A lightweight IoT device identification method based on message length distribution similarity

Abstract

The present invention discloses a lightweight IoT device identification method based on message length distribution similarity, which belongs to the field of network security technology. First, IoT device traffic data is collected, and the length and direction of each message within a fixed time interval of each device are extracted to construct a data set, and the data set is divided into a training set, a validation set, and a test set. According to the training set data, three types of feature tables are calculated to describe the similarity of message length distribution: a message length distribution stability table, a recognition table, and a frequency table. The three feature tables are randomly combined to obtain seven combinations. The seven feature table combinations are traversed, and the feature table combination with the highest classification accuracy of the validation set samples is selected as the optimal message length distribution similarity table combination. The optimal table combination is used in the test set to calculate the length distribution similarity between the sample and each device, and the device label with the highest similarity is taken as the classification label to quickly identify the type of IoT device.

Description

Technical Field

[0001] The present invention relates to a lightweight Internet of Things device identification method based on message length distribution similarity, belonging to the technical field of network security. Background Art

[0002] IoT devices are devices that can communicate with each other without direct human interaction, such as smart sockets, smart lights, cameras, and temperature / humidity sensors. Today, IoT devices are diverse, massive, and growing rapidly. Due to their diverse access methods, limited computing performance, and non-standard communication standards, IoT devices present security risks such as side-channel information leaks, firmware vulnerabilities, and communication protocol vulnerabilities. IoT devices are constantly exposed to security threats such as botnets, distributed denial of service attacks, and malware attacks.

[0003] IoT device identification technology provides an effective way to solve the above-mentioned IoT security issues. Network security managers need to obtain IoT device information through IoT device identification technology while fully considering user privacy. Combined with the IoT device vulnerability database, they can discover known security vulnerabilities and prevent risks through system firmware upgrades, product replacements, etc.

[0004] Existing IoT device identification methods are categorized as active or passive, depending on whether they actively send data packets to the network. However, existing methods still have limitations. Active IoT device identification methods require identifying the target to be identified, actively sending probe packets to the target, and receiving the returned data, which can have a certain impact on the network. Passive IoT device identification methods do not directly affect network traffic, but rather passively monitor traffic information. Existing methods often extract numerous features from packets and use supervised learning or deep learning methods to train classification models, which consumes significant computing resources. Summary of the Invention

[0005] To address the limitations of the existing technology, the present invention proposes a lightweight IoT device identification method based on message length distribution similarity, which achieves lower energy consumption and higher efficiency without affecting the network traffic status.

[0006] A lightweight IoT device identification method based on message length distribution similarity, the specific steps are as follows:

[0007] Step 1: Collect IoT device traffic through data collection equipment and process the IoT device traffic: divide the traffic data into a series of raw traffic samples according to device instances and fixed time intervals.

[0008] Distinguish each device instance by IP address or MAC address, and the device instance serial number is recorded as D i, a specific device instance is divided according to a fixed time interval, and the time sequence number is recorded as T m , divide the IoT device traffic into several original traffic samples

[0009] Step 2: Extract the message length distribution characteristics from the original traffic samples to form the IoT traffic dataset.

[0010] Based on raw traffic samples Calculate the frequency distribution of each message length and direction. A specific combination of message length and direction is denoted as (l,d), where l represents the message length in bytes and d represents the message direction. The direction field d is 0 if the message originates from the IoT device and 1 otherwise.

[0011] count[D i ,T m ,(l,d)] represents device D i No. T m The number of times a specific message (l, d) appears in a sample.

[0012] count[D i ,T m ] indicates device D i No. T m The total number of messages in the sample.

[0013] freq[D i ,T m ,(l,d)] represents device D i No. T m The frequency of occurrence of a specific message (l, d) in a sample.

[0014]

[0015] Data concentration device D i No. T m A sample consists of a key-value pair of each message size, direction and frequency {(l,d),freq[D i ,T m ,(l,d)]}.

[0016] Step 3: Divide the IoT traffic dataset into training set, validation set and test set according to the set ratio.

[0017] Step 4: Using the training set data, three feature tables are calculated to describe the similarity of message length distribution.

[0018] The three types of feature tables are: stability table, recognition table and frequency table. The calculation process is as follows:

[0019] (1) Calculation stability table

[0020] Let training set device D be i The set of all specific messages (l,d) in Note that the training set device D i The total number of training samples is The stability table is composed of the stability tables of each device. Each device stability table consists of key-value pairs, with key (l, d)∈ The value is

[0021] Device D i The stability value of a specific message (l,d) in The calculation formula is:

[0022]

[0023] (2) Calculate the recognition table

[0024] Let D be the number of devices in the training set. The recognition degree of messages with specific length and direction in the training set is:

[0025]

[0026] The recognition table consists of key-value pairs, where the key (l, d) is taken from the set of all device messages in the training set. The union of

[0027] (3) Calculate frequency table

[0028] The set of average occurrence frequencies of each specific message (l, d) in each device constitutes a frequency table.

[0029] Device D i The average frequency of a specific combination (l,d) in

[0030] The frequency table is composed of the frequency table of each device. The frequency table of each device is composed of key-value pairs. The value is

[0031] Step 5: Randomly select at least one feature table from the three feature tables to obtain seven combinations of feature table combinations, and use the validation set to select the best feature table combination.

[0032] The specific method for selecting the best feature table combination is:

[0033] First, under a certain feature table combination, each sample in the validation set is traversed, and the length distribution similarity between each validation set sample and each device in the training set is calculated. The training set device label with the highest similarity is taken as the sample classification prediction label.

[0034] Each validation set sample and IoT device D i Similarity Each message (l, d) in the sample and the device D i The similarity is summed up.

[0035]

[0036] Specifically, define the parameters: FQ-ENABLE (Boolean type), DE-ENABLE (Boolean type), FREQ-ENABLE (Boolean type); FQ-ENABLE corresponds to the stability table, DE-ENABLE corresponds to the recognition table, and FREQ-ENABLE corresponds to the frequency table.

[0037] When the FQ-ENABLE, DE-ENABLE, and FREQ-ENABLE parameters are all set to 1:

[0038]

[0039] If FQ-ENABLE is 0, In the expression The value is always 1;

[0040] If DE-ENABLE is 0, then In the expression The value is always 1;

[0041] If FREQ-ENABLE is 0, then In the expression The value is always 1.

[0042] Then, by comparing the predicted labels with the true labels, the classification accuracy of the validation set samples is obtained.

[0043] This method is used to traverse the seven feature table combinations, and the classification accuracy of the validation set samples in each combination is obtained. The feature table combination with the highest classification accuracy of the validation set samples is selected as the best feature table combination.

[0044] Step 6: Based on the optimal feature table combination, IoT device identification is performed on the samples to be classified in the test set.

[0045] Traverse each sample in the test set, calculate the similarity between the sample and the length distribution of each device in the training set, and take the training set device label with the highest similarity as the sample classification prediction label.

[0046] The technical solution of the present invention has the following advantages:

[0047] (1) The present invention proposes a lightweight IoT device identification method based on message length distribution similarity, which takes traffic data in a certain time window as the analysis object and is applicable to idle traffic and working traffic states.

[0048] (2) The present invention converts several training set traffic instances at fixed time intervals into three feature tables that describe the similarity of message length distribution. The sample recognition time does not increase with the number of training set traffic instances, enabling rapid recognition of IoT devices.

[0049] (3) The message length distribution similarity algorithm fully considers the scarcity of message frequency and the similarity of occurrence probability of specific IoT devices, and has strong robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is an overall flow chart of the lightweight IoT device identification method based on message length distribution similarity of the present invention. DETAILED DESCRIPTION

[0051] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0052] The present invention provides a lightweight IoT device identification method based on message length distribution similarity. The method first collects IoT device traffic data, extracts the message length distribution of each device within a fixed time interval as a sample to construct a dataset, and then divides the dataset into a training set, a validation set, and a test set according to a certain ratio. To describe the length distribution characteristics of each IoT device in the training set, a message length distribution stability table, a recognition table, and a frequency table are calculated based on the training set data. Seven combinations of at least one of the three feature tables are possible. The combination of the feature tables determines how the length distribution similarity is calculated.

[0053] For a specific feature table combination, we traverse each sample in the validation set and calculate the similarity between the length distribution of the sample and each device in the training set. We then select the training set device label with the highest similarity as the predicted label for the sample classification. By comparing the predicted label with the true label, we can obtain the classification accuracy of the validation set sample.

[0054] We traverse seven feature table combinations and calculate the classification accuracy of validation set samples for each combination. We select the feature table combination with the highest classification accuracy for validation set samples as the optimal feature table combination. Finally, we use the optimal feature table combination from the validation set to calculate length distribution similarity in the test set, allowing for rapid identification of IoT device types.

[0055] In order to achieve the purpose of the present invention, Figure 1 As shown, the specific technical steps of this solution are as follows:

[0056] Step 1: Collect IoT device traffic through data collection equipment.

[0057] IoT device traffic refers to the messages transmitted by IoT devices in a computer network. In this application example, the traffic collection point is located at the local area network gateway.

[0058] In one feasible implementation, port mirroring technology can be used to capture traffic data and obtain records of incoming and outgoing messages. Port mirroring, through a switch or router, forwards data traffic from one or more source ports to a designated port for network monitoring and traffic analysis. Port mirroring does not alter any information in the mirrored packets or affect the normal forwarding of the original packets.

[0059] Step 2: Process the IoT device traffic and divide the traffic data into a series of raw traffic samples according to the device instance and fixed time interval.

[0060] Distinguish each device instance by IP address or MAC address, and the device instance serial number is recorded as D i , a specific device instance is divided according to a fixed time interval, and the time sequence number is recorded as T m , divide the IoT device traffic into several original traffic samples

[0061] Step 3: Extract the message length distribution characteristics from the original traffic samples in step 2 to form an IoT traffic dataset.

[0062] Based on device D i No. T m Raw traffic samples Calculate the frequency distribution of each message size and direction. A specific combination of message length and direction is denoted as (l,d), where l represents the message size in bytes and d represents the message direction. The direction field d is 0 if the message originates from the IoT device and 1 otherwise.

[0063] count[D i ,T m ,(l,d)] represents device D i No. T m The number of times a specific message (l, d) appears in a sample.

[0064] count[D i ,T m ] indicates device D i No. T m The total number of messages in the sample.

[0065] freq[D i ,T m ,(l,d)] represents device D i No. Tm The frequency of occurrence of a specific message (l, d) in a sample.

[0066]

[0067] Data concentration device D i No. T m Each sample consists of a key-value pair of each message size, direction, and frequency. i ,T m ,(l,d)]}.

[0068] Exemplarily, the sample length distribution is expressed as follows:

[0069] {(342,0):0.5,(54,1):0.3,(83,1):0.05,(328,1):0.05,(98,1):0.05,(82,1):0.05}

[0070] Step 4: Divide the IoT traffic dataset generated in step 3 into training set, validation set, and test set according to a certain ratio.

[0071] The ratio of the training set, validation set, and test set can be set to 5:3:2. The timestamp of the training set data should be earlier than that of the validation set data and the test set data.

[0072] Step 5: Using the training data from step 4, calculate three feature tables describing the message length distribution according to a specific algorithm: stability table, recognition table, and frequency table. The combination of feature tables determines how the length distribution similarity is calculated.

[0073] 5.1 Calculation stability table

[0074] Specifically, the stability table reflects whether messages of a specific length and direction appear stably between samples of the same device. i The set of all specific messages (l,d) in Let training set device D be i The total number of training samples is Note device D i The stability value of a specific message (l,d) is The stability table is composed of the stability tables of each device. Each device stability table consists of key-value pairs. The value is

[0075] 5.2 Calculating the Recognition Table

[0076] The rarer the length and direction of a particular message is across all training devices, the higher its recognition score. The recognition score is designed to give higher weight to unique messages.

[0077] Let D be the number of devices in the training set, and the recognition degree of messages with specific length and direction in the training set is

[0078]

[0079] The recognition table consists of key-value pairs, where the key (l, d) is taken from the set of all device messages in the training set. The union of

[0080] 5.3 Calculating frequency table

[0081] The average occurrence frequency of each specific message (l, d) in each device forms a frequency table. The frequency table can be used to compare the similarity of specific message proportions between samples.

[0082] Device D i No. T m The probability of a specific message (l, d) appearing in a sample

[0083] Device D i The average frequency of a specific combination (l,d) in

[0084] The frequency table is composed of the frequency table of each device. The frequency table of each device is composed of key-value pairs. The value is

[0085] Step 6: Select at least one of the three feature tables obtained in Step 5. There are seven possible combinations. For each feature table combination, iterate over each sample in the validation set and calculate the similarity between the sample and the length distribution of each device in the training set. The training set device label with the highest similarity is used as the predicted label for the sample classification. By comparing the predicted label with the true label, the classification accuracy of the validation set sample can be obtained.

[0086] Traverse the seven feature table combinations and calculate the classification accuracy of the validation set samples under each combination. Select the feature table combination with the highest classification accuracy of the validation set samples as the best feature table combination.

[0087] Specifically, define the following parameters: FQ-ENABLE (Boolean type), DE-ENABLE (Boolean type), and FREQ-ENABLE (Boolean type). FQ-ENABLE corresponds to the stability table. DE-ENABLE corresponds to the recognition table. FREQ-ENABLE corresponds to the frequency table.

[0088] Samples and Equipment D i Similarity The sample messages (l, d) and device D i The similarity is summed up.

[0089]

[0090] When the FQ-ENABLE, DE-ENABLE, and FREQ-ENABLE parameters are all set to 1:

[0091]

[0092] If FQ-ENABLE is 0, In the expression The value is always 1.

[0093] If DE-ENABLE is 0, then In the expression The value is always 1.

[0094] If FREQ-ENABLE is 0, then In the expression The value is always 1.

[0095] Step 7: Based on the optimal feature table combination obtained in step 6, IoT device identification is performed on the test set samples.

[0096] Traverse each sample in the test set, calculate the similarity between the sample and the length distribution of each device in the training set, and take the training set device label with the highest similarity as the sample classification prediction label.

Claims

1. A lightweight IoT device identification method based on message length distribution similarity, characterized in that: The specific steps are as follows: Step 1: Use data collection equipment to collect IoT device traffic, process the IoT device traffic, and obtain original traffic samples; Distinguish each device instance by IP address or MAC address, and the device instance serial number is recorded as D i , a specific device instance is divided according to a fixed time interval, and the time sequence number is recorded as T m , divide the IoT device traffic into several original traffic samples Step 2: Extract the message length distribution characteristics from the original traffic samples to form an IoT traffic dataset; Based on raw traffic samples Calculate the frequency distribution of each message length and direction. A specific combination of message length and direction is denoted as (l,d), where l represents the message length in bytes and d represents the message direction. The direction field d is 0 when the message originates from the IoT device and 1 otherwise. freq[D i ,T m ,(l,d)] represents device D i No. T m The frequency of occurrence of a specific message (l,d) in a sample: Among them, count[D i ,T m ,(l,d)] represents device D i No. T m The number of times a specific message (l, d) appears in a sample; count[D i ,T m ] indicates device D i No. T m The total number of messages in a sample; The IoT traffic data set is: Device D in the data set i No. T m A sample consists of a key-value pair of each message size, direction and frequency {(l,d),freq[D i ,T m ,(l,d)]}; Step 3: Divide the IoT traffic dataset into training set, validation set, and test set according to the set ratio; Step 4: Using the training set data, three feature tables are calculated to describe the similarity of message length distribution. The three types of feature tables are: stability table, recognition table and frequency table. The calculation process is as follows: (1) Stability table Note that the training set device D i The set of all specific messages (l, d) is Note that the training set device D i The total number of training samples is The stability table is composed of the stability tables of each device. Each device stability table is composed of key-value pairs. The value is (2) Recognition table The recognition table consists of key-value pairs, where the key (l, d) is taken from the set of all device messages in the training set. The union of (3) Frequency table The average occurrence frequency of each specific message (l, d) in each device constitutes a frequency table; The frequency table is composed of the frequency table of each device, and the frequency table of each device is composed of key-value pairs. The value is Step 5: Randomly select at least one feature table from the three feature tables to obtain seven combinations of feature tables, and use the validation set to select the best feature table combination; The specific method for selecting the best feature table combination is: First, under a certain feature table combination, we traverse each sample in the validation set and calculate the similarity between the length distribution of each validation set sample and each device in the training set. We then take the training set device label with the highest similarity as the sample classification prediction label. Then, by comparing the predicted labels with the true labels, the classification accuracy of the validation set samples is obtained; Finally, we traverse the seven feature table combinations and obtain the classification accuracy of the validation set samples under each combination. We select the feature table combination with the highest classification accuracy of the validation set samples as the best feature table combination. Step 6: Based on the optimal feature table combination, IoT device identification is performed on the samples to be classified in the test set; Traverse each sample in the test set, calculate the similarity between the sample and the length distribution of each device in the training set, and take the training set device label with the highest similarity as the sample classification prediction label.

2. According to claim 1, a lightweight IoT device identification method based on message length distribution similarity is characterized in that: The device D i The stability value of a specific message (l,d) in The calculation formula is:

3. According to claim 1, a lightweight IoT device identification method based on message length distribution similarity is characterized in that: Let D be the number of devices in the training set. The recognition degree of messages with specific length and direction in the training set is:

4. The lightweight IoT device identification method based on message length distribution similarity according to claim 1 is characterized in that: Device D i The average frequency of a specific combination (l,d) in 5. The lightweight IoT device identification method based on message length distribution similarity according to claim 1 is characterized in that: Each sample in the validation set and each IoT device in the training set D i Length distribution similarity Each message (l, d) in the sample and the device D i The sum of similarities is obtained; Among them, each sample message (l, d) and device D i Similarity for: Define parameters: FQ-ENABLE, DE-ENABLE, FREQ-ENABLE; FQ-ENABLE corresponds to the stability table, DE-ENABLE corresponds to the recognition table, FREQ-ENABLE corresponds to the frequency table; If the FQ-ENABLE, DE-ENABLE, and FREQ-ENABLE parameters are all 1, the similarity As shown in the above formula; If FQ-ENABLE is 0, In the expression The value is always 1; If DE-ENABLE is 0, then In the expression The value is always 1; If FREQ-ENABLE is 0, then In the expression The value is always 1.

Images