Lightweight Internet of Things equipment identification method based on message length distribution similarity

Abstract

The invention discloses a lightweight Internet of Things equipment identification method based on message length distribution similarity, and belongs to the technical field of network security. The method comprises the following steps: firstly, collecting traffic data of Internet of Things equipment, extracting the length and direction of each message of each equipment in a fixed time interval to construct a data set, and dividing the data set into a training set, a verification set and a test set; calculating three types of feature tables for describing message length distribution similarity according to the training set data, wherein the three types of feature tables comprise a message length distribution stability table, an identification degree table and a frequency table; and randomly combining the three types of feature tables to obtain seven combination modes. And traversing the seven feature table combinations, and selecting the feature table combination with the highest verification set sample classification accuracy as the optimal message length distribution similarity table combination. And calculating the length distribution similarity between the sample and each device by using the optimal table combination in the test set, taking the device label with the highest similarity as a classification label, and quickly identifying the type of the Internet of Things device.

Description

Technical Field

[0001] The invention relates to a lightweight Internet of Things device identification method based on message length distribution similarity, and belongs to the technical field of network security. Background Art

[0002] IoT devices refer to devices that can communicate with each other and do not require direct human involvement, such as smart sockets, smart lights, cameras, temperature / humidity sensors, etc. Today, IoT devices are diverse, large in scale, and growing rapidly. Because IoT devices have a wide range of access methods, limited computing performance, and non-uniform communication standards, there are security risks such as side channel information leakage, firmware vulnerabilities, and communication protocol vulnerabilities. IoT devices have long faced security threats such as botnets, distributed denial of service attacks, and malware attacks.

[0003] IoT device identification technology provides an effective way to solve the above-mentioned IoT security issues. Network security managers need to obtain IoT device information through IoT device identification technology under the premise of fully considering user privacy. Combined with the IoT device vulnerability database, they can discover known security vulnerabilities and prevent risks through system firmware upgrades, product replacements, etc.

[0004] Among the existing IoT device identification methods, there are active identification methods and passive identification methods, depending on whether data packets are actively sent to the network. However, the existing methods still have limitations. In the active IoT device identification method, it is necessary to determine the target to be identified, and then actively send a detection packet to the target and receive its return data, which will have a certain impact on the network. The passive IoT device identification method does not directly affect the network traffic, but passively monitors the traffic information. Existing methods often extract many features from the message, use supervised learning or deep learning methods to train the classification model, and consume a lot of computing resources. Summary of the invention

[0005] In order to solve the limitations of the prior art, the present invention proposes a lightweight IoT device identification method based on message length distribution similarity, which achieves lower energy consumption and higher efficiency without affecting the network traffic status.

[0006] A lightweight IoT device identification method based on message length distribution similarity, the specific steps are as follows:

[0007] Step 1: Collect IoT device traffic through data collection equipment and process the IoT device traffic: divide the traffic data into a series of original traffic samples according to device instances and fixed time intervals.

[0008] Differentiate each device instance by IP address or MAC address, record the device instance serial number as Di, divide a specific device instance according to a fixed time interval, record the time serial number as Tm, and divide the IoT device traffic into several original traffic samples

[0009] Step 2: Extract the message length distribution characteristics from the original traffic samples to form the IoT traffic data set.

[0010] Based on raw traffic samples Calculate the frequency distribution of each message length and direction. The specific combination of message length and direction is recorded as (l, d), where l represents the message length in bytes; d represents the message direction. The direction field d is 0 when the message originates from the IoT device, otherwise it is 1.

[0011] count[Di,Tm,(l,d)] indicates the number of occurrences of a specific message (l,d) in the Tmth sample of device Di.

[0012] count[Di,Tm] represents the total number of messages in the Tmth sample of device Di.

[0013] freq[Di,Tm,(l,d)] represents the frequency of occurrence of a specific message (l,d) in the Tmth sample of device Di.

[0014]

[0015] The Tmth sample of device Di in the dataset consists of key-value pairs of each message size and direction and its frequency {(l,d),freq[Di,Tm,(l,d)]}.

[0016] Step three: divide the IoT traffic dataset into training set, validation set and test set according to the set ratio.

[0017] Step 4: Calculate three types of feature tables for describing the similarity of message length distribution using the training set data;

[0018] The three types of feature tables are: stability table, recognition table and frequency table, and the calculation processes are as follows:

[0019] (1) Calculation of stability table

[0020] The set of all specific messages (l, d) in the training set device Di is The total number of training samples of device Di in the training set is The stability table is composed of the stability tables of each device. Each device stability table consists of key-value pairs. Value is

[0021] Stability value of a specific message (l,d) in device Di The calculation formula is:

[0022]

[0023] (2) Calculate the recognition table

[0024] Let D be the number of devices in the training set. The recognition degree of messages with specific length and direction in the training set is:

[0025]

[0026] The recognition table consists of key-value pairs, and the key (l, d) is taken from the set of all device messages in the training set. The union of (l,d) .

[0027] (3) Calculate frequency table

[0028] The average occurrence frequency set of each specific message (l, d) in each device constitutes a frequency table.

[0029] The average frequency of a specific combination (l,d) in device Di

[0030] The frequency table is composed of the frequency table of each device. The frequency table of each device is composed of key-value pairs. Value is

[0031]

[0032] Step 5: randomly select at least one feature table from the three feature tables to obtain seven combinations of feature tables, and use the validation set to select the best feature table combination.

[0033] The specific method for selecting the best feature table combination is:

[0034] First, under a certain feature table combination, traverse each sample in the validation set, calculate the similarity of the length distribution of each validation set sample and each device in the training set, and take the training set device label with the highest similarity as the sample classification prediction label.

[0035] The similarity between each validation set sample and IoT device Di It is obtained by summing the similarities between each message (l, d) in the sample and the device Di.

[0036]

[0037] Specifically, define the parameters: FQ-ENABLE (Boolean type), DE-ENABLE (Boolean type), FREQ-ENABLE (Boolean type); FQ-ENABLE corresponds to the stability table, DE-ENABLE corresponds to the recognition table, and FREQ-ENABLE corresponds to the frequency table.

[0038] When the FQ-ENABLE, DE-ENABLE, and FREQ-ENABLE parameters are all 1:

[0039]

[0040] If FQ-ENABLE is 0, In the expression The value is always 1;

[0041] If DE-ENABLE is 0, then In the expression The value is always 1;

[0042] If FREQ-ENABLE is 0, then In the expression The value is always 1.

[0043] Then, by comparing the predicted labels with the true labels, the classification accuracy of the validation set samples is obtained.

[0044] This method is used to traverse the seven feature table combinations, and the classification accuracy of the validation set samples in each combination is obtained. The feature table combination with the highest classification accuracy of the validation set samples is selected as the best feature table combination.

[0045] Step six: Based on the optimal feature table combination, IoT devices are identified for the samples to be classified in the test set.

[0046] Traverse each sample in the test set, calculate the similarity between the sample and the length distribution of each device in the training set, and take the training set device label with the highest similarity as the sample classification prediction label.

[0047] The technical solution of the present invention has the following advantages:

[0048] (1) The present invention proposes a lightweight IoT device identification method based on message length distribution similarity, which takes traffic data in a certain time window as the analysis object and is applicable to idle traffic and working traffic states.

[0049] (2) The present invention converts several training set traffic instances at fixed time intervals into three types of feature tables for describing the similarity of message length distribution. The sample recognition time does not increase with the increase of the number of training set traffic, and can achieve rapid recognition of IoT devices.

[0050] (3) The message length distribution similarity algorithm fully considers the scarcity of message frequency and the similarity of occurrence probability of specific IoT devices, and has strong robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 The figure is an overall flow chart of the lightweight IoT device identification method based on message length distribution similarity of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0053] The present invention provides a lightweight IoT device identification method based on message length distribution similarity. First, the IoT device traffic data is collected, and the message length distribution of each device within a fixed time interval is extracted as a sample to construct a data set, and the data set is divided into a training set, a validation set, and a test set according to a certain ratio. In order to describe the length distribution characteristics of each IoT device in the training set, a message length distribution stability table, a recognition table, and a frequency table are calculated based on the training set data. At least one feature table is taken from the above three feature tables, and there are seven combinations. The combination of feature tables determines the length distribution similarity calculation method.

[0054] Under certain feature table combinations, traverse each sample in the validation set, calculate the similarity between the sample and the length distribution of each device in the training set, and take the training set device label with the highest similarity as the sample classification prediction label. By comparing the predicted label with the true label, the classification accuracy of the validation set sample can be obtained.

[0055] Traverse the seven feature table combinations and calculate the classification accuracy of the validation set samples under each combination. Select the feature table combination with the highest classification accuracy of the validation set samples as the best feature table combination. Finally, use the best feature table combination determined in the validation set to calculate the length distribution similarity in the test set to quickly identify the type of IoT devices.

[0056] In order to achieve the purpose of the present invention, Figure 1 As shown, the specific technical steps of this solution are as follows:

[0057] Step 1: Collect IoT device traffic through data collection equipment.

[0058] IoT device traffic refers to the messages transmitted by IoT devices in a computer network. In this application example, the traffic collection point is located at the LAN gateway.

[0059] In a feasible implementation, port mirroring technology can be used to capture traffic data, thereby obtaining records of incoming and outgoing messages. The port mirroring function refers to forwarding the data traffic of one or more source ports to a specified port through a switch or router to achieve network monitoring and traffic analysis. Port mirroring does not change any information of the mirrored message, nor does it affect the normal forwarding of the original message.

[0060] Step 2: Process the IoT device traffic and divide the traffic data into a series of raw traffic samples according to device instances and fixed time intervals.

[0061] Differentiate each device instance by IP address or MAC address, record the device instance serial number as Di, divide a specific device instance according to a fixed time interval, record the time serial number as Tm, and divide the IoT device traffic into several original traffic samples

[0062] Step 3: Extract the message length distribution characteristics from the original traffic sample in step 2 to form an IoT traffic data set.

[0063] Based on the Tmth raw traffic sample of device Di Calculate the frequency distribution of each message size and direction. The specific combination of message length and direction is recorded as (l, d), where l represents the message size in bytes; d represents the message direction. The direction field d is 0 when the message originates from the IoT device, otherwise it is 1.

[0064] count[Di,Tm,(l,d)] indicates the number of occurrences of a specific message (l,d) in the Tmth sample of device Di.

[0065] count[Di,Tm] represents the total number of messages in the Tmth sample of device Di.

[0066] freq[Di,Tm,(l,d)] represents the frequency of occurrence of a specific message (l,d) in the Tmth sample of device Di.

[0067]

[0068] The Tmth sample of device Di in the dataset consists of key-value pairs of each message size and direction and its frequency: {(l,d),freq[Di,Tm,(l,d)]}.

[0069] Exemplarily, the sample length distribution is expressed as follows:

[0070] {(342,0):0.5,(54,1):0.3,(83,1):0.05,(328,1):0.05,(98,1):0.05,(82,1):0.05}

[0071] Step 4: Divide the IoT traffic dataset generated in step 3 into training set, validation set and test set according to a certain ratio.

[0072] The ratio of the training set, validation set, and test set can be set to 5:3:2, and the timestamp of the training set data should be earlier than that of the validation set data and the test set data.

[0073] Step 5: Use the training set data in step 4 to calculate three types of feature tables describing the message length distribution characteristics according to a certain algorithm: stability table, recognition table and frequency table. The combination of feature tables determines the length distribution similarity calculation method.

[0074] 5.1 Calculation of stability table

[0075] Specifically, the stability table reflects whether the message of a specific length and direction appears stably between samples of the same device. The set of all specific messages (l, d) in the training set device Di is The total number of training samples of the training set device Di is The stability value of a specific message (l, d) in device Di is The stability table is composed of the stability tables of each device. Each device stability table consists of key-value pairs. Value is

[0076] 5.2 Calculation of recognition table

[0077] The rarer the length and direction of a particular message is across all training devices, the higher the recognition. The recognition term is designed to give higher weight to unique messages.

[0078] Let D be the number of devices in the training set, and the recognition degree of messages with specific length and direction in the training set is

[0079]

[0080] The recognition table consists of key-value pairs, and the key (l, d) is taken from the set of all device messages in the training set. The union of (l,d) .

[0081] 5.3 Calculating frequency table

[0082] The average occurrence frequency of each specific message (l, d) in each device constitutes a frequency table. The frequency table can be used to compare the similarity of specific message proportions between samples.

[0083] Probability of occurrence of a specific message (l, d) in the Tmth sample of device Di

[0084] The average frequency of a specific combination (l,d) in device Di

[0085] The frequency table is composed of the frequency table of each device. The frequency table of each device is composed of key-value pairs. Value is

[0086]

[0087] Step 6: Take at least one feature table from the three feature tables obtained in step 5. There are seven combinations. In the case of a certain feature table combination, traverse each sample in the validation set, calculate the length distribution similarity between the sample and each device in the training set, and take the training set device label with the highest similarity as the sample classification prediction label. By comparing the predicted label with the true label, the classification accuracy of the validation set sample can be obtained.

[0088] Traverse the seven feature table combinations and calculate the classification accuracy of the validation set samples in each combination. Select the feature table combination with the highest classification accuracy of the validation set samples as the best feature table combination.

[0089] Specifically, define the parameters: FQ-ENABLE (Boolean type), DE-ENABLE (Boolean type), FREQ-ENABLE (Boolean type). FQ-ENABLE corresponds to the stability table. DE-ENABLE corresponds to the recognition table. FREQ-ENABLE corresponds to the frequency table.

[0090] Similarity between the sample and the device Di It is obtained by summing the similarities between each sample message (l, d) and the device Di.

[0091]

[0092] When the FQ-ENABLE, DE-ENABLE, and FREQ-ENABLE parameters are all 1:

[0093]

[0094] If FQ-ENABLE is 0, In the expression The value is always 1.

[0095] If DE-ENABLE is 0, then In the expression The value is always 1.

[0096] If FREQ-ENABLE is 0, then In the expression The value is always 1.

[0097] Step 7: Identify IoT devices on the test set samples based on the best feature table combination obtained in step 6.

[0098] Traverse each sample in the test set, calculate the similarity between the sample and the length distribution of each device in the training set, and take the training set device label with the highest similarity as the sample classification prediction label.

Claims

1. A lightweight IoT device identification method based on message length distribution similarity, characterized in that: The specific steps are as follows: Step 1: collect IoT device traffic through data collection equipment, and process the IoT device traffic to obtain original traffic samples; Distinguish each device instance by IP address or MAC address, and record the device instance serial number as D i , divide the specific device instance into fixed time intervals, and the time sequence number is recorded as T m , divide the IoT device traffic into several original traffic samples Step 2: extract the message length distribution characteristics from the original traffic sample to form an IoT traffic data set; Based on raw traffic samples Calculate the frequency distribution of each message length and direction; the specific combination of message length and direction is recorded as (l, d), where l represents the message length in bytes; d represents the message direction. When the message originates from the IoT device, the direction field d is 0, otherwise it is 1; freq[D i ,T m ,(l,d)] represents device D i No. T m The frequency of occurrence of specific messages (l, d) in samples: Among them, count[D i ,T m ,(l,d)] represents device D i No. T m The number of occurrences of a specific message (l, d) in a sample; count[D i ,T m ] indicates device D i No. T m The total number of messages in a sample; The IoT traffic data set is: Device D in the data set i No. T m A sample consists of a key-value pair of each message size, direction and frequency {(l,d),freq[D i ,T m ,(l,d)]}; Step 3: Divide the IoT traffic data set into training set, validation set and test set according to the set ratio; Step 4: Calculate three types of feature tables for describing the similarity of message length distribution using the training set data; The three types of feature tables are: stability table, recognition table and frequency table, and the calculation processes are as follows: (1) Stability table Remember the training set device D i The set of all specific messages (l, d) is Remember the training set device D i The total number of training samples is The stability table is composed of the stability tables of each device. Each device stability table consists of key-value pairs. Value is (2) Recognition table The recognition table consists of key-value pairs, and the key (l, d) is taken from the set of all device messages in the training set. The union of (l,d) ; (3) Frequency table The average occurrence frequency set of each specific message (l, d) in each device constitutes a frequency table; The frequency table is composed of the frequency table of each device, and the frequency table of each device is composed of key-value pairs. Value is Step 5, randomly select at least one feature table from the three feature tables to obtain seven combinations of feature tables, and use the validation set to select the best feature table combination; The specific method for selecting the best feature table combination is: First, under a certain feature table combination, traverse each sample in the validation set, calculate the similarity of the length distribution between each validation set sample and each device in the training set, and take the training set device label with the highest similarity as the sample classification prediction label; Then, by comparing the predicted labels with the true labels, the classification accuracy of the validation set samples is obtained; Finally, we traverse the seven feature table combinations, obtain the classification accuracy of the validation set samples in each combination, and select the feature table combination with the highest classification accuracy of the validation set samples as the best feature table combination; Step 6: Identify IoT devices for the samples to be classified in the test set according to the best feature table combination; Traverse each sample in the test set, calculate the similarity between the sample and the length distribution of each device in the training set, and take the training set device label with the highest similarity as the sample classification prediction label.

2. According to claim 1, a lightweight IoT device identification method based on message length distribution similarity is characterized in that: The device D i The stability value of a specific message (l,d) in The calculation formula is:

3. According to claim 1, a lightweight IoT device identification method based on message length distribution similarity is characterized in that: Let D be the number of devices in the training set. The recognition degree of messages with specific length and direction in the training set is:

4. According to claim 1, a lightweight IoT device identification method based on message length distribution similarity is characterized in that: Equipment D i The average frequency of a specific combination (l,d) in 5. According to claim 1, a lightweight IoT device identification method based on message length distribution similarity is characterized in that: Each sample in the validation set and each IoT device in the training set D i The length distribution similarity Each message (l, d) in the sample and device D i The similarity of is summed up; Among them, each sample message (l, d) and device D i Similarity for: Define parameters: FQ-ENABLE, DE-ENABLE, FREQ-ENABLE; FQ-ENABLE corresponds to the stability table, DE-ENABLE corresponds to the recognition table, FREQ-ENABLE corresponds to the frequency table; If the FQ-ENABLE, DE-ENABLE, and FREQ-ENABLE parameters are all 1, the similarity As shown in the above formula; If FQ-ENABLE is 0, In the expression The value is always 1; If DE-ENABLE is 0, then In the expression The value is always 1; If FREQ-ENABLE is 0, then In the expression The value is always 1.

Images