A method for constructing IoT device behavior signature

Abstract

The present invention discloses a method for constructing a behavior signature of an IoT device. First, the mixed traffic captured by the gateway is processed, the traffic data packet information is extracted, and the traffic of various behaviors is segmented from the session level. Then, the packet-level information such as the length, time interval, and protocol type of the traffic data packet is extracted, and the frequency of occurrence is calculated to construct a characteristic vector of the behavior. DBSCAN clustering is used to obtain respective clusters of different behaviors, and a signature of the behavior is automatically constructed from each behavior cluster. The signature consists of ordered data packet header information and statistical features, and contains key information for distinguishing similar device behaviors. The present invention can automatically process mixed traffic, construct signatures of different behaviors, and has extremely high unique identification.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things traffic security technology, and in particular to a method for constructing an IoT device behavior signature. Background Art

[0002] In recent years, the number of smart home Internet of Things (IoT) devices has grown rapidly. Various IoT devices are connected to the network through wired or wireless communication. Due to their convenience and connectivity, different types of devices provide users with more and more intelligent functions. However, due to their large number and fragile security, IoT devices are vulnerable to attacks. In recent years, there have been many IoT device attack incidents: privacy leakage, illegal intrusion, local network destruction, etc., causing serious economic losses. Therefore, it is very important to monitor the network behavior of IoT devices.

[0003] At present, the methods for device type and behavior identification mainly analyze the information contained in the traffic, generate fingerprints for the device and its behavior, or construct feature vectors for training to achieve identification of device type or behavior. However, most existing studies build device fingerprints based on labeled traffic collected in specific scenarios. Since IoT device communications generally use traffic encryption, statistical features such as average traffic size and average interval are often used in device fingerprints. These features are related to the network environment and may cause fingerprint deviations for the same device in different network environments. Manual re-training of the model is required, which makes the fingerprint unsuitable for cross-scenario use. On the other hand, due to the wide variety of IoT devices and mixed traffic, it is necessary to collect labeled data of a large number of devices in advance and build a fingerprint library in a centralized manner, resulting in a huge workload and fingerprint library size, and high storage requirements for edge detection devices such as deployed gateways.

[0004] As the import and export of IoT device traffic, the gateway directly deploys the traffic capture and processing system at the gateway to automatically build traffic fingerprints corresponding to different device behaviors, which can reduce the complexity of manual processing and decentralize the centralized fingerprint construction work. At the same time, since it is the work scene collection and construction, fingerprint deviation can be prevented, and the gateway only needs to store the IoT device behavior traffic fingerprint of its working environment, which will greatly reduce its fingerprint storage volume. Summary of the invention

[0005] In order to solve the technical problems that the fingerprint setting recognition of IoT devices is not universal and the huge fingerprint library leads to too high hardware requirements for detection equipment, the present invention provides a method for constructing IoT device behavior signatures by automatically processing gateway mixed traffic, thereby reducing resource overhead and achieving rapid deployment.

[0006] In order to achieve the above technical purpose, the technical solution of the present invention is:

[0007] A method for constructing an IoT device behavior signature is used to automatically construct signature information of device behavior from mixed unknown traffic, including the following steps:

[0008] Step 1: Distinguish the traffic of different IoT devices according to the MAC address, extract each session from the traffic of the same device, extract the timestamp information of the data packet from each session, and then divide the session according to the time interval;

[0009] Step 2: Filter the sessions to remove those that are not related to the behavior, then sort the filtered sessions according to the session start time, and then determine whether the sessions belong to the same behavior based on the preset time interval threshold, and merge the sessions belonging to the same behavior based on the determination result, and finally sort the data packets within each session according to the timestamp;

[0010] Step 3: using the packet size as a statistical factor, counting the frequency of occurrence of packets of the same size in the behavior, and extracting the packet information with the highest frequency to construct a feature vector of the behavior;

[0011] Step 4: Calculate the distance between each feature vector, and cluster the feature vectors of all behaviors based on the distance, so as to obtain a clustering cluster for each behavior;

[0012] Step 5: From the clustering clusters of each behavior, the data packet information including the size of the data packet is used as an identifier, and the signature information of the device behavior is constructed based on the identifier.

[0013] The method, step 1 comprises:

[0014] Step 1.1: For the mixed traffic of different IoT devices captured by the gateway, use the MAC address of each device as a unique identifier to distinguish and save the traffic of each device independently;

[0015] Step 1.2, for the traffic of each device, extract the corresponding two-way communication data packet according to the five-tuple information of each data packet in the traffic, thereby extracting each session composed of the two-way communication flow;

[0016] Step 1.3, extract the timestamp information of each data packet from the session, traverse each data packet in turn, and calculate the time interval with the previous data packet based on the timestamp information, and then divide the session into multiple sub-sessions based on the threshold set according to the actual working environment.

[0017] In the method described, in step 1.2, the five-tuple information of the data packet includes the source IP address, source port, destination IP address, destination port and transport layer protocol.

[0018] The method, step 2 comprises:

[0019] Step 2.1, check the protocol type and session duration of the session, and delete the sessions of specific protocol types and sessions exceeding the preset time threshold as background traffic, so as to retain only the sessions closely related to the behavior, and then sort the retained sessions according to the session start time;

[0020] Step 2.2, extract the start and end time of all the sorted sessions, and determine the time interval between the two sessions. If the interval between the end time of the previous session and the start time of the next session is less than the threshold, or there is a time overlap between the data packets of the two adjacent sessions, the sessions are considered to belong to the same behavior and merged, otherwise they are considered to belong to different behaviors.

[0021] Step 2.3, sort the data packets in all sessions that have completed step 2.2 according to the timestamps, so as to obtain the behavior traffic that has been segmented and processed.

[0022] In the method, in step 2.1, sessions with protocol types of DNS, SSDP, DHCP, mDNS and NTP, as well as sessions with a duration greater than 2 minutes are deleted.

[0023] The method described, in step 3, is to extract the data packet information of the top 5 data packets of the same size that appear most frequently in the behavior, wherein the data packet information includes 4 types, namely, the size of the data packet, the number of occurrences, the average time interval and the protocol type, thereby constructing a 20-dimensional unlabeled feature vector for the 5 data packets in the behavior.

[0024] The method described, the step 4 comprises:

[0025] Step 4.1, using Jaccard distance as the distance calculation object between each feature vector, the 4-dimensional features of each data packet in the feature vector are regarded as a set, thereby obtaining 5 sets, and then comparing the ratio of the intersection and union of the sets of two feature vectors to obtain the distance between the sets;

[0026] Step 4.2, cluster the feature vectors using the DBSCAN algorithm, using the distance obtained in step 4.1 as the basis for the distance measurement algorithm in the clustering process to obtain the clustering clusters of all behavior samples;

[0027] Step 4.3, label each cluster according to the clustering results to identify the behavior.

[0028] In the method described, in step 4.1, the calculation formula of Jaccard distance is:

[0029]

[0030] in, Indicates the 4 features contained in each set of a feature vector. Indicates behavior, Indicates data packets, Indicates the size of the data packet. Indicates the number of occurrences of the data packet, represents the average time interval, Indicates the protocol type. represents the intersection symbol, and ∪ represents the union symbol.

[0031] The method, step 5 comprises:

[0032] Step 5.1, for each behavior cluster, count the number of occurrences of all different data packet sizes, record the top 5 data packet sizes, and record the time interval between each data packet corresponding to the top 5 data packet sizes;

[0033] Step 5.2, calculating the average number of occurrences and average time interval between packets corresponding to the top 5 packet sizes as components of the signature;

[0034] Step 5.3, obtain the order of the first appearance of the data packets corresponding to the top 5 data packet sizes in all the behavior traffic of the clustering cluster, and use the order of appearance of the data packets with more appearances as the arrangement order in the signature, so as to construct the signature information of the device behavior.

[0035] The signature information representation of the device behavior finally constructed by the method described is as follows:

[0036] <size1-fre1-time1, size2-fre2-time2, size3-fre3-time3, size4-fre4-time4, size5-fre5-time5> -protocol. Size1 to size5 are arranged in the order of the data packets' occurrence time.

[0037] The technical effect of the present invention is that the present invention designs a behavior traffic segmentation algorithm to segment the traffic segments corresponding to different device behaviors from the gateway mixed traffic; then extracts the effective feature information of the data packet, constructs a multi-character feature vector, and designs a distance measurement algorithm for clustering the behavior traffic, and finally extracts the signature information of the device behavior from the clustering cluster results. The present invention can realize rapid and automatic deployment and processing, is applicable to various network scenarios, can accurately divide various behavior traffic from mixed and unknown traffic, and construct effective signatures, and compared with existing analysis methods, can reduce space and time overhead, and has strong scalability.

[0038] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention will be further described in detail below with reference to the accompanying drawings and specific examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is an overall framework diagram of the IoT device behavior signature construction method of the present invention.

[0040] Figure 2 This is a schematic diagram of device behavior traffic composition.

[0041] Figure 3 Schematic diagram for behavioral clustering and signature construction. DETAILED DESCRIPTION

[0042] In this embodiment, the home gateway used includes various IoT devices, including IoT devices directly connected to WiFi, and small IoT devices connected to the gateway through ZigBee and Z-Wave protocols to achieve communication with the cloud. When the user uses the device, the device will communicate with the cloud, send traffic data for interaction, and obtain instructions to complete the operation, which is regarded as behavioral traffic. When the device is idle, it will also maintain a simple communication process with the cloud, which is regarded as background traffic. This embodiment provides a method for constructing an IoT device behavior signature, which is used to illustrate the specific implementation steps of the present invention. The method of this embodiment can provide users with an accurate signature of device behavior and ensure the security of the device operation process.

[0043] Figure 1 This is the overall framework diagram given in this embodiment, and the specific steps are as follows:

[0044] Step 1: Capture the mixed traffic of all devices in the home gateway, distinguish the traffic of different devices according to the MAC address of each device, distinguish the device traffic according to the session, extract the timestamp information of the data packet from each session, and divide the session according to the time interval, including:

[0045] Step 1.1: For the mixed traffic of IoT devices captured by the router gateway, separate the traffic of each device according to the unique identification MAC address of the device and save it independently.

[0046] Step 1.2: For the traffic of each device, extract each session based on the five-tuple information of the data packet (<source IP address, source port, destination IP address, destination port, transport layer protocol>), and then save it separately as a pcap file.

[0047] Step 1.3: Extract the timestamp information of all packets from each session and traverse the time interval between two adjacent packets in the session in turn. The reason for this step is that the behavior traffic generated by the same or similar operations of the same device may have the same number of sessions, and the corresponding five-tuples are also the same, such as Figure 2 As shown in the figure, it includes two behaviors of the light bulb device, on and off. They have the same number of sessions and the same session quintuple. However, due to the triggering sequence of the behaviors, the sessions belonging to different behaviors have a large interval in time.

[0048] According to the statistics of the data set, the time interval threshold between data packets is set to λ=30s. The setting of the threshold is based on the observation and analysis of the working conditions of a large number of IoT devices. If the time interval between packets is greater than the threshold, the data stream is split and the session is split into sub-sessions to ensure that all the sessions obtained strictly belong to a certain behavior.

[0049] Step 2: Sort each session by time, and make a judgment based on the set time interval threshold, merge sessions belonging to the same behavior, and obtain all behaviors of the data packets arranged in order.

[0050] Step 2.1: Filter and sort all sessions according to the protocol type and session duration of device communication. For sessions with protocol types such as DNS, SSDP, DHCP, mDNS, NTP, and sessions with a duration greater than 2 minutes, treat them as background traffic and delete their corresponding sessions. Finally, only sessions closely related to the behavior are retained. Figure 2 Packets 4, 57, and 14-49 are all background traffic and will be removed. Finally, all remaining sessions are sorted according to the time when the session occurred.

[0051] Step 2.2: Merge all filtered sessions into the corresponding behavior traffic. Extract the start and end time of all sorted sessions, and judge based on the time interval between the end of the previous session and the start of the next session. If the time interval between the two sessions is less than the threshold If the time interval is greater than the threshold, the sessions are considered to belong to different behaviors.

[0052] Step 2.3: Count the timestamp information of all packets in the merged session. The reason for this step is that the packets originally belonging to different sessions may overlap in time. Therefore, it is necessary to sort the packets in the session according to the timestamps and finally obtain the behavior traffic after the segmentation process. Figure 2 In the example, the packets retained in the time range of packets 1-13 belong to the on behavior, and the packets retained in the time range of packets 50-62 belong to the off behavior.

[0053] Step 3: Extract the data packet information of each behavior, determine the data packet information with the top 5 occurrence frequencies, and construct the feature vector of the behavior.

[0054] Step 3.1: Extract the packet information in the behavior traffic and construct a feature vector. After a lot of observation and analysis, it is found that the same behavior traffic has some commonalities: some packets of a certain size will appear repeatedly in the behavior, and these packets each have a specific time interval between packets. Therefore, the top five packets in the behavior traffic are counted, and their four features of <size-number of occurrences-average time interval-protocol type> are extracted respectively, and a 4×5=20-dimensional feature vector is constructed as the feature expression of the behavior. Among them, for TCP / TLS connections, only packets containing "payload / application data" are retained. Through such filtering operations, it can be ensured that the extracted features only exist in valid packets that are closely related to the device behavior. The specific meaning of the features is shown in Table 1 below.

[0055]

[0056] Step 4: Design a distance measurement algorithm and use the DBSCAN clustering algorithm to cluster all behavior sample feature vectors. Figure 3 A schematic diagram of the process is shown to obtain clusters for each behavior.

[0057] Step 4.1: Using Jaccard distance as a reference, the distance measurement algorithm is improved. The 4-dimensional features (i.e., size, number of occurrences, average time interval, and protocol type) of each data packet in the feature vector are regarded as one set, a total of 5 sets. By comparing the ratio of the intersection and union of the sets of two feature vectors, the distance between them is calculated. Considering that each behavior sample may have a certain fluctuation, this embodiment sets a tolerance value for each feature to reduce the risk of false alarms.

[0058] (1)

[0059] For example, the feature vector of one sample of the photo behavior of the blink-camera device is [1476-42-0.785s-tcp, 123-3-0.135s-tcp, 587-1-0.785s-tcp, 331-1-0.888s-tcp, 187-1-0.255s-tcp], and the feature vector of the other sample is [1476-41-0.731s-tcp, 123-3-0.125s-tcp,1159-1-0.785s-tcp, 331-1-0.875s-tcp, 187-1-0.275s-tcp]. By comparing the similarity of the corresponding features of each data packet in the two samples, there are 4 sets of data packets that are similar, and the final similarity distance between s1 and s2 is (1-4 / 5) = 0.2.

[0060] Step 4.2: Use the DBSCAN algorithm to cluster the sample feature vectors. For the DBSCAN algorithm, eps and minPts are two key parameters. Eps specifies the neighborhood radius when determining the core point, and minPts is the minimum number of points in the neighborhood for a point to become a core point. In this embodiment, Eps = 0.2 and minPts = 2 or 3 are selected. Considering that the sample data packet may be affected by the network environment or the behavior contains a variety of different modes, minPts has a certain fluctuation value, and some behaviors may also generate multiple clusters.

[0061] Step 4.3: After clustering, the effectiveness of the clustering results is judged based on the clustering results and combined with expert knowledge, and each cluster is labeled in turn through manual assistance to identify the behavior.

[0062] Step 5: From each behavior cluster, statistically analyze the data packet information, extract unique identifying information, and construct one or more signature information for each behavior based on the type of pattern each behavior has. Data packet size, number of occurrences, time interval between data packets, and protocol type, as stable feature representations, can effectively distinguish different behaviors, so they are still used as the basis for signature construction in this embodiment. For each cluster of behavior traffic, signature extraction includes the following steps:

[0063] Step 5.1: Count the number of occurrences of all packet sizes, find the top five packet sizes, and record the inter-packet time interval of each packet included.

[0064] Step 5.2: Calculate the average number of occurrences and average time interval between packets of the above five sizes as components of the signature.

[0065] Step 5.3: Get the order in which the selected five sizes of packets first appear in all behavioral traffic of the cluster, and find out the main order of packet occurrence as the arrangement order in the signature.

[0066] The final behavioral traffic signature is expressed as:

[0067] <size1-fre1-time1, size2-fre2-time2, size3-fre3-time3, size4-fre4-time4, size5-fre5-time5> -protocol. Size1 to size5 are arranged in the order of the packet occurrence time. For some behaviors, there are multiple clusters. This is because some behaviors may contain different behavior patterns, so multiple signature information will be constructed. Finally, all generated signatures are saved in a signature library for subsequent traffic matching.

[0068] The correctness of the above embodiment is verified by specific tests below.

[0069] Test environment: The CPU is Intel Core i7-8750 processor, the memory is 16GB, and the experiment is carried out on a 64-bit operating system with Windows 10.

[0070] Test examples: behavioral traffic samples from 57 IoT devices, of which 38 devices are from the public dataset Mon(IoT)r, 13 devices are from the public dataset PINGPONG, and 6 devices are from device traffic collected in personal experiments, containing a total of 245 behaviors. All behavioral traffic samples are labeled when collected, but the labels are hidden during the implementation of the method and are only used to verify the accuracy of the signature.

[0071] By analyzing the behavioral traffic samples in the data set, the threshold parameters for behavioral traffic segmentation and the parameters in the clustering process are set, as shown in Table 2 below.

[0072]

[0073] After performing session segmentation, behavior sample division, and DBSCAN clustering on the three data sets and device behavior traffic, and verifying the consistency of all clustering results with the device behavior traffic labels, we use the clustering average accuracy to measure the consistency between the clustering cluster results and the actual behavior traffic samples. It can be seen that on the three data sets, the average accuracy of the clustering results has reached more than 99%, as shown in Table 3 below.

[0074]

[0075] Based on the above clustering results, 256 behavioral signatures were finally generated for 57 devices and 245 behaviors in the data set. Then, all the generated signatures were tested for uniqueness. By comparing the differences in the information in the signatures, the uniqueness of the behavioral traffic signatures of different devices was verified, and each behavior could be uniquely identified.

[0076] To further verify the uniqueness of the signature, we verify the similarity of the signatures in four scenarios: (1) the behavior of two different types of devices from the same manufacturer; (2) the behavior of the same type of devices from different manufacturers; (3) different update versions of the same device; and (4) the behavior of the device in two network environments. Table 4 below shows the comparison of the behavior signatures of some devices that meet the above comparison conditions.

[0077]

[0078] In each scenario, for the same behavior, the behavior signatures generated by the two devices are quite different. For the same device in different network environments, although the generated signatures are similar, there are still large differences in the number of occurrences of data packets. The results show that the behavior signature designed by the present invention can have good uniqueness and discrimination in different scenarios.

[0079] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for constructing IoT device behavior signatures, used to automatically construct device behavior signature information from mixed unknown traffic, characterized in that: The following steps are involved: Step 1: Distinguish the traffic of different IoT devices according to the MAC address, extract each session from the traffic of the same device, extract the timestamp information of the data packet from each session, and then divide the session according to the time interval; Step 2: Filter the sessions to remove those that are not related to the behavior, then sort the filtered sessions according to the session start time, and then determine whether the sessions belong to the same behavior based on the preset time interval threshold, and merge the sessions belonging to the same behavior based on the determination result, and finally sort the data packets within each session according to the timestamp; Step 3: using the packet size as a statistical factor, counting the frequency of occurrence of packets of the same size in the behavior, and extracting the packet information with the highest frequency to construct a feature vector of the behavior; Step 4: Calculate the distance between each feature vector, and cluster the feature vectors of all behaviors based on the distance, so as to obtain a clustering cluster for each behavior; Step 5: From the clustering clusters of each behavior, the data packet information including the size of the data packet is used as an identifier, and the signature information of the device behavior is constructed based on the identifier.

2. The method according to claim 1, characterized in that The step 1 comprises: Step 1.1: For the mixed traffic of different IoT devices captured by the gateway, use the MAC address of each device as a unique identifier to distinguish and save the traffic of each device independently; Step 1.2, for the traffic of each device, extract the corresponding two-way communication data packet according to the five-tuple information of each data packet in the traffic, thereby extracting each session composed of the two-way communication flow; Step 1.3, extract the timestamp information of each data packet from the session, traverse each data packet in turn, and calculate the time interval with the previous data packet based on the timestamp information, and then divide the session into multiple sub-sessions based on the threshold set according to the actual working environment.

3. The method according to claim 2, characterized in that In the step 1.2, the five-tuple information of the data packet includes the source IP address, source port, destination IP address, destination port and transport layer protocol.

4. The method according to claim 1, characterized in that: The step 2 comprises: Step 2.1, check the protocol type and session duration of the session, and delete the sessions of specific protocol types and sessions exceeding the preset time threshold as background traffic, so as to retain only the sessions closely related to the behavior, and then sort the retained sessions according to the session start time; Step 2.2, extract the start and end time of all the sorted sessions, and determine the time interval between the two sessions. If the interval between the end time of the previous session and the start time of the next session is less than the threshold, or there is a time overlap between the data packets of the two adjacent sessions, the sessions are considered to belong to the same behavior and merged, otherwise they are considered to belong to different behaviors. Step 2.3, sort the data packets in all sessions that have completed step 2.2 according to the timestamps, so as to obtain the behavior traffic that has been segmented and processed.

5. The method according to claim 4, characterized in that In step 2.1, sessions with protocol types of DNS, SSDP, DHCP, mDNS and NTP, as well as sessions with a duration greater than 2 minutes are deleted.

6. The method according to claim 1, characterized in that In the step 3, the packet information of the top 5 packets of the same size that appear most frequently in the behavior is extracted, wherein the packet information includes 4 types, namely, packet size, number of occurrences, average time interval and protocol type, thereby constructing a 20-dimensional unlabeled feature vector for the 5 packets in the behavior.

7. The method according to claim 6, characterized in that The step 4 comprises: Step 4.1, using Jaccard distance as the distance calculation object between each feature vector, the 4-dimensional features of each data packet in the feature vector are regarded as a set, thereby obtaining 5 sets, and then comparing the ratio of the intersection and union of the sets of two feature vectors to obtain the distance between the sets; Step 4.2, cluster the feature vectors using the DBSCAN algorithm, using the distance obtained in step 4.1 as the basis for the distance measurement algorithm in the clustering process to obtain the clustering clusters of all behavior samples; Step 4.3, label each cluster according to the clustering results to identify the behavior.

8. The method according to claim 7, characterized in that In step 4.1, the calculation formula of Jaccard distance is: ； in, Indicates the 4 features contained in each set of a feature vector. Indicates behavior, Indicates data packets, Indicates the size of the data packet. Indicates the number of occurrences of the data packet, represents the average time interval, Indicates the protocol type. represents the intersection symbol, and ∪ represents the union symbol.

9. The method according to claim 8, characterized in that The step 5 comprises: Step 5.1, for each behavior cluster, count the number of occurrences of all different data packet sizes, record the top 5 data packet sizes, and record the time interval between each data packet corresponding to the top 5 data packet sizes; Step 5.2, calculating the average number of occurrences and average time interval between packets corresponding to the top 5 packet sizes as components of the signature; Step 5.3, obtain the order of the first appearance of the data packets corresponding to the top 5 data packet sizes in all the behavior traffic of the clustering cluster, and use the order of appearance of the data packets with more appearances as the arrangement order in the signature, so as to construct the signature information of the device behavior.

10. The method according to claim 9, characterized in that The signature information representation of the final built device behavior is as follows: <size1-fre1-time1, size2-fre2-time2, size3-fre3-time3, size4-fre4-time4,size5-fre5-time5> -protocol, where size1 to size5 are arranged in the order of the data packets.

Images