Method, device, medium and equipment for identifying open flow internet of things device

Abstract

This application discloses a method, apparatus, medium, and device for identifying IoT devices in the face of open traffic. The method includes: acquiring traffic data of a device to be identified; inputting the traffic data into a device identification model to obtain the device type of the device to be identified output by the device identification model; the device type includes IoT device types stored in a database and unknown devices not stored in the database; wherein the device identification model is constructed based on the following steps: performing dimensionality reduction processing on sample traffic data of sample IoT devices, and mapping the dimensionality-reduced sample traffic data into a latent space to obtain clusters of sample IoT devices; and constructing the device identification model based on the traffic characteristics of each cluster. The method, apparatus, medium, and device provided in this application improve the accuracy of IoT device identification and are beneficial to improving the security of IoT devices.

Description

Technical Field

[0001] This application relates to the field of Internet of Things (IoT) technology, and more specifically, to a method, apparatus, medium, and device for identifying IoT devices in the face of open traffic. Background Technology

[0002] With the popularization and development of network and communication technologies, the Internet of Things (IoT) has been widely applied in various fields such as transportation, industry, healthcare, agriculture, retail, home, and military. A massive influx of miniature intelligent devices into the network, along with diverse applications and services, has provided people with a more convenient and comfortable lifestyle. However, the countless IoT devices and the complex and diverse applications of IoT also lead to severe security threats. Many IoT devices have pre-set weak keys and serious security vulnerabilities, allowing attackers to illegally intrude into and control IoT devices by guessing keys and exploiting vulnerabilities, thus reducing the security of user data.

[0003] Only by correctly identifying the types of IoT devices can targeted security strategies be deployed, security vulnerabilities patched, and timely security maintenance performed. Identifying IoT devices can also uncover abnormal or unknown devices in the network environment, allowing for rapid security maintenance based on unusual device behavior. Therefore, the identification of IoT devices is of great significance to cyberspace security.

[0004] Therefore, how to identify IoT devices and thus improve their security has become a pressing technical problem for the industry. Summary of the Invention

[0005] This application provides a method, apparatus, medium, and device for identifying IoT devices in the face of open traffic, in order to solve the technical problem of how to identify IoT devices in the prior art, thereby improving the security of IoT devices.

[0006] Firstly, this application provides a method for identifying IoT devices in the face of open traffic, including:

[0007] Obtain traffic data for the device to be identified;

[0008] The traffic data is input into the device identification model to obtain the device type of the device to be identified, which is output by the device identification model. The device type includes IoT device types stored in the database and unknown devices not stored in the database.

[0009] The device identification model is constructed based on the following steps:

[0010] The sample traffic data of the sample IoT devices is dimensionality reduced, and the dimensionality-reduced sample traffic data is mapped into the latent space to obtain the clusters of the sample IoT devices;

[0011] The device identification model is constructed based on the traffic characteristics of each cluster.

[0012] In some embodiments, the dimensionality reduction processing of the sample traffic data of the sample IoT devices includes:

[0013] Obtain session data from sample traffic data of sample IoT devices;

[0014] The session data is processed to generate a two-dimensional traffic data graph based on the processed session data;

[0015] The two-dimensional flow data graph is then subjected to dimensionality reduction processing;

[0016] The sample IoT devices include IoT devices in the sample network environment and IoT devices in the open source dataset;

[0017] The data processing includes: packet filtering, message removal, session segmentation, packet truncation and padding, protocol field processing, and data normalization.

[0018] In some embodiments, after generating a two-dimensional traffic data graph based on the processed session data, the method further includes:

[0019] An autoencoder is constructed based on the two-dimensional traffic data graph; the autoencoder includes an encoder, a classification layer, and a decoder.

[0020] The encoder is used to perform dimensionality reduction processing on the two-dimensional traffic data map and map the dimensionality-reduced two-dimensional traffic data map onto the latent space to obtain the clusters of each sample IoT device;

[0021] The encoder includes an output layer; the output layer is connected to the classification layer and is used to input the embedding vector into the classification layer; the embedding vector is determined based on the dimensionality-reduced two-dimensional traffic data graph.

[0022] The classification layer is used to predict the device type corresponding to the received embedding vector;

[0023] The decoder is used to restore the dimensionality-reduced two-dimensional traffic data graph to its original dimension and map it to the original space of the two-dimensional traffic data graph to obtain a two-dimensional reconstructed data graph.

[0024] In some embodiments, constructing the device identification model based on traffic characteristics in each cluster includes:

[0025] The encoder is obtained by training the initial encoder based on the cross-entropy of the initial encoder;

[0026] Based on the reconstruction error between the two-dimensional traffic data graph and the two-dimensional reconstructed data graph, the initial autoencoder is trained to obtain the autoencoder.

[0027] A cluster boundary model is constructed based on the distribution distance between the center vector of the cluster and the embedding vector in the cluster, and the reconstruction error;

[0028] Input the embedding vector into the cluster boundary model, obtain the matching probability between each embedding vector output by the cluster boundary model and each IoT device type, and obtain the probability vector;

[0029] The device recognition model is obtained by correcting the embedding vector output by the encoder based on the probability vector.

[0030] In some embodiments, the distribution distance is determined based on the following steps:

[0031] Determine the center vector of the cluster, and calculate the vector distance between the embedding vectors in the cluster and the center vector;

[0032] Based on the set sliding window size and the vector distance, the distribution distance between the embedding vector and the center vector is determined.

[0033] In some embodiments, inputting the traffic data into the device identification model to obtain the device type of the device to be identified output by the device identification model includes:

[0034] The traffic data is input into the device identification model to determine the matching probability between the traffic data and a first IoT device type; the first IoT device type is the IoT device type that matches the traffic data.

[0035] Compare the matching probability with the probability threshold of the first IoT device type;

[0036] If the matching probability is less than the probability threshold or the traffic data does not have a matching IoT device type, the device type of the device to be identified is determined to be an unknown device.

[0037] If the matching probability is greater than or equal to the probability threshold, the device type of the device to be identified is determined to be the first IoT device type.

[0038] In some embodiments, predicting the device type corresponding to the received embedding vector includes:

[0039] The embedding vector is parsed to determine the traffic characteristics corresponding to the embedding vector;

[0040] Predict the device type corresponding to the embedding vector based on the traffic characteristics.

[0041] Secondly, this application provides an IoT device identification device for open traffic, comprising:

[0042] The acquisition module is used to acquire traffic data of the device to be identified;

[0043] The identification module is used to input the traffic data into the device identification model to obtain the device type of the device to be identified output by the device identification model; the device type includes IoT device types stored in the database and unknown devices not stored in the database;

[0044] The construction module is used to perform dimensionality reduction processing on the sample traffic data of the sample IoT devices and map the dimensionality-reduced sample traffic data into a latent space to obtain the clusters of the sample IoT devices; based on the traffic characteristics in each cluster, the device identification model is constructed.

[0045] Thirdly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0046] Fourthly, this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to implement the above-described method when executing the program through the computer program.

[0047] The method, apparatus, medium, and device for identifying IoT devices in open traffic scenarios provided in this application reduce the dimensionality of sample traffic data to obtain clusters of IoT devices for each sample, thereby obtaining highly accurate traffic features corresponding to each cluster. A device identification model is constructed using these traffic features, improving the accuracy of IoT device identification. Even in open traffic scenarios, the device type of the device to be identified can be accurately identified. By classifying device types into known IoT device types and unknown devices, the IoT device type of the device to be identified or unknown devices can be detected. Based on the device type of the device to be identified, a security strategy can be formulated and security maintenance can be performed on the device to be identified. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0049] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 One of the flowcharts for an IoT device identification method for open traffic provided in an embodiment of this application;

[0051] Figure 2 This is a schematic diagram of the data processing flow provided in the embodiments of this application;

[0052] Figure 3 A second schematic flowchart illustrating the IoT device identification method for open traffic provided in this application embodiment;

[0053] Figure 4 One of the structural schematic diagrams of an IoT device identification device facing open traffic provided in an embodiment of this application;

[0054] Figure 5 A second schematic diagram of the structure of an IoT device identification device for open traffic provided in an embodiment of this application;

[0055] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0056] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0057] It should be noted that the terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0058] The IoT device identification method for open traffic provided in this application is applicable to terminals. Terminals can be various electronic devices with displays and web browsing capabilities, including but not limited to servers, smartphones, tablets, laptops, and desktop computers.

[0059] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of personal information all comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals.

[0060] Figure 1 This is one of the flowcharts illustrating the IoT device identification method for open traffic provided in the embodiments of this application, such as... Figure 1 As shown, the method includes steps 110 and 120. These method steps are merely one possible implementation of this application.

[0061] Step 110: Obtain traffic data of the device to be identified.

[0062] Specifically, the execution entity of the IoT device identification method for open traffic provided in this application embodiment is an IoT device identification device for open traffic. This device can be a hardware device independently installed in the terminal, or it can be a software program running in the terminal. For example, when the terminal is a mobile phone, the IoT device identification device for open traffic can be manifested as an application such as identification software in the mobile phone.

[0063] The device to be identified is a device in the current network environment that can output traffic data, such as a switch or light bulb that can be controlled by software.

[0064] Traffic data refers to the interaction data between the device to be identified and other devices through traffic. A data collection platform can be set up in the current network environment, and network sniffing tools can be deployed to capture the traffic data of the device to be identified.

[0065] For example, the server can be used as the gateway in the current network environment, and a data acquisition system and TCPDump tool can be installed. To enable the data acquisition platform to simultaneously capture traffic data from devices being identified via both wired and wireless transmissions, a wireless router can be connected to the server as a wireless repeater. For devices requiring special protocols to access the network, a smart gateway compatible with the specific protocol can be deployed before connecting it to the router or server. After the data acquisition platform is set up, TCPDump can be used to monitor the traffic data of all devices being identified on the local area network.

[0066] Step 120: Input traffic data into the device identification model to obtain the device type of the device to be identified output by the device identification model; the device type includes IoT device types stored in the database and unknown devices not stored in the database;

[0067] The device identification model is constructed based on the following steps:

[0068] The sample traffic data of the sample IoT devices is dimensionality reduced, and the dimensionality-reduced sample traffic data is mapped into the latent space to obtain the clusters of sample IoT devices;

[0069] Based on the traffic characteristics of each cluster, a device identification model is constructed.

[0070] Specifically, the device identification model is a neural network model used to identify device types.

[0071] Because the device type of the device to be identified may be an IoT device type already stored in the database; it may also be an IoT device type not stored in the database; or it may not belong to the category of IoT devices. In this embodiment of the application, IoT device types not stored in the database and devices that do not belong to the category of IoT devices are referred to as unknown devices.

[0072] Therefore, the device identification model in this application embodiment may output a specific IoT device type or an unknown device.

[0073] The actual scenario for identifying the device to be identified falls under the category of open traffic scenarios. Open traffic scenarios refer to situations where the network environment is open, and new, unknown devices will continuously emerge.

[0074] In open traffic scenarios, traffic data from unknown devices can affect the identification results of device identification models. To improve the accuracy of device identification models in open traffic scenarios, this application embodiment performs dimensionality reduction processing on the sample traffic data of sample IoT devices. The sample traffic data after dimensionality reduction will reveal more significant traffic characteristics.

[0075] Mapping the dimensionality-reduced sample traffic data onto a latent space yields the clusters of each sample IoT device. Extracting traffic features from each cluster reveals the traffic characteristics of each sample IoT device. Training the model based on these traffic features and their corresponding IoT device types results in a highly accurate device identification model.

[0076] The IoT device identification method for open traffic scenarios provided in this application reduces the dimensionality of sample traffic data to obtain clusters of IoT devices, thereby obtaining highly accurate traffic features corresponding to each cluster. A device identification model is constructed using these traffic features, improving the accuracy of IoT device identification. Even in open traffic scenarios, the device type of the device to be identified can be accurately identified. By classifying device types into known IoT device types and unknown devices, the IoT device type of the device to be identified or unknown devices can be detected. Based on the device type of the device to be identified, a security strategy can be formulated and security maintenance can be performed.

[0077] It should be noted that each implementation method of this application can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0078] In some embodiments, the sample traffic data of the sample IoT devices is subjected to dimensionality reduction processing, including:

[0079] Obtain session data from sample traffic data of sample IoT devices;

[0080] The session data is processed to generate a two-dimensional traffic data graph based on the processed session data;

[0081] Dimensionality reduction processing is performed on the two-dimensional flow data graph;

[0082] The sample IoT devices include IoT devices in the sample network environment and IoT devices in the open source dataset;

[0083] Data processing includes: packet filtering, message removal, session segmentation, packet truncation and padding, protocol field processing, and data normalization.

[0084] Specifically, a data acquisition platform can be built to collect sample traffic data from sample IoT devices. The sample traffic data of each sample IoT device can be distinguished and labeled using either a Media Access Control (MAC) address or an Internet Protocol (IP) address. The labeling granularity can be adjusted according to the actual use case.

[0085] For example, if only coarse-grained labels need to be identified, the sample traffic data can be labeled as device type; if the manufacturer level needs to be identified, the sample traffic data can be labeled as manufacturer-device type; if the manufacturer and model level needs to be identified, the sample traffic data can be labeled as manufacturer-device type-model. Labeling the sample traffic data is beneficial for subsequent tracking and processing.

[0086] The sample IoT devices include IoT devices in the sample network environment and IoT devices in the open-source dataset. The sample network environment is a manually constructed virtual network environment used to train the device recognition model.

[0087] Open-source datasets can be used to expand sample traffic data, improve the multi-source nature and diversity of sample traffic data, and thus improve the accuracy of device identification models.

[0088] Session data refers to continuous communication activities within a certain time frame. It has specific start and end markers and involves data exchange between two or more IoT devices. A session can contain multiple data streams and packets to implement a specific application or service.

[0089] IoT devices generate different session data based on specific business or service content. Therefore, there is a correlation between the data characteristics of session data in sample traffic data and the type characteristics of the corresponding IoT device type. By parsing the session data, the device type of IoT devices can be effectively identified.

[0090] To further reduce interference, the session data can be processed. Figure 2 This is a schematic diagram of the data processing flow provided in the embodiments of this application; such as Figure 2 As shown, the data processing includes steps 210 and 260.

[0091] Step 210: Data Packet Filtering

[0092] In scenarios with open traffic, the data packets of session data collected may include data from Dynamic Host Configuration Protocol (DHCP) and Network Time Protocol (NTP). These protocol data may contain fields such as IP and MAC addresses, which may interfere with the identification of IoT devices. Therefore, packet filtering is necessary.

[0093] Step 220: Message Removal

[0094] Irrelevant packet headers can be removed, while retaining bidirectional stream data in the session data and extracting payload data from the bidirectional stream data. This helps prevent the device recognition model from overfitting to irrelevant fields during training. The pcap file in the session data contains pcap file header information, and the packets in the sample traffic data also contain fields that can easily cause overfitting. Therefore, it is necessary to filter out fields that are prone to overfitting and remove packet header information.

[0095] Step 230, Session Segmentation

[0096] Session data is segmented using a 5-tuple (source IP address, source port, destination IP address, destination port, and protocol). Due to uncontrollable factors such as network congestion, captured session data packets may be out of order, lost, or duplicated. Therefore, sequence numbers and timestamp intervals can be used to determine whether certain packets need to be corrected or removed.

[0097] Step 240: Data Packet Truncation and Padding

[0098] Session data from IoT devices is typically short and compact, so standardized session data can be obtained through padding and truncation. The first 16 packets of session data can be used, and then the first 256 bytes of payload data can be truncated for standardization, with zero-padding applied to all missing portions.

[0099] Step 250, Protocol Field Processing

[0100] Taking the Transmission Control Protocol (TCP) packet format as an example, the TCP packet format includes a 4-bit version number, a 4-bit header length, and 3-bit flags, among other protocol fields. These compact fields are determined based on the header size, parsing time, and bandwidth resources. Truncation of these fields using 8 bits could easily lead to information loss. Therefore, the compact fields of the header data are normalized, automatically padding fields less than a multiple of 8 bits to 8 bits.

[0101] Step 260: Data Normalization

[0102] After converting each byte in the session data to decimal and then normalizing it, each byte is converted into a decimal number from 0 to 255, which normalizes the value range to the range [0, 1].

[0103] During data processing, session data needs to be kept in order, while the payload of the session data is retained for subsequent extraction of traffic features.

[0104] Steps 210 to 260 arrange the processed Transmission Control Protocol (TCP) traffic packets in the session data sequentially to form a two-dimensional traffic data graph, which can be a two-dimensional matrix. Dimensionality reduction processing can be performed on the two-dimensional traffic data graph to extract traffic features.

[0105] The IoT device identification method for open traffic provided in this application can remove interference data and improve the accuracy of the device identification model by acquiring session data from sample traffic data and processing the session data.

[0106] In some embodiments, after generating a two-dimensional traffic data graph based on the processed session data, the method further includes:

[0107] An autoencoder is constructed based on a two-dimensional traffic data graph; the autoencoder includes an encoder, a classification layer, and a decoder.

[0108] The encoder is used to reduce the dimensionality of the two-dimensional traffic data graph and map the dimensionality-reduced two-dimensional traffic data graph to the latent space to obtain the clusters of each sample IoT device;

[0109] The encoder includes an output layer; the output layer is connected to the classification layer and is used to input the embedding vector into the classification layer; the embedding vector is determined based on the dimensionality-reduced two-dimensional flow data graph;

[0110] The classification layer is used to predict the device type corresponding to the received embedding vector;

[0111] The decoder is used to restore the dimensionality-reduced two-dimensional traffic data graph to its original dimension and map it to the original space of the two-dimensional traffic data graph to obtain the two-dimensional reconstructed data graph.

[0112] Specifically, an autoencoder can be constructed based on a two-dimensional flow data graph. Autoencoders have strong feature representation capabilities and can use the input two-dimensional flow data graph as the learning target for representation learning.

[0113] The autoencoder in this application includes an encoder, a classification (Softmax) layer, and a decoder.

[0114] The encoder can also act as a feature extractor, reducing the dimensionality of the two-dimensional traffic data graph and mapping the dimensionality-reduced two-dimensional traffic data graph of each sample IoT device to a low-dimensional compact latent space, generating clusters of each sample IoT device, and extracting the traffic features of the sample IoT devices from the clusters.

[0115] The encoder includes an output layer, also known as an embedding layer. The output layer is connected to the classification layer and is used to input the embedding vector into the classification layer. The embedding vector can be determined based on the dimensionality-reduced two-dimensional flow data graph.

[0116] The encoder's output layer dimension is smaller than the dimension of the input two-dimensional traffic data graph, thus requiring dimensionality reduction processing of the two-dimensional traffic data graph. Dimensionality reduction processing can remove redundant information in the two-dimensional traffic data graph and obtain traffic features that are strongly correlated with IoT device types.

[0117] The encoder's output layer sends the embedding vector to the classification layer. Based on the received embedding vector, the classification layer predicts the device type corresponding to that vector. The classification layer can also parse the embedding vector to determine the traffic characteristics it corresponds to; based on these traffic characteristics, it predicts the device type corresponding to the embedding vector.

[0118] The decoder is used to restore the dimensionality-reduced 2D traffic data graph to its original dimensions and map it back to the original space of the 2D traffic data graph, resulting in a reconstructed 2D data graph. The decoder's output layer can serve as the reconstruction layer, outputting the reconstructed 2D data graph. The reconstructed 2D data graph has the same dimensions as the 2D traffic data graph.

[0119] The IoT device identification method for open traffic provided in this application constructs an autoencoder through a two-dimensional traffic data graph. The classification layer of the autoencoder can output the device type corresponding to the embedding vector, thereby determining the device type of the sample IoT device corresponding to the embedding vector, which improves the identification accuracy of the device identification model.

[0120] In some embodiments, a device identification model is constructed based on traffic characteristics in each cluster, including:

[0121] The initial encoder is trained based on its cross-entropy to obtain the encoder.

[0122] Based on the reconstruction error between the two-dimensional traffic data graph and the two-dimensional reconstructed data graph, the initial autoencoder is trained to obtain the autoencoder.

[0123] A cluster boundary model is constructed based on the distribution distance between the cluster center vector and the embedding vectors in the cluster, as well as the reconstruction error.

[0124] Input the embedding vector into the cluster boundary model, obtain the matching probability of each embedding vector output by the cluster boundary model with each IoT device type, and obtain the probability vector;

[0125] The device recognition model is obtained by correcting the embedding vector output by the encoder based on the probability vector.

[0126] Specifically, the initial encoder can be trained using cross-entropy. The formula for calculating cross-entropy L1 is as follows:

[0127]

[0128] Here, cross-entropy is calculated from multiple samples. M represents the number of input two-dimensional traffic data graphs, N represents the number of IoT device types in the database, and C... j Let x represent the sample space of the j-th type of IoT device in the database. i This represents the i-th two-dimensional flow data graph input, δ ij Represents a symbolic function. p(C j |x i ) represents the Softmax function. L represents the embedding vector obtained from the feature embedding layer. The embedding vector contains N values, and k means traversing these N values.

[0129] After training the initial encoder using cross-entropy, to ensure that the embedded vectors (Logits layer vectors) output by the trained encoder can effectively capture and reconstruct the traffic features in the two-dimensional traffic data graph, the initial autoencoder can be further trained based on the reconstruction error between the two-dimensional traffic data graph and the two-dimensional reconstructed data graph. The formula for calculating the reconstruction error L2 is as follows:

[0130]

[0131] Where f represents the encoder and g represents the decoder.

[0132] This application embodiment trains an autoencoder based on L1 and L2 algorithms, which can automatically extract traffic features of sample IoT devices, output more accurate embedding vectors, and obtain a more accurate two-dimensional reconstructed data map. The higher the accuracy of the embedding vector, the stronger its correlation with the type features of the corresponding IoT device type.

[0133] Since the sample IoT devices all belong to the IoT device types stored in the database, and are all known IoT device types, it is necessary to output the type of the unknown device in order to complete the detection of unknown devices. Therefore, this embodiment of the application corrects the embedding vector output by the encoder by constructing a cluster boundary model, thereby obtaining a final device identification model that can identify known IoT device types and detect unknown devices.

[0134] Spatial probability distributions can be used for spatial modeling to obtain cluster boundary models. Spatial probability distributions include the Weibull distribution or the normal distribution, among others.

[0135] Taking the Weibull distribution as an example, the embedding vector in the embedding space generally follows a Weibull distribution. The Weibull distribution, also known as the Weibull distribution, is the theoretical basis for reliability analysis and lifetime verification. It can model the distribution region in the embedding space, thereby assessing the probability that a point in the embedding space belongs to a certain region. The higher the probability, the more likely the point is to belong to that region. In this embodiment, the embedding vector is a point in the embedding space. The Weibull cumulative distribution function ρ is shown below:

[0136]

[0137] Among them, D(L; C i ) represents L and C i The distance metric function between them, η i and κ i These are hyperparameters controlling the distribution scale and shape, respectively, while α controls the reliability. After training the autoencoder based on L1 and L2, the embedding vectors L = f(x) = (L1, L2, ..., L...) of all two-dimensional flow data graphs can be obtained. N ).

[0138] Typically D(L;C) i ) is based on L and C i Using the Euclidean distance of the mean vector as a metric, this application improves and optimizes the distance metric function by adding reconstruction error as feedback, so that the Weibull distribution considers both the cluster distance and the reconstruction distance in the embedding space during the modeling process. The cluster distance is the distribution distance between the embedding vector and the center vector in the embedding space, and the reconstruction distance is the offset distance between the reconstructed vector after reconstruction by the autoencoder and the input embedding vector. The metric formula in this application is D(L; C). i (x; x′) is shown below:

[0139]

[0140] Where ω is the training parameter, μ represents the center vector of the cluster, and the reconstructed vector x′=g(x).

[0141] In fitting the Weibull distribution, a hyperparameter n can be introduced, called the tail size, which represents the n embedding vectors furthest from the cluster center vector. Therefore, we only need to find the n furthest embedding vectors to model the Weibull distribution space.

[0142] To reduce the impact of embedding vectors that are far from the center vector on the modeling results in the Weibull distribution space modeling process, the n farthest embedding vectors can be smoothed. The n smoothed embedding vectors that are farthest away are selected, and spatial modeling is performed according to the Weibull distribution to obtain the cluster boundary model of each cluster.

[0143] The distance of each embedding vector from its center vector in the embedding space can be calculated. Based on a smoothing algorithm, the n furthest distances are obtained. The n furthest embedding vectors corresponding to these n furthest distances are then fitted with a Weibull distribution. Finally, using the obtained cluster boundary model, the probability that the smoothed n furthest embedding vectors belong to each IoT device type is calculated, resulting in the probability vector w. i :

[0144] w i =MODEL.score(x i )

[0145] The embedding vector output by the encoder in the autoencoder can be corrected by probability vector correction, and the boundaries between IoT device types and unknown devices in the database can be distinguished, thereby realizing the detection of unknown devices.

[0146] Taking an example where there are N types of IoT devices stored in the database, after calibration, the autoencoder generates the final device recognition model. At this point, the original N outputs are transformed into N+1 outputs, thus enabling the device recognition model to perform the identification of known IoT device types and the detection of unknown devices.

[0147] The embedding vectors and probability vectors of N IoT device types can be multiplied by a Hadamard product to identify the input traffic data as traffic data of IoT device types or detect it as traffic data of unknown devices.

[0148]

[0149] The corrected embedding vector is then input into the classification layer, which can output N+1 different results.

[0150] The IoT device identification method for open traffic provided in this application improves the accuracy of identification by training the autoencoder with cross-entropy and reconstruction error. By constructing a cluster boundary model and further optimizing the embedding vector output by the encoder with the probability vector output by the cluster boundary model, it can identify known IoT device types and detect unknown devices.

[0151] In some embodiments, the distribution distance is determined based on the following steps:

[0152] Determine the center vector of the cluster, and calculate the vector distance between the embedding vectors in the cluster and the center vector;

[0153] Based on the set sliding window size and vector distance, the distribution distance between the embedded vector and the center vector is determined.

[0154] Specifically, the cluster center vector μ is calculated, and the n furthest embedding vectors are selected for Weibull distribution fitting. However, outliers may exist among the selected n embedding vectors, which clearly do not belong to any cluster and may affect the modeling effect of the cluster boundaries.

[0155] Therefore, a sliding window is set with a size of k. When selecting n embedding vectors, a redundant selection method is adopted, that is, the n+k-1 embedding vectors farthest from the center vector are selected, and the distance d from the center vector is calculated. i :

[0156] d i =||l i -μ||2

[0157] Sort the obtained distances in ascending order, and denote them as d1, d2, ..., d n+k-1 Then, the distribution distance d′ of the embedding vector is obtained by averaging the data over each k data points within a window. The specific calculation formula is shown below:

[0158]

[0159]

[0160] ...

[0161]

[0162] After calculating n+k-1 distances, the sliding window is moved. Each time the window is moved, the k data points in the next window are averaged within the window, resulting in n smoothed vector distances d′. This embodiment uses these smoothed vector distances to model the Weibull distribution space, obtaining a cluster boundary model.

[0163] The IoT device identification method for open traffic provided in this application reduces the impact of outliers on the modeling of the row-Weibull distribution space by smoothing the embedded vector and calculating the distribution distance between the embedded vector and the center vector, thereby improving the identification accuracy.

[0164] In some embodiments, step 120 includes:

[0165] Input traffic data into the device identification model to determine the matching probability between the traffic data and the first IoT device type; the first IoT device type is the IoT device type that matches the traffic data.

[0166] Compare the matching probability with the probability threshold of the first IoT device type;

[0167] If the matching probability is less than the probability threshold or there is no matching IoT device type in the traffic data, the device type of the device to be identified is determined to be an unknown device.

[0168] If the matching probability is greater than or equal to the probability threshold, the device type of the device to be identified is determined to be the first IoT device type.

[0169] Specifically, obtain the minimum discriminant confidence p for each known IoT device type on the training set. i The minimum discrimination confidence level is used as the probability threshold for each type of IoT device.

[0170] The first IoT device type is a known IoT device type, that is, an IoT device type stored in the database.

[0171] If the matching probability is less than the probability threshold for the first IoT device type, the identification result of the traffic data is an unknown device.

[0172] If the traffic data does not match the IoT device type, then the identification result of the traffic data is an unknown device.

[0173] If the matching probability is greater than or equal to the probability threshold of the first IoT device type, then the identification result of the traffic data is the first IoT device type.

[0174] The IoT device identification method for open traffic provided in this application embodiment can determine whether the device type of the device to be identified is a first IoT device by using a probability threshold, thereby improving the accuracy of identification.

[0175] Figure 3 This is a second schematic flowchart illustrating the IoT device identification method for open traffic provided in this application embodiment, as shown below. Figure 3 As shown, the method includes:

[0176] Step 310: Collect, analyze, and label traffic data from IoT devices.

[0177] Step 320: Extract session data from the traffic data, and preprocess the session data through packet filtering, message removal, packet truncation and padding, field normalization, and normalization to obtain a two-dimensional traffic data graph of the session data.

[0178] Step 330: Construct an autoencoder based on the two-dimensional traffic data graph. The encoder in the autoencoder also acts as a feature extractor, mapping various sample traffic data into a low-dimensional, compact latent space to generate clusters of IoT devices. Traffic features of IoT devices are extracted from these clusters, and the predicted device category is output using a Softmax layer. The decoder in the autoencoder is used to obtain the two-dimensional reconstructed data graph corresponding to the two-dimensional traffic data graph.

[0179] Step 340: Cluster boundary optimization, smoothing the embedding vector.

[0180] Step 350: Calculate the distribution distance between the cluster center vector and the embedding vector of each IoT device in the latent space, calculate the reconstruction error between the two-dimensional traffic data graph and the two-dimensional reconstructed data graph, and use the reconstruction error as the reconstruction distance. Based on the sum of the distribution distance and the reconstruction distance, perform spatial modeling on the boundaries of all clusters to obtain the cluster boundary model.

[0181] Step 360: Calculate the probability that the input traffic data belongs to each IoT device type based on the cluster boundary model, forming a probability vector. Then, correct the encoder's prediction result in the autoencoder based on the probability vector. This identifies the input traffic data as traffic data belonging to IoT device types or detects it as traffic data from unknown devices.

[0182] The IoT device identification method for open traffic provided in this application embodiment can identify the IoT device type of the device to be identified or detect unknown devices, thereby formulating security policies and performing security maintenance for the device to be identified based on the device type, with a high identification accuracy.

[0183] The following describes the IoT device identification apparatus for open traffic provided in the embodiments of this application. The IoT device identification apparatus for open traffic described below can be referred to in correspondence with the IoT device identification method for open traffic described above.

[0184] Figure 4 This is one of the structural schematic diagrams of an IoT device identification device for open traffic provided in the embodiments of this application, such as... Figure 4 As shown, the device includes an acquisition module 410, an identification module 420, and a construction module 430.

[0185] The acquisition module is used to acquire traffic data of the device to be identified;

[0186] The identification module is used to input traffic data into the device identification model and obtain the device type of the device to be identified output by the device identification model; the device type includes IoT device types stored in the database and unknown devices not stored in the database;

[0187] The module is used to perform dimensionality reduction on the sample traffic data of sample IoT devices and map the dimensionality-reduced sample traffic data into a latent space to obtain the clusters of sample IoT devices; based on the traffic characteristics in each cluster, a device identification model is constructed.

[0188] Specifically, according to embodiments of this application, any number of modules among the acquisition module, identification module, and construction module can be merged into one module, or any one of these modules can be split into multiple modules.

[0189] Alternatively, at least some of the functionality of one or more of these modules can be combined with at least some of the functionality of other modules and implemented in a single module.

[0190] According to embodiments of this application, at least one of the acquisition module, identification module, and construction module can be at least partially implemented as a hardware circuit, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in a suitable combination of any of them.

[0191] Alternatively, at least one of the acquisition module, identification module, and construction module can be implemented at least partially as a computer program module that can perform corresponding functions when the computer program module is run.

[0192] The IoT device identification device for open traffic provided in this application embodiment reduces the dimensionality of sample traffic data to obtain clusters of various sample IoT devices, thereby obtaining highly accurate traffic features corresponding to each cluster. A device identification model is constructed using these traffic features, improving the accuracy of IoT device identification. Even in open traffic scenarios, the device type of the device to be identified can be accurately identified. By classifying device types into known IoT device types and unknown devices, the IoT device type of the device to be identified or unknown devices can be detected. Based on the device type of the device to be identified, a security strategy can be formulated for the device to be identified, and security maintenance can be performed.

[0193] In some embodiments, the building module includes a dimensionality reduction submodule, which is used for:

[0194] Obtain session data from sample traffic data of sample IoT devices;

[0195] The session data is processed to generate a two-dimensional traffic data graph based on the processed session data;

[0196] Dimensionality reduction processing is performed on the two-dimensional flow data graph;

[0197] The sample IoT devices include IoT devices in the sample network environment and IoT devices in the open source dataset;

[0198] Data processing includes: packet filtering, message removal, session segmentation, packet truncation and padding, protocol field processing, and data normalization.

[0199] In some embodiments, the build module further includes a first build submodule, the first build submodule being configured to:

[0200] An autoencoder is constructed based on a two-dimensional traffic data graph; the autoencoder includes an encoder, a classification layer, and a decoder.

[0201] The encoder is used to reduce the dimensionality of the two-dimensional traffic data graph and map the dimensionality-reduced two-dimensional traffic data graph to the latent space to obtain the clusters of each sample IoT device;

[0202] The encoder includes an output layer; the output layer is connected to the classification layer and is used to input the embedding vector into the classification layer; the embedding vector is determined based on the dimensionality-reduced two-dimensional flow data graph;

[0203] The classification layer is used to predict the device type corresponding to the received embedding vector;

[0204] The decoder is used to restore the dimensionality-reduced two-dimensional traffic data graph to its original dimension and map it to the original space of the two-dimensional traffic data graph to obtain the two-dimensional reconstructed data graph.

[0205] In some embodiments, the build module further includes a second build submodule, the second build submodule being used for:

[0206] The initial encoder is trained based on its cross-entropy to obtain the encoder.

[0207] Based on the reconstruction error between the two-dimensional traffic data graph and the two-dimensional reconstructed data graph, the initial autoencoder is trained to obtain the autoencoder.

[0208] A cluster boundary model is constructed based on the distribution distance between the cluster center vector and the embedding vectors in the cluster, as well as the reconstruction error.

[0209] Input the embedding vector into the cluster boundary model, obtain the matching probability of each embedding vector output by the cluster boundary model with each IoT device type, and obtain the probability vector;

[0210] The device recognition model is obtained by correcting the embedding vector output by the encoder based on the probability vector.

[0211] In some embodiments, the second construction submodule is specifically used for:

[0212] Determine the center vector of the cluster, and calculate the vector distance between the embedding vectors in the cluster and the center vector;

[0213] Based on the set sliding window size and vector distance, the distribution distance between the embedded vector and the center vector is determined.

[0214] In some embodiments, the identification module is specifically used for:

[0215] Input traffic data into the device identification model to determine the matching probability between the traffic data and the first IoT device type; the first IoT device type is the IoT device type that matches the traffic data.

[0216] Compare the matching probability with the probability threshold of the first IoT device type;

[0217] If the matching probability is less than the probability threshold or the traffic data does not have a matching IoT device type, the device type of the device to be identified is determined to be an unknown device.

[0218] If the matching probability is greater than or equal to the probability threshold, the device type of the device to be identified is determined to be the first IoT device type.

[0219] In some embodiments, the first construction submodule is specifically used for:

[0220] The embedding vector is parsed to determine the traffic characteristics corresponding to the embedding vector;

[0221] Predict the device type corresponding to the embedded vector based on traffic features.

[0222] Figure 5 This is a second schematic diagram of the structure of an IoT device identification device for open traffic provided in an embodiment of this application; as shown below. Figure 5 As shown: The IoT device identification device for open traffic in this application embodiment includes:

[0223] The traffic acquisition and processing module 510 is used to collect, analyze, and label traffic data from IoT devices.

[0224] The session extraction module 520 is used to extract session data from traffic data and preprocess the session data through packet filtering, message removal, packet truncation and padding, field normalization, and normalization.

[0225] The feature extraction module 530 is used to construct an autoencoder, extract the traffic features of IoT devices, and obtain a two-dimensional reconstructed data map corresponding to the two-dimensional traffic data map.

[0226] The cluster boundary optimization module 540 is used for cluster boundary optimization and smoothing of the embedding vector.

[0227] The cluster spatial distribution modeling module 550 is used to calculate the center vector, distribution distance, and reconstruction distance of each cluster in the embedding space, and to perform spatial modeling of the boundaries of each cluster.

[0228] The identification and detection module 560 is used to identify known IoT device types and detect unknown devices.

[0229] It should be noted that the IoT device identification device for open traffic provided in this application embodiment can implement all the method steps implemented in the above-described IoT device identification method embodiment for open traffic and can achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0230] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communications bus 640. The processor 610 can call logical commands stored in the memory 630 to execute the above-described method, which includes:

[0231] Obtain traffic data for the device to be identified;

[0232] Input traffic data into the device identification model to obtain the device type of the device to be identified output by the device identification model; the device type includes IoT device types stored in the database and unknown devices not stored in the database;

[0233] The device identification model is constructed based on the following steps:

[0234] The sample traffic data of the sample IoT devices is dimensionality reduced, and the dimensionality-reduced sample traffic data is mapped into the latent space to obtain the clusters of sample IoT devices;

[0235] Based on the traffic characteristics of each cluster, a device identification model is constructed.

[0236] Furthermore, the logical commands in the aforementioned memory can be implemented as software functional modules and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several commands to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0237] The processor in the electronic device provided in this application embodiment can call logical instructions in the memory to implement the above method. Its specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effect, which will not be repeated here.

[0238] This application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments.

[0239] The specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effects, so it will not be repeated here.

[0240] This application provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0241] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0242] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0243] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for identifying IoT devices in the face of open traffic, characterized in that, include: Obtain traffic data for the device to be identified; The traffic data is input into the device identification model to obtain the device type of the device to be identified, which is output by the device identification model. The device type includes IoT device types stored in the database and unknown devices not stored in the database. The device identification model is constructed based on the following steps: The sample traffic data of the sample IoT devices is dimensionality reduced, and the dimensionality-reduced sample traffic data is mapped into the latent space to obtain the clusters of the sample IoT devices; The device identification model is constructed based on the traffic characteristics of each cluster; The step of constructing the device identification model based on traffic characteristics in each cluster includes: Based on the distribution distance between the center vector of the cluster and the embedding vectors in the cluster, and the reconstruction error, a cluster boundary model is constructed. The embedding vectors are input into the cluster boundary model, and the matching probability between each embedding vector output by the cluster boundary model and each IoT device type is obtained to obtain a probability vector. Based on the probability vector, the embedding vectors output by the encoder of the autoencoder are corrected to obtain the device recognition model. The autoencoder includes the encoder, the classification layer, and the decoder. The encoder includes an output layer, which is connected to the classification layer. The autoencoder is constructed based on a two-dimensional traffic data graph, which is obtained based on session data in the sample traffic data.

2. The method for identifying IoT devices facing open traffic according to claim 1, characterized in that, The step of dimensionality reduction processing of sample traffic data from sample IoT devices includes: Obtain session data from sample traffic data of sample IoT devices; The session data is processed to generate a two-dimensional traffic data graph based on the processed session data; The two-dimensional flow data graph is then subjected to dimensionality reduction processing; The sample IoT devices include IoT devices in the sample network environment and IoT devices in the open source dataset; The data processing includes: packet filtering, message removal, session segmentation, packet truncation and padding, protocol field processing, and data normalization.

3. The method for identifying IoT devices facing open traffic according to claim 2, characterized in that, The encoder is used to perform dimensionality reduction processing on the two-dimensional traffic data map and map the dimensionality-reduced two-dimensional traffic data map onto the latent space to obtain the clusters of each sample IoT device; The output layer is used to input the embedding vector into the classification layer; the embedding vector is determined based on the dimensionality-reduced two-dimensional traffic data graph. The classification layer is used to predict the device type corresponding to the received embedding vector; The decoder is used to restore the dimensionality-reduced two-dimensional traffic data graph to its original dimension and map it to the original space of the two-dimensional traffic data graph to obtain a two-dimensional reconstructed data graph.

4. The IoT device identification method for open traffic as described in claim 3, characterized in that, The construction of the autoencoder based on the two-dimensional flow data graph includes: The encoder is obtained by training the initial encoder based on the cross-entropy of the initial encoder; Based on the reconstruction error between the two-dimensional traffic data graph and the two-dimensional reconstructed data graph, the initial autoencoder is trained to obtain the autoencoder.

5. The method for identifying IoT devices facing open traffic according to claim 4, characterized in that, The distribution distance is determined based on the following steps, including: Determine the center vector of the cluster, and calculate the vector distance between the embedding vectors in the cluster and the center vector; Based on the set sliding window size and the vector distance, the distribution distance between the embedding vector and the center vector is determined.

6. The method for identifying IoT devices facing open traffic according to claim 1, characterized in that, The process of inputting the traffic data into the device identification model to obtain the device type of the device to be identified, as output by the device identification model, includes: The traffic data is input into the device identification model to determine the matching probability between the traffic data and a first IoT device type; the first IoT device type is the IoT device type that matches the traffic data. The matching probability is compared with the probability threshold of the first IoT device type; If the matching probability is less than the probability threshold or the traffic data does not have a matching IoT device type, the device type of the device to be identified is determined to be an unknown device. If the matching probability is greater than or equal to the probability threshold, the device type of the device to be identified is determined to be the first IoT device type.

7. The method for identifying IoT devices facing open traffic according to claim 3, characterized in that, The step of predicting the device type corresponding to the received embedding vector includes: The embedding vector is parsed to determine the traffic characteristics corresponding to the embedding vector; Predict the device type corresponding to the embedding vector based on the traffic characteristics.

8. An IoT device identification device for open traffic, characterized in that, include: The acquisition module is used to acquire traffic data of the device to be identified; The identification module is used to input the traffic data into the device identification model and obtain the device type of the device to be identified output by the device identification model; the device type includes IoT device types stored in the database and unknown devices not stored in the database; A construction module is used to perform dimensionality reduction processing on the sample traffic data of the sample IoT devices, and map the dimensionality-reduced sample traffic data into a latent space to obtain the clusters of the sample IoT devices; based on the traffic characteristics in each cluster, the device identification model is constructed. Specifically, the construction module is used to: obtain a two-dimensional traffic data map based on the session data in the sample traffic data; construct an autoencoder based on the two-dimensional traffic data map, the autoencoder including an encoder, a classification layer, and a decoder, the encoder including an output layer connected to the classification layer; construct a cluster boundary model based on the distribution distance between the center vector of the cluster and the embedding vector in the cluster, and the reconstruction error; input the embedding vector into the cluster boundary model; obtain the matching probability of each embedding vector output by the cluster boundary model with each IoT device type to obtain a probability vector; and correct the embedding vector output by the encoder based on the probability vector to obtain the device identification model.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the IoT device identification method for open traffic as described in any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the Internet of Things device identification method for open traffic as described in any one of claims 1 to 7 through the computer program.

Images