Internet of Things protocol automatic identification system and method based on data model

Abstract

The invention provides an Internet of Things protocol automatic identification system and method based on a data model, and relates to the technical field of Internet of Things protocol identification. The method comprises the following steps: performing data analysis on a communication data packet, extracting a protocol head feature, a data field feature and a time pattern feature of the communication data packet, generating an Internet of Things protocol feature, and performing protocol analysis on the Internet of Things protocol feature according to a first protocol identification model to obtain a first protocol identification result; acquiring transmission state monitoring data of the Internet of Things equipment about the communication data packet, extracting Internet of Things transmission characteristics of the communication data packet from the transmission state monitoring data, and performing transmission analysis on the Internet of Things transmission characteristics according to a second protocol identification model to obtain a second protocol identification result; and determining a protocol analysis list according to the first protocol identification result and the second protocol identification result, and performing data analysis on the communication data packet based on a plurality of analyzers in the protocol analysis list. According to the invention, the accuracy of automatic identification of the Internet of Things protocol is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things protocol identification, and in particular to a data model-based automatic identification system and method for Internet of Things protocols. Background Art

[0002] IoT devices typically use various standardized or customized communication protocols for data transmission, such as MQTT, CoAP, and HTTP. These protocols often have multiple versions or customized variants for different devices or application scenarios, requiring devices to identify and adapt to different protocol versions during communication. However, some protocol identification methods rely on manual configuration or version determination based on a single protocol field, lacking automation capabilities and making it difficult to cope with the dynamic changes and switching requirements of protocol versions in IoT systems.

[0003] In practice, some protocols may not clearly label version numbers, or the differences between versions cannot be identified through simple header field parsing. This problem is particularly prominent in custom or variant protocols. IoT devices often operate under varying network conditions or device states. These state changes can cause devices to switch to different protocol versions. Existing technologies cannot dynamically perceive device status and network environments, and lack effective automated version switching mechanisms. This leads to inefficient protocol identification and can even cause communication errors.

[0004] Therefore, how to provide an automated protocol identification solution based on data models is an urgent problem to be solved in the field of IoT protocol identification. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a data model-based automatic identification system and method for the Internet of Things protocol, so as to realize automatic and accurate identification of the Internet of Things protocol based on the data model.

[0006] To achieve the above objectives, the present invention provides, in a first aspect, a method for automatically identifying an Internet of Things protocol based on a data model, comprising: After receiving the communication data packet sent by the IoT device, the communication data packet is parsed to extract the protocol header features, data field features and time pattern features of the communication data packet; Performing feature splicing on the protocol header features, data field features, and time pattern features of the communication data packet to generate an IoT protocol feature, performing protocol analysis on the IoT protocol feature according to a first protocol identification model to obtain a first protocol identification result, wherein the first protocol identification model performs protocol analysis on the IoT protocol feature based on a protocol feature database, and the protocol feature database is constructed based on historical transmission data and protocol version data of IoT devices; Acquiring transmission status monitoring data about communication data packets from an IoT device, extracting IoT transmission features of the communication data packets from the transmission status monitoring data, and performing transmission analysis on the IoT transmission features according to a second protocol identification model to obtain a second protocol identification result, wherein the second protocol identification model performs transmission analysis on the IoT transmission features based on a distance feature analysis unit and a discrete feature analysis unit; A protocol parsing list is determined according to the first protocol identification result and the second protocol identification result, and data parsing is performed on the communication data packet based on a plurality of parsers in the protocol parsing list.

[0007] Preferably, performing protocol analysis on the IoT protocol features according to the first protocol identification model includes: Extracting a first IoT protocol feature vector corresponding to the IoT protocol feature, and correcting the first IoT protocol feature vector according to a corrected protocol feature vector of each protocol cluster in a protocol feature database to obtain a second IoT protocol feature vector for each protocol cluster; The IoT protocol features are matched with multiple protocol versions in a protocol feature database based on multiple second IoT protocol feature vectors, including determining the protocol feature vector of each protocol version and the second IoT protocol feature vector of the protocol cluster to which it belongs, performing similarity analysis on the protocol feature vector of each protocol version and the second IoT protocol feature vector of the protocol cluster to which it belongs, sorting multiple protocol versions according to similarity values ​​obtained from the similarity analysis, screening out multiple first target protocols from the multiple protocol versions and generating a first protocol identification result.

[0008] Preferably, the protocol feature database further includes: After collecting historical transmission data and protocol version data of IoT devices, protocol features of each protocol version are extracted based on the historical transmission data and protocol version data, and a protocol feature vector of each protocol version is constructed. Multiple protocol versions are clustered based on the protocol feature vectors to obtain multiple protocol clusters, a cluster center vector of each protocol cluster is determined, and a revised protocol feature vector of each protocol cluster is determined based on a preset reference feature vector. A protocol feature database including multiple protocol clusters and the revised protocol feature vector of each protocol cluster is constructed, wherein each protocol cluster includes the protocol feature vectors of multiple protocol versions. For the modified protocol feature vector: ; in, Indicates the modified protocol feature vector in The value of the feature dimension, The cluster center vector of the protocol cluster is The value of the feature dimension, Indicates the preset reference feature vector in The value of the feature dimension; The first IoT protocol feature vector is modified according to the modified protocol feature vector of each protocol cluster in the protocol feature database, including generating a second IoT protocol feature vector of each protocol cluster using the following formula: ; in, Indicates that the second IoT protocol feature vector is The value of the feature dimension, Indicates that the first IoT protocol feature vector is in The value of the feature dimension, The modified protocol feature vector of the protocol cluster to which the second IoT protocol feature vector belongs is in the The value of a feature dimension.

[0009] Preferably, performing transmission analysis on the IoT transmission characteristics according to the second protocol identification model includes: Inputting the IoT transmission feature of the communication data packet extracted from the transmission status monitoring data into a distance feature analysis unit, and generating a distance analysis result by the distance feature analysis unit, the distance analysis result including a first matching degree between the communication data packet and the plurality of protocol versions; Determining whether there is a protocol version with a first matching degree greater than a preset matching threshold, and if so, generating a second protocol identification result based on the first matching degrees of the communication data packet and the multiple protocol versions; Otherwise, the IoT transmission characteristics of the communication data packet extracted from the transmission status monitoring data are input into the discrete characteristic analysis unit to generate a discrete analysis result, the discrete analysis result includes the second matching degree between the communication data packet and multiple protocol versions, and the second protocol identification result is generated based on the second matching degree between the communication data packet and multiple protocol versions.

[0010] Preferably, the distance feature analysis unit and the discrete feature analysis unit further include: Extracting multiple transmission features of each protocol version based on historical transmission data of IoT devices and constructing multiple transmission feature vectors; calculating the standard deviation of each transmission feature parameter based on the multiple transmission feature vectors; calculating the fluctuation score of each protocol version by taking the arithmetic mean of the standard deviations of the multiple transmission feature parameters for each protocol version; and classifying the multiple protocol versions into a first protocol type and a second protocol type based on a preset fluctuation threshold; For multiple protocol versions in the first protocol type, calculating the mean of multiple transmission feature vectors of each protocol version to generate a transmission feature reference vector for each protocol version, and the distance feature analysis unit performing protocol matching on the IoT transmission feature of the communication data packet based on the transmission feature reference vector of each protocol version to generate a first matching degree between the communication data packet and the multiple protocol versions of the first protocol type; For multiple protocol versions of the second protocol type, a training data set is constructed based on multiple transmission feature vectors of each protocol version, and a protocol version label is added to each transmission feature vector in the training data set based on the protocol version corresponding to the transmission feature vector, and a transmission analysis model is trained based on the training data set. The training of the transmission analysis model takes multiple protocol version labels as training targets, and the discrete feature analysis unit analyzes the IoT transmission characteristics of the input communication data packet through the transmission analysis model to generate a second matching degree between the communication data packet and the protocol versions of the multiple second protocol types.

[0011] Preferably, determining a protocol parsing list according to the first protocol identification result and the second protocol identification result, and performing data parsing on the communication data packet based on multiple parsers in the protocol parsing list includes: Based on the first identification result and the second protocol identification result, multiple matching feature values ​​of each protocol version are determined, and the multiple matching feature values ​​of each protocol version are weighted and fused to generate a target matching degree for each protocol version. According to the target matching degree, the parsers corresponding to the multiple protocol versions are sorted and a protocol parsing list is generated. The communication data packets are parsed in descending order of the target matching degree through multiple parsers.

[0012] Preferably, the transmission analysis model is an improved convolutional neural network model, including adding a Softmax output layer in the convolutional neural network to output the probability distribution corresponding to each protocol version label.

[0013] A second aspect of the present invention provides an Internet of Things protocol automatic identification system based on a data model, which is used to implement any of the above-mentioned methods for automatically identifying an Internet of Things protocol based on a data model, including: A data receiving module is used to receive communication data packets sent by IoT devices; The protocol feature extraction module is used to parse the communication data packet after receiving it from the IoT device, and extract the protocol header features, data field features, and time pattern features of the communication data packet; a protocol feature analysis module, configured to perform feature splicing on the protocol header features, data field features, and time pattern features of the communication data packet to generate an IoT protocol feature, and perform protocol analysis on the IoT protocol feature according to a first protocol identification model to obtain a first protocol identification result, wherein the first protocol identification model performs protocol analysis on the IoT protocol feature based on a protocol feature database, which is constructed based on historical transmission data and protocol version data of IoT devices; The transmission feature extraction module is used to obtain the transmission status monitoring data of the IoT device about the communication data packet, and extract the IoT transmission features of the communication data packet from the transmission status monitoring data; a transmission feature analysis module, configured to perform transmission analysis on the IoT transmission feature according to a second protocol identification model to obtain a second protocol identification result, wherein the second protocol identification model performs transmission analysis on the IoT transmission feature based on a distance feature analysis unit and a discrete feature analysis unit; The data identification and parsing module is used to determine a protocol parsing list according to the first protocol identification result and the second protocol identification result, and perform data parsing on the communication data packet based on multiple parsers in the protocol parsing list.

[0014] The present invention has the following beneficial effects: The present invention analyzes data packets at the data level and accurately identifies the protocol version used by the device by conducting in-depth analysis of the multi-dimensional characteristics of the communication data packets, which has significant advantages in particular in scenarios with ambiguous versions or customized protocols. In addition, by combining the ability to perceive device status and network conditions, it further analyzes changes in protocol versions and integrates identification results from multiple angles to achieve more flexible and accurate protocol identification management. This mechanism is particularly suitable for addressing the challenges brought about by the complexity of IoT device protocols and dynamic switching requirements. It can be well applied to the field of automatic identification of IoT protocols, greatly improving the accuracy of automatic identification of IoT protocols. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flowchart of a method for automatically identifying Internet of Things protocols based on a data model is provided in an embodiment of the present invention.

[0016] Figure 2 This is a structural block diagram of an Internet of Things protocol automatic identification system based on a data model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0018] See also Figure 1 The first aspect of the present invention provides a method for automatically identifying an Internet of Things protocol based on a data model, comprising the following steps: Step S11: After receiving the communication data packet sent by the IoT device, the communication data packet is parsed to extract the protocol header features, data field features, and time pattern features of the communication data packet.

[0019] In this embodiment, communication data packets received from IoT devices are initially parsed to extract protocol header information, data entropy values, field feature information, time features, and transmission mode information. Protocol header information typically includes information such as the protocol version identifier, message type, and message length. Some protocols directly include version information, but in many cases, the version number is ambiguous and requires further analysis. Different protocol versions differ in the definition of command words and the use of flags. These differences can serve as an important basis for distinguishing protocol versions. For example, different versions of the MQTT protocol use different flags in CONNACK messages, thereby extracting protocol header features from communication data packets. Regarding data entropy values ​​and field feature information, different protocol versions may differ in message compression, encryption, and other processing. Analyzing the data fields of communication data packets and generating data field features can effectively capture these differences. Certain protocol versions have different communication frequencies in different scenarios. Extracting the transmission frequency, time interval, and delay pattern of data packets and generating time pattern features is also an important basis for distinguishing protocol versions.

[0020] Step S12: performing feature splicing on the protocol header features, data field features, and time pattern features of the communication data packet to generate IoT protocol features, performing protocol analysis on the IoT protocol features according to the first protocol recognition model, and obtaining a first protocol recognition result.

[0021] In this embodiment, the extracted protocol header features, data field features, and time pattern features will be spliced ​​into a complete feature and recorded as the Internet of Things protocol feature, which is used to represent the features of different protocol versions in the communication data packet at the data level. The Internet of Things protocol feature is analyzed using a pre-built first protocol identification model. The first protocol identification model infers the most likely protocol version by analyzing the Internet of Things protocol feature and comparing it with the data in the protocol feature database. The protocol feature database is constructed based on a large amount of historical transmission data and protocol version data, and can cover the protocol features of different Internet of Things devices in use. In this way, the protocol version used by the communication data packet can be automatically identified, the first protocol identification result can be generated, and a preliminary judgment of the protocol version can be achieved.

[0022] Step S13: Acquire the transmission status monitoring data of the IoT device about the communication data packet, extract the IoT transmission characteristics of the communication data packet from the transmission status monitoring data, perform transmission analysis on the IoT transmission characteristics according to the second protocol identification model, and obtain the second protocol identification result.

[0023] In this embodiment, after extracting the IoT protocol features of the communication data packet from the data layer, an analysis is also performed at the transmission status layer of the IoT device. When the IoT device operates under different network conditions and power consumption states, it may switch to different protocol versions. Simply relying on protocol fields or data packet content for identification may not be accurate enough. The device status and network environment provide important contextual information that can help accurately infer the device's protocol version. By obtaining and analyzing the transmission status monitoring data of the IoT device during the transmission of the communication data packet, the transmission status monitoring data may include information such as the device's current operating status, such as power consumption, CPU usage, storage status, and network conditions such as network latency, bandwidth, and packet loss rate. The IoT transmission features of the communication data packet are extracted from this data, which can be used to describe the communication environment and device operating status, and to a certain extent reflect the protocol version used by the device under different working conditions. The extracted IoT transmission features are analyzed using the second protocol identification model. The second protocol identification model uses a distance feature analysis unit and a discrete feature analysis unit to analyze real-time data from different angles based on historical data, predict the current device status and the protocol version that may be used under network conditions, and finally generate a second protocol identification result analyzed at the transmission status level to achieve further judgment of the protocol version.

[0024] Step S14: determining a protocol parsing list according to the first protocol identification result and the second protocol identification result, and performing data parsing on the communication data packet based on multiple parsers in the protocol parsing list.

[0025] In this embodiment, after obtaining the first and second protocol identification results, a comprehensive analysis is performed on these two results. The first and second protocol identification results respectively record the degree of match between the communication protocol corresponding to the communication data packet and multiple protocol versions after protocol version matching from different perspectives. The matching results at the data level and the transmission status level are integrated to generate a protocol parsing list. The protocol parsing list contains target matching degrees corresponding to multiple protocol versions. The higher the target matching degree, the closer the match between the communication protocol corresponding to the communication data packet and the protocol version. Based on the parsers corresponding to each protocol version, the parser corresponding to the most likely protocol version is first called to parse the communication data packet in the order listed in the protocol parsing list. If the first parser parses successfully and the result is reasonable, subsequent parsing is stopped and the protocol version is confirmed. If the parsing fails, the next parser is called until the most matching protocol version is successfully found. By completing the parsing of the data packet in this way, it can ensure that the communication protocol of the data packet can be correctly identified and processed in different scenarios, achieving efficient and automatic identification and parsing of IoT protocols.

[0026] It's worth noting that IoT devices use a wide variety of communication protocols, such as MQTT, CoAP, and HTTP. These protocols often evolve into multiple versions or variants as device manufacturers update and expand them. Some existing protocol identification mechanisms typically rely on manually configured protocol parsers or simple header fields for version identification. This is feasible when protocols are standardized and have a limited number of versions, but it struggles with the increasing diversity of protocol versions. In some scenarios, the protocol version used by a device may not be clear, or even the protocol version field may be omitted or not standardized, making it impossible to directly identify the version through the header field. In actual applications, IoT devices may also use multiple protocol versions due to different application scenarios, version updates, or device manufacturers. The data model-based IoT protocol automatic identification method provided by the present invention can automatically identify protocol versions through multi-dimensional feature extraction, rather than relying on a clear version field. This greatly enhances the automation and adaptability of protocol identification and addresses the parsing challenges associated with version diversity. Furthermore, the operating environment of IoT devices is often dynamic. For example, the device's power consumption may change (from high to low power) or the network environment may fluctuate (from high bandwidth to low bandwidth). In such cases, the device may need to dynamically switch to a different protocol version to adapt to the current environment. Some protocol identification methods usually rely on fixed parsing strategies to process received data packets, without fully considering the impact of differences in device status or network conditions. This results in the inability to successfully parse communication data in scenarios such as low power consumption or high latency.

[0027] The present invention analyzes data packets at the data level and accurately identifies the protocol version used by the device by conducting in-depth analysis of the multi-dimensional characteristics of the communication data packets, which has significant advantages in particular in scenarios with ambiguous versions or customized protocols. In addition, by combining the ability to perceive device status and network conditions, it further analyzes changes in protocol versions and integrates identification results from multiple angles to achieve more flexible and accurate protocol identification management. This mechanism is particularly suitable for addressing the challenges brought about by the complexity of IoT device protocols and dynamic switching requirements. It can be well applied to the field of automatic identification of IoT protocols, greatly improving the accuracy of automatic identification of IoT protocols.

[0028] As one implementation method, performing protocol analysis on the IoT protocol features according to the first protocol identification model in step S12 specifically includes: The first IoT protocol feature vector corresponding to the IoT protocol feature is extracted, where each element in the IoT protocol feature vector represents a feature related to one of the protocols in the IoT protocol feature, such as the packet sending frequency in the time pattern feature, the data entropy value in the data field feature, etc. The IoT protocol feature is quantified to generate the first IoT protocol feature vector, and the first IoT protocol feature vector is corrected according to the corrected protocol feature vector of each protocol cluster in the protocol feature database to obtain the second IoT protocol feature vector of each protocol cluster.

[0029] In this embodiment, in order to better adapt to the diversity and version evolution of the Internet of Things protocol, the first Internet of Things protocol feature vector is revised. Specifically, the first Internet of Things protocol feature vector is adjusted according to the revised protocol feature vector of each protocol cluster in the protocol feature database. The correction operation is implemented based on historical data and the feature change trend of the protocol cluster, which helps to capture the feature differences of the protocol version in different scenarios to enhance the accuracy of feature matching. After the correction is completed, each protocol cluster will obtain a second Internet of Things protocol feature vector, which is the revised Internet of Things protocol feature. When matching with multiple protocol versions in one of the protocol clusters based on the Internet of Things protocol feature, the second Internet of Things protocol feature vector is used to represent the Internet of Things protocol feature for matching.

[0030] Each protocol cluster contains multiple protocol versions. After determining the protocol feature vector of each protocol version and the second IoT protocol feature vector of the protocol cluster to which it belongs, a similarity analysis is performed on the protocol feature vector of each protocol version and the second IoT protocol feature vector of the protocol cluster to which it belongs. Specifically, the similarity of the two vectors is calculated to evaluate the matching degree between the characteristics of the current communication data packet and different protocol versions. The similarity analysis method can, for example, perform similarity calculation based on cosine similarity to obtain similarity values ​​between the current communication data packet and multiple protocol versions. In this way, the IoT protocol characteristics are matched with multiple protocol versions in each protocol cluster. Finally, the multiple protocol versions are sorted according to the similarity values ​​obtained by the similarity analysis, and multiple first target protocols are screened out from the multiple protocol versions to generate a first protocol identification result. The first target protocol is the most likely protocol version screened out based on the results of the similarity analysis. The first protocol identification result composed of multiple first target protocols can provide an important basis for subsequent data parsing and further processing.

[0031] As one implementation method, the above-mentioned protocol feature database is constructed in the following manner: After collecting the historical transmission data and protocol version data of the IoT devices, the protocol features of each protocol version are extracted based on the historical transmission data and protocol version data, and a protocol feature vector for each protocol version is constructed. The protocol feature vector specifically includes feature information such as the protocol header features, data field features, and time pattern features of the transmitted data.

[0032] Based on the protocol feature vector, multiple protocol versions are clustered to obtain multiple protocol clusters and the cluster center vector of each protocol cluster is determined. The cluster center vector of the protocol cluster represents the core features of the typical protocol version in the protocol cluster and can be used as the representative features of multiple protocol versions in the cluster for subsequent protocol matching and identification. Clustering of multiple protocol versions can adopt an algorithm based on distance, density or hierarchy for clustering. Clustering based on protocol feature vectors and determining the cluster center vector of each cluster are technical means well known to those skilled in the art and are not specifically limited in this embodiment. For example, after clustering multiple protocol versions based on the DBSCAN algorithm to obtain multiple protocol clusters, the modified protocol feature vector of each protocol cluster is determined based on the preset reference feature vector.

[0033] It's worth noting that differences may exist between different protocol versions. In particular, in some protocol versions, specific dimensions, such as header field length or temporal pattern features, vary significantly compared to other features. Simple protocol feature matching may fail to capture these differences, resulting in reduced matching accuracy. To achieve more accurate protocol matching, a modified protocol feature vector is determined for each protocol cluster. The purpose of the modified protocol feature vector is to first modify an unknown protocol feature based on the modified protocol feature vector of the protocol cluster when matching it with multiple protocol versions within the cluster. Matching analysis between different feature vectors is then performed to ensure that the unknown protocol feature better conforms to the characteristic patterns of the protocol cluster. The preset reference feature vector can be determined based on the initial protocol version or constructed by comprehensively analyzing the features of multiple protocol versions. Its primary function is to serve as a benchmark for analyzing the differences between different protocol versions. Ultimately, a protocol feature database is constructed, containing multiple protocol clusters and the modified protocol feature vectors for each protocol cluster. Each protocol cluster includes protocol feature vectors for multiple protocol versions.

[0034] Among them, for the generation of the modified protocol feature vector: ; in, Indicates the modified protocol feature vector in The value of the feature dimension, The cluster center vector of the protocol cluster is The value of the feature dimension, Indicates the preset reference feature vector in The value of a feature dimension.

[0035] The purpose of the above calculation is to determine the modified protocol feature vector by the ratio of the protocol cluster center vector and the preset reference feature vector. Each element in the modified protocol feature vector represents the change of one of the protocol features compared with the reference version. By modifying the protocol feature vector, the feature vector composed of the data to be analyzed can be enhanced or denoised to a certain extent, so that it is more consistent with the protocol version in the protocol cluster to be matched, and the feature differences between different protocol versions are better reflected.

[0036] After the protocol feature database is constructed, the first IoT protocol feature vector is modified according to the modified protocol feature vector of each protocol cluster in the protocol feature database, including generating a second IoT protocol feature vector for each protocol cluster using the following formula: ; in, Indicates that the second IoT protocol feature vector is The value of the feature dimension, Indicates that the first IoT protocol feature vector is in The value of the feature dimension, The modified protocol feature vector of the protocol cluster to which the second IoT protocol feature vector belongs is in the The value of a feature dimension.

[0037] In the IoT environment, different protocol versions may exhibit certain differences. For example, protocol header information may vary slightly between versions, and simply using uncorrected feature vectors may not accurately match these differences. By introducing cluster centroid vectors and a correction mechanism, we can more accurately capture the feature differences between protocol versions, significantly improving the accuracy of protocol version identification.

[0038] As one implementation method, the step S13 performs transmission analysis on the IoT transmission characteristics according to the second protocol identification model, including: The IoT transmission features of the communication data packet extracted from the transmission status monitoring data are input into the distance feature analysis unit, and the distance analysis results are generated by the distance feature analysis unit. The distance feature analysis unit generates the distance analysis results by calculating the distance between the data packet transmission features and the feature vectors of multiple protocol versions, which includes the first matching degree between the communication data packet and each protocol version, and is used to measure the similarity between the communication data packet and different protocol versions.

[0039] First, determine whether there is a protocol version with a first matching degree greater than a preset matching threshold. If there is a protocol version that meets the conditions, it is considered that these protocol versions have a high correlation with the current communication data packet. If so, a second protocol identification result is generated based on the first matching degree between the communication data packet and multiple protocol versions, and no further analysis is required.

[0040] If the distance feature analysis fails to find a highly matched protocol version, a more detailed analysis is performed. Specifically, the IoT transmission features extracted from the transmission status monitoring data are input into the discrete feature analysis unit, where another analysis mechanism is used to generate a discrete analysis result. The discrete analysis result includes a second degree of match between the communication packet and multiple protocol versions, that is, the matching of the packet transmission features with different protocol versions on the discrete features. The second protocol identification result is generated based on the second degree of match between the communication packet and multiple protocol versions.

[0041] For the above-mentioned distance feature analysis unit and discrete feature analysis unit, protocol matching is performed in the following manner: Different protocol versions may be highly adaptable to multiple device states and network conditions, even under extreme conditions. Other versions may only be suitable for use under relatively stable device states and network conditions. A conventional approach involves analyzing the transmission data of different protocol versions during use, identifying the device states and network conditions that are suitable for each protocol version, and constructing a feature vector that represents the appropriate device states and network conditions for each protocol version. This feature vector is then matched to different protocol versions based on a distance metric. Considering the differences in the applicable conditions of protocol versions, the above approach may not accurately capture patterns under these complex conditions.

[0042] In this case, multiple transmission features of each protocol version are extracted based on the historical transmission data of IoT devices and multiple transmission feature vectors are constructed. Each transmission feature vector represents the device status and network condition characteristics during one transmission process. The standard deviation of each transmission feature parameter is then calculated based on the multiple transmission feature vectors to measure the fluctuation of each transmission feature parameter under different conditions. The arithmetic average of the standard deviations of the multiple transmission feature parameters of each protocol version is summed to calculate the fluctuation score of each protocol version, which represents the degree of fluctuation of the multiple transmission feature vectors of a certain protocol version under different environments, and can reflect the stability of the device status and network conditions suitable for the protocol version.

[0043] Multiple protocol versions are categorized into a first protocol type and a second protocol type based on a preset fluctuation threshold. Protocol versions of the first protocol type have lower fluctuation scores, indicating that they are relatively stable under applicable device states and network conditions during use and are suitable for simple scenarios. In contrast, protocol versions of the second protocol type have higher fluctuation scores, indicating that they are applicable to a variety of device states and network conditions and are suitable for use in complex scenarios.

[0044] For multiple protocol versions in the first protocol type, the mean of multiple transmission feature vectors of each protocol version is calculated to generate a transmission feature reference vector for each protocol version. The transmission feature reference vector represents the working conditions applicable to the protocol version. The distance feature analysis unit performs protocol matching on the Internet of Things transmission features of the communication data packet based on the transmission feature reference vector of each protocol version, and generates a first matching degree between the communication data packet and the multiple protocol versions of the first protocol type based on the distance measurement algorithm.

[0045] For the multiple protocol versions of the second protocol type, due to their complex application scenarios, simple mean calculation and similarity analysis cannot be used to match them. Therefore, a machine learning-based transmission analysis model is used for analysis.

[0046] Specifically, a training data set is constructed based on multiple transmission feature vectors of each protocol version, and a protocol version label is added to each transmission feature vector in the training data set based on the protocol version corresponding to the transmission feature vector. The transmission analysis model is trained based on the training data set. The training of the transmission analysis model uses multiple protocol version labels as training targets, so that the trained transmission analysis model can predict the applicable protocol version in this case by analyzing the input IoT transmission characteristics, including analyzing the matching with multiple protocol versions. In actual application, the discrete feature analysis unit generates a second matching degree between the communication data packet and multiple second protocol type protocol versions through the model. This process better captures the characteristic changes of the protocol version under different transmission conditions through a more complex nonlinear analysis method.

[0047] In this embodiment, the transmission analysis model is an improved convolutional neural network model. The traditional convolutional neural network model includes an input layer, a convolution layer, a pooling layer and a fully connected layer. Its original output is an unnormalized class score, called logits. In this embodiment, a Softmax output layer is added to the traditional convolutional neural network model. The Softmax output layer is a Softmax activation function added to the last layer of CNN. The purpose is to convert the original output of the network, that is, the unnormalized class score, into a probability distribution, so that the output value of each class is between 0 and 1, and the sum of the probabilities of all categories is 1. Finally, the probability of each protocol version, that is, the matching degree, is output through the Softmax output layer.

[0048] This dual analysis approach avoids the need to perform complex machine learning model analysis on all protocol versions. For stable protocol versions of the first protocol type, simple feature mean calculation and distance matching are required, significantly reducing computational overhead. For protocol versions suitable for extreme conditions, the use of machine learning models ensures accurate and robust matching. This analysis method adapts to matching requirements for protocol versions of varying complexity, improving not only the accuracy and efficiency of protocol matching but also the system's adaptability and flexibility in IoT environments.

[0049] As one embodiment, in step S14, determining a protocol parsing list based on the first protocol identification result and the second protocol identification result, and performing data parsing on the communication data packet based on multiple parsers in the protocol parsing list specifically includes: Based on the first identification result and the second protocol identification result, multiple matching feature values ​​of each protocol version are determined, and the matching feature values ​​represent the degree of matching between the communication data packet and different protocol versions. Then, the multiple matching feature values ​​of each protocol version are weighted and fused to generate a comprehensive target matching degree for each protocol version, ensuring that the actual adaptation of the protocol version can be better reflected. Finally, the parsers corresponding to the multiple protocol versions can be sorted according to the target matching degree and a protocol parsing list can be generated. The communication data packet is parsed through multiple parsers in descending order of the target matching degree, so that those protocol versions that best match the data packet can be processed first, reducing unnecessary parsing attempts, shortening the time for the data packet to be correctly parsed, and facilitating the system to find the best matching protocol version in the shortest time.

[0050] See also Figure 2 The second aspect of the present invention provides an Internet of Things protocol automatic identification system based on a data model, including the following structure: A data receiving module is used to receive communication data packets sent by IoT devices; The protocol feature extraction module is used to parse the communication data packet after receiving it from the IoT device, and extract the protocol header features, data field features, and time pattern features of the communication data packet; a protocol feature analysis module, configured to perform feature splicing on the protocol header features, data field features, and time pattern features of the communication data packet to generate an IoT protocol feature, and perform protocol analysis on the IoT protocol feature according to a first protocol identification model to obtain a first protocol identification result, wherein the first protocol identification model performs protocol analysis on the IoT protocol feature based on a protocol feature database, which is constructed based on historical transmission data and protocol version data of IoT devices; The protocol analysis of the IoT protocol features is performed according to the first protocol identification model, including: Extracting a first IoT protocol feature vector corresponding to the IoT protocol feature, and correcting the first IoT protocol feature vector according to a corrected protocol feature vector of each protocol cluster in a protocol feature database to obtain a second IoT protocol feature vector for each protocol cluster; The IoT protocol features are matched with multiple protocol versions in a protocol feature database based on multiple second IoT protocol feature vectors, including determining the protocol feature vector of each protocol version and the second IoT protocol feature vector of the protocol cluster to which it belongs, performing similarity analysis on the protocol feature vector of each protocol version and the second IoT protocol feature vector of the protocol cluster to which it belongs, sorting multiple protocol versions according to similarity values ​​obtained from the similarity analysis, screening out multiple first target protocols from the multiple protocol versions and generating a first protocol identification result.

[0051] The transmission feature extraction module is used to obtain the transmission status monitoring data of the IoT device about the communication data packet, and extract the IoT transmission features of the communication data packet from the transmission status monitoring data; a transmission feature analysis module, configured to perform transmission analysis on the IoT transmission feature according to a second protocol identification model to obtain a second protocol identification result, wherein the second protocol identification model performs transmission analysis on the IoT transmission feature based on a distance feature analysis unit and a discrete feature analysis unit; The transmission analysis of the IoT transmission characteristics is performed according to the second protocol identification model, including: Inputting the IoT transmission feature of the communication data packet extracted from the transmission status monitoring data into a distance feature analysis unit, and generating a distance analysis result by the distance feature analysis unit, the distance analysis result including a first matching degree between the communication data packet and the plurality of protocol versions; Determining whether there is a protocol version with a first matching degree greater than a preset matching threshold, and if so, generating a second protocol identification result based on the first matching degrees of the communication data packet and the multiple protocol versions; Otherwise, the IoT transmission characteristics of the communication data packet extracted from the transmission status monitoring data are input into the discrete characteristic analysis unit to generate a discrete analysis result, the discrete analysis result includes the second matching degree between the communication data packet and multiple protocol versions, and the second protocol identification result is generated based on the second matching degree between the communication data packet and multiple protocol versions.

[0052] The data identification and parsing module is used to determine a protocol parsing list according to the first protocol identification result and the second protocol identification result, and perform data parsing on the communication data packet based on multiple parsers in the protocol parsing list.

[0053] The data model-based Internet of Things protocol automatic identification system provided in the embodiment of the present invention is used to implement the above-mentioned data model-based Internet of Things protocol automatic identification method, and the specific implementation process is not repeated here.

[0054] The foregoing description is merely a detailed description of the present invention, which is intended to enable those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art. Portions not described in detail in this specification are well known to those skilled in the art.

Claims

1. A method for automatic identification of Internet of Things protocols based on data model, characterized in that: include: After receiving the communication data packet sent by the IoT device, the communication data packet is parsed to extract the protocol header features, data field features and time pattern features of the communication data packet; Perform feature splicing on the protocol header features, data field features and time pattern features of the communication data packet to generate IoT protocol features, perform protocol analysis on the IoT protocol features according to the first protocol recognition model, and obtain a first protocol recognition result, wherein the first protocol recognition model performs protocol analysis on the IoT protocol features based on a protocol feature database, and the protocol feature database is constructed based on historical transmission data and protocol version data of IoT devices; Acquire the transmission status monitoring data of the IoT device about the communication data packet, extract the IoT transmission characteristics of the communication data packet from the transmission status monitoring data, perform transmission analysis on the IoT transmission characteristics according to the second protocol recognition model, and obtain the second protocol recognition result, wherein the second protocol recognition model performs transmission analysis on the IoT transmission characteristics based on the distance feature analysis unit and the discrete feature analysis unit; A protocol parsing list is determined according to the first protocol identification result and the second protocol identification result, and data parsing is performed on the communication data packet based on a plurality of parsers in the protocol parsing list.

2. According to the method of automatic identification of Internet of Things protocols based on data model in claim 1, it is characterized in that: The protocol analysis of the IoT protocol features is performed according to the first protocol identification model, including: Extracting a first IoT protocol feature vector corresponding to the IoT protocol feature, and correcting the first IoT protocol feature vector according to a corrected protocol feature vector of each protocol cluster in a protocol feature database to obtain a second IoT protocol feature vector of each protocol cluster; The IoT protocol features are matched with multiple protocol versions in a protocol feature database based on multiple second IoT protocol feature vectors, including determining the protocol feature vector of each protocol version and the second IoT protocol feature vector of the protocol cluster to which it belongs, performing similarity analysis on the protocol feature vector of each protocol version and the second IoT protocol feature vector of the protocol cluster to which it belongs, sorting multiple protocol versions according to similarity values ​​obtained from the similarity analysis, screening out multiple first target protocols from the multiple protocol versions and generating a first protocol identification result.

3. The method for automatic identification of Internet of Things protocols based on data model according to claim 2 is characterized in that: For the protocol signature database, also included: After collecting historical transmission data and protocol version data of IoT devices, extract the protocol features of each protocol version according to the historical transmission data and protocol version data and construct a protocol feature vector for each protocol version, cluster multiple protocol versions based on the protocol feature vector to obtain multiple protocol clusters, determine the cluster center vector of each protocol cluster, determine the modified protocol feature vector of each protocol cluster according to a preset reference feature vector, and construct a protocol feature database including multiple protocol clusters and the modified protocol feature vector of each protocol cluster, wherein each protocol cluster includes the protocol feature vectors of multiple protocol versions; For the modified protocol feature vector: ； in, Indicates the modified protocol feature vector in The value of the feature dimension, The cluster center vector of the protocol cluster is The value of the feature dimension, Indicates the preset reference feature vector in The value of the feature dimension; The first IoT protocol feature vector is modified according to the modified protocol feature vector of each protocol cluster in the protocol feature database, including generating a second IoT protocol feature vector of each protocol cluster using the following formula: ； in, Indicates that the second IoT protocol feature vector is The value of the feature dimension, Indicates that the first IoT protocol feature vector is The value of the feature dimension, The modified protocol feature vector of the protocol cluster to which the second IoT protocol feature vector belongs is The value of a feature dimension.

4. The method for automatic identification of Internet of Things protocols based on data model according to claim 3 is characterized in that: The transmission characteristics of IoT are analyzed according to the second protocol identification model, including: Inputting the IoT transmission feature of the communication data packet extracted from the transmission status monitoring data into the distance feature analysis unit, and generating a distance analysis result through the distance feature analysis unit, the distance analysis result including a first matching degree between the communication data packet and the multiple protocol versions; Determine whether there is a protocol version with a first matching degree greater than a preset matching threshold, and if so, generate a second protocol identification result based on the first matching degrees of the communication data packet and the plurality of protocol versions; Otherwise, the IoT transmission characteristics of the communication data packet extracted from the transmission status monitoring data are input into the discrete characteristic analysis unit to generate a discrete analysis result, the discrete analysis result includes a second matching degree between the communication data packet and multiple protocol versions, and a second protocol identification result is generated based on the second matching degree between the communication data packet and multiple protocol versions.

5. The method for automatic identification of Internet of Things protocols based on data model according to claim 4 is characterized in that: For the distance feature analysis unit and the discrete feature analysis unit, it also includes: Extract multiple transmission features of each protocol version according to the historical transmission data of the IoT device and construct multiple transmission feature vectors, calculate the standard deviation of each transmission feature parameter according to the multiple transmission feature vectors, perform arithmetic mean summation on the standard deviations of the multiple transmission feature parameters of each protocol version to calculate the fluctuation score of each protocol version, and divide the multiple protocol versions into a first protocol type and a second protocol type according to a preset fluctuation threshold; For multiple protocol versions in the first protocol type, the mean of multiple transmission feature vectors of each protocol version is calculated to generate a transmission feature reference vector of each protocol version, and the distance feature analysis unit performs protocol matching on the IoT transmission feature of the communication data packet based on the transmission feature reference vector of each protocol version to generate a first matching degree between the communication data packet and the multiple protocol versions of the first protocol type; For multiple protocol versions of the second protocol type, a training data set is constructed based on multiple transmission feature vectors of each protocol version, and a protocol version label is added to each transmission feature vector in the training data set based on the protocol version corresponding to the transmission feature vector, and a transmission analysis model is trained based on the training data set. The training of the transmission analysis model takes multiple protocol version labels as training targets, and the discrete feature analysis unit analyzes the IoT transmission characteristics of the input communication data packet through the transmission analysis model to generate a second matching degree between the communication data packet and the protocol versions of the multiple second protocol types.

6. The method for automatic identification of Internet of Things protocols based on data model according to claim 5 is characterized in that: Determining a protocol parsing list according to the first protocol identification result and the second protocol identification result, and performing data parsing on the communication data packet based on multiple parsers in the protocol parsing list, including: Based on the first identification result and the second protocol identification result, multiple matching feature values ​​for each protocol version are determined, and the multiple matching feature values ​​for each protocol version are weighted and fused to generate a target matching degree for each protocol version. According to the target matching degree, the parsers corresponding to the multiple protocol versions are sorted and a protocol parsing list is generated. According to the descending order of the target matching degree, data parsing is performed on the communication data packets through multiple parsers in sequence.

7. The method for automatic identification of Internet of Things protocols based on data model according to claim 5 is characterized in that: The transmission analysis model is an improved convolutional neural network model, which includes adding a Softmax output layer to the convolutional neural network to output the probability distribution corresponding to each protocol version label.

8. An Internet of Things protocol automatic identification system based on a data model, characterized in that: The system is used to implement the method for automatically identifying an Internet of Things protocol based on a data model as described in any one of claims 1 to 7, comprising: A data receiving module, used to receive communication data packets sent by IoT devices; The protocol feature extraction module is used to parse the communication data packet after receiving the communication data packet sent by the IoT device, and extract the protocol header feature, data field feature and time mode feature of the communication data packet; A protocol feature analysis module is used to perform feature splicing on the protocol header features, data field features and time pattern features of the communication data packet to generate IoT protocol features, and perform protocol analysis on the IoT protocol features according to a first protocol recognition model to obtain a first protocol recognition result, wherein the first protocol recognition model performs protocol analysis on the IoT protocol features based on a protocol feature database, and the protocol feature database is constructed based on historical transmission data and protocol version data of IoT devices; The transmission feature extraction module is used to obtain the transmission status monitoring data of the IoT device about the communication data packet, and extract the IoT transmission features of the communication data packet from the transmission status monitoring data; A transmission feature analysis module, used to perform transmission analysis on the IoT transmission feature according to a second protocol identification model to obtain a second protocol identification result, wherein the second protocol identification model performs transmission analysis on the IoT transmission feature based on a distance feature analysis unit and a discrete feature analysis unit; The data identification and analysis module is used to determine a protocol analysis list according to the first protocol identification result and the second protocol identification result, and perform data analysis on the communication data packet based on multiple parsers in the protocol analysis list.

Images