Dynamic protocol identification method and system based on multi-modal data

Abstract

The invention discloses a dynamic protocol identification method and system based on multi-modal data, and belongs to the technical field of data protocol identification, the identification method comprises the following steps: S1, calculating a data packet arrival interval time standard deviation, and judging whether a communication mode is a periodic or sudden communication mode; s2, dividing the data packet according to the length, calculating and counting the probability distribution of each interval, and calculating the Shannon entropy value of the data packet; s3, generating a protocol fingerprint according to the data packet arrival interval time standard deviation, the Shannon entropy value of the data packet and the probability distribution of each interval; and S4, classifying the data packets by adopting an edge AI classification engine. The protocol identification accuracy can be improved, and the network security protection capability is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data protocol identification, and in particular to a dynamic protocol identification method and system based on multimodal data. Background Art

[0002] Dynamic protocol identification is a technology that automatically and in real time determines the protocol type used by data transmission on a network. As network environments become increasingly complex and the variety of devices and communication protocols continues to expand, dynamic protocol identification is crucial. It breaks the limitations of traditional protocol identification, which relies on fixed rules or port numbers. It can flexibly handle a variety of known and unknown protocols, accurately analyzing network traffic.

[0003] In actual applications, dynamic protocol identification is widely used in the field of network security. It can monitor network protocols in real time, quickly identify potential malicious protocol behaviors, and promptly detect network attacks and abnormal traffic, providing strong support for network security protection. In the Internet of Things scenario, as the core technology for heterogeneous device access, it can be compatible with GB / T28181, GB35114, China Southern Power Grid PG, Hikvision Dahua and other manufacturers' private protocols to achieve interconnection between different devices. In enterprise network management, it helps administrators clearly understand the data transmission situation in the network, optimize network resource allocation, and improve network performance and efficiency.

[0004] However, current technologies for dynamic protocol identification have significant limitations. Many similar implementations only parse protocol header fields and lack dynamic analysis of communication behavior. This results in rigid identification capabilities, an inability to deeply explore the characteristics of protocols in actual interactions, and a difficulty adapting to complex and changing network environments. Regarding resource utilization, while cloud-based solutions offer powerful computing capabilities, they consume bandwidth exceeding 500Kbps per device. In edge computing scenarios, where device resources are limited and bandwidth requirements are stringent, this high resource consumption makes it difficult to meet demand and results in low resource efficiency. Furthermore, existing technologies lack scalability, and the adaptation of new protocols often relies on manual intervention. In the rapidly evolving IoT environment, with the constant emergence of new protocols, manual adaptation methods are unable to respond promptly, severely restricting the application and development of dynamic protocol identification technology. There is an urgent need to overcome these technical bottlenecks and integrate advanced technologies such as artificial intelligence and machine learning to promote smarter and more efficient dynamic protocol identification and further enhance its application value in various fields.

[0005] The information disclosed in this background technology section is only intended to enhance understanding of the overall background of the invention and should not be regarded as an admission or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the Invention

[0006] The purpose of the present invention is to provide a dynamic protocol identification method and system based on multimodal data, which can improve the accuracy of protocol identification, support multiple data sources and protocol types, be applicable to various network environments, identify and monitor network protocols in real time, and improve network security protection capabilities.

[0007] To achieve the above object, the present invention provides a dynamic protocol identification method based on multimodal data, comprising the following steps:

[0008] S1: Calculate the standard deviation of the inter-arrival time of data packets to determine whether the communication mode is periodic or bursty;

[0009] S2: Divide the data packets by length, calculate the probability distribution of each interval, and calculate the Shannon entropy value of the data packets;

[0010] S3: Generate a protocol fingerprint based on the standard deviation of the packet arrival interval, the Shannon entropy value of the packet, and the probability distribution of each interval;

[0011] S4: Use the edge AI classification engine to classify the data packet and determine whether it is a known protocol. The input layer of the edge AI classification engine receives a multi-dimensional feature vector, and the output layer maps it to a protocol label.

[0012] S5: If it is determined not to be a known protocol, a protocol fingerprint of the data packet is generated and the protocol library of the edge AI classification engine is updated;

[0013] S6: If it is determined to be a known protocol, the search strategy of the edge AI classification engine is optimized based on the current protocol type, channel load rate, and conflict rate.

[0014] In one embodiment of the present invention, in step S1, according to the data packet arrival time interval sequence T={t1, t2, ..., t n}, calculate the standard deviation σ of the packet arrival interval:

[0015]

[0016] In one embodiment of the present invention, in step S2, the length of the data packet is divided into k intervals, and the frequency f of each interval is counted. i , and normalized to a probability distribution Then, based on the probability distribution p i , calculate its Shannon entropy value H:

[0017]

[0018] In one embodiment of the present invention, in step S3, the standard deviation σ of the packet arrival interval, the Shannon entropy value H of the packet and the probability distribution p of each interval are calculated.i Concatenate into a multidimensional feature vector V:

[0019] V=[σ,H,p1,p2,...,p k ]

[0020] Then, a hash operation is performed on the multi-dimensional feature vector V to obtain the protocol fingerprint.

[0021] In one embodiment of the present invention, the multidimensional feature vector V is a 128-dimensional feature vector. When the multidimensional feature vector V is less than 128 dimensions, it is padded by padding, feature expansion or embedding mapping; when the multidimensional feature vector V is more than 128 dimensions, it is compressed by dimensionality reduction, feature selection or truncation.

[0022] In one embodiment of the present invention, in step S4, the convolution layer of the edge AI classification engine is a depthwise separable convolution.

[0023] In one embodiment of the present invention, in step S5, a protocol fingerprint is generated and the fingerprint library of the edge AI classification engine is updated based on the standard deviation of the packet arrival interval time, the Shannon entropy value of the packet, and the probability distribution of each interval; the fingerprint library stores fingerprint information in a structured manner, supporting conflict resolution, incremental updates, and fast matching.

[0024] In one embodiment of the present invention, in step S6, unified processing of heterogeneous protocols is first achieved through format adaptation and semantic mapping; then, the protocols are attached in a standardized format and metadata, and finally, the search strategy of the edge AI classification engine is optimized.

[0025] In one embodiment of the present invention, the process of optimizing the search strategy of the edge AI classification engine is as follows:

[0026] S601: Encode the current protocol type, channel load rate, and historical collision rate into a state matrix;

[0027] S602: Dynamically adjust the protocol priority using the reduction in the conflict rate as a reward value for machine learning;

[0028] S603: Update policy network parameters through Q-learning.

[0029] The present invention also provides a dynamic protocol identification system based on multimodal data, comprising:

[0030] The standard deviation acquisition module is used to calculate the standard deviation of the data packet arrival interval time and determine whether the communication mode is periodic or bursty;

[0031] The protocol complexity acquisition module is used to divide the data packet according to its length, calculate the probability distribution of each interval, and calculate the Shannon entropy value of the data packet;

[0032] The protocol fingerprint generation module is used to generate the protocol fingerprint based on the standard deviation of the packet arrival interval, the Shannon entropy value of the packet and the probability distribution of each interval;

[0033] A protocol classification module, which uses an edge AI classification engine to classify packets and determine whether they belong to a known protocol. The input layer of the edge AI classification engine receives a multidimensional feature vector V, and the output layer maps it to a protocol label.

[0034] The unknown protocol update module is used to generate the protocol fingerprint of the data packet when it is determined that the protocol is not an unknown protocol, and then update the protocol library of the edge AI classification engine;

[0035] The known protocol update module is used to optimize the search strategy of the edge AI classification engine based on the current protocol type, channel load rate and conflict rate when it is determined to be a known protocol.

[0036] Compared with the prior art, a dynamic protocol identification method and system based on multimodal data according to the present invention has the following advantages: 1. Real-time analysis of data packet timing characteristics (communication cycle, burst traffic ratio) and statistical characteristics (length distribution, information entropy) through edge computing to generate a dynamic fingerprint library; 2. Deployment of a small model (memory occupancy <50MB) to implement protocol type inference based on behavioral feature vectors; 3. Self-learning of protocol compatibility, using a reinforcement learning model to dynamically optimize the protocol conversion strategy, and reducing the conflict rate in multiple protocol mixed scenarios; 4. Feature extraction and classification are completed locally, and only metadata is uploaded (bandwidth occupancy <20Kbps / device). BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of a dynamic protocol identification method based on multimodal data according to an embodiment of the present invention;

[0038] Figure 2 4 is a schematic diagram of a dynamic protocol identification system based on multimodal data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific embodiments.

[0040] Unless expressly stated otherwise, throughout the specification and claims, the term "comprise" or variations such as "include" or "comprising", etc., will be understood to include the stated elements or components but not to exclude other elements or other components.

[0041] like Figure 1As shown, a dynamic protocol identification method based on multimodal data according to a preferred embodiment of the present invention includes the following steps:

[0042] S1: Calculate the standard deviation of the data packet arrival interval to determine whether the communication mode is periodic or bursty.

[0043] The standard deviation σ of the packet arrival interval can reflect the degree of fluctuation in the communication timing. If the interval time is highly regular (such as a fixed period), that is, the standard deviation approaches 0, it is a periodic communication mode. If the interval time is highly random and the standard deviation increases significantly, it is a bursty communication mode.

[0044] Assume that the time interval between arrival of data packets is T = {t1, t2, ..., t n}, the standard deviation is calculated as:

[0045]

[0046] By setting a threshold, the communication mode can be quantitatively distinguished. In the present invention, the threshold is set to 5ms, that is, if σ<5ms, it is judged as a periodic communication mode, and if σ≥5ms, it is judged as a bursty communication mode.

[0047] Although the binary classification of communication patterns into periodicity and burstiness in step S1 is not explicitly used in subsequent steps, the value of the standard deviation σ indirectly influences feature engineering, protocol fingerprint generation, classification model inference, and dynamic priority adjustment. Furthermore, this judgment process provides a foundation for dynamic analysis of protocol behavior, enhancing the system's adaptability to complex network environments. This is achieved through multidimensional feature fusion and implicit strategies.

[0048] S2: Divide the data packet by length, calculate the probability distribution of each interval, and calculate the Shannon entropy value of the data packet.

[0049] Specifically, the packet length is divided into k intervals (such as 0-64B, 65-128B, etc.), and the frequency f of each interval is counted. i , and normalized to a probability distribution Then, based on the probability distribution p i , calculate its Shannon entropy H to characterize the complexity of the protocol:

[0050]

[0051] For fixed-length heartbeat messages, the Shannon entropy is close to 0. For variable-length encryption protocols (such as video streaming), the Shannon entropy increases significantly and is greater than 3 bits.

[0052] The number of bins k should be determined based on a comprehensive consideration of protocol characteristics, data distribution, and computational efficiency. In practical applications, it is recommended to optimize the value of k through experiments or dynamic binning to balance feature expression capabilities and system performance.

[0053] S3: Generate a protocol fingerprint based on the standard deviation of the packet arrival interval, the Shannon entropy value of the packet, and the probability distribution of each interval.

[0054] Specifically, the standard deviation of the packet arrival interval σ, the Shannon entropy value H of the packet and the probability distribution p of each interval are i They are all normalized to the interval [0,1] and then concatenated into a multidimensional feature vector V:

[0055] V=[σ,H,p1,p2,...,p k ]

[0056] The multidimensional feature vector V is a 128-dimensional feature vector, which is typically a standardized requirement during the design phase. However, implementation requires flexible handling based on the original feature dimensionality: insufficient dimensionality is addressed through padding, feature expansion, or embedding mapping; excessive dimensionality is addressed through dimensionality reduction, feature selection, or truncation compression. The key goal is to ensure compatibility of the feature vector with the model input layer while preserving the critical information for protocol identification to the greatest extent possible.

[0057] Then, a hash operation is performed on the multi-dimensional feature vector V to obtain a unique protocol fingerprint.

[0058] Normalization is performed on all features (σ, H, P i ), but the specific method varies depending on the characteristics. The normalization of Shannon entropy H needs to be based on its theoretical maximum value (H max =log2k), ensuring that the normalized value does not exceed 1.

[0059] The standard deviation of the packet arrival interval σ, the Shannon entropy value H of the packet and the probability distribution p of each interval are used. i , and the fused protocol fingerprint, in the protocol conflict scenario, an innovative protocol similarity weight w is introduced:

[0060] w=α·σ+β·H+γ·KL(p i ||q i )

[0061] Among them, KL is the KL divergence of the length distribution, α, β, γ are dynamic adjustment coefficients, q i It is the length distribution of known protocols in the protocol library, which is used to quantify the similarity between the current data packet and the target protocol.

[0062] The dynamic adjustment coefficients α, β, and γ are initialized through experiments and optimized in real time by reinforcement learning to adapt to changes in the network environment, ensuring that the similarity weight w can dynamically reflect the importance of different features and improve the recognition robustness in protocol conflict scenarios.

[0063] The protocol similarity weight w quantifies the similarity of protocol features and implicitly plays a role in conflict resolution, reinforcement learning reward design, protocol library management, and other aspects. This mechanism enhances the system's adaptability in complex network environments and is one of the core innovations of the dynamic protocol identification method.

[0064] S4: Use the edge AI classification engine to classify the data packet to determine whether it is a known protocol. The input layer of the edge AI classification engine receives the multidimensional feature vector V, and the output layer is mapped to the protocol label.

[0065] In this step, the edge AI classification engine is based on a lightweight convolutional neural network (CNN), but with the following improvements: using depthwise separable convolution to replace the standard 3×3 convolution layer of the traditional CNN network, reducing the number of parameters (by 72%); using dynamic quantization-aware training to compress the model weights to 8-bit integers (memory usage <50MB); and using an adaptive pooling layer that can dynamically adjust the pooling kernel size according to the input feature dimension.

[0066] The edge AI classification engine runs locally on the gateway, supporting processing of >1000 packets per second. This engine achieves hardware acceleration by integrating NPU instruction set optimizations (such as ARM CMSIS-NN). It decouples feature extraction from classification tasks, enabling multi-threaded processing and pipelined parallel processing. It also achieves model pruning by removing redundant neurons (sparsity >60%) while retaining key feature paths. A measured processing speed of 1200 packets / s on a Raspberry Pi 4B (Cortex-A72) significantly improves the protocol recognition efficiency and responsiveness of edge devices.

[0067] S5: If it is determined not to be a known protocol, a protocol fingerprint of the data packet is generated and the protocol library of the edge AI classification engine is updated.

[0068] Specifically, similar to step S3, based on the standard deviation of packet inter-arrival times, the Shannon entropy of the packets, and the probability distribution of each interval, a protocol fingerprint is generated and the fingerprint library of the edge AI classification engine is updated. The fingerprint library stores fingerprint information in a structured manner, supporting conflict resolution, incremental updates, and fast matching.

[0069] S6: If it is determined to be a known protocol, the search strategy of the edge AI classification engine is optimized based on the current protocol type, channel load rate, and conflict rate.

[0070] Specifically, unified processing of heterogeneous protocols is first achieved through format adaptation and semantic mapping, relying on a lightweight engine to ensure real-time performance; then, the protocols are attached in a standardized format and metadata to ensure downstream system compatibility, and compression and QoS are used to improve transmission efficiency; finally, the search strategy of the edge AI classification engine is optimized, and continuous optimization of protocol priority and resource allocation is achieved through reinforcement learning and real-time monitoring.

[0071] Specifically, the process of optimizing the search strategy of the edge AI classification engine is as follows:

[0072] S601: Encode the current protocol type, channel load rate, and historical collision rate into a state matrix. The current protocol type is a protocol type vector, such as the softened version of the protocol type vector encoded using one-hot encoding, which is [0.9, 0.1, 0, ..., 0.3]. The channel load rate is the real-time channel occupancy rate, which reflects the degree to which the current channel is occupied by data transmission, such as 70%. The historical collision rate is the average collision rate within a sliding window. Sliding window is a data processing concept that defines a fixed-size interval on time series data. Over time, the average collision rate represents the proportion of data transmission conflicts within this sliding window, such as 5%.

[0073] An example matrix is:

[0074]

[0075] S602: Dynamically adjust the protocol priority using the reduction in the conflict rate as a reward value for machine learning.

[0076] Specifically, after each round of strategy execution, the change in conflict rate is calculated as the reward value R for machine learning:

[0077]

[0078] Among them, ΔConflictRate represents the change in conflict rate, that is, the difference between the previous conflict rate and the current conflict rate; PreviousRate is the conflict rate before the last round of strategy execution. If the conflict rate drops from 15% to 5%, then

[0079] This positive reward value indicates that the conflict rate has decreased after the policy is executed, which is a good result; if the conflict rate increases, the reward value will be negative, indicating that the current policy has led to worse network conditions.

[0080] S603: Update the policy network parameters θ through Q-learning:

[0081] θ t+1 =θ t +η·(R+γ′maxQ(st+1 ,a)-Q(s t ,a))

[0082] Among them, θ t is the policy network parameter at the current moment (round t); η is the learning rate, which controls the magnitude of parameter changes at each update; R is the reward value; γ' is the discount factor, which serves the long-term reward balance function module; Q(st,a) in Q-learning represents the state s t The core function of the algorithm is to guide the intelligent agent (such as the protocol recognition system) to select the optimal action under a specific state.

[0083] Q-learning is a model-free reinforcement learning algorithm used to learn the optimal strategy in a given environment. When inferring protocols, it prioritizes protocol conversion paths with high reward values ​​(e.g., LoRa over ZigBee).

[0084] like Figure 2 As shown, a dynamic protocol identification method based on multimodal data according to a preferred embodiment of the present invention includes:

[0085] Standard deviation acquisition module 1, used to calculate the standard deviation of the data packet arrival interval time and determine whether the communication mode is periodic or burst communication mode;

[0086] The protocol complexity acquisition module 2 is used to divide the data packet according to its length, calculate the probability distribution of each interval, and calculate the Shannon entropy value of the data packet;

[0087] The protocol fingerprint generation module 3 is used to generate a protocol fingerprint based on the standard deviation of the inter-arrival time of the data packets, the Shannon entropy value of the data packets and the probability distribution of each interval;

[0088] The protocol classification module 4 is used to classify the data packets using the edge AI classification engine to determine whether they belong to a known protocol. The input layer of the edge AI classification engine receives the multidimensional feature vector V, and the output layer maps it to the protocol label.

[0089] The unknown protocol update module 5 is used to generate a protocol fingerprint of the data packet and update the protocol library of the edge AI classification engine when it is determined that the protocol is not an established protocol;

[0090] The known protocol update module 6 is used to optimize the search strategy of the edge AI classification engine according to the current protocol type, channel load rate and conflict rate when it is determined to be a known protocol.

[0091] The foregoing descriptions of specific exemplary embodiments of the present invention are for purposes of illustration and description. These descriptions are not intended to limit the invention to the precise forms disclosed, and it is apparent that many variations and modifications are possible in light of the foregoing teachings. The exemplary embodiments have been selected and described for the purpose of explaining the specific principles of the invention and their practical application, thereby enabling those skilled in the art to realize and utilize a variety of exemplary embodiments of the invention and various options and modifications. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A dynamic protocol identification method based on multimodal data, characterized in that: The following steps are involved: S1: Calculate the standard deviation of the inter-arrival time of data packets to determine whether the communication mode is periodic or bursty; S2: Divide the data packets by length, calculate the probability distribution of each interval, and calculate the Shannon entropy value of the data packets; S3: Generate a protocol fingerprint based on the standard deviation of the packet arrival interval, the Shannon entropy value of the packet, and the probability distribution of each interval; S4: Use the edge AI classification engine to classify the data packet and determine whether it is a known protocol. The input layer of the edge AI classification engine receives a multi-dimensional feature vector, and the output layer maps it to a protocol label. S5: If it is determined not to be a known protocol, a protocol fingerprint of the data packet is generated and the protocol library of the edge AI classification engine is updated; S6: If it is determined to be a known protocol, the search strategy of the edge AI classification engine is optimized based on the current protocol type, channel load rate, and conflict rate.

2. The dynamic protocol identification method based on multimodal data according to claim 1, characterized in that: In step S1, according to the data packet arrival time interval sequence T={t1,t2,...,t n }, calculate the standard deviation σ of the packet arrival interval:

3. The dynamic protocol identification method based on multimodal data according to claim 1, characterized in that: In step S2, the packet length is divided into k intervals, and the frequency f of each interval is counted. i , and normalized to a probability distribution Then, based on the probability distribution p i , calculate its Shannon entropy value H:

4. The dynamic protocol identification method based on multimodal data according to claim 1, characterized in that: In step S3, the standard deviation of the packet arrival interval σ, the Shannon entropy value H of the packet and the probability distribution p of each interval are calculated. i Concatenate into a multidimensional feature vector V: V=[σ,H,p1,p2,...,p k ] Then, a hash operation is performed on the multi-dimensional feature vector V to obtain the protocol fingerprint.

5. The dynamic protocol identification method based on multimodal data according to claim 4, characterized in that: The multidimensional feature vector V is a 128-dimensional feature vector. When the multidimensional feature vector V is less than 128 dimensions, it is filled by padding, feature expansion or embedding mapping; when the multidimensional feature vector V is more than 128 dimensions, it is compressed by dimensionality reduction, feature selection or truncation.

6. The dynamic protocol identification method based on multimodal data according to claim 1, characterized in that: In step S4, the convolution layer of the edge AI classification engine is a depth-wise separable convolution.

7. The dynamic protocol identification method based on multimodal data according to claim 1, characterized in that: In step S5, a protocol fingerprint is generated and the fingerprint library of the edge AI classification engine is updated based on the standard deviation of the packet arrival interval, the Shannon entropy value of the packet, and the probability distribution of each interval; the fingerprint library stores fingerprint information in a structured manner, supporting conflict resolution, incremental updates, and fast matching.

8. The dynamic protocol identification method based on multimodal data according to claim 1, characterized in that: In step S6, unified processing of heterogeneous protocols is first achieved through format adaptation and semantic mapping; then, the protocols are attached in a standardized format and metadata, and finally, the search strategy of the edge AI classification engine is optimized.

9. The dynamic protocol identification method based on multimodal data according to claim 8, characterized in that: The process of optimizing the search strategy of the edge AI classification engine is as follows: S601: Encode the current protocol type, channel load rate, and historical collision rate into a state matrix; S602: Dynamically adjust the protocol priority using the reduction in the conflict rate as a reward value for machine learning; S603: Update policy network parameters through Q-learning.

10. A dynamic protocol identification system based on multimodal data, characterized in that: include: The standard deviation acquisition module is used to calculate the standard deviation of the data packet arrival interval time and determine whether the communication mode is periodic or bursty; The protocol complexity acquisition module is used to divide the data packet according to its length, calculate the probability distribution of each interval, and calculate the Shannon entropy value of the data packet; The protocol fingerprint generation module is used to generate the protocol fingerprint based on the standard deviation of the packet arrival interval, the Shannon entropy value of the packet and the probability distribution of each interval; A protocol classification module, which uses an edge AI classification engine to classify packets and determine whether they belong to a known protocol. The input layer of the edge AI classification engine receives a multidimensional feature vector V, and the output layer maps it to a protocol label. The unknown protocol update module is used to generate the protocol fingerprint of the data packet and update the protocol library of the edge AI classification engine when it is determined that the protocol is not known. The known protocol update module is used to optimize the search strategy of the edge AI classification engine based on the current protocol type, channel load rate and conflict rate when it is determined to be a known protocol.

Images