A dynamic protocol identification method and system based on multi-modal data

Abstract

The application discloses a kind of dynamic protocol identification method and system based on multi-modal data, belong to the technical field of data protocol identification, identification method includes: S1: calculate data packet arrival interval time standard deviation, judge communication mode is periodic or burst communication mode;S2: data packet is divided according to length, calculate and count the probability distribution of each interval, and calculate the Shannon entropy value of data packet;S3: according to data packet arrival interval time standard deviation, the Shannon entropy value of data packet and the probability distribution of each interval, generate protocol fingerprint;S4: using edge AI classification engine classifies data packet.The application can improve the accuracy of protocol identification, improve network security protection capability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data protocol identification, and in particular to a dynamic protocol identification method and system based on multi-modal data. BACKGROUND

[0002] Dynamic protocol identification is a technology that can automatically determine the type of protocol followed by data transmission in real time. In the context of increasingly complex network environments, with the increasing number of device types and communication protocols, dynamic protocol identification technology is crucial. It breaks the limitations of traditional reliance on fixed rules or port numbers to identify protocols, and can flexibly cope with various known and unknown protocols, accurately parsing network traffic.

[0003] In practical applications, dynamic protocol identification is widely used in the field of network security, which can monitor network protocols in real time, quickly identify potential malicious protocol behaviors, and timely detect network attacks and abnormal traffic, providing strong support for network security protection; in the Internet of Things scenario, as a core technology for accessing heterogeneous devices, it can be compatible with various manufacturers' private protocols such as GB / T28181, GB35114, Nanwang PG, and Hikvision Dahua, realizing the interconnection between different devices; in enterprise network management, it helps administrators clearly understand the data transmission situation in the network, optimizes network resource allocation, and improves network performance and efficiency.

[0004] However, the existing technology of dynamic protocol identification has obvious limitations. In terms of identification capability, many similar implementation schemes only analyze the protocol header field, lack dynamic analysis of communication behavior, resulting in rigid identification capability, and unable to deeply mine the characteristics of protocols in actual interaction, making it difficult to cope with complex and variable network environments. In terms of resource utilization, cloud-based solutions have strong computing power, but occupy more than 500Kbps per device in bandwidth, which is difficult to meet the requirements in edge computing scenarios where device resources are limited and bandwidth requirements are strict. Such high resource consumption has the problem of low resource efficiency. In addition, existing technologies also have poor scalability, and the adaptation of new protocols often relies on manual intervention. In the face of rapidly iterating Internet of Things environment, new protocols are emerging, and manual adaptation cannot respond in time, which seriously restricts the application and development of dynamic protocol identification technology. In the future, it is urgent to break through these technical bottlenecks, combine advanced technologies such as artificial intelligence and machine learning, and promote the development of dynamic protocol identification to be more intelligent and efficient, further improving its application value in various fields.

[0005] The information disclosed in this BACKGROUND section is only intended to increase an understanding of the general background of the application and should not be taken as an acknowledgement or any form of suggestion that this information forms prior art that is already known to those of ordinary skill in the art. SUMMARY

[0006] The application aims to provide a dynamic protocol identification method and system based on multi-modal data, which can improve the accuracy of protocol identification, support multiple data sources and protocol types, be suitable for various network environments, identify and monitor network protocols in real time, and improve network security protection capability.

[0007] To achieve the above-mentioned purpose, the application provides a dynamic protocol identification method based on multi-modal data, comprising the following steps:

[0008] S1: calculating the standard deviation of data packet arrival interval time to determine whether the communication mode is periodic or bursty communication mode;

[0009] S2: dividing data packets according to length, calculating the probability distribution of each interval, and calculating the Shannon entropy value of data packets;

[0010] S3: generating protocol fingerprints according to the standard deviation of data packet arrival interval time, the Shannon entropy value of data packets, and the probability distribution of each interval;

[0011] S4: classifying data packets by using an edge AI classification engine to determine whether it is a known protocol, wherein the input layer of the edge AI classification engine receives a multi-dimensional feature vector, and the output layer is mapped to a protocol label;

[0012] S5: if it is determined that it is not a known protocol, generating the protocol fingerprints of data packets, and updating the protocol library of the edge AI classification engine;

[0013] S6: if it is determined that it is a known protocol, optimizing the search strategy of the edge AI classification engine according to the current protocol type, channel load rate, and collision rate.

[0014] In an embodiment of the application, in step S1, the standard deviation σ of data packet arrival interval time is calculated according to the data packet arrival time interval sequence T={t1, t2,...,t n

[0015]

[0016] In an embodiment of the application, in step S2, the length of data packets is divided into k intervals, the frequency f i of each interval is counted, and normalized to a probability distribution Then, based on the probability distribution p i , the Shannon entropy value H thereof is calculated:

[0017]

[0018] In an embodiment of the application, in step S3, the standard deviation σ of data packet arrival interval time, the Shannon entropy value H of data packets, and the probability distribution p​i The splicing is a multi-dimensional feature vector V:

[0019] V=[σ,H,p1,p2,...,p k ]

[0020] Then, the multi-dimensional feature vector V is subjected to a hash operation to obtain a protocol fingerprint.

[0021] In an embodiment of the present application, the multi-dimensional feature vector V is a 128-dimensional feature vector, when the multi-dimensional feature vector V is less than 128-dimensional, it is filled, feature expansion or embedding mapping to make up; when the multi-dimensional feature vector V is more than 128-dimensional, it is compressed by dimension reduction, feature selection or truncation.

[0022] In an embodiment of the present application, in step S4, the convolution layer of the edge AI classification engine is a depth separable convolution.

[0023] In an embodiment of the present application, in step S5, according to the standard deviation of the data packet arrival interval time, the Shannon entropy value of the data packet and the probability distribution of each interval, the protocol fingerprint is generated and the fingerprint library of the edge AI classification engine is updated; the fingerprint library is structured to store fingerprint information, supports conflict resolution, incremental update and fast matching.

[0024] In an embodiment of the present application, in step S6, first, the heterogeneous protocol is uniformly processed through format adaptation and semantic mapping; then, the protocol is attached with metadata in a standard format, and finally, the search strategy of the edge AI classification engine is optimized.

[0025] In an embodiment of the present application, the process of optimizing the search strategy of the edge AI classification engine is as follows:

[0026] S601: encode the current protocol type, channel load rate and historical conflict rate into a state matrix;

[0027] S602: take the conflict rate reduction amplitude as the reward value of machine learning, and dynamically adjust the protocol priority;

[0028] S603: update the strategy network parameters through Q-learning.

[0029] The present application also provides a dynamic protocol identification system based on multi-modal data, comprising:

[0030] A standard deviation acquisition module is configured to calculate the standard deviation of the data packet arrival interval time, and determine whether the communication mode is a periodic or burst communication mode.

[0031] A protocol complexity acquisition module is configured to divide the data packet according to the length, calculate the probability distribution of each interval, and calculate the Shannon entropy value of the data packet.

[0032] a protocol fingerprint generation module configured to generate a protocol fingerprint according to a standard deviation of inter-arrival time of data packets, a Shannon entropy value of the data packets, and a probability distribution of each interval;

[0033] a protocol classification module configured to classify the data packets by using an edge AI classification engine, and determine whether the data packets are known protocols, wherein an input layer of the edge AI classification engine receives a multi-dimensional feature vector V, and an output layer of the edge AI classification engine is mapped to a protocol label;

[0034] an unknown protocol updating module configured to, when it is determined that the data packets are not known protocols, generate a protocol fingerprint of the data packets, and then update a protocol library of the edge AI classification engine;

[0035] a known protocol updating module configured to, when it is determined that the data packets are known protocols, optimize a search strategy of the edge AI classification engine according to a current protocol type, a channel load rate, and a conflict rate.

[0036] Compared with the prior art, the dynamic protocol identification method and system based on multi-modal data according to the present application have the following advantages: 1. By using edge computing to analyze the time sequence characteristics (communication period, burst traffic proportion) and statistical characteristics (length distribution, information entropy) of the data packets in real time, a dynamic fingerprint library is generated; 2. A small model (memory occupation < 50 MB) is deployed, and a protocol type inference is realized based on a behavior feature vector; 3. Protocol compatibility is self-learned, and a reinforcement learning model is used to dynamically optimize a protocol conversion strategy, thereby reducing the conflict rate in a mixed protocol scene; and 4. Feature extraction and classification are completed locally, and only metadata (bandwidth occupation < 20 Kbps / device) is uploaded. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a flowchart of a dynamic protocol identification method based on multi-modal data according to an embodiment of the present application;

[0038] Figure 2 is a schematic diagram of a dynamic protocol identification system based on multi-modal data according to an embodiment of the present application. DETAILED DESCRIPTION

[0039] The specific embodiments of the present application will be described in detail below with reference to the accompanying drawings, but it should be understood that the scope of protection of the present application is not limited by the specific embodiments.

[0040] Unless otherwise explicitly stated, throughout the specification and claims, the term "comprise" or variations such as "comprises" or "comprising" will be understood to imply the inclusion of a stated element or component but not the exclusion of any other element or component.

[0041] As Figure 1As shown, a dynamic protocol identification method based on multi-modal data according to a preferred embodiment of the present application comprises the following steps:

[0042] S1: Calculate the standard deviation of data packet arrival interval time, and determine whether the communication mode is periodic or bursty.

[0043] The standard deviation σ of data packet arrival interval time can reflect the fluctuation degree of communication timing. If the interval time is highly regular (such as fixed period), i.e. the standard deviation tends to 0, it is a periodic communication mode, and if the interval time has high randomness, the standard deviation significantly increases, it is a bursty communication mode.

[0044] Suppose the data packet arrival time interval sequence is T = {t1, t2,..., t n}, the standard deviation is calculated as:

[0045]

[0046] By setting a threshold, the communication mode can be quantitatively distinguished. In the present application, the threshold is set to 5ms, i.e. if σ < 5ms, it is determined as a periodic communication mode, and if σ ≥ 5ms, it is determined as a bursty communication mode.

[0047] The binary classification of periodicity / burstiness of the communication mode in step S1 is not explicitly used in the subsequent steps, but the value of the standard deviation σ indirectly affects feature engineering, protocol fingerprint generation, classification model inference, and dynamic priority adjustment. In addition, this judgment process provides a basis for dynamic analysis of protocol behavior, enhancing the adaptability of the system to complex network environments, and its role is achieved through multi-dimensional feature fusion and implicit strategies.

[0048] S2: Divide the data packets according to their lengths, calculate the probability distribution of each interval, and calculate the Shannon entropy value of the data packets.

[0049] Specifically, the data packet length is divided into k intervals (such as 0-64B, 65-128B, etc.), the frequency f i of each interval is counted, and normalized to a probability distribution Then, based on the probability distribution p i , the Shannon entropy value H is calculated to represent the protocol complexity:

[0050]

[0051] For fixed-length heartbeat messages, the Shannon entropy value is close to 0, and for variable-length encryption protocols (such as video streams), the Shannon entropy value significantly increases and is greater than 3 bits.

[0052] The determination of the number of intervals k needs to consider the protocol characteristics, data distribution, and computational efficiency. In practical applications, it is recommended to optimize the value of k through experiments or dynamic binning methods to balance feature representation capability and system performance.

[0053] S3: Generate a protocol fingerprint based on the standard deviation of the data packet arrival interval, the Shannon entropy of the data packets, and the probability distribution of each interval.

[0054] Specifically, the standard deviation of the data packet arrival interval σ, the Shannon entropy value H of the data packet, and the probability distribution p of each interval are used. i All values ​​are normalized to the [0,1] interval, and then concatenated to form a multidimensional feature vector V:

[0055] V=[σ,H,p1,p2,...,p k ]

[0056] The multidimensional feature vector V is a 128-dimensional feature vector, which is usually a standardized requirement during the design phase. In actual implementation, it needs to be flexibly handled according to the original feature dimensions: if the dimensions are insufficient, they are filled by padding, feature expansion, or embedding mapping; if the dimensions are too many, they are reduced by dimensionality reduction, feature selection, or truncation and compression. The core goal is to ensure the compatibility of the feature vector with the model input layer, while preserving the key information for protocol recognition to the greatest extent possible.

[0057] Then, a hash operation is performed on the multidimensional feature vector V to obtain a unique protocol fingerprint.

[0058] Normalization applies to all features (σ, H, P) i However, the specific method varies depending on the characteristics. Normalization of the Shannon entropy H needs to be based on its theoretical maximum value (H0). max =log2k), ensuring that the normalized value does not exceed 1.

[0059] The standard deviation of the data packet arrival interval σ, the Shannon entropy value H of the data packets, and the probability distribution p of each interval are used to determine the data packet arrival interval. i And the fused protocol fingerprint, in protocol conflict scenarios, introduces an innovative protocol similarity weight w:

[0060] w=α·σ+β·H+γ·KL(p i ||q i )

[0061] Where KL is the KL divergence of the length distribution, α, β, γ are dynamic adjustment coefficients, and q i It is the length distribution of known protocols in the protocol library, used to quantify the similarity between the current data packet and the target protocol.

[0062] The dynamic adjustment coefficients a, b, and g are initialized through experiments and optimized in real time depending on reinforcement learning to adapt to changes in the network environment and ensure that the similarity weight w can dynamically reflect the importance of different features to improve the recognition robustness in the protocol conflict scenario.

[0063] The protocol similarity weight w plays an implicit role in multiple aspects such as conflict resolution, reinforcement learning reward design, and protocol library management by quantifying the similarity of protocol features. This mechanism enhances the adaptive ability of the system in complex network environments and is one of the core innovations of the dynamic protocol recognition method.

[0064] S4: The edge AI classification engine is used to classify the data packet to determine whether it is a known protocol, wherein the input layer of the edge AI classification engine receives a multi-dimensional feature vector V, and the output layer is mapped to a protocol label.

[0065] In this step, the edge AI classification engine is based on a lightweight convolutional neural network (CNN), but with the following improvements: using depthwise separable convolution to replace the standard 3x3 convolution layer of the traditional CNN network, reducing the parameter amount by 72%; using dynamic quantization perception training to compress the model weight to 8-bit integer (memory occupancy <50MB); using an adaptive pooling layer that can dynamically adjust the pooling kernel size according to the input feature dimension.

[0066] The edge AI classification engine runs locally on the gateway and supports processing >1000 data packets per second. The edge AI classification engine is optimized through the integration of NPU instruction sets (such as ARM CMSIS-NN) to achieve hardware acceleration; decouples feature extraction and classification tasks to achieve multi-threaded processing and pipeline parallel processing; and removes redundant neurons (sparsity >60%) to retain key feature paths and achieve model pruning. The actual processing speed on Raspberry Pi 4B (Cortex-A72) is 1200 packets / s, significantly improving the protocol recognition efficiency and response capability of edge devices.

[0067] S5: If it is determined that it is not a known protocol, generate a protocol fingerprint of the data packet and update the protocol library of the edge AI classification engine.

[0068] Specifically, according to the data packet arrival interval time standard deviation, the Shannon entropy value of the data packet, and the probability distribution of each interval, the protocol fingerprint is generated and the fingerprint library of the edge AI classification engine is updated. In the fingerprint library, the fingerprint information is stored in a structured manner to support conflict resolution, incremental updating, and fast matching.

[0069] S6: If it is determined that it is a known protocol, the search strategy of the edge AI classification engine is optimized according to the current protocol type, channel load rate, and conflict rate.

[0070] Specifically, heterogeneous protocol unified processing is realized by format adaptation and semantic mapping, and real-time performance is ensured by relying on a lightweight engine. Then, the protocol is attached with metadata in a standardized format to ensure the compatibility of downstream systems, and the transmission efficiency is improved by compression and QoS. Finally, the search strategy of the edge AI classification engine is optimized, and the continuous optimization of protocol priority and resource allocation is realized through reinforcement learning and real-time monitoring.

[0071] Specifically, the process of optimizing the search strategy of the edge AI classification engine is as follows:

[0072] S601: Encode the current protocol type, channel load rate and historical conflict rate into a state matrix. Wherein, the current protocol type is a protocol type vector, such as the softened version of the protocol type vector encoded in One-hot, which is [0.9, 0.1, 0,..., 0.3]. The channel load rate is the real-time channel occupancy rate, which reflects the degree of current channel occupation by data transmission, such as 70%. The historical conflict rate is the average conflict rate in a sliding window, which is a concept of data processing that defines a fixed size interval on time series data. The average conflict rate indicates the proportion of data transmission conflicts within this sliding window, such as 5%.

[0073] The matrix example is as follows:

[0074]

[0075] S602: Take the conflict rate reduction amplitude as the reward value of machine learning, and dynamically adjust the protocol priority.

[0076] Specifically, after each round of strategy execution, the conflict rate change is calculated as the reward value R of machine learning:

[0077]

[0078] Wherein, ΔConflictRate represents the change of conflict rate, that is, the difference between the previous conflict rate and the current conflict rate; PreviousRate is the conflict rate before the previous strategy execution. For example, if the conflict rate decreases from 15% to 5%, then

[0079] This positive reward value indicates that the conflict rate has decreased after the strategy execution, which is a good result; if the conflict rate increases, the reward value will be negative, indicating that the current strategy has led to a worse network condition.

[0080] S603: Update the strategy network parameter θ by Q-learning:

[0081] θ t+1 = θ t + η·(R + γ'maxQ(st+1 a)-Q(s t ,a))

[0082] wherein, θ t is the policy network parameter at the current time (tth round); η is the learning rate, which controls the magnitude of parameter change at each update; R is the reward value; γ' is the discount factor, serving the long-term reward balance function module; Q(st,a) represents the long-term expected cumulative reward value of selecting action a in state s t in Q-learning, which plays a core role in guiding the agent (such as the protocol identification system) to select the optimal action in a specific state.

[0083] Q-learning is a model-free reinforcement learning algorithm used to learn the optimal policy in a given environment. In protocol inference, the protocol conversion path with high reward value is preferentially selected (such as LoRa is preferred to ZigBee).

[0084] As shown in Figure 2 , a dynamic protocol identification method based on multi-modal data according to the preferred embodiment of the present application comprises:

[0085] a standard deviation acquisition module 1 for calculating the standard deviation of the data packet arrival interval time to determine whether the communication mode is periodic or bursty communication mode;

[0086] a protocol complexity acquisition module 2 for dividing the data packets according to length, calculating the probability distribution of each interval, and calculating the Shannon entropy value of the data packets;

[0087] a protocol fingerprint generation module 3 for generating a protocol fingerprint according to the standard deviation of the data packet arrival interval time, the Shannon entropy value of the data packets, and the probability distribution of each interval;

[0088] a protocol classification module 4 for classifying the data packets using an edge AI classification engine to determine whether it is a known protocol, wherein the input layer of the edge AI classification engine receives a multi-dimensional feature vector V, and the output layer is mapped to a protocol label;

[0089] an unknown protocol update module 5 for generating a protocol fingerprint of the data packets and updating the protocol library of the edge AI classification engine when it is determined that it is not a known protocol;

[0090] a known protocol update module 6 for optimizing the search strategy of the edge AI classification engine according to the current protocol type, channel load rate, and collision rate when it is determined that it is a known protocol.

[0091] The foregoing description of specific exemplary embodiments of the application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise forms disclosed, and obviously many modifications and variations are possible in light of the above teaching. It is intended that the scope of the application be limited not with this detailed description, but rather by the claims appended hereto.

Claims

1. A dynamic protocol identification method based on multi-modal data, characterized in that, The method comprises the following steps: S1: calculating the standard deviation of the data packet arrival interval time to determine whether the communication mode is periodic or bursty; S2: dividing the data packet according to the length, calculating the probability distribution of each interval, and calculating the Shannon entropy value of the data packet; S3: generating a protocol fingerprint according to the standard deviation of data packet arrival interval time, Shannon entropy value of data packet, and probability distribution of each interval; in step S3, the standard deviation of data packet arrival interval time σ , Shannon entropy value of data packet H , and probability distribution of each interval p i are concatenated into a multi-dimensional feature vector V: ， Then, a hash operation is performed on the multi-dimensional feature vector V to obtain a protocol fingerprint; S4: using an edge AI classification engine to classify the data packet to determine whether it is a known protocol, wherein the input layer of the edge AI classification engine receives the multi-dimensional feature vector V, and the output layer is mapped to a protocol label; S5: if it is determined that it is not a known protocol, generating a protocol fingerprint of the data packet and updating the protocol library of the edge AI classification engine; in step S5, the protocol fingerprint is generated and the fingerprint library of the edge AI classification engine is updated according to the standard deviation of the data packet arrival interval time, the Shannon entropy value of the data packet, and the probability distribution of each interval; the fingerprint information is stored in the fingerprint library in a structured manner, supporting conflict resolution, incremental updating, and fast matching; S6: if it is determined that it is a known protocol, optimizing the search strategy of the edge AI classification engine according to the current protocol type, the channel load rate, and the conflict rate; in step S6, heterogeneous protocol unified processing is realized through format adaptation and semantic mapping; then, the protocol is attached with metadata in a standard format; finally, the search strategy of the edge AI classification engine is optimized; The process of optimizing the search strategy of the edge AI classification engine is as follows: S601: encoding the current protocol type, the channel load rate, and the historical conflict rate into a state matrix; S602: taking the conflict rate reduction amplitude as the reward value of machine learning to dynamically adjust the protocol priority; S603: updating the policy network parameters through Q-learning.

2. The dynamic protocol identification method based on multi-modal data of claim 1, wherein, In step S1, the standard deviation of the data packet arrival interval times is calculated from the sequence of data packet arrival time intervals :​ 。 3. The dynamic protocol identification method based on multi-modal data of claim 1, wherein, In step S2, the data packet length is divided into intervals, the frequency of each interval is counted , and normalized into a probability distribution ; then, based on the probability distribution , the Shannon entropy value H is calculated 。 4. The dynamic protocol identification method based on multi-modal data of claim 1, wherein, The multi-dimensional feature vector V is a 128-dimensional feature vector; when the multi-dimensional feature vector V is less than 128-dimensional, it is filled, feature expanded, or embedded mapped to complete; when the multi-dimensional feature vector V is more than 128-dimensional, it is reduced, feature selected, or truncated compressed.

5. The dynamic protocol identification method based on multi-modal data of claim 1, wherein, In step S4, the convolution layer of the edge AI classification engine is a depth separable convolution.

6. A dynamic protocol identification system based on the dynamic protocol identification method of any one of claims 1-5, characterized in that, It comprises: a standard deviation acquisition module for calculating the standard deviation of the data packet arrival interval time to determine whether the communication mode is periodic or bursty; a protocol complexity acquisition module for dividing the data packet according to the length, calculating the probability distribution of each interval, and calculating the Shannon entropy value of the data packet; a protocol fingerprint generation module for generating a protocol fingerprint according to the standard deviation of the data packet arrival interval time, the Shannon entropy value of the data packet, and the probability distribution of each interval; a protocol classification module for using an edge AI classification engine to classify the data packet to determine whether it is a known protocol, wherein the input layer of the edge AI classification engine receives the multi-dimensional feature vector V, and the output layer is mapped to a protocol label; an unknown protocol updating module for generating a protocol fingerprint of the data packet and updating the protocol library of the edge AI classification engine when it is determined that it is not a known protocol; a known protocol updating module for optimizing the search strategy of the edge AI classification engine according to the current protocol type, the channel load rate, and the conflict rate when it is determined that it is a known protocol.

Images