Industrial protocol identification method and system

Abstract

The invention relates to the technical field of industrial data automatic processing, and discloses an industrial protocol identification method and system, and the method comprises the steps: obtaining industrial data received based on a network interface in real time; performing key field identification on the industrial data to obtain a feature vector; inputting the feature vector into a pre-trained protocol type recognition model to obtain an initial protocol type recognition result; and based on the feature vector of the industrial data and the corresponding initial protocol type identification result, performing reinforcement learning by using a pre-trained reinforcement learning model to obtain a final protocol type corresponding to the industrial data. According to the method, the initial protocol type identification result is determined according to the feature vector of the industrial data, and the final protocol type is obtained by performing reinforcement learning by using the reinforcement learning model on the basis, so that the data acquisition efficiency is improved, and meanwhile, the expansibility, flexibility and economy of an industrial data acquisition system are improved; and the industrial system is promoted to develop towards a more efficient and intelligent direction.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial data automated processing, and in particular to an industrial protocol identification method and system. Background Art

[0002] Industrial data acquisition systems often face challenges such as device diversity and inconsistent protocols. Existing systems often rely on fixed protocol converters. These systems often require downtime for manual adjustments or reconfiguration when new devices or communication protocols are introduced, resulting in poor scalability, low flexibility, and high maintenance costs. Summary of the Invention

[0003] In view of this, the present invention provides an industrial protocol identification method and system to solve the problem that the existing industrial data acquisition system has low efficiency in identifying multiple types of industrial protocols, resulting in poor scalability and flexibility in maintaining the industrial data acquisition system and high operation and maintenance costs.

[0004] In a first aspect, the present invention provides an industrial protocol identification method, the method comprising:

[0005] Real-time acquisition of industrial data received via network interface;

[0006] Identifying key fields of the industrial data to obtain feature vectors;

[0007] Inputting the feature vector into a pre-trained protocol type recognition model to obtain an initial protocol type recognition result;

[0008] Based on the feature vector of industrial data and the corresponding initial protocol type recognition results, reinforcement learning is performed based on the pre-trained reinforcement learning model to obtain the final protocol type corresponding to the industrial data.

[0009] The industrial protocol identification method provided by the embodiments of the present invention receives industrial data in real time through a network interface, ensuring data timeliness. Based on the feature vectors of the industrial data and the corresponding initial protocol type identification results, a reinforcement learning model is then used to perform reinforcement learning to obtain the final protocol type. Reinforcement learning can be continuously optimized and adjusted according to actual conditions. For new or previously unencountered special industrial data or protocol changes, it can further learn and adapt based on existing knowledge. This allows the entire system to accurately identify protocol types even in complex and changing industrial environments, constantly updated equipment, and protocols. This avoids the drawbacks of traditional fixed protocol converters that require manual downtime or reconfiguration, improves the system's scalability and flexibility, reduces maintenance costs, and overall makes the operation and maintenance of industrial data acquisition systems more convenient and economical. After accurately identifying the protocol type, industrial data from different sources and protocols can be better integrated throughout the entire industrial system, achieving data interconnection and interoperability. Each link can accurately analyze the data based on the protocol, providing strong data support for optimizing production processes, improving equipment collaboration, and making more scientific management decisions, thereby promoting the development of the entire industrial system towards a more efficient and intelligent direction.

[0010] In an optional embodiment, identifying key fields of the industrial data to obtain a feature vector includes:

[0011] It is beneficial to preset the window size and the sliding step size to divide the acquired industrial data into multiple continuous data segments;

[0012] Parse each data fragment to identify key fields, including: data frame header, identification code, data type, data length, and checksum;

[0013] The key fields are normalized to obtain a feature vector.

[0014] The embodiment of the present invention utilizes a preset window size and a preset sliding step size to divide industrial data into multiple continuous data segments, thereby reducing the complexity of the entire data processing process and making the processing process more efficient. By analyzing these local samples, the characteristic changes of industrial data in different local areas can be better captured, and the key fields reflecting the essential characteristic information of the protocol can be extracted to more accurately determine the protocol type. After standardization processing, this enables the model to more effectively learn the relationship between the feature vector and the protocol type, thereby improving the accuracy and generalization ability of the model. At the same time, standardization processing also helps to reduce the impact of noise and outliers in the data on the model, so that the model can run more stably.

[0015] In an optional embodiment, the training process of the protocol type recognition model includes:

[0016] Labeling the corresponding classification label for the feature vector corresponding to each industrial data, wherein the classification label includes: a protocol type label and an abnormality label;

[0017] The feature vector with the classification label is used as training data, and the preset neural network model is trained using the preset loss function. The trained model is used as the protocol type recognition model to output the protocol type of industrial data or the probability corresponding to the anomaly;

[0018] According to the protocol type of the output industrial data or the probability corresponding to the anomaly, it is determined that the industrial data corresponds to the protocol type or is abnormal data.

[0019] The embodiment of the present invention utilizes feature vectors that carry the core features of industrial data, combined with classification labels, to enable the preset neural network model to learn the characteristic laws corresponding to different protocol types and abnormal situations from these valuable data, maximize the role of data, and enable the model to accurately summarize the characteristic patterns corresponding to each category. Then, when faced with new industrial data, effective judgments can be made based on these learned patterns. The judgment method based on output probability is more scientific and flexible.

[0020] In an optional embodiment, the feature vector based on the industrial data and the corresponding initial protocol type recognition result is subjected to reinforcement learning based on a pre-trained reinforcement learning model to obtain the final protocol type corresponding to the industrial data, including:

[0021] The feature vector of industrial data is used as the state space description of the preset reinforcement learning model;

[0022] An initial protocol type recognition result determined based on the feature vector of the industrial data is used as an action;

[0023] The reward rules are set based on the comparison between the initial protocol type recognition result and the true label, including: if the initial protocol type recognition result is consistent with the true label, the reward is positive, and if it is inconsistent, the reward is negative;

[0024] The final protocol type recognition result is obtained by reinforcing the learning model to maximize the cumulative reward.

[0025] The embodiment of the present invention uses the initial protocol type recognition result determined based on the characteristic vector of industrial data as an action, making full use of the preliminary judgment output by the previous protocol type recognition model. This initial result itself is obtained based on the analysis of data features. Although there may be certain errors, it contains certain effective information. Introducing it into the reinforcement learning model as an action can provide a valuable reference direction for further learning of the model, guiding the model to optimize and adjust on this basis, making the final protocol type recognition result more accurate, and realizing the effective integration of analysis results at different stages. The initial protocol type recognition in the early stage is a relatively fast preliminary judgment based on the existing model, and reinforcement learning is further improved on this basis. By integrating the initial result as an action into the reinforcement learning process, combined with the dynamic optimization capability of reinforcement learning, the advantages of multi-stage analysis can be brought into play, making up for the possible shortcomings of single-stage analysis, and improving the quality of overall protocol type recognition.

[0026] In an optional embodiment, the process of training the reinforcement learning model includes:

[0027] Randomly initialize the deep Q network parameters and the experience replay buffer D is empty;

[0028] Input the feature vector St of the industrial data sample into the deep Q network to obtain the Q value of the action;

[0029] Use the ε-greedy strategy to randomly select action At with ε probability and select the action with the largest current Q value with 1-ε probability;

[0030] According to the selected action At, output the protocol type;

[0031] If the action prediction is correct, the reward Rt = +1; if the prediction is wrong, the penalty Rt = -1;

[0032] Store the current experience (St, At, Rt, St+1) into the experience replay buffer D;

[0033] Randomly extract samples from cache D, and use the sampled experience to calculate the target Q value, and use the current network to estimate the value of the current state. The difference between the Q value estimate of the current network and the target Q value is calculated through the preset loss function, and the network parameters are optimized to reduce the loss function. When the preset number of training rounds or convergence conditions are reached, a trained reinforcement learning model is obtained, which is used to output the protocol type of industrial data.

[0034] During the reinforcement learning model training process, the embodiments of the present invention use an ε-greedy strategy, randomly selecting an action At with a probability of ε and selecting the action with the highest current Q-value with a probability of 1-ε. This balances the model's exploration of new actions with its utilization of existing, better-performing actions. Initially, the model has limited knowledge of industrial data protocol types and needs to randomly select actions with a certain probability to explore more possible protocol types and discover potentially better options. However, as training progresses and the model accumulates experience, the probability of selecting the action with the highest Q-value increases, allowing it to better utilize the learned effective strategy to accurately identify protocol types. This balancing mechanism helps the model learn more comprehensively and efficiently, avoids premature regression into local optimal solutions, and improves the model's ultimate performance. Industrial data is often complex and volatile, with new equipment and protocols constantly emerging. The ε-greedy strategy enables the model to discover new and appropriate methods for identifying protocol types in this complex and changing environment, ensuring it can adapt to the ever-changing realities of industrial scenarios and enhancing the model's adaptability to different industrial data protocol types.

[0035] In an optional embodiment, the method further includes: dynamically loading a corresponding parsing template from a preset database according to the protocol type, and parsing the industrial data using the parsing template.

[0036] The industrial sector often contains numerous different devices that may use a variety of communication protocols. Furthermore, with technological advancements and the introduction of new devices, the number of protocol types will continue to increase. The embodiments of the present invention utilize dynamic loading of parsing templates to effectively cope with this diversity of protocols. Whether facing existing common protocols or newly emerging protocols, as long as the corresponding parsing template is available in the preset database, the system can quickly load and use it to parse the corresponding industrial data. This eliminates the need for large-scale redesign or modification of the entire parsing system and allows for flexible adaptation to the industrial data parsing needs of different protocols, thus expanding its scope of application.

[0037] In a second aspect, the present invention provides an industrial protocol identification system, comprising:

[0038] Industrial data acquisition module, used to acquire industrial data received based on the network interface in real time;

[0039] A feature vector extraction module, configured to identify key fields of the industrial data to obtain feature vectors;

[0040] A first recognition module is used to input the feature vector into a pre-trained protocol type recognition model to obtain an initial protocol type recognition result;

[0041] The second recognition module is used to perform reinforcement learning based on the feature vector of the industrial data and the corresponding initial protocol type recognition results, and obtain the final protocol type corresponding to the industrial data using a pre-trained reinforcement learning model.

[0042] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the industrial protocol identification method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0043] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the industrial protocol identification method of the first aspect or any corresponding embodiment thereof.

[0044] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the industrial protocol identification method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 is a flow chart of an industrial protocol identification method according to an embodiment of the present invention;

[0047] Figure 2 is a flow chart of another industrial protocol identification method according to an embodiment of the present invention;

[0048] Figure 3 is a structural block diagram of an industrial protocol identification system according to an embodiment of the present invention;

[0049] Figure 4 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0051] In order to solve the problem that the existing industrial data acquisition system has low efficiency in identifying multiple types of industrial protocols, which leads to poor scalability and flexibility in maintaining the industrial data acquisition system and high operation and maintenance costs, an embodiment of the present invention provides an industrial protocol identification method embodiment. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0052] In this embodiment, an industrial protocol identification method is provided. Figure 1 FIG. 1 is a flow chart of an industrial protocol identification method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0053] S101, obtaining industrial data received based on a network interface in real time.

[0054] This embodiment of the present invention differs from the traditional single-monitoring mode by receiving the initial data packets of industrial data in real time through a network interface. It can capture all transmitted data at the data link layer. The captured data packets are cached in memory, ensuring the timeliness of the data and allowing subsequent analysis and processing to be quickly followed up, so that relevant production information can be grasped at the first moment. This avoids problems such as decision delays caused by data lag, making the information flow throughout the entire industrial process smoother and helping to improve production and operation efficiency.

[0055] S102, identifying key fields of industrial data to obtain feature vectors.

[0056] Specifically, embodiments of the present invention parse each data fragment and identify the data frame header, identification code, data type, data length, checksum, and key fields, which are key components of industrial data protocols. The data frame header typically contains information such as the protocol start flag. The identification code can be used to distinguish different devices or data sources. The data type determines the specific purpose of the data. The data length facilitates correct data reception and parsing. The checksum verifies the data's accuracy.

[0057] By identifying key fields in industrial data to obtain feature vectors, we can extract the information that best reflects the essential characteristics of the protocol. Using key features to determine the protocol type can more accurately focus on the core elements that determine the protocol type, compared to relying solely on a general analysis of the overall data, and reduce the occurrence of misjudgments.

[0058] S103: Input the feature vector into a pre-trained protocol type recognition model to obtain an initial protocol type recognition result.

[0059] The embodiment of the present invention obtains the initial protocol type recognition result with the help of a pre-trained protocol type recognition model. The pre-trained model has learned a large amount of sample data feature rules and can quickly give a relatively reliable preliminary judgment when faced with new data.

[0060] S104: Based on the feature vector of the industrial data and the corresponding initial protocol type recognition result, reinforcement learning is performed using a pre-trained reinforcement learning model to obtain the final protocol type corresponding to the industrial data.

[0061] The embodiment of the present invention utilizes reinforcement learning to continuously optimize and adjust according to actual conditions. It can further learn and adapt to some new special industrial data or protocol changes that have not been encountered before on the existing basis, so that the entire system can still accurately identify the protocol type when facing complex and changing industrial environments, constantly updated equipment and protocols, avoiding the disadvantages of traditional fixed protocol converters that require shutdown for manual adjustment or reconfiguration, and improving the scalability and flexibility of the system.

[0062] The industrial protocol identification method provided by the embodiment of the present invention receives industrial data in real time through a network interface, which can ensure the timeliness of the data. Based on the characteristic vector of the industrial data and the corresponding initial protocol type identification result, the reinforcement learning model is used to perform reinforcement learning to obtain the final protocol type, so that the entire system can still accurately identify the protocol type when facing a complex and changing industrial environment, constantly updated equipment and protocols, thereby improving the scalability and flexibility of the system, reducing maintenance costs, and making the operation and maintenance of the industrial data acquisition system more convenient and economical as a whole. After accurately identifying the protocol type, industrial data from different sources and different protocols can be better integrated in the entire industrial system to achieve data interconnection and interoperability. Each link can parse the data based on the accurate protocol, thereby providing strong data support for optimizing production processes, improving the ability of equipment to work together, making more scientific management decisions, etc., and promoting the development of the entire industrial system in a more efficient and intelligent direction.

[0063] The embodiment of the present invention also provides another industrial protocol identification method, such as Figure 2 As shown, the following steps are included:

[0064] S201, acquiring industrial data received via a network interface in real time.

[0065] For details, see step S101 and no further details will be given here.

[0066] S202: Identify key fields of industrial data to obtain feature vectors.

[0067] The embodiment of the present invention performs step S202, which specifically includes the following steps:

[0068] S2021, facilitating a preset window size and a preset sliding step size, dividing the acquired industrial data into a plurality of continuous data segments;

[0069] S2022, parse each data fragment to identify key fields, including: data frame header, identification code, data type, data length, and checksum;

[0070] S2023, normalize the key fields to obtain a feature vector.

[0071] In a specific embodiment, the method includes the following steps:

[0072] A1 sets the window size W and sliding step size S, and sets the window to move forward continuously to intercept continuous data fragments of the data packet.

[0073] A2, divide the received data stream D (data packet) into blocks according to the window W to form a window data set Dw = {d1, d2, ..., dn}.

[0074] A3 parses each window data di to obtain the following key fields:

[0075] ① Data frame header: By setting frame header identification rules, the embodiment of the present invention selects a start identifier to locate the starting point of the data packet;

[0076] ② Identification code: Extract the data identification code from the parsed frame structure for subsequent classification or pattern matching;

[0077] ③Data type: parse the data type (temperature, pressure, status value, etc.) according to the data frame header;

[0078] ④Data length and checksum: Extract the length identifier of the data field and calculate the checksum of the current data segment as a data integrity and verification feature.

[0079] A4 pre-processes the five key fields of the obtained data frame header, identification code, data type, data length, and checksum:

[0080] Header: A frame header is typically a fixed-length string or byte sequence. If it's a string, you can convert it into a numerical feature using one-hot encoding or word embedding. For byte-length data, you can directly use its binary representation.

[0081] Code: A code encodes a specific data type or information, typically a numeric value or string. If it's a numeric value, it can be used directly as a feature. If it's a string, one-hot encoding or numeric mapping can also be used.

[0082] Type: If the data type is a predefined category (such as sensor type, measurement unit, etc.), you can use one-hot encoding or directly map it to a numeric value. For example, temperature, pressure, humidity, etc. can be represented by numbers (0: temperature, 1: pressure, 2: humidity).

[0083] Length (data length): The data length is usually an integer and can be directly used as a numerical feature. The length feature does not require special encoding and the original value can be used directly.

[0084] Checksum: A checksum is usually an integer or floating point value that indicates data integrity or error detection. This value can be used directly as a feature.

[0085] After preprocessing, all these features (header, code, type, length, checksum) will be merged into a feature vector X:

[0086] X=[header_encoded, code_encoded, type_encoded, length, checksum_value].

[0087] For numerical features in the feature vector, the present embodiment uses the Z-Score normalization method. This method is accomplished by subtracting the feature value from its mean and then dividing it by the standard deviation, resulting in a feature mean of 0 and a standard deviation of 1. This is then used as input to the subsequent protocol type recognition model. The normalized feature vector enables the model to more effectively learn the relationship between the feature vector and the protocol type, improving the model's accuracy and generalization capabilities. At the same time, normalization also helps reduce the impact of noise and outliers in the data on the model, enabling the model to operate more stably.

[0088] S203: Input the feature vector into a pre-trained protocol type recognition model to obtain an initial protocol type recognition result.

[0089] Specifically, the training process of the protocol type recognition model includes the following steps:

[0090] B1. Label the corresponding classification label for the feature vector corresponding to each industrial data. The classification label includes: protocol type label and abnormal label.

[0091] Specifically, a classification label is assigned to each industrial data point. For example, label 0 indicates data anomaly, 1 indicates protocol type HTTP, 2 indicates protocol type FTP, 3 indicates protocol type TCP, etc. Form a label array Y = [1, 3, 1, 2, 0] # Classification label: 0 indicates data anomaly, 1 indicates protocol type HTTP, 2 indicates protocol type FTP, 3 indicates protocol type TCP.

[0092] B2, the feature vector with the classification label is used as training data, and the preset loss function is used to train the preset neural network model. The trained model is used as the protocol type recognition model to output the protocol type of industrial data or the probability corresponding to the anomaly.

[0093] In this embodiment of the present invention, the training data is labeled with both protocol type and anomaly markers, allowing the model to simultaneously learn the data characteristics of normal protocol types and possible anomaly data. This allows the model to not only accurately identify the protocol type corresponding to industrial data in practical applications, but also detect anomaly data, preventing the mixing of anomaly data that could lead to subsequent misjudgment of protocol types or affect the entire industrial data processing process, thereby enhancing the model's ability to cope with complex data situations.

[0094] B3, according to the protocol type of the output industrial data or the probability corresponding to the anomaly, determining whether the industrial data corresponds to the protocol type or is abnormal data.

[0095] Specifically, in real industrial scenarios, data may contain a certain degree of ambiguity or uncertainty, rather than being absolutely binary. Through probabilistic output, users can understand the likelihood of different protocol types or abnormal situations. For example, when the probability of a certain protocol type exceeds a certain value, it is determined to be that protocol type. This probabilistic judgment method is more scientific and flexible, and can better adapt to complex and changing industrial data environments.

[0096] In the embodiment of the present invention, the protocol type identification model used is a neural network model, and its structure is:

[0097] ① Network input layer: The feature vector X = [x1, x2, ..., xn] obtained from the data preprocessing stage, where n is the dimension of the feature. The number of nodes in the input layer of the network is the dimension n of the feature vector. In the embodiment of the present invention, the dimension is 5, namely Header, Code, Type, Length, Checksum. The feature vectors are input into the neural network one by one through the input layer.

[0098] ② Hidden layer: The number of nodes in each layer is set based on the input dimension and task complexity. The first hidden layer typically has more nodes than the input layer, and the number of nodes in subsequent hidden layers decreases layer by layer, forming a pyramid structure. In this embodiment, the number of input layer nodes is n = 5, so the number of hidden layer nodes can be designed to be [16, 8], that is, 16 nodes in the first hidden layer and 8 nodes in the second hidden layer. Adding an activation function after each hidden layer introduces nonlinearity, helping the network learn more complex feature relationships. Adding a Sigmoid activation function after each hidden layer introduces nonlinearity, helping the network learn more complex feature relationships.

[0099] ③ Output layer design: The output layer has different structures and activation functions according to different task types. The dimension of the output layer is equal to the number of categories C. The Softmax activation function is used to convert the network output into a category probability distribution and output the probability of each category. The output result is a vector [p1, p2, ..., pC], where p represents the probability of belonging to the i-th category.

[0100] ④ Loss function: Using binary cross entropy loss, the loss L is calculated using the following loss function formula (1), where y is the true label and p is the predicted probability:

[0101]

[0102] S204: Based on the feature vector of the industrial data and the corresponding initial protocol type recognition result, reinforcement learning is performed using a pre-trained reinforcement learning model to obtain the final protocol type corresponding to the industrial data.

[0103] Specifically, executing step S204 includes the following steps:

[0104] S2041, use the feature vector of industrial data as the state space description of the reinforcement learning model.

[0105] Specifically, state S represents the system's current observed data or environmental state. In the protocol recognition scenario, state S primarily comes from the aforementioned feature vector, representing the current state of the packet and containing key information about the packet. Assume a packet has the following features: header = 0x1234, code = 0x01, type = 0x03, length = 64, checksum = 0xabc. These values form a feature vector X = [0x1234, 0x01, 0x03, 64, 0xabc], which is state S.

[0106] S2042: An initial protocol type recognition result determined according to the feature vector of the industrial data is used as an action.

[0107] Specifically, action A is the decision the model takes based on state S. In protocol identification tasks, the model typically determines the protocol type of a packet or whether it is an anomaly based on input features. Specifically, the model selects the most likely protocol type or anomaly flag based on the current feature vector S. Assuming state S = [0x1234, 0x01, 0x03, 64, 0xabc], the protocol type identification model ultimately outputs a protocol category A = Protocol 1 or A = Anomaly.

[0108] S2043, setting a reward rule based on the comparison result between the initial protocol type recognition result and the true label, including: if the initial protocol type recognition result is consistent with the true label, the reward is a positive value, and if it is inconsistent, the reward is a negative value.

[0109] Specifically, the reward R is the feedback the model receives based on the effect of an action after it takes it, and is used to guide the learning process. Rewards are usually related to the model's prediction accuracy, with the goal of encouraging the model to make correct decisions. The reward in the protocol identification task is based on whether the model accurately predicts the protocol type of the data packet, or whether it successfully detects abnormal data. The reward R comes from the comparison between the model's prediction effect and the true label. If the model predicts correctly (that is, the predicted protocol class matches the actual protocol, or the data packet is correctly judged as abnormal), the reward is positive; if the model predicts incorrectly, the reward is negative. For example, if the protocol type identification model judges that the data packet belongs to protocol 1, and the actual protocol is also protocol 1, then the reward R = +1; if the protocol type identification judges that the data packet is abnormal, but it is actually a normal data packet, then the reward R = -1.

[0110] S2044, through the reinforcement learning of the reinforcement learning model to maximize the cumulative reward, the final protocol type recognition result is obtained.

[0111] The deep Q network (DQN) used in the reinforcement learning model in the embodiment of the present invention calculates the value of executing action A in state S: its input layer inputs the feature vector X (state S), the hidden layer includes multiple fully connected layers, and an activation function (such as a Sigmoid function) is introduced to extract deep features; the output layer outputs the Q value of all possible actions, with the dimension being the number of actions |A|, that is, the number of protocol categories. In this task, the action is to select a protocol category, such as HTTP, FTP, TCP, UDP, exceptions, etc. Q(S,A) represents the predicted value of each action output by the model. The Q value represents the cumulative reward that can be obtained after selecting a certain action (protocol) starting from the current state. The Q value is used to indicate the quality of selecting a certain action in a certain state. In protocol identification, it helps the system determine which protocol is the best to select under given data packet characteristics.

[0112] In one embodiment, Q-learning (DQN) is used to identify the protocol type of a data packet:

[0113] State S: feature vector of the data packet;

[0114] Action A: Select the protocol type;

[0115] Q-value: For each protocol category (action), a Q-value is assigned, which represents the expected reward of selecting that protocol in the current state. For example, if the packet feature vector shows that it best matches the HTTP protocol, then the Q-value for HTTP will be higher, indicating that selecting HTTP will bring a higher reward.

[0116] Status judgment: Based on the characteristics of the data packet (specific headers or fields), the "status" of the current data packet is determined, that is, which protocol it belongs to or whether it is an abnormal data packet. For example, if the characteristics of the data packet closely match those of the HTTP protocol (such as containing a GET request), its status is determined to be HTTP protocol. If the data packet does not match any known protocol characteristics (such as damaged content or unknown format), it can be determined that the data packet is abnormal.

[0117] Final selection (action selection): In DQN, the action with the highest Q value is the selected protocol type or state judgment: Input feature vector St, the network outputs the Q value corresponding to each action (protocol category). The action with the highest Q value is selected. For example, if the highest Q value is HTTP, HTTP is selected as the predicted protocol type for the current packet.

[0118] The process of training the reinforcement learning model in the implementation of the present invention includes:

[0119] C1, randomly initialize the deep Q network parameters, and the experience replay buffer D is empty;

[0120] C2, input the feature vector St of the industrial data sample into the deep Q network to obtain the Q value of the action;

[0121] C3, using the ε-greedy strategy, randomly selects action At with ε probability, and selects the action with the largest current Q value with 1-ε probability;

[0122] C4, outputs the protocol type according to the selected action At;

[0123] C5, if the action prediction is correct, reward Rt = +1; if the prediction is wrong, penalty Rt = -1;

[0124] C6, stores the current experience (St, At, Rt, St+1) into the experience replay buffer D;

[0125] C7 randomly extracts samples from cache D and uses the sampled experience to calculate the target Q value. It also uses the current network to estimate the value of the current state and calculates the difference between the current network's Q value estimate and the target Q value through a preset loss function.

[0126] C8 optimizes network parameters to reduce the loss function. When the preset number of training rounds or convergence conditions are reached, a trained reinforcement learning model is obtained to output the protocol type of industrial data.

[0127] The loss function Loss in the embodiment of the present invention is calculated by the following formula (2):

[0128] Loss=[R t +γ·maxQ(S t+1 , A′)-Q(S t , A t )] 2 (2)

[0129] Loss represents the difference between the current Q-value and the target Q-value; Rt represents the immediate reward obtained after taking action At in the current state St; γ represents a discount factor, which is used to weigh the importance of current rewards against future rewards, and ranges from 0 ≤ γ ≤ 1. A larger γ value indicates a greater focus on future rewards, while a smaller γ value indicates a greater focus on immediate rewards; Q(St,At) represents the estimated Q-value of taking action At in the current state St. The Q-value represents the cumulative reward obtained by taking an action starting from the current state; St+1 represents the next state after taking action At in the current state St; maxQ(St+1,A′) represents the maximum Q-value of all possible actions A′ in state St+1. This value represents the maximum future reward that can be obtained by following the optimal strategy starting from state St+1; A′ represents a possible action in state St+1S_{t+1}St+1.

[0130] The embodiment of the present invention uses an ε-greedy strategy when training the model, randomly selecting an action At with a probability of ε, and selecting the action with the largest current Q value with a probability of 1-ε, thereby balancing the model's exploration of new actions and the use of existing better actions. In the early stages of training, the model has limited knowledge of industrial data protocol types, and needs to explore more possible protocol types and discover potential better options by randomly selecting actions with a certain probability. As training progresses, the model accumulates a certain amount of experience, and the probability of selecting an action based on the maximum Q value increases, and it can better utilize the effective strategies that have been learned to accurately identify protocol types. This balancing mechanism helps the model learn more comprehensively and efficiently, avoids falling into a local optimal solution too early, improves the final performance of the model, ensures that it can cope with the ever-changing actual conditions in industrial scenarios, and enhances the model's adaptability to different industrial data protocol types.

[0131] In the embodiment of the present invention, when real-time data is accessed, the deep learning model dynamically analyzes the protocol type or determines the status of the data based on the feature vector, and continuously updates the deep Q network parameters through the online learning mechanism to improve the accuracy.

[0132] S205 , dynamically loading a corresponding parsing template from a preset database according to the protocol type, and parsing the industrial data using the parsing template.

[0133] Specifically, different protocol types have their own unique structures, rules, and data format requirements. The embodiment of the present invention can ensure that the parsing method adopted fully conforms to the characteristics of the protocol by dynamically loading the parsing template corresponding to the specific protocol type from a preset database. For example, some protocols have specific provisions for the order, length, encoding method, etc. of data fields. The corresponding parsing template will accurately extract and interpret each data part based on these rules, avoiding the deviation or error in understanding industrial data that may be caused by the use of a general but less accurate parsing method, thereby greatly improving the accuracy of industrial data parsing.

[0134] If a protocol standard is updated or improved, simply adjust the corresponding parsing template in the preset database. When the system subsequently parses industrial data, it can still accurately load the latest parsing template based on the protocol type and adapt to protocol changes in real time. This dynamic update capability ensures that the system can always keep pace with the ever-changing industrial protocol environment during long-term use, continuously and effectively parsing industrial data without problems such as parsing failures caused by protocol changes.

[0135] The embodiment of the present invention uses a structure based on a graph database to store templates to achieve rapid retrieval and updating; performs a version compatibility check when loading the parsing template, and tests the validity of the template in a sandbox environment. In the sandbox environment, various actual industrial data scenarios and possible abnormal situations can be simulated to conduct a comprehensive validity test on the parsing template, which can detect in advance possible problems that may arise in the template during the parsing process, such as inaccurate parsing of certain special format data, inability to correctly perform parsing operations under specific conditions, etc. After discovering the problem, the template can be optimized and adjusted in a timely manner to ensure that it can complete the task efficiently and accurately when it is officially applied to actual industrial data parsing work, thereby improving the quality and reliability of the template.

[0136] This embodiment also provides an industrial protocol identification system for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the systems described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0137] This embodiment provides an industrial protocol identification system, such as Figure 3 Shown, including:

[0138] The industrial data acquisition module 301 is used to acquire industrial data received based on a network interface in real time;

[0139] A feature vector extraction module 302 is used to identify key fields of industrial data to obtain feature vectors;

[0140] The first recognition module 303 is used to input the feature vector into a pre-trained protocol type recognition model to obtain an initial protocol type recognition result;

[0141] The second recognition module 304 is used to perform reinforcement learning based on the feature vector of the industrial data and the corresponding initial protocol type recognition result using a pre-trained reinforcement learning model to obtain the final protocol type corresponding to the industrial data.

[0142] In some optional implementations, the feature vector extraction module 302 includes:

[0143] A data partitioning unit, configured to partition the acquired industrial data into a plurality of continuous data segments according to a preset window size and a preset sliding step size;

[0144] Key field identification unit, used to parse each data segment to identify key fields, including: data frame header, identification code, data type, data length, checksum;

[0145] The feature vector output unit is used to normalize the key fields to obtain the feature vector.

[0146] In some optional implementations, the training process of the protocol type recognition model in the first recognition module 303 includes:

[0147] Labeling the corresponding classification label for the feature vector corresponding to each industrial data, wherein the classification label includes: a protocol type label and an abnormality label;

[0148] The feature vector with the classification label is used as training data, and the preset neural network model is trained using the preset loss function. The trained model is used as the protocol type recognition model to output the protocol type of industrial data or the probability corresponding to the anomaly;

[0149] According to the protocol type of the output industrial data or the probability corresponding to the anomaly, it is determined that the industrial data corresponds to the protocol type or is abnormal data.

[0150] In some optional implementations, the second identification module 304 includes:

[0151] A state space description unit, used to use the feature vector of industrial data as a state space description of a preset reinforcement learning model;

[0152] An action set unit, configured to use an initial protocol type recognition result determined according to a feature vector of industrial data as an action;

[0153] A reward setting unit sets reward rules based on the comparison between the initial protocol type recognition result and the true label, including: if the initial protocol type recognition result is consistent with the true label, the reward is positive, and if it is inconsistent, the reward is negative;

[0154] The recognition result output unit is used to maximize the cumulative reward through reinforcement learning of the reinforcement learning model to obtain the final protocol type recognition result.

[0155] In some optional implementations, the process of training the reinforcement learning model includes:

[0156] Randomly initialize the deep Q network parameters and the experience replay buffer D is empty;

[0157] Input the feature vector St of the industrial data sample into the deep Q network to obtain the Q value of the action;

[0158] Use the ε-greedy strategy to randomly select action At with ε probability and select the action with the largest current Q value with 1-ε probability;

[0159] According to the selected action At, output the protocol type;

[0160] If the action prediction is correct, the reward Rt = +1; if the prediction is wrong, the penalty Rt = -1;

[0161] Store the current experience (St, At, Rt, St+1) into the experience replay buffer D;

[0162] Randomly extract samples from cache D, and use the sampled experience to calculate the target Q value, and use the current network to estimate the value of the current state, and calculate the difference between the current network's Q value estimate and the target Q value through the preset loss function;

[0163] The network parameters are optimized to reduce the loss function. When the preset number of training rounds or convergence conditions are reached, a trained reinforcement learning model is obtained to output the protocol type of industrial data.

[0164] In some optional embodiments, the system further includes: a data parsing unit, configured to dynamically load a corresponding parsing template from a preset database according to the protocol type, and parse the industrial data using the parsing template.

[0165] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0166] The industrial protocol identification system in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0167] The embodiment of the present invention also provides a computer device having the above Figure 3 The industrial protocol identification system shown.

[0168] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 4As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0169] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0170] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.

[0171] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0172] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0173] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0174] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor central control system or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0175] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0176] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An industrial protocol identification method, characterized in that: include: Real-time acquisition of industrial data received via network interface; Identifying key fields of the industrial data to obtain feature vectors; Inputting the feature vector into a pre-trained protocol type recognition model to obtain an initial protocol type recognition result; Based on the feature vector of industrial data and the corresponding initial protocol type recognition results, reinforcement learning is performed based on the pre-trained reinforcement learning model to obtain the final protocol type corresponding to the industrial data.

2. The method according to claim 1, characterized in that The step of identifying key fields of the industrial data to obtain a feature vector includes: It is beneficial to preset the window size and the sliding step size to divide the acquired industrial data into multiple continuous data segments; Parse each data fragment to identify key fields, including: data frame header, identification code, data type, data length, and checksum; The key fields are normalized to obtain a feature vector.

3. The method according to claim 2, characterized in that The training process of the protocol type recognition model includes: Labeling the corresponding classification label for the feature vector corresponding to each industrial data, wherein the classification label includes: a protocol type label and an abnormality label; The feature vector with the classification label is used as training data, and the preset neural network model is trained using the preset loss function. The trained model is used as the protocol type recognition model to output the protocol type of industrial data or the probability corresponding to the anomaly; According to the protocol type of the output industrial data or the probability corresponding to the anomaly, it is determined that the industrial data corresponds to the protocol type or is abnormal data.

4. The method according to claim 1, wherein The feature vector based on the industrial data and the corresponding initial protocol type recognition result are used to perform reinforcement learning based on a pre-trained reinforcement learning model to obtain the final protocol type corresponding to the industrial data, including: The feature vector of industrial data is used as the state space description of the preset reinforcement learning model; An initial protocol type recognition result determined based on the feature vector of the industrial data is used as an action; The reward rules are set based on the comparison between the initial protocol type recognition result and the true label, including: if the initial protocol type recognition result is consistent with the true label, the reward is positive, and if it is inconsistent, the reward is negative; The final protocol type recognition result is obtained by reinforcing the learning model to maximize the cumulative reward.

5. The method according to claim 4, characterized in that The process of training the reinforcement learning model includes: Randomly initialize the deep Q network parameters and the experience replay buffer D is empty; Input the feature vector St of the industrial data sample into the deep Q network to obtain the Q value of the action; Use the ε-greedy strategy to randomly select action At with ε probability and select the action with the largest current Q value with 1-ε probability; According to the selected action At, output the protocol type; If the action prediction is correct, the reward Rt = +1; if the prediction is wrong, the penalty Rt = -1; Store the current experience (St, At, Rt, St+1) into the experience replay buffer D; Randomly extract samples from cache D, and use the sampled experience to calculate the target Q value, and use the current network to estimate the value of the current state, and calculate the difference between the current network's Q value estimate and the target Q value through the preset loss function; The network parameters are optimized to reduce the loss function. When the preset number of training rounds or convergence conditions are reached, a trained reinforcement learning model is obtained to output the protocol type of industrial data.

6. The method according to claim 1, characterized in that Also includes: The corresponding parsing template is dynamically loaded from a preset database according to the protocol type, and the industrial data is parsed using the parsing template.

7. An industrial protocol identification system, characterized in that: include: Industrial data acquisition module, used to acquire industrial data received based on the network interface in real time; A feature vector extraction module, configured to identify key fields of the industrial data to obtain feature vectors; A first recognition module is used to input the feature vector into a pre-trained protocol type recognition model to obtain an initial protocol type recognition result; The second recognition module is used to perform reinforcement learning based on the feature vector of the industrial data and the corresponding initial protocol type recognition results, and obtain the final protocol type corresponding to the industrial data using a pre-trained reinforcement learning model.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the industrial protocol identification method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the industrial protocol identification method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer to execute the industrial protocol identification method according to any one of claims 1 to 6.

Images