Protocol analysis method and device, electronic equipment and storage medium

Abstract

The invention discloses a protocol analysis method and device, electronic equipment and a storage medium, and relates to the technical field of information processing. The method comprises the following steps: acquiring N screen images and N original messages of industrial equipment; identifying the ith screen image in the N screen images to obtain at least one character sequence in the ith screen image, and generating a corresponding label according to each character sequence to form a label set; analyzing the ith original message in the N original messages to obtain a byte sequence of the ith original message; pairing the label set with the byte sequence to obtain a training sample pair corresponding to the ith screen image and the ith original message; and inputting the training sample pairs corresponding to the N screen images and the N original messages into an initial neural network model for model training to obtain a protocol analysis model so as to improve the protocol analysis capability of an unknown protocol.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a protocol parsing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of new-generation information technologies, especially the deep integration of technologies such as the Internet of Things (IoT), Artificial Intelligence (AI), big data, cloud computing, and 5G into the industrial sector, industrial digital transformation has become a core driving force for upgrading the manufacturing industry and promoting high-quality enterprise development. In the industrial field, communication protocols between devices are often the core of technological implementation. However, many protocols (especially proprietary protocols) lack public documentation, significantly increasing the difficulty of protocol parsing. Protocol reverse engineering is a key technology for solving this problem.

[0003] There are currently two main methods for protocol parsing. One method relies on the vendor's publicly available protocol specifications to develop corresponding parsing modules, but this is severely limited by the openness of the protocol (it is difficult to adapt when there are private protocols or incomplete documentation). The other method is based on machine learning models to extract features from the collected traffic and automatically classify protocols based on the extracted traffic features, but it can only identify the type of protocol and cannot parse the protocol content. Summary of the Invention

[0004] This application provides a protocol parsing method to improve the ability to parse unknown protocols.

[0005] Firstly, a protocol parsing method is provided, including:

[0006] Collect N screen images and N raw messages from industrial equipment; where N is an integer greater than 0.

[0007] The i-th screen image among the N screen images is identified to obtain at least one character sequence in the i-th screen image, and a corresponding label is generated based on each character sequence to form a label set; wherein i is an integer less than or equal to N;

[0008] The i-th original message among N original messages is parsed to obtain the byte sequence of the i-th original message; wherein the i-th screen image and the i-th original message are collected in the same time period;

[0009] Pair the tag set and the byte sequence to obtain the training sample pair corresponding to the i-th screen image and the i-th original message;

[0010] The training samples corresponding to the N screen images and N original messages are input into the initial neural network model for model training to obtain the protocol parsing model; wherein, the protocol parsing model is used to parse the input unknown protocol to be parsed and output the protocol parsing result of the unknown protocol.

[0011] Using the above method, multiple device operating status parameters (also referred to as fields) are identified and extracted from the acquired screen images, forming a tag set. This eliminates the need for manual annotation, significantly reducing annotation costs and improving the automation level of annotation. In the protocol parsing model, the original messages acquired at the same time are parsed to obtain the corresponding byte sequence, and this byte sequence is aligned with the tag set identified based on the screen images. This achieves semantic alignment between fields and device operating status parameters. The resulting protocol parsing model overcomes the bottleneck of existing solutions that only identify protocol types but cannot understand protocol content.

[0012] In one possible design, the process of recognizing the i-th screen image among the N screen images to obtain at least one character sequence in the i-th screen image, and generating corresponding tags based on each character sequence to form a tag set, includes:

[0013] Perform text detection on the i-th screen image to obtain at least one text region in the i-th screen image that contains text;

[0014] Based on the tilt of each text region, the at least one text region is corrected to obtain the at least one corrected text region;

[0015] Identify at least one corrected local region to obtain at least one character sequence in the i-th screen image;

[0016] Invalid characters in the at least one character sequence are removed according to the cleaning rules to obtain at least one target character sequence;

[0017] A tag corresponding to the first target character sequence is generated based on the parameter type and parameter value of the device operating status parameter contained in the first target character sequence; wherein, the first target character sequence is any one of the at least one target character sequence.

[0018] This design allows for the extraction of parameter types and values ​​of device status operating parameters from screen images using an OCR architecture. These parameters can then be used to construct labels for the original messages at the same time, eliminating the need for manual annotation, significantly reducing annotation costs, and improving the level of automation in annotation.

[0019] In one possible design, the step of inputting the training samples corresponding to the N screen images and N original messages into an initial neural network model for model training to obtain a protocol parsing model includes:

[0020] Dimensionality reduction is performed on the multiple bytes contained in each byte sequence to obtain the embedding vector sequences of N byte sequences respectively;

[0021] Feature encoding is performed on multiple embedding vectors in the embedding vector sequence of each byte sequence to obtain the byte feature sequences of each of the N byte sequences;

[0022] For each byte feature sequence, a prediction result set is obtained for each of the N byte sequences; wherein, each of the prediction result sets contains at least one device operating state parameter's parameter prediction type and parameter prediction value, and the start position and end position of each device operating state parameter in the byte sequence;

[0023] The loss value of the protocol parsing model is calculated based on the prediction result set of each of the N byte sequences and the label set of each of the N byte sequences.

[0024] When the loss value does not meet the threshold, the parameters of the protocol parsing model are updated until the loss value of the updated protocol parsing model meets the threshold.

[0025] This design constructs end-to-end semantic associations for training samples based on the alignment results of labels and byte sequences identified from screen images at the same time. These associations are then used to train a protocol parsing model, enabling end-to-end prediction and structural reconstruction of field positions, types, and values ​​within the protocol, thereby improving the ability to parse protocols for unknown protocols in the future.

[0026] In one possible design, the loss value satisfies the following expression:

[0027]

[0028] Among them, the The s represents the start and end positions of the i-th device operating status parameter in any byte sequence within the byte sequence. i ,e i The actual start and actual end positions of the i-th device operating status parameter in the byte sequence are characterized. The parameter prediction type characterizing the operating state parameter of the i-th device, the k i The parameter type representing the operating status parameter of the i-th device in the tag set corresponding to the byte sequence, the The predicted value of the parameter characterizing the operating state parameter of the i-th device, the vi The parameter value representing the operating status parameter of the i-th device in the tag set corresponding to the byte sequence.

[0029] In one possible design, after inputting the training samples corresponding to the N screen images and N original messages into the initial neural network model for model training to obtain the protocol parsing model, the method further includes:

[0030] The currently acquired unknown protocol is input into the protocol parsing model to predict the protocol parsing result of the unknown protocol; wherein, the protocol parsing result includes the start and end positions of each device operating status parameter in the unknown protocol, and the parameter type and parameter value corresponding to each device operating status parameter.

[0031] In one possible design, after acquiring N screen images from the industrial equipment, the process further includes:

[0032] Image preprocessing operations are performed on the N screen images respectively to obtain N processed screen images; wherein, the image preprocessing operations include at least one of the following: grayscale processing, binarization processing, noise reduction processing, contrast enhancement processing, and image region of interest cropping processing.

[0033] This design, through the aforementioned series of image preprocessing operations, can improve the accuracy of text recognition in screen images.

[0034] Secondly, a protocol parsing apparatus is provided, comprising:

[0035] The acquisition module is used to acquire N screen images and N raw messages from industrial equipment; wherein N is an integer greater than 0.

[0036] The recognition module is used to recognize the i-th screen image among the N screen images, obtain at least one character sequence in the i-th screen image, and generate a corresponding label based on each character sequence to form a label set; wherein, i is an integer less than or equal to N;

[0037] The parsing module is used to parse the i-th original message among N original messages to obtain the byte sequence of the i-th original message; wherein the i-th screen image and the i-th original message are acquired in the same time period;

[0038] The training sample construction module is used to pair the tag set and the byte sequence to obtain the training sample pair corresponding to the i-th screen image and the i-th original message;

[0039] The model training module is used to input the training samples corresponding to the N screen images and N original messages into the initial neural network model for model training to obtain the protocol parsing model; wherein, the protocol parsing model is used to parse the input unknown protocol to be parsed and output the protocol parsing result of the unknown protocol.

[0040] In one possible design, the identification module is specifically used for:

[0041] Perform text detection on the i-th screen image to obtain at least one text region in the i-th screen image that contains text;

[0042] Based on the tilt of each text region, the at least one text region is corrected to obtain the at least one corrected text region;

[0043] Identify at least one corrected local region to obtain at least one character sequence in the i-th screen image;

[0044] Invalid characters in the at least one character sequence are removed according to the cleaning rules to obtain at least one target character sequence;

[0045] A tag corresponding to the first target character sequence is generated based on the parameter type and parameter value of the device operating status parameter contained in the first target character sequence; wherein, the first target character sequence is any one of the at least one target character sequence.

[0046] Thirdly, an electronic device is provided, comprising:

[0047] A memory for storing computer programs; a processor for executing the computer programs stored in the memory to implement the method steps described in any one of the first aspects.

[0048] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the method steps described in any one of the first aspects.

[0049] For the various aspects of the second to fourth aspects mentioned above, and the technical effects that each aspect may achieve, please refer to the above description of the technical effects that can be achieved for the first aspect or the various possible solutions in the first aspect, which will not be repeated here. Attached Figure Description

[0050] Figure 1 This is a schematic diagram illustrating the application scenarios applicable to the embodiments of this application;

[0051] Figure 2A flowchart illustrating a protocol parsing method provided in this application embodiment;

[0052] Figure 3 This application provides a logical diagram illustrating the generation of a tag.

[0053] Figure 4 This application provides a logical diagram illustrating byte sequence parsing in an embodiment.

[0054] Figure 5 A logical diagram illustrating the training and application of a protocol parsing model provided in this application embodiment;

[0055] Figure 6 This is a schematic diagram of the structure of a protocol parsing device provided in an embodiment of this application;

[0056] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0058] The terms "first" and "second" in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. The term "multiple" in this application can mean at least two, for example, two, three, or more, and the embodiments of this application do not impose limitations.

[0059] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These embodiments should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that in the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solutions of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0060] To better understand the embodiments of this application, the technical terms involved in the embodiments of this application will be explained below.

[0061] (1) A convolutional neural network (CNN) consists of one or more convolutional layers and a fully connected layer at the top, and may also include structures such as associated weights and pooling layers.

[0062] (2) Optical Character Recognition (OCR) is a process that processes and recognizes images or videos containing text content and extracts the text, characters and layout information contained therein.

[0063] (3) The Transformer architecture is a deep learning architecture that employs an attention mechanism, which assigns different weights based on the varying importance of different parts of the input data. Models using this architecture are primarily used in natural language processing and computer vision.

[0064] (4) A Programmable Logic Controller (PLC) is a digital electronic system designed for industrial applications. It uses a programmable memory to store programs, execute logical operations, sequential control, timing, counting, and arithmetic operations, and control various types of machinery or production processes through digital or analog inputs / outputs.

[0065] The following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0066] Figure 1This is a schematic diagram illustrating an application scenario applicable to the embodiments of this application. As shown in the figure, the application scenario mainly includes: industrial equipment 10 and server 11. The industrial equipment 10 and server 11 can interact with each other via a communication network, which can employ communication methods including wireless communication and wired communication.

[0067] In some application scenarios, industrial equipment 10 can access the network and communicate with server 11 through cellular mobile communication technology, which may include 5th generation mobile networks (5G) technology and 6G technology.

[0068] In some application scenarios, industrial equipment 10 can access the network and communicate with server 11 via short-range wireless communication, which may include Wireless Fidelity (Wi-Fi) technology.

[0069] This application embodiment does not impose any limitation on the number of the above-mentioned devices, such as Figure 1 As shown, only industrial equipment 10 and server 11 are described as examples. The following is a brief introduction to each of the above devices and their respective functions.

[0070] Industrial equipment 10 is a device that can provide voice and / or data connectivity to users, including: handheld terminal devices with wireless connectivity, vehicle-mounted terminal devices, etc. For example, industrial equipment 10 includes, but is not limited to: wireless terminal devices in industrial control, wireless terminal devices in autonomous driving, wireless terminal devices in smart grids, wireless terminal devices in transportation safety, wireless terminal devices in smart cities, or wireless terminal devices in smart homes, etc.

[0071] Furthermore, the industrial equipment 10 may be equipped with a protocol parsing client, which can be software (e.g., an app, a browser, etc.), or a webpage, a mini-program, etc. In this embodiment, the industrial equipment 10 can use the aforementioned protocol parsing client and can perform protocol parsing interaction with the server 11. For example, after a user opens the corresponding client, the user can initiate a protocol parsing request carrying an unknown protocol on the user interface and send it to the server 11.

[0072] Furthermore, server 11 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0073] Furthermore, in this embodiment, the server 11 may be equipped with a protocol parsing device corresponding to the client. This protocol parsing device is used to train a protocol parsing model and to process protocol parsing requests for various unknown protocols initiated by the user on the user interface of the industrial equipment 10 using the trained protocol parsing model. This protocol parsing model overcomes the bottleneck of existing solutions that only identify protocol types and cannot understand protocol content, supporting field-level parsing; it can jointly model field boundaries, protocol types, and parameter values ​​in unknown protocols, improving the accuracy and generalization ability of protocol reconstruction.

[0074] To further illustrate the technical solutions provided in the embodiments of this application, a detailed description is provided below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of this application provide method operation steps as shown in the following embodiments or drawings, the method may include more or fewer operation steps based on conventional or non-inventive methods. In steps where there is no logically necessary causal relationship, the execution order of these steps is not limited to the execution order provided in the embodiments of this application. In actual processing or when the device executes the method, it may be executed in the order shown in the embodiments or drawings, or in combination.

[0075] Figure 2 This is a flowchart illustrating a protocol parsing method provided in an embodiment of this application. This process can be executed by a protocol parsing device to improve the ability to parse unknown protocols. Figure 2 As shown, the process includes the following steps:

[0076] S201: Collect N screen images and N raw messages from industrial equipment.

[0077] In this step, N is an integer greater than 0, and its specific value can be adjusted according to actual needs. No restrictions are imposed here.

[0078] In some implementations, images can be captured from the screens of PLCs or instruments on industrial equipment at set time intervals using deployed industrial cameras or USB cameras. The captured N screen images can then display the respective equipment operating status parameters in real time.

[0079] In other embodiments, after acquiring N screen images of the industrial equipment, image preprocessing operations can be performed on each of the N screen images to obtain N processed screen images. These image preprocessing operations include at least one of the following: grayscale conversion, used to convert color screen images to grayscale images to reduce computational load; binarization, used to convert screen images to pure black and white images to highlight the contrast between text and background; denoising, used to remove noise (such as white spots caused by dust) from the screen images; contrast enhancement, used to adjust the black and white contrast to make the text clearer; and image region of interest cropping, used to enhance the quality of the screen images, and to crop out only the text region containing the text if the screen image also contains other irrelevant parts (such as buttons or indicator lights). Through this series of image preprocessing operations, the accuracy of text recognition in the screen images can be improved.

[0080] In some implementations, raw messages at the same time can be obtained from the communication interface of the industrial equipment while the screen images are being acquired at time intervals.

[0081] S202: Recognize the i-th screen image among the above N screen images to obtain at least one character sequence in the i-th screen image, and generate corresponding labels according to each character sequence to form a label set, where i is an integer less than or equal to N.

[0082] In some implementations, the process of constructing the tag set can be performed via steps A1-A5:

[0083] Step A1: Perform text detection on the i-th screen image to obtain at least one text region in the i-th screen image that contains text.

[0084] In some implementations, scene text detection algorithms (such as differentiable binarization, deep learning algorithms that detect multiple frames in a single pass, etc.) can be used to perform text detection on the i-th screen image to obtain at least one text region (also called a text box) containing text in the i-th screen image.

[0085] Step A2: Correct the at least one text region according to the tilt of each text region to obtain the at least one corrected text region.

[0086] In some implementations, the tilt of each text region can be determined based on its position in the screen image. Based on the tilt of each text region, the tilted text regions can be rotated, aligned, etc., so that the corrected text regions meet the horizontal recognition conditions, thereby improving the accuracy of subsequent text recognition.

[0087] Step A3: Recognize at least one text region after the above correction to obtain at least one character sequence in the i-th screen image.

[0088] In some implementations, an OCR architecture can be used to recognize at least one of the corrected text regions. For example, taking a text region as an example, a convolutional neural network in the OCR architecture can be used to extract features from the text region to obtain the key text feature sequence in the text region (such as F = f). cnn (I), F=[f1,f2,…,f T ],f t ∈R d f t The length of the t-th text feature is represented by d, and the dimension of each text feature is represented by d. A bidirectional long short-term memory network is used to model the context of this text feature sequence to obtain a context-aware feature sequence (e.g., H = f). seq (F)=BiLSTM(F)=[h1,h2,…,h T ],h t ∈R d′ , where h t These are context-aware features at each time step, for example, h t When determining whether a vertical line is "1" or "l", the system references whether the preceding character is "F:" or "S:" to make a more accurate judgment. This solves the semantic understanding problems of "line of text", "field sequence", and "moving text". The context-aware feature sequence is then input into the decoding module to obtain the final character sequence.

[0089] Optionally, the decoding module can be a Connectionist Temporal Classification (CTC) module, where each time step h... t It will be mapped to a score vector of characters through a linear classifier, and then the score vector will be transformed into a probability distribution (e.g., ...) through an activation function. This is used to represent the probability of identifying a character as any character in the character set at time step t, where: Σ is the character set (e.g., the set of all numbers, letters, colons, etc.), +1 in |E|+1 represents a special whitespace character, W is the weight, and b is the bias parameter. Through this connection-based temporal classification, duplicate characters can be merged and whitespace characters removed, ultimately obtaining the most probable character sequence, thus improving the quality of the character sequence.

[0090] Step A4: Remove invalid characters from at least one character sequence according to the cleaning rules to obtain at least one target character sequence.

[0091] In some implementations, since the character sequence output by the OCR architecture may contain noise or inconsistent formatting, predefined cleaning rules (regular expression rules) can be used to clean the character sequence, thereby removing invalid characters (redundant characters, units, punctuation, etc.) to obtain the target character sequence.

[0092] Step A5: Generate a tag for each target character sequence based on the parameter type and parameter value of the device operating status parameters contained in each target character sequence, forming a tag set.

[0093] For example, taking a target character sequence as an example, the parameter types and values ​​of the device operating status parameters in the target character sequence are extracted. The parameter types are then used as the keys in key-value pairs, and the parameter values ​​are used as the values ​​in key-value pairs, forming a label in the form of a key-value pair structure. k i ,v i The parameter type and parameter value respectively represent the device operating status parameters in the i-th target character sequence.

[0094] Optionally, equipment operating status parameters include, but are not limited to: process variables (such as pressure parameters, flow parameters, temperature parameters, liquid level parameters, speed parameters, etc.), electrical parameters (such as current parameters, voltage parameters, power parameters, frequency parameters, etc.), equipment status (such as running, stopped, fault, alarm, etc.), and counting and statistics (such as output, running time, power consumption, etc.).

[0095] In some implementations, the tags in the above key-value pair structure can also be encapsulated into JSON format and used as supervisory tags for the original message to be input into the protocol parsing model, supporting subsequent prediction of the parameter types and values ​​of device operating status parameters in unknown protocols.

[0096] Based on steps A1-A5 above, Figure 3 This is a logical diagram illustrating a tag generation method provided in an embodiment of this application. Figure 3 In the middle, the image acquisition device 30 acquires data from industrial equipment (such as...) Figure 1 The screen image of the industrial equipment 10 shown is obtained; image preprocessing is performed on the screen image to obtain the processed screen image; the processed screen image is recognized using an OCR architecture, and the recognized character sequence is cleaned to obtain the target character sequence; the key-value pairs of the equipment operating status parameters in the target character sequence are extracted to generate a label in the form of a key-value pair structure.

[0097] Through the above steps A1-A5, combined with the OCR architecture, the parameter types and values ​​of the device status operation parameters in the screen image are extracted and constructed as tags for the original message at the same time. This eliminates the need for manual annotation, significantly reduces annotation costs, and improves the level of automation in annotation.

[0098] S203: Parse the i-th original message among the N original messages to obtain the byte sequence of the i-th original message. The i-th screen image and the i-th original message were collected in the same time period.

[0099] In this step, a matching communication protocol can be used to parse the original message to obtain the byte sequence of the original message. The aforementioned same time period can include the same moment, or it can include acquiring the corresponding original message at intervals of a few milliseconds (or a few seconds) after acquiring the screen image at a certain moment, or acquiring the corresponding screen image at intervals of a few milliseconds (or a few seconds) after acquiring the original message at a certain moment.

[0100] S204: Pair the tag set extracted from the i-th screen image with the byte sequence parsed from the i-th original message to obtain the training sample pair corresponding to the i-th screen image and the i-th original message.

[0101] In this step, the tag sets extracted from other screen images at the same time are paired with the byte sequences parsed from the original messages at the same time to obtain training sample pairs corresponding to other screen images at the same time and the original messages. This results in a large number of training sample pairs, which together constitute the training sample set (e.g., ...). Among them, B i =[b1,b2,…,b n ] is the byte sequence of the i-th original message. It is the j-th tag corresponding to the i-th byte sequence.

[0102] S205: Input the training samples corresponding to N screen images and N original messages into the initial neural network model to train the model and obtain the protocol parsing model. This protocol parsing model is used to parse the unknown protocol to be parsed and output the protocol parsing result of the unknown protocol.

[0103] In some implementations, the training process of the protocol parsing model can be performed through the following steps B1-B5:

[0104] Step B1: Perform dimensionality reduction on the multiple bytes contained in each byte sequence to obtain the embedding vector sequences of each of the N byte sequences.

[0105] For example, a word vector model from natural language processing can be used to analyze the byte order of each character (e.g., B = [b1, b2, ..., b...)). N ],bi ∈{0,1,…,255}, where b i The values ​​of the i-th byte (ranging from 0 to 255) are used to perform dimensionality reduction on the multiple bytes contained in the original message. Each byte is mapped to a d-dimensional dense vector, resulting in the embedding vector sequences of the N byte sequences (e.g., X = f). embed (B) = [x1, x2, ..., x N ],x i ∈R d , where x i The embedding vector represents the i-th byte, and d represents the dimension of the embedding vector (e.g., 64, 128, etc.). This process preserves the contextual information and semantic distribution characteristics of the bytes.

[0106] Step B2: Encode the features of multiple embedding vectors in the embedding vector sequence of each byte sequence to obtain the byte feature sequences of each of the N byte sequences.

[0107] For example, the embedding vector sequence of each byte sequence can be input into a Transformer network. The multi-head self-attention mechanism of this network can capture the long-distance dependencies of device state operating parameters (also known as fields) across byte sequences, thereby obtaining the byte feature sequences of each of the N byte sequences (e.g., H = f). trans (X)=[h1,h2,…,h N ],h i ∈R d′ , where h i d represents the byte characteristics of the i-th byte in the context. ′ (The vector dimension representing the byte features output). This process highlights the richer semantic expression of the byte.

[0108] Step B3: Predict each byte feature sequence to obtain a prediction result set for each of the N byte sequences; wherein, each prediction result set contains at least one device operating status parameter's parameter prediction type and parameter prediction value, and the start and end positions of each device operating status parameter in the byte sequence.

[0109] For example, a classifier (such as a span-based structure) can be used to model the byte feature sequence encoded by a Transformer network at the field level, enabling field position prediction (the starting position of the device operating status parameter in the byte sequence), field type classification (the predicted parameter type of the device operating status parameter), and field value regression (the predicted parameter value of the device operating status parameter). Optionally, field position prediction can be achieved using span... s,e =H[s:e] indicates that field type classification can be achieved through... This indicates that field value regression can be achieved through... This indicates that H[s:e] represents the start and end positions of a device's operating status parameter in the byte sequence, and W... cls The weight parameters b represent the pooling layer. cls The bias parameters of the pooling layer are represented by pool(·), and pool(·) represents the function of the pooling layer; b reg The bias parameter W of the regression layer reg The weight parameters characterize the regression layer.

[0110] After the above process, the final set of predicted byte sequences is obtained. in, This represents the start and end positions of the i-th device operating status parameter in any byte sequence. The parameter prediction type characterizing the operating state parameter of the i-th device. The predicted value of the parameter representing the operating status parameter of the i-th device. For example, the 20th to 23rd bytes in the byte sequence are the "temperature" field (device operating status parameter), and the parameter prediction type predicted by the protocol parsing model is temperature, with a predicted value of 37.5℃.

[0111] Step B4: Calculate the loss value of the protocol parsing model based on the prediction result set of each of the N byte sequences and the label set of each of the N byte sequences.

[0112] Optionally, the loss value satisfies the following expression:

[0113]

[0114] in, s represents the start and end positions of the i-th device operating status parameter in any byte sequence. i ,e i The actual start and end positions of the i-th device's operating status parameters in the byte sequence (available from open-source documentation) are represented. The parameter prediction type characterizing the operating state parameter of the i-th device, k i The parameter type that represents the operating status parameter of the i-th device in the tag set corresponding to the byte sequence. The predicted value of the parameter representing the operating state parameter of the i-th device, v i The parameter value representing the operating status parameter of the i-th device in the tag set corresponding to the byte sequence.

[0115] Step B5: When the loss value does not meet the threshold, update the parameters of the protocol parsing model until the loss value of the updated protocol parsing model meets the threshold.

[0116] In some implementations, the weight parameters of different layers in the protocol parsing model can be updated to improve the generalization ability of the protocol model.

[0117] Based on steps B1-B5 above, Figure 4 This is a logical diagram illustrating byte sequence parsing as provided in an embodiment of this application. Figure 4 In this process, the byte sequence is input into the embedding layer (such as the Byte2Vec layer), and the output is the embedding vector sequence; the embedding vector sequence is input into the Transformer network, and the output is the byte feature sequence; the byte feature sequence is input into the span-based structure pair for prediction, and the output is the prediction result set.

[0118] Through the above steps B1-B5, the alignment results of the labels and byte sequences identified by the screen images at the same time are used to construct training samples to form end-to-end semantic associations; these are then used to train a protocol parsing model to complete end-to-end prediction and structural reconstruction of field positions, field types, and field values ​​in the protocol, thereby improving the ability to parse protocols of unknown protocols in the future.

[0119] In other implementations, after obtaining the trained protocol parsing model, the currently acquired unknown protocol (also known as unknown message) can be input into the protocol parsing model to predict the protocol parsing result of the unknown protocol. The protocol parsing result includes the start and end positions of each device operating status parameter in the unknown protocol, the parameter type and parameter value corresponding to each device operating status parameter, thereby realizing the reconstruction and semantic parsing of the unknown protocol, and thus supporting the automatic identification, parsing and adaptation of various protocols without the need for open source documentation provided by the manufacturer.

[0120] Based on the above Figure 1 The protocol parsing method shown is as follows: Figure 5 This is a logical diagram illustrating the training and application of a protocol parsing model, as provided in an embodiment of this application. Figure 5In this process, according to a set cycle, image acquisition device 30 captures images of the screen of industrial equipment, obtains screen images, and uses an OCR architecture to perform visual recognition on the screen images to obtain the corresponding tag set. At the same time, the terminal obtains the original messages of the industrial equipment through a serial port or serial port, parses them to obtain byte sequences, pairs the byte sequences with the tag set to form training sample pairs, inputs the formed training sample pairs into the initial neural network model for training, and generates a protocol parsing model. Subsequently, when an unknown protocol to be parsed is captured from the industrial equipment, the unknown protocol is input into the protocol parsing model for inference and prediction, and the protocol parsing result of the unknown protocol is output.

[0121] It should be noted that the above protocol parsing methods include, but are not limited to, applications in the following scenarios: industrial data acquisition, protocol gateway development, edge intelligent operation and maintenance, etc., which can effectively improve the adaptability and intelligence level of industrial networks.

[0122] Based on the same technical concept, this application also provides a protocol parsing device, which can implement the protocol parsing method flow described above in this application.

[0123] Figure 6 This is a schematic diagram of a protocol parsing device provided in an embodiment of this application. Figure 6 The device includes a data acquisition module 601, an identification module 602, a parsing module 603, a training sample construction module 604, and a model training module 605.

[0124] The acquisition module 601 is used to acquire N screen images and N raw messages from industrial equipment; wherein N is an integer greater than 0.

[0125] The recognition module 602 is used to recognize the i-th screen image among the N screen images, obtain at least one character sequence in the i-th screen image, and generate a corresponding label according to each character sequence to form a label set; wherein, i is an integer less than or equal to N.

[0126] The parsing module 603 is used to parse the i-th original message among N original messages to obtain the byte sequence of the i-th original message; wherein the i-th screen image and the i-th original message are collected in the same time period.

[0127] The training sample construction module 604 is used to pair the tag set and the byte sequence to obtain the training sample pair corresponding to the i-th screen image and the i-th original message.

[0128] The model training module 605 is used to input the training samples corresponding to the N screen images and N original messages into the initial neural network model for model training to obtain a protocol parsing model; wherein, the protocol parsing model is used to parse the input unknown protocol to be parsed and output the protocol parsing result of the unknown protocol.

[0129] In some embodiments, the identification module 602 is specifically used for:

[0130] Text detection is performed on the i-th screen image to obtain at least one text region containing text in the i-th screen image; the at least one text region is corrected according to the tilt of each text region to obtain at least one corrected text region; the at least one corrected text region is identified to obtain at least one character sequence in the i-th screen image; invalid characters in the at least one character sequence are removed according to cleaning rules to obtain at least one target character sequence; a label corresponding to the first target character sequence is generated according to the parameter type and parameter value of the device operating status parameter contained in the first target character sequence; wherein, the first target character sequence is any one of the at least one target character sequence.

[0131] It should be noted that the apparatus provided in this application embodiment can implement all the method steps in the above method embodiment and achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0132] Based on the same technical concept, this application also provides an electronic device that can realize the function of the aforementioned protocol parsing device.

[0133] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0134] At least one processor 701 and a memory 702 connected to at least one processor 701. In this embodiment, the specific connection medium between the processor 701 and the memory 702 is not limited. Figure 7 The example shown is the connection between processor 701 and memory 702 via bus 700. Bus 700 is... Figure 7 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. The 700 bus can be divided into address bus, data bus, control bus, etc., for ease of representation. Figure 7 The term is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, the processor 701 can also be called a controller; there is no restriction on the name.

[0135] In this embodiment, memory 702 stores instructions executable by at least one processor 701. By executing the instructions stored in memory 702, at least one processor 701 can perform a data processing method as described above. Processor 701 can implement... Figure 6 The functions of each module in the device shown.

[0136] The processor 701 is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory 702 and calling data stored in memory 702, the processor can perform various functions and process data, thereby monitoring the device as a whole.

[0137] In this embodiment, processor 701 may include one or more processing units. Processor 701 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into processor 701. In some embodiments, processor 701 and memory 702 may be implemented on the same chip; in some embodiments, they may be implemented separately on independent chips.

[0138] The processor 701 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of a protocol parsing method disclosed in the embodiments of this application can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0139] Memory 702, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 702 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 702 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 702 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0140] By designing and programming the processor 701, the code corresponding to one of the protocol parsing methods described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute it during operation. Figure 2 The illustrated embodiment presents a protocol parsing method. How to design and program the processor 701 is a technique well-known to those skilled in the art and will not be described further here.

[0141] It should be noted that the electronic device provided in this application embodiment can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Here, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail.

[0142] Based on the same technical concept, embodiments of this application provide a computer storage medium, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the protocol parsing methods discussed above. Since the principle by which the above-described computer storage medium solves the problem is similar to that of a protocol parsing method, the implementation of the above-described computer storage medium can be referred to the implementation of the method, and repeated details will not be elaborated further.

[0143] In specific implementation, computer storage media can include: Universal Serial Bus Flash Drive (USB), portable hard drive, Read-Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, and other storage media that can store program code.

[0144] Based on the same technical concept, this application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the protocol parsing methods discussed above. Since the principle by which the above computer program product solves the problem is similar to that of a protocol parsing method, the implementation of the above computer program product can be referred to the implementation of the method, and repeated details will not be described again.

[0145] Computer program products may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0146] The methods in this application can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in this application are performed, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, network equipment, user equipment, core network equipment, OAM, or other programmable devices.

[0147] The computer program or instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions may be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; or an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0148] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0149] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that specifies the functions in one or more boxes. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including the instruction device, which is implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0150] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0151] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if these modifications and variations of the present invention fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A protocol parsing method, characterized in that, include: Collect N screen images and N raw messages from industrial equipment; where N is an integer greater than 0. The i-th screen image among the N screen images is identified to obtain at least one character sequence in the i-th screen image, and a corresponding label is generated based on each character sequence to form a label set; wherein i is an integer less than or equal to N; The i-th original message among N original messages is parsed to obtain the byte sequence of the i-th original message; wherein the i-th screen image and the i-th original message are collected in the same time period; Pair the tag set and the byte sequence to obtain the training sample pair corresponding to the i-th screen image and the i-th original message; The training samples corresponding to the N screen images and N original messages are input into the initial neural network model for model training to obtain the protocol parsing model; wherein, the protocol parsing model is used to parse the input unknown protocol to be parsed and output the protocol parsing result of the unknown protocol.

2. The method as described in claim 1, characterized in that, The step involves recognizing the i-th screen image among the N screen images to obtain at least one character sequence in the i-th screen image, and generating a corresponding label based on each character sequence to form a label set, including: Perform text detection on the i-th screen image to obtain at least one text region in the i-th screen image that contains text; Based on the tilt of each text region, the at least one text region is corrected to obtain the at least one corrected text region; The corrected text region is identified to obtain at least one character sequence in the i-th screen image; Invalid characters in the at least one character sequence are removed according to the cleaning rules to obtain at least one target character sequence; Based on the parameter type and parameter value of the device operating status parameters contained in each target character sequence, a tag corresponding to each target character sequence is generated, forming the tag set.

3. The method as described in claim 1, characterized in that, The step of inputting the training samples corresponding to the N screen images and N original messages into the initial neural network model for model training to obtain the protocol parsing model includes: Dimensionality reduction is performed on the multiple bytes contained in each byte sequence to obtain the embedding vector sequences of N byte sequences respectively; Feature encoding is performed on multiple embedding vectors in the embedding vector sequence of each byte sequence to obtain the byte feature sequences of each of the N byte sequences; For each byte feature sequence, a prediction result set is obtained for each of the N byte sequences; wherein, each of the prediction result sets contains at least one device operating state parameter's parameter prediction type and parameter prediction value, and the start position and end position of each device operating state parameter in the byte sequence; The loss value of the protocol parsing model is calculated based on the prediction result set of each of the N byte sequences and the label set of each of the N byte sequences. When the loss value does not meet the threshold, the parameters of the protocol parsing model are updated until the loss value of the updated protocol parsing model meets the threshold.

4. The method as described in claim 3, characterized in that, The loss value satisfies the following expression: Among them, the The s represents the start and end positions of the i-th device operating status parameter in any byte sequence within the byte sequence. i ,e i The actual start and actual end positions of the i-th device operating status parameter in the byte sequence are characterized. The parameter prediction type characterizing the operating state parameter of the i-th device, the k i The parameter type representing the operating status parameter of the i-th device in the tag set corresponding to the byte sequence, the The predicted value of the parameter characterizing the operating state parameter of the i-th device, the v i The parameter value representing the operating status parameter of the i-th device in the tag set corresponding to the byte sequence.

5. The method as described in claim 1, characterized in that, After inputting the training samples corresponding to the N screen images and N original messages into the initial neural network model for model training to obtain the protocol parsing model, the process further includes: The currently acquired unknown protocol is input into the protocol parsing model to predict the protocol parsing result of the unknown protocol; wherein, the protocol parsing result includes the start and end positions of each device operating status parameter in the unknown protocol, and the parameter type and parameter value corresponding to each device operating status parameter.

6. The method according to any one of claims 1-5, characterized in that, After acquiring N screen images from the industrial equipment, the process also includes: Image preprocessing operations are performed on the N screen images respectively to obtain N processed screen images; wherein, the image preprocessing operations include at least one of the following: grayscale processing, binarization processing, noise reduction processing, contrast enhancement processing, and image region of interest cropping processing.

7. A protocol parsing apparatus, characterized in that, include: The acquisition module is used to acquire N screen images and N raw messages from industrial equipment; wherein N is an integer greater than 0. The recognition module is used to recognize the i-th screen image among the N screen images, obtain at least one character sequence in the i-th screen image, and generate a corresponding label based on each character sequence to form a label set; wherein, i is an integer less than or equal to N; The parsing module is used to parse the i-th original message among N original messages to obtain the byte sequence of the i-th original message; wherein the i-th screen image and the i-th original message are acquired in the same time period; The training sample construction module is used to pair the tag set and the byte sequence to obtain the training sample pair corresponding to the i-th screen image and the i-th original message; The model training module is used to input the training samples corresponding to the N screen images and N original messages into the initial neural network model for model training to obtain the protocol parsing model; wherein, the protocol parsing model is used to parse the input unknown protocol to be parsed and output the protocol parsing result of the unknown protocol.

8. The apparatus as claimed in claim 7, characterized in that, The identification module is specifically used for: Perform text detection on the i-th screen image to obtain at least one text region in the i-th screen image that contains text; Based on the tilt of each text region, the at least one text region is corrected to obtain the at least one corrected text region; Identify at least one corrected local region to obtain at least one character sequence in the i-th screen image; Invalid characters in the at least one character sequence are removed according to the cleaning rules to obtain at least one target character sequence; A tag corresponding to the first target character sequence is generated based on the parameter type and parameter value of the device operating status parameter contained in the first target character sequence; wherein, the first target character sequence is any one of the at least one target character sequence.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a computer program stored in the memory, implements the method of any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1-6.

Images