Communication protocol analysis method and system, computer device and computer program product

Abstract

The application is suitable for the technical field of communication, and provides a communication protocol analysis method and system, computer equipment and computer program product, comprising: capturing an original message through a network interface and obtaining a numerical sequence by data preprocessing of the original message; inputting the numerical sequence into a pre-training model to obtain a protocol label, the protocol label having a corresponding confidence; performing syntax structure analysis on the original message based on the protocol label and the confidence corresponding to the protocol label to obtain structure information of the original message; and performing semantic information analysis on the original message based on the structure information to obtain semantic information of the original message. The application can realize efficient and accurate automatic identification of any original message and complete accurate analysis of the semantic information of the original message.

Description

Technical Field

[0001] This application belongs to the field of communication technology, and in particular relates to a communication protocol parsing method, system, computer equipment, and computer program product. Background Technology

[0002] Industrial control systems are widely used in key areas such as intelligent manufacturing, energy, and transportation. Their network environments are filled with a variety of industrial communication protocols, such as Modbus, PROFIBUS (Process Field Bus), PROFINET (Process Field Network), OPC UA (OLE for Process Control Unified Architecture), EtherNet / IP (Ethernet Industrial Protocol), and CANopen (Controller Area Network open). In addition, there are numerous proprietary or industry-customized protocols.

[0003] Current communication protocol parsing technologies heavily rely on expert experience. When new devices are added or protocols change, engineers must manually analyze protocol documents (if available) or reverse engineer them using packet capture tools to parse the message structure and write specific parsing scripts or drivers. This process is time-consuming, labor-intensive, and requires highly specialized personnel. Furthermore, once the protocol version is updated or the device is replaced, the original parsing logic may become invalid, requiring manual intervention and adjustment, making it difficult for the system to adapt. For unpublished proprietary protocols, there is a lack of effective automated parsing methods, severely hindering the transformation of legacy systems and the extraction of data value.

[0004] Therefore, there is an urgent need for a communication protocol parsing method to solve the problems existing in the current technology. Summary of the Invention

[0005] In view of this, embodiments of this application provide a communication protocol parsing method, system, computer device, and computer program product to achieve efficient and accurate automated identification of any original message and complete the precise parsing of the semantic information of the original message.

[0006] The first aspect of this application provides a communication protocol parsing method, including:

[0007] The original packets are captured through the network interface and the data preprocessed to obtain a numerical sequence.

[0008] The numerical sequence is input into a pre-trained model to obtain a protocol label, which has a corresponding confidence level.

[0009] Based on the protocol tag and the confidence level corresponding to the protocol tag, the original message is parsed to obtain the structural information of the original message;

[0010] The semantic information of the original message is obtained by parsing the original message based on the structural information.

[0011] In one implementation of the first aspect, the step of capturing the original packet through a network interface and performing data preprocessing on the original packet to obtain a numerical sequence includes:

[0012] The original packet physical signal is captured through the network interface and interpreted into a binary data stream. The physical signal is a level signal.

[0013] The binary data stream is converted into protocol data units according to the data link layer protocol.

[0014] By stripping the header information of the network layer and transport layer, the application layer data payload in the protocol data unit is extracted to obtain the application layer message;

[0015] A numerical sequence is generated based on the application layer message.

[0016] In one implementation of the first aspect, generating a numerical sequence based on the application layer message includes:

[0017] According to the temporal and spatial order of each byte in the application layer message, each byte of the application layer message is arranged sequentially to form a one-dimensional sequence;

[0018] A numerical sequence is obtained by numerically mapping each byte in the one-dimensional sequence, and the numerical sequence includes the integer corresponding to the byte;

[0019] The message association information of the numerical sequence is generated based on the session identifier, communication endpoint, and timing context of the application layer message.

[0020] In one implementation of the first aspect, the step of inputting the numerical sequence into a pre-trained model to obtain a protocol label, the protocol label having a corresponding confidence level, includes:

[0021] The numerical sequence is input into a pre-trained model to obtain the feature vector of the numerical sequence, and the feature vector is a mathematical representation of the overall pattern of the numerical sequence;

[0022] Load the prototype feature vector of a known protocol and calculate the target distance between the feature vector and the prototype feature vector;

[0023] The known protocol corresponding to the prototype feature vector with the smallest target distance is determined as the protocol label of the numerical sequence, and the confidence level of the protocol label is generated based on the target distance.

[0024] In one implementation of the first aspect, the step of parsing the original message based on the protocol tag and the confidence level corresponding to the protocol tag to obtain the structural information of the original message includes:

[0025] If the confidence level of the protocol tag is greater than the first preset confidence level, the communication protocol corresponding to the protocol tag is determined to be the communication protocol of the original message;

[0026] The original message is parsed using the communication protocol to obtain structural information.

[0027] If the confidence level of the protocol tag is less than the second preset confidence level, the original message is parsed to obtain structural information based on at least one of the value domain mutation information, byte statistical patterns, function code and length field in the numerical sequence.

[0028] In one implementation of the first aspect, when the confidence level of the protocol tag is greater than a first preset confidence level, the step of performing semantic information parsing on the original message based on the structural information to obtain the semantic information of the original message includes:

[0029] Load the semantic knowledge base according to the communication protocol;

[0030] Based on the semantic knowledge base, the function code, address field, and data field in the structural information are mapped to the semantic information of the original message;

[0031] When the confidence level of the protocol tag is less than the second preset confidence level, the step of parsing the semantic information of the original message based on the structural information to obtain the semantic information of the original message includes:

[0032] Based on the structural information, combined with prior knowledge of known protocols, the context of the original message, and statistical cluster analysis, at least one semantic inference information of the original message is generated, and each semantic inference information has a confidence level.

[0033] The semantic information of the original message is determined based on the confidence level of the semantic inference information.

[0034] In one implementation of the first aspect, the method further includes:

[0035] Map the structural information and the semantic information to the target data model;

[0036] Semantic actions are extracted from the structural information by combining the semantic information;

[0037] When the target protocol is a known protocol, the target data model is mapped and assembled according to the target protocol and the semantic action to obtain the target message under the target protocol;

[0038] When the target protocol is unknown, the target data model is mapped and assembled according to the configuration driver information and the semantic action to obtain the target message under the target protocol.

[0039] A second aspect of this application provides a communication protocol parsing system, including:

[0040] The data processing module is used to capture raw packets through a network interface and perform data preprocessing on the raw packets to obtain a numerical sequence.

[0041] The protocol label module is used to input the numerical sequence into the pre-trained model to obtain a protocol label, and the protocol label has a corresponding confidence level.

[0042] The structure parsing module is used to perform syntax structure parsing on the original message based on the protocol tag and the confidence level corresponding to the protocol tag to obtain the structure information of the original message;

[0043] The semantic parsing module is used to perform semantic information parsing on the original message based on the structural information to obtain the semantic information of the original message.

[0044] A third aspect of this application provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0045] A fourth aspect of this application provides a computer program product including a computer program that, when run, causes the method described in the first aspect to be executed.

[0046] The beneficial effects of the first aspect of the embodiments of this application are as follows: the original message is captured through the network interface and the data of the original message is preprocessed to obtain a numerical sequence. The numerical sequence is input into a pre-trained model to obtain a protocol label. The protocol label has a corresponding confidence level. Based on the protocol label and the confidence level of the protocol label, the original message is parsed to obtain the structural information of the original message. Based on the structural information, the original message is parsed to obtain the semantic information of the original message. This realizes efficient and accurate automatic identification of any original message and completes the accurate parsing of the semantic information of the original message.

[0047] It is understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram illustrating the implementation flow of the communication protocol parsing method provided in the embodiments of this application;

[0050] Figure 2 This is a schematic diagram illustrating the implementation flow of the communication protocol parsing method provided in the embodiments of this application;

[0051] Figure 3 This is a schematic diagram illustrating the implementation flow of the communication protocol parsing method provided in the embodiments of this application;

[0052] Figure 4 This is a schematic diagram illustrating the implementation flow of the communication protocol parsing method provided in the embodiments of this application;

[0053] Figure 5 This is a schematic diagram illustrating the implementation flow of the communication protocol parsing method provided in the embodiments of this application;

[0054] Figure 6 This is a schematic diagram of the communication protocol parsing system provided in an embodiment of this application;

[0055] Figure 7 This is a schematic diagram of the computer device provided in an embodiment of this application;

[0056] Figure 8 This is a schematic diagram of a computer program product provided in an embodiment of this application. Detailed Implementation

[0057] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0058] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0059] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0060] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0061] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0062] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0063] This application provides a communication protocol parsing method for efficiently and accurately identifying the communication protocol of an original message and parsing it to obtain the semantic information of the original message. The method captures the original message through a network interface and preprocesses it to obtain a numerical sequence. This numerical sequence is then input into a pre-trained model to obtain a protocol label, which has a corresponding confidence level. Based on the protocol label and its corresponding confidence level, the method performs syntactic structure parsing on the original message to obtain its structural information. Finally, based on the structural information, the method performs semantic information parsing on the original message to obtain its semantic information. This achieves efficient and accurate automated identification of any original message and precise parsing of its semantic information.

[0064] The communication protocol parsing method provided in this application can be applied to computer devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), desktop computers, and cloud servers. This application does not impose any restrictions on the specific type of computer device.

[0065] like Figure 1 The present application provides a communication protocol parsing method, including:

[0066] Step S1: Capture the original message through the network interface and perform data preprocessing on the original message to obtain a numerical sequence.

[0067] In the application, raw messages from the industrial communication field are captured through the network interface, and the communication protocol of the raw messages is the industrial communication protocol.

[0068] In applications, the core objective of data preprocessing is to losslessly and structurally elevate the physical layer's binary bitstream into semantic information understandable by the application layer, such as application layer messages. This process follows the layered model of communication systems, but the processing logic is reversed. Numerical sequence generation is a crucial step in transforming application layer messages into model inputs. Its core logic is to abandon any semantic parsing based on prior knowledge and adopt the lowest-level, universal numerical representation.

[0069] Step S2: Input the numerical sequence into the pre-trained model to obtain a protocol label, which has a corresponding confidence level.

[0070] In the application, the preprocessed numerical sequence is input into a pre-trained AI model to obtain at least one protocol label. Each protocol label has a corresponding confidence level, which represents the degree of similarity between the protocol label and the actual communication protocol of the original message.

[0071] In applications, protocol tags include protocol type and version information. The identification result of the protocol tag acts as a "master switch" or "routing table" for the entire parsing process, determining subsequent resource allocation and parsing strategies. For example, guiding syntax parsing: high-confidence protocol tags (greater than 90%): the system loads the corresponding protocol parsing template. For instance, after identifying it as Modbus TCP, the model focuses on the precise parsing of specific fields such as the MBAP header, transaction ID, and unit identifier, greatly improving the accuracy and efficiency of parsing.

[0072] Low-confidence identification (less than 50%) triggers "exploration mode." The model cannot rely on strong prior knowledge and must utilize its general sequence understanding capabilities to analyze the message structure from scratch. This consumes more computational resources and has relatively lower accuracy, but it is an essential capability for the system to handle unknown protocols. Semantic mapping is determined: the protocol type is directly associated with the semantic knowledge base. For example, if PROFIBUS is identified, the system will retrieve the PROFIBUS function code dictionary and device address mapping table from the knowledge base, transforming the parsed raw values ​​into business semantics such as "starting the motor" or "overheating." The protocol tag identification result determines the standardized output data model format and the rule set for protocol conversion. Different target systems (such as SCADA and MES) may require different data formats; the protocol type is crucial for selecting the correct format.

[0073] In application, a low match rate (e.g., less than 50%) is not a failure, but a high-value diagnostic signal. Its engineering significance lies in triggering a tiered processing flow: high confidence (>85%): automatic processing, with the result directly entering the business system; medium confidence (50%-85%): the result is marked as "pending verification," enters the manual review queue, and the model's original output is recorded simultaneously. This significantly narrows the scope of manual investigation, reducing it from "massive amounts of unknown data" to "a small amount of suspicious data."

[0074] Low confidence (<50%): Triggers a "New Protocol Discovery" alert. The system automatically archives these messages and their context (session pairs) for expert analysis, continuously expanding the model's training data. It provides a starting point for analysis; even with only 50% confidence, the model's output of the "most likely protocol" provides valuable clues. Engineers can prioritize comparative analysis starting with this protocol family (e.g., it might match "some TCP-based binary protocol") instead of starting from scratch. It demonstrates "honesty": a method that reports low confidence is far more reliable than a system that "blindly trusts" and gives incorrect results. It establishes a foundation of trust for human-machine collaboration.

[0075] Step S3: Based on the protocol tag and the confidence level corresponding to the protocol tag, perform syntax structure parsing on the original message to obtain the structural information of the original message.

[0076] The application parses the raw message byte-by-byte or field-by-field. If the confidence level is greater than the field-by-field threshold (e.g., 90%), it enters the "field-by-field parsing" fast track. The protocol template is loaded, and key information is extracted directly. If the confidence level is below the byte-by-byte threshold, it enters the "byte-by-byte analysis" deep exploration track. The model activates its sequence labeling capabilities to analyze the message structure from scratch.

[0077] In application, even in "field-by-field parsing" mode, if a parsed value is clearly unreasonable (e.g., the length field value exceeds the total message length), it will automatically downgrade to "byte-by-byte analysis" mode for re-examination of that area to ensure robustness. Field lengths are not fixed and are determined by protocol definition or inferred through pattern analysis. The confidence level of the protocol tag is a "decision signal" for selecting a parsing strategy, not the sole basis for field length. "Byte-by-byte" parsing offers a general capability to handle the unknown and discover structure; "field-by-field" parsing utilizes known information for efficient extraction.

[0078] The inventiveness of the method provided in this application lies in the fact that when no protocol template is available (i.e., the "field-by-field" path is blocked), the system can still complete the parsing task through "byte-by-byte" intelligent analysis, thereby achieving adaptive communication protocol parsing.

[0079] Step S4: Based on the structural information, perform semantic information parsing on the original message to obtain the semantic information of the original message.

[0080] like Figure 2 As shown, in one embodiment, step S1, which involves capturing the original packet through a network interface and performing data preprocessing on the original packet to obtain a numerical sequence, includes:

[0081] Step S11: Capture the original packet physical signal through the network interface and interpret the physical signal as a binary data stream. The physical signal is a level signal.

[0082] In applications, network interface cards (NICs) or serial devices perform analog-to-digital conversion, interpreting level signals (such as differential signals in Ethernet or voltage differences in RS-485) into raw binary bit streams. The above process focuses on bit synchronization and frame delimitation (such as Ethernet preambles and start-of-frame delimiters).

[0083] Step S12: Convert the binary data stream into protocol data units according to the data link layer protocol.

[0084] In applications, the frame header and trailer are parsed according to data link layer protocols (such as Ethernet frames and serial frame formats), and the frame check sequence is verified. After stripping the frame encapsulation, the network / transport layer PDUs (Protocol Data Units) are obtained, such as IP (Internet Protocol) packets and Modbus RTU (Remote Terminal Unit) ADUs (Application Data Units). The above process ensures the integrity and correctness of the frame.

[0085] Step S13: By stripping the header information of the network layer and transport layer, the application layer data payload in the protocol data unit is extracted to obtain the application layer message.

[0086] In application, the header information of the network layer (IP header) and transport layer (TCP / UDP header) is further stripped away to extract the application layer payload. TCP stands for Transmission Control Protocol; UDP stands for User Datagram Protocol. For example, for Modbus TCP, it is the PDU after removing the MBAP (Modbus Application Protocol) header; for serial protocols, it is the data field after removing the address and check fields. After the above process, a pure application layer message representing industrial operation instructions or data is obtained.

[0087] Step S14: Generate a numerical sequence based on the application layer message.

[0088] In traditional applications, this requires prior knowledge of the protocol specifications for parsing. This application, however, directly feeds the unparsed application layer message into a large AI model, allowing the model to "guess" its internal structure. This application achieves a general numerical representation at the underlying level by converting the application layer message into a numerical sequence.

[0089] like Figure 3 As shown, in one embodiment, step S14, generating a numerical sequence based on the application layer message, includes:

[0090] Step S141: Arrange each byte of the application layer message into a one-dimensional sequence according to the temporal and spatial order of each byte in the application layer message.

[0091] In this application, bytes are chosen as the atomic unit. Bytes are the most basic and universal addressing unit in computer systems, and the de facto foundation of all protocols. Choosing bytes as the basic processing unit, rather than bits or multi-byte fields, ensures the universal applicability of the method provided in this application. Then, each byte is arranged sequentially according to the temporal and spatial order of the message's appearance on the network, forming a one-dimensional sequence. This sequence accurately records all the original information of the message, including potential protocol headers, data, padding, etc., without any information loss. For example, a message "A5 5A 0701 02 03 04 B3 C4" is directly converted into the sequence [165, 90, 7, 1, 2, 3, 4, 179, 196].

[0092] Step S142: Perform numerical mapping on each byte in the one-dimensional sequence to obtain a numerical sequence, wherein the numerical sequence includes the integer corresponding to the byte.

[0093] In the application, each byte (an 8-bit unsigned integer) is mapped to an integer space of 0-255. This mapping is deterministic and one-to-one. The pre-trained model is able to learn the association between specific numerical patterns (such as 165 and 90 appearing consecutively, possibly representing a frame header) and protocol syntax within this integer sequence.

[0094] In applications, the model requires input with fixed dimensions, but the message length is variable. Therefore, smart padding is used for short messages. Smart padding is not simply filling in 0s, but using a special value that rarely appears in normal messages (such as 0xFF or 0x00) as the padding character, and attaching a valid length identifier to the input to tell the model which are real data and which are padding.

[0095] For long messages, truncation is performed, preserving key regions. Analysis shows that critical protocol information (such as function codes and addresses) is usually located at the beginning of the message. Therefore, the tail data is truncated first. Simultaneously, the model can be trained to handle "segmented input," dividing long messages into multiple segments for sequential processing.

[0096] Step S143: Generate message association information of the numerical sequence based on the session identifier, communication endpoint and timing context of the application layer message.

[0097] The purpose of message association is to reconstruct the context of communication, as information from a single request or response message is incomplete. The core principle of association is to match identifiers and network 5-tuples within a time window. The three elements of association include session identifier, communication endpoint, and timing context.

[0098] Session identifiers are fields extracted from each message that uniquely identify a "transaction." For example, ModbusTCP uses the transaction identifier in the MBAP header. The same request and response have the same transaction ID. General TCP protocols use TCP sequence numbers and acknowledgment numbers. The acknowledgment number of a response message equals the sequence number of the request message plus the length of the requested data. Custom protocols may include auto-incrementing message IDs or sequence numbers.

[0099] In applications, network layer communication endpoint information is used to segment different data streams. A communication endpoint consists of a network 5-tuple: {source IP, source port, destination IP, destination port, transport layer protocol}. All messages in a session should have the same 5-tuple (the source / destination of requests and responses should be reversed). This process ensures that communications between different devices are not incorrectly associated.

[0100] In applications, timing context refers to requests and responses that are sequential in time. For example, by setting a reasonable timeout window (e.g., 1-5 seconds), after capturing a request message, the system will search for a matching response within that timeout window from messages with the same 5-tuple (but with the source / destination swapped). If no response is found after the timeout, it is determined that either the request or the response was lost.

[0101] In application, the specific process involves message classification, where captured messages are routed to different "virtual channels" based on the 5-tuple. Request detection, within a channel, identifies request messages based on protocol characteristics (such as known port numbers, specific function code patterns) or heuristic rules (such as the message initiating the connection is usually a request). Identifier extraction extracts session identifiers (such as transaction IDs) from request messages. Response matching searches for messages in subsequent arrivals that meet the following conditions: belong to the same channel (5-tuple swaps), contain a session identifier that matches the request message, and arrive after the request and before timeout. Session construction pairs the matched request and response messages to form a session pair and assigns it a unique session ID. Session pairs can serve as a larger input unit for the model, allowing the model to learn the logical relationship between "questions and answers."

[0102] like Figure 4 As shown, in one embodiment, step S2 involves inputting the numerical sequence into a pre-trained model to obtain a protocol label, the protocol label having a corresponding confidence level, including:

[0103] Step S21: Input the numerical sequence into the pre-trained model to obtain the feature vector of the numerical sequence. The feature vector is a mathematical representation of the overall pattern of the numerical sequence.

[0104] In application, "overall pattern" is a comprehensive concept. It doesn't refer to a specific field, but rather a collection of multi-dimensional and multi-level statistical and structural characteristics presented by the message as a complete sequence. The model analyzes the following patterns in parallel: Macro-statistical characteristics: Message length distribution: The message length of some protocols is fixed or varies within a very small range (such as some PROFIBUS messages), while others are highly variable (such as OPC UA). Byte value distribution: Analyzing the entropy value of each byte position in the message. The bytes in the protocol header often have lower entropy values ​​(more fixed), while the bytes in the data area have higher entropy values ​​(more random). Structural characteristics: Magic number, identifying whether there is a specific, fixed byte sequence at the beginning of the message (such as 00 00 in Modbus TCP, although not a strict magic number, the pattern is fixed); Field alignment, analyzing whether the message exhibits a certain alignment pattern (such as 2-byte or 4-byte alignment), which suggests the data type that the protocol may use (16-bit integer, 32-bit floating-point number). Session interaction characteristics, request-response patterns: Analyzing whether there is a stable correspondence between the length and structure of request messages and the length and structure of response messages. For example, a short request always followed by a long response might indicate a "read data" operation. The model integrates these features from different dimensions into a high-dimensional, comprehensive feature vector for matching with known protocols.

[0105] Step S22: Load the prototype feature vector of the known protocol and calculate the target distance between the feature vector and the prototype feature vector.

[0106] The preprocessed numerical sequence [x1, x2, x3, ..., xn] is input into the model. The model uses its complex neural network layer (Transformer encoder) to transform this sequence into a fixed-length, high-dimensional, dense feature vector, which is the mathematical representation of the message's "overall pattern".

[0107] During model training, the model learns one or more prototype feature vectors for each protocol it has learned (such as Modbus TCP, OPC UA), which can be understood as anchor points for that type of protocol in the feature space. During inference, the "distance" between the feature vectors and each known protocol prototype vector is calculated, such as cosine similarity or Euclidean distance.

[0108] Step S23: Determine the known protocol corresponding to the prototype feature vector with the smallest target distance as the protocol label of the numerical sequence, and generate the confidence level of the protocol label based on the target distance.

[0109] These "distances" are transformed using a Softmax function, normalizing them into a probability distribution. The matching degree (e.g., 90%) is this probability value. It represents the probability that the current message belongs to the target protocol among all candidate protocols.

[0110] For example, suppose there are three protocols (A, B, C), and the original similarity scores of the current message with them are S_A=8, S_B=2, and S_C=0, respectively. After Softmax analysis: P_A≈0.9975, P_B≈0.0025, P_C≈0.0001. Protocol A is ultimately confirmed, with a matching degree of 99.75%.

[0111] In application, the known protocol corresponding to the prototype feature vector with the smallest target distance is determined as the protocol label of the numerical sequence, and the matching degree is used as the confidence degree of the protocol label.

[0112] In one embodiment, step S3, which involves parsing the original message based on the protocol tag and the confidence level corresponding to the protocol tag to obtain the structural information of the original message, includes:

[0113] Step S31: If the confidence level of the protocol tag is greater than the first preset confidence level, determine that the communication protocol corresponding to the protocol tag is the communication protocol of the original message.

[0114] In application, the first preset confidence level is 90% or other high values. If the confidence level of the protocol tag is greater than the first preset confidence level, it means that the feature vector of the current message is very close to the prototype vector of a certain protocol in the feature space, almost overlapping, and they belong to the same type of communication protocol.

[0115] Step S32: Based on the communication protocol, perform syntax structure parsing on the original message to obtain structural information.

[0116] In application, if the confidence level of a protocol tag exceeds a pre-set confidence level, the template for that protocol tag is retrieved from the knowledge base. This template explicitly specifies the name, starting position, length, and data type of each field. The model then knows that "starting from byte 0, the first two bytes are the transaction ID; the next two bytes are the protocol identifier..." In this case, parsing is performed directly field by field, resulting in the highest efficiency and accuracy.

[0117] In applications, structural information includes function codes, address fields, and data fields.

[0118] Step S33: If the confidence level of the protocol tag is less than the second preset confidence level, perform syntax structure parsing on the original message to obtain structural information based on at least one of the value domain mutation information, byte statistical regularity, function code and length field in the numerical sequence.

[0119] In applications, if the confidence level of the protocol tag is less than the second preset confidence level, it is impossible to obtain such a definite template. In this case, it is necessary to utilize its general field boundary recognition capability learned during pre-training and fine-tuning, which is "byte-by-byte analysis".

[0120] In applications, abrupt changes in value ranges—a significant difference between the values ​​of two adjacent bytes—may indicate a switch from a header field with fixed values ​​to a data field with greater variability. Statistical patterns: Some byte values ​​have low entropy (fixed or minimally variable values, such as function codes), while subsequent positions show a sudden increase in entropy (random values, such as data payloads). This point of entropy abrupt change may be the field boundary. Function code patterns: The model has learned that certain specific values ​​(such as 01, 03, 10) appearing at specific offsets in the message (usually near the beginning) have a high probability of being "function codes." After identifying the function code, the possible positions and sizes of subsequent fields (such as address and length) can be inferred based on common patterns. Length field: The model can identify fields in certain protocols used to declare the "length of the data portion." Once the value L of this length field is parsed, it can directly locate the boundary of the data field for the next L bytes.

[0121] In one embodiment, when the confidence level of the protocol tag is greater than a first preset confidence level, step S4, which involves semantically parsing the original message based on the structural information to obtain the semantic information of the original message, includes:

[0122] Step S41: Load the semantic knowledge base according to the communication protocol.

[0123] In the application, the corresponding "dictionary" is loaded from the structured semantic knowledge base according to the protocol type and version. Function code dictionary: This is a direct mapping table. For example, 03 -> Function: Read holding register, 06 -> Function: Write to a single register, 10 -> Function: Write to multiple registers. Address mapping table in the database maps logical addresses in the protocol to physical entities. For example, holding register address 0x0000 -> Device: Boiler 1, Variable: Temperature setpoint, Unit: °C; Coil address 0x0001 -> Device: Motor 2, Variable: Start / Stop status. Data type specification: Defines how the raw byte sequence is interpreted as a meaningful numerical value. For example, 2 bytes, big-endian, unsigned integer; 4 bytes, IEEE (Institute of Electrical and Electronics Engineers) 754 floating-point number.

[0124] Step S42: Map the function code, address field, and data field in the structural information to the semantic information of the original message according to the semantic knowledge base.

[0125] In the application, the parsed function codes are looked up in the function code dictionary to obtain a readable operation description. The parsed address fields are looked up in the address mapping table to obtain the actual device or variable represented by that address. The parsed data fields are decoded according to the data type specification to obtain the final value.

[0126] For example, in the Modbus TCP protocol, the output structure information includes function code: 0x10, starting address: 0x0000, number of registers: 0x0002, and data: 0x4248 0x0000 (4 bytes). The semantic parsing process is as follows: look up the function code dictionary: 0x10 -> "write multiple holding registers"; look up the address mapping table: 0x0000 -> "boiler 1 temperature setpoint"; decode by data type: data 0x42480000 is interpreted as IEEE 754 floating-point number -> 50.0, and the final output is: "Instruction: Set the temperature setpoint of boiler 1 to 50.0°C".

[0127] If the confidence level of the protocol tag is less than the second preset confidence level, step S4, which involves semantically parsing the original message based on the structural information to obtain the semantic information of the original message, includes:

[0128] Step S43: Based on the structural information, combined with prior knowledge of known protocols, the context of the original message, and statistical cluster analysis, at least one semantic inference information of the original message is generated, and each semantic inference information has a confidence level.

[0129] In applications where the confidence level of the protocol label is less than a second pre-set confidence level, and no readily available semantic knowledge base is available, semantic parsing is achieved through a multi-factor, interpretable reasoning process. This process is not a "black box" of "input equals output." Specifically, it utilizes pre-trained prior knowledge; the model has learned common patterns of hundreds of common protocols during the training phase. It knows that "a byte near the beginning of a message is likely a function code," "the next two bytes are likely an address," and "data fields are usually located in the latter part of the message." This prior knowledge allows the model to make educated guesses even when faced with unknown protocols. Contextual session analysis (the most crucial means): The model does not analyze a message in isolation. It always understands request and response messages as a pair. For example, it observes a short request message followed by a long response message containing a large amount of data.

[0130] Inferring, "This is likely a command to read data. A field in the request message specifies what to read, and the data field in the response message is the value read back." Observing a request message containing a piece of data, followed by a short response message indicating only success. Subsequently, another request to read the status returned changed data. Inferring, "This is likely a command to write data. The data in the request message is the value to be set." Statistical and cluster analysis: Long-term monitoring of network traffic, performing cluster analysis on the packets. For example, if a certain type of packet always appears periodically and the data field changes very little, it may be inferred as a "heartbeat" or "periodic status reporting." If a certain type of packet always appears immediately after a certain event (such as pressing a button), it may be inferred as a "control command."

[0131] In application, hypotheses and confidence scores are generated. Based on the above analysis, the model generates one or more semantic hypotheses. For example, "with a 75% confidence level, byte [1] is a function code with a value of 0xA0 representing 'read'; bytes [2-3] are the starting address 0x0100; bytes [4-5] are the data length." This confidence level will be dynamically adjusted as more of the same session patterns are observed. Example of an unknown protocol: Input session pair:

[0132] Request: [0xAA, 0xBB, 0x01, 0x00, 0x02]; Response: [0xAA, 0xBB, 0x41, 0x48, 0x00, 0x00] (assuming 0x41480000 is the byte representation of the floating-point number 50.0). The model inference process for this example refers to the following: Pattern matching: short request, long response and floating-point data -> high probability of "read operation"; Field association: 0x0100 (256) and 0x02 (2) in the request may represent "address 256" and "request 2 registers" respectively; Semantic generation: the model may output "suspected read operation: 2 registers were read from address 256, and a floating-point value of 50.0 (unit unknown) was obtained", with a confidence level, such as 70%.

[0133] Step S44: Determine the semantic information of the original message based on the confidence level of the semantic inference information.

[0134] In application, the semantic inference information with the highest confidence level is determined as the semantic information of the original message.

[0135] like Figure 5 As shown, in one embodiment, the method further includes:

[0136] Step S51: Map the structural information and the semantic information to the target data model.

[0137] In applications, the parsed structural and semantic information is mapped to a unified, standardized data model (e.g., based on JSON (JavaScript Object Notation), XML (Extensible Markup Language), or OPC UA information model). Structural information includes a list of message field boundaries, the raw byte value of each field, and the inferred role of each field (e.g., function code, starting address, data length, data payload, CRC (Cyclic Redundancy Check)). For example, [{starting byte: 0, length: 1, role: "function code", raw value: "0x10"}, {starting byte: 2, length: 2, role: "starting address", raw value: "0x0000"}, ...]. Semantic information includes operation type (e.g., read, write, heartbeat), target object (e.g., device ID, variable address, register number), data value (decoded numerical value, e.g., 50.0), physical meaning (e.g., "boiler temperature setpoint"), and unit (e.g., "°C"). For example, {Operation: "Write", Target: "Boiler 1 Temperature Setpoint", Value: 50.0, Unit: "°C"}.

[0138] In applications, structural and semantic information are combined with contextual information to represent a standard data structure. Contextual information includes session ID (associated request-response pair), timestamp, and communication direction (request / response), for example {session ID: "sess_12345", timestamp: "2023-10-27T10:00:00.000Z", direction: "request"}.

[0139] In application, for example, the structural and semantic information, combined with the above contextual information, can be represented as a JSON object as follows:

[0140] {

[0141] "metadata": {

[0142] "protocol": "Modbus_TCP"

[0143] "version": "1.0",

[0144] "confidence": 0.98

[0145] "session_id": "sess_12345",

[0146] "timestamp": "2023-10-27T10:00:00.000Z",

[0147] "direction": "request"

[0148] },

[0149] "syntax": {

[0150] "function_code": {"raw_value": "0x10", "role": "function_code"},

[0151] "starting_address": {"raw_value": "0x0000", "role": "address"},

[0152] "quantity": {"raw_value": "0x0002", "role": "data_length"},

[0153] "data": {"raw_value": "0x42480000", "role": "payload"}

[0154] },

[0155] "semantics": {

[0156] "operation": "WRITE_MULTIPLE_REGISTERS",

[0157] "target_device":"Boiler_1",

[0158] "target_variable": "Temperature_Setpoint",

[0159] "value": 50.0

[0160] "unit": "°C"

[0161] }}.

[0162] Step S52: Extract semantic actions from the structural information by combining the semantic information.

[0163] In applications, regardless of whether the source protocol is Modbus, PROFIBUS, or a proprietary protocol, its core business intent can be described using a set of common metaphrases, which extract the common semantic actions from the semantics section of the aforementioned "structured information." For example, the operation is: READ / WRITE / COMMAND / SUBSCRIBE; the target is: {device_id}.{variable_name} (e.g., "Boiler_1.Temperature_Setpoint"); and the value is: 50.0 (with type and unit information).

[0164] Step S53: If the target protocol is a known protocol, perform protocol mapping and assembly on the target data model according to the target protocol and the semantic action to obtain the target message under the target protocol.

[0165] In application, when the target protocol is a known protocol, a complete knowledge base of the target protocol (such as OPC UA) is known. Based on the target protocol and the semantic actions, the target data model is mapped and assembled to obtain the target message under the target protocol. The mapping process includes operation mapping: mapping the general operation WRITE to the OPC UA WriteRequest service call; address mapping: mapping the general target Boiler_1.Temperature_Setpoint to the OPC UA NodeId, for example, ns=1; s=Boiler1 / TempSetpoint; and data encoding: encoding the value 50.0 according to the standard OPC UA data type (such as Float).

[0166] Step S54: When the target protocol is unknown, the target data model is mapped and assembled according to the configuration driver information and the semantic action to obtain the target message under the target protocol.

[0167] In applications where the target protocol is unknown, a template predefined by the engineer for the target private protocol is used, which configures the driver information and specifies the format of its function code, address field, and data field. The conversion engine then becomes the filler for this template.

[0168] In applications, if the structure of a target protocol is learned online (by monitoring its traffic), it's possible to attempt to "rewrite" a message based on the learned structure and general semantics. For example, knowing that the target protocol's "write operation" function code is 0xA5 and its data field is a 4-byte floating-point number, such a message can be assembled. In this case, the output will have lower confidence and requires rigorous testing.

[0169] In applications, target messages are sent to target devices or networks.

[0170] In one embodiment, the detailed training process of the pre-trained model is as follows:

[0171] Phase 1: Data Preparation. Raw data collection and aggregation of multi-source data: public protocol specification libraries (Modbus, PROFIBUS, OPC UA, etc.); simulated data generator (covering all function codes and data type combinations); real industrial network traffic (anonymized); protocol variant data (enhancing model robustness). Data cleaning and preprocessing, including: noise filtering (removing non-industrial protocol traffic); invalid packet removal (length anomalies, checksum errors); session stream reassembly (based on 5-tuples and transaction IDs); byte serialization (uniformly converting to integer sequences). Fine-grained data annotation includes a multi-level annotation system, protocol type and version labels; byte-level field boundary annotation (BIO scheme); semantic role annotation (function codes, addresses, data, etc.); session context annotation (request-response pairing). Dataset partitioning, including a pre-training set: 1 billion+ unlabeled packets (for self-supervised learning); a fine-tuning set: 10 million+ labeled samples (for supervised training); and a test set: reserved data (for final evaluation).

[0172] Phase Two: Pre-training. Model architecture initialization, basic configuration, Transformer encoder architecture, byte-level vocabulary (256 tokens), multi-scale attention mechanism, relative position encoding. Pre-training task design: Masked byte prediction: In Python, input: [0xA5, 0x5A, [MASK], 0x01, 0x02, [MASK], 0x04], objective: predict the masked byte [0x07, 0x03], purpose: learn the protocol's local structure and dependencies. Next byte prediction: In Python, input: [0xA5, 0x5A, 0x07, 0x01, 0x02], objective: predict the next byte 0x03, purpose: learn the protocol state machine. Protocol contrast learning: In Python: positive samples: (request message, corresponding response message) negative samples: (Modbus message, PROFIBUS message), purpose: learn protocol discrimination ability. Syntax Tree Prediction: Constructing an abstract syntax tree for a known protocol to predict the tree structure corresponding to a message. Objective: To learn the hierarchical organization of protocols. Optimization and Convergence: Training parameters: Batch size: 1024-2048, Learning rate: Warm-up + linear decay, Optimizer: AdamW + gradient pruning. Convergence Criteria: Stable perplexity on the validation set, early stopping mechanism triggered when accuracy saturates on each task.

[0173] Phase 3: Supervised Fine-tuning, including a multi-task learning architecture: a shared backbone network, a pre-trained Transformer encoder, a dedicated output head, a protocol classification head (fully connected layer + Softmax), a field detection head (CRF (Conditional Random Fields) layer + BIO (Beginning, Inside, Outside, sequence labeling scheme) annotation), and a semantic parsing head (Multilayer Perceptron). Progressive training strategies: For basic protocols, focus on clearly structured protocols such as Modbus and PROFIBUS, emphasizing field boundary detection training. For complex protocols, introduce complex protocols such as OPCUA and EtherNet / IP to enhance semantic understanding. For proprietary protocols, utilize few-shot learning techniques and apply meta-learning frameworks. Learning schedule: from simple to complex: fixed-length → variable-length protocols; from known to unknown: standard protocols → protocol variants; from independent to interactive: single message → complete session.

[0174] Phase Four: Reinforcement Learning Optimization. The parsing environment is constructed, with a state space of current message sequence and parsing history; an action space consisting of field boundary prediction and semantic label assignment; and a reward function.

[0175] In Python:

[0176] defreward_function(parsing_result, ground_truth):

[0177] accuracy=calculate_overlap(parsing_result, ground_truth);

[0178] consistency=check_structure_consistency(parsing_result);

[0179] confidence=model_confidence_score(parsing_result);

[0180] return0.6*accuracy+0.3*consistency+0.1*confidence.

[0181] The PPO (Proximal Policy Optimization) algorithm training consists of a policy network (fine-tuned model), a value network (extra-trained critic network), and an update mechanism (proximal policy optimization to ensure training stability).

[0182] Phase Five: Continuous Learning. Online learning mechanism, real-time data stream: continuous monitoring of industrial network traffic;

[0183] Active learning: Low-confidence samples are prioritized for labeling; Incremental training: The model is updated periodically with new data. Catastrophic forgetting protection and resilient weight consolidation: Important parameters are regularized; Experience replay: Core sample buffer; Knowledge distillation: New and old models are trained together. Version management system and model snapshots: Every important version is saved; Performance monitoring: Real-time tracking of model performance; Rollback mechanism: Quick recovery when performance degrades.

[0184] Phase Six: Model Deployment. Model compression, knowledge distillation, teacher model: complete large model; student model: lightweight network; distillation loss: KL (Kullback-Leibler) divergence + task loss. Quantization optimization, precision conversion: FP32→INT8; calibration dataset: representative industrial traffic; quantization-aware training: simulated quantization effect. Inference optimization, engine integration: TensorRT, ONNXRuntime (Open Neural Network Exchange Runtime). Caching mechanism: common protocol template caching; batch processing: parallel parsing of multiple packets. Performance verification, accuracy metrics: protocol recognition accuracy >98%; field parsing F1 score >95%; end-to-end success rate >97%; efficiency metrics: single packet parsing latency <10ms; throughput >1000pps (Packets Per Second); memory usage <2GB.

[0185] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0186] This application also provides a communication protocol parsing system for performing the steps described in the communication protocol parsing method embodiments. The communication protocol parsing system can be a virtual appliance in a computer device, run by the computer device's processor, or it can be the computer device itself.

[0187] like Figure 6 As shown in the figure, this application provides a communication protocol parsing system 600, including:

[0188] Data processing module 601 is used to capture raw packets through a network interface and perform data preprocessing on the raw packets to obtain a numerical sequence.

[0189] The protocol label module 602 is used to input the numerical sequence into the pre-trained model to obtain a protocol label, wherein the protocol label has a corresponding confidence level;

[0190] The structure parsing module 603 is used to perform syntax structure parsing on the original message based on the protocol tag and the confidence level corresponding to the protocol tag to obtain the structure information of the original message;

[0191] The semantic parsing module 604 is used to perform semantic information parsing on the original message based on the structural information to obtain the semantic information of the original message.

[0192] In one embodiment, the data processing module 601 is configured to:

[0193] The original packet physical signal is captured through the network interface and interpreted into a binary data stream. The physical signal is a level signal.

[0194] The binary data stream is converted into protocol data units according to the data link layer protocol.

[0195] By stripping the header information of the network layer and transport layer, the application layer data payload in the protocol data unit is extracted to obtain the application layer message;

[0196] A numerical sequence is generated based on the application layer message.

[0197] In one embodiment, the data processing module 601 is configured to:

[0198] According to the temporal and spatial order of each byte in the application layer message, each byte of the application layer message is arranged sequentially to form a one-dimensional sequence;

[0199] A numerical sequence is obtained by numerically mapping each byte in the one-dimensional sequence, and the numerical sequence includes the integer corresponding to the byte;

[0200] The message association information of the numerical sequence is generated based on the session identifier, communication endpoint, and timing context of the application layer message.

[0201] In one embodiment, the protocol tag module 602 is used for:

[0202] The numerical sequence is input into a pre-trained model to obtain the feature vector of the numerical sequence, and the feature vector is a mathematical representation of the overall pattern of the numerical sequence;

[0203] Load the prototype feature vector of a known protocol and calculate the target distance between the feature vector and the prototype feature vector;

[0204] The known protocol corresponding to the prototype feature vector with the smallest target distance is determined as the protocol label of the numerical sequence, and the confidence level of the protocol label is generated based on the target distance.

[0205] In one embodiment, the structure parsing module 603 is used for:

[0206] If the confidence level of the protocol tag is greater than the first preset confidence level, the communication protocol corresponding to the protocol tag is determined to be the communication protocol of the original message;

[0207] The original message is parsed using the communication protocol to obtain structural information.

[0208] If the confidence level of the protocol tag is less than the second preset confidence level, the original message is parsed to obtain structural information based on at least one of the value domain mutation information, byte statistical patterns, function code and length field in the numerical sequence.

[0209] In one embodiment, when the confidence level of the protocol tag is greater than a first preset confidence level, the semantic parsing module 604 is configured to:

[0210] Load the semantic knowledge base according to the communication protocol;

[0211] The function code, address field, and data field in the structural information are mapped to the semantic information of the original message based on the semantic knowledge base.

[0212] If the confidence level of the protocol tag is less than the second preset confidence level, the semantic parsing module 604 is configured to:

[0213] Based on the structural information, combined with prior knowledge of known protocols, the context of the original message, and statistical cluster analysis, at least one semantic inference information of the original message is generated, and each semantic inference information has a confidence level.

[0214] The semantic information of the original message is determined based on the confidence level of the semantic inference information.

[0215] In one embodiment, the method further includes a protocol conversion module, configured to:

[0216] Map the structural information and the semantic information to the target data model;

[0217] Semantic actions are extracted from the structural information by combining the semantic information;

[0218] When the target protocol is a known protocol, the target data model is mapped and assembled according to the target protocol and the semantic action to obtain the target message under the target protocol;

[0219] When the target protocol is unknown, the target data model is mapped and assembled according to the configuration driver information and the semantic action to obtain the target message under the target protocol.

[0220] In applications, the modules in the communication protocol parsing system can be software program modules, or they can be implemented through different logic circuits integrated in the processor, or they can be implemented through multiple distributed processors.

[0221] Figure 7This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 7 As shown, the computer device 7 of this embodiment includes: at least one processor 70 ( Figure 7 (Only one is shown in the diagram) a processor, a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70, wherein the processor 70 executes the computer program 72 to implement the steps in any of the above-described communication protocol parsing method embodiments.

[0222] The computer device may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that... Figure 7 The computer device 7 is merely an example and does not constitute a limitation on the computer device 7. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0223] The processor 70 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0224] In some embodiments, the memory 71 may be an internal storage unit of the computer device 7, such as a hard disk or memory of the computer device 7. In other embodiments, the memory 71 may be an external storage device of the computer device 7, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 7. Furthermore, the memory 71 may include both internal and external storage units of the computer device 7. The memory 71 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0225] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0226] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0227] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0228] like Figure 8 As shown, this application provides a computer program product 800, including a computer program 72. When the computer program 72 is run, the steps in the above-described communication protocol parsing method embodiments are executed.

[0229] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / computer equipment, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0230] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0231] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0232] In the embodiments provided in this application, it should be understood that the disclosed computer devices and methods can be implemented in other ways. For example, the computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0233] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0234] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A communication protocol parsing method, characterized in that, include: The original packets are captured through the network interface and the data preprocessed to obtain a numerical sequence. The numerical sequence is input into a pre-trained model to obtain a protocol label, which has a corresponding confidence level. Based on the protocol tag and the confidence level corresponding to the protocol tag, the original message is parsed to obtain the structural information of the original message; The semantic information of the original message is obtained by parsing the original message based on the structural information; The step of capturing raw packets through a network interface and performing data preprocessing on the raw packets to obtain a numerical sequence includes: The original packet physical signal is captured through the network interface and interpreted into a binary data stream. The physical signal is a level signal. The binary data stream is converted into protocol data units according to the data link layer protocol. By stripping the header information of the network layer and transport layer, the application layer data payload in the protocol data unit is extracted to obtain the application layer message; A numerical sequence is generated based on the application layer message; The generation of a numerical sequence based on the application layer message includes: According to the temporal and spatial order of each byte in the application layer message, each byte of the application layer message is arranged sequentially to form a one-dimensional sequence; A numerical sequence is obtained by numerically mapping each byte in the one-dimensional sequence, and the numerical sequence includes the integer corresponding to the byte; The message association information of the numerical sequence is generated based on the session identifier, communication endpoint, and timing context of the application layer message.

2. The communication protocol parsing method as described in claim 1, characterized in that, The step of inputting the numerical sequence into a pre-trained model to obtain a protocol label, wherein the protocol label has a corresponding confidence level, including: The numerical sequence is input into a pre-trained model to obtain the feature vector of the numerical sequence, and the feature vector is a mathematical representation of the overall pattern of the numerical sequence; Load the prototype feature vector of a known protocol and calculate the target distance between the feature vector and the prototype feature vector; The known protocol corresponding to the prototype feature vector with the smallest target distance is determined as the protocol label of the numerical sequence, and the confidence level of the protocol label is generated based on the target distance.

3. The communication protocol parsing method as described in claim 1, characterized in that, The step of parsing the original message based on the protocol tag and the confidence level corresponding to the protocol tag to obtain the structural information of the original message includes: If the confidence level of the protocol tag is greater than the first preset confidence level, the communication protocol corresponding to the protocol tag is determined to be the communication protocol of the original message. The original message is parsed using the communication protocol to obtain structural information. If the confidence level of the protocol tag is less than the second preset confidence level, the original message is parsed to obtain structural information based on at least one of the value domain mutation information, byte statistical patterns, function code and length field in the numerical sequence.

4. The communication protocol parsing method as described in claim 3, characterized in that, When the confidence level of the protocol tag is greater than a first preset confidence level, the step of parsing the semantic information of the original message based on the structural information to obtain the semantic information of the original message includes: Load the semantic knowledge base according to the communication protocol; Based on the semantic knowledge base, the function code, address field, and data field in the structural information are mapped to the semantic information of the original message; When the confidence level of the protocol tag is less than the second preset confidence level, the step of parsing the semantic information of the original message based on the structural information to obtain the semantic information of the original message includes: Based on the structural information, combined with prior knowledge of known protocols, the context of the original message, and statistical cluster analysis, at least one semantic inference information of the original message is generated, and each semantic inference information has a confidence level. The semantic information of the original message is determined based on the confidence level of the semantic inference information.

5. The communication protocol parsing method according to any one of claims 1 to 4, characterized in that, The method further includes: Map the structural information and the semantic information to the target data model; Semantic actions are extracted from the structural information by combining the semantic information; When the target protocol is a known protocol, the target data model is mapped and assembled according to the target protocol and the semantic action to obtain the target message under the target protocol; When the target protocol is unknown, the target data model is mapped and assembled according to the configuration driver information and the semantic action to obtain the target message under the target protocol.

6. A communication protocol parsing system, characterized in that, include: The data processing module is used to capture raw packets through a network interface and perform data preprocessing on the raw packets to obtain a numerical sequence; capture the physical signals of the raw packets through the network interface and interpret the physical signals as binary data streams, wherein the physical signals are level signals; convert the binary data streams into protocol data units according to the data link layer protocol; extract the application layer data payload from the protocol data units by stripping the header information of the network layer and transport layer to obtain application layer packets; generate a numerical sequence based on the application layer packets; arrange each byte of the application layer packets sequentially according to the temporal and spatial order of each byte in the application layer packets to form a one-dimensional sequence; perform numerical mapping on each byte in the one-dimensional sequence to obtain a numerical sequence, wherein the numerical sequence includes the integer corresponding to the byte; and generate packet association information of the numerical sequence based on the session identifier, communication endpoint, and timing context of the application layer packets. The protocol label module is used to input the numerical sequence into the pre-trained model to obtain a protocol label, and the protocol label has a corresponding confidence level. The structure parsing module is used to perform syntax structure parsing on the original message based on the protocol tag and the confidence level corresponding to the protocol tag to obtain the structure information of the original message; The semantic parsing module is used to perform semantic information parsing on the original message based on the structural information to obtain the semantic information of the original message.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A computer program product, characterized in that, Includes a computer program, which, when run, causes the method described in any one of claims 1 to 5 to be performed.

Images