Data-driven extensible communication protocol analysis system, method, equipment and medium

Abstract

The invention discloses a data-driven extensible communication protocol analysis system, method and device and a medium, and relates to the technical field of communication protocol processing and software automation, and the system comprises a log preprocessing module which is used for forming a binary protocol frame to be analyzed; the frame analysis module is used for carrying out structured analysis on the binary protocol frame and comprises a protocol atom library used for storing and analyzing an atom analysis function of a basic data field; the atomic type mapping table is used for associating an atomic type identifier declared in a protocol description file with the atomic analytic function; and the data stream analysis and visualization module is used for generating visual output based on the data stream context formed in the frame analysis process. According to the invention, the analysis of the newly added protocol field only needs to register a new atomic function in the protocol atomic library without modifying the core architecture of the system, and the newly added protocol type only needs to establish a new frame structure mapping relation, so that the system can quickly adapt to the evolution requirements of various communication protocols.

Description

Technical Field

[0001] This invention relates to the field of communication protocol processing and software automation technology, and in particular to a data-driven, scalable communication protocol parsing system, method, device, and medium. Background Technology

[0002] In critical fields such as industrial automation and power systems, reliable data communication between equipment and systems relies on a series of standardized communication protocols. These protocols define complex frame structures, diverse data unit types, and strict timing logic, forming the cornerstone for data acquisition, equipment monitoring, and system scheduling. Accurate and efficient parsing of these protocol messages constitutes a core technical challenge for various host computer software, data gateways, and monitoring platforms.

[0003] Existing methods achieve the conversion between data item identifiers of different protocols by establishing a data item mapping table, solving the problem of not needing to write a separate parsing function for each data item in protocol conversion scenarios, thereby improving development efficiency when multiple protocols are compatible. However, this existing method has a fundamental deficiency: it is essentially a protocol conversion scheme built on top of a complete underlying parser, rather than a bottom-up general parsing framework. It cannot handle core tasks such as identifying and extracting protocol frames from raw binary data streams and parsing complex nested structures. Its flexibility is limited to data mapping within existing protocol frameworks, and it does not address the core contradiction between the protocol parsing logic itself and the code coupling. Therefore, it fails to solve deeper industry pain points such as flexible definition of protocol structure, visualization of the parsing process, and automatic document synchronization.

[0004] Another existing method solves the problem of separating protocol description from program code and implementing automatic data update notification through a signal-slot mechanism by defining an XML-formatted protocol file and using a template engine to convert it into compilable code. However, it has significant shortcomings: First, when the protocol changes, users must directly modify the existing XML configuration file structure. This process is extremely cumbersome when dealing with complex protocols, and because XML configuration is usually a tightly integrated whole, any addition, deletion, or modification of nodes constitutes a "destructive modification" to the core configuration logic, which is prone to introducing errors and causing compatibility issues. Second, and more fundamentally, because its architecture lacks an atomic abstraction and reuse mechanism for protocol elements, a large number of repetitive XML code segments appear when describing complex protocols composed of basic units. This makes the configuration file lengthy, difficult to understand, and structurally rigid, and instead of improving maintainability, it significantly worsens it.

[0005] In summary, existing technologies lack a solution that can fundamentally achieve flexible configuration of protocol parsing logic, a clear and modular architecture, and unified management and visual traceability of parsed data streams. Therefore, there is an urgent need in this field for an innovative technical solution to completely resolve the core pain points caused by traditional implementation methods, such as a black-box development process, poor system scalability, maintenance difficulties, and opaque data streams. Summary of the Invention

[0006] In view of the above-mentioned problems, the present invention provides a data-driven scalable communication protocol parsing system, method, device and medium.

[0007] Therefore, the problem that this invention aims to solve is that the existing technology lacks a solution that can fundamentally achieve flexible configuration of protocol parsing logic, clear and modular architecture, and unified management and visual traceability of parsed data streams.

[0008] To address the aforementioned technical problems, this invention provides the following technical solution: a data-driven, scalable communication protocol parsing system, comprising: a log preprocessing module for identifying and extracting target communication protocol frames from raw log data to form binary protocol frames to be parsed; a frame parsing module for performing structured parsing of the binary protocol frames, the frame parsing module comprising: a protocol atomic library for storing atomic parsing functions for parsing basic data fields; an atomic type mapping table for associating atomic type identifiers declared in the protocol description file with the atomic parsing functions; a frame type library for storing the frame type structure of the communication protocol; a frame type mapping table for associating frame type identifiers with corresponding parsing sequences; and a data flow parsing and visualization module for generating visualization output based on the data flow context formed during frame parsing, comprising: a visualization template library for storing visualization template rendering functions; and a template mapping table for associating visualization type identifiers with corresponding template rendering functions.

[0009] As a preferred embodiment of the data-driven scalable communication protocol parsing system described in this invention, the binary protocol frame to be parsed is formally defined as follows: in, This is the original log text that was entered. This is a function that encapsulates the logic for protocol identification and data extraction. The output of the function is the protocol frame containing the original binary data that has been successfully identified and extracted.

[0010] As a preferred embodiment of the data-driven scalable communication protocol parsing system described in this invention, the atomic parsing function is an independent mapping unit, and its behavior is formally defined as follows: in, This is a buffer for the raw binary data input. This is the current parsing offset in the data buffer; Contextual information, used to transmit and store shared information across atoms; For the corresponding executable atomic function entities in the library; For data in a specific format that has been parsed; This is the updated parsed offset, used by the next atomic function. This is the updated parsing context, reflecting any possible modifications to the shared state made during this parsing process.

[0011] As a preferred embodiment of the data-driven scalable communication protocol parsing system described in this invention, the frame type structure consists of an ordered parsing sequence composed of several atomic type identifiers; in the frame parsing module, the frame parsing engine obtains the corresponding atomic type identifier sequence from the frame type mapping table based on the frame type identifier during the parsing process, and obtains the corresponding atomic parsing function according to the atomic type mapping table to execute the parsing operation sequentially, and continuously updates the parsing offset and parsing context information during the parsing process.

[0012] As a preferred embodiment of the data-driven scalable communication protocol parsing system described in this invention, in the data stream parsing and visualization module, the visualization engine selects the corresponding template rendering function based on the template mapping table and converts the complete data stream context accumulated during frame parsing into a visualization output document.

[0013] To address the aforementioned technical problems, this invention provides the following technical solution: a data-driven, scalable communication protocol parsing method, comprising: constructing basic components for protocol parsing based on protocol technical documents; performing communication protocol parsing based on the obtained basic components; collecting the input, output, offset changes, and context states of each atomic parsing function during the communication protocol parsing process to form a data flow context; and selecting a corresponding template rendering function from a visualization template library based on a preset visualization type to convert the data flow context into a visualization output document.

[0014] As a preferred embodiment of the data-driven scalable communication protocol parsing method described in this invention, the construction of the basic components for protocol parsing includes: extracting basic field definitions from the protocol technical documents and establishing corresponding atomic parsing functions for each field to construct a protocol atomic library; associating atomic type identifiers with the atomic parsing functions to establish an atomic type mapping table; combining several atomic type identifiers to form a parsing sequence corresponding to the frame type according to the frame structure defined in the protocol to construct a frame type library; associating frame type identifiers with the parsing sequence to establish a frame type mapping table; determining the visualization presentation method according to the protocol document and user requirements, designing and implementing a template rendering function for generating visualization output, and assigning corresponding visualization type identifiers to form a template mapping table for rendering output of the data stream context after parsing, thus constructing a visualization template library.

[0015] As a preferred embodiment of the data-driven scalable communication protocol parsing method described in this invention, the execution of communication protocol parsing includes: identifying and extracting target communication protocol frames from raw log data; converting the protocol frames into standardized binary protocol frames; identifying the corresponding frame type identifier based on the protocol frame characteristics; obtaining a parsing sequence based on the frame type mapping table; sequentially calling atomic parsing functions according to the atomic type mapping table to perform field parsing and continuously updating the parsing offset and parsing context information; and when the data body contains duplicate structured objects, cyclically calling the corresponding atomic parsing functions to complete multi-object processing.

[0016] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of a data-driven scalable communication protocol parsing method as described above.

[0017] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a data-driven, scalable communication protocol parsing method as described above.

[0018] The beneficial effects of this invention are as follows: This patent achieves high system scalability through atomic component design and modular architecture. Parsing new protocol fields only requires registering new atomic functions in the protocol atomic library, without modifying the core system architecture; the frame type library supports dynamic expansion, and adding new protocol types only requires establishing new frame structure mapping relationships; the visualization template library allows for flexible addition of new display formats and rendering methods as needed. This hierarchical expansion mechanism enables the system to quickly adapt to the evolving needs of various communication protocols.

[0019] Based on the single responsibility principle of atomic functions and standardized data flow management, each parsing unit in the system evolves independently, significantly reducing system coupling. The configurable architecture allows protocol changes to be implemented by updating configuration files, avoiding the cumbersome process of modifying core code as in traditional methods. A clear data flow tracing mechanism ensures the complete controllability of the parsing process state, greatly improving the system's maintainability and stability.

[0020] This patent enables complete visualization of the protocol parsing process, transforming traditional binary protocol parsing from a "black box" operation into a transparent "white box" process. Real-time data stream tracing capabilities allow developers to intuitively observe the complete transformation path of data from the raw binary stream to structured results. Combined with standardized debugging interfaces, this significantly shortens fault location time and provides comprehensive infrastructure support for automated testing. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a structural diagram of a data-driven scalable communication protocol parsing system in Example 1.

[0023] Figure 2 This is a flowchart of a data-driven scalable communication protocol parsing method in Example 2.

[0024] Figure 3 This is a flowchart illustrating the construction of basic components for a data-driven, scalable communication protocol parsing method in Example 2.

[0025] Figure 4 This is a flowchart illustrating the communication protocol parsing process of a data-driven, scalable communication protocol parsing method in Example 2.

[0026] Figure 5 This is a visual flowchart of a data-driven scalable communication protocol parsing method in Example 2. Detailed Implementation

[0027] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0028] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0029] Example 1, referring to Figure 1 This is the first embodiment of the present invention, which provides a data-driven, scalable communication protocol parsing system. This example is perfectly adapted to the parsing requirements of the IEC 60870-5-104 protocol (hereinafter referred to as IEC104). As a standard communication protocol for power automation systems, the hierarchical frame structure and standardized data unit definition of this protocol are highly compatible with the atomic parsing architecture and data-driven concept proposed in this invention. The following section will select some typical fields of this protocol to specifically demonstrate the implementation process of the technical solution of this invention.

[0030] The specific structure of the system of this invention is as follows: Figure 1 As shown, it includes: The log preprocessing module is used to identify and extract target communication protocol frames from raw log data, forming binary protocol frames to be parsed. This module serves as the input to the protocol parsing process and is specifically designed to process the raw IEC104 log files generated between the network dispatch center and the power plant. Through processing steps such as protocol feature identification, data frame boundary detection, and integrity verification, it accurately extracts binary protocol frames conforming to the IEC104 specification from the mixed communication logs, providing standardized data input for subsequent protocol parsing processes. Its core operation can be formally defined as: in, This is the original log text that was entered. This is a function that encapsulates the logic for protocol identification and data extraction. The output of the function is the protocol frame containing the original binary data that has been successfully identified and extracted.

[0031] The frame parsing module is used to perform structured parsing of binary protocol frames. The frame parsing module includes: The protocol atomic library stores atomic parsing functions for parsing basic data fields. As a core component of the frame parsing engine, it is a pre-built, extensible repository of parsing functions. This library not only contains a series of atomic parsing functions that follow the single responsibility principle, but also maintains a crucial atomic type mapping table.

[0032] An atomic analytic function is an independent mapping unit whose behavior is formally defined as follows: in, This is a buffer for the raw binary data input. This is the current parsing offset in the data buffer; Contextual information, used to transmit and store shared information across atoms; For the corresponding executable atomic function entities in the library; For data in a specific format that has been parsed; This is the updated parsed offset, used by the next atomic function. This is the updated parsing context, reflecting any possible modifications to the shared state made during this parsing process.

[0033] The core atomic parsing functions include: start character parsing (get_start_char_info), APDU length parsing (get_apdu_length_info), control field parsing (get_control_field_info), normalized value parsing (get_NVA_info), quality descriptor parsing (get_QDS_info), and seven-byte time parsing (get_CP56Time2a_info).

[0034] An atomic type mapping table, used to associate atomic type identifiers declared in the protocol description file with atomic resolution functions, is formally defined as follows: in, For atom type mapping table, This is an identifier for the atomic function type declared in the protocol description file.

[0035] Based on the core atom parsing function, the atom type mapping table can be represented as: Protocol Atomic Library By definition, the entire atomic library can be formally defined as a binary tuple containing a mapping table and a set of functions: At runtime, the frame parsing engine, based on the atomic type sequence specified in the context (previously parsed fields), queries... To obtain the corresponding function entity The data is processed sequentially, gradually transforming the original binary input stream into a final structured data object. This dynamic binding mechanism, implemented through a mapping table, is key to the high scalability and runtime flexibility achieved in this patent.

[0036] The frame type library stores the frame type structure of communication protocols. As the core of the protocol parsing system's structure definition, it is a pre-built, extensible frame structure repository. This library maintains a crucial frame type mapping table, used to dynamically associate specific frame type identifiers appearing in the protocol with their corresponding atomic function execution sequences, thereby fully defining the parsing logic for that type of data frame.

[0037] A frame type mapping table, used to associate frame type identifiers with corresponding parsed sequences, is formally defined as follows: in, For frame type mapping table, This is a frame type identifier identified within the protocol context; For a given frame structure, a sequence of atomic functions for parsing that frame type is defined. Indexed by quantity.

[0038] The frame type mapping table can be represented as: in, When parsing a time-stamped single-point information frame, the system sequentially calls the normalized value, quality descriptor, and seven-byte time parsing atomic function; the parsing of Application Protocol Control Information (APCI) frames follows the same principle. This frame structure definition method based on atomic function sequences ensures the standardization and scalability of protocol parsing, while providing an architectural foundation for the dynamic combination of protocol frames.

[0039] Frame type library The complete definition is: To further explain, the frame type structure consists of an ordered parsed sequence of several atomic type identifiers, and its formal definition is: in, The total number of atoms is identified by the atom type.

[0040] In the frame parsing module, the frame parsing engine retrieves the corresponding atomic type identifier sequence from the frame type mapping table based on the frame type identifier during the parsing process. It then obtains the corresponding atomic parsing function according to the atomic type mapping table and executes the parsing operations sequentially, continuously updating the parsing offset and parsing context information throughout the process. This mechanism of mapping frame type to function sequence is key to the flexible definition and dynamic adaptation of the protocol structure in this patent.

[0041] And a data stream parsing and visualization module, used to generate visualization output based on the data stream context formed during frame parsing, including: The visual template library stores visual template rendering functions; it is a pre-built, extensible template repository that maintains a key template mapping table.

[0042] Template mapping tables are used to associate visual type identifiers with corresponding template rendering functions, and their formal definition is: in, For template mapping table, For visual types of identifiers; This is the corresponding executable template rendering function entity in the library.

[0043] In this example of the IEC104 protocol implementation scheme, the following is defined: This example includes the following two core rendering functions: To generate structured Markdown reports with good readability and hierarchical formatting, suitable for technical document generation and system debugging; It displays data stream content in a concise text mode, providing the most basic information presentation, suitable for rapid verification and logging.

[0044] Each template rendering function is an independent mapping unit, and its behavior is formally defined as follows: in, The data flow context contains a complete record of all intermediate states and data throughout the entire parsing process, from raw binary data to the final structured result. The generated visual output is in a human-readable format.

[0045] Visual Template Library By definition, the entire visualization template library can be formally defined as a binary tuple containing a template mapping table and a set of template functions: in, This is a collection of template rendering functions.

[0046] In the data stream parsing and visualization module, the visualization engine selects the corresponding template rendering function based on the template mapping table and converts the complete data stream context accumulated during frame parsing into a visual output document. This dynamic rendering mechanism implemented through the template mapping table is the key to achieving transparency in the parsing process and visualization of debugging information in this patent.

[0047] Example 2, refer to Figures 2-5This is the second embodiment of the present invention, which differs from the first embodiment in that: a data-driven scalable communication protocol parsing method includes, as follows: Figure 2 As shown: Step a: Build the basic components for protocol parsing based on the protocol technical documents, specifically as follows: Figure 3 As shown: Step a-1: Extract the basic field definitions from the protocol technical document and establish a corresponding atomic parsing function for each field to build a protocol atomic library. The aim is to transform the field specifications defined in the communication protocol technical document into a formally defined protocol atomic library. The input is the protocol technical document, and the output is the initialized protocol atomic library.

[0048] Step a-1-1: Parse the protocol's basic fields. Read and parse the protocol technical document, extracting all explicitly defined basic data fields and their parsing rules. This process can be formally represented as: in, For input protocol technical documents; For field extraction functions; This is the extracted set of basic fields, including the parsing rules for these fields. Basic fields are the smallest logical units that constitute a protocol frame, including but not limited to: start character, length field, control field, address field, checksum, etc.

[0049] Step a-1-2: Establish a set of atomic types and assign a unique atomic function type identifier to each extracted basic data field. This process forms a set of atom types T, which can be formally represented as: in, For the first The atomic type identifiers corresponding to each basic field For any, For the first One basic field.

[0050] Step a-1-3: Implement the atomic function entity. Based on the parsing rules for each field, implement the specific function that conforms to the atomic function interface: All implemented atomic functions constitute the atomic function set. .

[0051] Step a-1-4: Establish an atom type mapping table to associate atom type identifiers with atom resolution functions: This mapping table constitutes the protocol atomic library. The core architecture.

[0052] Through the above steps, the field specifications in the protocol technical document are systematically converted into an executable protocol atomic library, providing the foundation for subsequent frame parsing. This mapping-based architecture allows the system to easily add new... Mapping pairs are used to extend parsing capabilities.

[0053] Step a-2: Based on the frame structure defined in the protocol, combine several atomic type identifiers to form a parsing sequence of the corresponding frame type to build a frame type library. The aim is to extract frame type definitions from the protocol technical documents and transform them into a formally defined frame type library. The input is the protocol technical documents and the constructed protocol atomic library, and the output is the initialized frame type library.

[0054] Step a-2-1: Parse the frame type definition, read the protocol technical document, and extract all frame types and their structural descriptions defined therein. This process can be formally represented as: in, For input protocol technical documents; For frame type extraction functions, This is the extracted set of frame type identifiers, containing a structural description of each frame type.

[0055] Step a-2-2 establishes a frame structure mapping, for each frame type identifier Based on its structural description, corresponding atomic functions are selected from the protocol atomic library to construct an ordered sequence of atomic functions. This process can be formally represented as: in, Furthermore, the sequence order is determined by the structural description of the frame type.

[0056] Step a-2-3 establishes a frame type mapping table, associates frame type identifiers with parsing sequences, and establishes a mapping relationship from frame type identifiers to frame structures.

[0057] Step a-2-4 verifies the integrity of the frame structure. Each frame structure is verified to ensure it can completely parse all fields of the corresponding frame type. This verification process can be formally represented as: in, To verify the function, ensure that the frame structure can handle all data fields of the frame type.

[0058] Through the above steps, the frame type definitions in the protocol technical documents are systematically converted into an executable frame type library, providing structural definition capabilities for subsequent frame parsing. This mapping-based architecture allows the system to support new frame types simply by adding new mapping pairs.

[0059] Step a-3: Determine the visualization presentation method based on the protocol document and user requirements, design and implement the template rendering function for generating visualization output, and assign corresponding visualization type identifiers to form a template mapping table to build a visualization template library for rendering output of the data stream context after parsing.

[0060] Step a-3-1 analyzes the visualization requirements. Based on the specific visualization requirements written by the user, the corresponding template rendering function is designed and implemented. This process can be formally represented as: in, Define a single visualization requirement, explicitly specifying the output format, data mapping relationships, and display style; This refers to the process of manually implementing template functions based on user requirements.

[0061] Step a-3-2 establishes a visual type mapping, assigning a corresponding visual type identifier to each implemented template function.

[0062] Step a-3-3: Build a visual template library, integrating all implemented template functions and mapping tables to construct a complete visual template library. Step a-3-4: Verify the conformity of the template function. Verify the implemented template function to ensure that its output fully meets the user's requirements.

[0063] Through the above steps, the user's specific visualization needs are accurately translated into corresponding template functions, and a complete visualization template library is built. This method ensures that each template function can accurately meet the user's specific visualization requirements, achieving a high degree of customization and flexibility.

[0064] Step b: Based on the obtained basic components, perform communication protocol parsing. Its input is typically a system log file containing communication records or other files carrying the original protocol data, specifically as follows: Figure 4 The following are included: Step b-1: Identify and extract the target communication protocol frame from the raw log data, and convert the protocol frame into a standardized binary protocol frame. This process can be formally represented as: in, The original log file that was input; This is a protocol frame extraction function; This refers to the set of valid protocol frames extracted, i.e., the set of binary protocol frames.

[0065] Step b-2: Based on the protocol atom library and frame type library constructed in step a, perform structured parsing of the binary protocol frames. This process can be formally represented as follows: in, The result of parsing the structured data. This is the main function for frame parsing. This is the input raw binary protocol frame.

[0066] The specific analysis process includes: Step b-2-1 Frame type identification: Identify the corresponding frame type identifier based on the protocol frame characteristics. in, This is a frame type identification function.

[0067] Step b-2-2 Obtain the parsed sequence: Based on the identified frame type, obtain the parsed sequence using the frame type mapping table. in, Identify the sequence of atomic function types. To access members in an object, For use Use it as a key to retrieve a value from the mapping table.

[0068] Step b-2-3 executes the parsing sequentially, calling the atomic parsing function in turn according to the atomic type mapping table, performing field parsing and continuously updating the parsing offset and parsing context information.

[0069] Step b-2-4 Multi-object processing: When the data body contains duplicate structure objects, the corresponding atomic parsing function is called repeatedly until all object data is processed, thus completing the multi-object processing.

[0070] Step c: During protocol parsing, a data stream is formed and visualized. During communication protocol parsing, the input, output, offset changes, and context state of each atomic parsing function are collected to form a data stream context. Based on a preset visualization type, a corresponding template rendering function is selected from the visualization template library to convert the data stream context into a visualization output document, as detailed below. Figure 5 The following are included: Step c-1: Collect complete state information generated during protocol frame parsing and construct a standardized data flow context. This context includes the original binary data, the input and output states during the execution of each atomic function, the parsing offset change trajectory, and a complete evolution record of the context information.

[0071] Step c-2: Based on user configuration or system preset, select the appropriate visualization type from the template mapping table of the visualization template library and determine the corresponding template rendering function.

[0072] Step c-3: Take the constructed data stream context as input and pass it to the selected template rendering function to execute the rendering process and generate a visual output that meets the user's needs.

[0073] Step c-4: Output the visualization content generated by the template rendering function in the specified format, completing the entire conversion process from protocol parsing data stream to a readable visualization report.

[0074] Example 3, the third embodiment of the present invention, differs from the previous two embodiments in that: if the function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0075] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0076] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0077] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented in combination with any of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0078] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A data-driven, scalable communication protocol parsing system, characterized in that: include, The log preprocessing module is used to identify and extract target communication protocol frames from raw log data to form binary protocol frames to be parsed. A frame parsing module is used to perform structured parsing of the binary protocol frame. The frame parsing module includes: Protocol Atom Library: This library stores atomic parsing functions for resolving basic data fields. An atom type mapping table is used to associate the atom type identifiers declared in the protocol description file with the atom parsing functions; Frame type library, used to store the frame type structure of communication protocols; A frame type mapping table is used to associate frame type identifiers with corresponding parsing sequences; And a data stream parsing and visualization module, used to generate visualization output based on the data stream context formed during frame parsing, including: A visual template library for storing visual template rendering functions; The template mapping table is used to associate visual type identifiers with corresponding template rendering functions.

2. The data-driven scalable communication protocol parsing system as described in claim 1, characterized in that: The formal definition of the binary protocol frame to be parsed is expressed as the formula: in, This is the original log text that was entered. This is a function that encapsulates the logic for protocol identification and data extraction. The output of the function is the protocol frame containing the original binary data that has been successfully identified and extracted.

3. The data-driven scalable communication protocol parsing system as described in claim 2, characterized in that: The atomic analytic function is an independent mapping unit, and its behavior is formally defined as follows: in, This is a buffer for the raw binary data input. This is the current parsing offset in the data buffer; Contextual information, used to transmit and store shared information across atoms; For the corresponding executable atomic function entities in the library; For data in a specific format that has been parsed; This is the updated parsed offset, used by the next atomic function. This is the updated parsing context, reflecting any possible modifications to the shared state made during this parsing process.

4. The data-driven scalable communication protocol parsing system as described in claim 3, characterized in that: The frame type structure consists of an ordered parsing sequence of several atomic type identifiers; In the frame parsing module, the frame parsing engine obtains the corresponding atomic type identifier sequence from the frame type mapping table based on the frame type identifier during the parsing process, and obtains the corresponding atomic parsing function according to the atomic type mapping table to execute the parsing operation sequentially, and continuously updates the parsing offset and parsing context information during the parsing process.

5. The data-driven scalable communication protocol parsing system as described in claim 4, characterized in that: In the data stream parsing and visualization module, the visualization engine selects the corresponding template rendering function based on the template mapping table and converts the complete data stream context accumulated during frame parsing into a visualization output document.

6. A data-driven scalable communication protocol parsing method, using the data-driven scalable communication protocol parsing system as described in any one of claims 1 to 5, characterized in that: include, Build the basic components for protocol parsing based on the protocol technical documents; Based on the obtained basic components, perform communication protocol parsing; During the communication protocol parsing process, the input, output, offset changes, and context state of each atomic parsing function are collected to form a data stream context; Based on the preset visualization type, the corresponding template rendering function is selected from the visualization template library to convert the data flow context into a visualization output document.

7. The data-driven scalable communication protocol parsing method as described in claim 6, characterized in that: The basic components for constructing protocol parsing include... Extract the basic field definitions from the protocol technical documents, and build a corresponding atomic parsing function for each field to construct the protocol atomic library; Associate the atom type identifier with the atom parsing function to establish an atom type mapping table; Based on the frame structure defined in the protocol, several atomic type identifiers are combined to form a parsing sequence of the corresponding frame type, and a frame type library is constructed. Associate the frame type identifier with the parsed sequence to establish a frame type mapping table; Based on the protocol documents and user requirements, determine the visualization presentation method, design and implement the template rendering function for generating visualization output, and assign corresponding visualization type identifiers to form a template mapping table, which is used to render the data flow context after parsing and build a visualization template library.

8. The data-driven scalable communication protocol parsing method as described in claim 7, characterized in that: The execution of communication protocol parsing includes, Identify and extract target communication protocol frames from raw log data, and convert the protocol frames into standardized binary protocol frames; Identify the corresponding frame type identifier based on protocol frame characteristics; The parsing sequence is obtained based on the frame type mapping table; Based on the atomic type mapping table, the atomic parsing function is called sequentially to perform field parsing and continuously update the parsing offset and parsing context information; When the data body contains duplicate structured objects, the corresponding atomic parsing function is called repeatedly to complete the processing of multiple objects.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the data-driven scalable communication protocol parsing system according to any one of claims 6 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the data-driven scalable communication protocol parsing system according to any one of claims 6 to 8.

Images