Byte stream data analysis method and device based on deserialization

Abstract

The embodiment of the invention provides a byte stream data analysis method and device based on deserialization, and the method comprises the steps: defining a universal analysis interface, loading an XML analysis configuration file, and automatically constructing a DataFormat object based on the XML analysis configuration file and deserialization; byte stream data to be analyzed is obtained through a universal analysis interface, the data type of the byte stream data is recognized through a DataFormat object, and corresponding analysis logic is called through the universal analysis interface according to the data type to analyze the byte stream data. According to the embodiment of the invention, the expandability and maintainability of the byte stream message analysis function of the ground subsystem maintenance software of the signal system are improved.

Description

Technical Field

[0001] This document relates to the field of computer technology, and in particular to a byte stream data parsing method and device based on deserialization. Background Art

[0002] The ground subsystem application software of the subway signal system (hereinafter referred to as the application software) generates a large amount of status information that needs to be saved during operation. This information is not directly stored in the industrial computer disk, but is generally sent to the maintenance software for processing via Ethernet. That is, the application software needs to reduce the structured status information into a one-dimensional byte stream array and send it to the maintenance software via UDP. The maintenance software receives the byte stream information and parses it according to the status information interface. The common practice is to define the data structure based on the size and meaning of each field defined by the interface, and implement the corresponding parsing code. Once the state information that needs to be transmitted changes, the parsing code needs to be changed accordingly. In order to facilitate maintenance and expansion and reduce changes in the parsing code, it is necessary to separate the interface definition and the parsing logic to enhance the scalability and maintainability of the ground subsystem maintenance software's message parsing function.

[0003] XML (Extensible Markup Language) is a markup language used to mark the structure of electronic files. XML can be used to represent any structured information, such as text, data, graphics, etc., and can be easily parsed and processed by computer programs. It has the characteristics of strong scalability, structured, readable and cross-platform. XML deserialization refers to the process of converting XML documents into objects or data structures. After XML deserialization processing, different interfaces are parsed using common parsing logic. Summary of the invention

[0004] The purpose of the present invention is to provide a byte stream data parsing method and device based on deserialization, aiming to solve the above-mentioned problems in the prior art.

[0005] The present invention provides a byte stream data parsing method based on deserialization, comprising:

[0006] Define a general parsing interface, load an XML parsing configuration file, and automatically construct a DataFormat object by deserialization based on the XML parsing configuration file;

[0007] The byte stream data with parsing is obtained through a universal parsing interface, the data type of the byte stream data is identified through a DataFormat object, and the corresponding parsing logic is called through the universal parsing interface according to the data type to parse the byte stream data.

[0008] The present invention provides a byte stream data parsing device based on deserialization, comprising:

[0009] Load the deserialization module, which is used to define the general parsing interface, load the XML parsing configuration file, and deserialize and automatically construct the DataFormat object based on the XML parsing configuration file;

[0010] The parsing module is used to obtain the parsed byte stream data through the general parsing interface, identify the data type of the byte stream data through the DataFormat object, and parse the byte stream data by calling the corresponding parsing logic through the general parsing interface according to the data type.

[0011] An embodiment of the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the above-mentioned deserialization-based byte stream data parsing method when executed by the processor.

[0012] An embodiment of the present invention further provides a computer-readable storage medium, on which a program for implementing information transmission is stored. When the program is executed by a processor, the steps of the above-mentioned deserialization-based byte stream data parsing method are implemented.

[0013] By adopting the embodiment of the present invention, the scalability and maintainability of the byte stream message parsing function of the signal system ground subsystem maintenance software are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0015] Figure 1 is a flowchart of a byte stream data parsing method based on deserialization according to an embodiment of the present invention;

[0016] Figure 2 is a flowchart of the detailed processing of the byte stream data parsing method based on deserialization according to an embodiment of the present invention;

[0017] Figure 3 It is a schematic diagram of an example of writing an XML configuration file according to an embodiment of the present invention;

[0018] Figure 4 is a schematic diagram of ATS-ZC input information parsing according to an embodiment of the present invention;

[0019] Figure 5is a schematic diagram of ATS-ZC input parsing structured data according to an embodiment of the present invention;

[0020] Figure 6 Schematic diagram of a train application example of an embodiment of the present invention;

[0021] Figure 7 is a schematic diagram of a byte stream data parsing device based on deserialization according to an embodiment of the present invention;

[0022] Figure 8 is a schematic diagram of an electronic device according to an embodiment of the present invention;

[0023] Fig. 9 It is a schematic diagram of the processing steps of the byte stream data parsing method based on deserialization according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will be combined with the drawings in one or more embodiments of this specification to clearly and completely describe the technical solutions in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0025] Method Embodiment

[0026] According to an embodiment of the present invention, a byte stream data parsing method based on deserialization is provided. Figure 1 is a flowchart of a byte stream data parsing method based on deserialization according to an embodiment of the present invention. Figure 1 As shown, the byte stream data parsing method based on deserialization according to an embodiment of the present invention specifically includes:

[0027] Step S101, defining a general parsing interface, loading an XML parsing configuration file, parsing the definition in the XML parsing configuration file, deserializing and automatically constructing a DataFormat object, wherein the DataFormat object includes multiple Package objects, each Package object includes multiple Zones, and each Zone contains a specific parsing type instance; wherein the XML parsing configuration file specifically includes: all possible Package types and their corresponding XML tags, defining multiple Zones in each Package, each Zone containing a specific parsing type and its configuration, and using XML tags to represent parsers of different parsing types and necessary parameters for configuration;

[0028] Step S102, obtain the parsed byte stream data through the general parsing interface, identify the type of the current data packet according to the previous bytes of the byte stream data through the DataFormat object, and find the corresponding Package object; traverse each Zone object in the Package object, and in each Zone object, execute the parsing logic in sequence according to the definition of the parsing type, and store or output the parsing result in a predetermined format; during the parsing process, if an unrecognizable data packet type or a parsing error is encountered, capture the exception and perform corresponding error handling, wherein the general parsing interface includes: basic methods of all parsing types, wherein the basic methods of the parsing types specifically include: execution entry of the parsing logic, and the parsing types specifically include: basic parsing types, composite parsing types and container types, wherein for multiple independent Zone objects or Package objects, multi-threading or multi-processing is used for parallel parsing:

[0029] For the basic analysis type, directly call its Analysis method to parse the corresponding byte data; for the compound analysis type, call the corresponding sub-analysis type for analysis according to its internal logic.

[0030] During the parsing process, if an unrecognizable data packet type or a parsing error is encountered, the exception is captured and the corresponding error handling is performed.

[0031] In an embodiment of the present invention, in order to optimize the processing flow, the following optimization operations may be performed: before parsing the byte stream data, the byte stream data may be pre-read and pre-processed; for recurring parsing types or parsing configurations, the repetitive parsing configurations may be reduced by caching.

[0032] It can be seen from the above technical solution that the embodiment of the present invention defines the data structure of the subway signal system application software status information in XML, matches the byte stream parsing rules defined by the interface, and includes basic parsing types used for inter-system communication such as bit, byte, ushort, uint, long, and Debugchar. It is also necessary to define and display compound parsing types such as loops and selections. The parsing of complex data structures is achieved by combining the basic parsing types and compound parsing types. Finally, a parsing program combined with a configuration file can parse any type of message byte stream, which effectively improves the scalability and maintainability of the parsing.

[0033] The above technical solution of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings.

[0034] The embodiment of the present invention adopts an object-oriented software development method to build a flexible and scalable byte stream message parsing system by defining a general parsing interface (IItem) and its implementation class. The parsing program architecture design mainly includes the following key parts:

[0035] Common parsing interface (IItem): defines the basic methods of all parsing types, including the execution entry of the parsing logic (such as the Analysis method).

[0036] Basic parsing types: such as Bit, Byte, Ushort, Uint, etc. These types are directly derived from the Item base class to implement specific data parsing logic.

[0037] Composite parsing types: such as ItemIf, ItemFor, ItemSwitch, etc. These types contain multiple sub-parsing types, which are combined for parsing according to specific logic (such as conditional judgment, loop, selection).

[0038] Container types: such as Zone, Package, and DataFormat, are used to organize and manage different parsing types to implement partition management and automatic matching parsing of data.

[0039] Table 1 General parsing interface design

[0040]

[0041] Table 2 Basic analysis type design

[0042]

[0043] Table 3 Composite analysis type design

[0044]

[0045]

[0046] The execution flow of the parsing program is as follows Figure 2 and Fig. 9 As shown, its application scenarios can be Figure 6 The scenario shown. The specific processing includes the following:

[0047] (1) Initialize the parsing configuration: load the XML parsing configuration file, parse the definitions in the configuration file, and deserialize to automatically build a DataFormat object, which contains multiple Package objects. Each Package object contains multiple Zones, and each Zone contains a specific parsing type instance.

[0048] Among them, Figure 3As shown in the figure, the parsing configuration file is written based on the XML format and mainly includes the following parts: DataFormat definition: lists all possible Package types and their corresponding XML tags. Package definition: defines multiple zones in each package, each zone contains a specific parsing type and its configuration (such as starting position, byte length, etc.). Parsing type definition: uses XML tags to represent different types of parsers (basic parsing type or composite parsing type) and configures necessary parameters.

[0049] (2) Read byte stream data: Get the byte stream data to be parsed from the network or other data sources. ATS-ZC input information parsing is as follows: Figure 4 The parsed structured data input by ATS-ZC is shown as Figure 5 shown.

[0050] (3) Identify the data packet type: The DataFormat object first identifies the type of the current data packet based on the first few bytes of the byte stream (usually the header information of the data frame) and finds the corresponding Package.

[0051] (4) Execute parsing logic:

[0052] a) Traverse each Zone in the Package.

[0053] b) In each zone, the parsing logic is executed in sequence according to the definition of the parsing type (basic parsing type or composite parsing type).

[0054] i. For basic parsing types, directly call its Analysis method to parse the corresponding byte data.

[0055] ii. For compound parsing types, according to their internal logic (such as conditional judgment, loop, etc.), call the corresponding sub-parsing type for parsing.

[0056] c) Store or return the parsing results in a predetermined format.

[0057] (5) Error handling: During the parsing process, if an unrecognizable data packet type or a parsing error is encountered, the program should be able to capture the exception and perform appropriate error handling.

[0058] (6) Result output: The parsing results are output in a structured form for use by upper-level applications.

[0059] In the embodiment of the present invention, in order to improve the parsing performance, for multiple independent zones or packages, multi-threading or multi-processing can be used for parallel parsing. For repeated data packet types or parsing configurations, a cache can be used to reduce the overhead of repeated parsing configurations. Before parsing, the data is pre-read and pre-processed, such as calculating checksums, judging data integrity, etc., to reduce invalid parsing.

[0060] In summary, the technical solution of the embodiment of the present invention proposes a byte stream message parsing method based on deserialization, which aims to improve the parsing efficiency and maintainability of the signal system ground subsystem maintenance software. The method realizes flexible parsing of complex byte stream messages by defining a set of general parsing interfaces (IItem) and their implementation classes. The core lies in separating the parsing logic from the data structure, defining the parsing rules of the message through the XML configuration file, thereby realizing the dynamic configurability of the interface definition.

[0061] In terms of technical implementation, the embodiment of the present invention adopts an object-oriented software design method, designs basic parsing types (such as Bit, Byte, etc.) and compound parsing types (such as ItemIf, ItemFor, etc.), as well as container types (Zone, Package, DataFormat) to organize and manage parsing logic. The parsing program builds a parsing model by loading an XML configuration file, automatically matches and executes the corresponding parsing logic according to the input byte stream data, and finally restores the byte stream into structured data for use by upper-layer applications.

[0062] To further optimize performance, the embodiments of the present invention also introduce parallel parsing, caching mechanism and pre-reading preprocessing and other technical means, which significantly improve the efficiency and accuracy of message parsing. When the communication interface between systems changes, parsing messages only requires changing the parsing configuration file, which reduces the difficulty of maintenance.

[0063] Device Example 1

[0064] According to an embodiment of the present invention, a byte stream data parsing device based on deserialization is provided. Figure 7 Schematic diagram of a byte stream data parsing device based on deserialization according to an embodiment of the present invention. Figure 7 As shown, the byte stream data parsing device based on deserialization according to an embodiment of the present invention specifically includes:

[0065] A deserialization module 70 is loaded, which is used to define a general parsing interface, load an XML parsing configuration file, parse the definition in the XML parsing configuration file, and automatically construct a DataFormat object by deserialization. The DataFormat object includes multiple Package objects, each Package object includes multiple Zones, and each Zone contains a specific parsing type instance; wherein the XML parsing configuration file specifically includes: all possible Package types and their corresponding XML tags, multiple Zones are defined in each Package, each Zone contains a specific parsing type and its configuration, and XML tags are used to represent parsers of different parsing types and necessary parameters for configuration.

[0066] The parsing module 72 is used to obtain the byte stream data with parsing through the general parsing interface, identify the type of the current data packet according to the front bytes of the byte stream data through the DataFormat object, and find the corresponding Package object; traverse each Zone object in the Package object, execute the parsing logic in sequence according to the definition of the parsing type in each Zone object, and store or output the parsing result in a predetermined format; in the parsing process, if an unrecognizable data packet type or a parsing error is encountered, the exception is captured and the corresponding error processing is performed, wherein the general parsing interface includes: the basic methods of all parsing types, wherein the basic methods of the parsing types specifically include: the execution entry of the parsing logic, and the parsing types specifically include: basic parsing types, composite parsing types and container types, wherein for multiple independent Zone objects or Package objects, multi-threading or multi-processing is used for parallel parsing. .

[0067] The analysis module 72 is specifically used for: for a basic analysis type, directly calling its Analysis method to parse the corresponding byte data; for a compound analysis type, calling the corresponding sub-analysis type for analysis according to its internal logic.

[0068] During the parsing process, if an unrecognizable data packet type or a parsing error is encountered, the exception is captured and the corresponding error handling is performed.

[0069] The parsing module 72 is further used to: pre-read and pre-process the byte stream data before parsing the byte stream data; and reduce repeated parsing configurations by caching for repeated parsing types or parsing configurations.

[0070] The embodiment of the present invention is a device embodiment corresponding to the above method embodiment. The specific operations of each module can be understood by referring to the description of the method embodiment, which will not be repeated here.

[0071] Device Example 2

[0072] An embodiment of the present invention provides an electronic device, such as Figure 8 As shown, it includes: a memory 80, a processor 82, and a computer program stored in the memory 80 and executable on the processor 82. When the computer program is executed by the processor 82, the steps described in the method embodiment are implemented.

[0073] Device Example 3

[0074] An embodiment of the present invention provides a computer-readable storage medium, on which a program for implementing information transmission is stored. When the program is executed by the processor 82, the steps described in the method embodiment are implemented.

[0075] The computer-readable storage medium in this embodiment includes, but is not limited to, ROM, RAM, magnetic disk or optical disk, etc.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A byte stream data parsing method based on deserialization, characterized in that: include: Define a general parsing interface, load an XML parsing configuration file, parse the definition in the XML parsing configuration file, and automatically construct a DataFormat object based on the deserialization of the parsed definition, wherein the DataFormat object includes multiple Package objects, each Package object includes multiple Zones, and each Zone contains a specific parsing type instance; wherein parsing the definition in the XML parsing configuration file specifically includes: parsing all possible Package types and their corresponding XML tags, defining multiple Zones in each Package, each Zone contains a specific parsing type and its configuration, and using XML tags to represent parsers of different parsing types and necessary parameters for configuration; Obtain the parsed byte stream data through the universal parsing interface, identify the type of the current data packet according to the previous bytes of the byte stream data through the DataFormat object, and find the corresponding Package object; traverse each Zone object in the Package object, and in each Zone object, execute the parsing logic in sequence according to the definition of the parsing type, and store or output the parsing result in a predetermined format; during the parsing process, if an unrecognizable data packet type or a parsing error is encountered, capture the exception and perform corresponding error handling, wherein the universal parsing interface includes: basic methods of all parsing types, wherein the basic methods of the parsing types specifically include: execution entry of the parsing logic, and the parsing types specifically include: basic parsing types, composite parsing types, and container types, wherein for multiple independent Zone objects or Package objects, multi-threading or multi-processing is used for parallel parsing.

2. The method according to claim 1, characterized in that In each Zone object, the parsing logic is executed in order according to the definition of the parsing type, including: For basic parsing types, directly call its Analysis method to parse the corresponding byte data; For compound parsing types, the corresponding sub-parsing types are called for parsing according to their internal logic.

3. The method according to claim 2, characterized in that The method further comprises: Before parsing the byte stream data, the byte stream data is pre-read and pre-processed.

4. The method according to claim 2, characterized in that: The method further comprises: For repetitive parsing types or parsing configurations, reduce duplicate parsing configurations through caching.

5. A byte stream data parsing device based on deserialization, characterized in that: include: Load a deserialization module, which is used to define a general parsing interface, load an XML parsing configuration file, parse the definition in the XML parsing configuration file, and automatically construct a DataFormat object by deserialization. The DataFormat object includes multiple Package objects, each Package object includes multiple Zones, and each Zone contains a specific parsing type instance; wherein the XML parsing configuration file specifically includes: all possible Package types and their corresponding XML tags, multiple Zones are defined in each Package, each Zone contains a specific parsing type and its configuration, and XML tags are used to represent parsers of different parsing types and necessary parameters for configuration; A parsing module is used to obtain parsed byte stream data through a universal parsing interface, identify the type of the current data packet according to the previous bytes of the byte stream data through a DataFormat object, and find the corresponding Package object; traverse each Zone object in the Package object, and in each Zone object, execute the parsing logic in sequence according to the definition of the parsing type, and store or output the parsing results in a predetermined format; during the parsing process, if an unrecognizable data packet type or a parsing error is encountered, the exception is captured and corresponding error handling is performed, wherein the universal parsing interface includes: basic methods of all parsing types, wherein the basic methods of the parsing types specifically include: execution entry of the parsing logic, and the parsing types specifically include: basic parsing types, composite parsing types, and container types, wherein for multiple independent Zone objects or Package objects, multi-threading or multi-processing is used for parallel parsing.

6. The device according to claim 5, characterized in that The analysis module is specifically used for: For basic parsing types, directly call its Analysis method to parse the corresponding byte data; For compound parsing types, the corresponding sub-parsing types are called for parsing according to their internal logic.

7. The device according to claim 5, characterized in that The parsing module is further used for: Before parsing the byte stream data, the byte stream data is pre-read and pre-processed.

8. The device according to claim 5, characterized in that The parsing module is further used for: For repetitive parsing types or parsing configurations, reduce duplicate parsing configurations through caching.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the deserialization-based byte stream data parsing method as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an implementation program for information transmission, and when the program is executed by a processor, the steps of the deserialization-based byte stream data parsing method according to any one of claims 1 to 4 are implemented.

Images