Deserialization-based byte stream data parsing method and device

Abstract

The embodiment of the present specification provides a byte stream data analysis method and device based on deserialization, wherein the method comprises the following steps: defining a general analysis interface, loading an XML analysis configuration file, and automatically constructing a DataFormat object based on the XML analysis configuration file; obtaining byte stream data to be analyzed through the general analysis interface, identifying the data type of the byte stream data through the DataFormat object, and analyzing the byte stream data through the general analysis interface according to the data type by calling corresponding analysis logic. The embodiment of the present specification improves the scalability and maintainability of the byte stream message analysis function of the signal system ground subsystem maintenance software.

Description

TECHNICAL FIELD

[0001] The present document relates to the technical field of computers, and in particular to a byte stream data analysis method and device based on deserialization. BACKGROUND

[0002] The ground subsystem application software (hereinafter referred to as application software) of a subway signal system periodically generates a large amount of state information that needs to be saved during operation. These information is not directly stored in the disk of an industrial computer, but is generally sent to maintenance software for processing through Ethernet, that is, the application software needs to reduce the dimensionality of structured state information to a one-dimensional byte stream array and send it to the maintenance software through UDP. The maintenance software receives the byte stream information and parses it according to the state information interface. The common practice is to define a data structure based on the size and meaning of each field defined in the interface, and implement the corresponding parsing code. Once the state information to be transmitted changes, the parsing code needs to be changed accordingly. In order to facilitate maintenance and expansion, and reduce the changes of the parsing code, the interface definition and parsing logic need to be separated to enhance the extensibility and maintainability of the ground subsystem maintenance software parsing message function.

[0003] XML (Extensible Markup Language) is a markup language used to mark the structure of electronic files. XML can be used to represent any structured information, such as text, data, graphics, etc., and can be easily parsed and processed by computer programs. It has the characteristics of strong extensibility, structured, strong readability and cross-platform. XML deserialization refers to the process of converting an XML document into an object or data structure. After XML deserialization processing, different interfaces are parsed using a general parsing logic. SUMMARY

[0004] The purpose of the present application is to provide a byte stream data analysis method and device based on deserialization, which aims to solve the above-mentioned problems in the prior art.

[0005] The present application provides a byte stream data analysis method based on deserialization, comprising:

[0006] Defining a general parsing interface, loading an XML parsing configuration file, and automatically building a DataFormat object based on the XML parsing configuration file through deserialization;

[0007] Obtaining byte stream data to be parsed through the general parsing interface, identifying the data type of the byte stream data through the DataFormat object, and parsing the byte stream data through the general parsing interface according to the data type by calling the corresponding parsing logic.

[0008] The present application provides a byte stream data analysis device based on deserialization, comprising:

[0009] The loading deserialization module is used for defining a general parsing interface, loading an XML parsing configuration file, and deserializing to automatically build a DataFormat object based on the XML parsing configuration file;

[0010] The parsing module is used for obtaining the byte stream data with parsing through the general parsing interface, identifying the data type of the byte stream data through the DataFormat object, and calling corresponding parsing logic through the general parsing interface according to the data type to parse the byte stream data.

[0011] The embodiment of the present application also provides an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, when the computer program is executed by the processor, the steps of the deserialization-based byte stream data parsing method are implemented.

[0012] The embodiment of the present application also provides a computer readable storage medium, and the computer readable storage medium stores an information transmission implementation program, when the program is executed by a processor, the steps of the deserialization-based byte stream data parsing method are implemented.

[0013] By adopting the embodiment of the present application, the extensibility and maintainability of the byte stream message parsing function of the signal system ground subsystem maintenance software are improved. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the one or more embodiments of the present application or the prior art, the drawings needed to be used in the embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0015] Figure 1 is a flow chart of the deserialization-based byte stream data parsing method of the embodiment of the present application;

[0016] Figure 2 is a flow chart of the detailed processing of the deserialization-based byte stream data parsing method of the embodiment of the present application;

[0017] Figure 3 is a schematic diagram of an xml configuration file writing example of the embodiment of the present application;

[0018] Figure 4 is a schematic diagram of ATS-ZC input information parsing of the embodiment of the present application;

[0019] Figure 5Fig. 1 is a schematic diagram of an ATS-ZC input parsing structured data according to an embodiment of the present application;

[0020] Figure 6 Fig. 2 is a schematic diagram of a train application example according to an embodiment of the present application;

[0021] Figure 7 Fig. 3 is a schematic diagram of a deserialization-based byte stream data parsing device according to an embodiment of the present application;

[0022] Figure 8 Fig. 4 is a schematic diagram of an electronic device according to an embodiment of the present application;

[0023] Figure 9 Fig. 5 is a schematic diagram of processing steps of a deserialization-based byte stream data parsing method according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the technical personnel in the art better understand the technical solutions in the one or more embodiments of the present specification, the technical solutions in the one or more embodiments of the present specification will be clearly and completely described below in conjunction with the drawings in the one or more embodiments of the present specification. Obviously, the described embodiments are only a part of the embodiments of the present specification, not all embodiments. Based on the one or more embodiments of the present specification, all other embodiments obtained by those of ordinary skill in the art without creative labor should belong to the protection scope of the present document.

[0025] Method embodiments

[0026] According to an embodiment of the present application, a deserialization-based byte stream data parsing method is provided, Figure 1 Fig. 6 is a flowchart of a deserialization-based byte stream data parsing method according to an embodiment of the present application, as Figure 1 shown, the deserialization-based byte stream data parsing method according to an embodiment of the present application specifically includes:

[0027] Step S101, defining a general parsing interface, loading an XML parsing configuration file, parsing the definitions in the XML parsing configuration file, deserializing and automatically building a DataFormat object, the DataFormat object including a plurality of Package objects, each Package object including a plurality of Zones, and each Zone containing specific parsing type instances; wherein the XML parsing configuration file specifically includes: all possible Package types and their corresponding XML tags, defining a plurality of Zones in each Package, each Zone containing specific parsing types and their configurations, using XML tags to represent different parsing type parsers and necessary configuration parameters;

[0028] In step S102, the byte stream data with parsing is acquired through a general parsing interface, the type of the current data packet is recognized according to the front bytes of the byte stream data through a DataFormat object, and a corresponding Package object is found; each Zone object in the Package object is traversed, in each Zone object, the parsing logic is executed according to the definition of the parsing type in sequence, and the parsing result is stored or output in a predetermined format; in the parsing process, if an unrecognizable data packet type or a parsing error is encountered, an exception is captured and corresponding error processing is performed, wherein the general parsing interface includes basic methods of all parsing types, the basic methods of the parsing types specifically include an execution entrance of the parsing logic, the parsing types specifically include a basic parsing type, a composite parsing type and a container type, and for multiple independent Zone objects or Package objects, multi-threading or multi-processing is used for parallel parsing.

[0029] For the basic parsing type, the Analysis method thereof is directly called to parse the corresponding byte data; for the composite parsing type, corresponding sub-parsing types are called for parsing according to the internal logic thereof.

[0030] In the parsing process, if an unrecognizable data packet type or a parsing error is encountered, an exception is captured and corresponding error processing is performed.

[0031] In the embodiment of the application, in order to optimize the processing flow, the following optimization operations can be performed: before the parsing of the byte stream data is performed, the byte stream data is pre-read and pre-processed; for a repeatedly occurring parsing type or parsing configuration, the repeated parsing configuration is reduced through caching.

[0032] As can be seen from the above technical solution, in the embodiment of the application, the data structure of the metro signal system application software state information is defined in XML, the byte stream parsing rule defined by the interface is matched, and the basic parsing types used for inter-system communication, such as bit (bit), byte (byte), ushort (short integer), uint (integer), long (integer number), Debugchar (string) and the like, are contained. It is also necessary to define and display the composite parsing types such as loop and selection. The basic parsing types and the composite parsing types are comprehensively used to realize the parsing of complex data structures. Finally, a parsing program combination configuration file can parse any type of report byte stream, and the scalability and maintainability of the parsing are effectively improved.

[0033] The above technical solution of the embodiment of the application will be described in detail below with reference to the accompanying drawings.

[0034] The embodiment of the application adopts object-oriented software development method, and constructs a flexible and extensible byte stream message analysis system by defining a general analysis interface (IItem) and implementation classes thereof. The analysis program architecture design mainly includes the following key parts:

[0035] General analysis interface (IItem): defines basic methods of all analysis types, including an execution entry of analysis logic (such as an Analysis method).

[0036] Basic analysis types: such as Bit, Byte, Ushort, Uint, etc., which are directly derived from the Item base class to implement specific data analysis logic.

[0037] Composite analysis types: such as ItemIf, ItemFor, ItemSwitch, etc., which internally contain multiple sub-analysis types, and combine the sub-analysis types to perform analysis according to specific logic (such as conditional judgment, loop, selection).

[0038] Container types: such as Zone, Package, DataFormat, used for organizing and managing different analysis types to realize data partition management and automatic matching analysis.

[0039] Table 1 general analysis interface design

[0040]

[0041] Table 2 basic analysis type design

[0042]

[0043] Table 3 composite analysis type design

[0044]

[0045]

[0046] The execution flow of the analysis program is shown in Figure 2 and Figure 9 , and the application scenario can be the scenario shown in Figure 6 . Specifically, the following processing is included:

[0047] (1) Initialize analysis configuration: load the XML analysis configuration file, analyze the definitions in the configuration file, and deserialize to automatically build a DataFormat object, which contains multiple Package objects, each Package object contains multiple Zones, and each Zone contains specific analysis type instances.

[0048] For example, Figure 3As shown, the writing of the parsing configuration file is based on XML format, mainly including the following parts: DataFormat definition: lists all possible Package types and their corresponding XML tags. Package definition: defines multiple Zones within each Package, each Zone containing specific parsing types and their configurations (such as starting position, byte length, etc.). Parsing type definition: uses XML tags to represent different types of parsers (basic parsing type or composite parsing type) and configures necessary parameters.

[0049] (2) Read byte stream data: obtain the byte stream data to be parsed from the network or other data sources. ATS-ZC input information parsing is as shown in Figure 4 . ATS-ZC input parsing structured data is as shown in Figure 5 .

[0050] (3) Identify packet type: the DataFormat object first identifies the type of the current data packet according to the first few bytes of the byte stream (usually the header information of the data frame), and finds the corresponding Package.

[0051] (4) Execute parsing logic:

[0052] a) Traverse each Zone within the Package.

[0053] b) In each Zone, according to the definition of the parsing type (basic parsing type or composite parsing type), execute the parsing logic in order.

[0054] i. For basic parsing types, directly call their Analysis methods to parse the corresponding byte data.

[0055] ii. For composite parsing types, according to their internal logic (such as conditional judgment, loop, etc.), call the corresponding sub-parsing types for parsing.

[0056] c) Store or return the parsing results in the predetermined format.

[0057] (5) Error handling: if an unrecognized packet type or parsing error is encountered during parsing, the program should be able to catch exceptions and perform corresponding error handling.

[0058] (6) Result output: output the parsing results in a structured form for use by the upper layer application.

[0059] In the embodiment of the present application, in order to improve the analysis performance, for multiple independent Zones or Packages, multi-threading or multi-processing can be used for parallel analysis. For repeated packet types or analysis configurations, a cache can be used to reduce the overhead of repeated analysis configurations. Before analysis, pre-reading and preprocessing of data are performed, such as calculating checksum, judging data integrity, etc., to reduce invalid analysis.

[0060] In summary, the technical scheme of the embodiment of the present application proposes a byte stream message analysis method based on deserialization, aiming to improve the analysis efficiency and maintainability of the ground subsystem maintenance software of the signal system. The method realizes flexible analysis of complex byte stream messages by defining a set of general analysis interfaces (IItem) and their implementation classes. The core is to separate the analysis logic from the data structure, and define the analysis rules of the message through an XML configuration file, thereby realizing the dynamic configurability of the interface definition.

[0061] In terms of technical implementation, the embodiment of the present application adopts an object-oriented software design method, designs basic analysis types (such as Bit, Byte, etc.) and composite analysis types (such as ItemIf, ItemFor, etc.), as well as container types (Zone, Package, DataFormat) to organize and manage the analysis logic. The analysis program constructs an analysis model by loading an XML configuration file, automatically matches and executes the corresponding analysis logic according to the input byte stream data, and finally restores the byte stream into structured data for use by the upper layer application.

[0062] To further optimize performance, the embodiment of the present application also introduces parallel analysis, cache mechanism, and pre-reading and preprocessing techniques, which significantly improve the efficiency and accuracy of message analysis. The change of the communication interface between systems requires only the change of the analysis configuration file, reducing the maintenance difficulty.

[0063] Device embodiment one

[0064] According to the embodiment of the present application, a byte stream data analysis device based on deserialization is provided, Figure 7 is a schematic diagram of the byte stream data analysis device based on deserialization of the embodiment of the present application, as Figure 7 shown, the byte stream data analysis device based on deserialization according to the embodiment of the present application specifically includes:

[0065] The loading deserialization module 70 is configured to define a general parsing interface, load an XML parsing configuration file, parse definitions in the XML parsing configuration file, and deserialize an automatically constructed DataFormat object, wherein the DataFormat object includes a plurality of Package objects, each of the Package objects includes a plurality of Zones, and each of the Zones contains a specific parsing type instance.

[0066] The parsing module 72 is configured to obtain byte stream data with parsing through the general parsing interface, identify a type of a current data packet according to a front byte of the byte stream data through the DataFormat object, find a corresponding Package object, traverse each Zone object in the Package object, execute parsing logic in each Zone object according to definitions of parsing types in sequence, and store or output a parsing result in a predetermined format, and capture an exception and perform corresponding error processing if an unrecognizable data packet type or a parsing error is encountered in the parsing process.

[0067] The parsing module 72 is specifically configured to directly call an Analysis method of a basic parsing type to parse corresponding byte data, and call corresponding sub-parsing types to perform parsing according to internal logic of a composite parsing type for the composite parsing type.

[0068] In the parsing process, if an unrecognizable data packet type or a parsing error is encountered, an exception is captured and corresponding error processing is performed.

[0069] The parsing module 72 is further configured to perform pre-reading and preprocessing on the byte stream data before performing parsing on the byte stream data, and reduce repeated parsing configurations through caching for repeated parsing types or parsing configurations.

[0070] The embodiment of the application is a device embodiment corresponding to the method embodiment, and the specific operations of each module can be understood with reference to the description of the method embodiment, which will not be described herein.

[0071] The device embodiment two

[0072] An electronic device is provided, comprising: a memory 80, a processor 82, and a computer program stored in the memory 80 and executable on the processor 82, wherein the computer program is executed by the processor 82 to implement the steps as described in the method embodiments. Figure 8

[0073] Device embodiment three

[0074] A computer readable storage medium is provided, wherein the computer readable storage medium stores an implementation program of information transmission, and the program is executed by the processor 82 to implement the steps as described in the method embodiments.

[0075] The computer readable storage medium described in the embodiments includes, but is not limited to, ROM, RAM, magnetic or optical disk, etc.

[0076] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.​

Claims

1. A method for parsing byte stream data based on deserialization, characterized in that, Comprise: Defining a general parsing interface, loading an XML parsing configuration file, parsing definitions in the XML parsing configuration file, deserializing an automatically constructed DataFormat object based on the parsed definitions, the DataFormat object including a plurality of Package objects, each Package object including a plurality of Zones, and each Zone containing specific parsing type instances; wherein parsing the definitions in the XML parsing configuration file specifically includes: parsing all possible Package types and their corresponding XML tags, defining a plurality of Zones in each Package, each Zone containing specific parsing types and their configurations, and using XML tags to represent different parsing type parsers and necessary configuration parameters; Obtaining byte stream data to be parsed through a general parsing interface, identifying the type of the current data packet according to the first bytes of the byte stream data through the DataFormat object, and finding the corresponding Package object; traversing each Zone object in the Package object, executing parsing logic in each Zone object according to the definition of the parsing type in order, and storing or outputting the parsing results in a predetermined format; if an unrecognized data packet type or parsing error is encountered during parsing, capturing exceptions and performing corresponding error handling, wherein the general parsing interface includes basic methods of all parsing types, and the basic methods of the parsing types specifically include: execution entry of parsing logic, and the parsing types specifically include: basic parsing types, composite parsing types, and container types, wherein for a plurality of independent Zone objects or Package objects, multi-threading or multi-processing is used for parallel parsing.

2. The method of claim 1, wherein, In each Zone object, the parsing logic is executed in order according to the definition of the parsing type, which specifically includes: For basic parsing types, directly call their Analysis method to parse the corresponding byte data; For composite parsing types, call the corresponding sub-parsing types for parsing according to their internal logic.

3. The method of claim 2, wherein, The method further comprises: Before parsing the byte stream data, pre-reading and preprocessing the byte stream data.

4. The method of claim 2, wherein, The method further comprises: For repeated parsing types or parsing configurations, reduce repeated parsing configurations through caching.

5. A deserialization-based byte stream data parsing apparatus, characterized by comprising: Comprise: The loading deserialization module is configured to define a general parsing interface, load an XML parsing configuration file, parse definitions in the XML parsing configuration file, and deserialize an automatically constructed DataFormat object including a plurality of Package objects, each of which includes a plurality of Zones, and each of which contains a specific parsing type instance. The parsing module is configured to obtain byte stream data to be parsed through the general parsing interface, identify the type of the current data packet according to the first bytes of the byte stream data through the DataFormat object, and find the corresponding Package object. Each Zone object in the Package object is traversed, and in each Zone object, the parsing logic is executed in order according to the definition of the parsing type, and the parsing result is stored or output in a predetermined format. During the parsing process, if an unrecognized data packet type or a parsing error is encountered, an exception is captured and corresponding error handling is performed. The general parsing interface includes basic methods of all parsing types, and the basic methods of the parsing types specifically include an execution entry of the parsing logic. The parsing types specifically include basic parsing types, composite parsing types, and container types. For a plurality of independent Zone objects or Package objects, multi-threading or multi-processing is used for parallel parsing.

6. The apparatus of claim 5, wherein, The parsing module is specifically configured to: For the basic parsing type, directly call its Analysis method to parse the corresponding byte data. For the composite parsing type, call the corresponding sub-parsing type for parsing according to its internal logic.

7. The apparatus of claim 5, wherein, The parsing module is further configured to: Before parsing the byte stream data, pre-reading and preprocessing the byte stream data.

8. The apparatus of claim 5, wherein, The parsing module is further configured to: For repeated parsing types or parsing configurations, reduce repeated parsing configurations through caching.

9. An electronic device, comprising: The computer program is stored on the memory and executable on the processor, and when the computer program is executed by the processor, the steps of the deserialization-based byte stream data parsing method according to any one of claims 1-4 are implemented. The computer readable storage medium stores an information transmission implementation program, and the program is executed by the processor to implement the steps of the deserialization-based byte stream data parsing method according to any one of claims 1-4.

10. A computer-readable storage medium, characterized in that, ​

Images