PCAP data parsing method based on serialization protocol, and apparatus and device

Abstract

The present invention relates to a PCAP data parsing method based on a serialization protocol, and an apparatus and a device. The method comprises: defining, to be message files, information exchanged between software of a fully automatic train supervision system, and using a compilation tool to compile the defined message files into source code files; defining a message dictionary, which is used for storing the message names and message types of the message files; capturing the source code files and saving same as PCAP data packets, deserializing, on the basis of the defined dictionary, application-layer data of each data packet among the PCAP data packets so as to generate a deserialized byte string, and storing or transmitting the deserialized byte string in text form; and reconstructing the PCAP data packets, so as to implement the deserialization parsing and visual analysis of PCAP data. Compared with the prior art, the present invention has the advantages of implementing PCAP data parsing based on a serialization protocol, improving the readability and maintainability of PCAP data, etc.

Description

PCAP data parsing method, apparatus and equipment based on serialization protocol Technical Field

[0001] This invention relates to the field of network technology, and in particular to a PCAP data parsing method, apparatus, and device based on a serialization protocol. Background Technology

[0002] Serialization protocols refer to the process of transforming data structures or object states into a form that can be stored or transmitted. During serialization, an object is broken down into a series of bytes, which can be reassembled in a specific order to reconstruct the object. This process ensures the integrity and consistency of the object, allowing it to be correctly reconstructed across different systems or platforms.

[0003] PCAP (Packet Capture) is a common network data capture format that allows users to capture network packets in their raw, unmodified form. It's implemented at the operating system kernel level, enabling the capture of network packets and performing various operations such as filtering and deep analysis. A PCAP file is a file format in which network packets captured using PCAP technology are stored. PCAP file parsing technology involves reading and analyzing PCAP files to extract the network packet information they contain. It is widely used in network communication, security auditing, protocol analysis, and other fields.

[0004] Due to the simplicity and scalability of serialization protocols, fully automated train monitoring systems currently use efficient data serialization for data transmission between software. However, it is currently impossible to read and parse PCAP data files captured by various software programs based on serialization protocols. This limits the understanding and analysis of data information between software programs in the fully automated train monitoring system. Therefore, developing a PCAP data parsing method based on serialization protocols is of great significance.

[0005] How to achieve PCAP data parsing based on serialization protocols has become a technical problem that needs to be solved.

[0006] Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art by providing a PCAP data parsing method, apparatus, and device based on a serialization protocol.

[0008] The objective of this invention can be achieved through the following technical solutions:

[0009] According to one aspect of the present invention, a PCAP data parsing method based on a serialization protocol is provided, the method comprising the following steps:

[0010] Step S1: Define the information exchanged between the software of the fully automatic train monitoring system as message files, and use a compilation tool to compile the defined message files into source code files;

[0011] Step S2: Define a message dictionary to store the message names and message types of the message files in step S1;

[0012] Step S3: Capture the source code file from step S1 and save it as a PCAP data packet. Based on the dictionary defined in step S2, deserialize the application layer data of each data packet in the PCAP data packet to generate a deserialized byte string, and store or transmit it in text form.

[0013] Step S4: Reconstruct the PCAP data packet to achieve parsing and visualization analysis of PCAP data deserialization.

[0014] Preferably, the source code file is used to generate message classes and for encoding / decoding.

[0015] More preferably, the encoding and decoding specifically involves converting a message object into binary data or converting binary data into a message object.

[0016] Preferably, in step S2, the key of the dictionary is the ID of the message, and the value of the dictionary is the type of the message.

[0017] Preferably, step S3 includes:

[0018] a) Read the global header of the PCAP data packet;

[0019] b) Traverse the header and content of each packet in the PCAP data packet;

[0020] c) Obtain the data content of each data packet, extract the number of bytes that the application layer needs to deserialize and the corresponding message type in each data packet according to the dictionary defined in step S2, and use them as parameters to call the deserialization function to perform the deserialization operation.

[0021] More preferably, the process of calling the deserialization function to perform the deserialization operation includes:

[0022] First, obtain the message class from the source code file in step S1;

[0023] Next, an empty message object is created using the message class, and the byte string that needs to be deserialized is parsed into the properties of the message object by using the codec function in the message class;

[0024] Then, iterate through each field and its corresponding value in the message object and concatenate them as strings;

[0025] Finally, the concatenated string is encoded into ASCII code and stored or transmitted as text.

[0026] More preferably, the global header of the PCAP data packet is read in binary mode.

[0027] Preferably, the process of reconstructing the PCAP data packet includes:

[0028] Create a new PCAP packet object to store the parsed PCAP data;

[0029] The global header of the PCAP data packet, the header of each data packet, and the byte string deserialized from the application layer are concatenated and written into a new PCAP data packet object to form a new PCAP data packet.

[0030] According to another aspect of the present invention, a PCAP data parsing apparatus based on a serialization protocol is provided. The apparatus includes a message file compilation module, a deserialization module, a PCAP packet reconstruction module, and a PCAP data parsing module connected in sequence.

[0031] Preferably, the message file compilation module is used to define the information exchanged between software during the operation of the fully automatic train monitoring system as message files, and to use a compilation tool to compile the defined message files into source code files.

[0032] Preferably, the deserialization module reads the global header of the PCAP data packet in binary mode, traverses the header and content of each data packet, and calls the deserialization function to parse the number of bytes that the application layer needs to deserialize in each data packet and the corresponding message type, storing or transmitting it in text form.

[0033] Preferably, the reconstructed PCAP packet module is used to reconstruct PCAP packets, that is, to concatenate the global header of the PCAP data packet, the header of each data packet, and the byte string deserialized by the application layer as the data packet content and write them into a new PCAP file.

[0034] Preferably, the PACP data parsing module is used to parse the PCAP data deserialization and perform visualization analysis.

[0035] According to a third aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described thereon.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1) In the design phase, this invention defines the information exchanged between software in the fully automatic train monitoring system as message files. Combined with a defined dictionary for storing message names and message types, by deserializing the PCAP application layer data of the serialization protocol and reconstructing the PCAP data packets, the parsed PCAP data packets contain both underlying network protocol information and readable application layer data. Compared with the prior art, reconstructing the PCAP data packets makes parsing and analyzing PCAP data more convenient and efficient, while also improving data reliability and security.

[0038] 2) This invention includes the implementation of deserialization technology, which can restore serialized data to the original data; compared with the prior art, the implementation of deserialization technology can improve the readability and maintainability of data.

[0039] 3) This invention enables the visualization analysis of PCAP data, allowing users to analyze and understand PCAP data more intuitively; compared with existing technologies, the implementation of visualization analysis can improve the efficiency and accuracy of data analysis. Attached Figure Description

[0040] Figure 1 is a schematic diagram of the process for generating source code files in this invention;

[0041] Figure 2 is a flowchart illustrating the PCAP data parsing method of this invention;

[0042] Figure 3 is a schematic diagram of the deserialization process in this invention;

[0043] Figure 4 is a schematic diagram of the original PCAP data packet in this invention;

[0044] Figure 5 is a schematic diagram of the reconstructed PCAP data packet in this invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0046] This embodiment relates to a PCAP data parsing method based on a serialization protocol. The method deserializes the application layer data and then rewrites the PCAP data packet while keeping the message structure unchanged. The application layer data is replaced with deserialized data to form a new PCAP packet, thereby solving the problem of unreadable PCAP data packets in the original fully automatic train monitoring system and realizing the parsing of application layer data.

[0047] As shown in Figure 2, the method includes the following steps:

[0048] Step S1: First, define the information exchanged between software during the operation of the fully automatic train monitoring system as message files. Use a compilation tool to compile the defined message files into source code files. These source code files provide functions for generating message classes and codecs, and can convert message objects into binary data or binary data into message objects.

[0049] Step S2: Define a message dictionary to store message names and message types. The key of the dictionary is the ID value of all messages in the message file, and the ID value is the message type in the message file. The message ID is used as the key of the dictionary, and the message type is used as the value of the dictionary.

[0050] Step S3: The PCAP data packet consists of three parts: Global Header, Packet Header, and Packet Data. The global header of the original PCAP data packet is read in binary mode. The global header contains global information for each PCAP file, including the following fields:

[0051] Magic Number: This is a marker indicating the beginning of a file and is used to identify the file and its byte order. It can be 0xa1b2c3d4 or 0xd4c3b2a1, representing big-endian mode and raw byte order, respectively.

[0052] Major and Minor version numbers: These indicate the version information of the current file.

[0053] ThisZone (Local Time Zone): If GMT is used, this value is 0.

[0054] SigFigs (timestamp precision): Indicates the precision of the timestamp.

[0055] SnapLen (Maximum Storage Length): Sets the maximum length of the captured data packets.

[0056] LinkType: Indicates the link type, such as Ethernet.

[0057] Step S4: Traverse the header and content of each original PCAP data packet. The header of a data packet typically contains the following structure: 1. Timestamp, indicating the timestamp information of the captured data packet, usually with precision at the second and microsecond level; 2. Data packet length, indicating the actual length of the data packet, including the sum of the header and body; 3. Capture length, indicating the actual length of the data packet captured and stored in the PCAP file, usually the capture length is less than or equal to the data packet length; 4. Uncaptured length, indicating the length of data packets not captured in the PCAP file. If the capture length is equal to the data packet length, the uncaptured length is 0.

[0058] Step S5: Obtain the data content of each original PCAP data packet. Based on the defined message dictionary, extract the number of bytes that the application layer needs to deserialize in each data packet and the corresponding message type, and use these as parameters to call the deserialization function.

[0059] Step S6: This step describes the role of the deserialization function. First, it retrieves the message class from the source code file defined in Step 1. An empty message object is created using this message class, and the codec function in the message class is used to parse the byte string to be deserialized into the attributes of the message object. Then, each field and its corresponding value in the message object are iterated over, and they are concatenated as strings. Finally, the resulting string is encoded into ASCII code for storage or transmission as text. Step S7: A new PCAP packet object is created to store the parsed PCAP data.

[0060] Step S8: Concatenate the global header of the PCAP data packet, the header of each data packet, and the byte string deserialized from the application layer as the data packet content and write them into a new PCAP file to form a new PCAP data packet.

[0061] Step S9: Open the newly generated PCAP data packet to perform PCAP data deserialization parsing and visualization analysis.

[0062] This embodiment also relates to a PCAP data parsing method based on a serialization protocol. This embodiment will be described in detail with reference to a single message in PCAP.

[0063] Step S101, as shown in Figure 1, generates the corresponding message class according to the message definition:

[0064] / / Define a message to represent a control or status request.

[0065] Message

[0066] {

[0067] / / Optional field, unique identifier for the message

[0068] optional MSGID msg_id = 1; / / MSGID field, used to uniquely identify the message.

[0069] / / Optional field, identifier of RTU (Remote Terminal Unit)

[0070] optional int32 rtu_id = 2; / / RTU_ID field, representing the identifier of the remote terminal unit.

[0071] / / Optional field, control mode status, indicating the current control mode status.

[0072] optional int32 control_mode_status = 3; / / The CONTROL_MODE_STATUS field indicates the current control mode status.

[0073] / / Optional field, the control mode type of the request

[0074] optional CONTROL_MODE_TYPE control_mode_request_status = 4; / / The CONTROL_MODE_REQUEST_STATUS field indicates the control mode type requested.

[0075] }

[0076] / / Define an enumeration type to represent different control mode request types

[0077] enum CONTROL_MODE_TYPE

[0078] {

[0079] / / No request for control mode change

[0080] NOT_REQUEST = 0; / / A value of 0 indicates that no control mode change has been requested.

[0081] / / Center requests local control

[0082] CENTER_REQUEST_LOCAL_CONTROL = 1; / / A value of 1 indicates that the central request is to hand over control to the local system.

[0083] / / Center request center control

[0084] CENTER_REQUEST_CENTER_CONTROL = 2; / / A value of 2 indicates that the center requests to retain or regain control.

[0085] / / Local requests and local control

[0086] LOCAL_REQUEST_LOCAL_CONTROL = 3; / / A value of 3 indicates that the local request retains control.

[0087] / / Local request center control

[0088] LOCAL_REQUEST_CENTER_CONTROL = 4; / / A value of 4 indicates that the local request will relinquish control to the center.

[0089] }

[0090] Step S102, as shown in Figure 2, import the message file defined in step S1, traverse the message queue, compile each message file taken from the queue, and generate source code file.

[0091] Step S103: Use the message ID from step S101 as the key of the dictionary and the message type as the value of the dictionary.

[0092] Step S104: Capture the transmitted source code file using a packet capture tool (such as Wireshark) to obtain the read PCAP data packet (here referred to as the original PCAP data packet). Assume that the original PCAP packet contains only one message, and this message is the message listed in step S101. Assign the global header of the read original PCAP data packet to the global header (global_header), and write the global_header to the new PCAP file;

[0093] The original PCAP data is shown in Figure 4.

[0094] Step S105: Obtain the header and content of this data packet, extract the number of bytes that the application layer needs to deserialize and the corresponding message type, and use them as parameters to call the deserialization function;

[0095] Step S106, as shown in Figure 3, involves the deserialization function first retrieving the message class from the module defined in Step 1. Using this message class, an empty message object is created, and the codec function is called to parse the byte string to be deserialized into the attributes of the message object. Then, each field and its corresponding value in the message object is iterated over, and they are concatenated as strings. Finally, the resulting string is encoded into ASCII code.

[0096] Step S107: Create a new PCAP package object;

[0097] Step S108: Concatenate the global header, data packet header, and application layer deserialized byte string and write them into a new PCAP file;

[0098] Step S109: Use the Wireshark analysis tool to open the generated PCAP packet and parse out the serialized PCAP data as shown in Figure 5.

[0099] This embodiment also relates to a PCAP data parsing device based on a serialization protocol. The device includes a message file compilation module, a deserialization module, a PCAP packet reconstruction module, and a PCAP data parsing module connected in sequence.

[0100] in:

[0101] The message file compilation module is used to define the information exchanged between software during the operation of the fully automatic train monitoring system as message files, and to use compilation tools to compile the defined message files into source code files.

[0102] The deserialization module reads the global header of the original PCAP data packet in binary mode, traverses the header and content of each data packet, and parses the number of bytes that the application layer needs to deserialize and the corresponding message type in each data packet by calling the deserialization function, storing or transmitting it in text form.

[0103] The PCAP packet reconstruction module is used to reconstruct PCAP packets, which involves concatenating the global header of the PCAP data packets, the header of each data packet, and the application-layer deserialized byte strings as the data packet content and writing them into a new PCAP file.

[0104] The PACP data parsing module is used to parse and visualize PCAP data deserialization.

[0105] This embodiment also relates to an electronic device, including a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0106] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0107] The processing unit executes the various methods and processes described above, such as methods S1 to S9. For example, in some embodiments, methods S1 to S9 may be implemented as computer software programs tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of methods S1 to S9 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S9 by any other suitable means (e.g., by means of firmware).

[0108] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0109] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0110] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0111] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A PCAP data parsing method based on a serialization protocol, characterized in that, The method comprises the following steps: Step S1, defining information exchanged between software of the full-automatic train monitoring system as a message file, and using a compiling tool to compile the defined message file into a source code file; Step S2, defining a message dictionary for storing the message name and message type of the message file in step S1; Step S3, capturing the source code file in step S1 and saving it as a PCAP data packet, deserializing the application layer data of each data packet in the PCAP data packet according to the message dictionary defined in step S2, generating a deserialized byte string, and storing or transmitting the deserialized byte string in a text form; Step S4, reconstructing the PCAP data packet to realize the parsing and visual analysis of the PCAP data deserialization.

2. The PCAP data parsing method based on a serialization protocol according to claim 1, characterized in that, The source code file is used to generate a message class and a coding and decoding.

3. The PCAP data parsing method based on a serialization protocol according to claim 2, characterized in that, The coding and decoding specifically refers to converting a message object into binary data or converting binary data into a message object.

4. The PCAP data parsing method based on serialization protocol according to claim 1, characterized in that, In step S2, the key of the dictionary is the ID of the message, and the value of the dictionary is the type of the message.

5. The PCAP data parsing method based on serialization protocol according to claim 1, characterized in that, The process of step S3 comprises: a) reading the global header of the PCAP data packet; b) traversing the packet header and packet content of each data packet in the PCAP data packet; c) obtaining the data content of each data packet, taking the number of bytes of the application layer that need to be deserialized and the corresponding message type in each data packet as parameters according to the dictionary defined in step S2, and calling a deserialization function to perform a deserialization operation.

6. The PCAP data parsing method based on a serialization protocol according to claim 5, characterized in that, The process of calling the deserialization function to perform the deserialization operation comprises: First, obtaining the message class from the source code file in step S1; Next, creating an empty message object using the message class, and parsing the byte string that needs to be deserialized into the properties of the message object by using the coding and decoding function in the message class; Then, traversing each field and corresponding value in the message object, and concatenating them in a string form; Finally, encoding the concatenated result string into ASCII code and storing or transmitting it in a text form.

7. The PCAP data parsing method based on a serialization protocol according to claim 5, characterized in that, The global header of the PCAP data packet is read in a binary mode.

8. The PCAP data parsing method based on serialization protocol according to claim 1, characterized in that, The process of reconstructing the PCAP data packet comprises: Creating a new PCAP packet object for storing the parsed PCAP data; Concatenating and writing the global header of the PCAP data packet, the packet header of each data packet, and the byte string deserialized in the application layer as the packet content into the new PCAP packet object to form a new PCAP data packet.

9. An apparatus for parsing PCAP data based on a serialization protocol according to the method of claim 1, characterized in that, The device comprises a message file compiling module, a deserialization module, a PCAP packet reconstructing module, and a PACP data parsing module connected in sequence.

10. The apparatus of claim 9, wherein, The message file compiling module is used to define the information exchanged between software of the full-automatic train monitoring system during operation as a message file, and to compile the defined message file into a source code file using a compiling tool.

11. The apparatus of claim 9, wherein, The reverse serialization module reads the global header of the PCAP data packet in binary mode, traverses the packet header and packet content of each data packet, calls a reverse serialization function to parse the number of bytes in each data packet that need to be reverse serialized in the application layer and the corresponding message type, and stores or transmits in text form.

12. The apparatus of claim 9, wherein, The reconstruction PCAP packet module is used for reconstructing the PCAP packet, that is, splicing and writing the global header of the PCAP data packet, the packet header of each data packet and the byte string of the application layer reverse serialization as the packet content into a new PCAP file.

13. The apparatus of claim 9, wherein, The PACP data analysis module is used for realizing the reverse serialization analysis of the PCAP data and can be used for visual analysis.

14. An electronic device comprising a memory and a processor, said memory having stored thereon a computer program, characterized in that, The processor implements the method in any one of claims 1-8 when executing the program.

Images