Data serialization and deserialization method

Abstract

The embodiment of the invention provides a data serialization and deserialization method. The data serialization and deserialization method comprises the following steps: designing a Protobuf data structure for describing an ROS message; mapping data of the ROS message into a Protobuf message structure, and serializing the data into a binary format; the method comprises the following steps: publishing serialized Probuf binary data through a topic of an ROS; and deserializing the data, namely deserializing the binary data into a Probuf message structure at a receiving end, and mapping the Probuf message structure back to an ROS message structure for processing. According to the method, the message transmission efficiency and the cross-platform compatibility in the ROS environment are improved, so that the ROS node can efficiently perform data transmission in the cross-platform and cross-language environment, and the quick serialization and deserialization of the ROS message are realized by converting the existing ROS message format into the Protobuf format and utilizing the characteristics of high efficiency and cross-platform of the Protobuf.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing and communication in a robot operating system, and in particular to a data serialization and deserialization method. Background Art

[0002] At present, the Robot Operating System (ROS) is a robot development framework widely used in the world, mainly used for the development and testing of robot software. One of the main functions of ROS is to achieve data exchange between different nodes through the message passing mechanism. The ROS custom message serialization mechanism is its core component, which is used to package and unpack the data transmitted between nodes.

[0003] In the current ROS, the communication between nodes mainly relies on its custom message format, which supports multiple message types, such as sensor_msgs, geometry_msgs, etc. During the data transmission process, the message content is encoded into a binary format through serialization, which is convenient for data transmission between different nodes. The serialization method of ROS is based on its internally defined message structure, which has good compatibility and flexibility for conventional message transmission. However, this custom serialization mechanism has performance bottlenecks when processing large-scale data, especially in systems with high-frequency data transmission and limited resources.

[0004] ROS's custom serialization mechanism provides developers with a flexible way of messaging, especially with good adaptability in inter-node communication of different types of data. However, with the expansion of application scenarios, the default serialization method of ROS has gradually shown some shortcomings in applications with high performance requirements. First, the mechanism has low transmission efficiency when processing large amounts of data (such as images, point clouds, etc.), resulting in increased communication delays. Secondly, due to the lack of cross-platform compatibility of the custom format, integration between different operating systems and programming languages ​​is more complicated. In addition, the existing serialization format is not ideal for resource-constrained environments such as embedded systems or distributed robot clusters, and there is a large resource occupation and data transmission overhead. Summary of the invention

[0005] In view of the above problems existing in the prior art, an embodiment of the present invention provides a data serialization and deserialization method, which improves the efficiency and cross-platform compatibility of message transmission in the ROS environment, and is particularly suitable for data exchange and remote communication of embedded systems, large-scale robot clusters and distributed control systems. The core of the method is to use Protobuf in the ROS system to serialize and deserialize data, so that ROS nodes can efficiently transmit data in a cross-platform and cross-language environment. By converting the existing ROS message format into the Protobuf format, the high efficiency and cross-platform characteristics of Protobuf are utilized to realize the rapid serialization and deserialization of ROS messages.

[0006] An embodiment of the present invention provides a data serialization and deserialization method, including:

[0007] Step 1: Design a Protobuf data structure for describing robot operating system messages;

[0008] Step 2: Map the data of the Robot OS message into the Protobuf message structure and serialize it into binary format;

[0009] Step 3: Publish the serialized Protobuf binary data through the Robot OS topic;

[0010] Step 4: Data deserialization, specifically, deserializing the binary data into a Protobuf message structure at the receiving end and mapping it back to the robot operating system message structure for processing.

[0011] In some embodiments of the present invention, in step 1, the method comprises:

[0012] A Protobuf message structure is defined in the image file to describe the robot operating system data that needs to be transmitted, wherein the Protobuf message includes multiple data types, and the multiple data types include at least sensor data and image data.

[0013] In some embodiments of the present invention, in step 2, the method comprises:

[0014] In the robot operating system node, after receiving the data, the data in the robot operating system message is filled into the corresponding Protobuf message structure, and the serialization function of Protobuf is called to convert it into binary data.

[0015] In some embodiments of the present invention, in step 2,

[0016] The data in the serialized Robot OS message can be published through the Robot OS topic.

[0017] In some embodiments of the present invention, in step 2,

[0018] The Robot OS node serialization process includes:

[0019] receiving robot operating system messages, including images or sensor data;

[0020] Create the corresponding Protobuf message object and map the field values ​​of the robot operating system message to the Protobuf message;

[0021] Call the Protobuf serialization function to serialize the Protobuf message into binary format;

[0022] Publish the binary data via the Robot OS topic.

[0023] In some embodiments of the present invention, in step 5, the method specifically includes:

[0024] After the receiving end subscribes to the topic, it receives serialized binary data;

[0025] Through the deserialization function of Protobuf, the binary data is restored to the Protobuf message object, and the field values ​​therein are extracted, wherein the field values ​​can be remapped back to the robot operating system message structure for use by the system.

[0026] In some embodiments of the present invention, in step 5,

[0027] The robot operating system node deserialization process includes:

[0028] Receive binary data from a Robot OS topic;

[0029] Call the Protobuf deserialization function to convert binary data into a Protobuf message;

[0030] Extract data from Protobuf messages and map back to Robot OS messages;

[0031] Use the restored Robot OS message for subsequent processing or display.

[0032] In some embodiments of the present invention, the data serialization and deserialization method further includes:

[0033] Set up publishing nodes and subscription nodes, where:

[0034] Setting up the publishing node includes, in the robot operating system, listening to sensor data or image data, serializing the data into Protobuf format, and publishing it through the robot operating system topic;

[0035] Setting up a subscription node includes subscribing to published messages, receiving Protobuf serialized binary data, deserializing the data and restoring the original Robot OS message structure for use.

[0036] Compared with the prior art, the existing robot operating system data serialization mechanism shows certain deficiencies in high-performance, cross-platform and resource-constrained application scenarios. First, the default serialization method of the robot operating system has low transmission efficiency when processing large amounts of data (such as images, lidar data, etc.), resulting in increased communication delays. Secondly, the serialization mechanism lacks compatibility in multi-platform integration, especially in the data transmission between different operating systems and programming languages. In addition, for environments such as embedded systems and distributed robot clusters, the existing serialization format will cause a large resource consumption, which is not conducive to the real-time and efficiency of the system. Therefore, the technical problem of the present invention is how to improve the efficiency and cross-platform compatibility of data serialization in the robot operating system, especially in scenarios with high-frequency data transmission and limited resources, and optimize the performance and flexibility of message transmission.

[0037] Compared with the existing robot operating system data serialization technology, the present invention brings the following beneficial effects: 1. Efficiency improvement: by using Protobuf for data serialization, the transmission efficiency of messages is significantly improved, especially in the scenario of large data volume transmission, the communication delay is reduced, and the real-time performance of the system is enhanced. 2. Cross-platform compatibility: the present invention supports multiple operating systems and programming languages, and can achieve cross-platform compatibility of data transmission without additional adaptation work, simplifying the integration work between different platforms. 3. Resource saving: by optimizing the embedded system, the consumption of computing resources and memory in the serialization and deserialization process is reduced, so that the present invention can also run efficiently in resource-constrained embedded devices; 4. Flexible scalability: developers can easily define and expand the robot operating system message format, and use the flexibility and scalability of Protobuf to adapt to different application requirements. 5. Data compression and efficient transmission: the present invention effectively saves bandwidth and improves data transmission efficiency by compressing large data volume messages, and is particularly suitable for application scenarios that are sensitive to bandwidth, such as robot remote monitoring and control. The present invention not only improves the efficiency and cross-platform compatibility of data transmission in the robot operating system, but also reduces resource consumption, adapts to a variety of application scenarios and hardware platforms, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1The present invention provides a flowchart of a method for serializing and deserializing data. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0040] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0041] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0042] It should also be understood that, although the present application has been described with reference to some specific examples, those skilled in the art will be able to realize many other equivalent forms of the present application that have the features described in the claims and are therefore within the scope of protection defined thereby.

[0043] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0044] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to determine the true intent based on the user's historical operations and to avoid unnecessary or redundant details that make the present application unclear. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but are merely used as the basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0045] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0046] The embodiment of the present invention provides a data serialization and deserialization method, such as Figure 1 As shown, the method includes:

[0047] Step 1: Design a Protobuf data structure for describing robot operating system messages; specifically, define the Protobuf message structure: Design a Protobuf data structure for describing ROS messages. First, you need to define the Protobuf message structure in the .proto file to describe the ROS data that needs to be transmitted. Protobuf messages can include a variety of data types, such as sensor data, image data, etc.

[0048] Step 2: Map the data of the robot operating system message to the Protobuf message structure and serialize it into a binary format; specifically, perform data serialization: map the data of the ROS message to the Protobuf message structure and serialize it into a binary format. In the ROS node, after receiving the data, we fill the data in the ROS message into the corresponding Protobuf message structure, and then call the serialization function of Protobuf to convert it into binary data. The serialized data can be published through the ROS topic. ROS node serialization process: 1. Receive ROS messages (for example, images or sensor data). 2. Create the corresponding Protobuf message object and map the field values ​​of the ROS message to the Protobuf message. 3. Call the serialization function of Protobuf to serialize the Protobuf message into a binary format. 4. Publish the binary data through the ROS topic.

[0049] Step 3: Perform data transmission and publish serialized Protobuf binary data through the ROS topic;

[0050] Step 4: Deserialize the data. Specifically, the receiver deserializes the binary data into a Protobuf message structure and maps it back to the ROS message structure for processing. After the receiver subscribes to the topic, it receives the serialized binary data. Through the deserialization function of Protobuf, the binary data can be restored to a Protobuf message object and the field values ​​can be extracted. Subsequently, these field values ​​can be remapped back to the ROS message structure for system use.

[0051] ROS node deserialization process: 1. Receive binary data in the ROS topic. 2. Call the Protobuf deserialization function to convert the binary data into a Protobuf message. 3. Extract data from the Protobuf message and map it back to the ROS message. 4. Use the restored ROS message for subsequent processing or display.

[0052] The settings of the publishing node and the subscription node are as follows:

[0053] Publishing node: In the ROS system, it listens to sensor data or image data, serializes the data into Protobuf format, and publishes it through ROS topics.

[0054] Subscriber node: subscribes to published messages and receives binary data serialized by Protobuf. Deserializes the data and restores the original ROS message structure for use.

[0055] To facilitate understanding of the above technical solution, the following is an explanation in conjunction with the specific specification drawings and specific examples, as follows:

[0056] First embodiment: Protobuf serialization implementation of standard ROS messages.

[0057] In the first embodiment of the present invention, for standard message types (such as sensor_msgs, geometry_msgs, etc.) in the ROS system, the efficiency of message transmission is improved by serializing and deserializing messages using Protobuf. The specific implementation steps are as follows:

[0058] Message definition conversion: Use the message definition conversion tool to convert the .msg format of ROS standard messages (such as sensor_msgs / Image) to the .proto format. The converted Protobuf message structure can be directly used for serialization processing.

[0059] Serialization process: When a ROS node sends an Image type message, the Protobuf serialization module serializes the message into a compact binary format and transmits it through the ROS communication mechanism.

[0060] Deserialization process: After receiving the serialized message, the receiving node uses the Protobuf deserialization module to restore it to a ROSImage type message for processing by other nodes.

[0061] In this way, this embodiment achieves efficient cross-platform message transmission in the ROS system and significantly reduces the transmission delay of large amounts of data (such as image data).

[0062] Second embodiment: Optimized large data volume message processing

[0063] In the second embodiment, the present invention introduces data compression and batch processing mechanisms for large data volume messages (such as point cloud data) in the ROS system, further optimizing the data transmission efficiency. The specific process is as follows:

[0064] Batch processing: When transmitting a large amount of point cloud data, the system will group the data into batches, and each batch of data will be compressed and transmitted after being serialized by Protobuf.

[0065] Compressed transmission: Use the GZIP algorithm to compress serialized data to reduce network bandwidth usage, which is especially suitable for low-bandwidth network environments.

[0066] Decompression and deserialization: The receiving end first decompresses the received batch data, and then uses the Protobuf module to deserialize the data to restore it to the standard ROS point cloud message format.

[0067] It can be seen from the above technical solutions that the data serialization and deserialization method provided by the above embodiments of the present invention has the following beneficial effects compared with the existing ROS data serialization technology: 1. Efficiency improvement: by using Protobuf for data serialization, the message transmission efficiency is significantly improved, especially in the scenario of large data volume transmission, the communication delay is reduced, and the real-time performance of the system is enhanced. 2. Cross-platform compatibility: the present invention supports a variety of operating systems and programming languages, and can achieve cross-platform compatibility of data transmission without additional adaptation work, which simplifies the integration work between different platforms. 3. Resource saving: by optimizing the embedded system, the consumption of computing resources and memory in the serialization and deserialization process is reduced, so that the present invention can also run efficiently in resource-constrained embedded devices; 4. Flexible scalability: developers can easily define and expand the ROS message format, and use the flexibility and scalability of Protobuf to adapt to different application requirements. 5. Data compression and efficient transmission: by compressing large data volume messages, the present invention effectively saves bandwidth and improves data transmission efficiency, which is particularly suitable for application scenarios that are sensitive to bandwidth, such as robot remote monitoring and control. The present invention not only improves the efficiency of data transmission and cross-platform compatibility in the ROS system, but also reduces resource consumption, adapts to a variety of application scenarios and hardware platforms, and has broad application prospects.

[0068] The above embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention. The protection scope of the present invention is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present invention, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.

Claims

1. A data serialization and deserialization method, characterized in that: include: Step 1: Design a Protobuf data structure for describing robot operating system messages; Step 2: Map the data of the Robot OS message into the Protobuf message structure and serialize it into binary format; Step 3: Publish the serialized Protobuf binary data through the Robot OS topic; Step 4: Data deserialization, specifically, deserializing the binary data into a Protobuf message structure at the receiving end and mapping it back to the robot operating system message structure for processing.

2. The data serialization and deserialization method according to claim 1, characterized in that: In step 1, the method comprises: A Protobuf message structure is defined in the image file to describe the robot operating system data that needs to be transmitted, wherein the Protobuf message includes multiple data types, and the multiple data types include at least sensor data and image data.

3. The data serialization and deserialization method according to claim 2, characterized in that: In step 2, the method comprises: In the robot operating system node, after receiving the data, the data in the robot operating system message is filled into the corresponding Protobuf message structure, and the serialization function of Protobuf is called to convert it into binary data.

4. The data serialization and deserialization method according to claim 3, characterized in that: In step 2, The data in the serialized Robot OS message can be published through the Robot OS topic.

5. The data serialization and deserialization method according to claim 4, characterized in that: In step 2, The Robot OS node serialization process includes: receiving robot operating system messages, including images or sensor data; Create the corresponding Protobuf message object and map the field values ​​of the robot operating system message to the Protobuf message; Call the Protobuf serialization function to serialize the Protobuf message into binary format; Publish the binary data via the Robot OS topic.

6. The data serialization and deserialization method according to claim 5, characterized in that: In step 5, the method specifically includes: After the receiving end subscribes to the topic, it receives serialized binary data; Through the deserialization function of Protobuf, the binary data is restored to the Protobuf message object, and the field values ​​therein are extracted, wherein the field values ​​can be remapped back to the robot operating system message structure for use by the system.

7. The data serialization and deserialization method according to claim 6, characterized in that: In step 5, The robot operating system node deserialization process includes: Receive binary data from a Robot OS topic; Call the Protobuf deserialization function to convert binary data into a Protobuf message; Extract data from Protobuf messages and map back to Robot OS messages; Use the restored Robot OS message for subsequent processing or display.

8. The data serialization and deserialization method according to claim 7, characterized in that: Also includes: Set up publishing nodes and subscription nodes, where: Setting up the publishing node includes, in the robot operating system, listening to sensor data or image data, serializing the data into Protobuf format, and publishing it through the robot operating system topic; Setting up a subscription node includes subscribing to published messages, receiving Protobuf serialized binary data, deserializing the data and restoring the original Robot OS message structure for use.

Images