Recording and playback method and system for sensor data in automatic driving

Abstract

The invention relates to a recording and playback method and system for sensor data in automatic driving, and the method comprises the steps: transmitting sensor data msg collected by each sensor to an upper computer, and triggering a recording function when the upper computer receives a frame of sensor data msg; the size maxmsgsize of the sensor data msg is defined based on the type of the sensor data msg, and a message header BAGHEAD is defined based on the sensor id, the size maxmsgsize and a message timestamp; after the message header BAGHEAD and the sensor data msg are serialized into a cache region buf, all content of the buf is written into a file msg.bag to obtain a message frame, and recording of the sensor data msg is completed; after the content of the file msg.bag is read into a data buffer area buf, data frames in the buf are analyzed in a deserialization mode, and the sensor data msg are sent to a sensor for playback based on a message timestamp control playback frequency; the problems that hardware equipment is insufficient and no open source software is used for recording sensor data in automatic driving are solved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method and system for recording and replaying sensor data in autonomous driving. Background Art

[0002] In autonomous driving, it is often necessary to record chassis CAN format data and play back sensor CANFD format data for problem debugging and performance optimization.

[0003] There are hardware devices for direct recording on the market, such as canoe, etc. These devices are expensive, have limited corresponding recording interfaces, and may require multiple recordings, which is not convenient for mass production. There are also recordings through ROS, but most controllers do not support ROS. Summary of the invention

[0004] In view of the technical problems existing in the prior art, the present invention provides a method and system for recording and replaying sensor data in autonomous driving, which solves the problem of insufficient hardware equipment and no open source software for recording sensor data in autonomous driving.

[0005] According to a first aspect of the present invention, a method for recording and replaying sensor data in autonomous driving is provided, comprising:

[0006] Step 1, sending the sensor data msg collected by each sensor to the host computer, and triggering the recording function when the host computer receives a frame of the sensor data msg;

[0007] Step 2, defining the size max_msg_size of the sensor data msg based on the type of the sensor data msg, and defining the message header BAG_HEAD based on the sensor id, the size max_msg_size and the message timestamp time_stamp;

[0008] Step 3, after serializing the message header BAG_HEAD and the sensor data msg into the buffer area buf, write the entire content of buf into the file msg.bag to obtain a message frame, and complete the recording of the sensor data msg;

[0009] Step 4: after reading the content of the file msg.bag into the data buffer buf, deserialize and parse the data frame in buf, control the playback frequency based on the message timestamp and send the sensor data msg to the sensor for playback.

[0010] Based on the above technical solution, the present invention can also make the following improvements.

[0011] Optionally, in step 2, defining the size max_msg_size of the sensor data msg based on the type of the sensor data msg includes:

[0012] When the sensor data id = canfd, define max_msg_size = sizeof(CanFdFrame) + 8, where CanFdFrame is the data type of canfd; sizeof is a function representing the size of a variable or type;

[0013] When the sensor data id = SerialPortData, define max_msg_size = sizeof(SerialPortData) + 8, where SerialPortData is the data type of the serial port.

[0014] Optionally, serializing the message header BAG_HEAD and the sensor data msg in step 3 includes: defining a data frame, the first 16 bits of the data frame are the message header BAG_HEAD, and the last n bits are the sensor data msg; n represents the length of the sensor data msg; and each of the data frames is connected in sequence.

[0015] Optionally, before writing the entire contents of buf into the file msg.bag, step 3 further includes: calculating a CRC checksum of the encoded message part and storing it in buf.

[0016] Optionally, the process of writing the entire contents of buf into the file msg.bag in step 3 includes:

[0017] The message frame is saved in binary form into a txt text file, with each message frame corresponding to one line.

[0018] Optionally, the process of deserializing and parsing the data frame in buf in step 4 includes:

[0019] The sensor data msg and its corresponding message timestamp time_stamp are saved as a data frame in bag_list; the data frames in bag_list are sorted in ascending and descending order according to the sequence numbers corresponding to their message timestamps time_stamps.

[0020] Optionally, in the process of sending the sensor data msg to the sensor for playback in step 4, the sensor data msg are defined in order of timestamps as msg1, msg2, ..., msgN, where N represents the total number of sensor data msg; the process of playing back msg1 and msg2 includes:

[0021] Record the time when data msg1 starts to be sent as start_time, start the timer timer, and record the time of the timer timer as time1; after playing back data msg1 and preparing to play back msg2, record the time of the timer timer as time2, then the program running time elapsed = time2-time1; when elapsed>=T, send msg2 to the sensor;

[0022] time1 and time2 are the durations of msg1 and msg2 respectively.

[0023] According to a second aspect of the present invention, there is provided a recording and playback system for sensor data in autonomous driving, comprising: a host computer and various sensors;

[0024] Each sensor sends the collected sensor data msg to the host computer;

[0025] When the host computer receives a frame of the sensor data msg, the recording function is triggered; based on the type of the sensor data msg, the size max_msg_size of the sensor data msg is defined, and the message header BAG_HEAD is defined based on the sensor id, the size max_msg_size and the message timestamp time_stamp; after the message header BAG_HEAD and the sensor data msg are serialized into the buffer area buf, the entire content of buf is written into the file msg.bag to obtain a message frame, and the recording of the sensor data msg is completed;

[0026] After the host computer reads the content of the file msg.bag into the data buffer buf, it deserializes and parses the data frame in buf, controls the playback frequency based on the message timestamp, and sends the sensor data msg to the sensor for playback.

[0027] According to a third aspect of the present invention, there is provided an electronic device comprising a memory and a processor, wherein the processor is used to implement the steps of a method for recording and replaying sensor data in autonomous driving when executing a computer management program stored in the memory.

[0028] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium on which a computer management program is stored. When the computer management program is executed by a processor, the steps of a method for recording and replaying sensor data in autonomous driving are implemented.

[0029] The present invention provides a recording and playback method, system, electronic device and storage medium for sensor data in autonomous driving. During recording, data of multiple different sensors can be saved in the same package, which is universal and simplifies the implementation method. It is safe through serialization and message type, length and CRC verification. When recording is triggered by each frame of sensor, data is written in text and shared data is protected by locks, without garbled characters or data loss. The code ensures that the recorded data includes a complete message header, data and its CRC verification, thereby ensuring the integrity and reliability of the recorded data. The integrity of the data is critical for subsequent data analysis, recovery or reproduction of problem events, especially when data tracing or verification is required. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A flowchart of recording and playback of sensor data in autonomous driving provided by the present invention;

[0031] Figure 2 A schematic diagram of the format of a data frame stored in buf during a recording process of an embodiment of the present invention is provided;

[0032] Figure 3 A schematic diagram of the format of a data frame stored in buf during playback of an embodiment of the present invention is provided;

[0033] Figure 4 A structural block diagram of a sensor data recording and playback system for autonomous driving provided by the present invention;

[0034] Figure 5 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0035] Figure 6 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0036] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0037] Figure 1 A flow chart of a method for recording and replaying sensor data in autonomous driving provided by the present invention, such as Figure 1 As shown, the method includes:

[0038] Step 1: Send the sensor data msg collected by each sensor to the host computer. When the host computer receives a frame of sensor data msg, the recording function is triggered.

[0039] Step 2: define the size max_msg_size of the sensor data msg based on the type of the sensor data msg, and define the message header BAG_HEAD based on the sensor id, the size max_msg_size and the message timestamp time_stamp.

[0040] Step 3, after serializing the message header BAG_HEAD and the sensor data msg into the buffer area buf, write the entire content of buf into the file msg.bag to obtain a message frame, completing the recording of the sensor data msg.

[0041] Step 4: After reading the contents of the file msg.bag into the data buffer buf, deserialize and parse the data frame in buf, and control the playback frequency based on the message timestamp to send the sensor data msg to the sensor for playback.

[0042] The present invention provides a method for recording and replaying sensor data in autonomous driving. The method records data using C++ code on a host computer to solve the problem of insufficient hardware equipment and the problem of no open source software for recording data. The method ensures the security of recorded data through serialization encryption. The method has no restrictions on the format of recorded data, thus solving the problem that tools on the market can only record specific data. The method also ensures the periodicity and authenticity of data through timestamp playback alignment, thus establishing a complete recording and playback system without data loss.

[0043] Example 1

[0044] Embodiment 1 of the present invention is an embodiment of a method for recording and replaying sensor data in an autonomous driving system provided by the present invention. Figure 1 It can be seen that the embodiment of the method includes:

[0045] Step 1: Send the sensor data msg collected by each sensor to the host computer. When the host computer receives a frame of sensor data msg, the recording function is triggered.

[0046] Step 2: define the size max_msg_size of the sensor data msg based on the type of the sensor data msg, and define the message header BAG_HEAD based on the sensor id, the size max_msg_size and the message timestamp time_stamp.

[0047] In a possible implementation manner, in step 2, defining the size max_msg_size of the sensor data msg based on the type of the sensor data msg includes:

[0048] When the id of the sensor data is canfd, max_msg_size is defined as sizeof(CanFdFrame)+8, where CanFdFrame is the data type of canfd; sizeof represents a function of the size of a variable or type.

[0049] When the sensor data id = SerialPortData, define max_msg_size = sizeof(SerialPortData) + 8, where SerialPortData is the data type of the serial port.

[0050] Step 3, after serializing the message header BAG_HEAD and the sensor data msg into the buffer area buf, write the entire content of buf into the file msg.bag to obtain a message frame, completing the recording of the sensor data msg.

[0051] like Figure 2 The schematic diagram of the format of the data frame stored in buf during the recording process of the embodiment of the present invention is shown, combined with Figure 2 It can be seen that in a possible implementation method, serializing the message header BAG_HEAD and the sensor data msg in step 3 includes: defining a data frame, the first 16 bits of the data frame are the message header BAG_HEAD, and the last n bits are the sensor data msg; n represents the length of the sensor data msg; and each data frame is connected in sequence.

[0052] In a possible embodiment, before step 3 writes the entire content of buf into the file msg.bag, it also includes: calculating the CRC-32 checksum of the encoded message part and storing it in buf.

[0053] The file msg.bag includes the message header BAG_HEAD, sensor data msg and CRC.

[0054] In a possible implementation manner, the process of writing the entire content of buf into the file msg.bag in step 3 includes:

[0055] Save the message frame in binary form to txt text, with each message frame corresponding to one line.

[0056] Step 4: After reading the contents of the file msg.bag into the data buffer buf, deserialize and parse the data frame in buf, and control the playback frequency based on the message timestamp to send the sensor data msg to the sensor for playback.

[0057] like Figure 3 The schematic diagram of the format of the data frame stored in buf during the playback process of the embodiment of the present invention is shown in FIG. Figure 3It can be seen that in a possible embodiment, the process of deserializing and parsing the data frame in buf in step 4 includes:

[0058] A sensor data msg and its corresponding message timestamp time_stamp are saved as a data frame in bag_list; each data frame in bag_list is sorted in ascending and descending order according to the sequence number corresponding to its message timestamp time_stamp.

[0059] In a possible implementation manner, in the process of completing the playback of the sensor data msg in step 4, the sensor data msg are defined in order of timestamps as msg1, msg2, ... msgN, where N represents the total number of sensor data msg; the process of playing back msg1 and msg2 includes:

[0060] Record the time when data msg1 starts to be sent as start_time, start the timer timer, and record the time of the timer timer as time1; after playing back data msg1 and preparing to play back msg2, record the time of the timer timer as time2, then the program running time elapsed = time2-time1; when elapsed>=T, send msg2 to the sensor.

[0061] time1 and time2 are the durations of msg1 and msg2 respectively.

[0062] Use the trigger mechanism to trigger the recording function every time a message frame is received

[0063] Example 2

[0064] Embodiment 2 provided by the present invention is an embodiment of a recording and playback system for sensor data in autonomous driving provided by the present invention. Figure 4 A structural diagram of a recording and playback system for sensor data in autonomous driving provided by an embodiment of the present invention, combined with Figure 4 It can be seen that the embodiment of the system includes: a host computer and various sensors.

[0065] Each sensor sends the collected sensor data msg to the host computer.

[0066] When the host computer receives a frame of sensor data msg, the recording function is triggered; the size max_msg_size of the sensor data msg is defined based on the type of the sensor data msg, and the message header BAG_HEAD is defined based on the sensor id, the size max_msg_size and the message timestamp time_stamp; after serializing the message header BAG_HEAD and the sensor data msg into the buffer area buf, the entire content of buf is written into the file msg.bag to obtain a message frame, completing the recording of the sensor data msg.

[0067] After the host computer reads the content of the file msg.bag into the data buffer buf, it deserializes and parses the data frame in buf, controls the playback frequency based on the message timestamp, and sends the sensor data msg to the sensor for playback.

[0068] It can be understood that the system for recording and replaying sensor data in autonomous driving provided by the present invention corresponds to the method for recording and replaying sensor data in autonomous driving provided by the aforementioned embodiments. The relevant technical features of the system for recording and replaying sensor data in autonomous driving can refer to the relevant technical features of the method for recording and replaying sensor data in autonomous driving, which will not be repeated here.

[0069] See also Figure 5 , Figure 5 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 5 As shown, an embodiment of the present invention provides an electronic device, including a memory 1310, a processor 1320, and a computer program 1311 stored in the memory 1310 and executable on the processor 1320. When the processor 1320 executes the computer program 1311, the following steps are implemented: the sensor data msg collected by each sensor is sent to the host computer, and when the host computer receives a frame of sensor data msg, the recording function is triggered; the size max_msg_size of the sensor data msg is defined based on the type of the sensor data msg, and the message header BAG_HEAD is defined based on the sensor id, the size max_msg_size and the message timestamp time_stamp; after serializing the message header BAG_HEAD and the sensor data msg into the buffer area buf, the entire content of buf is written into the file msg.bag to obtain a message frame, and the recording of the sensor data msg is completed; after reading the content of the file msg.bag into the data buffer buf, the data frame in buf is deserialized and parsed, and the playback frequency is controlled based on the message timestamp to send the sensor data msg to the sensor for playback.

[0070] See also Figure 6 , Figure 6Schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. Figure 6 As shown, this embodiment provides a computer-readable storage medium 1400, on which a computer program 1411 is stored. When the computer program 1411 is executed by a processor, the following steps are implemented: the sensor data msg collected by each sensor is sent to the host computer, and when the host computer receives a frame of sensor data msg, the recording function is triggered; the size max_msg_size of the sensor data msg is defined based on the type of the sensor data msg, and the message header BAG_HEAD is defined based on the sensor id, the size max_msg_size and the message timestamp time_stamp; after serializing the message header BAG_HEAD and the sensor data msg into the buffer area buf, the entire content of buf is written into the file msg.bag to obtain a message frame, and the recording of the sensor data msg is completed; after reading the content of the file msg.bag into the data buffer buf, the data frame in buf is deserialized and parsed, and the playback frequency is controlled based on the message timestamp to send the sensor data msg to the sensor for playback.

[0071] The embodiment of the present invention provides a recording and playback method, system, electronic device and storage medium for sensor data in autonomous driving, defines a basic message header class, instantiates the message header class according to the type of sensor data received, and the message header class includes the sensor type, data channel, and data timestamp; defines a data buffer area to store the received raw data of the sensor, message header, message length and crc32 check digit; serializes the data in the data buffer area and saves it in binary form to a txt text, with each message frame corresponding to one line; controls the playback frequency by the timestamp during playback to ensure the authenticity of the data. Its beneficial effects include:

[0072] When recording, data from multiple different sensors can be saved in the same package, which is universal and simplifies the implementation method. It is secure through serialization and message type, length, and CRC verification. When recording is triggered by each frame of the sensor, the data is written via text and shared data is protected by a lock, without garbled characters or data loss. The code ensures that the recorded data includes a complete message header, data, and its CRC verification, ensuring the integrity and reliability of the recorded data. The integrity of the data is critical for subsequent data analysis, recovery, or reproduction of problem events, especially when data traceability or verification is required.

[0073] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0074] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0076] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0078] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0079] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for recording and replaying sensor data in autonomous driving, characterized in that: The method comprises: Step 1, sending the sensor data msg collected by each sensor to the host computer, and triggering the recording function when the host computer receives a frame of the sensor data msg; Step 2, defining the size max_msg_size of the sensor data msg based on the type of the sensor data msg, and defining the message header BAG_HEAD based on the sensor id, the size max_msg_size and the message timestamp time_stamp; Step 3, after serializing the message header BAG_HEAD and the sensor data msg into the buffer area buf, write the entire content of buf into the file msg.bag to obtain a message frame, and complete the recording of the sensor data msg; Step 4: after reading the content of the file msg.bag into the data buffer buf, deserialize and parse the data frame in buf, control the playback frequency based on the message timestamp and send the sensor data msg to the sensor for playback.

2. The method according to claim 1, characterized in that: Defining the size max_msg_size of the sensor data msg based on the type of the sensor data msg in step 2 includes: When the sensor data id = canfd, define max_msg_size = sizeof(CanFdFrame) + 8, where CanFdFrame is the data type of canfd; sizeof is a function representing the size of a variable or type; When the sensor data id = SerialPortData, define max_msg_size = sizeof(SerialPortData) + 8, where SerialPortData is the data type of the serial port.

3. The method according to claim 1, characterized in that: The step 3 of serializing the message header BAG_HEAD and the sensor data msg includes: defining a data frame, the first 16 bits of the data frame are the message header BAG_HEAD, and the last n bits are the sensor data msg; n represents the length of the sensor data msg; and each of the data frames is connected in sequence.

4. The method according to claim 1, characterized in that: Before writing the entire contents of buf into the file msg.bag, step 3 further includes: calculating the CRC checksum of the encoded message part and storing it in buf.

5. The method according to claim 1, characterized in that The process of writing the entire contents of buf into the file msg.bag in step 3 includes: The message frame is saved in binary form into a txt text file, with each message frame corresponding to one line.

6. The method according to claim 1, characterized in that The process of deserializing and parsing the data frame in buf in step 4 includes: The sensor data msg and its corresponding message timestamp time_stamp are saved as a data frame in bag_list; the data frames in bag_list are sorted in ascending and descending order according to the sequence numbers corresponding to their message timestamps time_stamps.

7. The method according to claim 1, characterized in that In the process of sending the sensor data msg to the sensor for playback in step 4, the sensor data msg are defined in order of timestamps as msg1, msg2, ..., msgN, where N represents the total number of sensor data msg; The process of replaying msg1 and msg2 includes: Record the time when data msg1 starts to be sent as start_time, start the timer timer, and record the time of the timer timer as time1; after playing back data msg1 and preparing to play back msg2, record the time of the timer timer as time2, then the program running time elapsed = time2-time1; when elapsed>=T, send msg2 to the sensor; time1 and time2 are the durations of msg1 and msg2 respectively.

8. A recording and playback system for sensor data in autonomous driving, characterized in that: The system comprises: a host computer and various sensors; Each sensor sends the collected sensor data msg to the host computer; When the host computer receives a frame of the sensor data msg, the recording function is triggered; based on the type of the sensor data msg, the size max_msg_size of the sensor data msg is defined, and the message header BAG_HEAD is defined based on the sensor id, the size max_msg_size and the message timestamp time_stamp; after the message header BAG_HEAD and the sensor data msg are serialized into the buffer area buf, the entire content of buf is written into the file msg.bag to obtain a message frame, and the recording of the sensor data msg is completed; After the host computer reads the content of the file msg.bag into the data buffer buf, it deserializes and parses the data frame in buf, controls the playback frequency based on the message timestamp, and sends the sensor data msg to the sensor for playback.

9. An electronic device, characterized in that: It includes a memory and a processor, and the processor is used to implement the steps of the method for recording and replaying sensor data in autonomous driving as described in any one of claims 1 to 7 when executing a computer management program stored in the memory.

10. A computer-readable storage medium, characterized in that: A computer management program is stored thereon, and when the computer management program is executed by the processor, the steps of the method for recording and replaying sensor data in autonomous driving as described in any one of claims 1 to 7 are implemented.

Images