Data processing method and system based on automatic driving simulation platform

Abstract

The invention discloses a data processing method and system based on an automatic driving simulation platform, and belongs to the technical field of automatic driving, and the technical key points are that the method comprises the following steps: 1, obtaining simulation data of automatic driving; 2, serializing the simulation data to obtain serialized data of an image and a truth value; 3, according to the time and type information of the serialized data, archiving and storing the serialized data; 4, matching the serialized data, generating label data, and constructing a data set; and 5, storing the data set, simulating a plurality of driving scenes by using a simulation environment, and recording, processing and storing various data in the simulation environment in real time, including image, radar and positioning data. These data can be used for the advantages of improving automatic driving algorithms, verifying the safety and performance of the system, and optimizing driving decisions and path planning.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a data processing method and system based on an autonomous driving simulation platform. Background Art

[0002] With the rapid development of autonomous driving technology, simulation platforms have become an important tool for evaluating and verifying autonomous driving systems. The algorithm model training of traditional autonomous driving systems usually relies on actual road tests, and the actual collection of large-scale data sets involves huge costs, including sensor equipment, human resources, and travel expenses.

[0003] In addition, there are safety risks in collecting autonomous driving data on actual roads. Therefore, it is urgent to process the data generated by the autonomous driving simulation platform to construct a data set to support the development and optimization of autonomous driving algorithms, thereby improving the safety and reliability of the system. Summary of the invention

[0004] In view of the deficiencies in the prior art, an embodiment of the present invention aims to provide a data processing method and system based on an autonomous driving simulation platform to solve the problems in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: A data processing method and system based on an autonomous driving simulation platform, comprising the following steps: Step 1: Obtain simulation data for autonomous driving; Step 2: Serializing the simulation data to obtain serialized data of images and true values; Step 3: Archiving and storing the serialized data according to the time and type information of the serialized data; Step 4: Match the serialized data, generate label data, and construct a data set; Step 5: Store the data set.

[0006] As a further solution of the present invention, the simulation data in step one includes raw data generated by the autonomous driving platform, and the raw data is data output by acquisition cameras, ultrasonic radars, millimeter-wave radars, lidars, and fusion sensors.

[0007] As a further solution of the present invention, the step 2 includes converting the format of the simulation data, wherein the image data is converted into a jpeg format and the true value data is converted into a Json format.

[0008] As a further solution of the present invention, the step three includes generating a storage directory according to the current time information and the type information of the serialized data, and storing the corresponding serialized data according to the generated storage directory.

[0009] As a further solution of the present invention, step four includes pairing each image serialization data and its corresponding true value serialization data according to the timestamp information of the image data and the true value data, generating a Json tag file to store the correspondence between the image serialization data file name and the true value serialization data, and generating a data set in combination with the serialization data.

[0010] As a further solution of the present invention, step five includes collecting simulation data of the autonomous driving simulation platform, serializing the simulation data to obtain serialized data of images and true values, and generating a storage directory based on time information and type information of the serialized data to store corresponding serialized data.

[0011] As a further solution of the present invention, it includes: An acquisition module is used to obtain simulation data of the autonomous driving platform; A conversion module, used for serializing the simulation data to obtain serialized data of an image and a true value; An archiving module, used to generate an archiving directory to store the serialized data according to the time and the type information of the serialized data; A matching module, used to match the serialized data, generate a label file, and construct a data set; A storage module is used to store the data set.

[0012] In summary, compared with the prior art, the embodiments of the present invention have the following beneficial effects: 1. Improve data processing efficiency The present invention can convert different types of data (such as images, radar data, positioning data, etc.) into a serialized format that is easy to store and manage by serializing the autonomous driving simulation data. This can greatly improve the efficiency of data processing and facilitate subsequent data analysis and utilization.

[0013] 2. Convenient data storage and management By archiving and storing data according to time and type information, the system can automatically manage large amounts of simulation data and ensure that the storage structure of different data sources is clear and well-organized. This archiving mechanism not only facilitates data search and retrieval, but also prevents data loss or confusion, thereby effectively improving data reliability and integrity.

[0014] 3. Accurate construction and label generation of data sets The data processing method provided by the present invention can generate a label file based on serialized data matching, automatically label the simulation data with corresponding labels, and construct a high-quality data set. This function can effectively improve the accuracy of data labeling, avoid subjectivity and errors in manual labeling, and thus provide a high-quality training data set for the training of autonomous driving algorithms.

[0015] 4. Automated data storage and construction process The data storage module automatically stores the constructed data set to the specified storage medium, reducing the complexity and error rate of manual operations and improving work efficiency. In addition, the automated data processing process also reduces the workload of R&D personnel in the data preparation process, allowing them to focus more on algorithm optimization and system verification.

[0016] In order to more clearly illustrate the structural features and effects of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of a data processing method based on an autonomous driving simulation platform in an embodiment of the invention.

[0018] Figure 2 This is an implementation diagram of the data processing method based on the autonomous driving simulation platform in an embodiment of the invention.

[0019] Figure 3 It is a structural block diagram of a data processing method based on an autonomous driving simulation platform in an embodiment of the invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0021] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0022] In one embodiment, a data processing method and system based on an autonomous driving simulation platform, see Figure 1 to Figure 3 , including the following steps: Step 1: Obtain simulation data for autonomous driving; Step 2: Serializing the simulation data to obtain serialized data of images and true values; Step 3: Archiving and storing the serialized data according to the time and type information of the serialized data; Step 4: Match the serialized data, generate label data, and construct a data set; Step 5: Store the data set.

[0023] For further information, see Figure 1 to Figure 3 The simulation data in step 1 includes the original data generated by the autonomous driving platform, and the original data is the data output by the acquisition camera, ultrasonic radar, millimeter wave radar, lidar and fusion sensor.

[0024] For further information, see Figure 1 to Figure 3 , the step 2 includes converting the format of the simulation data, wherein the image data is converted into the jpeg format and the true value data is converted into the Json format.

[0025] For further information, see Figure 1 to Figure 3 The step three includes generating a storage directory according to the current time information and the type information of the serialized data, and storing the corresponding serialized data according to the generated storage directory.

[0026] For further information, see Figure 1 to Figure 3 The step 4 includes pairing each image serialization data and its corresponding true value serialization data according to the timestamp information of the image data and the true value data, generating a Json tag file to store the correspondence between the image serialization data file name and the true value serialization data, and generating a data set in combination with the serialization data.

[0027] For further information, see Figure 1 to Figure 3 , the step five includes collecting simulation data of the autonomous driving simulation platform, serializing the simulation data to obtain serialized data of images and true values, and generating a storage directory according to time information and type information of the serialized data to store the corresponding serialized data.

[0028] For further information, see Figure 1 to Figure 3 ,include: An acquisition module is used to obtain simulation data of the autonomous driving platform; A conversion module, used for serializing the simulation data to obtain serialized data of an image and a true value; An archiving module, used to generate an archiving directory to store the serialized data according to the time and the type information of the serialized data; A matching module, used to match the serialized data, generate a label file, and construct a data set; A storage module is used to store the data set.

[0029] In this embodiment, Figure 1 This is a data processing method based on an autonomous driving simulation platform according to an embodiment of the present application. Figure 1As shown, according to the data processing method based on the autonomous driving simulation platform according to the embodiment of the present application, the steps include: S101: Obtaining simulation data for autonomous driving.

[0030] In one embodiment of the present application, the simulation data includes raw data generated by an autonomous driving platform, and the simulation data for autonomous driving is obtained, including: collecting raw byte data output by cameras, ultrasonic radars, millimeter-wave radars, lidars, and fusion sensors.

[0031] Specifically, combined Figure 2 As shown, first start the autonomous driving simulation platform, and write a callback function according to the API interface specification provided by the simulation platform to establish a connection with the simulation platform and receive the original byte data sent by the simulation platform.

[0032] S102: Serializing the simulation data to obtain serialized data of images and true values, including: converting the simulation data into a format, wherein the image data is converted into a JPEG format, and the true value data is converted into a JSON format.

[0033] Combination Figure 2 As shown in FIG. 1 , as a specific example, in the process of serializing the simulation data, the serialization logic is implemented based on the C++ programming language.<jpeglib.h> Process the received image byte data, first set the jpeg compression parameters, then read and traverse the image byte data, and convert it into color RGB image serialization data in jpeg format.<json.h> The library processes the true value data, constructs a Json::Value object, converts the true value data into a key-value pair format and stores it in the Json::Value object, and then converts the Json::Value object into a string and writes it into the corresponding Json file.

[0034] S103: Archiving and storing the serialized data according to the time and type information of the serialized data, including: generating a storage directory according to the current time information, such as year, month, day, hour and minute information, and the type information of the serialized data, and storing the corresponding serialized data according to the generated storage directory.

[0035] Combination Figure 2 As shown, as a specific example, according to the current year, month, day, hour, minute and data type information, a storage directory "year / month / day / hour / minute / data type / " is recursively generated, and the corresponding serialized data is archived according to the generated storage directory.

[0036] S104: Match the serialized data stored in the archive to generate a data set, including: pairing each image serialized data and its corresponding true value serialized data according to the timestamp information of the image data and the true value data, and generating a Json tag file to store the correspondence between the image file name and the true value data, and generating a data set in combination with the serialized data.

[0037] S105: storing the data set.

[0038] According to the data processing method based on the autonomous driving simulation platform of the embodiment of the present application, the simulation data of the autonomous driving simulation platform is collected, the simulation data is serialized, and the serialized data of the image and the true value are obtained. According to the time information and the type information of the serialized data, a storage directory is generated to store the corresponding serialized data. According to the timestamp information of the serialized data, each image serialized data and its corresponding true value serialized data are paired, and a Json tag file is generated to store the correspondence between the image file name and the true value data, and a data set is generated in combination with the serialized data. The use of this automated method to generate a data set reduces the huge cost involved in actually collecting large-scale data sets.

[0039] Figure 3 1 is a structural block diagram of a data processing system based on an autonomous driving simulation platform according to an embodiment of the present application. Figure 3 As shown, a structural block diagram of a data processing system based on an autonomous driving simulation platform according to an embodiment of the present application includes: an acquisition module 310, a conversion module 320, an archiving module 330, a matching module 340 and a storage module 350, wherein: An acquisition module 310 is used to acquire simulation data of the autonomous driving platform; The conversion module 320 is used to perform serialization processing on the simulation data to obtain serialized data of the image and the true value; An archiving module 330 is used to generate an archiving directory to store the serialized data according to the time and the type information of the serialized data; A matching module 340, used to match the serialized data, generate a label file, and construct a data set; The storage module 350 is used to store the data set.

[0040] According to the data processing system based on the autonomous driving simulation platform of the embodiment of the present application, the simulation data of the autonomous driving simulation platform is collected, the simulation data is serialized, and the serialized data of the image and the true value are obtained. According to the time information and the type information of the serialized data, a storage directory is generated to store the corresponding serialized data. According to the timestamp information of the serialized data, each image serialized data and its corresponding true value serialized data are paired, and a Json tag file is generated to store the correspondence between the image file name and the true value data, and a data set is generated in combination with the serialized data. The use of this automated method to generate a data set reduces the huge cost involved in actually collecting large-scale data sets.

[0041] For the specific definition of the data processing system based on the autonomous driving simulation platform, please refer to the definition of the data processing method based on the autonomous driving simulation platform mentioned above, which will not be repeated here. The various modules of the above-mentioned data processing system based on the autonomous driving simulation platform can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0042] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A data processing method based on an autonomous driving simulation platform, characterized in that: The following steps are involved: Step 1: Obtain simulation data for autonomous driving; Step 2: Serializing the simulation data to obtain serialized data of images and true values; Step 3: Archiving and storing the serialized data according to the time and type information of the serialized data; Step 4: Match the serialized data, generate label data, and construct a data set; Step 5: Store the data set.

2. The data processing method based on the autonomous driving simulation platform according to claim 1, characterized in that: The simulation data in step one includes raw data generated by the autonomous driving platform, and the raw data is data output by acquisition cameras, ultrasonic radars, millimeter-wave radars, lidars, and fusion sensors.

3. The data processing method based on the autonomous driving simulation platform according to claim 1, characterized in that: The step 2 includes converting the format of the simulation data, wherein the image data is converted into the jpeg format and the true value data is converted into the Json format.

4. The data processing method based on the autonomous driving simulation platform according to claim 1, characterized in that: The step three includes generating a storage directory according to the current time information and the type information of the serialized data, and storing the corresponding serialized data according to the generated storage directory.

5. The data processing method based on the automatic driving simulation platform according to claim 1, characterized in that: The step four includes pairing each image serialization data and its corresponding true value serialization data according to the timestamp information of the image data and the true value data, generating a Json tag file to store the correspondence between the image serialization data file name and the true value serialization data, and generating a data set in combination with the serialization data.

6. The data processing method and system based on the autonomous driving simulation platform according to claim 1, characterized in that: The step five includes collecting simulation data of the autonomous driving simulation platform, serializing the simulation data to obtain serialized data of images and true values, and generating a storage directory based on time information and type information of the serialized data to store the corresponding serialized data.

7. The system corresponding to the data processing method based on the autonomous driving simulation platform according to claim 1, characterized in that: include: An acquisition module is used to obtain simulation data of the autonomous driving platform; A conversion module, used for serializing the simulation data to obtain serialized data of an image and a true value; An archiving module, used to generate an archiving directory to store the serialized data according to the time and the type information of the serialized data; A matching module, used to match the serialized data, generate a label file, and construct a data set; A storage module is used to store the data set.

Images