A method and system for producing an autonomous driving data set

Through digital twin technology, the autonomous driving vehicle and environmental model is built in the simulator, and the sensors and environmental parameters are configured, which solves the problem of difficulty in collecting and labeling of autonomous driving data sets, and achieves rapid and large-scale multi-scene and multi-task data set generation. Meet the needs of autonomous driving systems.

CN115257785BActive Publication Date: 2024-11-22TONGJI UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210865589.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-21
Publication Date
2024-11-22
Estimated Expiration
2042-07-21

AI Technical Summary

Technical Problem

The existing autonomous driving data sets cannot effectively capture the variability of the real world environment, and the truth value annotation of advanced tasks requires a lot of manpower and material resources, which is difficult to meet the needs of large-scale, multi-scenario, and multi-tasks.

Method used

Through digital twin technology, a physical model of autonomous driving vehicles and environment is established, a simulator is imported for simulation, arranging on-board sensors and environmental parameters, collecting and preprocessing data, and generating a multi-sensor and multi-scene autonomous driving data set.

Benefits of technology

It realizes efficient and easy-to-use production of autonomous driving data sets, solves the problem of difficulty in collecting and labeling real data sets, and can quickly and largely produce comprehensive data sets of multi-scene and multi-tasks to meet the needs of autonomous driving systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115257785B_ABST
    Figure CN115257785B_ABST
Patent Text Reader

Abstract

The present invention relates to a method and system for producing an autonomous driving data set, the method comprising: according to the model of the autonomous driving vehicle and the real operating environment, using digital twin technology to establish a physical model corresponding to the autonomous driving vehicle and an environmental map corresponding to the real environment and importing them into an autonomous driving simulator; configuring in the autonomous driving simulator based on test requirements, including configuring the parameters of the on-board sensors, setting environmental parameters, importing the physical models of pedestrians and other vehicles, and setting their behavior modes; obtaining the operating mode according to the test requirements, making the physical model corresponding to the autonomous driving vehicle operate according to the operating mode in the autonomous driving simulator, collecting information from the on-board sensors during the operation, performing time synchronization and preprocessing, and storing the obtained information in the autonomous driving data set. Compared with the prior art, the present invention performs simulation in a virtual world, effectively solving the problem of difficulty in collecting and labeling real autonomous driving data sets.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of digital twin and autonomous driving, and in particular to a method and system for producing an autonomous driving data set for digital twin. Background Art

[0002] Adapting to the ever-changing environment is an inevitable key safety challenge for all autonomous driving systems. To this end, autonomous driving algorithms require massive amounts of data for training and verification. However, existing image and video driving datasets cannot capture the variability of the real world environment, and the true values ​​of many high-level tasks cannot be given directly, which usually requires a lot of manpower and material resources for labeling. Therefore, how to provide an easily accessible, large-scale, multi-scenario, multi-task comprehensive dataset has become an urgent need in the industry. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for producing an autonomous driving data set in order to overcome the defects of the above-mentioned prior art.

[0004] The purpose of the present invention can be achieved by the following technical solutions:

[0005] According to a first aspect of the present invention, a method for preparing an autonomous driving dataset is provided, comprising the following steps:

[0006] According to the model of the autonomous driving vehicle, the physical model corresponding to the autonomous driving vehicle is established through digital twin technology and imported into the autonomous driving simulator;

[0007] Based on the actual environment in which the autonomous vehicle operates, an environmental map corresponding to the real environment is established through digital twin technology and imported into the autonomous driving simulator;

[0008] Build physical models of pedestrians and other vehicles through digital twin technology;

[0009] Obtaining test requirements and configuring the autonomous driving simulator based on the test requirements, including configuring parameters of on-board sensors, setting environmental parameters, importing physical models of pedestrians and other vehicles, and setting their behavior modes;

[0010] The operating mode is obtained according to the test requirements, and the physical model corresponding to the autonomous driving vehicle is operated according to the operating mode in the autonomous driving simulator. During the operation, the on-board sensors collect information and perform time synchronization and preprocessing, and the obtained information is stored in the autonomous driving data set.

[0011] Furthermore, the physical model and environmental map corresponding to the autonomous driving vehicle are established in UE4 through digital twin technology, which is the digital virtualization in the digital twin. After verification of mechanics, kinematics and dynamics, it is imported into the CARLA autonomous driving simulator.

[0012] Furthermore, the coordinate system of the physical model of the autonomous driving vehicle established in UE4 is consistent with the coordinate system definition in the CARLA autonomous driving simulator, following the definition of the left-handed coordinate system with the X-axis facing forward in UE4. The coordinate conversion method is: use the scipy transform library to calculate the product of the rotation matrix and then convert it into the (position, rotation) format in the CARLA autonomous driving simulator.

[0013] Furthermore, the “establishing an environmental map corresponding to the real environment through digital twin technology based on the real environment in which the autonomous driving vehicle operates” specifically includes:

[0014] Environmental information of the real operating environment is collected through sensors, including but not limited to binocular cameras and lidars, and Colmap is used to jointly densely reconstruct the environment information, which is then imported into UE4 to repair the missing parts and optimize the details, including but not limited to text, traffic signs, potholes and cracks on the road, to obtain an environmental map.

[0015] Furthermore, the "configuration in the autonomous driving simulator based on the test requirements" also includes configuring special sensors, including but not limited to semantic segmentation cameras, target detection cameras, and optical flow cameras. The special sensors rely on parameters and data in the autonomous driving simulator, including but not limited to the physical model and motion pattern corresponding to the autonomous driving vehicle, the environmental map corresponding to the real environment, on-board sensors, and the physical models and behavior patterns of pedestrians and other vehicles. The special sensors are used to perform perception tasks including but not limited to semantic segmentation, target detection, and optical flow, and the obtained information is stored in the autonomous driving data set.

[0016] Furthermore, the vehicle-mounted sensors include but are not limited to cameras, lidars, millimeter-wave radars, IMUs, and wheel odometers. The parameters for configuring the vehicle-mounted sensors are specifically: setting the installation position of each vehicle-mounted sensor, setting the specifications of each vehicle-mounted sensor, and setting the working parameters of each vehicle-mounted sensor.

[0017] Furthermore, the environment parameters are set as follows:

[0018] Obtain the test requirements, parse the environmental requirements corresponding to the test requirements, and change the environmental parameters in the autonomous driving simulator based on the environmental requirements. The environmental parameters include but are not limited to cloud cover, rainfall, water accumulation, wind strength, solar azimuth, solar altitude, fog density, fog height, visible distance, humidity, scattering intensity, Mie scattering ratio, Rayleigh scattering ratio, and sandstorm intensity.

[0019] Furthermore, the “obtaining the operation mode according to the test requirements” is divided into a fixed operation mode and a real-time control mode. The fixed operation mode is as follows:

[0020] A preset driving route and driving data are obtained, and a physical model corresponding to the autonomous driving vehicle drives along the driving route according to the preset driving data;

[0021] The real-time control modes are as follows:

[0022] An input signal is obtained using an external device, and a physical model corresponding to the autonomous driving vehicle drives according to the input signal. The external device includes but is not limited to a keyboard, a handle, a steering wheel, and a somatosensory device, and the input information is a driving input signal of the vehicle.

[0023] Furthermore, the use of a data acquisition program to collect information during the "collecting information and performing time synchronization by on-board sensors during operation" involves three levels of time synchronization: one is the time synchronization between the CARLA autonomous driving simulator server and the client of the data set acquisition program; the second is the synchronization between the various actors in the CARLA autonomous driving simulator; and the third is the time synchronization between the various on-board sensors. The three levels of time synchronization are implemented by a thread-safe buffer queue.

[0024] According to a second aspect of the present invention, there is provided an autonomous driving dataset production system, based on the autonomous driving dataset production method according to the first aspect, comprising:

[0025] An input module, used to input test requirements, the model of the autonomous driving vehicle, and the real environment in which the autonomous driving vehicle operates;

[0026] Modeling module, used to establish the physical model corresponding to the autonomous driving vehicle through digital twin technology, establish the environmental map corresponding to the real environment through digital twin technology, and establish the physical model of pedestrians and other vehicles through digital twin technology;

[0027] A configuration module is configured in the autonomous driving simulator based on the test requirements, including configuring parameters of on-board sensors, setting environmental parameters, importing physical models of pedestrians and other vehicles, and setting their behavior modes;

[0028] A simulation module is used to make the physical model corresponding to the autonomous driving vehicle run according to the operation mode in the autonomous driving simulator, and the vehicle-mounted sensors collect information and perform time synchronization during the operation;

[0029] The dataset module is used to preprocess the collected information and store it into the autonomous driving dataset.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] (1) The present invention proposes an efficient and easy-to-use method for producing an autonomous driving dataset. By constructing a digital twin of the autonomous driving vehicle and the environment and performing simulation in a virtual world, the problem of difficulty in collecting and labeling real autonomous driving datasets is effectively solved.

[0032] (2) The present invention is based on virtual simulation and takes advantage of the fact that environmental parameters can be easily controlled to design a user-friendly scenario simulator that can tailor simulation scenarios to the needs of autonomous driving tasks.

[0033] (3) The present invention gives full play to the advantages of virtual simulation and can quickly and massively produce multi-sensor, multi-scenario comprehensive autonomous driving data sets to meet the multi-scenario, multi-task requirements of autonomous driving data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a flow chart of the method of the present invention;

[0035] Figure 2 is an example of an autonomous driving image data set collected by the method of the present invention;

[0036] Figure 3 It is an example of the environmental diversity of the data set collected by the method of the present invention. DETAILED DESCRIPTION

[0037] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0038] Embodiment 1:

[0039] With the combination of virtual simulation technology with artificial intelligence, 5G communication, cloud computing, high-precision maps and other technologies, the field of autonomous driving has flourished and has continuously entered the field of digital twins that combine virtuality and reality. Virtual scenes are constructed based on real data to simulate urban traffic flows and predict traffic scenes. Accordingly, the present invention provides a method for producing an autonomous driving data set, such as Figure 1 As shown, the following steps are included:

[0040] S1. According to the model of the autonomous driving vehicle, the physical model corresponding to the autonomous driving vehicle is established through digital twin technology and imported into the autonomous driving simulator;

[0041] It is understandable that the data set is produced for the autonomous driving of vehicles. Therefore, the autonomous driving vehicle here refers to the vehicle that needs to be targeted, and does not limit the vehicle's driving mode in subsequent steps to autonomous driving. In fact, in order to collect data in various scenarios and situations, the vehicle's driving mode is set by the staff according to needs.

[0042] In this embodiment, a physical model corresponding to the autonomous driving vehicle is established in UE4 through digital twin technology, which is the digital virtualization in the digital twin. After verification of mechanics, kinematics, and dynamics, it is imported into the CARLA autonomous driving simulator.

[0043] The coordinate system of the physical model of the autonomous driving vehicle established in UE4 is consistent with the coordinate system definition in the CARLA autonomous driving simulator, following the definition of the left-handed coordinate system with the positive X-axis facing forward in UE4. The coordinate conversion method is: use the scipy transform library to calculate the product of the rotation matrix and then convert it to the (position, rotation) format in the CARLA autonomous driving simulator.

[0044] S2. Based on the real environment in which the autonomous driving vehicle operates, an environmental map corresponding to the real environment is established through digital twin technology and imported into the autonomous driving simulator;

[0045] In this embodiment, environmental information of the real operating environment is collected by sensors, such as actual urban roads, buildings and other data. The sensors include but are not limited to binocular cameras and lidars. Colmap is used to jointly densely reconstruct the point cloud into a .pcd format, and then imported into UE4 for repair and optimization. The missing parts are repaired and the details are optimized, including but not limited to text, traffic signs, potholes and cracks on the road, etc., to obtain a 3D environment map model containing real detailed information such as text, traffic signs, potholes and cracks on the road.

[0046] S3. Establish physical models of pedestrians and other vehicles through digital twin technology;

[0047] S4. Obtain test requirements and configure the autonomous driving simulator based on the test requirements, including configuring the parameters of on-board sensors, setting environmental parameters, importing physical models of pedestrians and other vehicles, and setting their behavior modes;

[0048] On-board sensors include but are not limited to cameras, lidars, millimeter-wave radars, IMUs, wheel odometers, etc. The internal and external parameters of sensors such as cameras and radars are set in the CARLA simulator. The parameters of the on-board sensors are configured as follows: setting the installation position of each on-board sensor, setting the specifications of each on-board sensor, and setting the working parameters of each on-board sensor, including the resolution, FOV, frame rate, optical center, lateral / longitudinal distortion, etc. of the camera (monocular / binocular / depth / event camera), the number of lines, range, rotation frequency, upper / lower FOV, etc. of the lidar, the accelerometer deviation and gyroscope deviation of the IMU, and the range, frequency, horizontal / vertical FOV, etc. of the millimeter-wave radar.

[0049] In addition to traditional sensors such as cameras and radars, special sensors are also configured to meet the truth requirements of specific perception tasks. Special sensors include but are not limited to semantic segmentation cameras, target detection cameras, optical flow cameras, etc. Special sensors rely on parameters and data in the autonomous driving simulator, including but not limited to the physical model and motion mode corresponding to the autonomous driving vehicle, the environmental map corresponding to the real environment, on-board sensors, and the physical models and behavior patterns of pedestrians and other vehicles. Special sensors are used to perform perception tasks including but not limited to semantic segmentation, target detection, optical flow, etc., and the obtained information is stored in the autonomous driving dataset. This type of special sensor does not exist in reality. Its data is obtained by joint preprocessing of traditional sensors and autonomous driving simulator data. Usually, the parameter settings are consistent with those of traditional sensors and the pose is aligned. Since the vehicle / person / map data in the autonomous driving simulator is known, it is easier to obtain reliable truth values ​​of perception tasks such as semantic segmentation and target detection than in the real dataset. In fact, the data of special sensors is obtained by combining algorithms such as semantic segmentation / stereo matching / target detection with city maps and other vehicle / pedestrian models in UE4 and CARLA.

[0050] In this embodiment, the baseline of the binocular camera is set to 0.3m, the resolution is 1920*1080, the FOV is 90, the frame rate is 25Hz, the ISO is 100, the gamma is 2.2, the lens distortion parameters are set to the default, the number of lines of the laser radar is set to 128, the range is default, the rotation frequency is 10, the upper / lower FOV is 10 and -30 respectively, and the basic parameters of the depth camera, semantic segmentation camera, and optical flow camera are consistent with the left camera of the binocular camera and aligned with the left camera.

[0051] In order to solve the problem that existing data sets are difficult to reflect the variability of the real-world environment, this application obtains test requirements, parses the environmental requirements corresponding to the test requirements, such as weather, light, humidity, etc., and changes the environmental parameters in the autonomous driving simulator based on the environmental requirements. The environmental parameters include but are not limited to cloud cover, rainfall, water accumulation, wind strength, solar azimuth, solar altitude, fog density, fog height, visible distance, humidity, scattering intensity, Mie scattering ratio, Rayleigh scattering ratio, sandstorm intensity, etc.

[0052] Taking a sunny day as an example, in this embodiment, cloudiness=0.2, percipitation=0.2, sun_altitude_angle=80.0 are set, and other parameters remain default.

[0053] The number, physical model and behavior pattern of pedestrians and other vehicles can be adjusted freely; in order to ensure the scenario diversity of the data set, specific scenarios are constructed by setting parameters such as the number of other vehicles and pedestrians and behavior patterns, such as: pedestrians who suddenly cross the road, vehicles coming from the side at the intersection, vehicles running red lights, etc. It can be understood that when modeling, the number of pedestrians and other vehicles can be set as needed, the age, height, weight, etc. of pedestrians can be set separately, and the model and location of other vehicles can be set separately. When determining the behavior pattern, the behavior pattern can include cruising according to traffic rules, running red lights, speeding, intentional collisions, human manipulation, etc., where human manipulation can simulate some complex accident scenarios to test the generalization ability of the autonomous driving algorithm. Specifically, the route and speed of pedestrians and other vehicles, stop and wait, etc. can be set. In this embodiment, the number of other vehicles is set to 200, the number of pedestrians is set to 50, vehicles and pedestrians cruise according to the specified route, and the physical simulation range is set to 30m around the autonomous driving vehicle to save computing resources.

[0054] After setting up according to the above steps, Figure 3 As shown, Figure 3 Examples are given for setting different environmental parameters, the number of pedestrians and other vehicles, and behavior patterns in CARLA.

[0055] S5. Obtain the operation mode according to the test requirements, and make the physical model corresponding to the autonomous driving vehicle run according to the operation mode in the autonomous driving simulator. During the operation, the on-board sensors collect information and perform time synchronization and preprocessing, and the obtained information is stored in the autonomous driving data set.

[0056] "Obtaining the operation mode according to test requirements" is divided into fixed operation mode and real-time control mode. The fixed operation mode is as follows:

[0057] A preset driving route and driving data are obtained, and a physical model corresponding to the autonomous driving vehicle drives along the driving route according to the preset driving data;

[0058] The real-time control modes are as follows:

[0059] Use external devices to obtain input signals, and the physical model corresponding to the autonomous driving vehicle drives according to the input signals. The external devices include but are not limited to keyboards, handles, steering wheels, and somatosensory devices. The input information is the driving input signal of the vehicle. For example, in this embodiment, the handle is used to linearly control the throttle, direction, and brake of the data collection vehicle, and the vehicle drives in compliance with traffic regulations in the city to achieve behavior simulation and collect data.

[0060] In "Information is collected by on-board sensors and time is synchronized during operation", a data collection program is used to collect information, which involves three levels of time synchronization: one is the time synchronization between the CARLA autonomous driving simulator server and the client; the second is the synchronization between the various actors (vehicles, pedestrians, weather, traffic signals, etc.) in the CARLA autonomous driving simulator; and the third is the time synchronization between the various on-board sensors. The three levels of time synchronization are implemented by a thread-safe buffer queue.

[0061] In order to meet the usage requirements of specific training tasks, the collected raw data is preprocessed by dimension transformation, filtering, encoding, format conversion, etc., and stored in the required data format. In this embodiment, the RGB and depth maps are stored in .png format after channel transformation and filtering, the radar point cloud is stored in .pcd format, the pose and timestamp are stored in the standard format of KITTI, and the optical flow is stored in TIFF format.

[0062] The image dataset sample obtained according to the above-mentioned autonomous driving dataset preparation method is as follows: Figure 2 As shown, Figure 2 Examples of images in different formats are given in .

[0063] The present invention also provides an autonomous driving data set production system, comprising:

[0064] An input module, used to input test requirements, the model of the autonomous driving vehicle, and the real environment in which the autonomous driving vehicle operates;

[0065] Modeling module, used to establish the physical model corresponding to the autonomous driving vehicle through digital twin technology, establish the environmental map corresponding to the real environment through digital twin technology, and establish the physical model of pedestrians and other vehicles through digital twin technology;

[0066] Configuration module, which is configured in the autonomous driving simulator based on test requirements, including configuring the parameters of on-board sensors, setting environmental parameters, importing physical models of pedestrians and other vehicles, and setting their behavior modes;

[0067] A simulation module is used to make the physical model corresponding to the autonomous driving vehicle run according to the operation mode in the autonomous driving simulator, and the vehicle-mounted sensors collect information and perform time synchronization during the operation;

[0068] The dataset module is used to preprocess the collected information and store it into the autonomous driving dataset.

[0069] The present invention proposes a method and system for producing an autonomous driving dataset for digital twins to meet the requirements of the autonomous driving field for large-scale, easy-to-access, multi-scenario, and multi-task datasets. The dataset production method can conveniently obtain a comprehensive dataset with comprehensive sensors and mainstream perception task truth values. It can also record the discrete and continuous changes over time in the environment such as illumination, cloud cover, rain and fog intensity, and density of vehicles and pedestrians, helping the autonomous driving system to improve its perception capabilities for specific tasks and its generalization capabilities in complex and changing environments.

[0070] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for preparing an autonomous driving dataset. It is characterized in that The following steps are involved: According to the model of the autonomous driving vehicle, the physical model corresponding to the autonomous driving vehicle is established through digital twin technology and imported into the autonomous driving simulator; Based on the actual environment in which the autonomous vehicle operates, an environmental map corresponding to the real environment is established through digital twin technology and imported into the autonomous driving simulator; Build physical models of pedestrians and other vehicles through digital twin technology; Obtaining test requirements and configuring the autonomous driving simulator based on the test requirements, including configuring parameters of on-board sensors, setting environmental parameters, importing physical models of pedestrians and other vehicles, and setting their behavior modes; Obtain the operation mode according to the test requirements, and make the physical model corresponding to the autonomous driving vehicle run according to the operation mode in the autonomous driving simulator. During the operation, the on-board sensors collect information and perform time synchronization and preprocessing, and the obtained information is stored in the autonomous driving data set; Use digital twin technology to build the physical model and environment map corresponding to the autonomous driving vehicle in UE4, and import it into the CARLA autonomous driving simulator after verification; The coordinate system of the physical model of the autonomous driving vehicle established in UE4 is consistent with the coordinate system definition in the CARLA autonomous driving simulator, following the definition of the left-handed coordinate system with the positive X-axis facing forward in UE4. The coordinate conversion method is: use the scipy transform library to calculate the product of the rotation matrix and then convert it into the [position, rotation] format in the CARLA autonomous driving simulator; The environmental map corresponding to the real environment in which the autonomous driving vehicle operates is established by using the digital twin technology as follows: Environmental information of the real operating environment is collected through sensors, including binocular cameras and lidars, and Colmap is used to jointly reconstruct dense information, which is then imported into UE4 to repair missing parts and optimize details, including text, traffic signs, potholes and cracks on the road, to obtain an environmental map; The configuration in the autonomous driving simulator based on the test requirements also includes configuring special sensors, the special sensors include semantic segmentation cameras, target detection cameras, and optical flow cameras. The special sensors rely on parameters and data in the autonomous driving simulator, including physical models and motion patterns corresponding to autonomous driving vehicles, environmental maps corresponding to real environments, on-board sensors, and physical models and behavior patterns of pedestrians and other vehicles. The special sensors are used to perform semantic segmentation, target detection, and optical flow perception tasks, and the obtained information is stored in the autonomous driving data set; The vehicle-mounted sensors include cameras, laser radars, millimeter-wave radars, IMUs, and wheel odometers. The parameters for configuring the vehicle-mounted sensors specifically include: setting the installation position of each vehicle-mounted sensor, setting the specifications of each vehicle-mounted sensor, and setting the working parameters of each vehicle-mounted sensor; Set the environment parameters as follows: Obtain test requirements, parse out the environmental requirements corresponding to the test requirements, and change the environmental parameters in the autonomous driving simulator based on the environmental requirements, wherein the environmental parameters include cloud cover, rainfall, water accumulation, wind intensity, solar azimuth, solar altitude, fog density, fog height, visual distance, humidity, scattering intensity, Mie scattering ratio, Rayleigh scattering ratio, and sandstorm intensity; The operation mode obtained according to the test requirements is divided into a fixed operation mode and a real-time control mode. The fixed operation mode is as follows: A preset driving route and driving data are obtained, and a physical model corresponding to the autonomous driving vehicle drives along the driving route according to the preset driving data; The real-time control modes are as follows: An input signal is obtained using an external device, and a physical model corresponding to the autonomous driving vehicle drives according to the input signal. The external device includes a keyboard, a handle, a steering wheel, and a somatosensory device. The input signal is a driving input signal of the vehicle.

2. The method for preparing an autonomous driving dataset according to claim 1, It is characterized in that The information is collected by the on-board sensors during operation and the data collection program is used for time synchronization, which involves three levels of time synchronization: one is the time synchronization between the CARLA autonomous driving simulator server and the client of the data set collection program; the second is the synchronization between the various actors in the CARLA autonomous driving simulator; and the third is the time synchronization between the various on-board sensors. The time synchronization at these three levels is implemented by a thread-safe buffer queue.

3. A system for producing autonomous driving datasets, It is characterized in that The method for producing an autonomous driving data set according to any one of claims 1 to 2 comprises: An input module, used to input test requirements, the model of the autonomous driving vehicle, and the real environment in which the autonomous driving vehicle operates; Modeling module, used to establish the physical model corresponding to the autonomous driving vehicle through digital twin technology, establish the environmental map corresponding to the real environment through digital twin technology, and establish the physical model of pedestrians and other vehicles through digital twin technology; A configuration module is configured in the autonomous driving simulator based on the test requirements, including configuring parameters of on-board sensors, setting environmental parameters, importing physical models of pedestrians and other vehicles, and setting their behavior modes; A simulation module is used to make the physical model corresponding to the autonomous driving vehicle run according to the operation mode in the autonomous driving simulator, and during the operation, the vehicle-mounted sensors collect information and perform time synchronization; The dataset module is used to pre-process the collected information and store it into the autonomous driving dataset.