A method for batch generation of perception datasets for autonomous driving
By constructing simulated road scenarios and using statistical and machine learning methods to generate autonomous driving perception data sets, the problems of low data set generation efficiency and insufficient sample differences in the existing technology are solved, and efficient and specific perception data set generation is achieved. .
Patent Information
- Application Number
- CN202411216297.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2024-12-20
- Estimated Expiration
- 2044-09-02
AI Technical Summary
The existing autonomous driving perception data set generation method relies on fixed-configured sensors and complex scenario models to build, resulting in low data set generation efficiency and insufficient sample differences, making it difficult to meet the data needs in complex traffic scenarios.
By obtaining real road scene data, building simulated road scenes and generating road flow distribution models using statistical and machine learning methods, randomly selecting simulated vehicle elements as observation objects, configuring simulation sensors to obtain perceptual data, and generating perceptual data sets through data annotation. Repeat the process until the preset number is reached.
It realizes efficient batch generation of autonomous driving perception data sets, improves the specificity and diversity of the data set, reduces dependence on real road scene data and traffic flow data, and improves the data set generation efficiency.
Smart Images
Figure CN118708981B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of intelligent driving simulation data processing, and in particular to a method for batch generating perception data sets for autonomous driving. Background Art
[0002] Autonomous driving technology can improve traffic safety, reduce traffic accidents, and make driving convenient and comfortable. Before fully autonomous driving can be achieved, many technical problems need to be solved, one of which is how to generate enough data sets to train autonomous driving models.
[0003] Dataset generation refers to obtaining or constructing a set of data sets through various means to train machine learning models. In autonomous driving technology, data set generation is a very important link. Due to the complexity and safety requirements of autonomous driving technology, a large number of data sets are needed to support the training of autonomous driving models.
[0004] Existing dataset generation technologies are mostly based on fixed-configuration sensors that collect data in each simulated scenario. This not only results in a huge time-consuming process of repeatedly building scenarios, but also requires a large amount of diverse scenario inputs to enrich the perception dataset, further increasing the difficulty of dataset generation. Traditional methods are limited by the fact that sensors are installed on stationary vehicles and rely on the natural changes of the vehicle and its surroundings to capture data. When faced with complex traffic scenarios such as peak-hour congestion, it is difficult to efficiently obtain highly differentiated data samples due to slow environmental changes, which affects the specificity of the dataset and the training effect.
[0005] In the process of implementing the above-mentioned related technologies, the inventors found that the related technologies have the following problems:
[0006] The perception dataset generation method provided by the related technology only generates data once in each frame of the scene. It relies on the construction of a large number of scene models and cannot make full use of the scene data. The large number of scenes currently generated using traffic flow construction methods such as SUMO make it difficult to ensure the diversity of dataset samples. Summary of the invention
[0007] In order to solve the above problems existing in the prior art, the present invention provides a method for batch generation of perception data sets for autonomous driving, so as to realize large-scale and efficient generation of specific perception data sets. The technical solution provided by the present invention is as follows:
[0008] According to one aspect of an embodiment of the present invention, a method for batch generating a perception data set for autonomous driving is provided, wherein the method comprises:
[0009] S10: Acquire real road scene data corresponding to the real road scene, wherein the real road scene data includes vehicle element data, pedestrian element data and environment element data;
[0010] S20: Calculate a real road flow distribution model corresponding to the real road scene data using statistical and machine learning methods, and generate a simulated road scene corresponding to the real road scene data according to the road flow distribution model, wherein the simulated road scene includes simulated vehicle elements, simulated pedestrian elements, and simulated environment elements;
[0011] S30: Determine any simulated vehicle element in the simulated road scene as an observation object for observation, configure a preset simulation sensor at a preset position of the observation object, and obtain perception data of the observation object on each surrounding simulated road scene element through the simulation sensor, and then mark the simulated road scene element corresponding to each perception data in the simulated road scene according to the acquired perception data;
[0012] S40: updating any unobserved simulated vehicle element in the simulated road scene as an observed object, and executing the step of configuring a preset simulated sensor at a preset position of the observed object in S30 until all simulated vehicle elements are observed, thereby generating a perception data set for autonomous driving;
[0013] S50: Repeat S10 to S40 until the number of perception data sets for autonomous driving meets the preset generation number.
[0014] Preferably, before S30, the method further includes:
[0015] If it is detected that the simulated road scene has damage or missing preset simulated road scene elements, each damaged or missing simulated road scene element is determined as a to-be-processed simulated element, and a replacement simulated road scene element corresponding to each to-be-processed simulated element is added to the simulated road scene according to a preset element adding rule;
[0016] The preset simulated road scene elements include vehicle elements, pedestrian elements and environmental elements;
[0017] The preset element adding rule includes the corresponding adding position conditions of each replacement simulated road scene element in the preset simulated road scene.
[0018] Preferably, the substitute simulated road scene elements are various simulated road scene elements pre-stored in a road scene element database, and the substitute simulated road scene elements are obtained by simulating different real road scene elements.
[0019] Preferably, the method further comprises:
[0020] If it is detected that the number of simulated road scene elements in the simulated road scene does not meet the preset number, each simulated road scene element that does not meet the preset number will be determined as a simulation element to be added, and replacement simulated road scene elements corresponding to each simulation element to be added will be added to the simulated road scene according to the preset element adding rules.
[0021] Preferably, S30 further includes:
[0022] After executing the step of configuring a preset simulation sensor at the preset position of the observation object, the simulated vehicle element is removed from the simulated road scene, and then the step of obtaining the perception data of the observation object on the surrounding simulated road scene elements through the simulation sensor is executed, and then the simulated road scene elements corresponding to each perception data in the simulated road scene are marked according to the acquired perception data, and after the marking is completed, the observation object is restored in the simulated road scene.
[0023] Preferably, S30 further includes:
[0024] After executing the step of obtaining the perception data of the observed object on each surrounding simulated road scene element through the simulation sensor, the perception data is preprocessed and cleaned, and then based on the preprocessed and cleaned perception data, the step of labeling the simulated road scene elements corresponding to each perception data in the simulated road scene is executed.
[0025] Preferably, the simulated sensor includes a simulated millimeter-wave radar and a simulated camera.
[0026] Compared with the prior art, the method for batch generating perception data sets for autonomous driving provided by the present invention has the following advantages:
[0027] The present invention provides a method for batch generating perception data sets for autonomous driving, which comprises acquiring real road scene data corresponding to a real road scene, wherein the real road scene data comprises vehicle element data, pedestrian element data and environmental element data; using statistical and machine learning methods to calculate a real road flow distribution model corresponding to the real road scene data, and generating a simulated road scene corresponding to the real road scene data according to the road flow distribution model, wherein the simulated road scene comprises simulated vehicle elements, simulated pedestrian elements and simulated environmental elements; determining any simulated vehicle element in the simulated road scene as an observation object for observation, configuring a preset simulation sensor at a preset position of the observation object, and acquiring perception data of the observation object on each surrounding simulated road scene element through the simulation sensor, and then annotating the simulated road scene elements corresponding to each perception data in the simulated road scene according to the acquired perception data; updating any unobserved simulated vehicle element in the simulated road scene as an observation object, repeating the above steps until each simulated vehicle element is observed, and generating a perception data set for autonomous driving, thereby realizing high-efficiency mass generation of specific perception data sets through the acquisition of a small amount of real road scene data and traversal observation of each simulated vehicle element in the simulated road scene.
[0028] It should be noted that the present invention statically generates simulated road scenes based on simple real road scene data, and acquires perception data of each simulated element of the simulated road scene from different perspectives of the simulated vehicle elements by means of the sequential movement configuration of each simulated vehicle element in the static simulated road scene through the simulation sensor. There is no need to rely on a large amount of traffic flow data for the generation of complex dynamic simulated road scenes, nor is there a need for a large amount of real road scene data for a huge amount of simulated road scenes. Batch generation of various types of perception data sets for autonomous driving can be achieved, thereby achieving the technical effect of improving the efficiency of generating perception data sets for autonomous driving.
[0029] In addition, the present invention also utilizes simulated road scene elements to fuse real scene elements in real road scenes, thereby realizing diversified generation of simulated road scenes and further improving the generation efficiency of specific perception data sets. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0031] Figure 1 It is a method flow chart of a method for batch generating a perception data set according to an exemplary embodiment of the present invention.
[0032] Figure 2The figure is a schematic diagram of a simulated road scene according to an exemplary embodiment of the present invention.
[0033] Figure 3 FIG. 4 is a schematic diagram of another simulated road scene according to an exemplary embodiment of the present invention.
[0034] Figure 4 FIG. 4 is a schematic diagram of another simulated road scene according to an exemplary embodiment of the present invention.
[0035] Figure 5 FIG. 4 is a schematic diagram of another simulated road scene according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION
[0036] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is described in detail below in combination with specific embodiments (but not limited to the embodiments) and the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0037] After constructing a simulated road scene based on the real road scene, the existing autonomous driving perception data set generation method usually uses a fixed simulated vehicle in the simulated road scene as the observation target to obtain perception data of other surrounding simulated elements. When a large number of perception data acquisition operations are required, the existing autonomous driving perception data set generation method can only obtain perception data of surrounding simulated elements through different modeling of a large number of simulated road scenes, and then obtain perception data of surrounding simulated elements through simulated sensors fixedly configured on a single simulated vehicle in each simulated road scene; or, after complex dynamic generation of each simulated element in a single simulated road scene, continuously obtain perception data of surrounding simulated elements through simulated sensors fixedly configured on a single simulated vehicle in the simulated road scene. Obviously, in the existing autonomous driving perception data set generation method, each simulated road scene can only obtain perception data corresponding to a single simulated vehicle, and a large number of simulated road scene modeling operations and simulation element dynamic generation operations are required, and the utilization rate of perception data acquisition of simulated road scenes is extremely low.
[0038] In order to avoid the above technical problems, the present invention shows as follows Figure 1 A method flow chart of a method for batch generation of perception data sets for autonomous driving is shown in FIG. Figure 1 As shown, the batch generation method of the perception data set for autonomous driving includes:
[0039] S10: Acquire real road scene data corresponding to the real road scene, wherein the real road scene data includes vehicle element data, pedestrian element data and environment element data.
[0040] The means of acquiring the real road scene data corresponding to the real road scene can be on-site collection, or obtaining traffic flow data through a real traffic database.
[0041] In a feasible implementation, the embodiment of the present invention uses the open source NGSIM database to obtain traffic flow data, and maps it to a two-dimensional world coordinate system through coordinate conversion technology, thereby ensuring the accuracy and practicality of the data.
[0042] S20: Calculate a real road traffic distribution model corresponding to the real road scene data using statistical and machine learning methods, and generate a simulated road scene corresponding to the real road scene data based on the road traffic distribution model, wherein the simulated road scene includes simulated vehicle elements, simulated pedestrian elements, and simulated environmental elements.
[0043] It should be noted that each simulated road scene included in the simulated road scene, such as simulated vehicle elements, simulated pedestrian elements, and simulated environment elements, corresponds one-to-one to each real road scene element included in each real road scene included in the real road scene data. For example, the position and motion state of the simulated vehicle element a in the simulated road scene correspond to the position and motion state of the vehicle element A in the real road scene, and the position and motion state of the simulated pedestrian element h in the simulated road scene correspond to the position and motion state of the pedestrian element H in the real road scene.
[0044] like Figure 2 As shown, it discloses a scene schematic diagram of a simulated road scene, which includes simulated vehicle elements, simulated pedestrian elements, and simulated environment elements.
[0045] Optionally, in the Unity environment, the present invention obtains a corresponding simulated road scene based on the real road scene data recorded in the database, such as vehicle type, location coordinates and other information, and deploys a model on the virtual scene ground according to the traffic flow.
[0046] S30: Determine any simulated vehicle element in the simulated road scene as an observation object for observation, configure a preset simulation sensor at a preset position of the observation object, and obtain perception data of the observation object on each surrounding simulated road scene element through the simulation sensor, and then mark the simulated road scene elements corresponding to each perception data in the simulated road scene based on the acquired perception data.
[0047] For example, Figure 3In the scene diagram of the simulated road scene shown, when the simulation sensor is set on the simulated vehicle element a as the observation object, the perception data of the simulated vehicle element a on other simulated vehicle elements b, c, d, e, f, g, other simulated pedestrian elements h, i, j, and other simulated environment elements m, k can be obtained.
[0048] Optionally, the labeling method may include: generating a 3D bounding box based on position and size information of other simulated road scene elements obtained from the perception data.
[0049] Preferably, the simulated sensor includes a simulated millimeter-wave radar, a simulated camera, etc.
[0050] When the simulated sensor is a simulated millimeter-wave radar, the acquired perception data is millimeter-wave data; when the simulated sensor is a simulated camera, the acquired perception data is image data.
[0051] Preferably, before S30, the method further includes:
[0052] S21: If it is detected that the simulated road scene has damage or missing preset simulated road scene elements, the damaged or missing simulated road scene elements are determined as simulation elements to be processed, and alternative simulated road scene elements corresponding to the simulation elements to be processed are added to the simulated road scene according to preset element adding rules; the preset simulated road scene elements include vehicle elements, pedestrian elements and environmental elements; the preset element adding rules include the corresponding adding position conditions of each alternative simulated road scene element in the preset simulated road scene.
[0053] The implementation of step S21 can enrich the simulated road scene element content included in the simulated road scene and avoid missing perception data types for different perception targets.
[0054] Preferably, the substitute simulated road scene elements are various simulated road scene elements pre-stored in a road scene element database, and the substitute simulated road scene elements are obtained by simulating different real road scene elements.
[0055] The preset element adding rules must follow the basic laws of the road driving environment. The preset element adding rules can be edited and modified by the staff according to the actual road scene requirements; for example, the simulated vehicle elements included in the replacement simulated road scene elements cannot be added to the positions of other simulated road scene elements; for example, the driving direction of the simulated vehicle elements needs to be consistent with the driving direction of the front and rear simulated vehicle elements; when it is necessary to obtain perception data under extreme road scenarios, the staff can edit and modify the preset element adding rules to create simulated reproductions of scenes such as vehicle accidents.
[0056] It should be noted that the damaged or missing simulated road scene elements can be calculated according to the similarity algorithm. For example, the simulated road scene x only includes simulated vehicle element a, simulated vehicle element b, simulated environment element k, and simulated environment element m. The server traverses the features of each simulated road scene element in the simulated road scene x according to the road scene element database. When it is detected that the simulated road scene x does not have similarities with each simulated pedestrian element that meet the preset conditions, the simulated pedestrian element is determined as the simulated element to be processed, and the simulated pedestrian element h is added as a replacement simulated road scene element from each simulated pedestrian element to the preset position of the sidewalk of the simulated road scene x according to the preset element addition rule.
[0057] For example, when the Figure 3 The simulated road scene shown in the figure has a damaged simulated vehicle element g, and the simulated vehicle element g is determined to be a simulated element to be processed, and the simulated vehicle element g1 is added from each simulated vehicle element to the preset position of the road in the simulated road scene according to the preset element adding rule, such as Figure 4 shown.
[0058] Optionally, the Unity 3D engine can be used as a rendering platform, and a custom script can be used to efficiently load the three-dimensional model in the road scene element database and render it in real time at the corresponding position of the simulated road scene.
[0059] Preferably, the method further comprises:
[0060] If it is detected that the number of simulated road scene elements in the simulated road scene does not meet the preset number, each simulated road scene element that does not meet the preset number will be determined as a simulation element to be added, and replacement simulated road scene elements corresponding to each simulation element to be added will be added to the simulated road scene according to the preset element adding rules.
[0061] Similarly, a large number of the same simulated road scene elements can enrich the stability of the perception data and avoid the reduction of perception data accuracy caused by the abnormality of a single simulated road scene element.
[0062] Preferably, S30 further includes:
[0063] After executing the step of configuring a preset simulation sensor at the preset position of the observation object, the simulated vehicle element is removed from the simulated road scene, and then the step of obtaining the perception data of the observation object on the surrounding simulated road scene elements through the simulation sensor is executed, and then the simulated road scene elements corresponding to each perception data in the simulated road scene are marked according to the acquired perception data, and after the marking is completed, the simulated vehicle element is restored in the simulated road scene.
[0064] For example, when the observed object is Figure 3 When the simulated vehicle element d in the simulated road scene is detected, after the step of configuring a preset simulated sensor at the preset position of the simulated vehicle element d, the simulated vehicle element d is removed from the simulated road scene, as shown in FIG5 , and then the step of obtaining the perception data of the simulated vehicle element d on the surrounding simulated road scene elements through the simulated sensor is performed, and then the simulated road scene elements corresponding to the perception data in the simulated road scene are marked according to the obtained perception data, and after the marking is completed, the simulated vehicle element d is restored in the simulated road scene, that is, Figure 3 .
[0065] The above steps acquire perception data by transforming the simulated vehicle element as the observation object each time, and at the same time reproduce the previous simulated vehicle element and hide the current simulated vehicle element until all simulated vehicle elements in the scene are traversed and image data or millimeter wave data are collected, thereby batch generating perception data sets.
[0066] It should be noted that, since the shapes and sizes of the simulated vehicle elements as the observation objects may be different, in the process of the simulated sensor following the movement of the observation object and collecting perception data, if the simulated sensor itself attached to the observation object is not removed and directly configured according to the same sensor external parameters, it is easy to cause the sensor position to be located inside the observation object body, causing a penetration phenomenon, thereby seriously affecting the data quality generated by the sensor. The present invention directly removes the observation object attached to the simulated sensor in the simulated road scene, while retaining the attachment position of the simulated sensor to the observation object, which can avoid the influence of the penetration phenomenon on the perception data set.
[0067] Preferably, S30 further includes:
[0068] After executing the step of obtaining the perception data of the simulated vehicle element on the surrounding simulated road scene elements through the simulation sensor, the perception data is preprocessed and cleaned, and then based on the preprocessed and cleaned perception data, the step of labeling the simulated road scene elements corresponding to each perception data in the simulated road scene is executed.
[0069] Preprocessing and cleaning the collected perception data can ensure its quality and accuracy.
[0070] S40: Update any unobserved simulated vehicle element in the simulated road scene to an observation object, and execute the step of configuring a preset simulated sensor at a preset position of the observation object in S30 until all simulated vehicle elements are observed, thereby generating a perception data set for autonomous driving.
[0071] S50: Repeat S10 to S40 until the number of perception data sets for autonomous driving meets the preset generation number.
[0072] It should be noted that compared with the existing method for generating autonomous driving perception data sets, the present invention can statically generate simulated road scenes based on simple real road scene data, and through the sequential movement configuration of each simulated vehicle element in the static simulated road scene by the simulation sensor, the perception data of each simulated element of the simulated road scene can be acquired from different simulated vehicle element perspectives. There is no need to rely on a large amount of traffic flow data for the generation of complex dynamic simulated road scenes, nor is there a need for a large amount of real road scene data for a huge amount of simulated road scenes. Batch generation of multiple types of perception data sets for autonomous driving can be achieved, thereby achieving the technical effect of improving the efficiency of generating perception data sets for autonomous driving.
[0073] Compared with the prior art, the method for batch generating perception data sets for autonomous driving provided by the present invention has the following advantages:
[0074] The present invention provides a method for batch generating perception data sets for autonomous driving, which comprises acquiring real road scene data corresponding to a real road scene, wherein the real road scene data comprises vehicle element data, pedestrian element data and environmental element data; using statistical and machine learning methods to calculate a real road flow distribution model corresponding to the real road scene data, and generating a simulated road scene corresponding to the real road scene data according to the road flow distribution model, wherein the simulated road scene comprises simulated vehicle elements, simulated pedestrian elements and simulated environmental elements; determining any simulated vehicle element in the simulated road scene as an observation object for observation, configuring a preset simulation sensor at a preset position of the observation object, and acquiring perception data of the observation object on each surrounding simulated road scene element through the simulation sensor, and then annotating the simulated road scene elements corresponding to each perception data in the simulated road scene according to the acquired perception data; updating any unobserved simulated vehicle element in the simulated road scene as an observation object, repeating the above steps until each simulated vehicle element is observed, and generating a perception data set for autonomous driving, thereby realizing high-efficiency mass generation of specific perception data sets through the acquisition of a small amount of real road scene data and traversal observation of each simulated vehicle element in the simulated road scene.
[0075] In addition, the present invention also utilizes simulated road scene elements to fuse real scene elements in real road scenes, thereby realizing diversified generation of simulated road scenes, and further improving the generation efficiency of specific perception data sets. Those skilled in the art will easily think of other embodiments of the present invention after considering the specification and practicing the invention herein. The present invention is intended to cover any variations, uses or adaptive changes of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the technical field that are not disclosed in the present invention. It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof.
Claims
1. A method for batch generation of perception data sets for autonomous driving, characterized in that: The method comprises: S10: Acquire real road scene data corresponding to the real road scene, wherein the real road scene data includes vehicle element data, pedestrian element data and environment element data; S20: Calculate a real road flow distribution model corresponding to the real road scene data using statistical and machine learning methods, and generate a simulated road scene corresponding to the real road scene data according to the road flow distribution model, wherein the simulated road scene includes simulated vehicle elements, simulated pedestrian elements, and simulated environment elements; S30: Determine any simulated vehicle element in the simulated road scene as an observation object for observation, configure a preset simulation sensor at a preset position of the observation object, and obtain perception data of the observation object on each surrounding simulated road scene element through the simulation sensor, and then mark the simulated road scene element corresponding to each perception data in the simulated road scene according to the acquired perception data; S40: updating any unobserved simulated vehicle element in the simulated road scene as an observed object, and executing the step of configuring a preset simulated sensor at a preset position of the observed object in S30 until all simulated vehicle elements are observed, thereby generating a perception data set for autonomous driving; S50: Repeat S10 to S40 until the number of perception data sets for autonomous driving meets the preset generation number.
2. The method according to claim 1, characterized in that: Before S30, the method further includes: If it is detected that the simulated road scene has damage or missing preset simulated road scene elements, each damaged or missing simulated road scene element is determined as a to-be-processed simulated element, and a replacement simulated road scene element corresponding to each to-be-processed simulated element is added to the simulated road scene according to a preset element adding rule; The preset simulated road scene elements include vehicle elements, pedestrian elements and environmental elements; The preset element adding rule includes the corresponding adding position conditions of each replacement simulated road scene element in the preset simulated road scene.
3. The method according to claim 2, characterized in that The substitute simulated road scene elements are various simulated road scene elements pre-stored in a road scene element database, and the substitute simulated road scene elements are obtained by simulating different real road scene elements.
4. The method according to claim 2, characterized in that: The method further comprises: If it is detected that the number of simulated road scene elements in the simulated road scene does not meet the preset number, each simulated road scene element that does not meet the preset number will be determined as a simulation element to be added, and replacement simulated road scene elements corresponding to each simulation element to be added will be added to the simulated road scene according to the preset element adding rules.
5. The method according to claim 1, characterized in that The S30 also includes: After executing the step of configuring a preset simulation sensor at the preset position of the observation object, the simulated vehicle element is removed from the simulated road scene, and then the step of obtaining the perception data of the observation object on the surrounding simulated road scene elements through the simulation sensor is executed, and then the simulated road scene elements corresponding to each perception data in the simulated road scene are marked according to the acquired perception data, and after the marking is completed, the observation object is restored in the simulated road scene.
6. The method according to claim 1, characterized in that The S30 also includes: After executing the step of obtaining the perception data of the observed object on each surrounding simulated road scene element through the simulation sensor, the perception data is preprocessed and cleaned, and then based on the preprocessed and cleaned perception data, the step of labeling the simulated road scene elements corresponding to each perception data in the simulated road scene is executed.
7. The method according to claim 1, characterized in that The simulated sensor includes a simulated millimeter wave radar and a simulated camera.