Automatic driving test scene generation method and system based on natural driving data
By combining the generated adversarial network and the conditional generation adversarial network with deep learning models, the problems of data redundancy, insufficient scene diversity and difficulty in generating dynamic scenarios in autonomous driving tests are solved, and diverse and highly realistic test scenarios are generated, which improves the coverage and accuracy of autonomous driving tests.
Patent Information
- Application Number
- CN202510628589.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-15
- Publication Date
- 2025-08-22
AI Technical Summary
The prior art has problems such as data redundancy and noise, insufficient scene diversity and difficulty in generating dynamic scenarios in autonomous driving tests, resulting in inaccurate tests and incomplete coverage.
Generative adversarial networks and conditional generation adversarial networks are used to combine deep learning models to preprocess, feature extraction and scene reconstruction of natural driving data to generate diverse and highly realistic test scenarios. Dynamic interactive features are extracted through joint training of generators and discriminators, and test scenarios are generated based on different traffic environment conditions.
With the reduction of manual intervention, the accurate generation of diverse test scenarios is improved, the coverage and realism of autonomous driving tests is solved, the problems of data redundancy and insufficient scenario diversity are supported, and the dynamic update and expansion of test scenario data are supported.
Smart Images
Figure CN120524809A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of autonomous driving function testing, and in particular to a method and system for generating autonomous driving test scenarios based on natural driving data. Background Art
[0002] With the rapid development of autonomous driving technologies (such as intelligent driving and advanced driver assistance systems), efficient and comprehensive testing of autonomous driving systems has become a focus of industry attention. Traditional testing methods rely primarily on manually designed test scenarios. While these methods can cover some typical scenarios, they fail to fully reflect the complexity and diversity of real-world road conditions. Furthermore, these manually designed scenarios often lack sufficient randomness and dynamism, making them inadequate for verifying the robustness of autonomous driving systems in complex environments.
[0003] In recent years, test scenario generation methods based on naturalistic driving data have gained increasing attention. Naturalistic driving data can realistically reflect the complex traffic conditions in the road environment, including multiple factors such as vehicles, pedestrians, road signs, and weather conditions. However, existing technologies for processing naturalistic driving data have the following problems:
[0004] i. Data redundancy and noise: Natural driving data often contains a large amount of redundant information and noise. Directly using this data will lead to inefficient and inaccurate test scenarios.
[0005] ii. Insufficient scenario diversity: Existing methods can only extract a limited number of scenario types when generating test scenarios, making it difficult to cover all possible traffic conditions.
[0006] iii. Difficulty in generating dynamic scenarios: Dynamic interactions in natural driving data (such as game behaviors between vehicles) are difficult to model accurately, resulting in a lack of realism in the generated test scenarios. Summary of the Invention
[0007] In view of the above, the present invention aims to provide a method and system for generating autonomous driving test scenarios based on natural driving data to solve the technical problems mentioned above.
[0008] The technical solution adopted in the present invention is as follows:
[0009] In a first aspect, the present invention provides a method for generating an autonomous driving test scenario based on natural driving data, comprising:
[0010] Preprocessing the collected natural driving data;
[0011] Extract scene features from pre-processed natural driving data;
[0012] Reconstructing the scene features using a generative adversarial network to generate a diverse first test scene;
[0013] A conditional generative adversarial network is used to combine the scenario features and / or the first test scenario with different traffic environment conditions to generate a diversified second test scenario.
[0014] In at least one possible implementation, the preprocessing includes:
[0015] Cleaning and labeling the natural driving data, and removing redundant information and noise therein;
[0016] The state information of key traffic participants is obtained therefrom, wherein the traffic participants include at least: vehicles, pedestrians, and traffic signs.
[0017] In at least one possible implementation, the scene feature extraction includes: combining a convolutional neural network and a long short-term memory network, and using at least state information of traffic participants to extract traffic scene features.
[0018] In at least one possible implementation, generating a first diversified test scenario includes:
[0019] By jointly training the generator and discriminator of the generative adversarial network, dynamic interaction information is learned from the traffic scene characteristics of natural driving data;
[0020] A first test scenario is generated according to the dynamic interaction information.
[0021] In at least one possible implementation, the scene generation method includes a mechanism for evaluating and adjusting the complexity of the generated scene:
[0022] Before actually deploying the first test scenario or the second test scenario, scoring the scenario complexity;
[0023] According to the scoring results and the requirements of the autonomous driving function to be tested, the parameters of the first test scenario or the second test scenario are adjusted in a targeted manner.
[0024] In at least one possible implementation, the first test scenario or the second test scenario is combined with digital twin technology to construct a virtual mirror of the real road environment, which is used to generate a test scenario synchronized with the real traffic environment in real time.
[0025] In at least one possible implementation manner, the traffic environment conditions include at least weather, lighting, traffic density, roads, and buildings.
[0026] In a second aspect, the present invention provides an autonomous driving test scenario generation system based on natural driving data, comprising:
[0027] A preprocessing module, used to preprocess the collected natural driving data;
[0028] Feature extraction module, used to extract scene features from pre-processed natural driving data;
[0029] A first scenario generation module is configured to reconstruct the scenario features using a generative adversarial network to generate a diversified first test scenario;
[0030] The second scenario generation module uses a conditional generative adversarial network to combine the scenario features and / or the first test scenario, as well as different traffic environment conditions, to generate a diversified second test scenario.
[0031] Compared to existing technologies, the key design concept of this invention lies in leveraging the dynamic scene reconstruction approach of a generative adversarial network (GAN). This approach uses a deep learning model to automatically analyze natural driving data and extract key scene features, thereby generating diverse and highly realistic test scenarios. Specifically, the GAN's generator and discriminator are jointly trained to extract complex dynamic interaction features from natural driving data and reconstruct highly dynamic and realistic test scenarios. Furthermore, by introducing a conditional GAN (C-GAN), diverse test scenarios are generated based on varying traffic conditions, such as weather, lighting, and traffic density. This significantly enhances the coverage of autonomous driving test scenarios, even encompassing edge cases within many test scenarios. This invention accurately generates diverse and realistic test scenarios while minimizing manual intervention. This effectively addresses current issues in autonomous driving testing, such as data redundancy, insufficient scene diversity, and difficulty generating dynamic scenarios. It also reliably supports the dynamic updating and expansion of test scenario data. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described below with reference to the accompanying drawings, in which:
[0033] Figure 1 A schematic diagram of a flow chart of a method for generating an autonomous driving test scenario based on natural driving data provided by an embodiment of the present invention;
[0034] Figure 2 A block diagram of a system for generating autonomous driving test scenarios based on natural driving data provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0035] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.
[0036] Before developing the following embodiments of the present invention, it is necessary to sort out several technical difficulties of the present invention:
[0037] (1) Extraction and modeling of dynamic interaction features: Since the dynamic interaction behaviors in natural driving data are complex and changeable (such as vehicle lane changes, pedestrians crossing, etc.), the following embodiments consider how to accurately extract and reconstruct these behaviors.
[0038] (2) Ensuring scenario diversity: When generating test scenarios, the present invention further considers how to ensure the diversity and coverage of scenarios, especially how to avoid generating repeated or overly simple scenarios.
[0039] The present invention proposes an embodiment of a method for generating an autonomous driving test scenario based on natural driving data. Specifically, Figure 1 shown, including:
[0040] Step S1: pre-processing the collected natural driving data;
[0041] Specifically, this can include at least cleaning and labeling the natural driving data, removing redundant information and noise, and extracting status information of key traffic participants (such as vehicles, pedestrians, and traffic signs). Furthermore, noise filtering can be performed using DBSCAN-based trajectory clustering to remove drift points. For semantic labeling, multi-sensor fusion labeling tools (LiDAR + camera) can be used to effectively improve labeling accuracy.
[0042] Step S2: extracting scene features from the pre-processed natural driving data;
[0043] In actual operation, a convolutional neural network (CNN) and a long short-term memory network (LSTM) can be combined, and at least the state information of the traffic participants can be used to extract traffic scene features, so as to identify several representative traffic scenes.
[0044] Here, an implementation example of a dual-feature extraction network is provided, wherein for the application scenario of the present invention, the dual features mainly refer to static features and dynamic features.
[0045] (1) About static feature extraction
[0046] Input: High-precision map (OpenDRIVE format);
[0047] Network: Graph Convolutional Network (GCN) + Attention Pooling;
[0048] Output: 256-dimensional road feature vector (including road curvature, slope, and sign semantics).
[0049] (2) About dynamic feature extraction
[0050] Input: target trajectory (targets mainly refer to vehicles / pedestrians)
[0051] Network: Spatiotemporal Transformer (time series length is set to 5s)
[0052] Output: interaction threat matrix + intention probability distribution.
[0053] Step S3: reconstructing the scene features using a generative adversarial network (GAN) to generate a diversified first test scene;
[0054] For example, by jointly training the generator and discriminator of a generative adversarial network, dynamic interaction information (such as vehicle lane changes, pedestrian crossing, etc.) can be learned from the traffic scene characteristics of natural driving data, and different first test scenarios with high dynamics and realism can be generated based on the dynamic interaction information, thereby ensuring that the generated scenarios are highly real and dynamic.
[0055] For the first test scenario, i.e., the generation of the basic scenario, it can be supplemented that the generator architecture preferably adopts a U-Net structure, combined with the above-mentioned static features injected as skip-connection; while the discriminator can use a multi-scale PatchGAN (local 10m + global 100m). In some embodiments of the present invention, the loss function is designed as:
[0056]
[0057] in: To generate adversarial loss, the true and false probabilities of the discriminator output are calculated;
[0058] For the interaction force field loss, the interaction force field prediction network is pre-trained Calculate the force field difference between the real scene and the generated scene:
[0059]
[0060] Validate the physics engine for loss, detecting violations of physics (such as vehicle penetration) through simulation environments such as Carla and calculating penalties.
[0061] Schematically, where =0.5, =0.2 is the weight coefficient optimized by grid search.
[0062] Step S4: Using a conditional generative adversarial network (Conditional GAN), combining the scenario features and / or the first test scenario, as well as different traffic environment conditions, to generate a diversified second test scenario.
[0063] In other words, a conditional generative adversarial network (GAN) is used to generate a diverse set of secondary test scenarios based on varying traffic conditions (such as weather, lighting, and traffic density). This significantly enhances the coverage of test scenarios, ultimately resulting in a wider range of traffic conditions, including marginal and low-probability events. In practice, the generation of the secondary test scenarios, or expanded scenarios, can include conditional encoding: weather and lighting parameters are injected by the AdaIN module. For edge case generation, for example, when the input conditions are [visibility < 50m, traffic density > 0.8], a Gaussian mixture sampling mode can be activated.
[0064] The above-mentioned embodiments, through the accurate extraction and modeling of dynamic interaction features and the combination of generative adversarial networks (GANs) and conditional generative adversarial networks (Conditional GANs), ensure that the generated scenarios can not only reflect the complex dynamic behaviors in real road environments, but also cover various possible and changing traffic conditions.
[0065] It can also be added here that some preferred embodiments of the present invention also include a mechanism for evaluating and adjusting the complexity of the generated scenario, that is, before the generated scenario is actually deployed, its complexity is scored, and based on the scoring results, the scenario parameters are adjusted and optimized in a targeted manner according to the test requirements of a certain autonomous driving function.
[0066] The following evaluation indicators and optimization strategies can be used as reference for implementation. Evaluation indicator table:
[0067]
[0068] Afterwards, reinforcement learning (PPO algorithm) can be used to dynamically adjust the generator parameters to execute the optimization strategy.
[0069] Based on the several embodiments provided by the present invention above, it can be expanded and supplemented that, by developing a virtual-reality combined test platform, the virtual scene generated by this solution can be seamlessly embedded in the real road test to enhance the comprehensiveness of the test; and the technical concept of the present invention can be combined with digital twin technology to construct a virtual mirror image of the real road environment, thereby generating a test scene synchronized with the real environment in real time.
[0070] Finally, in combination with the embodiments of the aforementioned methods and solutions, from the perspective of practical application, it can be added that the aforementioned data preprocessing can rely on: NVIDIA DGX-2 cluster for processing, with a processing speed of ≥1000 frames / second; and dual feature extraction can refer to the following:
[0071] Static branch: deploy the GCN model (PyTorch-Geometric library);
[0072] Dynamic branching: Customized Transformer (supports CUDA acceleration).
[0073] Scene generation can be divided into:
[0074] Basic generation unit: 4-layer 3D convolution generator (1.2M parameters);
[0075] Conditional control unit: FPGA implements AdaIN real-time encoding.
[0076] For the automated testing platform involved in the evaluation feedback, Carla+ROS2 can be integrated to support parallel testing of 20 scenarios per second.
[0077] In summary, the key design concept of this invention lies in leveraging the dynamic scene reconstruction approach of a generative adversarial network (GAN). This approach uses a deep learning model to automatically analyze natural driving data and extract key scene features, thereby generating diverse and highly realistic test scenarios. Specifically, the GAN's generator and discriminator are jointly trained to extract complex dynamic interaction features from natural driving data and reconstruct highly dynamic and realistic test scenarios. Furthermore, by introducing a conditional GAN (C-GAN), diverse test scenarios are generated based on varying traffic conditions, such as weather, lighting, and traffic density. This significantly enhances the coverage of autonomous driving test scenarios, even encompassing edge cases within many test scenarios. This invention accurately generates diverse and realistic test scenarios while minimizing manual intervention. This effectively addresses current issues in autonomous driving testing, such as data redundancy, insufficient scene diversity, and the difficulty of generating dynamic scenarios. It also reliably supports the dynamic updating and expansion of test scenario data.
[0078] Corresponding to the above embodiments and preferred solutions, the present invention also provides an embodiment of an autonomous driving test scenario generation system based on natural driving data, such as Figure 2 As shown, it may specifically include the following components:
[0079] A preprocessing module 201 is used to preprocess the collected natural driving data;
[0080] A feature extraction module 202 is used to extract scene features from the pre-processed natural driving data;
[0081] A first scenario generation module 203 is configured to reconstruct the scenario features using a generative adversarial network to generate a diversified first test scenario;
[0082] The second scenario generation module 204 uses a conditional generative adversarial network to combine the scenario features and / or the first test scenario, as well as different traffic environment conditions, to generate a diversified second test scenario.
[0083] It should be understood that the above Figure 2 The division of components in the system for generating autonomous driving test scenarios based on natural driving data is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into a single physical entity or physically separated. Furthermore, these components can all be implemented as software called through processing elements; some can also be implemented as software called through processing elements, while others can be implemented as hardware. For example, one of the aforementioned modules can be a standalone processing element or integrated into a single chip in an electronic device. The implementation of other components is similar. Furthermore, these components can all or partially be integrated together or implemented independently. During implementation, each step of the above method or each of the above components can be completed by hardware integrated logic circuits in the processor element or by software instructions.
[0084] For example, the above components may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more microprocessors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, these components may be integrated together to form a system-on-a-chip (SOC).
[0085] If the expressions expressing directions are mentioned in the embodiments of the present invention, they are relative concepts based on the embodiments. In addition, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can represent: a, b, c, a and b, a and c, b and c or a, b and c, where a, b, c can be single or multiple.
[0086] The above describes in detail the structure, features and effects of the present invention based on the embodiments shown in the drawings, but the above is only a preferred embodiment of the present invention. It should be noted that the technical features involved in the above embodiments and their preferred modes can be reasonably combined and matched into a variety of equivalent schemes by those skilled in the art without departing from or changing the design ideas and technical effects of the present invention; therefore, the scope of implementation of the present invention is not limited to what is shown in the drawings. Any changes made in accordance with the concept of the present invention, or modifications to equivalent embodiments with equivalent changes, which still do not exceed the spirit covered by the description and drawings, should be within the scope of protection of the present invention.
Claims
1. A method for generating autonomous driving test scenarios based on natural driving data, characterized in that: include: Preprocessing the collected natural driving data; Extract scene features from pre-processed natural driving data; Reconstructing the scene features using a generative adversarial network to generate a diverse first test scene; A conditional generative adversarial network is used to combine the scenario features and / or the first test scenario with different traffic environment conditions to generate a diversified second test scenario.
2. The method for generating autonomous driving test scenarios based on natural driving data according to claim 1, characterized in that: The pretreatment includes: Cleaning and labeling the natural driving data, and removing redundant information and noise therein; The state information of key traffic participants is obtained therefrom, wherein the traffic participants include at least: vehicles, pedestrians, and traffic signs.
3. The method for generating autonomous driving test scenarios based on natural driving data according to claim 2, characterized in that: The scene feature extraction includes: combining a convolutional neural network and a long short-term memory network, and using at least state information of traffic participants to extract traffic scene features.
4. The method for generating autonomous driving test scenarios based on natural driving data according to claim 1, wherein: Generating a diversified first test scenario includes: By jointly training the generator and discriminator of the generative adversarial network, dynamic interaction information is learned from the traffic scene characteristics of natural driving data; A first test scenario is generated according to the dynamic interaction information.
5. The method for generating autonomous driving test scenarios based on natural driving data according to claim 1, wherein: The scenario generation method includes a mechanism for evaluating and adjusting the complexity of the generated scenario: Before actually deploying the first test scenario or the second test scenario, scoring the scenario complexity; According to the scoring results and the requirements of the autonomous driving function to be tested, the parameters of the first test scenario or the second test scenario are adjusted in a targeted manner.
6. The method for generating autonomous driving test scenarios based on natural driving data according to claim 1, characterized in that: The first test scenario or the second test scenario is combined with digital twin technology to construct a virtual mirror of the real road environment, which is used to generate a test scenario synchronized with the real traffic environment in real time.
7. The method for generating an autonomous driving test scenario based on natural driving data according to any one of claims 1 to 6, characterized in that: The traffic environment conditions include at least weather, lighting, traffic density, roads, and buildings.
8. A system for generating autonomous driving test scenarios based on natural driving data, characterized in that: include: A preprocessing module, used to preprocess the collected natural driving data; Feature extraction module, used to extract scene features from pre-processed natural driving data; A first scenario generation module is configured to reconstruct the scenario features using a generative adversarial network to generate a diversified first test scenario; The second scenario generation module uses a conditional generative adversarial network to combine the scenario features and / or the first test scenario, as well as different traffic environment conditions, to generate a diversified second test scenario.