Intelligent driving algorithm simulation test method and device, equipment and storage medium

Abstract

The invention discloses an intelligent driving algorithm simulation test method, device and equipment and a storage medium, relates to the technical field of intelligent driving testing, and discloses an intelligent driving algorithm simulation test method which comprises the following steps: acquiring vehicle-end environment data and road-end environment data, obtaining a target scene according to the vehicle end environment data and the road end environment data; according to the vehicle end environment data, the road end environment data and the target scene, training based on a diffusion transformation model to obtain a world model; generating a simulation scene according to the target scene and the world model; and performing intelligent driving algorithm simulation test according to the simulation scene. Through vehicle end and road end environment data fusion and training based on a diffusion transformation model, high-fidelity simulation scene generation and efficient simulation testing of an intelligent driving algorithm are realized.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving test technology, and in particular to intelligent driving algorithm simulation test methods, devices, equipment and storage media. Background Art

[0002] With the rapid development of artificial intelligence technology, intelligent driving technology has gradually become the core innovation direction of the automotive industry, and end-to-end intelligent driving methods are gradually replacing traditional modular algorithm systems. Traditional intelligent driving solutions usually divide the overall algorithm into independent modules such as perception, decision-making, planning and control, and each module transmits data through an interface, but this modular design often brings problems such as increased system latency, local optimality and high algorithm complexity. In addition, traditional solutions lack generalization capabilities when faced with rare long-tail scenarios, and are difficult to cope with complex and diverse actual road environments. In contrast, the end-to-end intelligent driving solution directly maps the input sensor data to the vehicle's driving control output through deep learning technology, which simplifies the data flow process and improves the overall response speed and robustness of the system.

[0003] However, the development and verification of end-to-end intelligent driving algorithms have put forward higher requirements for simulation testing technology, especially the need for high-quality and high-fidelity simulation scenarios to ensure the reliability and safety of the algorithms in real environments. However, existing simulation technologies mainly rely on traditional simulation software to generate scenarios, which are far from the actual road scenarios. In particular, when generating rare dangerous scenarios and long-tail cases, the efficiency is low and it is difficult to meet the simulation requirements of end-to-end algorithms.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide an intelligent driving algorithm simulation test method, device, equipment and storage medium, aiming to solve the technical problem of insufficient fidelity of simulation scenes in end-to-end intelligent driving simulation testing.

[0006] To achieve the above objectives, the present application proposes a simulation test method for an intelligent driving algorithm, the method comprising:

[0007] Acquire vehicle-side environmental data and road-side environmental data, and obtain a target scene according to the vehicle-side environmental data and the road-side environmental data;

[0008] According to the vehicle-side environment data, the road-side environment data and the target scene, training is performed based on a diffusion transformation model to obtain a world model;

[0009] Generate a simulation scene according to the target scene and the world model;

[0010] Perform intelligent driving algorithm simulation test according to the simulation scenario.

[0011] In one embodiment, the step of generating a simulation scene according to the target scene and the world model comprises:

[0012] Obtaining dense description information of the target scene according to the visual language model;

[0013] Obtaining initial road scene information according to the world model;

[0014] A simulation scene is generated according to the initial road scene and the dense description information.

[0015] In one embodiment, the step of generating a simulation scene according to the initial road scene and the dense description information includes:

[0016] Querying the initial road scene according to the dense description information to obtain a target road scene;

[0017] According to the target road scene, a multi-view road scene is obtained;

[0018] Scene generalization is performed based on the multi-view road scene to generate a simulation scene.

[0019] In one embodiment, the step of performing scene generalization according to the multi-view road scene to generate a simulation scene includes:

[0020] According to the multi-view road scene, scene weather, current vehicle behavior, pedestrian information, and target vehicle behavior are obtained;

[0021] Performing environment generalization according to the scene weather to obtain an environment generalization set;

[0022] Generalizing the vehicle behavior according to the current vehicle behavior to obtain a generalized vehicle behavior set;

[0023] Generalize the road scene according to the pedestrian information and the target vehicle behavior to obtain a road scene generalization set;

[0024] The scene generalization is performed according to the environment generalization set, the vehicle behavior generalization set and the road scene generalization set to generate a simulation scene.

[0025] In one embodiment, the step of obtaining a world model by training based on a diffusion transformation model according to the vehicle-side environment data, the road-side environment data and the target scene comprises:

[0026] Obtaining a training data set according to the vehicle-side environmental data and the road-side environmental data;

[0027] Performing data preprocessing on the training data set to obtain a preprocessed data set;

[0028] According to the preprocessed data set, pre-training is performed based on a diffusion transformation model to obtain an initial world model;

[0029] A world model is obtained according to the preprocessed data set, the target scene and the initial world model.

[0030] In one embodiment, the step of obtaining the world model according to the preprocessed data set, the target scene and the initial world model comprises:

[0031] According to the preprocessed data set, a typical driving scene is obtained;

[0032] According to the typical driving scenario and the target scenario, enhanced training is performed based on the initial world model to obtain a world model.

[0033] In one embodiment, the step of acquiring vehicle-side environment data and road-side environment data, and obtaining a target scene according to the vehicle-side environment data and the road-side environment data includes:

[0034] Acquire vehicle-side environmental data and road-side environmental data, and fuse the vehicle-side environmental data with the road-side environmental data to obtain target environmental data;

[0035] Analyze the target environment data to obtain dangerous scenarios and abnormal driving behaviors;

[0036] A target scene is obtained according to the dangerous scene and the abnormal driving behavior.

[0037] In addition, to achieve the above purpose, the present application also proposes an intelligent driving algorithm simulation test device, the intelligent driving algorithm simulation test device comprising:

[0038] An acquisition module, used to acquire vehicle-side environmental data and road-side environmental data, and obtain a target scene according to the vehicle-side environmental data and the road-side environmental data;

[0039] A training module, configured to perform training based on a diffusion transformation model according to the vehicle-side environment data, the road-side environment data and the target scene to obtain a world model;

[0040] A generation module, used to generate a simulation scene according to the target scene and the world model;

[0041] The test module is used to perform intelligent driving algorithm simulation test according to the simulation scenario.

[0042] In addition, to achieve the above-mentioned objectives, the present application also proposes an intelligent driving algorithm simulation test device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the intelligent driving algorithm simulation test method as described above.

[0043] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the intelligent driving algorithm simulation test method described above are implemented.

[0044] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the intelligent driving algorithm simulation test method described above.

[0045] One or more technical solutions proposed in this application have at least the following technical effects:

[0046] Through comprehensive analysis of vehicle-side environmental data and road-side environmental data, representative target scenes are generated; the diffusion transformation model is used to efficiently train vehicle-side and road-side environmental data and target scenes, and a world model that can capture the characteristics of complex road scenes is constructed; based on the target scenes and world models, highly realistic and multi-perspective simulation scenes are generated, and intelligent driving algorithms are simulated and tested through these scenes. Through the above technical means, the problems of insufficient scene generation accuracy and low coverage of complex scenes in traditional simulation methods are overcome, so that simulation scenes can be closer to the real environment, thereby improving the accuracy and applicability of intelligent driving algorithms under extreme and complex conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0049] Figure 1 A flowchart of the first embodiment of the intelligent driving algorithm simulation test method of the present application is provided;

[0050] Figure 2A schematic diagram of the world model principle provided for the first embodiment of the intelligent driving algorithm simulation test method of this application;

[0051] Figure 3 A flowchart diagram of the second embodiment of the intelligent driving algorithm simulation test method of the present application;

[0052] Figure 4 A schematic diagram of the training and use of the world model provided in Example 2 of the intelligent driving algorithm simulation test method of this application;

[0053] Figure 5 A schematic diagram of simulation scenario generation provided in Example 2 of the intelligent driving algorithm simulation test method of this application;

[0054] Figure 6 This is a schematic diagram of the module structure of the intelligent driving algorithm simulation test device according to an embodiment of the present application;

[0055] Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the intelligent driving algorithm simulation test method in the embodiment of the present application.

[0056] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0057] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0058] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0059] The main solution of the embodiment of the present application is: to obtain vehicle-side environmental data and road-side environmental data, and to obtain a target scene based on the vehicle-side environmental data and the road-side environmental data; to perform training based on a diffusion transformation model according to the vehicle-side environmental data, the road-side environmental data and the target scene to obtain a world model; to generate a simulation scene according to the target scene and the world model; and to perform a simulation test of an intelligent driving algorithm according to the simulation scene.

[0060] In this embodiment, for ease of description, the following description is made using the identification of intelligent driving algorithm simulation test equipment as the execution subject.

[0061] Due to the lack of fidelity of simulation scenes in the prior art end-to-end intelligent driving simulation test, this application provides a solution, which generates representative target scenes through comprehensive analysis of vehicle-side environmental data and road-side environmental data; uses the diffusion transformation model to efficiently train vehicle-side and road-side environmental data and target scenes, and builds a world model that can capture the characteristics of complex road scenes; based on the target scenes and world models, highly realistic and multi-perspective simulation scenes are generated, and intelligent driving algorithms are simulated and tested through these scenes. Through the above technical means, the problems of insufficient scene generation accuracy and low coverage of complex scenes in traditional simulation methods are overcome, so that the simulation scenes can be closer to the real environment, thereby improving the test accuracy and applicability of intelligent driving algorithms under extreme and complex conditions.

[0062] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device that can realize the above functions, a cloud, an intelligent driving algorithm simulation test device, etc. The following takes the intelligent driving algorithm simulation test device as an example to illustrate this embodiment and the following embodiments.

[0063] Based on this, the embodiment of the present application provides a simulation test method for an intelligent driving algorithm, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the intelligent driving algorithm simulation test method of the present application.

[0064] In this embodiment, the intelligent driving algorithm simulation test method includes steps S10 to S40:

[0065] Step S10, acquiring vehicle-side environmental data and road-side environmental data, and obtaining a target scene according to the vehicle-side environmental data and the road-side environmental data;

[0066] It should be noted that vehicle-side environmental data is the surrounding environment information collected in real time by sensors installed on the vehicle, mainly including road conditions, the location of traffic participants and their dynamic behavior. Sensors on the vehicle include but are not limited to cameras, millimeter-wave radars, lidars and ultrasonic sensors. Cameras are used to capture image information of road markings, traffic signs and other vehicles; millimeter-wave radars are used to detect the position, speed and direction of surrounding objects; lidars provide accurate three-dimensional environmental modeling; and ultrasonic sensors are used for close-range object detection. These data are collected at a high frequency to provide a real-time view of the vehicle's surroundings. For example, the camera on the vehicle captures the status of the traffic light on the road ahead, and the millimeter-wave radar detects a decelerating vehicle 30 meters ahead. The vehicle-side environmental data will include the status of the traffic light and the speed and position of the vehicle ahead. After the data is collected, the data can be collected through T-Box, mobile APP, pre-installed collection equipment, etc., and the vehicle-side environmental data can be sent to the cloud. Road-side environmental data is real-time traffic and environmental information collected by fixed sensor equipment on the road, covering a wide range of road conditions, including but not limited to roadside cameras, traffic light sensors and geomagnetic coils. Roadside cameras are used to record all traffic participants on the road, including but not limited to vehicles, pedestrians or the dynamic behavior of vehicles; traffic light sensors monitor the status and changes of traffic lights; geomagnetic coils record the speed and flow information of vehicles passing through specific sections of the road. These data can make up for the field of view limitations of vehicle-side sensors and provide background information on the overall traffic environment. After collecting the data, it can be uploaded to the cloud for storage and processing via wireless networks.

[0067] In addition, it should be noted that the target scene refers to a dangerous driving situation with specific meaning or importance extracted from the vehicle-side environmental data and the road-side environmental data, which is used to simulate and evaluate the performance of intelligent driving algorithms in complex or extreme situations. These scenarios usually include abnormal driving behaviors, such as sudden lane changes or sudden braking, traffic conflicts, such as the risk of approach between vehicles and pedestrians or other vehicles, and driving scenarios in harsh environments, such as rainy and snowy weather or low visibility. The extraction of target scenes requires the fusion and analysis of the original data to identify dangerous features in the scene and classify the scene. For example, at a traffic light intersection, it is detected that the vehicle is turning left, and a reverse vehicle suddenly appears in front and pedestrians are crossing the zebra crossing on the right. This complex traffic situation can be defined as a target scene of a dangerous driving scenario.

[0068] It is understandable that the vehicle-side sensors collect dynamic information around the vehicle, including the speed and position of the vehicle in front, the information of road signs, the position of pedestrians and their dynamic behavior. At the same time, the road-side equipment collects overall traffic information, such as the status of traffic lights, the movement trajectory of other vehicles in the surrounding road sections, and traffic flow data. The collected vehicle-side environmental data and road-side environmental data are aligned in time and space dimensions, and the data fusion algorithm is used to remove redundant information, eliminate noise and fill in possible data information gaps to generate consistent road environment data. The fused environmental data is analyzed by the dangerous scene recognition model. The model evaluates the data from the dimensions of traffic conflicts, abnormal driving behaviors, and environmental risks, and identifies situations with potential dangerous characteristics, such as detecting that the vehicle is accelerating through the yellow light and pedestrians are entering the zebra crossing area from the right. Finally, the situation is extracted as a target scene for subsequent intelligent driving algorithm simulation testing.

[0069] In a feasible implementation, step S10 may include steps S11 to S13:

[0070] Step S11, acquiring vehicle-side environmental data and road-side environmental data, and fusing the vehicle-side environmental data and the road-side environmental data to obtain target environmental data;

[0071] It should be noted that the target environment data is a comprehensive data set formed by integrating vehicle-side environment data and road-side environment data. It contains all the characteristics of both, making up for the limitations of a single data source and providing complete data support for further identification and analysis of dangerous scenes. The target environment data can include real-time driving information in front of the vehicle, as well as the location and behavior of pedestrians at intersections provided by roadside cameras.

[0072] It is understandable that the preliminary fusion operation may include time synchronization and spatial mapping. The vehicle-side sensors collect dynamic data around the vehicle, such as the position, speed and traffic sign information of the vehicle ahead, and the road-side sensor equipment collects the overall traffic flow, traffic light status and pedestrian behavior of the road section. By performing time synchronization and spatial mapping operations on these two types of data, it can be ensured that the data sources are consistent in time and geographical location. After the preliminary fusion, the data fusion algorithm is used to remove redundant information and repair missing data, and the two types of data are integrated into unified target environment data. For example, the vehicle-side data identifies that the vehicle on the road ahead is slowing down, and the road-side data provides information on the switching of traffic lights. These two types of data are fused to obtain complete target environment data to provide support for subsequent analysis.

[0073] Step S12, analyzing the target environment data to obtain dangerous scenarios and abnormal driving behaviors;

[0074] It should be noted that dangerous scenarios are traffic situations with potential safety hazards analyzed from the target environment data, such as possible collisions between vehicles or conflicts with pedestrians. Abnormal driving behavior refers to situations where the behavior of vehicles or traffic participants deviates from normal traffic rules or driving habits, such as sudden braking, driving out of lane or driving in the wrong direction.

[0075] It is understandable that the dynamic information in the target environment data is analyzed, and pattern recognition algorithms or machine learning models are used to detect potential dangers and abnormal behaviors. For example, by analyzing the acceleration changes of the vehicle, the trajectory deviation, or the direction of pedestrian movement, dangerous scenes such as the vehicle is about to rear-end or side collision can be identified, and abnormal driving behaviors such as a vehicle suddenly changing lanes or speeding can be detected. Dangerous scenes and abnormal behaviors can be extracted.

[0076] Step S13, obtaining a target scene according to the dangerous scene and the abnormal driving behavior.

[0077] It is understandable that by extracting features and performing correlation analysis on dangerous scenes and abnormal driving behaviors, combined with the time, location and traffic rules information in the scene, a complete target scene is generated. For example, at a traffic light intersection, when a vehicle turns left, a dangerous sudden braking behavior occurs in front, and at the same time, a pedestrian on the zebra crossing on the right crosses the road without observing the traffic light. These features together form a complex target scene that can accurately describe the current dangerous driving conditions.

[0078] Step S20, training based on a diffusion transformation model according to the vehicle-side environment data, the road-side environment data and the target scene to obtain a world model;

[0079] It should be noted that the Diffusion Transformer (DiT) model is a deep learning method based on the denoising diffusion probability model and the Transformer architecture. It extracts the features of the data by gradually denoising through the diffusion process, and uses the Transformer architecture to capture the complex spatiotemporal dependencies in the data. The core idea of ​​the diffusion process is to map complex data to a high-dimensional distribution, and gradually restore the target data through the reverse process of denoising, thereby generating high-quality road scenes. The world model is a high-precision road scene generation model generated after training the diffusion transformer model. It can generate multi-perspective scenes that conform to real physical laws based on the input data, and is particularly good at handling complex dynamic environments and long-tail scenes. Long-tail scenes refer to special or extreme driving situations that have a low probability of occurring in actual traffic but are of great significance to the testing and optimization of intelligent driving algorithms, such as complex weather, sudden accidents, or rare traffic behaviors.

[0080] It can be understood that the vehicle-side environmental data, road-side environmental data, and target scene are input into the diffusion transformation model, and the potential features in the data are extracted through the gradual diffusion and denoising mechanism. In this process, the model gradually learns the dynamic relationship between the vehicle-side and road-side data, as well as the dangerous behavior patterns in the target scene. Preliminary training is performed on large-scale ordinary scene data to extract common features. After preliminary training, extreme data in the target scene is added to further strengthen learning of long-tail cases and complex scene features to optimize the parameters and weights of the model. After multiple rounds of training, the diffusion transformation model completes the comprehensive learning of scene features and can generate a world model with high quality, diversity, and realism.

[0081] In a feasible implementation, step S20 may include steps S21 to S24:

[0082] Step S21, obtaining a training data set according to the vehicle-side environment data and the road-side environment data;

[0083] It should be noted that the training data set is a data set extracted from the vehicle-side environmental data and the road-side environmental data for training the model, which contains various features describing the traffic scene, such as vehicle speed, position, road status, pedestrian dynamics, etc.

[0084] It is understandable that the vehicle-side environmental data includes dynamic information of the vehicle, such as speed, position and direction, as well as the status of traffic signs and signal lights, while the road-side environmental data includes behavioral data of traffic participants and overall traffic flow. By integrating these two types of data, removing redundant data and supplementing missing information, a comprehensive training data set can be formed.

[0085] Step S22, performing data preprocessing on the training data set to obtain a preprocessed data set;

[0086] It can be understood that data preprocessing is the operation of normalizing, cleaning, and optimizing the training data set in order to improve the data quality and make it meet the input requirements of model training. Preprocessing steps usually include data denoising, missing value filling, time series synchronization, normalization, and feature extraction.

[0087] It is understandable that a series of data cleaning and optimization operations are performed on the training data set, including time series alignment to ensure the consistency of vehicle-side and road-side data; filling missing values ​​and removing noise to improve data quality; converting all numerical features to a uniform magnitude through normalization operations; and finally generating a standardized data format suitable for the input of the diffusion transformation model through feature extraction methods. The preprocessed data set is complete, accurate, and highly available.

[0088] Step S23, performing pre-training based on the diffusion transformation model according to the pre-processed data set to obtain an initial world model;

[0089] It should be noted that the initial world model is a model version generated after preliminary training of the diffusion transformation model on the preprocessed dataset, which can capture basic road scene characteristics and dynamic behaviors for further enhanced training.

[0090] It can be understood that the preprocessed data set is input into the diffusion transformation model, and the potential features in the data are gradually extracted through the diffusion mechanism. The model learns the relationship between the dynamics of vehicles and pedestrians in the road scene, the state of traffic signals and road conditions. The model performs self-supervised learning at this stage to extract basic traffic scene features, such as the driving behavior of vehicles in different lanes and the impact of traffic light changes on traffic flow. After training is completed, the initial world model is generated, which can output reasonable road scene videos that conform to the physical laws of the real world.

[0091] Step S24, obtaining a world model according to the preprocessed data set, the target scene and the initial world model.

[0092] It is understandable that the preprocessed data set and the target scene features are input into the initial world model together, and the model's ability to understand and generate dangerous driving situations and complex scenes is enhanced through further training. At the same time, the target scene can strengthen the model's ability to handle dangerous or rare events, and ultimately generate a world model with high accuracy and strong generalization ability.

[0093] In a feasible implementation, step S24 may include steps S241 to S242:

[0094] Step S241, obtaining a typical driving scene according to the preprocessed data set;

[0095] It should be noted that typical driving scenarios are representative traffic scenarios extracted from the preprocessed data set, which are used to strengthen the world model's learning of key features, covering typical behaviors such as normal driving, traffic signal changes, and multi-vehicle interactions, such as dangerous edge scenarios combined with straight driving, left turns at intersections, lane changes, etc.

[0096] Step S242: performing enhanced training based on the initial world model according to the typical driving scenario and the target scenario to obtain a world model.

[0097] It is understandable that by inputting typical driving scenes and target scenes into the initial world model for training and fine-tuning the model parameters, the world model can better understand human driving behavior, especially typical driving behavior, and then output higher-quality scene videos, enhancing the model's generation capabilities in common and complex scenes. Through multiple rounds of enhanced training and optimization of the model's parameters, a world model with high precision, diversity, and long-tail scene generation capabilities can be obtained.

[0098] Step S30, generating a simulation scene according to the target scene and the world model;

[0099] It should be noted that the simulation scenario is a virtual road scenario generated by the world model, which is used to test and verify the intelligent driving algorithm. The simulation scenario includes the road layout, the dynamic behavior of traffic participants, and environmental conditions such as weather and lighting, and can effectively simulate the various complex situations that may be encountered in the real driving environment.

[0100] It is understandable that the world model generates a high-precision virtual scene by analyzing the specific dynamic features in the target scene and combining traffic rules and physical laws. For example, the target scene is described as a complex situation of turning left at a traffic light intersection. The world model will construct the traffic light status, the dynamic position of pedestrians, and the relative position relationship between the vehicle and other vehicles based on this scene. Then, the generated simulation scene is diversified and expanded using scene generalization technology, such as introducing different weather conditions, light changes, or adjustments to traffic density to further enrich the complexity of the scene. The generated simulation scene can include multi-dimensional and multi-perspective road scenarios, which can provide a comprehensive testing environment for intelligent driving algorithms.

[0101] Reference Figure 2 , Figure 2 Schematic diagram of the world model principle provided in Example 1 of the intelligent driving algorithm simulation test method of this application.

[0102] like Figure 2As shown in the figure, firstly, data is collected through the data collection vehicle and the vehicle-road-cloud integration to obtain road environment data including image data and point cloud data. These data reflect the visual information and three-dimensional spatial structure of the traffic scene, and then are input into the world model. The core of the world model includes Diffusion and Transformer modules. The Diffusion module is used to gradually extract the potential features in the data through the denoising diffusion mechanism, while the Transformer module is used to capture the spatiotemporal dependencies and dynamic features in the data. By combining these two modules, the world model can understand the input data and generate road scenes with real physical laws and semantic logic. In addition, the world model can adjust the generated scene behavior and dynamic content according to the input action priors and semantic commands, and finally output multi-perspective road scenes. The multi-perspective road scenes can display the road environment and the dynamic behavior of traffic participants from different observation angles, providing highly realistic virtual scenes for intelligent driving simulation tests.

[0103] Step S40: performing a simulation test of an intelligent driving algorithm according to the simulation scenario.

[0104] It should be noted that the intelligent driving algorithm refers to the core algorithm system used to control the autonomous driving vehicle, and its main functions include sensing the environment, making decisions on vehicle behavior, path planning, and execution control. The intelligent driving algorithm relies on input scene data and sensor information, and calculates the optimal driving behavior of the vehicle through technologies such as deep learning and machine learning. In this embodiment, the intelligent driving algorithm can be an end-to-end intelligent driving algorithm. The end-to-end intelligent driving algorithm refers to an intelligent driving technology paradigm, which is different from the traditional intelligent driving algorithm. In the traditional intelligent driving algorithm, the intelligent driving system is usually subdivided into multiple modules, such as perception, decision-making, planning, control, etc. Each module works independently and designs interfaces between modules to transfer data. This design may lead to an increase in the overall system delay and local optimal problems. At the same time, for rare scenarios, traditional algorithms may not be able to effectively generalize and respond. The end-to-end intelligent driving solution directly maps inputs, such as sensor data, images, etc., to outputs, such as control commands, through artificial intelligence technologies such as deep learning, without the need for complex modular processing procedures in the middle.

[0105] In addition, it should be noted that simulation testing is a method of verifying intelligent driving algorithms in a virtual environment, aiming to evaluate the performance and stability of the algorithms under various complex driving conditions. Simulation testing can simulate long-tail scenarios and dangerous scenarios that are difficult to occur frequently in reality on a large scale and efficiently, helping to discover potential problems in the algorithm and optimize its performance.

[0106] It is understandable that the generated simulation scenario is loaded into the intelligent driving simulation platform and provided to the intelligent driving algorithm as input. The algorithm performs perception, decision-making and control operations based on the environmental data in the simulation scenario, such as vehicle position, pedestrian dynamics, signal light status and road layout. During the test, the performance of the algorithm in complex scenarios is evaluated by monitoring the behavior of the algorithm, such as the braking response time of the vehicle, whether the planning of the steering path is reasonable, and the timeliness of pedestrian avoidance. For dangerous events in specific scenarios, such as emergency braking of the vehicle in front when the traffic light switches, the simulation test can observe whether the algorithm can quickly detect the danger and take appropriate avoidance measures. By running simulation scenarios of different complexity and diversity multiple times, the robustness and reliability of the intelligent driving algorithm under various conditions can be fully evaluated.

[0107] In addition, the results generated by simulation tests can be used to verify, evaluate and optimize intelligent driving algorithms. The performance of the algorithm in complex scenarios can be analyzed, its decision-making accuracy and execution efficiency in different weather conditions, dynamic behaviors and emergencies can be detected, the defects and deficiencies of the algorithm in long-tail scenarios and extreme conditions can be found, and the adjustment and optimization of algorithm parameters can be guided through regression analysis and performance evaluation of test results to improve its responsiveness and response strategies to dangerous scenarios. In addition, the results of simulation tests can also be used for iterative updates of training data sets, and the newly identified scene features and behavior patterns can be reintroduced into the training process through a feedback mechanism to further improve the generalization ability and robustness of the algorithm, and ultimately provide sufficient data support for actual road testing, ensuring the safety and stability of intelligent driving algorithms in real driving environments, while accelerating the mass production deployment and commercialization of intelligent driving systems.

[0108] This embodiment provides a simulation test method for an intelligent driving algorithm, which collects multi-dimensional real-time environmental data through vehicle-side sensors and road-side equipment, combines time synchronization and spatial mapping to perform data fusion, and generates target environmental data; then uses a visual-language model to densely describe the target scene, extracts dynamic elements, environmental conditions, and participant behavior characteristics in the scene, and forms a structured language expression; then, combines these description information with a world model, and the world model extracts potential scene features through diffusion mechanism denoising, and captures the spatiotemporal dependencies in the data based on the Transformer architecture, thereby generating an initial road scene with high fidelity; on this basis, multi-perspective extension technology is used to generate scenes from different observation angles such as the vehicle perspective, pedestrian perspective, and global perspective, and further generalizes environmental conditions, vehicle behaviors, and road scenes to generate simulation scenes covering a variety of weather, lighting, traffic behaviors, and complex dynamic interactions. These technical means effectively solve the technical problems of insufficient test data for intelligent driving algorithms in long-tail scenarios and complex dynamic environments, difficulty in efficiently reproducing real road scenarios, and limited scene diversity and coverage. They achieve the beneficial effect of efficiently and automatically generating high-precision and diversified simulation scenarios, and provide a highly realistic input environment for simulation testing and verification of intelligent driving algorithms.

[0109] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , step S30 of the intelligent driving algorithm simulation test method includes steps S31 to S33:

[0110] Step S31, obtaining dense description information of the target scene according to the visual language model;

[0111] It should be noted that the visual language model is a multimodal model that combines computer vision and natural language processing technology. It is used to extract visual information from images or videos and convert this information into language descriptions. The model can understand and express the visual elements and their relationships in complex scenes, and generate accurate and detailed descriptions. In addition, the dense description information is a comprehensive and detailed language expression of each element in the target scene, including traffic participants, environmental conditions, dynamic behaviors and their relationships in the scene. It aims to capture all the key details of the scene through detailed descriptions for further analysis or generation of simulation scenes.

[0112] It is understandable that through the visual language model, the multi-dimensional data of the target scene is input into the model, including data features of the vehicle side and the road side, such as the trajectory of the vehicle, the dynamic behavior of pedestrians, and the status of traffic signals. The visual language model analyzes the visual information in the target scene through the visual feature extraction module, such as extracting the position, behavior and environmental characteristics of each traffic participant. Through the language generation module, these visual features are converted into semantic dense description information. The dense description information expresses the dynamic situation of the target scene in detail by including all the key elements in the scene and their interrelationships. For example, after the visual language model recognizes that three vehicles are in emergency braking on the highway, the rear of one of the vehicles has obvious collision marks, the vehicle in front slows down and the distance is rapidly shortened, the weather is light rain, and the road is slippery, it generates the description information "On the highway, the light rain weather caused the road to be slippery, a white car suddenly braked in an emergency, and the two vehicles behind it had a chain rear-end collision due to untimely braking. The accident occurred in the middle lane of the three-lane road. The distance in front of the white car was short, and there was a risk of approaching another black SUV. There was also a truck in the right lane that was driving at high speed."

[0113] Step S32, obtaining initial road scene information according to the world model;

[0114] It should be noted that the initial road scene information is the basic road scene data generated by the world model according to the input conditions. The initial road scene information may include long-tail cases, including but not limited to the geometric layout of the road, the initial positions and dynamic behaviors of traffic participants. It is usually presented in the form of dynamic video or data, which can describe the key elements in the scene and their relationships.

[0115] It is understandable that a trained world model can generate a large amount of realistic road videos or dynamic data based on the input target scene features. The input feature information includes road type, such as urban roads or highways, traffic light status, location and behavior of participants, and environmental conditions, such as weather and lighting. The world model generates initial road scene information based on the traffic rules, physical laws, and scene features learned during training.

[0116] For example, the world model can generate a video of a highway where the road is slippery due to light rain, the car is driving in the middle lane, a truck in front suddenly loses control and tilts to the right lane, a car in the right lane suddenly changes lanes to the left lane to avoid the truck, and there is another car driving at high speed in the left lane. This video is the initial road scene information, and the world model can generate a large amount of initial road scene information, which can make the test case more complete.

[0117] Step S33: generating a simulation scene according to the initial road scene and the dense description information.

[0118] It should be noted that the simulation scene is a high-fidelity dynamic road environment generated by combining the initial road scene and dense description information, which is used for testing and training intelligent driving algorithms. The simulation scene not only includes the static structure of the road, but also incorporates the dynamic behavior of traffic participants, changes in environmental conditions and complex interactions, and can truly simulate various conditions in actual driving scenarios.

[0119] It is understandable that the fusion of the initial road scene and the dense description information can generate a complete simulation scene. By integrating the initial road scene and the dense description information, the generated simulation scene has a high degree of authenticity and dynamics, fully reproducing the various details and interactions in the complex traffic environment. This provides rich and diverse scenarios for the simulation test of intelligent driving algorithms, which can more effectively evaluate and improve the performance of the algorithms.

[0120] In a feasible implementation manner, step S33 may include steps S331 to S333:

[0121] Step S331, querying the initial road scene according to the dense description information to obtain a target road scene;

[0122] It should be noted that the target road scenario is a dangerous road environment generated by analyzing dense description information and automatically retrieving it, which accurately corresponds to the characteristics of the long-tail case scenario and can serve as the basis for subsequent simulation scenario generation.

[0123] It is understandable that the dense description information generated by the large visual language model includes all the key elements and dynamic behaviors in the long-tail case scene, such as vehicle location, pedestrian behavior, road conditions, and signal light status. By matching the dense description information with the feature data in the initial road scene, the elements in the initial road scene are searched and aligned to find the scene closest to the dense description. The target road scene is generated through this automated query and matching, which can accurately reflect the complex characteristics of the long-tail case scene. Exemplarily, the dense description information is "During the red light at the intersection, a car suddenly ran a red light, and a pedestrian was crossing on the zebra crossing on the right." By querying in the initial road scene, the system retrieves a target road scene that includes a traffic light intersection, pedestrians, and the dynamics of vehicles running red lights.

[0124] Step S332, obtaining a multi-view road scene according to the target road scene;

[0125] It should be noted that the multi-perspective road scene is a virtual road scene generated based on the target road scene, which can display the traffic environment and dynamic elements from different observation angles. These perspectives usually include the vehicle perspective, pedestrian perspective, other vehicle perspective, and global bird's-eye view perspective. The multi-perspective road scene can include the position, behavior, and relationship of different participants in the scene. For example, the vehicle perspective can simulate the situation observed by the driver while driving, such as the dynamics of the vehicle in front and the status of traffic lights; the global perspective can show the positional relationship of all traffic participants in the scene and the overall traffic flow. The multi-perspective road scene can fully restore the details and dynamic interactions of the road environment, providing richer reference data for the testing and optimization of intelligent driving algorithms.

[0126] It is understandable that the information in the target road scene is expanded to generate content that observes the scene from different perspectives, such as the vehicle perspective, global perspective, pedestrian perspective, etc. Based on the spatial information and dynamic data in the scene, the system reconstructs the road structure and participant behavior from each perspective, so that the characteristics of the scene are fully displayed from multiple dimensions. For example, the vehicle perspective can show the dynamics of the vehicle ahead running a red light, the global perspective can show the overall positional relationship between vehicles, pedestrians and traffic lights, and the pedestrian perspective can show the movement of pedestrians on the zebra crossing and the approaching process of vehicles running a red light.

[0127] Step S333: generalize the scene according to the multi-view road scene to generate a simulation scene.

[0128] It should be noted that scenario generalization is the process of expanding and transforming the original target scenario, generating new scenarios by adjusting the behavior of traffic participants, road conditions or environmental factors to increase the diversity and complexity of the scenarios.

[0129] It is understandable that on the basis of the generated multi-perspective road scene, the scene is expanded and transformed by adjusting the behavioral characteristics of traffic participants, environmental conditions and road parameters to generate multiple different versions of simulation scenes. For example, the number of pedestrians can be increased or decreased, the speed or path of vehicles running red lights can be changed, and special weather conditions such as rainy or foggy days can be introduced to increase the complexity of the scene. The purpose of scene generalization is to cover more potential test conditions and extreme situations by changing the original target scene, thereby improving the adaptability of intelligent driving algorithms in diverse scenarios.

[0130] Reference Figure 4 , Figure 4 A schematic diagram of the training and use of the world model provided in Example 2 of the intelligent driving algorithm simulation test method of this application.

[0131] like Figure 4As shown in the figure, in the training phase of the world model, data is collected by data collection vehicles and vehicle-road-cloud integration. The collected data includes image data and point cloud data. Image data mainly comes from visual information captured by cameras, while point cloud data comes from devices such as lidar to reflect three-dimensional spatial structures. These data are used to build the world model after training. In the use phase of the world model, test scenarios and long-tail cases are first input, and their features are described and provided to the trained world model. The world model generates basic road scenes based on the input, and further expands the scenes through generalization technology, including adjusting weather, traffic participant behavior, etc., and finally generates multi-perspective road scenes. Multi-perspective road scenes can display road environments and dynamic elements from different observation angles, providing support for simulation testing and verification of intelligent driving algorithms.

[0132] In a feasible implementation manner, step S333 may include steps S3331 to S3335:

[0133] Step S3331, obtaining scene weather, current vehicle behavior, pedestrian information, and target vehicle behavior according to the multi-view road scene;

[0134] It should be noted that scene weather refers to the environmental conditions in the simulation scene, including lighting, such as daytime, nighttime, precipitation, such as sunny, rainy, snowy, visibility, such as foggy or cloudy, etc., which are important factors affecting the driving environment. In addition, the current vehicle behavior refers to the dynamic behavior of the vehicle in the scene, including acceleration, deceleration, steering, lane change, parking, etc. It reflects the decision of the vehicle in the current scene when responding to environmental changes or interacting with other participants. In addition. Pedestrian information refers to the characteristics related to pedestrians in the scene, including the location, direction of movement, speed, behavioral intention, such as whether to cross the zebra crossing, etc. In addition, the target vehicle behavior refers to other vehicles in the scene, which can be the dynamic behavior of key vehicles that have an impact on the vehicle, such as going straight, changing lanes, emergency braking, etc.

[0135] It can be understood that key elements related to the environment and the behavior of participants are extracted from the multi-view road scene. The scene weather is identified by analyzing factors such as light and humidity in the scene, such as sunny days, light rain or night. The current vehicle behavior is extracted from the trajectory and dynamic data of the vehicle, such as turning left, accelerating or emergency braking. Pedestrian information is based on the position, speed and direction of pedestrians in the scene, such as pedestrians moving near zebra crossings. The target vehicle behavior is determined by analyzing the dynamic characteristics of other vehicles in the scene, such as the sudden braking of the vehicle in front or the lane change of the vehicle on the right.

[0136] Step S3332, generalizing the environment according to the scene weather to obtain an environment generalization set;

[0137] It should be noted that the environmental generalization set is a diverse set of environments generated by changing scene weather conditions, such as weather type, lighting conditions, etc., and is used to test the robustness of intelligent driving algorithms under different environmental conditions.

[0138] It is understandable that the environment generalization set is generated by adjusting the weather conditions according to the scene weather in the initial scene. First, the weather characteristics of the current scene, such as light intensity, precipitation type, and visibility, are analyzed to identify the key factors affecting driving behavior. Then, a variety of scenes are generated by changing these factors, such as changing sunny days to rainy days, changing daytime to night, and adding slippery road characteristics to foggy days. Exemplarily, in a scene at a city intersection, the scene weather is sunny during the day. Environmental generalization can generate a variety of new scenes, including rain at night, heavy fog during the day, and light snow and slippery road conditions at night.

[0139] Step S3333, generalizing the vehicle behavior according to the current vehicle behavior to obtain a generalized vehicle behavior set;

[0140] It should be noted that the vehicle behavior generalization set is a set of scenarios generated by changing the behavior of the vehicle in the scenario, such as adjusting the vehicle's speed, lane change timing, or braking intensity, to simulate different driving dynamics.

[0141] It can be understood that according to the current vehicle behavior, a generalized set of vehicle behaviors is generated by adjusting the dynamic features of the ego vehicle. First, the behavioral features of the ego vehicle in the scene are extracted, such as acceleration, deceleration, or lane change. Then, by changing the parameters of these behavioral features, such as speed, braking force, or lane change timing, multiple possible driving scenarios are generated. This generalization is used to test the performance of the ego vehicle in different dynamic behaviors and the robustness of the algorithm.

[0142] Step S3334, generalizing the road scene according to the pedestrian information and the target vehicle behavior to obtain a road scene generalization set;

[0143] It should be noted that the generalized set of road scenes is a set of scenes generated by changing pedestrian information and target vehicle behavior, such as adjusting the position and movement direction of pedestrians and the speed and path of target vehicles, to cover more complex traffic interaction scenarios.

[0144] It should be noted that, according to the pedestrian information and the target vehicle behavior, the generalized set of road scenes is generated by adjusting the dynamic characteristics of pedestrians and the behavioral characteristics of target vehicles. The position, speed, direction and whether the pedestrians cross the road are analyzed, and the behavioral characteristics of the target vehicles, such as changing lanes, braking or accelerating, are extracted. Then, different road interaction scenes are generated by adjusting the parameters of these features. For example, the movement path and speed of pedestrians can be changed, or the interaction behavior between the target vehicle and pedestrians can be adjusted. More pedestrians and vehicles can also be generated into the road scene, such as whether to avoid pedestrians or whether to drive according to traffic rules.

[0145] Step S3335, performing scene generalization according to the environment generalization set, the vehicle behavior generalization set and the road scene generalization set to generate a simulation scene.

[0146] It can be understood that by combining all possible combinations of the generalized set of environment, the generalized set of vehicle behavior, and the generalized set of road scenes, a variety of simulation scenarios can be generated. The simulation scenarios cover different weather conditions, vehicle behaviors, and pedestrian dynamics, which can provide rich input data for the testing of intelligent driving algorithms and ensure the accuracy of the tests.

[0147] Reference Figure 5 , Figure 5 A schematic diagram of simulation scenario generation provided in Example 2 of the intelligent driving algorithm simulation test method of this application.

[0148] like Figure 5 As shown in the figure, firstly, a road scene is input, which includes specific driving behaviors such as straight driving, left turning at intersections, and lane changing. The road scene is densely described through the visual-language model (VLM), and the key elements and dynamic information in the scene are extracted, such as the behavior of vehicles, road structure, and the status of traffic participants. These description information is then passed to the world model. The core of the world model is composed of the Diffusion module and the Transformer module. The Diffusion module gradually extracts and restores the potential features in the data through the diffusion mechanism, while the Transformer module is used to capture the temporal dynamics and spatial characteristics of the scene, and combines action priors and semantic commands to logically constrain the scene and generate behaviors. Finally, a multi-view road scene is generated through scene generalization. The multi-view road scene can display complex road environments and dynamic behaviors from different observation angles, providing highly realistic simulation results.

[0149] This embodiment provides a simulation test method for an intelligent driving algorithm. A visual-language model densely describes a road scene, extracts key elements such as dynamic behaviors, environmental conditions, and positions of traffic participants in the scene, and combines these semantic information with a trained world model. The world model gradually denoises and restores potential features of the scene through a diffusion mechanism, and captures temporal dynamics and spatial dependencies in the data based on a Transformer structure to generate initial road scene information. On this basis, scene contents from different observation angles such as the vehicle perspective, pedestrian perspective, and global perspective are generated through multi-perspective expansion technology to fully display the mutual relationships and dynamic behaviors of traffic participants. Finally, environmental conditions, vehicle behaviors, and pedestrian dynamics are adjusted and combined through scene generalization technology to generate diversified simulation scenes covering different weather, road conditions, and complex interactive relationships, thereby solving the technical problems of lack of high-quality test data, insufficient scene coverage, and weak ability to cope with extreme situations in intelligent driving algorithms in long-tail scenarios and complex dynamic environments, achieving efficient generation of high-fidelity, multi-perspective, and diversified simulated road scenes, providing highly realistic input conditions for comprehensive testing and verification of intelligent driving algorithms, and significantly improving the adaptability and robustness of the algorithms in complex scenarios.

[0150] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the intelligent driving algorithm simulation test method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0151] This application also provides an intelligent driving algorithm simulation test device, please refer to Figure 6 , the intelligent driving algorithm simulation test device comprises:

[0152] An acquisition module 10 is used to acquire vehicle-side environmental data and road-side environmental data, and obtain a target scene according to the vehicle-side environmental data and the road-side environmental data;

[0153] A training module 20, configured to perform training based on a diffusion transformation model according to the vehicle-side environment data, the road-side environment data and the target scene to obtain a world model;

[0154] A generating module 30, configured to generate a simulation scene according to the target scene and the world model;

[0155] The testing module 40 is used to perform a simulation test of the intelligent driving algorithm according to the simulation scenario.

[0156] The intelligent driving algorithm simulation test device provided by the present application adopts the intelligent driving algorithm simulation test method in the above embodiment, which can solve the technical problem of insufficient fidelity of simulation scenes in end-to-end intelligent driving simulation tests. Compared with the prior art, the beneficial effects of the intelligent driving algorithm simulation test device provided by the present application are the same as the beneficial effects of the intelligent driving algorithm simulation test method provided by the above embodiment, and the other technical features in the intelligent driving algorithm simulation test device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0157] In one embodiment, the generation module 30 is further used to obtain dense description information of the target scene according to the visual language model; obtain initial road scene information according to the world model; and generate a simulation scene according to the initial road scene and the dense description information.

[0158] In one embodiment, the generation module 30 is also used to query the initial road scene according to the dense description information to obtain a target road scene; obtain a multi-perspective road scene according to the target road scene; and generalize the scene according to the multi-perspective road scene to generate a simulation scene.

[0159] In one embodiment, the generation module 30 is further used to obtain scene weather, current vehicle behavior, pedestrian information and target vehicle behavior based on the multi-perspective road scene; generalize the environment based on the scene weather to obtain an environment generalization set; generalize the vehicle behavior based on the current vehicle behavior to obtain a vehicle behavior generalization set; generalize the road scene based on the pedestrian information and the target vehicle behavior to obtain a road scene generalization set; generalize the scene based on the environment generalization set, the vehicle behavior generalization set and the road scene generalization set to generate a simulation scene.

[0160] In one embodiment, the training module 20 is further used to obtain a training data set based on the vehicle-side environmental data and the road-side environmental data; perform data preprocessing on the training data set to obtain a preprocessed data set; perform pre-training based on a diffusion transformation model based on the preprocessed data set to obtain an initial world model; and obtain a world model based on the preprocessed data set, the target scene and the initial world model.

[0161] In one embodiment, the training module 20 is further used to obtain a typical driving scene according to the preprocessed data set; and to perform enhanced training based on the initial world model according to the typical driving scene and the target scene to obtain a world model.

[0162] In one embodiment, the acquisition module 10 is also used to acquire vehicle-side environmental data and road-side environmental data, and fuse the vehicle-side environmental data with the road-side environmental data to obtain target environmental data; analyze the target environmental data to obtain dangerous scenarios and abnormal driving behaviors; and obtain target scenarios based on the dangerous scenarios and the abnormal driving behaviors.

[0163] The present application provides an intelligent driving algorithm simulation test device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the intelligent driving algorithm simulation test method in the above-mentioned embodiment one.

[0164] Reference below Figure 7 , which shows a schematic diagram of the structure of an intelligent driving algorithm simulation test device suitable for implementing the embodiment of the present application. The intelligent driving algorithm simulation test device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The intelligent driving algorithm simulation test equipment shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0165] like Figure 7As shown, the intelligent driving algorithm simulation test equipment may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the intelligent driving algorithm simulation test equipment are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other via a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the intelligent driving algorithm simulation test device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows an intelligent driving algorithm simulation test device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or provided alternatively.

[0166] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0167] The intelligent driving algorithm simulation test equipment provided by the present application adopts the intelligent driving algorithm simulation test method in the above embodiment, which can solve the technical problem of insufficient fidelity of simulation scenes in end-to-end intelligent driving simulation tests. Compared with the prior art, the beneficial effects of the intelligent driving algorithm simulation test equipment provided by the present application are the same as the beneficial effects of the intelligent driving algorithm simulation test method provided by the above embodiment, and the other technical features in the intelligent driving algorithm simulation test equipment are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0168] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0169] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0170] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the intelligent driving algorithm simulation test method in the above-mentioned embodiment.

[0171] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0172] The above-mentioned computer-readable storage medium may be included in the intelligent driving algorithm simulation test device; or it may exist independently without being assembled into the intelligent driving algorithm simulation test device.

[0173] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the intelligent driving algorithm simulation test equipment, the intelligent driving algorithm simulation test equipment: obtains vehicle-side environmental data and road-side environmental data, and obtains a target scene based on the vehicle-side environmental data and the road-side environmental data; performs training based on the diffusion transformation model according to the vehicle-side environmental data, the road-side environmental data and the target scene to obtain a world model; generates a simulation scene according to the target scene and the world model; and performs intelligent driving algorithm simulation test according to the simulation scene.

[0174] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0175] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0176] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0177] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned intelligent driving algorithm simulation test method, and can solve the technical problem of insufficient fidelity of simulation scenes in end-to-end intelligent driving simulation tests. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the intelligent driving algorithm simulation test method provided in the above-mentioned embodiment, and will not be repeated here.

[0178] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned intelligent driving algorithm simulation test method when executed by a processor.

[0179] The computer program product provided by this application can solve the technical problem of insufficient fidelity of simulation scenes in end-to-end intelligent driving simulation tests. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as those of the intelligent driving algorithm simulation test method provided by the above embodiment, and will not be elaborated here.

[0180] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A simulation test method for an intelligent driving algorithm, characterized in that: The method comprises: Acquire vehicle-side environmental data and road-side environmental data, and obtain a target scene according to the vehicle-side environmental data and the road-side environmental data; According to the vehicle-side environment data, the road-side environment data and the target scene, training is performed based on a diffusion transformation model to obtain a world model; Generate a simulation scene according to the target scene and the world model; Perform intelligent driving algorithm simulation test according to the simulation scenario.

2. The method according to claim 1, characterized in that The step of generating a simulation scene according to the target scene and the world model comprises: Obtaining dense description information of the target scene according to the visual language model; Obtaining initial road scene information according to the world model; A simulation scene is generated according to the initial road scene and the dense description information.

3. The method according to claim 2, characterized in that The step of generating a simulation scene according to the initial road scene and the dense description information comprises: Querying the initial road scene according to the dense description information to obtain a target road scene; According to the target road scene, a multi-view road scene is obtained; Scene generalization is performed based on the multi-view road scene to generate a simulation scene.

4. The method according to claim 3, characterized in that The step of generalizing the scene according to the multi-view road scene to generate a simulation scene comprises: According to the multi-view road scene, scene weather, current vehicle behavior, pedestrian information, and target vehicle behavior are obtained; Performing environment generalization according to the scene weather to obtain an environment generalization set; Generalizing the vehicle behavior according to the current vehicle behavior to obtain a generalized vehicle behavior set; Generalize the road scene according to the pedestrian information and the target vehicle behavior to obtain a road scene generalization set; The scene generalization is performed according to the environment generalization set, the vehicle behavior generalization set and the road scene generalization set to generate a simulation scene.

5. The method according to claim 1, characterized in that The step of obtaining a world model by training based on the diffusion transformation model according to the vehicle-side environment data, the road-side environment data and the target scene comprises: Obtaining a training data set according to the vehicle-side environmental data and the road-side environmental data; Performing data preprocessing on the training data set to obtain a preprocessed data set; According to the preprocessed data set, pre-training is performed based on a diffusion transformation model to obtain an initial world model; A world model is obtained according to the preprocessed data set, the target scene and the initial world model.

6. The method according to claim 5, characterized in that The step of obtaining a world model according to the preprocessed data set, the target scene and the initial world model comprises: According to the preprocessed data set, a typical driving scene is obtained; According to the typical driving scenario and the target scenario, enhanced training is performed based on the initial world model to obtain a world model.

7. The method according to claim 1, characterized in that The step of acquiring vehicle-side environmental data and road-side environmental data, and obtaining a target scene according to the vehicle-side environmental data and the road-side environmental data comprises: Acquire vehicle-side environmental data and road-side environmental data, and fuse the vehicle-side environmental data with the road-side environmental data to obtain target environmental data; Analyze the target environment data to obtain dangerous scenarios and abnormal driving behaviors; A target scene is obtained according to the dangerous scene and the abnormal driving behavior.

8. An intelligent driving algorithm simulation test device, characterized in that: The device comprises: An acquisition module, used to acquire vehicle-side environmental data and road-side environmental data, and obtain a target scene according to the vehicle-side environmental data and the road-side environmental data; A training module, configured to perform training based on a diffusion transformation model according to the vehicle-side environment data, the road-side environment data and the target scene to obtain a world model; A generation module, used to generate a simulation scene according to the target scene and the world model; The test module is used to perform intelligent driving algorithm simulation test according to the simulation scenario.

9. An intelligent driving algorithm simulation test device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the intelligent driving algorithm simulation test method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the intelligent driving algorithm simulation test method as described in any one of claims 1 to 7 are implemented.

Images