Integrated training test method for automatic driving algorithm

Abstract

The invention discloses an integrated training test method for an automatic driving algorithm. Closed-loop iteration of training data targeted generation, algorithm integrated accelerated training and algorithm defect test diagnosis can be realized under a unified architecture. Around efficient iteration and performance improvement of an automatic driving algorithm, the method comprises the following specific steps: constructing a training data standardized link, providing a training data targeted generation and generalization method, and realizing efficient supply and distribution balance of data; an algorithm integrated training platform supporting modularization, end-to-end and other technical routes is established, and algorithm cooperative training acceleration is promoted; an automatic driving algorithm closed-loop defect test and diagnosis mechanism based on a virtual perception injection real vehicle test method is designed, targeted feedback and optimization basis are provided for data targeted generation, a training and test integrated data closed-loop method is finally formed, the algorithm iteration upgrading and real vehicle landing process can be accelerated, and the training and test efficiency is improved. And systematic support is provided for constructing a safe, intelligent and efficient automatic driving system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to an integrated training and testing method for automatic driving algorithms. BACKGROUND

[0002] With the rapid iteration of automatic driving technology, current domestic and foreign manufacturers have shifted from focusing on the first half of technology accumulation to the second half of algorithm commercial landing application. However, there are still many problems with algorithms in complex open scenarios. The current mainstream approach is to continuously collect and label data, expand the training set, and verify and test the algorithm in combination with diversified scenarios. However, this linear "collection-training-testing" process has many shortcomings: first, the training data structure is not unified, making it difficult to support efficient use across scenarios and algorithms; second, high-quality training data is sparse, data expansion lacks targeting, is inefficient and has a long cycle; third, the training and testing stages are disconnected, and cannot form a closed-loop feedback for data generation and algorithm improvement.

[0003] Therefore, it is urgent to establish a closed-loop method that can connect data, training and testing under a unified architecture, through targeted generation and generalization of training data, efficient integration and acceleration of training of algorithms, and closed-loop defect diagnosis based on virtual perception injection of real vehicle testing, to provide systematic support for automatic driving algorithm iteration and upgrade and real vehicle landing. SUMMARY

[0004] The present application provides an integrated training and testing method for automatic driving algorithms to solve the problem of long iteration and verification period of automatic driving algorithms and unsatisfactory improvement of automatic driving algorithms due to the lack of feedback mechanism in the training and testing stages and insufficient coverage of long-tail scenario data in the prior art.

[0005] TECHNICAL SCHEME An integrated training and testing method for automatic driving algorithms, comprising the following steps: S1 data standardization; Data standardization is performed on open source data sets from different sources to unify the input-output structure of training data; then a multi-dimensional scene classification system is established for the labeled road traffic data; S2 data targeted generation and category reconstruction; First, the defect scenarios of data targeted generation are determined; for the determined defect scenarios, a scene generation model is used for data targeted generation to improve the amount of training samples in the defect scenarios; a scene refinement reconstruction iterative method is used to refine the granularity of scene categories, enhance data targeted reinforcement and algorithm training effect; S3 integrated acceleration training; After the scene data is generated and completed, the algorithm is accelerated; S4 closed-loop test and defect diagnosis Based on the closed-loop test and defect diagnosis method, the dynamic closed-loop test of the automatic driving algorithm in different scenes and difficulties is carried out, the algorithm defects are diagnosed and used for algorithm iterative training; S5 loop execution step; Loop execution steps S1-S4 realize the continuous optimization of algorithm performance.

[0006] Specifically as follows: S1 data standardization: S11 data standardization processing: First, data standardization processing is carried out for different sources of open source data sets, the input-output structure of training data is unified, and the compatibility of the training test integrated closed-loop platform provided by the application to different source training data is ensured.

[0007] The input data is composed of three parts of vehicle perception data, vehicle state information and map and environment information: 1. Perception data: (1) camera image sequence: , wherein , respectively represent the image height, width and channel number; (2) laser radar point cloud: , wherein represents three-dimensional coordinates, represents the reflection intensity; 2. Vehicle state information: Stored in the format of dynamic state vector: , wherein is the longitudinal velocity, is the acceleration, is the steering wheel angle, is the yaw rate, is the heading angle; 3. Map and environment information (1) high-precision map topological data: , wherein L is a lane set, is an intersection set; (2) dynamic traffic signal: red light state , wherein 0 represents red light, 1 represents yellow light, and 2 represents green light; Based on the above, the formal expression of the training data in different scenes can be formed: The output data is the planning and control command of the vehicle in the future time domain, specifically including: 1. Predicted trajectory wherein represents the predicted position coordinates, represents the heading angle, is the future time step, and the maximum prediction time is . 2. Control commands vehicle chassis longitudinal control throttle opening , brake intensity , lateral control front wheel steering angle . 3. Risk assessment indicators (optional output) collision risk score: traffic rule compliance: Therefore, the mapping relationship of the model as a whole is: wherein, , The original data is processed into training data in a unified form, meeting the training requirements of different algorithms such as segmentation and end-to-end. S12 Scene classification grading A multi-dimensional scene classification system is established for the labeled road traffic data to meet the needs of dynamic adjustment of test intensity, targeted generation of defect scene categories and generalization in subsequent links.

[0008] Among them, the present application divides the traffic scene into three elements: environmental attributes, road topology and interaction mode, and considers that the three elements are strongly related to the perception and planning control ability of the algorithm, so the scene is classified according to the above elements, and the target scene data is positioned with fine granularity.

[0009] The grading reference indicators specifically include: (1) Environmental attribute dimension: day / night, sunny / rain / snow / fog, low visibility, sudden change of light (such as tunnel entrance and exit), etc. (2) Road topology dimension: expressway, urban trunk road, branch road, roundabout, intersection, etc. (3) Interaction mode dimension: single-lane following, lane changing, merging, unprotected left turn, pedestrian crossing, non-motor vehicle mixed traffic, etc.

[0010] On the basis of classification, a comprehensive difficulty index is defined to measure the challenge of the scene to the automatic driving algorithm, as follows: wherein, To assess the difficulty (number of targets, occlusion ratio, degree of influence of lighting / weather); Topological difficulty (number of lanes, number of conflict points, road curvature, etc.); The difficulty of the interaction (number of interaction targets, proportion of conflict zones, and intensity of interaction). This is a weighting parameter that can be set according to actual needs.

[0011] Based on the comprehensive difficulty index The numerical range is used to divide the overall difficulty level of the scene, ensuring the comparability of difficulty level classifications under multi-source data conditions. The threshold is determined manually based on experience, according to the statistical distribution characteristics of different data sources. ,but: Low difficulty (L): For example, a single-lane straight road with good lighting; Medium difficulty (M): For example, multiple lanes merging, and significant changes in lighting; High Difficulty (H): Such as complex intersections, multi-purpose traffic, and severe weather.

[0012] S2 Data Targeting Generation and Category Reconstruction S21 Data Targeting Generation This invention provides a process for targeted generation of training data for autonomous driving algorithms: First, identify the defect scenarios for targeted data generation, and then generate subsequent data within these target scenario categories.

[0013] A "defect scenario" is defined as a scenario in which the key performance indicators of the algorithm in a certain scenario fail to reach the preset threshold during the testing of autonomous driving algorithms, thus triggering emergency obstacle avoidance or being taken over by the driver.

[0014] The key performance indicators include, but are not limited to: (1) Perception layer metrics: Average Precision AP (Average Precision AP), Multiple Object Tracking Accuracy MOTA (Multiple Object Tracking Accuracy MOTA), False Detection / False Detection Rate; (2) Prediction layer indicators: Average Displacement Error (ADE) and Final Displacement Error (FDE). (3) Planning and control layer indicators: collision rate, path deviation rate, braking / steering stability.

[0015] Define scene performance score: , in For the first The normalized score of each indicator, As weight.

[0016] If the algorithm is in the category Difficulty Scene Overall score satisfy: If so, the scenario category is determined to be a defective scenario; if If it is, then it is determined to be a boundary scenario and further repeated testing is required; if If so, the scenario is determined to be a non-defect scenario. Where: difficulty , The difficulty level is defined in S12; and The preset performance threshold is determined by statistical analysis of the test performance of different types of human drivers and manual calibration in combination with the actual difficulty of typical scenarios, so as to ensure the rationality of defect scenario identification.

[0017] Among them, the scene performance score for defect scene discrimination is obtained from the previous round of S4 closed-loop testing and defect diagnosis. In the initial round, the default scene set is used directly to enter the training phase.

[0018] Next, for the identified defect scenarios, this invention uses a scenario generation model based on the previous round of defect scenario diagnosis results to generate targeted data based on the following scenario feature dimensions, thereby increasing the training sample size under defect scenarios. This invention maps the three main components of a traffic scenario—road topology, interaction patterns, and environmental attributes—to three dimensions: road topology, traffic events, and environmental perception. It also adds behavioral perturbations to enhance the generalization of scenario data and enable targeted data generation. (1) Road topology dimension: road topology The spatial environment in which traffic scenarios occur serves as the basic spatial constraint between the measured subject and the traffic flow interaction object, including road boundaries, lane division and intersection, and traffic flow direction.

[0019] (2) Traffic incident dimension: road topology in defective scenarios With background traffic flow Under these conditions, dynamic events that interact with the algorithm under test are automatically constructed through a traffic simulation platform.

[0020] (3) Behavioral disturbance dimension: For the defective scenario, disturbance modeling of traffic participants is introduced. For example, randomly setting background vehicle behaviors such as lane changing, sudden braking, and acceleration, or increasing traffic signal delays and anomalies, to construct more complex interactive data.

[0021] (4) Perception dimension: Add natural environment perception elements to the interactive scene. Based on the vehicle's sensor layout and spatial location, visual perception data is generated using the 3D Gaussian Scene Generation (3DGS) method, and the realism of defective scenes is enhanced by the Generative Adversarial Network (GAN) method.

[0022] A set of sensing data The generation process can be described as follows: In the formula This refers to the method for generating realistic scenes using 3DGS generation combined with GAN enhancement. These correspond to road topology, background traffic flow, random behavioral disturbances of participants, and natural environmental elements, respectively. Information is generated by the vehicle's own sensors.

[0023] The number of new samples generated through the above process is defined as: in The generated scene-aware data Compared with the original defect scene perception data Sample size This is the amplification factor, which is dynamically set according to the severity of the defect.

[0024] Finally, the generated defect scenario extended samples The data is merged with the original scene data to form the training set input for the current round, thereby achieving targeted reinforcement of the training data for the defective scenes of the tested algorithm. The targeted generation results are then transformed into new data samples and fed back into the training process. S22 Scene Refinement and Reconstruction While performing targeted data generation, this invention proposes a scene refinement and reconstruction iterative method to refine the granularity of scene categories and enhance the effects of targeted data reinforcement and algorithm training.

[0025] This invention defines "scene refinement and reconstruction" as follows: During multiple rounds of training, when the performance index improvement of the algorithm is insufficient to exceed the preset incremental threshold after several targeted generation and training in a certain type of defect scene, the system automatically triggers a finer-grained splitting and reconstruction of the scene to obtain new scene categories and expand the data.

[0026] First, clarify the triggering conditions for scene refinement and reconstruction: Let a certain scene be related to a difficulty category. The performance metrics after the t-th round of training are The previous performance indicators were If the following conditions are met: and continuous None of the rounds exceeded the threshold This triggers the scene iteration process. This is the performance increment threshold. This represents the minimum number of consecutive detection rounds.

[0027] Next, scene feature extraction and clustering are performed. For the scene category that triggers the iteration, scene feature vectors are extracted: in For the number of target elements in the scene, These are the lighting parameters. Due to weather conditions, For road topology indicators, The interaction strength.

[0028] The feature vectors are input into a clustering algorithm based on a Gaussian mixture model (GMM), assuming the sample distribution is as follows. A weighted combination of Gaussian distributions: Hyperparameters are obtained through iterative estimation. , respectively corresponding to the first The weights, mean vectors, and covariance matrices of each Gaussian component are used to subdivide the original scene category into multiple subclasses. .

[0029] After obtaining a new subclass Then, the S21 data targeting generation step is performed to obtain an expanded sample set. It is then integrated with the existing training set to form a new round of iterative training data.

[0030] After scene iteration processing, the original defect scene is refined into more comprehensive sub-scenes, enabling the algorithm to perform reinforcement learning for different fine-grained features.

[0031] S3 Integrated Accelerated Training After the scene data is generated and completed, the algorithm is trained at an accelerated pace. "Accelerated training" refers to dynamically adjusting the distribution of training data based on the test evaluation results after each round of training, in order to achieve targeted enhancement of the algorithm for defective scenes and improve the overall convergence speed.

[0032] Specifically, the following steps are included: S31 Training Data Management First, based on the scenario data described in S1 / S2, the user prepares the training data through the following process: (1) Accessing standardized scenario data: Read training data from the standardized database formed in step S1, integrate datasets from different sources, and uniformly format and store the data from different sources to ensure a unified input structure; (2) Perform dynamic data matching: Based on the feedback results of the previous training evaluation and the distribution of scene categories in the data, automatically match data of different difficulties and categories to obtain the dataset used in this training.

[0033] S32 Algorithm Training Parameter Optimization After each round of training data preparation and allocation is completed, training parameters are set according to the following process, and model training is performed: (1) Access the algorithm model under test: Access the autonomous driving algorithm under test through the standard interface and initialize the model parameters; (2) Training process execution: Call the training set after S31 allocation for batch training and model parameter iteration; (3) Dynamic parameter adjustment: Based on the feedback results of the previous round of S33 evaluation, the learning rate and loss function weights are automatically adjusted to enhance the learning effect of defective scenarios and shorten the model convergence time.

[0034] S33 Inter-round Test Evaluation During training, intermediate rounds of open-loop testing are performed to verify whether the algorithm training has converged and to conduct quantitative evaluations of targeted access to training data. The steps include: (1) Automatically generate test sets: During the training process, the standardized scene data obtained from the S31 training data management is called, and test scene sets between rounds are generated according to the difficulty level. (2) Indicator calculation: Quantitatively evaluate the performance of the model in the perception, prediction, planning and control stages, and calculate key indicators including perception accuracy, prediction error, collision rate and control stability. (3) Defect identification: If the overall score of a certain scenario is lower than the preset threshold, it is marked as a defect scenario, and the test results are returned to S31.

[0035] S34 Results Feedback and Data Feedback After each round of data matching, parameter adjustment, model training, and inter-round testing is completed, the test results generated by S33 are fed back to S31 to form a new round of training data input for a new round of integrated accelerated training, realizing a training-evaluation-feedback cycle.

[0036] The entire process repeats until the model meets the preset performance threshold in the S33 test, at which point the integrated accelerated training process ends and enters the S4 closed-loop testing and defect diagnosis process.

[0037] S4 Closed-Loop Testing and Defect Diagnosis: After the algorithm integration and accelerated training are completed, based on the closed-loop testing and defect diagnosis method described in this invention, dynamic closed-loop testing is performed on the autonomous driving algorithm under different scenarios and difficulties to diagnose algorithm defects and use them for algorithm iterative training.

[0038] Unlike the open-loop testing and evaluation in S33 used to determine training convergence and accelerate training, the method described in this section is a comprehensive closed-loop testing and defect diagnosis for autonomous driving algorithms, used for quantitative evaluation of autonomous driving algorithm capabilities, defect exposure, and algorithm improvement.

[0039] S41 Virtual Perception Injection Real Vehicle Testing Method This invention proposes a virtual perception injection method for real-vehicle testing, used to reproduce complex traffic scenarios under real-vehicle conditions and to specifically verify autonomous driving algorithms. This method combines virtually generated perception information with the real vehicle state, enabling the tested autonomous driving system to accept virtual environment input within the real vehicle control framework. This allows for coverage of more long-tail scenarios and extreme situations while ensuring safety.

[0040] 1. Method Definition Virtual perception injection into real vehicle testing refers to directly injecting perception information (including target detection results, traffic flow information, environmental disturbances, etc.) generated by a virtual simulation platform into the vehicle's domain controller in a real vehicle operating environment. This makes the sensor input received by the autonomous driving algorithm a combination of "real sensing data + virtual generated data", thereby completing closed-loop decision-making and execution under real vehicle conditions.

[0041] 2. Input Construction Virtual traffic flow injection: Based on the road topology and traffic elements of the labeled scene, interactive virtual traffic participants (such as oncoming vehicles, pedestrians, and bicycles) are generated in the simulation platform, and their movement trajectories and interactive behaviors are constructed. Virtual sensor perception injection: Construct corresponding virtual perception data through sensor modeling and image / point cloud generation methods, such as camera images in low visibility conditions and radar point cloud noise in rain and snow weather; Environmental disturbance injection: Introduce sudden disturbances into the scenario, such as sudden braking of the vehicle in front, sudden obstacles, or abnormal traffic signals, to increase the intensity of the test.

[0042] 3. Actual vehicle operation and control In a closed testing environment, a real vehicle equipped with a domain controller receives hybrid perception input (real + virtual) and executes subsequent prediction, planning, and control decisions based on this input. In this way, the vehicle can perform steering, braking, and acceleration under realistic dynamic conditions, resulting in test results that more closely resemble actual road performance.

[0043] 4. Enhanced human-computer interaction To enhance testing diversity, intelligent targets and remote drivers can be introduced into the testing range: Intelligent targets, as interactive physical traffic participants, interact with the virtual traffic flow and affect the tested vehicles. The driving simulator supports remote human driver control of the target vehicle, realizing dynamic three-way interaction between the intelligent target object, the algorithm under test, and the background traffic flow in the "human on the loop", enhancing the complexity and randomness of the test scenario.

[0044] 5. Evaluation and Feedback During testing, the platform collects key performance indicators of the tested model under different scenarios, including detection accuracy, trajectory prediction error, collision rate, and longitudinal and lateral control stability. If the indicators are lower than the set threshold, the scenario is determined to be a defective scenario, and its features are fed back to S21 for subsequent scenario iteration and target data generation.

[0045] S42 Closed-Loop Dynamic Test The closed-loop dynamic testing method proposed in this invention is a process of progressively verifying and providing feedback on autonomous driving algorithms by combining virtual perception injection into real-vehicle testing with scenario difficulty grading and dynamic adjustment mechanisms. This method not only verifies the model's performance in different types of scenarios, but also gradually approaches the algorithm's boundary by dynamically adjusting the testing difficulty, forming an efficient training-testing closed loop.

[0046] 1. Initial test set generation Based on the data standardization and scenario difficulty classification results, the current round of test scenario set is defined as follows: In the formula This indicates the difficulty level of the i-th type of scenario. The test subset below.

[0047] 2. Performance Index Calculation In each type of scenario Run the algorithm under test and calculate its performance score in different scenarios according to the definition in S21: In the formula For the first Key performance indicators, including sensing accuracy: Prediction error Collision rate lateral deviation Longitudinal acceleration stability . Assigning weights to each indicator. .

[0048] 3. Dynamic difficulty adjustment In a certain type of scenario Overall rating satisfy: This increases the testing difficulty for that scenario category. (e.g., from low difficulty L to medium difficulty M); like If the scenario is deemed defective, the defect diagnosis process will begin. like If the original difficulty is maintained, the test should be repeated to verify stability.

[0049] in The base value is consistent with S21, and dynamically adjusted based on manual experience.

[0050] 4. Closed-loop iterative execution After each round of testing, a set of test results for different categories of scenarios is obtained: like If the training termination condition is triggered, it indicates that the algorithm has reached the expected performance. Otherwise, Defect scenario set The data flows back to S21, triggering scenario iteration and targeted data generation, and entering the next training-testing cycle.

[0051] S43 Defect Location Diagnosis 1. Defect scenario localization Based on the defect scene localization method in the S21 data targeted generation process described in this invention, the defect scene of the current training round of the algorithm under test is located. For the set of scenarios that were judged to be defective Further analysis of the indicators reveals: perception indicators, prediction error, collision rate, and control stability, which correspond to defects such as insufficient perception, trajectory prediction, planning safety, and execution control, respectively.

[0052] 2. Result reflux For scene set In the defective scenario described in S22, calculate the feature vector based on the scenario: The data is fed back to S2 for generation and scenario iteration, serving as the basis for the next round of data expansion and generalization, thus achieving a closed loop between testing and training.

[0053] Based on the test results and scene performance scores obtained in S43-1: when the reliability index of defect localization in all scenes is lower than the threshold, the training termination condition is triggered, the training process is terminated, and the algorithm completes the training iteration; otherwise, defect scene localization and attribution analysis are performed. Based on the results of multi-round virtual perception injection into real vehicle tests, statistical analysis is performed on the same type of scenarios to identify stability issues of the algorithm in such scenarios. By combining iteration logs, defects are attributed to their causes, and the categories of defect sources are identified; repeated tests are conducted on the defect scenarios to verify the test results. The defect scenario feature indicators are fed back to the data-targeted generation process for the generation and generalization of relevant scenario data. S5 loop execution steps: This invention employs a training-testing-diagnosis-generation iterative mechanism to continuously optimize algorithm performance. In each iteration, the platform sequentially completes S1 data standardization, S2 targeted data generation and category reconstruction, S3 ensemble accelerated training, and S4 closed-loop testing and defect diagnosis.

[0054] When the overall performance indicators during the testing phase Satisfy the preset threshold for all scene categories When the system determines that the training has converged, it triggers the termination condition. If any scene category exists Then the corresponding defect scenario set The data flows back to S2 to perform data expansion and fine-grained iteration of the scenario.

[0055] This cycle continues until the performance metrics of all test scenarios stabilize within the threshold range. Through this closed-loop mechanism, the present invention enables the adaptive evolution of autonomous driving algorithms in long-tail and complex scenarios, ensuring that model performance gradually converges with each iteration and meets performance requirements.

[0056] Beneficial effects Compared with the prior art, the present invention has the following advantages: 1. Forming a closed-loop architecture of data reinforcement, accelerated training, and dynamic testing: Unlike the traditional fragmented linear process of "collection, training, and testing", this invention realizes a full-link loop on a unified platform, enabling data generation and algorithm improvement to drive each other and significantly improve iteration efficiency.

[0057] 2. Support for targeted enhancement and scene iteration: Through defect scene identification, targeted data generation and fine-grained scene iteration, this invention can quickly cover long-tail and extreme scenes, avoiding the inefficient method of simply relying on large-scale data collection.

[0058] 3. Accelerate training and testing collaboratively: The platform can dynamically integrate test feedback during the training process, automatically adjust data ratios and parameter optimization strategies, accelerate the correction of weak points in the algorithm, improve algorithm training efficiency, and significantly shorten the R&D cycle.

[0059] 4. Constructing a closed-loop dynamic test for real vehicles: The test method proposed in this invention introduces a dynamic adjustment mechanism for test difficulty while ensuring real vehicle dynamic conditions. This improves test coverage, reduces the risk of testing in real dangerous scenarios, and achieves accurate location of defects in the tested algorithm and closed-loop feedback of results. Attached Figure Description

[0060] Figure 1 This is a flowchart illustrating an integrated training and testing data closed-loop platform and method for autonomous driving algorithms provided by the present invention. Figure 2 A schematic diagram illustrating the data-targeted generation and category reconstruction process provided by this invention; Figure 3 This is a schematic diagram of the integrated accelerated training process provided by the present invention; Figure 4 A schematic diagram of the virtual perception injection real vehicle test architecture provided by this invention; Detailed Implementation The technical solution provided in this application will be further described below with reference to specific embodiments and accompanying drawings. The advantages and features of this application will become clearer from the following description.

[0061] Currently, mainstream autonomous driving algorithm training and testing schemes are relatively fragmented. Most schemes use independently organized, large-scale training datasets to train the proposed algorithm once. Then, another set of testing standards is used to verify the algorithm's reliability probabilistically through long-term, long-mileage simulation tests and real-vehicle tests. However, long-tail scenarios have a low probability of occurrence, which may lead to a situation where the failure rate in some dangerous scenarios is high, but the overall success rate data is still excellent, thus masking real problems.

[0062] Furthermore, even when some defective scenarios are discovered, the generalization of training data is mostly compensated by increasing the amount of data collected and filtering. Traditional closed-loop data schemes for training and testing are difficult to achieve efficient iteration of autonomous driving algorithms.

[0063] To address the aforementioned problems, this invention provides an integrated training and testing method for autonomous driving algorithms. It accelerates algorithm training and testing through efficient scene data generalization, enabling efficient iteration of the autonomous driving algorithm and resulting in an autonomous driving algorithm with strong generalization capabilities. Figure 1 As shown, the method includes the following steps: S1 model training basic data acquisition: The following section uses a typical scenario of "unprotected left turn at an urban intersection" to specifically illustrate the integrated training and testing method for autonomous driving algorithms of the present invention.

[0064] The tested autonomous vehicle acquires environmental information through onboard sensors: camera images LiDAR point cloud and the vehicle's own dynamic state , where represent longitudinal velocity, acceleration, steering angle, yaw rate, and heading angle, respectively. The input also includes a high-precision map. Traffic signal status .

[0065] The model output includes future predicted trajectories. and control commands and risk assessment indicators Therefore, it can be formalized as: After standardizing and labeling the collected data, the scenes are classified according to road topology, interaction mode, and environmental attributes, and a comprehensive difficulty index is defined: in, To perceive the difficulty, For topology difficulty, To increase the difficulty of interaction, For weights. According to... The numerical range divides the scene into three difficulty levels: low, medium, and high.

[0066] S2 Data Targeted Generation and Category Reconstruction: (e.g.) Figure 2 ) S21 Data Targeting Generation In the previous S4 testing phase, if a certain scenario category The following set of performance metrics Weighted score: like Below the threshold Then determine This is a defective scenario.

[0067] To address the deficiencies, the platform generates traffic event disturbances (such as sudden braking and crossing) through traffic simulation, enhances perception samples in low-light / severe weather conditions using GAN and 3DGS, and simulates random lane changes and signal anomalies of background vehicles to generate new sample sets. ,as follows: In the formula This is a method for generating realistic scenes, using 3DGS generation combined with GAN enhancement. These correspond to road topology, background traffic flow, random behavioral disturbances of participants, and natural environmental elements, respectively. Information is generated by the vehicle's own sensors.

[0068] Road topology information includes topological structure, lane line sets, conflict zone sets, and traffic signals. Background traffic flow includes interacting vehicles, pedestrians, and non-motorized vehicles. Environmental factors consider illumination disturbances, weather disturbances, and object occlusion. The generation process considers vehicle sensor types and specific configuration parameters, including sensor spatial locations and camera and LiDAR calibration data.

[0069] The number of new samples generated through the above process is defined as: in The generated scene-aware data Compared with the original defect scene perception data Sample size This is the amplification factor, which is generally set to 1 to 2 depending on the severity of the defect.

[0070] S22 Scene Refinement and Reconstruction If, after several rounds of training, the performance increment... Less than the preset incremental threshold And continuous If the conditions are not met, a scene iteration is triggered. Feature vector extraction: Clustering was performed using a Gaussian mixture model. Hyperparameters are obtained through iterative estimation. The original scene category is further subdivided into multiple subcategories. Then, targeted generation is performed on each subclass to expand the training samples.

[0071] S3 integrated accelerated training (e.g.) Figure 3 ) After the scene data undergoes S1 data standardization and S2 data targeted generation, it enters the integrated accelerated training phase. This phase dynamically adjusts the distribution of training data and training parameters to achieve targeted enhancement of the autonomous driving algorithm for defective scenarios and improve overall convergence speed.

[0072] 1. Data manipulation Before the algorithm under test is connected, the user first retrieves the standardized scene data from the standardized database and executes the matching strategy to calculate the scene weights. : in, Scene category The number of samples. If For scenarios with defects identified in the previous training round, the platform dynamically increases their weight: in, For adjustment coefficients, if the scenario For defective scenarios Select 1 if the data ratio is 1, otherwise select 0. The dataset after adjusting the data ratio is used in the subsequent training phase to ensure that the defective scenarios are reinforced in the training set.

[0073] 2. Training Management During the training phase, scene categories are used. The weights are dynamically adjusted to calculate the training loss function, whereby... Scene category Loss: This strengthens the impact of defective scenarios on the model during overall training, accelerating optimization and convergence. Simultaneously, the optimal model parameters are saved in real-time based on the validation results in the next step, avoiding overfitting.

[0074] 3. Testing and Evaluation During training, the testing and evaluation module calls a standardized scenario library to generate a test set: Key performance indicators (KPIs) of the model in the perception, prediction, planning, and control stages are calculated, such as perception accuracy (AP), trajectory prediction error (ADE), collision rate, and lateral and longitudinal control stability, and an open-loop evaluation comprehensive score is generated. , The open-loop score for each key indicator sub-item: like If the scenario is not found, it is marked as a defective scenario and returned to the data management process in Section S3 for the next round of training. This process repeats continuously until... .in These are empirical values ​​in S2. This is the adjustment value used during open-loop verification, typically ranging from 1 to 1.2.

[0075] Through the above process, the integrated accelerated training platform in this embodiment realizes a unified closed loop of "data operation - training management - test evaluation", which enables the algorithm under test to quickly access data reinforcement, complete training optimization, and obtain real-time performance evaluation and defect feedback in each iteration.

[0076] S4 Closed-Loop Testing and Defect Diagnosis: After the autonomous driving algorithm under test completes integrated accelerated training, a closed-loop testing and defect diagnosis process integrating virtual and real environments is executed. This process combines virtual simulation with real vehicle testing environments, and through the dynamic interaction of virtual traffic flow, intelligent target objects, and remote driving, it achieves multi-dimensional verification and defect exposure of the algorithm.

[0077] S41 Virtual Perception Injection Real Vehicle Testing Process like Figure 4 A schematic diagram of the real-vehicle test architecture for injecting virtual perception; In a closed testing ground, a testing environment was built that included real vehicles, intelligent targets, and remote driving control terminals.

[0078] Virtual traffic flow information, including background vehicles, pedestrians, bicycles, and other traffic participants, is generated through a simulation platform. Virtual perception data, such as scarce perception scenarios like low light, rain / snow, and fog / haze, is generated based on scene generation algorithms and sensor models. This virtual perception data is injected into the real vehicle domain controller via an interface and fused with real sensor inputs to form a hybrid input signal, enabling the algorithm to receive "virtual-real" environmental information within the real vehicle control framework. To increase scenario complexity and uncertainty, controllable intelligent targets, such as remotely drivable electric dummy cars or simulated pedestrians, are deployed in the test field. Driving simulators control these remote target vehicles to achieve dynamic interactive testing of "humans on a loop." For example, in an unprotected left-turn scenario, remotely driven vehicles can create oncoming traffic interference to verify the decision-making rationality and reaction latency of the algorithm under test.

[0079] The tested autonomous driving algorithm runs in a virtual-real fusion test environment. The platform collects key performance indicators of each module, including perception accuracy and false detection rate; trajectory prediction error; path collision rate; longitudinal and lateral control stability; and decision delay time in real time.

[0080] In the scene collection After executing the virtual perception injection real vehicle test process, the performance is scored according to the S21 comprehensive performance definition: like This increases the difficulty. If between and Between these thresholds, the scenario is deemed a boundary scene and requires repeated testing; otherwise, it falls below a preset threshold. This type of scenario is identified as a defect. A set of defective scenarios is obtained. The defect types are classified according to the performance degradation stage, including insufficient perception, prediction deviation, planning and decision-making, and control execution defects.

[0081] For defect scenarios, extract their feature vectors. The data is then fed back to the data-targeted generation module S2. In the next iteration, the system prioritizes data generation and generalization for this type of scenario to achieve targeted reinforcement. If the overall scenario score is higher than [a certain value] after two consecutive rounds of testing... Then the algorithm training process ends.

[0082] S5 loop execution steps: In this embodiment, the platform achieves continuous optimization of the autonomous driving algorithm through a cyclical iterative mechanism of "training-testing-diagnosis-generation".

[0083] In the initial stage, the system forms the first training set based on the S1 data standardization and scenario classification results; the initial model training is completed through the S3 integrated accelerated training platform to obtain the initial version of the algorithm model.

[0084] After the initial training is completed, the virtual-real fusion closed-loop test procedure described in S4 is executed. If the algorithm's performance score in any scene category is lower than the threshold... If a scenario meets the threshold requirement, it is marked as a defective scenario, and the S2 data targeting and scenario generalization process is triggered; if all scenarios meet the threshold requirement, the algorithm training terminates.

[0085] Next, the data generation and retraining phase begins. The defect scene features are fed back to the S2 module, where the scene generation model performs targeted data generation and feature expansion. The newly generated sample data is then fused with the original dataset and re-entered into the S3 training platform for accelerated training. During training, the platform automatically adjusts the defect scene weights and dynamically updates the training parameters and loss function based on test feedback.

[0086] The system repeatedly executes the "training-testing-diagnostics-generation" loop until the performance metrics for each scenario category are stably met: This means that all test scenarios pass the preset threshold standard. At this point, the platform triggers the training termination flag and outputs the final converged algorithm model.

[0087] The above description is merely a description of preferred embodiments of this application and is not intended to limit the scope of this application in any way. Any changes or modifications made by those skilled in the art based on the above-disclosed technical content should be considered as equivalent and valid embodiments and fall within the scope of protection of the technical solution of this application.

Claims

1. An integrated training and testing method for autonomous driving algorithms, characterized in that, Includes the following steps: S1 data standardization; Data standardization is performed on open-source datasets from different sources to unify the input-output structure of training data; then, a multi-dimensional scene classification system is established for the labeled road traffic data. S2 Data Targeted Generation and Category Reconstruction; First, identify the defect scenarios for targeted data generation; for the identified defect scenarios, use a scenario generation model to generate targeted data, thereby increasing the number of training samples in the defect scenarios. The scenario refinement and reconstruction iteration method is used to refine the granularity of scenario categories and enhance the data targeting and algorithm training effect. S3 integrates accelerated training; After the scene data is generated and completed, the algorithm is trained to speed up the process. S4 Closed-Loop Testing and Defect Diagnosis; Based on closed-loop testing and defect diagnosis methods, dynamic closed-loop testing of autonomous driving algorithms is carried out under different scenarios and difficulties to diagnose algorithm defects and use them for algorithm iterative training. S5 executes the steps in a loop; Steps S1-S4 are executed repeatedly to continuously optimize the algorithm's performance.

2. The integrated training and testing method for autonomous driving algorithms according to claim 1, characterized in that, Step S1 is as follows: S11 data standardization processing: First, data standardization is performed on open-source datasets from different sources to unify the input-output structure of training data; Input data includes: vehicle perception data, vehicle status information, and map and environmental information; The output data consists of the vehicle's planning and control commands in the future time domain, specifically including: predicted trajectory, control commands, and risk assessment indicators. S12 Scene Classification and Grading Traffic scenarios are broken down into three main components: environmental attributes, road topology, and interaction patterns. Scenarios are classified according to these components to achieve fine-grained positioning of target scenario data. Based on the classification, a comprehensive difficulty index is defined. This is used to measure the challenge of a scenario to autonomous driving algorithms, as follows: in, To perceive the difficulty; Topology difficulty; The difficulty level is determined by the interaction. These are weight parameters; Based on the comprehensive difficulty index The numerical range is used to divide the overall difficulty level of the scene, in order to ensure the comparability of difficulty level divisions under multi-source data conditions: Low difficulty L: ; Medium difficulty M: ; High-difficulty H: ; Among them, threshold The determination is made manually based on experience, taking into account the statistical distribution characteristics of different data sources.

3. The integrated training and testing method for autonomous driving algorithms according to claim 1, characterized in that, Step S2 is as follows: S21 Data Targeting Generation First, identify the defective scenarios for targeted data generation, and then generate subsequent data within this target scenario category. The "defect scenario" is defined as: during the testing of autonomous driving algorithms, the key performance indicators of the algorithm in a certain scenario fail to reach the preset threshold, thus triggering emergency obstacle avoidance or being taken over by the driver. The key performance indicators include: (1) Perception layer indicators: target detection accuracy, target tracking accuracy, false detection / false detection rate; (2) Prediction layer indicators: average trajectory prediction error, final position error; (3) Planning and control layer indicators: collision rate, path deviation rate, braking / steering stability; Define scene performance score: ， in For the first The normalized score of each indicator, As weight; If the algorithm is in the category Difficulty Scene Overall score satisfy: If so, the scene category is determined to be a defective scene; if If it is, then it is determined to be a boundary scenario and further repeated testing is required; if If the scenario is not a defective scenario, then it is determined that the scenario is not a defective scenario; where and The preset performance threshold; The difficulty level is defined in S12; Among them, the scene performance score for defect scene discrimination is obtained from the previous round of S4 closed-loop testing and defect diagnosis. The initial round directly uses the default scene set to enter the training phase. Next, for the identified defect scenarios, based on the scenario feature dimension, the scenario generation model is used to generate targeted data based on the previous round of defect scenario diagnosis results, thereby increasing the training sample size under the defect scenarios. The three components of a traffic scenario—road topology, interaction patterns, and environmental attributes—are mapped to three dimensions: road topology, traffic events, and environmental perception. Behavioral perturbations are added to enhance the generalization of scenario data, enabling targeted data generation. (1) Road topology dimension: road topology The spatial environment in which traffic scenarios occur serves as the basic spatial constraints between the measured subject and the traffic flow interaction objects, including road boundaries, lane division and intersection, and traffic flow direction; (2) Traffic incident dimension: road topology in defective scenarios With background traffic flow Under certain conditions, dynamic events that interact with the algorithm under test are automatically constructed through a traffic simulation platform; (3) Behavioral disturbance dimension: For the defective scenario, disturbance modeling of traffic participants is introduced. To construct more complex interactive data; (4) Perception dimension: Add natural environment perception elements to the interactive scene. Based on the vehicle's sensor layout and spatial location, visual perception data is generated using the 3D Gaussian scene generation (3DGS) method, and the realism of the defect scene is enhanced by the generative adversarial network (GAN) method. A set of sensing data The generation process is described as follows: In the formula A method for generating real-world scenarios. These correspond to road topology, background traffic flow, random behavioral disturbances of participants, and natural environmental elements, respectively. Information is generated by the vehicle's own sensors; The number of new samples generated through the above process is defined as: in The generated scene-aware data Compared with the original defect scene perception data Sample size This is the amplification factor, dynamically set according to the severity of the defect; Finally, the generated defect scenario extended samples The data is merged with the original scene data to form the training set input for the current round, thereby achieving targeted reinforcement of the training data for the defective scenes of the tested algorithm. The targeted generation results are then transformed into new data samples and fed back into the training process. S22 Scene Refinement and Reconstruction While performing targeted data generation, the scene category granularity is refined through scene refinement and reconstruction iteration method to enhance the data targeting and algorithm training effect. First, the trigger condition for scene refinement and reconstruction is: suppose a certain scene belongs to a certain difficulty category. The performance metrics after the t-th round of training are The previous performance indicators were If the following conditions are met: and continuous None of the rounds exceeded the threshold This triggers the scene iteration process, where This is the performance increment threshold. Minimum number of consecutive detection rounds; Next, scene feature extraction and clustering are performed; for the scene category that triggers the iteration, scene feature vectors are extracted: in For the target number of scenes, These are the lighting parameters. Due to weather conditions, For road topology indicators, Interaction strength; The feature vectors are input into a clustering algorithm based on a Gaussian Mixture Model (GMM), assuming the sample distribution is as follows. A weighted combination of Gaussian distributions: Hyperparameters are obtained through iterative estimation. , respectively corresponding to the first The weights, mean vectors, and covariance matrices of each Gaussian component are used to subdivide the original scene category into multiple subclasses. ; After obtaining a new subclass Then, the S21 data targeting generation step is performed to obtain an expanded sample set. This data is then integrated with the existing training set to form a new round of iterative training data. After scene iteration processing, the original defect scene is refined into more comprehensive sub-scenes, enabling the algorithm to perform reinforcement learning for different fine-grained features.

4. The integrated training and testing method for autonomous driving algorithms according to claim 1, characterized in that, Step S3 is as follows: S31 Training Data Management First, based on the scene data described in S1 / S2, the training data is prepared through the following process: (1) Accessing standardized scenario data: Read training data from the standardized database formed in step S1, integrate datasets from different sources, and uniformly format and store the data from different sources to ensure a unified input structure; (2) Perform dynamic data matching: Based on the feedback results of the previous training evaluation and the distribution of scene categories in the data, automatically match data of different difficulties and categories to obtain the dataset used in this training. S32 Algorithm Training Parameter Optimization After each round of training data preparation and allocation is completed, training parameters are set according to the following process, and model training is performed: (1) Access the algorithm model under test: Access the autonomous driving algorithm under test through the standard interface and initialize the model parameters; (2) Training process execution: Call the training set after S31 allocation for batch training and model parameter iteration; (3) Dynamic parameter adjustment: Based on the feedback results of the previous round of S33 evaluation, the learning rate and loss function weights are automatically adjusted to enhance the learning effect of defective scenarios and shorten the model convergence time; S33 Inter-round Test Evaluation During training, intermediate rounds of open-loop testing are performed to verify whether the algorithm training has converged and to conduct quantitative evaluations of targeted access to training data. The steps include: (1) Automatically generate test sets: During the training process, the standardized scene data obtained by the S31 training data management is called, and test scene sets between rounds are generated according to the difficulty level. (2) Calculation of indicators: The performance of the model in the perception, prediction, planning and control stages is quantitatively evaluated, and key indicators are calculated, including perception accuracy, prediction error, collision rate and control stability. (3) Defect identification: If the comprehensive score of a certain scenario is lower than the preset threshold, it is marked as a defect scenario, and the test results are returned to S31; S34 Results Feedback and Data Feedback After each round of data matching, parameter adjustment, model training, and inter-round testing is completed, the test results generated by S33 are fed back to S31 to form a new round of training data input into a new round of integrated accelerated training, realizing the training-evaluation-feedback cycle; The entire process repeats until the model meets the preset performance threshold in the S33 test, at which point the integrated accelerated training process ends and enters the S4 closed-loop testing and defect diagnosis process.

5. The integrated training and testing method for autonomous driving algorithms according to claim 1, characterized in that, Step S4 is as follows: S41 Virtual Perception Injection Real Vehicle Testing Method By combining virtually generated perception information with the real vehicle state, the tested autonomous driving system can accept virtual environment input within the real vehicle control framework, thereby covering more long-tail scenarios and extreme situations while ensuring safety; specifically as follows: (1) Method definition Virtual perception injection into real vehicle testing refers to directly injecting the perception information generated by the virtual simulation platform into the vehicle's domain controller in the real vehicle operating environment, so that the sensor input received by the autonomous driving algorithm is a combination of real sensing data and virtual generated data, thereby completing closed-loop decision-making and execution under real vehicle conditions. (2) Input Construction Virtual traffic flow injection: Based on the road topology and traffic elements of the labeled scene, interactive virtual traffic participants are generated in the simulation platform, and their movement trajectories and interactive behaviors are constructed. Virtual sensor perception injection: Constructing corresponding virtual perception data through sensor modeling and image / point cloud generation methods; Environmental disturbance injection: Introduce sudden disturbances into the scene to increase the intensity of the test; (3) Actual vehicle operation and control In a closed testing ground, the actual vehicle is equipped with a domain controller to receive mixed sensing input and execute subsequent prediction, planning and control decision-making processes based on this; (4) Enhanced human-computer interaction Introducing intelligent target objects and remote drivers into the test field, the intelligent target objects, as interactive physical traffic participants, work together with the virtual traffic flow to affect the vehicle under test; the driving simulator supports remote drivers to control the target vehicle, realizing the three-way dynamic interaction of "human on the loop" between the intelligent target object, the algorithm under test, and the background traffic flow, enhancing the complexity and randomness of the test scenario. (5) Evaluation and feedback During the testing process, the platform collects key performance indicators of the tested model in different scenarios. If the indicator is lower than the set threshold, the scenario is determined to be a defective scenario, and its features are fed back to S21 for subsequent scenario iteration and target data generation. S42 Closed-Loop Dynamic Test The closed-loop dynamic testing method is a process of verifying and providing feedback on autonomous driving algorithms step by step, based on virtual perception injection into real vehicle testing and combined with scenario difficulty grading and dynamic adjustment mechanisms; the details are as follows: (1) Initial test set generation Based on the data standardization and scenario difficulty classification results, the current round of test scenario set is defined as follows: In the formula This indicates the difficulty level of the i-th type of scenario. The test subset below; (2) Calculation of performance indicators In each type of scenario Run the algorithm under test and calculate its performance score in different scenarios according to the definition in S21: In the formula For the first Key performance indicators, including sensing accuracy: Prediction error Collision rate lateral deviation Longitudinal acceleration stability ; Assigning weights to each indicator. ; (3) Dynamic difficulty adjustment In a certain type of scenario Overall rating satisfy: This increases the testing difficulty for that scenario category. ; like If the scenario is deemed defective, the defect diagnosis process will begin. like If so, maintain the original difficulty and repeat the test to verify stability; in The base value is consistent with S21, and dynamically adjusted based on manual experience. (4) Closed-loop iterative execution After each round of testing, a set of test results for different categories of scenarios is obtained: like If the training termination condition is triggered, it indicates that the algorithm has reached the expected performance. Otherwise, Defect scenario set The data flows back to S21, triggering scene iteration and targeted data generation, and entering the next training-testing cycle. S43 Defect Location Diagnosis (1) Defect scenario localization Locate the defect scenarios in the current training round of the algorithm under test; For the set of scenarios that were judged to be defective The indicators are broken down and analyzed: perception indicators, prediction error, collision rate, and control stability, which correspond to defects such as insufficient perception, trajectory prediction, planning safety, and execution control, respectively. (2) Result reflux For scene set In the defective scenario described in S22, calculate the feature vector based on the scenario: The data is fed back to S2 for generation and scenario iteration, serving as the basis for the next round of data expansion and generalization, thus achieving a closed loop between testing and training. Based on the test results and scene performance scores, when the reliability index of defect localization in all scenes is lower than the threshold, the training termination condition is triggered, the training process is terminated, and the algorithm completes the training iteration. Otherwise, perform defect scenario localization and attribution analysis; Based on the results of multi-round virtual perception injection into real vehicle tests, statistical analysis is performed on the same type of scenarios to identify stability issues of the algorithm in such scenarios. By combining iteration logs, defects are attributed to specific causes, and the categories of defect sources are identified. Repeated tests were conducted on the defective scenarios to verify the test results; The defect scenario feature indicators are fed back to the data-targeted generation process for the generation and generalization of relevant scenario data.

Images