An integrated training and testing method for automatic driving algorithms

Abstract

The application discloses an integrated training and testing method for automatic driving algorithms, which can realize closed-loop iteration of training data target generation, algorithm integrated accelerated training, and algorithm defect testing and diagnosis under a unified architecture. Around the efficient iteration and performance improvement of automatic driving algorithms, the specific steps include: constructing a training data standardization link, proposing a training data target generation and generalization method, realizing efficient data supply and balanced distribution; establishing an algorithm integrated training platform supporting modularization and end-to-end and other multi-technology routes, promoting algorithm collaborative training acceleration; designing an automatic driving algorithm closed-loop defect testing and diagnosis mechanism based on virtual perception injection real vehicle testing method, providing targeted feedback and optimization basis for data target generation, and finally forming a training and testing integrated data closed-loop method, which can accelerate algorithm iteration and upgrade and real vehicle landing process, and provide systematic support for building a safe, intelligent and efficient automatic driving system.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to an integrated training and testing method for autonomous driving algorithms. Background Technology

[0002] With the rapid iteration of autonomous driving technology, domestic and international manufacturers have shifted from the first half, focusing on technological accumulation, to the second half, focusing on the commercial application of algorithms. However, algorithms still face numerous challenges in complex and open scenarios. The current mainstream approach involves continuously collecting and labeling data to expand the training set, and then verifying and testing the algorithm in diverse scenarios. However, this linear "collection-training-testing" process has several shortcomings: First, the training data structure is inconsistent, making it difficult to support efficient utilization across scenarios and algorithms; second, high-quality training data is sparse, data expansion lacks targeting, and is inefficient and time-consuming; third, the training and testing stages are disconnected, failing to create a closed-loop feedback loop for data generation and algorithm improvement.

[0003] Therefore, there is an urgent need to establish a closed-loop method that can connect data, training, and testing under a unified architecture. This method can provide systematic support for the iterative upgrade of autonomous driving algorithms and their deployment in real vehicles by targeting and generalizing training data, accelerating training through efficient integration of algorithms, and performing closed-loop defect diagnosis based on virtual perception injection into real vehicle testing. Summary of the Invention

[0004] This invention provides an integrated training and testing method for autonomous driving algorithms, which solves the problems in the prior art, such as long iteration and verification cycles and unsatisfactory improvement effects of autonomous driving algorithms due to the lack of feedback mechanisms in the training and testing stages and insufficient coverage of long-tail scenario data.

[0005] Technical solution

[0006] An integrated training and testing method for autonomous driving algorithms includes the following steps:

[0007] S1 data standardization;

[0008] Data standardization is performed on open-source datasets from different sources to unify the input-output structure of training data; then, a multi-dimensional scene classification system is established for the labeled road traffic data.

[0009] S2 Data Targeted Generation and Category Reconstruction;

[0010] First, identify the defect scenarios for targeted data generation; for the identified defect scenarios, use a scenario generation model to generate targeted data, increasing the training sample size under the defect scenarios; use a scenario refinement and reconstruction iterative method to refine the granularity of scenario categories, enhancing the data targeting and algorithm training effects.

[0011] S3 integrates accelerated training;

[0012] After the scene data is generated and completed, the algorithm is trained to speed up the process.

[0013] S4 Closed-Loop Testing and Defect Diagnosis;

[0014] Based on closed-loop testing and defect diagnosis methods, dynamic closed-loop testing of autonomous driving algorithms is carried out under different scenarios and difficulties to diagnose algorithm defects and use them for algorithm iterative training.

[0015] S5 executes the steps in a loop;

[0016] Steps S1-S4 are executed repeatedly to continuously optimize the algorithm's performance.

[0017] Specifically as follows:

[0018] S1 Data Standardization:

[0019] S11 data standardization processing:

[0020] First, data standardization is performed on open-source datasets from different sources to unify the input-output structure of training data, ensuring the compatibility of the integrated training and testing closed-loop platform provided by this invention with training data from different sources.

[0021] The input data consists of three parts: vehicle perception data, vehicle status information, and map and environmental information.

[0022] 1. Sensing data:

[0023] (1) Camera image sequence: ,in , These represent the image height, width, and number of channels, respectively.

[0024] (2) LiDAR point cloud: ,in Represents three-dimensional coordinates. Indicates the intensity of reflection;

[0025] 2. Vehicle status information:

[0026] Stored in dynamic state vector format:

[0027] in For longitudinal velocity, For acceleration, For steering wheel angle, The yaw rate is angular velocity. For heading angle;

[0028] 3. Map and Environmental Information

[0029] (1) High-precision map topology data: Where L is the set of lanes, For the set of intersections;

[0030] (2) Dynamic traffic signals: traffic light status In this system, 0 represents a red light, 1 represents a yellow light, and 2 represents a green light.

[0031] In summary, formal representations of training data in different scenarios can be formed:

[0032]

[0033] The output data consists of the vehicle's planning and control commands for the future time domain, specifically including:

[0034] 1. Predicting the trajectory

[0035]

[0036] in Indicates the predicted location coordinates. Indicates the heading angle. For the future There are [number] time steps, with the longest prediction time being [time]. ;

[0037] 2. Control Commands

[0038] Vehicle chassis longitudinal control throttle opening Braking strength Lateral control of front wheel steering angle ;

[0039] 3. Risk assessment indicators (optional output)

[0040] Collision risk rating:

[0041] Traffic rule compliance:

[0042] Therefore, the overall mapping relationship of the model is as follows:

[0043]

[0044] in, ,

[0045] The raw data is processed into training data in a unified form to meet the training requirements of different algorithms, such as segmented and end-to-end.

[0046] S12 Scene Classification and Grading

[0047] A multi-dimensional scenario classification system is established for the labeled road traffic data to meet the needs of dynamic adjustment of test intensity, targeted generation and generalization of defect scenario categories in subsequent stages.

[0048] This invention breaks down traffic scenarios into three main components: environmental attributes, road topology, and interaction patterns. It also considers these three components to be strongly correlated with the algorithm's perception and planning control capabilities. Therefore, scenarios are classified according to these components to achieve fine-grained positioning of target scenario data.

[0049] The specific reference indicators for grading include:

[0050] (1) Environmental attribute dimension: day / night, sunny / rainy / snowy / fog, low visibility, sudden changes in light (such as entering and exiting tunnels), etc.;

[0051] (2) Road topology dimension: highways, urban arterial roads, branch roads, roundabouts, intersections, etc.;

[0052] (3) Interaction mode dimension: single lane following, lane change, lane merging, unprotected left turn, pedestrian crossing, non-motorized vehicle mixed traffic, etc.

[0053] Based on the classification, a comprehensive difficulty index is defined. This is used to measure the challenge of a scenario to autonomous driving algorithms, as follows:

[0054]

[0055] in,

[0056] To assess the difficulty (number of targets, occlusion ratio, degree of influence of lighting / weather);

[0057] Topological difficulty (number of lanes, number of conflict points, road curvature, etc.);

[0058] The difficulty of the interaction (number of interaction targets, proportion of conflict zones, and intensity of interaction).

[0059] This is a weighting parameter that can be set according to actual needs.

[0060] Based on the comprehensive difficulty index The numerical range is used to divide the overall difficulty level of the scene, ensuring the comparability of difficulty level classifications under multi-source data conditions. The threshold is determined manually based on experience, according to the statistical distribution characteristics of different data sources. ,but:

[0061] Low difficulty (L): For example, a single-lane straight road with good lighting;

[0062] Medium difficulty (M): For example, multiple lanes merging, and significant changes in lighting;

[0063] High Difficulty (H): Such as complex intersections, multi-purpose traffic, and severe weather.

[0064] S2 Data Targeting Generation and Category Reconstruction

[0065] S21 Data Targeting Generation

[0066] This invention provides a process for targeted generation of training data for autonomous driving algorithms:

[0067] First, identify the defect scenarios for targeted data generation, and then generate subsequent data within these target scenario categories.

[0068] A "defect scenario" is defined as a scenario in which the key performance indicators of the algorithm in a certain scenario fail to reach the preset threshold during the testing of autonomous driving algorithms, thus triggering emergency obstacle avoidance or being taken over by the driver.

[0069] The key performance indicators include, but are not limited to:

[0070] (1) Perception layer metrics: Average Precision AP (Average Precision AP), Multiple Object Tracking Accuracy MOTA (Multiple Object Tracking Accuracy MOTA), False Detection / False Detection Rate;

[0071] (2) Prediction layer indicators: Average Displacement Error (ADE) and Final Displacement Error (FDE).

[0072] (3) Planning and control layer indicators: collision rate, path deviation rate, braking / steering stability.

[0073] Define scene performance score:

[0074] ,

[0075] in For the first The normalized score of each indicator, As weight.

[0076] If the algorithm is in the category Difficulty Scene Overall score satisfy: If so, the scenario category is determined to be a defective scenario; if If it is, then it is determined to be a boundary scenario and further repeated testing is required; if If so, the scenario is determined to be a non-defect scenario. Where: difficulty , The difficulty level is defined in S12; and The preset performance threshold is determined by statistical analysis of the test performance of different types of human drivers and manual calibration in combination with the actual difficulty of typical scenarios, so as to ensure the rationality of defect scenario identification.

[0077] Among them, the scene performance score for defect scene discrimination is obtained from the previous round of S4 closed-loop testing and defect diagnosis. In the initial round, the default scene set is used directly to enter the training phase.

[0078] Next, for the identified defect scenarios, this invention uses a scenario generation model based on the previous round of defect scenario diagnosis results to generate targeted data based on the following scenario feature dimensions, thereby increasing the training sample size under defect scenarios.

[0079] This invention maps the three main components of a traffic scenario—road topology, interaction patterns, and environmental attributes—to three dimensions: road topology, traffic events, and environmental perception. It also adds behavioral perturbations to enhance the generalization of scenario data and enable targeted data generation.

[0080] (1) Road topology dimension: road topology The spatial environment in which traffic scenarios occur serves as the basic spatial constraint between the measured subject and the traffic flow interaction object, including road boundaries, lane division and intersection, and traffic flow direction.

[0081] (2) Traffic incident dimension: road topology in defective scenarios With background traffic flow Under these conditions, dynamic events that interact with the algorithm under test are automatically constructed through a traffic simulation platform.

[0082] (3) Behavioral disturbance dimension: For the defective scenario, disturbance modeling of traffic participants is introduced. For example, randomly setting background vehicle behaviors such as lane changing, sudden braking, and acceleration, or increasing traffic signal delays and anomalies, to construct more complex interactive data.

[0083] (4) Perception dimension: Add natural environment perception elements to the interactive scene. Based on the vehicle's sensor layout and spatial location, visual perception data is generated using the 3D Gaussian Scene Generation (3DGS) method, and the realism of defective scenes is enhanced by the Generative Adversarial Network (GAN) method.

[0084] A set of sensing data The generation process can be described as follows:

[0085]

[0086] In the formula This refers to the method for generating realistic scenes using 3DGS generation combined with GAN enhancement. These correspond to road topology, background traffic flow, random behavioral disturbances of participants, and natural environmental elements, respectively. Information is generated by the vehicle's own sensors.

[0087] The number of new samples generated through the above process is defined as:

[0088]

[0089] in The generated scene-aware data Compared with the original defect scene perception data Sample size This is the amplification factor, which is dynamically set according to the severity of the defect.

[0090] Finally, the generated defect scenario extended samples The data is merged with the original scene data to form the training set input for the current round, thereby achieving targeted reinforcement of the training data for the defective scenes of the tested algorithm. The targeted generation results are then transformed into new data samples and fed back into the training process.

[0091] S22 Scene Refinement and Reconstruction

[0092] While performing targeted data generation, this invention proposes a scene refinement and reconstruction iterative method to refine the granularity of scene categories and enhance the effects of targeted data reinforcement and algorithm training.

[0093] This invention defines "scene refinement and reconstruction" as follows: During multiple rounds of training, when the performance index improvement of the algorithm is insufficient to exceed the preset incremental threshold after several targeted generation and training in a certain type of defect scene, the system automatically triggers a finer-grained splitting and reconstruction of the scene to obtain new scene categories and expand the data.

[0094] First, clarify the triggering conditions for scene refinement and reconstruction: Let a certain scene be related to a difficulty category. The performance metrics after the t-th round of training are The previous performance indicators were If the following conditions are met: and continuous None of the rounds exceeded the threshold This triggers the scene iteration process. This is the performance increment threshold. This represents the minimum number of consecutive detection rounds.

[0095] Next, scene feature extraction and clustering are performed. For the scene category that triggers the iteration, scene feature vectors are extracted:

[0096]

[0097] in For the number of target elements in the scene, These are the lighting parameters. Due to weather conditions, For road topology indicators, The interaction strength.

[0098] The feature vectors are input into a clustering algorithm based on a Gaussian mixture model (GMM), assuming the sample distribution is as follows. A weighted combination of Gaussian distributions:

[0099]

[0100] Hyperparameters are obtained through iterative estimation. , respectively corresponding to the first The weights, mean vectors, and covariance matrices of each Gaussian component are used to subdivide the original scene category into multiple subclasses. .

[0101] After obtaining a new subclass Then, the S21 data targeting generation step is performed to obtain an expanded sample set. It is then integrated with the existing training set to form a new round of iterative training data.

[0102] After scene iteration processing, the original defect scene is refined into more comprehensive sub-scenes, enabling the algorithm to perform reinforcement learning for different fine-grained features.

[0103] S3 Integrated Accelerated Training

[0104] After the scene data is generated and completed, the algorithm is trained at an accelerated pace. "Accelerated training" refers to dynamically adjusting the distribution of training data based on the test evaluation results after each round of training, in order to achieve targeted enhancement of the algorithm for defective scenes and improve the overall convergence speed.

[0105] Specifically, the following steps are included:

[0106] S31 Training Data Management

[0107] First, based on the scenario data described in S1 / S2, the user prepares the training data through the following process:

[0108] (1) Accessing standardized scenario data: Read training data from the standardized database formed in step S1, integrate datasets from different sources, and uniformly format and store the data from different sources to ensure a unified input structure;

[0109] (2) Perform dynamic data matching: Based on the feedback results of the previous training evaluation and the distribution of scene categories in the data, automatically match data of different difficulties and categories to obtain the dataset used in this training.

[0110] S32 Algorithm Training Parameter Optimization

[0111] After each round of training data preparation and allocation is completed, training parameters are set according to the following process, and model training is performed:

[0112] (1) Access the algorithm model under test: Access the autonomous driving algorithm under test through the standard interface and initialize the model parameters;

[0113] (2) Training process execution: Call the training set after S31 allocation for batch training and model parameter iteration;

[0114] (3) Dynamic parameter adjustment: Based on the feedback results of the previous round of S33 evaluation, the learning rate and loss function weights are automatically adjusted to enhance the learning effect of defective scenarios and shorten the model convergence time.

[0115] S33 Inter-round Test Evaluation

[0116] During training, intermediate rounds of open-loop testing are performed to verify whether the algorithm training has converged and to conduct quantitative evaluations of targeted access to training data. The steps include:

[0117] (1) Automatically generate test sets: During the training process, the standardized scene data obtained from the S31 training data management is called, and test scene sets between rounds are generated according to the difficulty level.

[0118] (2) Indicator calculation: Quantitatively evaluate the performance of the model in the perception, prediction, planning and control stages, and calculate key indicators including perception accuracy, prediction error, collision rate and control stability.

[0119] (3) Defect identification: If the overall score of a certain scenario is lower than the preset threshold, it is marked as a defect scenario, and the test results are returned to S31.

[0120] S34 Results Feedback and Data Feedback

[0121] After each round of data matching, parameter adjustment, model training, and inter-round testing is completed, the test results generated by S33 are fed back to S31 to form a new round of training data input for a new round of integrated accelerated training, realizing a training-evaluation-feedback cycle.

[0122] The entire process repeats until the model meets the preset performance threshold in the S33 test, at which point the integrated accelerated training process ends and enters the S4 closed-loop testing and defect diagnosis process.

[0123] S4 Closed-Loop Testing and Defect Diagnosis:

[0124] After the algorithm integration and accelerated training are completed, based on the closed-loop testing and defect diagnosis method described in this invention, dynamic closed-loop testing is performed on the autonomous driving algorithm under different scenarios and difficulties to diagnose algorithm defects and use them for algorithm iterative training.

[0125] Unlike the open-loop testing and evaluation in S33 used to determine training convergence and accelerate training, the method described in this section is a comprehensive closed-loop testing and defect diagnosis for autonomous driving algorithms, used for quantitative evaluation of autonomous driving algorithm capabilities, defect exposure, and algorithm improvement.

[0126] S41 Virtual Perception Injection Real Vehicle Testing Method

[0127] This invention proposes a virtual perception injection method for real-vehicle testing, used to reproduce complex traffic scenarios under real-vehicle conditions and to specifically verify autonomous driving algorithms. This method combines virtually generated perception information with the real vehicle state, enabling the tested autonomous driving system to accept virtual environment input within the real vehicle control framework. This allows for coverage of more long-tail scenarios and extreme situations while ensuring safety.

[0128] 1. Method Definition

[0129] Virtual perception injection into real vehicle testing refers to directly injecting perception information (including target detection results, traffic flow information, environmental disturbances, etc.) generated by a virtual simulation platform into the vehicle's domain controller in a real vehicle operating environment. This makes the sensor input received by the autonomous driving algorithm a combination of "real sensing data + virtual generated data", thereby completing closed-loop decision-making and execution under real vehicle conditions.

[0130] 2. Input Construction

[0131] Virtual traffic flow injection: Based on the road topology and traffic elements of the labeled scene, interactive virtual traffic participants (such as oncoming vehicles, pedestrians, and bicycles) are generated in the simulation platform, and their movement trajectories and interactive behaviors are constructed.

[0132] Virtual sensor perception injection: Construct corresponding virtual perception data through sensor modeling and image / point cloud generation methods, such as camera images in low visibility conditions and radar point cloud noise in rain and snow weather;

[0133] Environmental disturbance injection: Introduce sudden disturbances into the scenario, such as sudden braking of the vehicle in front, sudden obstacles, or abnormal traffic signals, to increase the intensity of the test.

[0134] 3. Actual vehicle operation and control

[0135] In a closed testing environment, a real vehicle equipped with a domain controller receives hybrid perception input (real + virtual) and executes subsequent prediction, planning, and control decisions based on this input. In this way, the vehicle can perform steering, braking, and acceleration under realistic dynamic conditions, resulting in test results that more closely resemble actual road performance.

[0136] 4. Enhanced human-computer interaction

[0137] To enhance testing diversity, intelligent targets and remote drivers can be introduced into the testing range:

[0138] Intelligent targets, as interactive physical traffic participants, interact with the virtual traffic flow and affect the tested vehicles.

[0139] The driving simulator supports remote human driver control of the target vehicle, realizing dynamic three-way interaction between the intelligent target object, the algorithm under test, and the background traffic flow in the "human on the loop", enhancing the complexity and randomness of the test scenario.

[0140] 5. Evaluation and Feedback

[0141] During testing, the platform collects key performance indicators of the tested model under different scenarios, including detection accuracy, trajectory prediction error, collision rate, and longitudinal and lateral control stability. If the indicators are lower than the set threshold, the scenario is determined to be a defective scenario, and its features are fed back to S21 for subsequent scenario iteration and target data generation.

[0142] S42 Closed-Loop Dynamic Test

[0143] The closed-loop dynamic testing method proposed in this invention is a process of progressively verifying and providing feedback on autonomous driving algorithms by combining virtual perception injection into real-vehicle testing with scenario difficulty grading and dynamic adjustment mechanisms. This method not only verifies the model's performance in different types of scenarios, but also gradually approaches the algorithm's boundary by dynamically adjusting the testing difficulty, forming an efficient training-testing closed loop.

[0144] 1. Initial test set generation

[0145] Based on the data standardization and scenario difficulty classification results, the current round of test scenario set is defined as follows:

[0146]

[0147] In the formula This indicates the difficulty level of the i-th type of scenario. The test subset below.

[0148] 2. Performance Index Calculation

[0149] In each type of scenario Run the algorithm under test and calculate its performance score in different scenarios according to the definition in S21:

[0150]

[0151] In the formula For the first Key performance indicators, including sensing accuracy: Prediction error Collision rate lateral deviation Longitudinal acceleration stability . Assigning weights to each indicator. .

[0152] 3. Dynamic difficulty adjustment

[0153] In a certain type of scenario Overall rating satisfy:

[0154]

[0155] This increases the testing difficulty for that scenario category. (e.g., from low difficulty L to medium difficulty M);

[0156] like If the scenario is deemed defective, the defect diagnosis process will begin.

[0157] like If the original difficulty is maintained, the test should be repeated to verify stability.

[0158] in The base value is consistent with S21, and dynamically adjusted based on manual experience.

[0159] 4. Closed-loop iterative execution

[0160] After each round of testing, a set of test results for different categories of scenarios is obtained:

[0161]

[0162] like If the training termination condition is triggered, it indicates that the algorithm has reached the expected performance.

[0163] Otherwise, Defect scenario set The data flows back to S21, triggering scenario iteration and targeted data generation, and entering the next training-testing cycle.

[0164] S43 Defect Location Diagnosis

[0165] 1. Defect scenario localization

[0166] Based on the defect scene localization method in the S21 data targeted generation process described in this invention, the defect scene of the current training round of the algorithm under test is located.

[0167] For the set of scenarios that were judged to be defective Further analysis of the indicators reveals: perception indicators, prediction error, collision rate, and control stability, which correspond to defects such as insufficient perception, trajectory prediction, planning safety, and execution control, respectively.

[0168] 2. Result reflux

[0169] For scene set In the defective scenario described in S22, calculate the feature vector based on the scenario:

[0170]

[0171] The data is fed back to S2 for generation and scenario iteration, serving as the basis for the next round of data expansion and generalization, thus achieving a closed loop between testing and training.

[0172] Based on the test results and scene performance scores obtained in S43-1: when the reliability index of defect localization in all scenes is lower than the threshold, the training termination condition is triggered, the training process is terminated, and the algorithm completes the training iteration; otherwise, defect scene localization and attribution analysis are performed.

[0173] Based on the results of multi-round virtual perception injection into real vehicle tests, statistical analysis is performed on the same type of scenarios to identify stability issues of the algorithm in such scenarios.

[0174] By combining iteration logs, defects are attributed to their causes, and the categories of defect sources are identified; repeated tests are conducted on the defect scenarios to verify the test results.

[0175] The defect scenario feature indicators are fed back to the data-targeted generation process for the generation and generalization of relevant scenario data.

[0176] S5 loop execution steps:

[0177] This invention employs a training-testing-diagnosis-generation iterative mechanism to continuously optimize algorithm performance. In each iteration, the platform sequentially completes S1 data standardization, S2 targeted data generation and category reconstruction, S3 ensemble accelerated training, and S4 closed-loop testing and defect diagnosis.

[0178] When the overall performance indicators during the testing phase Satisfy the preset threshold for all scene categories When the system determines that the training has converged, it triggers the termination condition.

[0179] If any scene category exists Then the corresponding defect scenario set The data flows back to S2 to perform data expansion and fine-grained iteration of the scenario.

[0180] This cycle continues until the performance metrics of all test scenarios stabilize within the threshold range. Through this closed-loop mechanism, the present invention enables the adaptive evolution of autonomous driving algorithms in long-tail and complex scenarios, ensuring that model performance gradually converges with each iteration and meets performance requirements.

[0181] Beneficial effects

[0182] Compared with the prior art, the present invention has the following advantages:

[0183] 1. Forming a closed-loop architecture of data reinforcement, accelerated training, and dynamic testing: Unlike the traditional fragmented linear process of "collection, training, and testing", this invention realizes a full-link loop on a unified platform, enabling data generation and algorithm improvement to drive each other and significantly improve iteration efficiency.

[0184] 2. Support for targeted enhancement and scene iteration: Through defect scene identification, targeted data generation and fine-grained scene iteration, this invention can quickly cover long-tail and extreme scenes, avoiding the inefficient method of simply relying on large-scale data collection.

[0185] 3. Accelerate training and testing collaboratively: The platform can dynamically integrate test feedback during the training process, automatically adjust data ratios and parameter optimization strategies, accelerate the correction of weak points in the algorithm, improve algorithm training efficiency, and significantly shorten the R&D cycle.

[0186] 4. Constructing a closed-loop dynamic test for real vehicles: The test method proposed in this invention introduces a dynamic adjustment mechanism for test difficulty while ensuring real vehicle dynamic conditions. This improves test coverage, reduces the risk of testing in real dangerous scenarios, and achieves accurate location of defects in the tested algorithm and closed-loop feedback of results. Attached Figure Description

[0187] Figure 1 This is a flowchart illustrating an integrated training and testing data closed-loop platform and method for autonomous driving algorithms provided by the present invention.

[0188] Figure 2 A schematic diagram illustrating the data-targeted generation and category reconstruction process provided by this invention;

[0189] Figure 3 This is a schematic diagram of the integrated accelerated training process provided by the present invention;

[0190] Figure 4 A schematic diagram of the virtual perception injection real vehicle test architecture provided by this invention; Detailed Implementation

[0191] The technical solution provided in this application will be further described below with reference to specific embodiments and accompanying drawings. The advantages and features of this application will become clearer from the following description.

[0192] Currently, mainstream autonomous driving algorithm training and testing schemes are relatively fragmented. Most schemes use independently organized, large-scale training datasets to train the proposed algorithm once. Then, another set of testing standards is used to verify the algorithm's reliability probabilistically through long-term, long-mileage simulation tests and real-vehicle tests. However, long-tail scenarios have a low probability of occurrence, which may lead to a situation where the failure rate in some dangerous scenarios is high, but the overall success rate data is still excellent, thus masking real problems.

[0193] Furthermore, even when some defective scenarios are discovered, the generalization of training data is mostly compensated by increasing the amount of data collected and filtering. Traditional closed-loop data schemes for training and testing are difficult to achieve efficient iteration of autonomous driving algorithms.

[0194] To address the aforementioned problems, this invention provides an integrated training and testing method for autonomous driving algorithms. It accelerates algorithm training and testing through efficient scene data generalization, enabling efficient iteration of the autonomous driving algorithm and resulting in an autonomous driving algorithm with strong generalization capabilities. Figure 1 As shown, the method includes the following steps:

[0195] S1 model training basic data acquisition:

[0196] The following section uses a typical scenario of "unprotected left turn at an urban intersection" to specifically illustrate the integrated training and testing method for autonomous driving algorithms of the present invention.

[0197] The tested autonomous vehicle acquires environmental information through onboard sensors: camera images LiDAR point cloud and the vehicle's own dynamic state , where represent longitudinal velocity, acceleration, steering angle, yaw rate, and heading angle, respectively. The input also includes a high-precision map. Traffic signal status .

[0198] The model output includes future predicted trajectories. and control commands and risk assessment indicators Therefore, it can be formalized as:

[0199]

[0200] After standardizing and labeling the collected data, the scenes are classified according to road topology, interaction mode, and environmental attributes, and a comprehensive difficulty index is defined:

[0201]

[0202] in, To perceive the difficulty, For topology difficulty, To increase the difficulty of interaction, For weights. According to... The numerical range divides the scene into three difficulty levels: low, medium, and high.

[0203] S2 Data Targeted Generation and Category Reconstruction: (e.g.) Figure 2 )

[0204] S21 Data Targeting Generation

[0205] In the previous S4 testing phase, if a certain scenario category The following set of performance metrics Weighted score:

[0206]

[0207] like Below the threshold Then determine This is a defective scenario.

[0208] To address the deficiencies, the platform generates traffic event disturbances (such as sudden braking and crossing) through traffic simulation, enhances perception samples in low-light / severe weather conditions using GAN and 3DGS, and simulates random lane changes and signal anomalies of background vehicles to generate new sample sets. ,as follows:

[0209]

[0210] In the formula This is a method for generating realistic scenes, using 3DGS generation combined with GAN enhancement. These correspond to road topology, background traffic flow, random behavioral disturbances of participants, and natural environmental elements, respectively. Information is generated by the vehicle's own sensors.

[0211] Road topology information includes topological structure, lane line sets, conflict zone sets, and traffic signals. Background traffic flow includes interacting vehicles, pedestrians, and non-motorized vehicles. Environmental factors consider illumination disturbances, weather disturbances, and object occlusion. The generation process considers vehicle sensor types and specific configuration parameters, including sensor spatial locations and camera and LiDAR calibration data.

[0212] The number of new samples generated through the above process is defined as:

[0213]

[0214] in The generated scene-aware data Compared with the original defect scene perception data Sample size This is the amplification factor, which is generally set to 1 to 2 depending on the severity of the defect.

[0215] S22 Scene Refinement and Reconstruction

[0216] If, after several rounds of training, the performance increment... Less than the preset incremental threshold And continuous If the conditions are not met, a scene iteration is triggered. Feature vector extraction:

[0217]

[0218] Clustering was performed using a Gaussian mixture model.

[0219]

[0220] Hyperparameters are obtained through iterative estimation. The original scene category is subdivided into multiple subcategories. Then, targeted generation is performed on each subclass to expand the training samples.

[0221] S3 integrated accelerated training (e.g.) Figure 3 )

[0222] After the scene data undergoes S1 data standardization and S2 data targeted generation, it enters the integrated accelerated training phase. This phase dynamically adjusts the distribution of training data and training parameters to achieve targeted enhancement of the autonomous driving algorithm for defective scenarios and improve overall convergence speed.

[0223] 1. Data manipulation

[0224] Before the algorithm under test is connected, the user first retrieves the standardized scene data from the standardized database and executes the matching strategy to calculate the scene weights. :

[0225]

[0226] in, Scene category The number of samples. If For scenarios with defects identified in the previous training round, the platform dynamically increases their weight:

[0227]

[0228] in, For adjustment coefficients, if the scenario For defective scenarios Select 1 if the data ratio is 1, otherwise select 0. The dataset after adjusting the data ratio is used in the subsequent training phase to ensure that the defective scenarios are reinforced in the training set.

[0229] 2. Training Management

[0230] During the training phase, scene categories are used. The weights are dynamically adjusted to calculate the training loss function, whereby... Scene category Loss:

[0231]

[0232] This strengthens the impact of defective scenarios on the model during overall training, accelerating optimization and convergence. Simultaneously, the optimal model parameters are saved in real-time based on the validation results in the next step, avoiding overfitting.

[0233] 3. Testing and Evaluation

[0234] During training, the testing and evaluation module calls a standardized scenario library to generate a test set:

[0235]

[0236] Key performance indicators (KPIs) of the model in the perception, prediction, planning, and control stages are calculated, such as perception accuracy (AP), trajectory prediction error (ADE), collision rate, and lateral and longitudinal control stability, and an open-loop evaluation comprehensive score is generated. , The open-loop score for each key indicator sub-item:

[0237]

[0238] like If the scenario is not found, it is marked as a defective scenario and returned to the data management process in Section S3 for the next round of training. This process repeats continuously until... .in These are empirical values ​​in S2. This is the adjustment value used during open-loop verification, typically ranging from 1 to 1.2.

[0239] Through the above process, the integrated accelerated training platform in this embodiment realizes a unified closed loop of "data operation - training management - test evaluation", which enables the algorithm under test to quickly access data reinforcement, complete training optimization, and obtain real-time performance evaluation and defect feedback in each iteration.

[0240] S4 Closed-Loop Testing and Defect Diagnosis:

[0241] After the autonomous driving algorithm under test completes integrated accelerated training, a closed-loop testing and defect diagnosis process integrating virtual and real environments is executed. This process combines virtual simulation with real vehicle testing environments, and through the dynamic interaction of virtual traffic flow, intelligent target objects, and remote driving, it achieves multi-dimensional verification and defect exposure of the algorithm.

[0242] S41 Virtual Perception Injection Real Vehicle Testing Process

[0243] like Figure 4 A schematic diagram of the real-vehicle test architecture for injecting virtual perception;

[0244] In a closed testing ground, a testing environment was built that included real vehicles, intelligent targets, and remote driving control terminals.

[0245] Virtual traffic flow information, including background vehicles, pedestrians, bicycles, and other traffic participants, is generated through a simulation platform. Virtual perception data, such as scarce perception scenarios like low light, rain / snow, and fog / haze, is generated based on scene generation algorithms and sensor models. This virtual perception data is injected into the real vehicle domain controller via an interface and fused with real sensor inputs to form a hybrid input signal, enabling the algorithm to receive "virtual-real" environmental information within the real vehicle control framework.

[0246] To increase scenario complexity and uncertainty, controllable intelligent targets, such as remotely drivable electric dummy cars or simulated pedestrians, are deployed in the test field. Driving simulators control these remote target vehicles to achieve dynamic interactive testing of "humans on a loop." For example, in an unprotected left-turn scenario, remotely driven vehicles can create oncoming traffic interference to verify the decision-making rationality and reaction latency of the algorithm under test.

[0247] The tested autonomous driving algorithm runs in a virtual-real fusion test environment. The platform collects key performance indicators of each module, including perception accuracy and false detection rate; trajectory prediction error; path collision rate; longitudinal and lateral control stability; and decision delay time in real time.

[0248] In the scene collection After executing the virtual perception injection real vehicle test process, the performance is scored according to the S21 comprehensive performance definition:

[0249]

[0250] like This increases the difficulty. If between and Between these thresholds, the scenario is deemed a boundary scene and requires repeated testing; otherwise, it falls below a preset threshold. This type of scenario is identified as a defect. A set of defective scenarios is obtained. The defect types are classified according to the performance degradation stage, including insufficient perception, prediction deviation, planning and decision-making, and control execution defects.

[0251] For defect scenarios, extract their feature vectors. The data is then fed back to the data-targeted generation module S2. In the next iteration, the system prioritizes data generation and generalization for this type of scenario to achieve targeted reinforcement. If the overall scenario score is higher than [a certain value] after two consecutive rounds of testing... Then the algorithm training process ends.

[0252] S5 loop execution steps:

[0253] In this embodiment, the platform achieves continuous optimization of the autonomous driving algorithm through a cyclical iterative mechanism of "training-testing-diagnosis-generation".

[0254] In the initial stage, the system forms the first training set based on the S1 data standardization and scenario classification results; the initial model training is completed through the S3 integrated accelerated training platform to obtain the initial version of the algorithm model.

[0255] After the initial training is completed, the virtual-real fusion closed-loop test procedure described in S4 is executed. If the algorithm's performance score in any scene category is lower than the threshold... If a scenario meets the threshold requirement, it is marked as a defective scenario, and the S2 data targeting and scenario generalization process is triggered; if all scenarios meet the threshold requirement, the algorithm training terminates.

[0256] Next, the data generation and retraining phase begins. The defect scene features are fed back to the S2 module, where the scene generation model performs targeted data generation and feature expansion. The newly generated sample data is then fused with the original dataset and re-entered into the S3 training platform for accelerated training. During training, the platform automatically adjusts the defect scene weights and dynamically updates the training parameters and loss function based on test feedback.

[0257] The system repeatedly executes the "training-testing-diagnostics-generation" loop until the performance metrics for each scenario category are stably met:

[0258]

[0259] This means that all test scenarios pass the preset threshold standard. At this point, the platform triggers the training termination flag and outputs the final converged algorithm model.

[0260] The above description is merely a description of preferred embodiments of this application and is not intended to limit the scope of this application in any way. Any changes or modifications made by those skilled in the art based on the above-disclosed technical content should be considered as equivalent and valid embodiments and fall within the scope of protection of the technical solution of this application.

Claims

1. An integrated training test method for an automatic driving algorithm, characterized in that, Comprising the following steps: S1 data standardization; Data standardization is performed for open-source datasets from different sources, and the input-output structure of the training data is unified; then a multi-dimensional scene classification system is established for the annotated road traffic data; S2 data target generation and category reconstruction; First, determine the defect scene of data target generation; for the determined defect scene, use the scene generation model to perform data target generation to improve the training sample quantity under the defect scene; Use the scene refinement reconstruction iterative method to refine the scene category granularity, enhance the data target reinforcement and algorithm training effect; S3 integrated accelerated training; After the scene data generation is completed, the algorithm is accelerated for training; S4 closed-loop testing and defect diagnosis; Based on the closed-loop testing and defect diagnosis method, the automatic driving algorithm is dynamically tested in different scenes and difficulties, the algorithm defects are diagnosed, and the algorithm iterative training is used; S5 execute steps S1-S4 in a loop to continuously optimize the algorithm performance; Step S2 is specifically: S21 data target generation First, determine the defect scene of data target generation, and perform subsequent data generation under this target scene category; Define the "defect scene" as: in the automatic driving algorithm test process, the key performance indicators of the algorithm running in a certain scene do not reach the preset threshold, and the scene triggers emergency obstacle avoidance or is taken over by the driver; The key performance indicators include: (1) Perception layer indicators: target detection accuracy, target tracking accuracy, false positive / negative rate; (2) Prediction layer indicators: average trajectory prediction error, final position error; (3) Planning and control layer indicators: collision rate, path deviation rate, braking / steering stability; Define the scene performance score: Wherein, the scene performance score of the defect scene is obtained from the S4 closed-loop test and defect diagnosis of the last round, and the default scene set is directly used in the initial round to enter the training link; ， wherein is the first normalized score of the item indicator, is the weight; If the algorithm is in the category Difficulty Scene Overall score satisfy: If the scene category is determined to be a defective scene; if If it is, then it is determined to be a boundary scenario and further repeated testing is required; if If the scenario is not a defective scenario, then it is determined that the scenario is not a defective scenario; where and The preset performance threshold; The difficulty level is defined in S12; Next, for the determined defect scene, based on the scene feature dimension, use the scene generation model to perform data target generation based on the defect scene diagnosis result of the last round to improve the training sample quantity under the defect scene; The three elements of road topology, interaction mode and environmental attribute of the traffic scene are corresponded to the three dimensions of road topology, traffic event and environmental perception, and behavior disturbance is added to increase the scene data generalization, and data target generation is carried out: The number of new samples generated through the above process is defined as: (1) Road topology dimension: Road topology The spatial environment for the occurrence of traffic scenarios, the basic spatial constraints for the interaction between the measured subject and the traffic flow object, including road boundaries, lane division and intersection, and traffic flow direction; (2) Traffic event dimension: Road topology under defect scenarios with background traffic flow condition, automatically construct dynamic events interacting with the measured algorithm through the traffic simulation platform; (3) Behavior disturbance dimension: For the defect scene, the disturbance of traffic participants is introduced to model the more difficult interaction data; ​ (4) Perception dimension: add natural environment perception elements to the interaction scene Based on the arrangement of the self-vehicle sensors and the spatial position, the 3DGS method is used to generate visual perception data, and the GAN method is used to enhance the authenticity of the defective scene. A set of perception data The generation process of the perception data is expressed as: In the formula A method for generating a real scene, Corresponding to the road topology, background traffic flow, participant random behavior disturbance and natural environment elements, The information of the vehicle's own sensors S22 scene refinement reconstruction wherein are generated scene-aware data are original defect scene-aware data is the sample size, is a magnification factor, dynamically set according to defect severity; Finally, the generated defect scene extension samples are merged with the original scene data to form the training set input of the current round, the target reinforcement of the defect scene training data of the algorithm under test is realized, the target generation result is converted into new data samples, and the new data samples are returned to the training link. While performing data target generation, the scene category granularity is refined through the scene refinement reconstruction iterative method to enhance the data target reinforcement and algorithm training effect; Next, scene feature extraction and clustering are performed; for the scene categories that trigger iteration, scene feature vectors are extracted: Firstly, the scene refinement reconstruction trigger condition is: let a scene and difficulty category The performance index after the tth round of training is The performance index of the last round is If the following condition is met: And the performance index has not exceeded the threshold value for consecutive rounds , the scene iteration process is triggered, wherein is a performance increment threshold value, is a minimum number of consecutive detection rounds; After scene iteration processing, the original defect scene is refined into sub-scenes that cover more comprehensive features, so that the algorithm can perform reinforcement learning for different fine-grained features. wherein is a number of scene targets, is a lighting parameter, is a weather condition, is a road topology index, is an interaction intensity; The feature vector is inputted into a clustering algorithm based on Gaussian Mixture Model (GMM), assuming that the sample distribution is a weighted combination of Gaussian distributions: Estimating hyperparameters by iteration , respectively corresponding to the weight, mean vector and covariance matrix of the first Gaussian component, thereby subdividing the original scene class into multiple subclasses ; After obtaining the new subclass Then, a data targeting generation step S21 is performed to obtain an extended sample set and is fused with the existing training set to form a new round of iteration training data; Step S1 is specifically: 2.The method of claim 1, wherein, S11 data standardization processing: First, data standardization processing is performed for open-source datasets from different sources, and the input-output structure of the training data is unified; ​ The input data includes vehicle perception data, vehicle state information, and map and environment information; The output data is a planning and control command of the vehicle in a future time domain, specifically including a predicted trajectory, a control command, and a risk assessment index; S12 scene classification and grading The traffic scene is divided into three elements: environmental attributes, road topology, and interaction mode. The scene is classified according to the above elements to achieve fine-grained positioning of the target scene data. On the basis of classification, define the comprehensive difficulty index for measuring the challenge of the scene to the automatic driving algorithm, as follows: wherein, is a perception difficulty; is a topology difficulty; is an interaction difficulty; is a weight parameter; According to the numerical interval of the comprehensive difficulty index , the scene difficulty comprehensive level is divided, which is used to ensure the comparability of the difficulty level division under the condition of multi-source data: Low difficulty L: ; Medium difficulty M: ; High difficulty H: ; wherein the threshold value According to the statistical distribution characteristics of different data sources, it is determined by artificial experience. 3.The method of claim 1, wherein, Step S3 is specifically: S31 training data management First, based on the scene data described in S1 / S2, the following process is used to prepare training data: (1) Access standardized scene data: read training data from the standardized database formed in step S1, integrate data sets from different sources, and store different sources of data in a unified format to ensure uniform input structure; (2) Dynamic data matching: based on the feedback results of the last round of training evaluation and the scene category distribution in the data, automatically match data of different difficulty and category to obtain the data set used in this round of training; S32 algorithm training parameter optimization After each round of training data preparation and matching, the training parameters are set based on the following process and the model is trained: (1) Access the algorithm model to be tested: access the tested autonomous driving algorithm through a standard interface and initialize the model parameters; (2) Training process execution: call the training set matched in S31 for batch training and model parameter iteration; (3) Dynamic parameter adjustment: automatically adjust the learning rate and loss function weight based on the feedback results of the last round S33 evaluation, strengthen the learning effect of defect scenes, and shorten the model convergence time; S33 round-to-round test evaluation Intermediate round open-loop test verification is performed during the training process to determine whether the algorithm training has converged and to quantitatively evaluate the targeted training data. The steps include: (1) Automatically generate a test set: call the standardized scene data obtained from S31 training data management during the training process, and generate a round-to-round test scene set according to the difficulty level; (2) Index calculation: quantitatively evaluate the performance of the model in perception, prediction, planning, and control, and calculate key indicators including perception accuracy, prediction error, collision rate, and control stability; (3) Defect identification: if the comprehensive score of a certain type of scene is lower than a preset threshold, it is marked as a defect scene, and the test results are fed back to S31; S34 result feedback and data backflow After each round of data matching-parameter adjustment-model training-round-to-round testing is completed, the test results generated in S33 are fed back to S31 to form new training data input for the next round of integrated accelerated training, realizing the cycle of training-evaluation-feedback; The whole process is repeated until the model meets the preset performance threshold in S33 testing, and the integrated accelerated training process ends, entering the S4 closed-loop test and defect diagnosis process.

4. The method of claim 1, wherein the method is integrated training and testing of an autonomous driving algorithm. Step S4 is specifically: S41 virtual perception injection real vehicle test method The virtual generated perception information is combined with the real vehicle state, so that the measured automatic driving system can accept virtual environment input in the real vehicle control framework, thereby covering more long-tail scenarios and extreme cases while ensuring safety. Specifically as follows: (1) Method definition Virtual perception injection real vehicle test refers to injecting the perception information generated by the virtual simulation platform into the domain controller of the vehicle in the real vehicle running environment, so that the sensor input received by the automatic driving algorithm is a combination of real sensor data and virtual generated data, and then the closed-loop decision and execution under real vehicle conditions are completed; (2) Input construction Virtual traffic flow injection: based on the road topology and traffic elements of the annotated scene, interactive virtual traffic participants are generated in the simulation platform, and their motion trajectories and interaction behaviors are constructed; Virtual sensor perception injection: virtual perception data is constructed through sensor modeling and image / point cloud generation methods; Environmental disturbance injection: introduce sudden disturbances in the scene to improve test intensity; (3) Real vehicle running and control In a closed test site, the real vehicle carries a domain controller to receive mixed perception input and execute subsequent prediction, planning and control decision-making processes; (4) Human-computer interaction enhancement Introduce intelligent target objects and remote drivers in the test field. Intelligent target objects act as interactive physical traffic participants and interact with virtual traffic flow to affect the measured vehicle. The driving simulator supports remote drivers to control the target vehicle, realizing the dynamic interaction of intelligent target objects-measured algorithm-background traffic flow in the "man-in-the-loop" mode, enhancing the complexity and randomness of the test scene; (5) Evaluation and feedback During the test, the platform collects key performance indicators of the measured model under different scenes. If the indicators are below the set threshold, the scene is determined to be a defect scene, and its features are fed back to S21 for subsequent scene iteration and targeted data generation; S42 Closed-loop dynamic testing The closed-loop dynamic testing method is a process of verifying and feeding back the automatic driving algorithm step by step based on virtual perception injection real vehicle testing, combined with scene difficulty grading and dynamic adjustment mechanism. Specifically as follows: (1) Initial test set generation According to the data standardization and scene difficulty grading results, define the current round test scene set: wherein represents the test subset for the i-th scenario at difficulty level under. (2) Performance indicator calculation In each type of scenario The tested algorithm is run under each scenario, and the algorithm performance score is calculated according to S21 definition: In the formula For the first Key performance indicators, including sensing accuracy: Prediction error Collision rate lateral deviation Longitudinal acceleration stability ; Assigning weights to each indicator. ; (3) Dynamic difficulty adjustment If the comprehensive score under certain scenarios satisfies: ​ then the test difficulty of the scene class is raised ; If , the scene is determined as a defect scene, and the defect diagnosis link is entered. If then maintain original difficulty and repeat test to verify stability; wherein The base value is consistent with S21, and the dynamic adjustment is calibrated according to artificial experience. (4) Closed-loop iteration execution After each round of testing, a set of test results under different categories of scenes is obtained: If then the training termination condition is triggered and the algorithm is judged to have reached the desired performance; Otherwise, set the defect scenario set in back to S21, trigger scenario iteration and targeted data generation, and enter the next round of training-test cycle; S43 Defect positioning and diagnosis (1) Defect scene positioning Position the defect scene of the measured algorithm in the current training round; a scene set determined to be defective , and performing index split analysis: perception index, prediction error, collision rate, control stability, corresponding to perception deficiency defects, trajectory prediction defects, planning safety defects and execution control defects, respectively; (2) Result backflow For defect scenes in the scene set compute a feature vector based on the scene described in S22. Backflow to S2 data generation and scene iteration as the basis for data expansion and generalization in the next round, realizing the closed loop of testing and training; According to the obtained test results and scene performance scores, when the positioning confidence indicators of all scene defects are below the threshold, the training termination condition is triggered, and the training process is terminated, and the algorithm completes the training iteration; Otherwise, perform defect scene positioning and attribution analysis; Based on the results of multiple rounds of virtual perception injection real vehicle testing, statistical analysis is performed on the same category of scenes to identify the stability problems of the algorithm in that category of scenes; Combine the iteration log to attribute the defects and identify the source category of the defects; Repeat testing for defect scenarios, verify test results; Reflux defect scenario feature indicators to data targeting generation processes for related scenario data generation and generalization.

Images