A Performance Boundary Adaptive Sampling Method for an Autonomous Driving System
Through the performance boundary adaptive sampling method, performance boundary sampling points are generated using brainstorm optimization algorithm and classification algorithm, which solves the problem of performance testing of autonomous driving systems in complex scenarios, improves testing efficiency and safety, and achieves performance boundary sampling with high coverage and accuracy.
Patent Information
- Application Number
- CN202211293693.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-21
- Publication Date
- 2025-07-01
- Estimated Expiration
- 2042-10-21
AI Technical Summary
It is difficult for autonomous driving systems to effectively conduct performance testing in complex real driving scenarios, resulting in safety hazards and unpredictable behaviors. The existing algorithms cannot accurately process high-dimensional large-scale data, resulting in long data acquisition cycles and low information volume.
Adaptive sampling method for performance boundary is adopted to optimize the classification results and classification efficiency of samples by generating scene state space vectors, evaluating performance indicators of autonomous driving system, iteratively generating new scene input state vectors, identifying performance patterns, and using a classification algorithm based on brainstorm optimization algorithm to generate performance boundary sampling points to optimize the classification results and classification efficiency of samples.
It achieves the maximum coverage and accuracy of boundary sampling with fewer samples, improves the efficiency and safety guarantee of autonomous driving system testing, reduces the data acquisition cycle and increases the amount of data information.
Smart Images

Figure CN115565267B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of autonomous driving, and more particularly to a method for adaptively sampling the performance boundary of an autonomous driving system. Background Art
[0002] An autonomous intelligent system is different from traditional artificial intelligence applications. It does not follow the process of data input, feature extraction, feature selection, logical reasoning, and prediction. Instead, it is a knowledge base established by computer learning of model parameters accumulated over a long time, so as to generate advanced cognitive results. There is a "black box" that people cannot understand between the data input by the autonomous intelligent system and the decisions it outputs. It is difficult for operators to understand the decisions made by the system in complex environments. Therefore, the decision results of autonomous systems are uncertain, and uncertainty brings unpredictability. Difficulty in prediction means difficulty in prevention, which may bring a dangerous situation that harms humans.
[0003] In recent years, autonomous intelligent systems have been widely applied to the software and hardware of artificial intelligence, showing vigorous vitality in the field of autonomous intelligent driving technology and demonstrating the deterministic value in uncertain times. Companies such as Google, Tesla, Baidu, Tencent, Uber, and Zoox have successively invested a lot of energy in this field, bringing the rapid development of the driverless industry and also generating a large number of problems. And the safety issue is currently the biggest problem in the application field of autonomous intelligent driving technology. In March 2018, an autonomous vehicle operated by Uber hit and killed a woman in Tempe, Arizona, USA. At the time of the accident, the vehicle was operating in autonomous driving mode, but the driver was in the car. After relevant investigation and analysis, the autonomous vehicle "saw" the woman but did not brake, and at the same time, the autonomous driving system did not generate a fault warning message. In fact, the autonomous driving system first identified the victim as an unknown object, then as a vehicle, and then as a bicycle. The autonomous driving system did not take any action within 6 to 13 seconds before the collision and only requested emergency braking 1.3 seconds before the collision. In addition to the problems caused by the black box characteristics, the autonomous driving system also faces other serious problems, including attacks on the vulnerability of artificial intelligence faced by the autonomous driving recognition module, decision-making module, etc. Therefore, there is no intelligent driving without safety, and safety is the key to determining the future development of intelligent driving.
[0004] Given the safety hazards faced by intelligent driving, intelligent driving testing has become an essential part of intelligent driving applications. Intelligent driving testing is used to verify and evaluate the reliability of technologies, enabling safer and more efficient driverless driving. The research difficulty in the performance testing of autonomous driving systems lies in how to analyze dynamic high-dimensional large-scale data. The model for autonomous driving system performance testing needs to support fault injection, automated testing, and report generation. Subsequently, it also needs to support the generalization of test cases. According to the requirements for the evaluation of autonomous driving systems, a high-precision scenario test dataset with traffic agents having complex, realistic, and free behaviors and high-fidelity physical characteristics should be generated. Continuously sample and evaluate the test set, and finally output the system evaluation results. The evaluation based on scenario test data can be modeled as a high-dimensional large-scale data analysis problem. System evaluation needs to consider the characteristics of large-scale optimization problems, data-driven optimization problems, and computationally expensive problems, establish a surrogate model, and build a set of performance boundaries of the system based on a small amount of sampled data to address the limitations of road testing.
[0005] Currently, for autonomous vehicles to play a perceptual role in rapidly changing complex real-world driving scenarios, a vast amount of road scene data needs to be annotated by professionals behind the scenes, thus being transformed into data support for the test system. Testing the task completion ability of autonomous systems such as autonomous driving usually requires monitoring specified system objectives, such as performance objectives (e.g., accuracy, precision, and recall), and ensuring that no data bias is introduced into the system. The result of this test may be retraining the system with an updated test dataset. A test scenario can be regarded as a single sample of the entire test space. An immediate problem the system encounters is that when attempting to simulate real tasks, the number of parameters in the test space increases rapidly. As the tasks and environments become more complex, the number of parameters in the test space is too large, leading to the well-known curse of dimensionality. The number of samples will thus be severely restricted. Therefore, how to reduce the data collection cycle and increase the data information content is a key factor in accelerating the landing and iteration cycle of intelligent driving-related applications, saving R & D time and costs, and is also of great significance for accelerating the scenario-based implementation, safe landing, improvement of user experience, and safe driving of the autonomous driving industry. From a macroscopic perspective, existing algorithms cannot accurately handle the infinitely possible long-tail scenarios in complex traffic environments, making the coverage of high-quality scenario data even more important. Specifically in terms of the landing requirements of autonomous driving, high-quality scenario data has also become the key to leading the competition in various businesses.
[0006] For realistic test scenarios, the dimensionality of the configuration space and the computational cost of high-fidelity simulation rule out exhaustive or uniform sampling. Therefore, we must carefully select the scenarios to be simulated, and generally speaking, the performance boundaries have the maximum information content.
[0007] An embodiment of the present invention provides a method for adaptively sampling the performance boundary of an autonomous driving system, including:
[0008] According to different scenario states of autonomous driving tests, configure and generate a scenario state space vector, and construct a scenario input state vector based on one of the scenario state space vectors;
[0009] Evaluate the performance metrics of the autonomous driving system under different scenarios, obtain a score vector, and construct a score sample set according to the score vector;
[0010] According to the generated scenario input state vector, use an adaptive search algorithm to iteratively generate a new scenario input state vector and identify performance patterns;
[0011] Input the new scenario input state vector into the autonomous driving system simulation model for simulation, generate a score vector, and construct a labeled sample set according to the generated scenario input state vector and score vector;
[0012] Adopt a classification algorithm based on the brainstorm optimization algorithm to obtain the labeled sample sets under different performance patterns;
[0013] Construct performance boundary pairs according to the labeled sample sets under different performance patterns, and construct a performance boundary set according to the performance boundary pairs;
[0014] Execute the brainstorm optimization algorithm on the labeled samples of different performance patterns to generate performance boundary sampling points, including:
[0015] Probability parameters And probability parameters Are respectively used to control the generation of new samples in the case of one cluster or two clusters. The expressions for generating new samples include formulas (1) and (2):
[0016] ; (1)
[0017] = ; (2)
[0018] Where, Is the d-th dimension of That is selected to generate a new individual; And Are samples of adjacent performance patterns; Is the d-th dimension of the newly generated individual ; Is the d-th dimension of the selected better newly generated individual ; Is a Gaussian random function with a mean of And a variance of ; is a step function and also a Gaussian random function is the coefficient of is a random number between
[0019] The width of the boundary region after generating the new sample is within the range, that is < , and now it is added to the boundary set and update the value of, that is ;
[0020] Iterate in combination with the optimized classification algorithm based on the brainstorm optimization algorithm to generate the final performance boundary.
[0021] Preferably, the different scenario states of the autonomous driving test include:
[0022] Autonomous driving environment;
[0023] Autonomous driving task environment;
[0024] Parameter configuration of the autonomous vehicle.
[0025] Preferably, configuring and generating a scenario state space vector according to the different scenario states of the autonomous driving test, and constructing a scenario input state vector according to one of the scenario state space vectors, includes:
[0026] State space = , which contains n elements, and each element represents a value of one of a series of multiple variables representing the environment, task or parameters of the autonomous vehicle;
[0027] After instantiating each element within the state space range, it is passed to each scenario input state vector of the simulation = , where i , and respectively represent and instantiated elements of.
[0028] Preferably, evaluating the performance metrics of the autonomous driving system to obtain a score vector, and constructing a score sample set of the score vector, includes:
[0029] Evaluating the performance metrics of the autonomous driving system;
[0030] Among them, the performance metrics include: task completion, number of safety violations, number of waypoints reached, fuel consumption;
[0031] Output a score vector according to the performance metrics = ;
[0032] Conduct N experiments to obtain a score sample set composed of N score vectors = .
[0033] Preferably, the expression of the adaptive search algorithm includes:
[0034]
[0035] Among them, F is the simulation model function of the autonomous driving system, which accepts a set of N scene input state vectors = ... and returns a set of N score vector sample sets = , and the marked samples output by the adaptive search function are a set of . of marked samples.
[0036] Preferably, the performance modes include:
[0037] Complete success TS of reaching two waypoints and successfully identifying and avoiding fixed and moving obstacles;
[0038] Safe success MS of only reaching the recovery waypoint and successfully identifying and avoiding fixed and moving obstacles;
[0039] Task success SS of only reaching the recovery waypoint and failing to successfully identify and avoid fixed and moving obstacles;
[0040] Complete failure of not reaching any waypoint and failing to successfully identify and avoid fixed and moving obstacles TF.
[0041] Preferably, the optimization classification algorithm of the brainstorm optimization algorithm includes:
[0042] Take the k of the KNN algorithm and the number of training samples for each category as the optimization decision variables;
[0043] Define the preliminary optimization objective function:
[0044]
[0045] Among them , m is the total number of classification categories, is the number of training samples for category ;
[0046] Take different values of k and as solutions, and take the classification error rate as the function value;
[0047] Apply the brainstorm optimization algorithm to the setting of k and to find the optimal combination of k and .
[0048] The embodiment of the present invention provides a performance boundary adaptive sampling method for an autonomous driving system. Compared with the prior art, its beneficial effects are as follows:
[0049] Applying the BSO algorithm to the performance boundary sampling problem to generate new samples can maximize the coverage and accuracy of boundary sampling with fewer samples. The present invention combines the BSO algorithm and the classification algorithm to optimize the selection of the classification algorithm and the data training set, which can improve the classification efficiency and the classification accuracy rate. At the same time, applying the BSO algorithm to the performance boundary sampling problem to generate new samples can maximize the coverage and accuracy of boundary sampling with fewer samples. Description of the Drawings
[0050] Figure 1 is the overall flowchart of a performance boundary adaptive sampling method for an autonomous driving system provided by an embodiment of the present invention;
[0051] Figure 2 is the flowchart of the improved brainstorm optimization algorithm for a performance boundary adaptive sampling method for an autonomous driving system provided by an embodiment of the present invention;
[0052] Figure 3 is the performance mode classification of a performance boundary adaptive sampling method for an autonomous driving system provided by an embodiment of the present invention, where (a) represents the TS / MS boundary pair and (b) represents the TF / SS boundary pair;
[0053] Figure 4 is a simple test scenario of an autonomous driving vehicle for a performance boundary adaptive sampling method for an autonomous driving system provided by an embodiment of the present invention. Detailed Embodiment
[0054] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0055] See Figures 1 to 4, the present invention provides an efficient performance boundary sampling method based on the BSO algorithm to solve problems such as the massive data and high-dimensional characteristics of the configuration space in autonomous driving test scenarios, the high cost of simulation calculations, and the severely limited number of sampling samples. These problem characteristics affect the test efficiency of the autonomous driving system and reduce the safety guarantee of the autonomous driving system. The present invention uses the BSO algorithm to improve performance indicators such as the accuracy and recall rate of performance boundary sampling. Determine the evaluation requirements, use the BSO algorithm to iteratively sample the system performance boundary, generate multiple groups of samples, continuously approach the area where the performance boundary exists with a high probability, and identify and evaluate the performance boundary. Return the set of sample points where the performance boundary is found to facilitate the efficient testing of the autonomous driving system.
[0056] S1: Scenario description, set the test scenario state space vector and the test scenario input state vector. The state space is defined by a set of parameter configuration files (referred to as state space files) that describe the environmental settings, task elements, and autonomous driving vehicles in the environment. These settings include the ranges of different simulation elements, such as the time of day, the number and location of obstacles, different task types, etc. The number of variable simulation elements in the state space constitutes its dimension. The state space = , which contains n elements. Each element in the state space vector represents a variable in the environment, task, or autonomous driving vehicle parameters with a series of possible values (obstacle position, time window, task priority, etc.). The various scenario input state vectors passed to the simulation = , where i , and respectively represent and instantiated elements, and is created based on the specific instantiation of each element within their respective state space ranges. For example, define the task of the autonomous driving vehicle to be tested: reach the recovery waypoint after passing the target waypoint, and identify and avoid fixed and moving obstacles during the process. Define the state space = , which contains 5 elements. In the state space vector represents the position where obstacles are placed in the test space, then represents the position of fixed obstacle 1, represents the position of fixed obstacle 2, represents the position of moving obstacle 1, represents the task priority including task priority, safety priority, speed priority, etc., represents the time window including time limit and recorded time, etc., It is indicated that environmental factors include visibility, temperature, humidity, wind direction magnitude, etc., It is indicated that the parameters of the autonomous vehicle include the steering rotation speed of the steering wheel, the driving speed of the vehicle, the hardware configuration, etc. Based on the state space range of each element, the input state vectors of each scenario passed to the simulation are instantiated and created. = , and the scenario shown in Figure 3 is obtained.
[0057] S2: Metric design, set the score space vector of the test scenario, and the metrics for scoring the system tasks and performance modes based on externally observable attributes. Output the score vector = . The sample set of N score vectors is defined as = . Each individual in the score vector represents a performance metric, and these metrics include discrete metrics such as task completion, the number of safety violations, the number of waypoints reached, etc., or continuous metrics such as fuel consumption. Through these metrics, the intelligence of the autonomous driving system can be evaluated. For example, define the task of the autonomous vehicle to be tested as passing through the target waypoint and reaching the recovery waypoint, and during the process, identify and avoid fixed and moving obstacles; at the same time, establish 4 performance modes: total success (TS) of reaching two waypoints and successfully identifying and avoiding fixed and moving obstacles, safety success (MS) of only reaching the recovery waypoint and successfully identifying and avoiding fixed and moving obstacles, task success (SS) of only reaching the recovery waypoint and failing to successfully identify and avoid fixed and moving obstacles, and total failure (TF) of not reaching any waypoint and failing to successfully identify and avoid fixed and moving obstacles, as Figure 4 shown. Output the score vector = . In the output score vector represents the score of whether the target waypoint is successfully reached. If it is successfully reached, it is 25; if it is not successfully reached, it is 0. represents the score of whether the recovery waypoint is successfully reached. If it is successfully reached, it is 15; if it is not successfully reached, it is 0. represents the score of whether a fixed obstacle is successfully identified. If it is successfully identified, it is 15; if it is not successfully identified, it is 0. represents the score of whether a moving obstacle is successfully identified. If it is successfully identified, it is 20; if it is not successfully identified, it is 0. represents the score of whether a fixed obstacle is successfully avoided. If it is successfully avoided, it is 20; if it is not successfully avoided, it is 0. The score indicating whether the moving obstacle is successfully avoided is 20 for successful avoidance and 0 for unsuccessful avoidance. In the specific case in the figure below, if both fixed and moving obstacles can be correctly identified, the score for complete success (TS) is 150, the score for safe success (SS) is 125, the score for mission success (MS) is 105, and the score for complete failure (TF) is 90. The sample set of N score vectors obtained after N experiments is defined as
[0058] =[ ].
[0059] S3: Dataset generation, building a simulation model for the autonomous driving system, F( ) = , which accepts a set of N scene input state vectors =[ ... ] and returns a set of N score vector samples =[ The simulation framework manages the initiation of the run and the parsing of the results. The target system simulation framework performs the simulation of the tasks described in the state space file. It takes the scenario state in the test generation software as input, and after the simulation is completed, the results are scored and returned to the test generation software.
[0060] Data sampling, the adaptive search algorithm iteratively generates new scenario states to run in simulation based on previous results. Run all submitted scenarios, identify performance patterns, and sort the test scenarios according to their distance from the performance boundary. Define the adaptive search function as follows:
[0061]
[0062] Where F is the dataset generation function, which accepts a set of N scene input state vectors =[ ... ] and returns a set of N score vector samples =[ ], adapt the search function Output is a group ] are labeled samples.
[0063] S3.1: By determining the boundary function:
[0064]
[0065] Among them, the output B=[ ] represents the set of boundary pairs of different types of performance transformations, L is the number of identified performance patterns, and N is The number of samples in. Each boundary is the performance mode and is the boundary sample set of, for example, a set of examples that may be on the boundary between a completed task and a failed task, while another set may contain examples on the boundary between a successful return and an unsuccessful return 。
[0066] S3.2: Evaluation process. The optimization goal is to generate a set of samples to define the region where the performance boundary appears with the highest possible resolution. The brainstorming algorithm can "refine" the search region through clustering operations. After multiple iterations, all solutions are likely to be clustered into a small search region with a high probability. New solutions are generated by mutating the cluster center or other existing solutions, thereby controlling the development region of the algorithm. Therefore, use the brainstorming algorithm to generate a set of performance boundary samples , and the evaluation function is to maximize the volume of the sampled boundary region ( ) for all boundaries in the set of boundary pairs B to find the minimum possible value of the width of the boundary region .
[0067] S3.3: Classify the samples according to the performance mode, identify the performance mode in the score space and use KNN to classify the samples. Once the samples are classified according to their performance mode, they group together adjacent sets of samples
[0068] During classification, the test data is divided into training samples and test samples. For the problem of sampling the performance boundary of an autonomous driving system, there is a large amount of test scenario data in the test space of the samples, which will lead to low classification efficiency and classification accuracy. For multi-classification problems, for each class, selecting an appropriate number of samples to enter the training sample set is an effective means to improve classification efficiency and classification accuracy. The following uses the basic KNN algorithm as an example for illustration
[0069] For each classification category, it is difficult to set appropriate sample numbers and optimization parameters. For the KNN algorithm, take k and the number of training samples for each class as the optimization decision variables, and set the preliminary optimization objective function:
[0070]
[0071] where , m is the total number of classification categories, is the number of training samples for class . Take different values of k and as solutions, and take the classification error rate as the function value. Apply the BSO algorithm to k and The setting can quickly find the optimal combination of k and, improving the classification accuracy and efficiency.
[0072] Apply the combined algorithm based on the BSO algorithm and the optimized classification algorithm to the performance boundary sampling problem to classify the performance patterns and optimize the classification results and efficiency of the samples.
[0073] This method is also applicable to other classification algorithms with test samples and classification samples, such as k the weighted nearest neighbor (k-weighted Nearest Neighbor, KWNN) algorithm, etc.
[0074] S3.4: After classifying the samples according to the performance patterns, form the boundary by performing pairwise comparisons between each performance pattern with different performance patterns. Use the KNN detection algorithm to determine the nearest neighbor of each sample with different performance patterns. Whether there are samples within the range, the nearest neighbor distances under different performance patterns will be added to the final boundary set, that is < . Then, construct the final boundary set according to the definition = , where a and b represent the performance patterns and . The boundary pair , i is composed of the points in the sampling set and satisfies: : ,| - |
[0075] Form the boundary by generating new sampling points through mutation between each performance pattern with different performance patterns. The probability parameter and the probability parameter are respectively used to control the generation of new samples in the case of one cluster or two clusters. The expressions for generating new samples include formula (1) and formula (2):
[0076] (1)
[0077] = (2)
[0078] Among them, is the d-th dimension of selected to generate a new individual; and are samples of adjacent performance patterns; is the newly generated individual at the d-th dimension; is the selected and better newly generated individual at the d-th dimension; is a Gaussian random function with a mean of and a variance of ; is a step size function and also a Gaussian random function as the coefficient; is a random number between;
[0079] The width of the boundary region after generating the new sample is within the range, that is < , and now it is added to the boundary set and update the value, that is .
[0080] S4: Generate a new sample and add it to the boundary set , if the iteration has not ended, initialize the existing boundary set and continue to iterate through steps S3.3 and S3.4 until the iteration ends. As the number of required samples increases, continuously approach the minimum possible value of the width of the boundary region. After the iteration ends, construct the final boundary set = .
[0081] Applying the BSO algorithm to the performance boundary sampling problem to generate new samples can achieve maximizing the coverage and accuracy of boundary sampling with fewer samples.
[0082] This method is also applicable to other boundary sampling problems of autonomous intelligent systems, such as military war simulations, aerospace systems, etc.
[0083] The above-disclosed are only several specific embodiments of the present invention. Those skilled in the art can make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the present invention. However, the embodiments of the present invention are not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of the present invention.
Claims
1. A performance boundary adaptive sampling method for an autonomous driving system, characterized in that, Including: Configuring and generating a scenario state space vector according to different scenario states of an autonomous driving test, and constructing a scenario input state vector according to one of the scenario state space vectors; Evaluating the performance metrics of the autonomous driving system in different scenarios to obtain a score vector, and constructing a score sample set according to the score vector; According to the generated scenario input state vector, using an adaptive search algorithm to iteratively generate a new scenario input state vector and identify performance patterns; Inputting the new scenario input state vector into the autonomous driving system simulation model for simulation to generate a score vector, and constructing a labeled sample set according to the generated scenario input state vector and score vector; Using a classification algorithm based on the brainstorm optimization algorithm to obtain labeled sample sets under different performance patterns; Constructing performance boundary pairs according to the labeled sample sets under different performance patterns, and constructing a performance boundary set according to the performance boundary pairs; Performing the brainstorm optimization algorithm on the labeled samples of different performance patterns to generate performance boundary sampling points, including: Probability parameter and the probability parameter are respectively used to control the generation of new samples in the case of one cluster or two clusters. The expressions for generating new samples include Formula (1) and Formula (2): ; (1) = ;(2) Among them, is the d-th dimension of selected to generate a new individual; and are samples of adjacent performance patterns; is the d-th dimension of the newly generated individual ; is the d-th dimension of the selected better newly generated individual ; is a Gaussian random function with a mean of and a variance of ; is a step function and also the coefficient of the Gaussian random function ; is a random number between; The width of the boundary region after the new sample is generated is within the range, that is < , and now it is added to the boundary set , update the value of, that is ; Combining with the optimized classification algorithm based on the brainstorm optimization algorithm for iteration to generate the final performance boundary.
2. The performance boundary adaptive sampling method for an autonomous driving system according to claim 1, wherein The different scenario states of the autonomous driving test include: Autonomous driving environment; Autonomous driving task environment; Parameter configuration of the autonomous driving vehicle.
3. The performance boundary adaptive sampling method of an autonomous driving system according to claim 1, characterized in that The configuring and generating a scenario state space vector according to different scenario states of the autonomous driving test, and constructing a scenario input state vector according to one of the scenario state space vectors includes: State space = , which contains n elements, each representing a value of one of a series of multiple variables representing environmental, task, or autonomous vehicle parameters; After instantiating each element within the state space range, it is passed to the input state vectors of each scenario for simulation = , where i , and respectively represent and instantiated elements.
4. The performance boundary adaptive sampling method of an autonomous driving system according to claim 1, characterized in that, The evaluating the performance metrics of the autonomous driving system to obtain a score vector, and constructing a score sample set of the score vector according to the score vector includes: Evaluating the performance metrics of the autonomous driving system; Among them, the performance metrics include: task completion, number of safety violations, number of waypoints reached, fuel consumption; Output a score vector according to the performance metrics = ; Conduct N experiments to obtain a score sample set composed of N score vectors = 。 5. The performance boundary adaptive sampling method for an autonomous driving system according to claim 1, wherein The expression of the adaptive search algorithm includes: Among them, F is the simulation model function of the autonomous driving system, which accepts a set of N scenario input state vectors = ... and returns a set of N score vector sample sets = . The adaptive search function outputs which is a set of labeled samples of 6. The performance boundary adaptive sampling method of an autonomous driving system according to claim 5, characterized in that The performance patterns include: Completely successful TS of reaching two waypoints and successfully identifying and avoiding fixed and moving obstacles; Safe successful MS of only reaching the recovery waypoint and successfully identifying and avoiding fixed and moving obstacles; Task successful SS of only reaching the recovery waypoint and failing to successfully identify and avoid fixed and moving obstacles; Completely failed TF of not reaching any waypoint and failing to successfully identify and avoid fixed and moving obstacles.
7. A performance boundary adaptive sampling method for an autonomous driving system according to claim 1, the optimized classification algorithm of the brainstorm optimization algorithm includes: Take the k of the KNN algorithm and the number of training samples for each category as the optimization decision variables; Defining a preliminary optimization objective function: Among them , m is the total number of categories for classification, is the number of training samples for category ; Taking different values of k and as solutions, and using the classification error rate as the function value; Apply the brainstorm optimization algorithm to the setting of k and to find the optimal combination of k and .
Citation Information
Patent Citations
Acceleration test system for automatic driving system
CN111814308A
Determining Performance of Autonomy Decision-Making Engines
US20190179738A1