A test scene selection method, device and equipment

Abstract

The application discloses a test scene selection method, device and equipment. A plurality of test scene data is acquired, a preset event prediction model is trained by using the plurality of test scene data, a target event prediction model is generated, when a plurality of alternative scene data is received, the target event prediction model is used to predict the accident occurrence probability corresponding to each alternative scene data, and finally, a target test scene is selected from the plurality of alternative scene data according to a test request and the accident occurrence probability in response to the received test request. Therefore, the scene selection is targeted according to the prediction of the accident occurrence probability and different test requests, and the test scene selection is more flexible and effective.

Description

Technical Field

[0001] This invention relates to the field of scene selection technology, and in particular to a method, apparatus and equipment for selecting test scenes. Background Technology

[0002] In recent years, with the rapid development of autonomous driving technology, the commercialization of autonomous vehicles has made great progress. Safety has always been a core issue in the development of intelligent vehicles. As one of the most important systems for the operation of intelligent vehicles, the autonomous driving system usually needs to undergo massive testing in different operating scenarios during version iteration.

[0003] Virtual simulation technology can simulate real-world driving environments and perform tests and verifications on computer software, offering low cost and high testing efficiency. Therefore, when testing autonomous driving systems, it is common practice to run the system in simulated scenarios to evaluate the capabilities of its algorithms and how they change.

[0004] However, in existing technologies, the test scenarios are usually randomly selected without any reference information, or certain high-incidence test scenarios are directly chosen. When the above methods cannot be flexibly adjusted according to different test objectives, the test flexibility is low. Summary of the Invention

[0005] This invention provides a method, apparatus, and device for selecting test scenarios, solving the problem that in the prior art, the selected test scenarios are usually randomly chosen without any reference information, or certain specific high-incidence test scenarios are directly selected. When the above methods cannot be flexibly adjusted according to different test objectives, the test flexibility is low.

[0006] The first aspect of this invention provides a method for selecting a test scenario, comprising:

[0007] Acquire data from multiple test scenarios;

[0008] A preset event prediction model is trained using data from multiple test scenarios to generate a target event prediction model;

[0009] When multiple candidate scenario data are received, the probability of an accident corresponding to each candidate scenario data is predicted by the target event prediction model.

[0010] In response to the received test request, a target test scenario is selected from multiple candidate scenario data according to the test request and the probability of the accident.

[0011] Optionally, the step of training a preset event prediction model using multiple test scenario data to generate a target event prediction model includes:

[0012] Data from each test scenario is extracted according to preset extraction conditions to obtain multiple scenario segments with takeover or non-takeover indicators.

[0013] Extract scene features from each of the scene segments;

[0014] The scene features are used to train a preset event prediction model to generate a target event prediction model.

[0015] Optionally, the step of extracting each test scene data according to preset extraction conditions to obtain multiple scene segments includes:

[0016] Test scenario data with takeover flags were selected as takeover scenario data;

[0017] Test scenario data with unmanaged flags were selected as non-managed scenario data;

[0018] After extracting a segment of a preset time period before each takeover moment from the takeover scenario data and setting a takeover status indicator, it is used as a scenario segment;

[0019] After extracting a segment of data from a non-takeover scenario within a preset time period and setting a takeover / non-takeover flag, it is used as a scenario segment;

[0020] The number of intercepted data points in the takeover scenario is equal to the number of intercepted data points in the non-takeover scenario.

[0021] Optionally, the step of extracting scene features in each of the scene segments includes:

[0022] Obstacles are identified frame by frame in each of the aforementioned scene segments;

[0023] Calculate the speed information and number of obstacles in each frame;

[0024] Using all the speed information, calculate the average speed and maximum speed of the obstacles in the scene segment;

[0025] Using all the aforementioned obstacle quantities, calculate the average number of obstacles and the maximum number of obstacles in the scene segment;

[0026] The average speed of the obstacles, the maximum speed of the obstacles, the average number of obstacles, and the maximum number of obstacles are concatenated into a vector form to generate scene features.

[0027] Optionally, the step of training a preset event prediction model using multiple scene features to generate a target event prediction model includes:

[0028] Each of the aforementioned scene features is input into a preset event prediction model to generate a prediction result corresponding to the scene segment to which each of the aforementioned scene features belongs;

[0029] The prediction results are matched with the takeover or non-takeover flags corresponding to the scene segments to calculate the prediction success rate of the event prediction model.

[0030] If the prediction success rate is less than a preset threshold, the model parameters of the event prediction model are adjusted according to the gradient, and the process jumps to the step of inputting each scene feature into the preset event prediction model to generate the prediction result corresponding to the scene segment to which each scene feature belongs.

[0031] If the prediction success rate is greater than or equal to a preset threshold, then the event prediction model at the current moment is determined as the target event prediction model.

[0032] Optionally, the step of predicting the probability of an accident corresponding to each of the candidate scenario data using the target event prediction model when multiple candidate scenario data are received includes:

[0033] When multiple candidate scenario data are received, the corresponding candidate scenario features are extracted from each candidate scenario data.

[0034] The target event prediction model uses the features of the alternative scenarios to predict the probability of an accident occurring corresponding to the data of the alternative scenarios.

[0035] Optionally, the step of selecting a target test scenario from multiple candidate scenario data according to the test request and the probability of the accident, in response to the received test request, includes:

[0036] Respond to received test requests and parse them to determine the current test content;

[0037] If the current test content is algorithm optimization, then a predetermined number of target test scenarios are selected from multiple candidate scenario data according to the probability of the accident occurring from high to low.

[0038] Optionally, it also includes:

[0039] If the current test content is a real road test simulation, then the probability of the accident occurring is used as the importance for sampling, and the target test scenario is selected from multiple candidate scenario data.

[0040] A second aspect of the present invention also provides a test scenario selection device, comprising:

[0041] The data acquisition module is used to acquire data from multiple test scenarios;

[0042] The model training module is used to train a preset event prediction model using multiple test scenario data to generate a target event prediction model.

[0043] The probability prediction module is used to predict the probability of an accident corresponding to each of the candidate scenario data when multiple candidate scenario data are received, through the target event prediction model.

[0044] The scenario selection module is used to respond to the received test request and select a target test scenario from multiple candidate scenario data according to the test request and the probability of the accident.

[0045] A third aspect of the present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the test scenario selection method as described in any one of the first aspects of the present invention.

[0046] As can be seen from the above technical solutions, the present invention has the following advantages:

[0047] This invention acquires data from multiple test scenarios, trains a pre-defined event prediction model using this data, and generates a target event prediction model. When multiple candidate scenario data are received, the target event prediction model predicts the probability of an accident corresponding to each candidate scenario. Finally, in response to a received test request, the target test scenario is selected from the multiple candidate scenario data according to the test request and the probability of accident occurrence. Therefore, by combining the prediction of the probability of accident occurrence with different test requests, scenario selection is made more flexible and effective. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating the steps of a test scenario selection method provided in Embodiment 1 of the present invention;

[0050] Figure 2 This is a flowchart illustrating the steps of a test scenario selection method provided in Embodiment 2 of the present invention;

[0051] Figure 3 This is a structural block diagram of a test scenario selection device provided in Embodiment 3 of the present invention. Detailed Implementation

[0052] This invention provides a method, apparatus, and device for selecting test scenarios, addressing the problem in existing technologies where test scenarios are typically randomly selected without any reference information, or where certain frequently occurring test scenarios are directly chosen. These methods lack flexibility and cannot be flexibly adjusted for different testing objectives.

[0053] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0054] Please see Figure 1 , Figure 1 The flowchart illustrates the steps of a test scenario selection method provided in Embodiment 1 of the present invention.

[0055] The present invention provides a test scenario selection method, comprising:

[0056] Step 101: Obtain data from multiple test scenarios;

[0057] Test scenario data refers to scenario data collected during routine road tests of autonomous vehicles. This includes scenario data where human intervention occurs, such as in the event of an accident, as well as scenario data where no intervention occurs, such as in the event of a smooth operation of the autonomous vehicle. The number of scenarios involving human intervention is far less than the number of scenarios without intervention.

[0058] In this embodiment of the invention, multiple test scenario data uploaded by at least one autonomous vehicle can be obtained through a processor or the cloud. The operating scenarios corresponding to the test scenario data can be the same or different.

[0059] To reduce resource consumption, this method can receive test scenario data uploaded from the autonomous driving system via a cloud server, select data through the cloud, or set it within the autonomous driving system, facilitating real-time debugging of the autonomous driving system by technicians.

[0060] Step 102: Train a preset event prediction model using data from multiple test scenarios to generate a target event prediction model;

[0061] After obtaining the test scenario data, multiple test scenario data can be used to train the preset event prediction model to generate the target event prediction model.

[0062] It should be noted that the event prediction model can be constructed using binary classification models, decision trees, random forests, or machine learning algorithms such as neural networks, to predict specific events from input test scenario data and determine the probability of a specific event occurring in the test scenario. This embodiment of the invention does not limit the type of specific model construction.

[0063] Step 103: When multiple candidate scenario data are received, the probability of an accident corresponding to each candidate scenario data is predicted by the target event prediction model.

[0064] In this embodiment, after training the model using test scenario data to obtain the target event prediction model, it is also necessary to test other alternative scenario data.

[0065] Therefore, when the processor or cloud receives multiple alternative scenario data, each alternative scenario data can be input into the target event prediction model. The target event prediction model then uses the features in each alternative scenario data as the data basis to predict the probability of an accident occurring in each alternative scenario.

[0066] Step 104: Respond to the received test request and select the target test scenario from multiple alternative scenario data according to the test request and the probability of the incident.

[0067] In this embodiment, after predicting the probability of accidents in each candidate scenario, the selection of scenarios can be selectively emphasized based on different input test requests. The system responds to external input test requests and parses them to determine the test content emphasized by the request. Then, according to the test content and the probability of accidents, the target test scenario is selected from multiple candidate scenario data to make targeted scenario selection and conduct more effective and accurate testing of the autonomous driving system.

[0068] In this embodiment of the invention, multiple test scenario data are acquired, and a preset event prediction model is trained using this data to generate a target event prediction model. When multiple candidate scenario data are received, the target event prediction model predicts the probability of an accident corresponding to each candidate scenario data. Finally, in response to a received test request, a target test scenario is selected from the multiple candidate scenario data according to the test request and the probability of accident occurrence. Thus, by combining the prediction of the probability of accident occurrence with different test requests, scenario selection is made more flexible and effective.

[0069] Please see Figure 2 , Figure 2 The flowchart illustrates the steps of a test scenario selection method provided in Embodiment 2 of the present invention.

[0070] The present invention provides a test scenario selection method, comprising:

[0071] Step 201: Obtain data from multiple test scenarios;

[0072] In this embodiment of the invention, the specific implementation process of step 201 is similar to that of step 101, and will not be repeated here.

[0073] Step 202: Extract data from each test scenario according to preset extraction conditions to obtain multiple scenario segments with takeover or non-takeover indicators;

[0074] Furthermore, step 202 may include the following sub-steps:

[0075] Test scenario data with takeover flags were selected as takeover scenario data;

[0076] Test scenario data with unmanaged flags were selected as non-managed scenario data;

[0077] After extracting a segment of a preset time period before each takeover moment from the takeover scenario data and setting a takeover status indicator, it is used as a scenario segment;

[0078] After extracting a segment of data from a non-takeover scenario within a preset time period and setting a takeover / non-takeover flag, it is used as a scenario segment;

[0079] The number of data points captured in the takeover scenario is equal to the number of data points captured in the non-takeover scenario.

[0080] In this embodiment, each test scenario data is marked with identification information indicating whether it has been taken over. After obtaining multiple test scenario data, the test scenario data containing the takeover identification is used as the takeover scenario data. Data segments within a preset time period, such as 5 seconds, before each takeover moment are extracted from the data segments. The takeover identification information set in the extracted data segments is marked as true value 1, thereby obtaining the scenario segments corresponding to the takeover scenarios.

[0081] Meanwhile, data with an unmanaged flag is selected from the test scenario data as non-managed scenario data. The non-managed scenario data is then segmented according to the preset time period, and the takeover flag set in the segmented data is marked as true value 0, thereby obtaining the scenario segment corresponding to the non-managed scenario.

[0082] It should be noted that the number of data segments extracted from the takeover scenario and the number of data segments extracted from the non-takeover scenario are equal, and the scenarios to which the takeover scenario data and the non-takeover scenario data belong can be the same or different.

[0083] Step 203: Extract scene features from each scene segment;

[0084] Optionally, step 203 may include the following sub-steps:

[0085] Identify obstacles frame by frame in each scene segment;

[0086] Calculate the speed information and number of obstacles in each frame;

[0087] Using all speed information, calculate the average speed and maximum speed of obstacles in the scene segment;

[0088] Using the total number of obstacles, calculate the average number of obstacles and the maximum number of obstacles in a scene segment;

[0089] The average speed of obstacles, the maximum speed of obstacles, the average number of obstacles, and the maximum number of obstacles are concatenated into a vector form to generate scene features.

[0090] In this embodiment, since scene segments typically consist of several seconds of data, they may not be sufficiently targeted for model training, resulting in low training efficiency. Therefore, obstacle identification can be performed frame-by-frame in each scene segment to count the number of obstacles and their speed information in each frame. Then, the average speed of obstacles in the scene segment is calculated using the ratio of all speed information to the number of frames, and the maximum value among the speed information is selected as the highest obstacle speed.

[0091] At the same time, the average number of obstacles in a scene segment is calculated by using the ratio of the total number of obstacles to the number of frames, and the maximum number of obstacles is selected as the maximum number of obstacles.

[0092] After calculating the above feature information, it is concatenated into a vector form according to the average speed of obstacles, the maximum speed of obstacles, the average number of obstacles, and the maximum number of obstacles, thereby generating the scene features corresponding to the scene segment.

[0093] In practical implementations, there are many methods for describing a scene. A common approach is to calculate features such as the average and maximum speed of obstacles within a given time period, the average number of obstacles per frame, and the maximum number of obstacles at any given moment. For example, assuming there are a total of n frames within 5 seconds, the information for the i-th frame is frame... i The information of the j-th obstacle is frame i .obstacle j The speed of the obstacle is frame i .obstacle j If speed is a given value, then the features listed above can be expressed as:

[0094] Average speed:

[0095] Top speed:

[0096] Average number of surrounding obstacles per frame:

[0097] Maximum number of obstacles at any given time:

[0098] After obtaining the above features, they are concatenated into a vector form to obtain the scene features (speed_avg, speed_max, num_avg, num_max).

[0099] It should be noted that, in addition to the speed and quantity features related to obstacles mentioned above, scene features may also include the behavioral features of the main vehicle, such as average speed and maximum speed; obstacle features, such as the average speed and congestion level of obstacles within a certain range; and other information features, such as map features (the proportion of time spent at intersections). This embodiment of the invention does not limit the types of features; they can be selected based on the training scenario. For example, if the tested scenario is a main road or a village within a city with a high degree of urbanization, features can be selected based on obstacles; if it is a highway or a desert with a low degree of urbanization, features can be selected based on the behavioral features of the main vehicle; if it is a complex scenario with many intersections, map features can be further added as scene features for subsequent model training.

[0100] Step 204: Train a pre-defined event prediction model using scene features to generate a target event prediction model;

[0101] In one example of the present invention, step 204 may include the following sub-steps:

[0102] Each scene feature is input into a pre-defined event prediction model to generate a prediction result for the scene segment to which each scene feature belongs.

[0103] The prediction success rate of the event prediction model is calculated by matching each prediction result with the takeover or non-takeover flag corresponding to the scene segment.

[0104] If the prediction success rate is less than the preset threshold, the model parameters of the event prediction model are adjusted according to the gradient, and the process jumps to the step of inputting each scene feature into the preset event prediction model to generate the prediction result corresponding to the scene segment to which each scene feature belongs.

[0105] If the prediction success rate is greater than or equal to the preset threshold, the event prediction model at the current moment will be determined as the target event prediction model.

[0106] In this embodiment, after obtaining the scene features, the scene features can be input into a preset event prediction model to predict specific events under the initial model parameters, so as to generate the prediction result of the scene segment to which the scene features belong.

[0107] Each prediction result is matched with the takeover / non-takeover flag corresponding to the scene segment. If the prediction result matches the result corresponding to the takeover / non-takeover flag, the model is considered to have successfully predicted the scene segment; otherwise, the model is considered to have failed to predict the scene segment. After statistically analyzing all prediction results, the ratio between the number of successful predictions and the total number of predictions is calculated to determine the prediction success rate of the event prediction model in the current prediction training.

[0108] If the prediction success rate is less than the preset threshold, it indicates that the accuracy of the event prediction model is limited. In this case, the model parameters can be adjusted using either a descent or ascending gradient method, and the features of each scene can be input into the adjusted event prediction model again to conduct an iterative training process. This continues until the prediction success rate is greater than or equal to the preset threshold, indicating that the model parameters have a high degree of adaptability. At this point, the event prediction model can be selected as the target event prediction model.

[0109] It's important to note that, taking the event prediction model as an example of a binary classification model, this problem is essentially a binary classification problem (the input is some features, and the output is 0 / 1). Such problems generally require machine learning methods, hence the use of a machine learning model. Model training requires the scene fragment features and ground truth values ​​obtained in the previous step. Many binary classification machine learning models can be used (logistic regression, decision trees, neural networks, etc.). This framework uses the LightGBM model, a decision tree-based algorithm, which has the advantage of supporting efficient parallel training.

[0110] The essence of binary classification models is to output a value between 0 and 1, usually with 0.5 as the boundary: if it is greater than 0.5, it is labeled as 1, otherwise it is labeled as 0, thus achieving binary classification.

[0111] The training data is (speed_avg, speed_max, num_avg, num_max) – label (0 / 1).

[0112] In practice, once the target event prediction model has been trained, it can be saved and called directly each time it is used, without the need for repeated training.

[0113] Step 205: When multiple candidate scenario data are received, the probability of an accident occurring corresponding to each candidate scenario data is predicted by the target event prediction model.

[0114] Optionally, step 205 may include the following sub-steps:

[0115] When multiple candidate scenario data are received, the corresponding candidate scenario features are extracted from each candidate scenario data.

[0116] The target event prediction model uses the features of alternative scenarios to predict the probability of an accident occurring corresponding to the data of alternative scenarios.

[0117] In this embodiment of the invention, after completing the target event prediction model and receiving multiple candidate scenario data, it indicates that test scenario selection is required. At this point, each candidate scenario data can be segmented and feature extraction can be performed on each frame segment, extracting the corresponding candidate scenario features from each candidate scenario data. These candidate scenario features are then input into the target event prediction model, which predicts the probability of an accident occurring for each candidate scenario data.

[0118] In the actual implementation, this step is similar to the training step, where each candidate scenario can also calculate its own features. Since the model from the previous step is needed, the type and method of scene features calculated in this step must be consistent with the previous step. Then, the model saved in the previous step is called to predict the probability of an accident occurring in each candidate scenario.

[0119] Step 206: Respond to the received test request and select the target test scenario from multiple alternative scenario data according to the test request and the probability of the incident.

[0120] In one example of the present invention, step 206 may include the following sub-steps:

[0121] Respond to received test requests and parse them to determine the current test content;

[0122] If the current test content is algorithm optimization, then a predetermined number of target test scenarios will be selected from multiple candidate scenario data according to the probability of the accident occurring from high to low.

[0123] In this embodiment, since different projects may have different test content or test objectives, the test request can be parsed through external input to determine the current test content.

[0124] The current testing content may include, but is not limited to, algorithm optimization or real road test simulation.

[0125] If the current test content is algorithm optimization, it means that it is necessary to find the problems that occur when the autonomous driving system runs in this scenario. At this time, a predetermined number of candidate scenarios can be selected from the candidate scenario data in descending order of the probability of accident occurrence as the target test scenarios.

[0126] In another example of the present invention, step 206 may further include the following sub-steps:

[0127] If the current test content is a real road test simulation, the probability of an accident is used as the importance for sampling, and the target test scenario is selected from multiple alternative scenario data.

[0128] In this embodiment, the current test content may also include real road test simulation. In this case, the probability of an accident can be used as the importance for sampling, and the target test scenario can be selected from multiple alternative scenario data.

[0129] In a road test, MPI (mileage per intervention) is typically used to measure the performance of an autonomous driving algorithm. If you want to estimate this value in simulation testing (testing all candidate scenarios would be extremely resource-intensive, allowing only a subset), you can't blindly select test scenarios randomly, nor can you only choose scenarios with a high predicted accident probability. For example, if the predicted probability of scenario A causing an incident is 0.9, then scenario A is considered to have an importance of 0.9. During sampling, there's a 0.9 probability of selecting this scenario. This value is indeed a constant across different sampling tests, but it's not guaranteed to be selected in every sampling, as there's still a 0.1 probability of not selecting it.

[0130] Importance sampling is a method used in statistics to estimate the properties of a distribution. This method samples from a different distribution and estimates the properties of the original distribution.

[0131] One purpose of road testing is to evaluate the capabilities of autonomous driving algorithms, and one metric is MPI (Mileage Intervention Index). Given a set of test scenarios, we want to predict the number of interventions through sampling tests. However, very few scenarios in the test scenario set actually involve interventions. If uniform sampling is used, the number of scenarios where an incident could occur will also be very small, resulting in a very large variance in the predicted number of interventions.

[0132] The sampling variance is:

[0133]

[0134] To reduce this variance, the following condition must be met: q * ∝p(x)f(x), if the road test sampling distribution is ignored, then it is q * ∝f(x), meaning we need to know relatively accurately whether an incident will occur (or make a probability prediction).

[0135] Optionally, after determining the target test scenario, the ratio between the takeover / disengagement flag and the probability of an accident corresponding to that target test scenario can be further calculated to unbiasedly calculate the number of takeovers during the road test.

[0136]

[0137] Where C is the expected value, i.e. the actual value of an incident; p(x) is the true distribution, which is already satisfied when the road test data is collected; and f(x) is whether an incident occurs.

[0138] If the probability of an incident occurring in this scenario is 0.1, then the probability of this scenario being selected is 0.1. If an incident does occur, then for the candidate scenarios, the count should be 1 / 0.1 = 10 times.

[0139] In this embodiment of the invention, multiple test scenario data are acquired, and a preset event prediction model is trained using this data to generate a target event prediction model. When multiple candidate scenario data are received, the target event prediction model predicts the probability of an accident corresponding to each candidate scenario data. Finally, in response to a received test request, a target test scenario is selected from the multiple candidate scenario data according to the test request and the probability of accident occurrence. Thus, by combining the prediction of the probability of accident occurrence with different test requests, scenario selection is made more flexible and effective.

[0140] Please see Figure 3 , Figure 3 A structural block diagram of a test scenario selection device according to Embodiment 3 of the present invention is shown.

[0141] This invention provides a test scenario selection device, comprising:

[0142] Data acquisition module 301 is used to acquire data from multiple test scenarios;

[0143] The model training module 302 is used to train a preset event prediction model using multiple test scenario data to generate a target event prediction model.

[0144] The probability prediction module 303 is used to predict the probability of an accident corresponding to each candidate scenario data through the target event prediction model when multiple candidate scenario data are received.

[0145] The scenario selection module 304 is used to respond to the received test request and select the target test scenario from multiple candidate scenario data according to the test request and the probability of the incident.

[0146] Optionally, the model training module 302 includes:

[0147] The segment extraction submodule is used to extract data from each test scenario according to preset extraction conditions, and obtain multiple scene segments with takeover or non-takeover indicators.

[0148] The feature extraction submodule is used to extract scene features from each scene segment;

[0149] The model training submodule is used to train a pre-defined event prediction model using scene features and generate a target event prediction model.

[0150] Optionally, the fragment extraction submodule is specifically used for:

[0151] Test scenario data with takeover flags were selected as takeover scenario data;

[0152] Test scenario data with unmanaged flags were selected as non-managed scenario data;

[0153] After extracting a segment of a preset time period before each takeover moment from the takeover scenario data and setting a takeover status indicator, it is used as a scenario segment;

[0154] After extracting a segment of data from a non-takeover scenario within a preset time period and setting a takeover / non-takeover flag, it is used as a scenario segment;

[0155] The number of data points captured in the takeover scenario is equal to the number of data points captured in the non-takeover scenario.

[0156] Optionally, the feature extraction submodule is specifically used for:

[0157] Identify obstacles frame by frame in each scene segment;

[0158] Calculate the speed information and number of obstacles in each frame;

[0159] Using all speed information, calculate the average speed and maximum speed of obstacles in the scene segment;

[0160] Using the total number of obstacles, calculate the average number of obstacles and the maximum number of obstacles in a scene segment;

[0161] The average speed of obstacles, the maximum speed of obstacles, the average number of obstacles, and the maximum number of obstacles are concatenated into a vector form to generate scene features.

[0162] Optionally, the model training submodule is specifically used for:

[0163] Each scene feature is input into a pre-defined event prediction model to generate a prediction result for the scene segment to which each scene feature belongs.

[0164] The prediction success rate of the event prediction model is calculated by matching each prediction result with the takeover or non-takeover flag corresponding to the scene segment.

[0165] If the prediction success rate is less than the preset threshold, the model parameters of the event prediction model are adjusted according to the gradient, and the process jumps to the step of inputting each scene feature into the preset event prediction model to generate the prediction result corresponding to the scene segment to which each scene feature belongs.

[0166] If the prediction success rate is greater than or equal to the preset threshold, the event prediction model at the current moment will be determined as the target event prediction model.

[0167] Optionally, the probability prediction module 303 is specifically used for:

[0168] When multiple candidate scenario data are received, the corresponding candidate scenario features are extracted from each candidate scenario data.

[0169] The target event prediction model uses the features of alternative scenarios to predict the probability of an accident occurring corresponding to the data of alternative scenarios.

[0170] Optionally, the scene selection module 304 is specifically used for:

[0171] Respond to received test requests and parse them to determine the current test content;

[0172] If the current test content is algorithm optimization, then a predetermined number of target test scenarios will be selected from multiple candidate scenario data according to the probability of the accident occurring from high to low.

[0173] Optionally, the scene selection module 304 is also specifically used for:

[0174] If the current test content is a real road test simulation, the probability of an accident is used as the importance for sampling, and the target test scenario is selected from multiple alternative scenario data.

[0175] This invention also provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the test scenario selection method as described in any embodiment of this invention.

[0176] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, modules, and sub-modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0177] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0178] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0179] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for selecting test scenarios, characterized in that, include: Acquire data from multiple test scenarios; A preset event prediction model is trained using data from multiple test scenarios to generate a target event prediction model; When multiple candidate scenario data are received, the probability of an accident corresponding to each candidate scenario data is predicted by the target event prediction model. In response to the received test request, a target test scenario is selected from multiple candidate scenario data according to the test request and the probability of the accident occurrence; The step of training a preset event prediction model using multiple test scenario data to generate a target event prediction model includes: Data from each test scenario is extracted according to preset extraction conditions to obtain multiple scenario segments with takeover or non-takeover indicators. Extract scene features from each of the scene segments; The scene features are used to train a preset event prediction model to generate a target event prediction model; The step of extracting scene features in each of the scene segments includes: Obstacles are identified frame by frame in each of the aforementioned scene segments; Calculate the speed information and number of obstacles in each frame; Using all the speed information, calculate the average speed and maximum speed of the obstacles in the scene segment; Using all the aforementioned obstacle quantities, calculate the average number of obstacles and the maximum number of obstacles in the scene segment; The average speed of the obstacles, the maximum speed of the obstacles, the average number of obstacles, and the maximum number of obstacles are concatenated into a vector form to generate scene features.

2. The method according to claim 1, characterized in that, The step of extracting test scene data according to preset extraction conditions to obtain multiple scene segments with takeover or non-takeover indicators includes: Test scenario data with takeover flags were selected as takeover scenario data; Test scenario data with unmanaged flags were selected as non-managed scenario data; After extracting a segment of a preset time period before each takeover moment from the takeover scenario data and setting a takeover status indicator, it is used as a scenario segment; After extracting a segment of data from a non-takeover scenario within a preset time period and setting a takeover / non-takeover flag, it is used as a scenario segment; The number of intercepted data points in the takeover scenario is equal to the number of intercepted data points in the non-takeover scenario.

3. The method according to claim 1, characterized in that, The step of training a preset event prediction model using multiple scene features to generate a target event prediction model includes: Each of the aforementioned scene features is input into a preset event prediction model to generate a prediction result corresponding to the scene segment to which each of the aforementioned scene features belongs; The prediction results are matched with the takeover or non-takeover flags corresponding to the scene segments to calculate the prediction success rate of the event prediction model. If the prediction success rate is less than a preset threshold, the model parameters of the event prediction model are adjusted according to the gradient, and the process jumps to the step of inputting each scene feature into the preset event prediction model to generate the prediction result corresponding to the scene segment to which each scene feature belongs. If the prediction success rate is greater than or equal to a preset threshold, then the event prediction model at the current moment is determined as the target event prediction model.

4. The method according to claim 1, characterized in that, The step of predicting the probability of an accident corresponding to each of the candidate scenario data when multiple candidate scenario data are received, using the target event prediction model, includes: When multiple candidate scenario data are received, the corresponding candidate scenario features are extracted from each candidate scenario data. The target event prediction model uses the features of the alternative scenarios to predict the probability of an accident occurring corresponding to the data of the alternative scenarios.

5. The method according to claim 1, characterized in that, The step of selecting a target test scenario from multiple candidate scenario data according to the test request and the probability of the accident, in response to the received test request, includes: Respond to received test requests and parse them to determine the current test content; If the current test content is algorithm optimization, then a predetermined number of target test scenarios are selected from multiple candidate scenario data according to the probability of the accident occurring from high to low.

6. The method according to claim 5, characterized in that, Also includes: If the current test content is a real road test simulation, then the probability of the accident occurring is used as the importance for sampling, and the target test scenario is selected from multiple candidate scenario data.

7. A test scenario selection device, characterized in that, include: The data acquisition module is used to acquire data from multiple test scenarios; The model training module is used to train a preset event prediction model using multiple test scenario data to generate a target event prediction model. The probability prediction module is used to predict the probability of an accident corresponding to each of the candidate scenario data when multiple candidate scenario data are received, through the target event prediction model. The scenario selection module is used to respond to the received test request and select a target test scenario from multiple candidate scenario data according to the test request and the probability of the accident occurrence; The model training module includes: The segment extraction submodule is used to extract the test scenario data according to preset extraction conditions to obtain multiple scene segments with takeover or non-takeover indicators. The feature extraction submodule is used to extract scene features from each of the scene segments; The model training submodule is used to train a preset event prediction model using the scene features and generate a target event prediction model. The feature extraction submodule is specifically used for: Obstacles are identified frame by frame in each of the aforementioned scene segments; Calculate the speed information and number of obstacles in each frame; Using all the speed information, calculate the average speed and maximum speed of the obstacles in the scene segment; Using all the aforementioned obstacle quantities, calculate the average number of obstacles and the maximum number of obstacles in the scene segment; The average speed of the obstacles, the maximum speed of the obstacles, the average number of obstacles, and the maximum number of obstacles are concatenated into a vector form to generate scene features.

8. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the steps of the test scenario selection method as described in any one of claims 1-6.

Images