Automatic driving test scene library generation method and device, and storage medium

Abstract

An autonomous driving test scenario library generation method, apparatus, and computer-readable storage medium are disclosed. The method includes: setting an initial virtual test scenario and a test task, the initial virtual test scenario including a vehicle under test and surrounding vehicles; constructing a vehicle behavior utility model, the vehicle behavior utility model including multiple different driving strategies; executing the test task on the vehicle under test, and changing the driving strategies of surrounding vehicles during the execution of the test task; obtaining test results; and generating a test scenario library based on the test results, the test scenario library including multiple virtual test scenarios with different difficulties for the vehicle under test and the driving strategies of surrounding vehicles corresponding to each virtual test scenario.

Description

Technical Field

[0001] This disclosure relates to, but is not limited to, the field of autonomous driving testing technology, and particularly to a method, apparatus, and storage medium for generating an autonomous driving test scenario library. Background Technology

[0002] Autonomous vehicles, as a typical intelligent system that can be implemented in our daily lives, use artificial intelligence technologies such as machine learning algorithms and are equipped with advanced sensors, computing components and actuators to achieve intelligent perception, decision-making and execution capabilities. They replace or partially replace human drivers to complete safe and efficient transportation and are an important component of intelligent transportation systems.

[0003] The intelligence of autonomous vehicles refers to the characteristics of autonomous vehicles that exhibit abilities similar to or comparable to human intelligence when completing tasks. Assessing and evaluating the level of intelligence of autonomous vehicles and determining their intelligence capability levels are crucial aspects of the design, implementation, and application of autonomous vehicles.

[0004] Unlike traditional autonomous vehicle testing, the testing and evaluation of the intelligence level of autonomous vehicles is a completely new field in automotive testing. By dissecting the internal structure of intelligence and deconstructing its external representation, the intelligence testing of autonomous vehicles requires the effective testing of various intelligent capabilities of autonomous vehicles. By setting different test indicators, the intelligence level of autonomous vehicles in achieving different goals such as safety and environmental protection can be evaluated. Summary of the Invention

[0005] This disclosure provides a method for generating an autonomous driving test scenario library, including:

[0006] Set up a virtual test scenario and test tasks, wherein the virtual test scenario includes the vehicle under test and surrounding vehicles;

[0007] A vehicle behavior utility model is constructed, which includes a variety of different driving strategies;

[0008] The test task is performed on the vehicle under test, and the driving strategies of surrounding vehicles are changed during the execution of the test task.

[0009] Obtain test results, and generate virtual test scenarios and / or test tasks with different levels of difficulty for the vehicle under test based on the test results.

[0010] This disclosure also provides an autonomous driving test simulation apparatus, including a memory; and a processor connected to the memory, the memory being used to store instructions, and the processor being configured to execute the steps of the autonomous driving test scenario library generation method described in any embodiment of this disclosure based on the instructions stored in the memory.

[0011] This disclosure also provides a storage medium storing a computer program that, when executed by a processor, implements the autonomous driving test scenario library generation method described in any embodiment of this disclosure.

[0012] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the application. Other advantages of this application can be realized and obtained by means of the solutions described in the description and the accompanying drawings. Attached Figure Description

[0013] The accompanying drawings are used to provide an understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0014] Figure 1 A flowchart illustrating an exemplary embodiment of this disclosure of a method for generating an autonomous driving test scenario library;

[0015] Figure 2 A schematic diagram illustrating a test task allocation method provided for an exemplary embodiment of this disclosure;

[0016] Figure 3 A schematic diagram illustrating an autonomous driving simulation test process and specific test steps, provided as an exemplary embodiment of this disclosure;

[0017] Figure 4 A schematic diagram of a vehicle cutting scenario on a one-way two-lane road, provided as an exemplary embodiment of this disclosure;

[0018] Figure 5A A schematic diagram of experimental results provided for an exemplary embodiment of this disclosure;

[0019] Figure 5B Another schematic diagram of experimental results provided for an exemplary embodiment of this disclosure;

[0020] Figure 6 A schematic diagram of the structure of an autonomous driving test scenario library generation device provided as an exemplary embodiment of this disclosure. Detailed Implementation

[0021] This application describes several embodiments, but these descriptions are exemplary and not restrictive, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0022] This application includes and contemplates combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed in this application may also be combined with any conventional features or elements to form a unique inventive scheme as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive schemes to form another unique inventive scheme as defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this application may be implemented individually or in any suitable combination. Therefore, the embodiments are not limited except by the limitations imposed by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

[0023] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that it does not depend on such a specific order. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation of the claims. Moreover, the claims concerning the method and / or process should not be limited to the steps performed in the written order, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments of this application.

[0024] Virtual simulation testing is an important means of assessing the intelligence level of autonomous vehicles. Virtual simulation testing searches for and samples key scenarios from autonomous vehicle driving datasets, enabling the acquisition of more road conditions that cannot be encountered during real-world testing in a short time, thus ensuring testing efficiency and scenario coverage.

[0025] Designing test tasks is a crucial step in realizing virtual simulation testing of the intelligence of autonomous vehicles. Introducing test task analysis into the testing process helps to understand the reasons for test success or failure, and allows for adjustments to the test tasks to facilitate subsequent optimization of test scenarios and further evaluation of the vehicle's intelligence level.

[0026] Current methods for generating virtual simulation test scenarios primarily focus on improving testing efficiency, neglecting the need for test scenarios of varying difficulty required to assess the vehicle's intelligence level. Furthermore, in lane-changing scenarios, only the impact of vehicles changing lanes on the tested vehicle is considered, ignoring the influence of other vehicles' non-lane-changing actions. This makes it impossible to sample test scenarios of varying difficulty. Typically, we lack knowledge of the intelligence level of the autonomous vehicle being tested, nor how it was trained (e.g., the amount and type of data used during training). Therefore, we cannot infer from its unique conditions which test scenarios are relatively difficult and which are relatively easy for the vehicle; in other words, we cannot determine the vehicle's actual level of intelligence.

[0027] This disclosure, taking into account the normal operation of components and functional devices such as sensors in autonomous vehicles, and after clarifying the testing requirements and objectives, proposes a method for generating an autonomous driving test scenario library, using intelligent testing that reflects safety as an example, to evaluate the intelligence level of autonomous vehicles.

[0028] like Figure 1 As shown, this disclosure provides a method for generating an autonomous driving test scenario library, including:

[0029] Step 101: Set up the initial virtual test scenario and test tasks. The initial virtual test scenario includes the vehicle under test and surrounding vehicles.

[0030] Step 102: Construct a vehicle behavior utility model, which includes various driving strategies;

[0031] Step 103: Perform a test task on the vehicle under test, and change the driving strategies of surrounding vehicles during the test task execution.

[0032] Step 104: Obtain test results and generate a test scenario library based on the test results. The test scenario library includes various virtual test scenarios with different difficulties for the tested vehicle and driving strategies for surrounding vehicles corresponding to each virtual test scenario.

[0033] The autonomous driving test scenario library generation method proposed in this disclosure uses the driving strategies of traffic participants in the virtual test scenario as adjustable elements of the virtual test scenario. By changing the driving strategies of surrounding vehicles during the execution of the test task and controlling the surrounding vehicles to simulate the impact of different driving behaviors on the test vehicle, virtual test scenarios with different difficulties for the test vehicle can be obtained. Finally, these scenarios are used to evaluate the intelligence level of the test vehicle to meet the needs of intelligence level assessment.

[0034] In some exemplary embodiments, a variety of different driving strategies include at least one aggressive driving strategy and at least one conservative driving strategy; the aggressive driving strategy tends to maintain a smaller minimum safe distance from the vehicle in front during following or lane changing than the conservative driving strategy tends to maintain a smaller minimum safe distance from the vehicle in front during following or lane changing.

[0035] In this embodiment of the disclosure, vehicles with an aggressive driving strategy tend to maintain a relatively close following distance or headway when following or changing lanes, and exhibit higher speeds and accelerations than normal during the process of completing a certain action. In contrast, vehicles with a conservative driving strategy tend to maintain a relatively large following distance or headway when following or changing lanes, and exhibit lower speeds and accelerations than normal during the process of completing a certain action.

[0036] In this embodiment of the disclosure, the various driving strategies may also include a normal driving strategy. Furthermore, in this embodiment of the disclosure, both the aggressive driving strategy and the conservative driving strategy may be set to multiple different levels; this embodiment of the disclosure does not impose any limitations on this.

[0037] In some exemplary implementations, during the execution of a test task, the driving strategies of surrounding vehicles are altered, including:

[0038] Determine all possible combinations of driving strategies for surrounding vehicles;

[0039] During the execution of the test task, each possible combination is tested one by one.

[0040] Since we don't know how the test vehicle was trained—for example, whether its training scenarios included adjusting the driving strategies of multiple surrounding vehicles or just one—we can help find more challenging scenarios for the test vehicle by varying the number of surrounding vehicles and their driving strategies.

[0041] In some exemplary implementations, the difficulty of the test task is defined as Where, ∑ t A i S is the total number of challenging events encountered by the test vehicle during the completion of test task i. A challenging event is defined as a collision between the test vehicle and surrounding vehicles occurring within a preset threshold time. i Let t be the total number of samples for test task i, where i is a natural number and t is the test time.

[0042] In some exemplary implementations, the difficulty of a virtual test scenario is defined as... Among them, S j Let A be the total number of samples for test task j, where j is a natural number and j ≠ i. ij Test task i and test task j are the number of challenging events encountered by the vehicle under test when concurrent tasks are performed, and μ is the weighting coefficient to mitigate the difficulty of concurrent tasks, 0≤μ≤1; concurrent tasks are defined as two or more tasks that are not related and have no logical temporal or spatial order and can be triggered at the same time and place.

[0043] While current research points out that the definition of test tasks is key to testing autonomous vehicles, it cannot meet the actual development needs of assessing the intelligence capability level of autonomous vehicles. At the same time, the lack of model descriptions of vehicle-to-vehicle interaction behaviors in scenarios leads to a lack of evidence for the differences in test tasks, and thus cannot provide methods for adjusting the difficulty required for different test tasks.

[0044] This disclosure addresses the issue of adjusting the difficulty of test tasks. Based on the method for constructing test tasks, it introduces a vehicle behavior utility model, constructs a behavior utility function to quantify the interaction behavior between vehicles, and constructs test tasks accordingly. By adjusting the number of vehicles that interact with the test vehicle or the driving strategies of these vehicles, the difficulty of the test task can be effectively adjusted.

[0045] Under the condition of clear testing requirements and testing objectives, this disclosure proposes a method for generating an autonomous driving test scenario library after constructing relevant virtual test scenarios (where the static environment is fixed and the only dynamic traffic participants are vehicles) in a virtual simulation test platform, in order to complete the comprehensive testing of the intelligence of autonomous vehicles.

[0046] In virtual test scenarios, to successfully pass tests under specific traffic conditions, the vehicle under test needs to successfully complete a series of test tasks. A test task refers to an activity that the autonomous vehicle needs to complete within a limited time in the test scenario. Therefore, setting measurable test tasks is indispensable and a prerequisite for evaluation. Test tasks can be broken down according to different test requirements. Test requirements can be single objectives for testing autonomous vehicles, such as the vehicle's intelligent safety, task execution efficiency, and comfort level, or they can be comprehensive tests encompassing multiple capabilities. The set of test tasks can be represented as consisting of several tasks, i.e., Ψ = {ψ1, ψ2, ...}. Different test tasks are generated according to each type of test requirement, and each test task can be further divided into several sub-tasks within a limited time and a defined space, i.e., ψ i ={ψ i1 ,ψ i2 The spatiotemporal relationships between subtasks are not fixed. Limited time and defined space refer to the spatiotemporal scope of the test task. For example, completing a traffic light test task means passing through a specific intersection within one minute, not passing through any intersection without a time limit. To facilitate the evaluation of the autonomous vehicle's task completion, embodiments of this disclosure provide a method for allocating test tasks, such as... Figure 2 As shown.

[0047] For example, taking safety-oriented intelligent function testing and characterization testing of autonomous vehicles as an example, in functional testing, the test tasks can be decomposed into several sub-tasks based on the different intelligent capabilities of the autonomous vehicle, such as perception, decision-making, and execution capabilities. In intelligent safety characterization testing, the different types of performance of the vehicle under test can be used as the basis for breaking down the test tasks, such as recognizing traffic lights at a certain time and place, determining whether to change lanes, and following the vehicle in front.

[0048] Different driving behaviors of traffic participants can be quantified using different behavioral utility functions, and the various factors influencing these behaviors can serve as inputs to the behavioral utility functions. This disclosure constructs a vehicle behavior utility model when the tested vehicle interacts with surrounding vehicles, which is used to subsequently analyze the impact of the interaction process between the tested vehicle and surrounding vehicles on the virtual test scenario and / or the difficulty of the test task.

[0049] Consider a behavioral utility function whose input is a series of independent variables: x1, x2, ..., x n (Velocity, position, acceleration, etc.), for any vehicle V (i) Define the vehicle's behavior utility function as U. i We consider the behavioral utility function to be a weighted sum of these independent variables; therefore, the behavioral utility function U... i The basic formula can be written as follows:

[0050]

[0051] Where, α k,i For variable x k The weighting coefficients. In actual use, the number of variables and weighting coefficients may vary for different vehicles. If there is an existing vehicle to be imitated, the parameters can be fitted using a fitting method, or a custom design can be made as needed. This disclosure uses the above-mentioned behavioral utility function U i The basic formula establishes a general form of expression for behavioral utility functions.

[0052] Considering these influencing factors x k This can be described using a Gaussian mixture distribution. Based on the properties of the Gaussian mixture distribution, when the classification is sufficiently reasonable, it can, to a certain extent, represent all possible distributions. Furthermore, these influencing factors can all be normalized (normalization eliminates the influence of different indicators having different magnitudes), meaning they have zero mean and unit variance. Therefore, vehicle V... (i) The prior distribution of behavioral utility can be represented as:

[0053]

[0054] Wherein, p(U i ) for U i The probability density distribution, (.) follows a standard normal distribution.

[0055] The behavioral utility of all traffic participants in the test scenario can be represented as a multivariate joint distribution:

[0056] p(U1,U2,…,U N ) = N(U;0,∑);

[0057] Where p(U1,U2,…,U) N ) represents U1 to U N The probability distribution function, N(U; 0, ∑), represents the distance from U1 to U2. N It follows an N-variable Gaussian distribution with mean 0 and covariance matrix Σ, U=[U1,U2,...,U N ] T ,and

[0058]

[0059] Where, ρ k,l V represents k With V l The correlation between the behaviors of the two vehicles

[0060] Considering the traffic participants in the case, they constitute a traffic group with undifferentiated preferences, meaning that the participants are influenced by factor x. i Maintaining consistency in preferences, such that for any pair of k and l, ρ k,l =ρ, k, and l are all between 1 and N, that is, each ρ in the matrix k,l All are equal. This is equivalent to, in a traffic environment, the tested vehicle knows that other vehicles around it have different preferences and knows the degree of overlap between these preferences and its own preferences, but does not need to identify the specific vehicle that matches its own preferences.

[0061] The aforementioned preferences are factors that influence participants' driving behavior. For example, the tested vehicle can detect in advance the presence of vehicles in the vicinity that are inconsistent with its own preferences (such as decision-making methods). However, the vehicle cannot confirm that the other vehicle is a threat until it engages in dangerous behavior, a setting that reflects real-world driving situations.

[0062] First, consider that the vehicle cannot know the driving preferences of other vehicles and only uses its own information to influence each decision. Clearly, this information is imperfect, meaning it contains noise residuals ν. k,i , and the measured value x k Together they form the true value: Correspondingly, vehicle V (i) The estimated behavioral utility can be rewritten as:

[0063]

[0064] Without loss of generality, consider E(v) k,i ) = 0, var(v k,i )=ε 2 Where ε is the environmental noise level related to the utility value, from which we can obtain the information about vehicle V. (i) Conditional probability density distribution of behavioral utility

[0065]

[0066] Referring to the formula above, we can obtain

[0067]

[0068]

[0069] Where, δ k,l This is the Kronecker delta function. The Kronecker function typically takes two integers as input; if they are equal, its output is 1, otherwise it is 0.

[0070] Therefore, when a vehicle's behavior is determined by its prior expectations and the information it has received,

[0071]

[0072] We will theoretically demonstrate that adjusting the behavioral utility of surrounding vehicles can create test tasks of varying difficulty. That is, in the simulation testing method disclosed herein, the vehicle under test can encounter more challenging events in the test scenario. For ease of analysis, in the following derivation, we will assign the vehicle under test the number V. (1) The surrounding vehicles are numbered V. (2) V (3) ,…,V (m) .

[0073] This disclosure does not consider situations such as negotiation or collaborative decision-making among vehicles. That is, traffic participants are not influenced by other participants or roadside equipment, and they make decisions and plans independently based on their own perception information to understand the preferences of surrounding vehicles. This represents the probability of a vehicle encountering a challenging event when completing a test task from an initial test scenario under a fixed vehicle driving strategy.

[0074] From the perspective of the vehicle being tested, its driving behavior is a crucial aspect reflecting the difficulty of the task. (Definition) Starting from a predetermined initial scenario, the test vehicle completes a pre-set test task, and the probability of performing a certain action is calculated by combining its own behavioral utility with that of surrounding vehicles (U, as a behavioral utility function, has no practical meaning; the purpose of choosing the utility value is primarily to rank the vehicle's behavioral performance). Among these, Let C be a random variable representing the estimated utility value of the tested vehicle's behavior at the current moment, where C = {C i Let}, i = 2, 3, ..., m be a combination of random variables representing the utility values ​​of the surrounding vehicles' behaviors. This disclosure does not consider surrounding vehicles that are far away, that is, it does not consider the phenomenon that there are cars in the virtual test scenario but they do not interact with the vehicle being tested.

[0075] Therefore, the challenge of virtual testing scenarios can be reflected in the behavior of the vehicle under test in the virtual testing scenario. The above formula can be rewritten using the law of total probability as follows:

[0076]

[0077] By altering the behavioral utility (or driving strategies) of surrounding vehicles, it is possible to find as many challenging scenarios as possible for the test vehicle. Theoretically, if we iterate through all possible combinations of behaviors from surrounding vehicles to generate all possible combinations of behaviors for the test vehicle in a specific scenario, we can divide the task into different levels of difficulty. However, iterating through all combinations is a non-deterministic polynomial hard (NP-hard) problem. Therefore, we need to classify and select the behavioral utility of surrounding vehicles to sample as many different difficulty levels of the set task as possible in a finite number of tests.

[0078] To simplify the formula below, we consider that the test requirements are determined, the basic test tasks Ψ and test indicators are set, and basic static and dynamic factors such as the road environment and the number and location of surrounding vehicles are limited.

[0079] For example, a semantically defined test requirement might be to test an autonomous vehicle's ability to recognize traffic conditions and safely steer at an intersection with four surrounding vehicles, all of which employ conservative strategies. Guided by this test requirement, the test scenario would specify the positions, attributes, and driving modes of the surrounding vehicles within the specific scenario. Based on the test metrics, specific autonomous driving test tasks would be designed, and a virtual test scenario would be initialized.

[0080] In other words, changes in the behavior of surrounding vehicles should be assessed under consistent test metrics, starting from the same initial virtual test scenario, based on different tasks Ψ. Under the same test scenario and the same test requirements, the test tasks are pre-defined and do not affect the results. Therefore, the above formula can be simplified to:

[0081]

[0082] When we adjust the behavior of a vehicle, such as V2, we can write it as:

[0083]

[0084] in:

[0085] Similarly, if we adjust the test task by changing the driving strategies of the two surrounding vehicles, the probability of a challenging event occurring in the test scenario is:

[0086]

[0087] in,

[0088]

[0089] As can be seen from the derivation, by changing the behavioral utility of surrounding vehicles and controlling the number of surrounding vehicles that are changed, test tasks of different difficulties can be obtained, thereby achieving the adjustment of the difficulty of the test task.

[0090] In an exemplary simulation scenario, we set up the utility function model of the following behavior of surrounding vehicles (the following distance and speed functions are respectively) as follows. We construct surrounding vehicles that adopt different driving strategies by adjusting the parameter values ​​in the model, and analyze their impact on the test vehicle to determine the difficulty of the test task.

[0091]

[0092] Where L(t) is the distance between the car in front and the car behind in the car-following model, x lead (t) and v lead (t) represents the distance and speed of the vehicle in front in the car-following model, respectively. follow (t) and v follow (t) represents the distance and speed of the following vehicle in the car-following model, respectively.

[0093] The speed of the following vehicle can then be calculated as follows:

[0094]

[0095] Among them, v max It is a configurable maximum speed (e.g., v). max L(t) can be set to 14 m / s, where T is the time interval (e.g., T can be set to 0.1 s). The ideal distance G is used to determine the final distance between the following vehicle and the target vehicle. If L(t) is less than G, the following vehicle will decelerate; if L(t) is greater than G, the following vehicle will accelerate until it reaches its maximum speed.

[0096] In this exemplary simulation scenario, we change the ideal distance G of the following vehicle and the maximum speed v. max The Gaussian distribution parameters are used to distinguish the following behavior of different strategies. This disclosure defines aggressive following behavior as maintaining a smaller ideal distance and having a slightly larger mean and variance of the maximum speed distribution, while conservative following behavior maintains a larger ideal distance and has smaller mean and variance of the maximum speed distribution.

[0097] In another exemplary simulation scenario, we adopt the MOBIL lane-changing behavior model as the specific lane-changing behavior utility function, as follows: the behavior utility function U is the sum of acceleration gains U acceleration If the total revenue exceeds the threshold Δα, a lane-changing decision is made.

[0098] U acceleration =Δαc+p(Δαn+Δαo)>Δα;

[0099] Where Δαc is the acceleration gain of the vehicle, p is the lane change coefficient, Δαn is the acceleration gain of the vehicle following in the target lane, Δαo is the acceleration gain of the vehicle following in the current lane, and Δα is the preset lane change threshold. Lane change is allowed when the sum of acceleration gains after lane change is greater than the preset lane change threshold Δα.

[0100] In this exemplary simulation scenario, we differentiate lane-changing behaviors based on different strategies by changing the lane-change coefficient p and the preset lane-change threshold Δα. For example, we can define the lane-change coefficient p for vehicles with an aggressive driving strategy as 0.1, meaning that the lane-change decision does not consider the safety of other vehicles; while the lane-change coefficient p for vehicles with a conservative driving strategy is 1, meaning that the lane-change decision considers the safety of other vehicles too much.

[0101] In this disclosure, we define the difficulty of the test task as proportional to the total number of challenging events encountered by the test vehicle in the scenario. We consider a challenging event to occur when the time to collision (TTC) between the two vehicles is greater than 1 second and less than 2.5 seconds. When the TTC is less than 1 second, we consider an unavoidable collision to be inevitable, one that even a human driver would be unable to brake safely. When the TTC is greater than 2.5 seconds, we consider that the vehicle will not encounter any difficulties related to following other vehicles.

[0102] The following section uses a vehicle-switching simulation scenario as an example to explain in detail the method for generating the autonomous driving test scenario library provided in this disclosure. When the vehicle under test and the initial scenario parameters remain unchanged, this disclosure can sample more challenging vehicle-switching scenarios and tasks by simultaneously changing the driving behavior of the vehicle switching and other surrounding vehicles.

[0103] like Figure 3 As shown, this disclosure provides an autonomous driving simulation process, which mainly includes the following steps: simulation initialization, behavior sampling, and intelligent evaluation.

[0104] Simulation initialization

[0105] First, an initial virtual test scenario needs to be set up as the test environment. In this initial test environment, the spatiotemporal layout of the scenario needs to be constructed and the components of the scenario need to be specified.

[0106] Without considering roadside units or other equipment support, this disclosure designs a typical one-way two-lane road vehicle-changing scenario. The distance between the starting position of the tested autonomous vehicle and the baseline is set as R, and the distance of the vehicle to the nearest lane line is R', as shown below. Figure 4As shown. The relationship between the tested automated vehicle (AV) and surrounding vehicles is described using a polar coordinate method with the tested vehicle as the axis. Let the distances between the four vehicles be (R1, D1), (R2, D2), and (R3, D3), where R... i This indicates the absolute distance between the centers of the vehicles; the AV driving direction is relative to the V. (i+1) The angular deviation between driving directions (i.e., the angular difference between the center of the other vehicle and the center of the AV, with the AV's front direction as the positive direction) is D. i .

[0107] The vehicle status information in the scene includes the vehicle's speed and acceleration. Specifically, V... (i) Velocity and acceleration constitute a pair of state variables (v i ,a i ), where i = 1, 2, 3, 4, and we set V (2) Prepare surrounding vehicles in the scene for lane changes. V (1) For the vehicle under test, and V (3) and V (4) They are all in a state of free driving.

[0108] The initialization parameters for the test scenario are shown in Table 1.

[0109]

[0110]

[0111] Table 1

[0112] Behavioral sampling

[0113] To generate test scenarios of varying difficulty, this disclosure employs a semantic-driven approach to describe the rich driving behaviors of a vehicle (i.e., different driving strategies can be described using language). Specifically, during the sampling process, this disclosure selects a behavioral utility function to describe the vehicle's driving behavior. This function takes the external environment and its own parameters as input, and outputs the vehicle's behavioral utility value at that moment. By adjusting the hyperparameters in the utility function, the heterogeneous output of the behavioral utility function is characterized.

[0114] Consider a utility function whose input is a series of independent variables: x1, x2, ..., x n For any vehicle V (i) Define the vehicle's behavioral utility as U. i We consider behavioral utility to be a weighted sum of these independent variables, therefore the basic formula for the utility function can be written as: Where, α k,i The weighting coefficients of the variables.

[0115] Based on observations, human drivers' driving strategies can generally be categorized into aggressive and conservative approaches. We believe that vehicle driving behavior is governed by a binary strategy, namely aggressive and conservative driving strategies, to reflect the "internal realism" of the vehicle, C = {'aggressive', 'conservative'}.

[0116] In this disclosure, the utility of driving behavior is defined as the probability of behavior switching. With consistent input variables to the utility function, the driving strategy adopted by a traffic participant influences the probability of performing the driving behavior.

[0117] Intelligent assessment

[0118] Intelligent Indicator Selection: In the intelligent safety test, we choose a Boolean value to determine whether the vehicle successfully achieves the safety objective within the appropriate time and space range. We determine the Boolean value by setting a threshold for Time-of-Collision (TTC). TTC, as one of the most commonly used safety performance indicators, can effectively reflect the safety status of autonomous vehicles. If the TTC exceeds the threshold, it indicates that the tested vehicle encountered a challenging event in the test scenario; in this case, the Boolean value B = 0. Otherwise, B = 1. Therefore, in this disclosure, the difficulty of the test scenario is defined as the number of challenging events encountered by the tested vehicle. The more challenging events the tested vehicle encounters, the greater the difficulty of the virtual test scenario in which the tested vehicle is located.

[0119] like Figure 5A As shown, the driving strategies of surrounding vehicles were adjusted in two test scenarios, Scenario 1 and Scenario 2, to obtain test scenarios with different levels of difficulty for the test vehicle. In Scenario 1, we only changed V... (2) In scenario 1, we changed the driving strategy, and in scenario 2, we simultaneously changed V. (2) and V (3) The driving strategy is described. We also provide a baseline scenario where all surrounding vehicles maintain normal driving strategies. In this embodiment, we set the TTC threshold to 2 seconds. We represent the results of 200 repeated experiments using a box plot, as shown below. Figure 5A As shown, Median is the median or mean value, and IQR is the interquartile range (also known as the interquartile range). From Figure 5A As can be seen, Scenario 2 presents more challenging scenarios compared to Scenario 1; similarly, compared to the baseline scenario, the tested vehicle encountered more challenging scenarios in both Scenario 1 and Scenario 2. This demonstrates that by changing the driving strategies of different numbers of surrounding vehicles, this disclosure causes the tested vehicle to encounter more challenging events, thereby sampling virtual test scenarios of varying difficulty.

[0120] Figure 5BAnother schematic diagram of experimental results provided for an exemplary embodiment of this disclosure, in Figure 5B In this paper, we will compare a lane-changing scenario library obtained using the autonomous driving test scenario library generation method disclosed herein with a lane-changing scenario library extracted by NGSIM. The NGSIM (Next Generation Simulation) dataset is driving data of US highways collected by the Federal Highway Administration (FHMA). It includes the vehicle driving conditions of all vehicles on roads such as US101 and I-80 over a time period. The data is acquired using cameras and then processed into trajectory point records. In the process of generating this lane-changing scenario library, the initial virtual test scenario we used is as follows: Figure 4 As shown, during the test, we assume that the driving strategy of the vehicle under test will not be changed, and we choose to traverse V. (2) V (3) and V (4) The driving strategies are shown in Table 2. For each strategy, we simulated 200 lane-changing scenarios to create a lane-changing scenario library, in order to reduce errors caused by randomness. Figure 5B As can be seen, the autonomous driving test scenario library generated in this disclosure is quite similar to the lane-changing scenario library extracted by NGSIM. In other words, by changing the driving strategies of surrounding vehicles during the execution of the test task, we can obtain a rich test scenario library that is quite similar to the test scenarios in the real scenario library in a limited number of tests.

[0121]

[0122] Table 2

[0123] This disclosure also provides an apparatus for generating an autonomous driving test scenario library, including a memory and a processor connected to the memory, the memory being used to store instructions, and the processor being configured to execute the steps of the autonomous driving test scenario library generation method as described in any embodiment of this disclosure based on the instructions stored in the memory.

[0124] like Figure 6As shown, in one example, the autonomous driving test scenario library generation device may include: a processor 610, a memory 620, and a bus system 630, wherein the processor 610 and the memory 620 are connected via the bus system 630, the memory 620 is used to store instructions, and the processor 610 is used to execute the instructions stored in the memory 620. Specifically, the processor 610 sets an initial virtual test scenario and test task, the initial virtual test scenario including the vehicle under test and surrounding vehicles; constructs a vehicle behavior utility model, the vehicle behavior utility model including multiple different driving strategies; executes the test task on the vehicle under test, and changes the driving strategies of surrounding vehicles during the execution of the test task; obtains test results, and generates a test scenario library based on the test results, the test scenario library including multiple virtual test scenarios with different difficulties for the vehicle under test and the driving strategies of surrounding vehicles corresponding to each virtual test scenario.

[0125] It should be understood that processor 610 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0126] Memory 620 may include read-only memory and random access memory, and provides instructions and data to processor 610. A portion of memory 620 may also include non-volatile random access memory. For example, memory 620 may also store device type information.

[0127] In addition to a data bus, the bus system 630 may also include a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 6 The general labeled all buses as Bus System 630.

[0128] In implementation, the processing performed by the processing device can be accomplished through integrated logic circuits in the hardware of the processor 610 or through software instructions. That is, the method steps of this embodiment can be executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other storage media. This storage medium is located in memory 620, and the processor 610 reads information from memory 620 and, in conjunction with its hardware, completes the steps of the aforementioned method. To avoid repetition, further details are omitted here.

[0129] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the autonomous driving test scenario library generation method as described in any embodiment of this disclosure. The method of generating the autonomous driving test scenario library by executing executable instructions is essentially the same as the method provided in the above embodiments of this disclosure, and will not be described in detail here.

[0130] In some possible implementations, various aspects of the autonomous driving test scenario library generation method provided in this application can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to perform the steps in the autonomous driving test scenario library generation method according to various exemplary embodiments of this application described above. For example, the computer device can execute the autonomous driving test scenario library generation method described in the embodiments of this application.

[0131] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0132] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0133] While the embodiments disclosed herein are as described above, the content is merely for the purpose of facilitating understanding of this disclosure and is not intended to limit the invention. Any person skilled in the art may make any modifications and changes to the form and details of the implementation without departing from the spirit and scope of this disclosure; however, the patent protection scope of this invention shall still be determined by the scope defined in the appended claims.

Claims

1. A method for generating an autonomous driving test scenario library, characterized in that, include: Set up an initial virtual test scenario and test tasks, wherein the initial virtual test scenario includes the vehicle under test and surrounding vehicles; A vehicle behavior utility model is constructed, which includes a variety of different driving strategies; The test task is performed on the vehicle under test, and the driving strategies of surrounding vehicles are changed during the execution of the test task. Obtain test results and generate a test scenario library based on the test results. The test scenario library includes multiple virtual test scenarios with different difficulties for the tested vehicle and driving strategies for surrounding vehicles corresponding to each virtual test scenario. The difficulty of the virtual test scenario is defined as follows: , For the tested vehicle to complete the test task The total number of challenging events encountered during the process, where a challenging event is defined as a collision between the tested vehicle and surrounding vehicles occurring within a preset threshold range. For the test task Total number of samples, Let t be a natural number and t be the test time. For testing tasks Total number of samples, For natural numbers, , For the test task With test tasks This represents the number of challenging events encountered by the vehicle under test during concurrent tasks. To mitigate the weighting of concurrent task difficulty, 0 ≤ ≤1, the concurrent tasks are defined as two or more tasks that are not related and have no logical temporal or spatial order, and can be triggered at the same time and place.

2. The method according to claim 1, characterized in that, The various driving strategies include at least one aggressive driving strategy and at least one conservative driving strategy; the aggressive driving strategy tends to maintain a smaller minimum safe distance from the vehicle in front during following or lane changing than the conservative driving strategy tends to maintain a smaller minimum safe distance from the vehicle in front during following or lane changing.

3. The method according to claim 1, characterized in that, The process of changing the driving strategies of surrounding vehicles during the execution of the test task includes: Determine all possible combinations of driving strategies for the surrounding vehicles; During the execution of the test task, each possible combination is tested one by one.

4. The method according to claim 1, characterized in that, The vehicle behavior utility model includes a car-following behavior model, which is expressed as follows: ； in, The distance between the car in front and the car behind. These represent the distance and speed of the vehicle in front, respectively. These represent the distance and speed of the following vehicle, respectively. This is the maximum acceleration; The formula for calculating the speed of the following vehicle is: ； Wherein, the ideal distance G and the maximum speed For the variables in the vehicle behavior utility model, when When the speed is less than G, the following vehicle slows down; when Greater than At that time, the following car accelerated until it reached its maximum speed; The aforementioned changes to the driving strategy of surrounding vehicles include: altering the ideal distance G and maximum speed of the following vehicle. The value of .

5. The method according to claim 1, characterized in that, The vehicle behavior utility model includes a lane-changing behavior model, which is expressed as follows: ； in, For acceleration benefit function, The acceleration gain of this vehicle is represented by p, where p is the lane change coefficient. For the acceleration gain of the vehicle behind in the target lane, For the acceleration benefit of the vehicle behind in this lane, The acceleration benefit function after a lane change is defined by a preset lane change threshold. The value is greater than the preset lane change threshold. Lane changing is permitted at this time; The method of changing the driving strategy of surrounding vehicles includes: changing the lane change coefficient p and the preset lane change threshold. The value of .

6. An apparatus for generating an autonomous driving test scenario library, characterized in that, The method includes a memory; and a processor connected to the memory, the memory being used to store instructions, the processor being configured to perform the steps of the autonomous driving test scenario library generation method as described in any one of claims 1 to 5 based on the instructions stored in the memory.

7. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method for generating an autonomous driving test scenario library as described in any one of claims 1 to 5.

Images