Method and device for determining capability boundary of automatic driving system

Abstract

The invention provides a method and device for determining the capability boundary of an automatic driving system, and relates to the technical field of automatic driving, and the method comprises the steps: defining a test scene space of a to-be-tested automatic driving system; using a pre-trained boundary exploration model to perform multiple rounds of iterative search in the test scene space, and determining the ability boundary of the automatic driving system according to the test result of the automatic driving system in the test scene searched each time; each capability boundary is a group of boundary test scene pairs, the test result of the automatic driving system under a first boundary test scene in each group of boundary test scene pairs is different from the test result under a second boundary test scene, and the spatial distance between the first boundary test scene and the second boundary test scene is smaller than a preset distance threshold value; and the boundary exploration model adjusts an exploration strategy according to a test result obtained by the executed iterative search. Therefore, the capability boundary of the automatic driving system under different conditions can be identified more efficiently and accurately.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and more particularly to a method and device for determining a capability boundary of an autonomous driving system. Background Art

[0002] With the rapid development of science and technology, self-driving cars have become a research field that has attracted much attention in intelligent transportation systems. By integrating advanced sensor technology, artificial intelligence algorithms and control systems, it is expected that they can reduce traffic accidents caused by human errors, improve road safety, and provide a more efficient and convenient way of travel. However, the safety of self-driving cars is one of the key challenges for their large-scale commercial use. In order to ensure the safe operation of vehicles in various complex environments, it is necessary to accurately evaluate and explore the boundaries of their self-driving capabilities.

[0003] Although current autonomous driving technology has demonstrated good performance under certain conditions, its capability boundaries, that is, the range of scenarios in which it can operate safely, are still limited. Capability boundaries refer to the range of driving scenarios in which the autonomous driving system can operate normally without failure or making wrong decisions. These boundaries define the various situations that the autonomous driving system can handle, including but not limited to different weather conditions, road conditions, traffic density, and interactions with other road users.

[0004] Exploring the boundaries of autonomous driving capabilities is crucial to understanding its limitations and further improving it. The existing practice is usually scenario-based testing, which evaluates the performance of autonomous driving systems through simulations or experiments on actual roads. Although this method is direct and effective, the scene space is extremely large due to the wide variety and variety of driving scenarios in reality. Faced with such a large collection of scenarios, using traditional means to conduct a comprehensive search is time-consuming and laborious, greatly increasing development costs and technical challenges. In addition, since traffic conditions in the real world are extremely complex and unpredictable, it is difficult to fully reflect all possible problems by relying only on a limited number of test scenarios generated by artificial construction or historical data, resulting in limited effectiveness and generalization of the test results. Summary of the invention

[0005] In view of this, the purpose of the present application is to provide a method and device for determining the capability boundary of an autonomous driving system, so as to solve the problems existing in the prior art of long search time, high cost, and limited validity and generalization ability of test results, so as to achieve the purpose of efficiently and accurately determining the capability boundary of an autonomous driving system.

[0006] An embodiment of the present application provides a method for determining a capability boundary of an autonomous driving system, the method comprising: Defining a test scenario space of the autonomous driving system to be tested; wherein the test scenario space includes multiple test dimensions, and the scenario parameters under each test dimension have a corresponding parameter value range; and forming a test scenario by combining parameter values ​​of the scenario parameters under each test dimension; A pre-trained boundary exploration model is used to perform multiple rounds of iterative searches in the test scene space, and the capability boundary of the autonomous driving system is determined according to the test results of the autonomous driving system in each test scene searched; wherein each capability boundary is a group of boundary test scene pairs, and the test results of the autonomous driving system in the first boundary test scene in each group of boundary test scene pairs are different from the test results in the second boundary test scene, and the spatial distance between the first boundary test scene and the second boundary test scene is less than a preset distance threshold; the boundary exploration model adjusts the exploration strategy according to the test results obtained from the iterative search that has been performed.

[0007] The embodiment of the present application further provides a device for determining a capability boundary of an autonomous driving system, the device comprising: A definition module, used to define a test scenario space of the autonomous driving system to be tested; wherein the test scenario space includes multiple test dimensions, and the scenario parameters under each test dimension have a corresponding parameter value range; a test scenario is formed by combining parameter values ​​of the scenario parameters under each test dimension; A search module is used to perform multiple rounds of iterative searches in the test scene space using a pre-trained boundary exploration model, and determine the capability boundary of the autonomous driving system according to the test results of the autonomous driving system in each test scene searched; wherein each capability boundary is a group of boundary test scene pairs, and the test results of the autonomous driving system in the first boundary test scene in each group of boundary test scene pairs are different from the test results in the second boundary test scene, and the spatial distance between the first boundary test scene and the second boundary test scene is less than a preset distance threshold; the boundary exploration model adjusts the exploration strategy according to the test results obtained from the iterative search that has been performed.

[0008] An embodiment of the present application also provides an electronic device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the method for determining the capability boundary of the autonomous driving system as described above are performed.

[0009] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for determining the capability boundary of the autonomous driving system as described above are executed.

[0010] The embodiment of the present application provides a method and device for determining the capability boundary of an autonomous driving system, constructs a scene space covering multiple test dimensions, and uses a pre-trained boundary exploration model to perform multiple rounds of iterative searches in the scene space, and adaptively and dynamically adjusts the search strategy according to the obtained search results, so as to quickly determine the capability boundary of the autonomous driving system. The capability boundary is defined as a pair of boundary test scenarios with different test results and a spatial distance less than a preset distance threshold. In this way, the capability boundary of the autonomous driving system under different conditions can be identified more efficiently, ensuring the comprehensiveness and accuracy of the test results, and reducing the large amount of time and resources required for traditional testing methods.

[0011] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0013] Figure 1 A flow chart showing a method for determining a capability boundary of an autonomous driving system provided in an embodiment of the present application is shown; Figure 2 A flow chart of a method for training a boundary exploration model provided in an embodiment of the present application is shown; Figure 3 A schematic diagram showing the structure of a device for determining a capability boundary of an autonomous driving system provided in an embodiment of the present application is shown; Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0014] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0015] Research has found that the existing approach to exploring the boundaries of autonomous driving capabilities is usually scenario-based testing, which evaluates the performance of autonomous driving systems through simulations or experiments on actual roads. Although this method is direct and effective, the variety and variety of driving scenarios in reality make the scenario space extremely large. Faced with such a large collection of scenarios, using traditional methods to conduct a comprehensive search is time-consuming and laborious, greatly increasing development costs and technical challenges. In addition, since traffic conditions in the real world are extremely complex and unpredictable, it is difficult to fully reflect all possible problems by relying only on a limited number of test scenarios that are manually constructed or generated from historical data, resulting in limited effectiveness and generalization of the test results.

[0016] Based on this, an embodiment of the present application provides a method for determining the capability boundary of an autonomous driving system to solve the problems of long search time, high cost, and limited validity and generalization ability of test results in the prior art, so as to achieve the purpose of efficiently and accurately determining the capability boundary of an autonomous driving system.

[0017] See also Figure 1 , Figure 1 A flowchart of a method for determining the capability boundary of an autonomous driving system provided in an embodiment of the present application. Figure 1 As shown in , the determination method provided in the embodiment of the present application includes: S101. Define a test scenario space for the autonomous driving system to be tested.

[0018] The test scene space includes multiple test dimensions, and the scene parameters under each test dimension have a corresponding parameter value range.

[0019] In specific implementation, multiple test dimensions and parameter value ranges corresponding to the scenario parameters under each test dimension can be constructed according to the actual application requirements and safety standards of the autonomous driving system to be tested. A test scenario is formed by combining the parameter values ​​of the scenario parameters under each test dimension, thereby forming a test scenario space covering a variety of driving conditions including a large number of test scenarios, so that the test scenario space can reflect various situations that may be encountered in the real world as widely as possible. These scenario parameters will have different effects on the performance of the autonomous driving system, thereby determining its "capability boundary". The goal of the subsequent steps is to find which parameter combinations under each test dimension will cause the autonomous driving system to fail, thereby determining the system's capability boundary. Exemplarily, the test dimensions may include self-driving vehicle driving parameters (such as vehicle speed, acceleration, steering angle), traffic conditions (such as traffic flow density), road surface conditions, and weather conditions.

[0020] At the same time, the autonomous driving system to be tested is connected to the scene space to ensure that the autonomous driving system can receive data input from the simulated test scene space and respond accordingly.

[0021] In one example, the test scene space is defined as , then:

[0022] in, Representative scenario variables under each test dimension. is a continuous vector with a defined maximum value. and minimum value .by For example, It can be visualized as the vehicle speed, which is expressed as ,in represents the minimum speed allowed for the vehicle, and is the maximum speed that the vehicle can reach. The entire test scene space Composed of all these variables, a multidimensional parameter space is formed. Each point Represents a specific test scenario in multidimensional space, i.e. . Represents a specific test scenario in the test scenario space.

[0023] S102. Use a pre-trained boundary exploration model to perform multiple rounds of iterative searches in the test scenario space, and determine the capability boundary of the autonomous driving system based on the test results of the autonomous driving system in each searched test scenario.

[0024] In the solution of the present application, each capability boundary is a set of boundary test scene pairs (that is, two scene points in the test scene space), and a pair of capability boundary points can be used. and In each set of boundary test scene pairs, the test result of the autonomous driving system in the first boundary test scene is different from the test result in the second boundary test scene, and the spatial distance between the first boundary test scene and the second boundary test scene is less than a preset distance threshold.

[0025] Here, the test results include pass and fail, and the test result under the first boundary test scenario is different from the test result under the second boundary test scenario, which specifically means: the test result under the first boundary test scenario is pass, and the test result under the first boundary test scenario is fail; or, the test result under the first boundary test scenario is fail, and the test result under the first boundary test scenario is pass.

[0026] This solution provides a way to define the capability boundary. The different test results under the two test scenarios indicate that there is a capability boundary of the autonomous driving system between the two test scenarios; and when the two test scenarios are close enough, it can be determined that a capability boundary is approached. The degree of closeness can be described by the spatial distance between the test scenarios. When the spatial distance is less than the preset distance threshold, the two test scenarios can be considered to be close enough, that is, a capability boundary has been found. Among them, the preset distance threshold can be determined during the model training process according to the parameter value range (value interval) corresponding to the scenario parameters under each test dimension.

[0027] The boundary exploration model in the embodiment of the present application can be a pre-trained neural network model based on deep reinforcement learning (DRL), which is trained through a large amount of generated data and can adaptively execute the best strategy for boundary search to effectively search for the capability boundary of autonomous driving in the test scene space. During multiple rounds of iterative search, the boundary exploration model in the embodiment of the present application can adaptively adjust the exploration strategy for the next round of iterative search based on the test results obtained from one or more rounds of iterative search that have been executed.

[0028] More specifically, updating the exploration strategy means that the boundary exploration model does not simply perform proximity traversal exploration, but instead updates its own exploration strategy based on the identified capability boundary points and the direction of gradient descent, and optimizes the search path in the test scenario space in order to find more capability boundary points more efficiently.

[0029] In a possible implementation, step S102 may include: S1021. For any round of iterative search, use the boundary exploration model to locate the capability boundary found in the previous round of iterative search in the test scenario space, and continue the current round of iterative search.

[0030] Here, after the first round of iterative search, in each subsequent round of iterative search, the boundary exploration model will first locate the position of the capability boundary point pair found in the previous round of iterative search in the test scenario space, that is, and ; After that, the updated optimal search strategy is executed to perform this round of iterative search on the boundary points near the located capability boundary points.

[0031] S1022. During this round of iterative search, whenever a group of test scene pairs is searched in the test scene space, the group of test scene pairs is imported into the autonomous driving system for testing.

[0032] In this step, whenever the boundary exploration model searches for a set of test scenario pairs in the test scenario space, the data of the set of test scenario pairs is input into the autonomous driving system for testing, and the response data of the autonomous driving system to each test scenario is obtained, thereby determining the test results of the autonomous driving system in each test scenario.

[0033] S1023. Use the boundary exploration model to determine the capability boundary of the autonomous driving system according to the test results of the autonomous driving system under each searched test scenario pair.

[0034] In this step, after obtaining a set of test results, the boundary exploration model can determine whether it is the capability boundary of the autonomous driving system based on the definition of the capability boundary mentioned above; if so, end this round of iterative search; if not, continue searching for the next set of test scenario pairs until a capability boundary is found in this round of iterative search.

[0035] In a possible implementation, step S1023 may include: Step a1: for a test scene pair searched for the first time, obtain a first test result of the autonomous driving system in the first test scene and a second test result in the second test scene in the test scene pair searched for this time.

[0036] Similarly, obtain the first test result of the autonomous driving system in the first test scenario and the second test result in the second test scenario in the test scenario pair first searched, and determine whether the first test result and the second test result are the same.

[0037] Step a2: When the first test result and the second test result are different, determine the spatial distance between the first test scene and the second test scene in the test scene pair.

[0038] The different test results include: the first test result is passed, and the second test result is not passed; or the first test result is not passed, and the second test result is passed. The parameter values ​​of the scene parameters under each test dimension in the test scene are used as the spatial coordinates of the test scene point in the test scene space, and the spatial distance between the first test scene and the second test scene can be determined by the distance formula.

[0039] Step a3: If the spatial distance is less than the preset distance threshold, the test scenario pair is determined as a capability boundary and this round of iterative search ends.

[0040] Step a4: If the spatial distance is not less than the preset distance threshold, the boundary exploration model uses a method of reducing the spatial distance to re-search for a set of new test scene pairs based on the test scene pair.

[0041] The method of reducing the spatial distance may include moving one point in the test scene pair toward another point, while the other point remains unchanged; or, the two points move toward each other at the same time. This can be expressed by the formula:

[0042] and / or,

[0043] In the formula, represents the first test scenario in the original test scenario pair, represents the second test scenario in the original test scenario pair; Indicates the spatial distance between the two; , used to control the convergence rate; represents the first test scenario in a new test scenario pair, represents the second test scenario in a new test scenario pair; Take specific still It should be determined according to the relative position of the test scene.

[0044] Step a5: reacquire a new first test result of the autonomous driving system in the first test scenario and a new second test result in the second test scenario in the new set of test scenario pairs.

[0045] Step a6: When the new first test result and the new second test result are different, return to the step of determining the spatial distance between the first test scene and the second test scene in the new test scene pair, that is, return to the above step a2.

[0046] Step a7: When the new first test result and the new second test result are the same, the new test scene with the changed test result in the new test scene pair and the original test scene corresponding to the new test scene with the changed test result are recombined to obtain a new test scene pair; and the process returns to the step of determining the spatial distance between the first test scene and the second test scene in the new test scene pair. That is, the process returns to the above step a2.

[0047] Here, when using the method of reducing the spatial distance to re-search a set of new test scenario pairs based on the original test scenario pairs, the new first test result may be the same as the new second test result. This means that due to the unknown nature of the capability boundary, the reduced spatial distance is too much, and one of the new test scenarios has crossed the capability boundary.

[0048] Exemplarily, the original test scenario pair is represented as [A1, B1]. The test result of the autonomous driving system in test scenario A1 is passed, and the test result in test scenario B1 is failed, which indicates that there is a certain capability boundary between A1 and B1. By reducing the spatial distance, test scene A1 becomes A2, and test scene B1 becomes B2, and a new set of test scenario pairs [A2, B2] are obtained by combining them.

[0049] The test result of the autonomous driving system in test scenario A2 is a failure, and the test result in test scenario B1 is also a failure, which are the same; comparing the new test scenario pair [A2, B2] and the original test scenario pair [A1, B1], it can be seen that the new test scenario with a changed test result is A2, and the original test scenario corresponding to A2 is A1. At this time, a new test scenario pair [A1, A2] is recombined. Obviously, the test results corresponding to A1 and A2 are different, and the spatial distance has also been shortened, that is, [A1, A2] is closer to the capability boundary than [A1, B1].

[0050] On the other hand, during this round of iterative search, for the test scenario pair searched for the first time, when the first test result and the second test result are the same, the test scenario pair is discarded, and a new set of test scenario pairs is searched again based on the capability boundaries successfully searched in the previous round of iterative search; and it is used as the test scenario pair searched for the first time, and the above step a1 is returned to be executed.

[0051] Furthermore, the determination method also includes: S103: Export the capability boundary of the autonomous driving system.

[0052] S104. When the boundary test scenario pairs corresponding to each capability boundary form a continuous boundary in the test scenario space, or when at least one test dimension reaches the spatial limit of the test scenario space, it is determined that the capability boundary of the autonomous driving system has been explored, and an autonomous driving capability boundary scenario set is formed based on the determined capability boundary of the autonomous driving system.

[0053] Here, the autonomous driving capability boundary scenario set is a collection of specific test scenarios, which consists of all the scenarios that meet the above and The scenario consists of a pair of capability boundary points that define the conditions. Since it can directly expose the limitations or deficiencies of autonomous driving, such scenarios have extremely high testing value.

[0054] For the above steps S103 and S104, whenever the boundary exploration model identifies a new autonomous driving capability boundary, the corresponding capability boundary point pair is exported and stored, and this process continues until all capability boundaries are explored. All capability boundaries are explored means that as the exploration process progresses, all data identified as capability boundary points will be recorded.

[0055] For an autonomous driving system, the scene space boundary points representing its capability boundary should be continuous, so it can be determined in the following way: when the boundary test scene pairs corresponding to each capability boundary form a continuous boundary in the test scene space, it is determined that the capability boundary of the autonomous driving system has been explored. Exemplarily, it can be formalized as a continuous boundary line in a two-dimensional space, as a continuous boundary surface in a three-dimensional space, and so on. In higher-dimensional spaces, computer software can also be used to determine whether the scene space boundary points are continuous. Alternatively, when at least one test dimension reaches the spatial limit of the constructed test scene space, it can also be determined that the capability boundary of the autonomous driving system has been explored.

[0056] The derived capability boundary points together constitute a continuous performance boundary, representing the capability range of the current autonomous driving system within the test scenario space.

[0057] See also Figure 2 , Figure 2 A flowchart of a method for training a boundary exploration model provided in an embodiment of the present application.

[0058] In a possible implementation, the boundary exploration model is a dual-depth Q network; wherein the main network is used to use - A greedy strategy is used, and according to the test results of the autonomous driving system under the test scenario, an execution action is selected from the action space; the target network is used to generate a Q value for the main network.

[0059] Here, a dual deep Q network (DDQN) and - Greedy strategy ( -greedy), which can ensure the model's ability to explore different strategies in the early stage and avoid the model falling into the local optimum in the later stage of training.

[0060] DDQN uses two different Q networks to select actions and evaluate action values ​​respectively, in order to reduce the overestimation of Q values. and target network . Main Network It is mainly used to select actions, that is, to find the action with the maximum potential reward in each state. Responsible for generating target Q values ​​for the main network, thereby evaluating the long-term value of the actions selected by the main network. The target network parameters are consistent with the main network, but the update speed is slower, and soft updates are usually used to ensure the stability of training.

[0061] The update rules in the training strategy are: and actions Get rewards , and update the Q value of the main network. DDQN uses two networks to update alternately, and the specific update formula is:

[0062] in is the immediate reward of the current state-action pair, is the discount factor used to control the impact of future rewards, Indicates that in the next state The Q-values ​​of the target network provide an estimate of future rewards during the update process, ensuring that the model does not overreact to short-term rewards during training.

[0063] Said - Greedy strategy means that in reinforcement learning training, in order to ensure that the model can explore the optimal strategy, - Greedy strategy, which randomly selects actions with a certain probability to increase the possibility of model exploration diversity. -The greedy strategy is used to balance the exploration and utilization of scene space points. In the early stages of training, a higher exploration rate is set. value, and gradually decreases as the training progresses. The value of .

[0064] Based on the test results, - Greedy strategy in each state Select an action , and execute the action to update the test point position. The choice of action depends on the model The policy output ensures a balance between exploration and exploitation. After executing the action, the model enters a new state , which is the updated test point position.

[0065] It should be noted that, in addition to - In addition to the greedy strategy, other methods of balancing exploration and exploitation can also be used, such as the Softmax strategy, the Upper Confidence Bound (UCB) strategy, etc. In addition to the dual deep Q network, the boundary exploration model can also be Dueling DQN, Rainbow DQN or Proximal Policy Optimization (PPO). These algorithms can also provide efficient policy learning capabilities and may perform well when dealing with high-dimensional state space and action space. For example, Dueling DQN can further improve the accuracy of value estimation by decomposing the value function into a state value function and an advantage function; Rainbow DQN integrates a variety of improvement techniques, such as priority experience replay, multi-step rewards, etc., which can enhance learning efficiency and performance.

[0066] like Figure 2 As shown in , the training method provided in the embodiment of the present application includes: S201. Set the state space, action space and state transfer function of the boundary exploration model.

[0067] The training environment of the boundary exploration model includes state space, action space and state transfer function. The iterative search process of the boundary of autonomous driving capability can be regarded as a Markov process, except that the state transfer step of the Markov process is not a time-related step but a step for each test.

[0068] The state space represents the set of all possible states of the agent in the environment; in the embodiment of the present application, each optional state included in the state space is a set of test scenario pairs, and the boundary exploration process can use a finite state set To represent, each state Represents the combination of test points currently explored. Assume that a pair of test points is , respectively represent the test point and the reference point (corresponding to S102 and ), and the comparison of the test results between these two points is used as the basis for boundary exploration. In addition, similar to the aforementioned step S101, it is also necessary to define the scene space of the autonomous driving system, connect the object to be tested as any available autonomous driving system, and normalize the variables in the scene space.

[0069] The action space defines all possible actions that the agent can take in each state; in each iterative search, the boundary exploration model chooses how to adjust the current test point position to form the action space In the embodiment of the present application, the optional actions include: 1. reducing the spatial distance between two boundary points (for selecting this action when approaching a boundary point); 2. randomly selecting a new point for testing (selecting this action when the test results of the two points are the same).

[0070] The state transition function defines how to move from one state to another after taking an action in a given state; as mentioned earlier, the boundary exploration environment satisfies the Markov property, that is, This state transition function represents the current state and the selected action Determine the next state .

[0071] S202: Randomly select an initial state from the state space.

[0072] S203. Select an execution action from the action space according to the test result of the automatic driving system in the test scenario pair corresponding to the initial state.

[0073] S204: Generate a transfer state according to the initial state, the execution action and the state transfer function.

[0074] Here, the autonomous driving system updates the state at each exploration step and provides environmental feedback. The formula can be expressed as:

[0075] S205. Generate a reward function according to the test result of the autonomous driving system in the current test scenario pair corresponding to the transfer state, and update the parameters of the boundary exploration model according to the reward function.

[0076] S206: Determine whether the current test scenario pair corresponding to the transfer state is a capability boundary.

[0077] Here, the test point and The distance in the scene space should be infinitely small in theory, so it is necessary to set a minimum distance threshold to determine the convergence of the point-to-point distance: (i.e. the distance between two points reaches the minimum threshold), the boundary is considered to be found and the boundary point is recorded. The distance in the test scene space keeps shrinking, and when the results of two test points are different and the distance is less than the threshold, it means that the capability boundary has been found.

[0078] If it is not a capability boundary, the process returns to the step of selecting an execution action from the action space (ie, the above-mentioned step S203 ) until it is determined that the test scenario pair corresponding to the regenerated transfer state is a capability boundary.

[0079] S207: If it is the capability boundary, determine whether the boundary exploration model has converged.

[0080] If not, return to the step of randomly selecting an initial state from the state space (ie, the above step S201 ) until the boundary exploration model converges, thereby obtaining the trained boundary exploration model.

[0081] During the training process, multiple rounds of execution are required to ensure that the model finds the boundary points of the system in all possible dimensional combinations. In this regard, the overall convergence condition of the boundary exploration model M is that the training loss function tends to be stable, that is, the global convergence standard of the model is reached. Specifically, after each action is executed, the experience replay is used to update the reward and the new state The parameters of the target network are updated at the same time , every Step 2: Copy the parameters of the main network to the target network. When the training loss of the model remains stable for at least N consecutive rounds, and the discovered boundary points are sufficient to cover all important scene combinations, the model is considered to have converged as a whole. After training is completed, it can execute the optimal boundary search strategy to explore the performance boundaries of autonomous driving.

[0082] For step S205, the reward function is a mechanism that provides feedback signals to guide model optimization and gradient update in the deep reinforcement learning process. In order to ensure that the boundary exploration model Learn an efficient boundary search strategy and design a reward function To guide the model to prioritize searching for potential boundary points.

[0083] More specifically, in step S205, the method of generating the reward function may include: Step b1: Determine whether the test results of the autonomous driving system in the two current test scenarios included in the current test scenario pair are the same.

[0084] Step b2: If they are the same, determine whether the current test scenario pair is the first test scenario pair with the same test result; if so, determine the reward function to be 0; if not, redetermine the current test scenario pair.

[0085] When the result is the same, that is, two test points and The test results of are the same, and this is the first time that the test results appear in a round of iterative search, then the current action No reward is given, i.e. .

[0086] If the same test results have already appeared in this round of iterative search, no reward will be given for the time being. This means that after the action is executed, one of the new test scenarios has mistakenly crossed the capability boundary. At this time, the current test scenario pair should be re-determined. The specific determination method can refer to the description in step a7.

[0087] Here, as mentioned above, the new first test result in the new current test scene pair re-determined in the manner of step a7 is different from the new second test result, and the spatial distance is also shortened. Therefore, step b5 can be skipped and the change trend can be determined to be shortened; the judgment of the change trend of the spatial distance in step b5 can also be performed again; and then the spatial distance is compared with the preset distance threshold to determine the reward function.

[0088] Step b3: If the test results of the autonomous driving system in the two current test scenarios included in the current test scenario pair are different, determine whether the current test scenario pair is the first test scenario pair in which different test results appear.

[0089] Step b4: If yes, determine that the reward function is the initial positive reward function.

[0090] In the case of different results, that is, the test point and The test results of are different, and this is the first time that the test results appear in a round of iterative search, then the initial positive reward function is given , indicating that the boundary may be approaching.

[0091] Step b5: If not, determine the change trend of the current spatial distance between the two test scenes and the spatial distance between the two test scenes corresponding to the previous transfer state.

[0092] Step b6: If the current spatial distance is shortened and is not less than the preset distance threshold, the reward function is determined to be a positive increasing reward function. ,in is the reward coefficient with decreasing distance.

[0093] Step b7: If the current spatial distance is less than the preset distance threshold, determining that the reward function is a maximum positive reward function. , and end the current round.

[0094] Furthermore, more dimensional information can be considered, such as the rate of change of the distance between test points, the number of consecutive failures or successes, etc., and a piecewise linear or nonlinear reward function can be introduced to reflect subtle changes in the boundary search process.

[0095] In an experiment, the settings of the neural network structure parameters and training hyperparameters for a given set of training processes are shown as follows: Boundary Exploration Model The neural network structure includes: Input layer: input current state The distance between the two test points ; Hidden layers: 3 fully connected layers, each containing 128, 64 and 32 nodes, and the activation function is ReLU.

[0096] Output layer: Outputs the Q value estimate of each action, including reducing the distance, maintaining the distance, etc.

[0097] Hyperparameter settings: Discount Factor ; Exploration Rate The initial value is 1 and decays linearly to 0.1; Learning Rate ; Minimum distance threshold (the minimum distance threshold in the scene space after normalization); Reward Value .

[0098] An embodiment of the present application provides a method for determining the capability boundary of an autonomous driving system, which utilizes a pre-trained boundary exploration model to efficiently identify the capability boundary of the autonomous driving system under various driving conditions.

[0099] First, by defining a scenario space covering various driving conditions and connecting the autonomous driving system to be tested into it, a testing environment is provided for subsequent boundary exploration.

[0100] On this basis, deep reinforcement learning technology is used to build an intelligent boundary exploration model, which can adaptively adjust the search strategy according to previous test results, and automatically find and locate performance boundary points in high-dimensional parameter space. The key to training the boundary exploration model is to design an effective reward mechanism to guide the learning process of the model. When two selected test points produce different test results, it indicates that there is a system performance boundary between the two points. At this time, positive rewards are given to encourage the model to further narrow the distance between the two points to accurately determine the boundary position; if the results of the two points are the same, no reward is given or only a low level of feedback signal is provided to avoid unnecessary exploration behavior.

[0101] The boundary exploration model uses a dual deep Q network (DDQN) architecture to optimize the search strategy. During the training process, by designing a reasonable reward function and adopting - Greedy strategy, the model can learn the optimal boundary search strategy from a large amount of generated data. By gradually reducing the probability of exploration with large randomness (i.e. reducing value), the model can gradually focus on the better solution after widely trying different strategies in the early stage.

[0102] After sufficient training, the boundary exploration model can quickly and accurately find the performance limit of the autonomous driving system in unknown scenarios. Applying this model to the testing of different types or versions of autonomous driving systems can quickly obtain their respective capability boundary information, helping developers to promptly discover potential safety hazards and take corresponding measures to improve them.

[0103] See also Figure 3 , Figure 3 This is a schematic diagram of a structure of a device for determining the capability boundary of an autonomous driving system provided in an embodiment of the present application. Figure 3 As shown in , the determining device 300 includes: A definition module 310 is used to define a test scenario space of the autonomous driving system to be tested; wherein the test scenario space includes multiple test dimensions, and the scenario parameters under each test dimension have a corresponding parameter value range; a test scenario is formed by combining the parameter values ​​of the scenario parameters under each test dimension; A search module 320 is used to perform multiple rounds of iterative searches in the test scene space using a pre-trained boundary exploration model, and determine the capability boundary of the autonomous driving system based on the test results of the autonomous driving system in each test scene searched; wherein each capability boundary is a group of boundary test scene pairs, and the test results of the autonomous driving system in the first boundary test scene in each group of boundary test scene pairs are different from the test results in the second boundary test scene, and the spatial distance between the first boundary test scene and the second boundary test scene is less than a preset distance threshold; the boundary exploration model adjusts the exploration strategy based on the test results obtained from the iterative search that has been performed.

[0104] Furthermore, when the search module 320 is used to perform multiple rounds of iterative searches in the test scenario space using a pre-trained boundary exploration model and determine the capability boundary of the autonomous driving system according to the test results of the autonomous driving system in each searched test scenario, the search module 320 is used to: For any round of iterative search, use the boundary exploration model to locate the capability boundary found in the previous round of iterative search in the test scenario space, and continue the current round of iterative search; During this round of iterative search, whenever a set of test scene pairs is searched in the test scene space, the set of test scene pairs is imported into the autonomous driving system for testing; The boundary exploration model is used to determine the capability boundary of the autonomous driving system according to the test results of the autonomous driving system under each searched test scenario pair.

[0105] Further, when the search module 320 is used to determine the capability boundary of the autonomous driving system according to the test results of the autonomous driving system under each searched test scenario pair using the boundary exploration model, the search module 320 is used to: For the test scene pair searched for the first time, obtaining a first test result of the autonomous driving system in the first test scene and a second test result in the second test scene in the test scene pair searched for the first time; When the first test result and the second test result are different, determining the spatial distance between the first test scene and the second test scene in the test scene pair; wherein the different test results include: the first test result is passed, and the second test result is failed; or, the first test result is failed, and the second test result is passed; If the spatial distance is less than the preset distance threshold, the test scenario pair is determined as a capability boundary, and the current round of iterative search ends; If the spatial distance is not less than the preset distance threshold, the boundary exploration model uses a method of reducing the spatial distance to re-search a set of new test scene pairs based on the test scene pair; Re-acquire a new first test result of the autonomous driving system in the first test scenario and a new second test result in the second test scenario in the set of new test scenario pairs; When the new first test result and the new second test result are different, returning to the step of determining the spatial distance between the first test scene and the second test scene in the new test scene pair; When the new first test result is the same as the new second test result, the new test scene whose test result has changed in the new test scene pair and the original test scene corresponding to the new test scene whose test result has changed are recombined to obtain a new set of test scene pairs; and return to execute the step of determining the spatial distance between the first test scene and the second test scene in the new test scene pair.

[0106] Further, when the search module 320 is used to use the boundary exploration model to determine the capability boundary of the autonomous driving system according to the test results of the autonomous driving system under each searched test scenario pair, the search module 320 is also used to: For the test scenario pair searched for the first time, when the first test result and the second test result are the same, the test scenario pair is discarded, and a new set of test scenario pairs is searched again based on the capability boundary searched in the previous round of iterations.

[0107] Furthermore, the determining device 300 further includes: an export module; the export module is used to: deriving the capability boundary of the autonomous driving system; When the boundary test scenario pairs corresponding to each capability boundary form a continuous boundary in the test scene space, or when at least one test dimension reaches the spatial limit of the test scene space, it is determined that the capability boundary of the autonomous driving system has been explored, and an autonomous driving capability boundary scenario set is formed based on the determined capability boundary of the autonomous driving system.

[0108] Furthermore, the determination device 300 further includes: a training module; the training module is used to train the boundary exploration model in the following manner: Setting the state space, action space and state transfer function of the boundary exploration model; wherein each optional state included in the state space is a set of test scenario pairs; randomly selecting an initial state from the state space; Selecting an execution action from the action space according to a test result of the autonomous driving system under the test scenario pair corresponding to the initial state; Generate a transition state according to the initial state, the execution action and the state transition function; generating a reward function according to a test result of the autonomous driving system under the current test scenario pair corresponding to the transfer state, and updating parameters of the boundary exploration model according to the reward function; Determine whether the current test scenario pair corresponding to the transfer state is a capability boundary; If it is not a capability boundary, returning to the step of selecting an execution action from the action space until it is determined that the test scenario pair corresponding to the regenerated transition state is a capability boundary; If it is the capability boundary, determining whether the boundary exploration model has converged; If it has not converged, return to the step of randomly selecting an initial state from the state space until the boundary exploration model converges, thereby obtaining the trained boundary exploration model.

[0109] Furthermore, when the training module is used to generate a reward function according to a test result of the autonomous driving system under the current test scenario pair corresponding to the transfer state, the training module is used to: Determining whether test results of the autonomous driving system in two current test scenarios included in the current test scenario pair are the same; If they are the same, determining whether the current test scenario pair is the first test scenario pair with the same test result; if so, determining the reward function to be 0; if not, re-determining the current test scenario pair; If the test results of the autonomous driving system in two current test scenarios included in the current test scenario pair are different, determining whether the current test scenario pair is a test scenario pair where different test results occur for the first time; If so, the reward function is determined to be the initial positive reward function; If not, determining the change trend of the current spatial distance between the two test scenes and the spatial distance between the two test scenes corresponding to the previous transfer state; If the current spatial distance is shortened and is not less than the preset distance threshold, determining that the reward function is a positive increasing reward function; If the current spatial distance is less than the preset distance threshold, the reward function is determined to be a maximum positive reward function.

[0110] Furthermore, the boundary exploration model is a dual-depth Q network; wherein the main network is used to use - A greedy strategy is used, and according to the test results of the autonomous driving system under the test scenario, an execution action is selected from the action space; the target network is used to generate a Q value for the main network.

[0111] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in , the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0112] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 as well as Figure 2 The steps of the method for determining the capability boundary of the autonomous driving system in the method embodiment shown, and the specific implementation method can be found in the method embodiment, which will not be repeated here.

[0113] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 as well as Figure 2 The steps of the method for determining the capability boundary of the autonomous driving system in the method embodiment shown, and the specific implementation method can be found in the method embodiment, which will not be repeated here.

[0114] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0115] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0118] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0119] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for determining the capability boundary of an autonomous driving system, characterized in that: The determination method comprises: Defining a test scenario space of the autonomous driving system to be tested; wherein the test scenario space includes multiple test dimensions, and the scenario parameters under each test dimension have a corresponding parameter value range; and forming a test scenario by combining parameter values ​​of the scenario parameters under each test dimension; A pre-trained boundary exploration model is used to perform multiple rounds of iterative searches in the test scene space, and the capability boundary of the autonomous driving system is determined according to the test results of the autonomous driving system in each test scene searched; wherein each capability boundary is a group of boundary test scene pairs, and the test results of the autonomous driving system in the first boundary test scene in each group of boundary test scene pairs are different from the test results in the second boundary test scene, and the spatial distance between the first boundary test scene and the second boundary test scene is less than a preset distance threshold; the boundary exploration model adjusts the exploration strategy according to the test results obtained from the iterative search that has been performed.

2. The determination method according to claim 1, characterized in that: Performing multiple rounds of iterative searches in the test scenario space using a pre-trained boundary exploration model, and determining the capability boundary of the autonomous driving system according to the test results of the autonomous driving system in each searched test scenario, including: For any round of iterative search, use the boundary exploration model to locate the capability boundary found in the previous round of iterative search in the test scenario space, and continue the current round of iterative search; During this round of iterative search, whenever a set of test scene pairs is searched in the test scene space, the set of test scene pairs is imported into the autonomous driving system for testing; The boundary exploration model is used to determine the capability boundary of the autonomous driving system according to the test results of the autonomous driving system under each searched test scenario pair.

3. The determination method according to claim 2, characterized in that: Determining the capability boundary of the autonomous driving system using the boundary exploration model according to the test results of the autonomous driving system under each searched test scenario pair includes: For the test scene pair searched for the first time, obtaining a first test result of the autonomous driving system in the first test scene and a second test result in the second test scene in the test scene pair searched for the first time; When the first test result and the second test result are different, determining the spatial distance between the first test scene and the second test scene in the test scene pair; wherein the different test results include: the first test result is passed, and the second test result is failed; or, the first test result is failed, and the second test result is passed; If the spatial distance is less than the preset distance threshold, the test scenario pair is determined as a capability boundary, and the current round of iterative search ends; If the spatial distance is not less than the preset distance threshold, the boundary exploration model uses a method of reducing the spatial distance to re-search a set of new test scene pairs based on the test scene pair; Re-acquiring a new first test result of the autonomous driving system in a first test scenario and a new second test result in a second test scenario in the set of new test scenario pairs; When the new first test result and the new second test result are different, returning to the step of determining the spatial distance between the first test scene and the second test scene in the new test scene pair; When the new first test result is the same as the new second test result, the new test scene whose test result has changed in the new test scene pair and the original test scene corresponding to the new test scene whose test result has changed are recombined to obtain a new set of test scene pairs; and return to execute the step of determining the spatial distance between the first test scene and the second test scene in the new test scene pair.

4. The determination method according to claim 3, characterized in that: Determining the capability boundary of the autonomous driving system using the boundary exploration model according to the test results of the autonomous driving system under each searched test scenario pair, further comprising: For the test scenario pair searched for the first time, when the first test result and the second test result are the same, the test scenario pair is discarded, and a new set of test scenario pairs is searched again based on the capability boundary searched in the previous round of iterations.

5. The determination method according to claim 1, characterized in that: The determination method further comprises: deriving the capability boundary of the autonomous driving system; When the boundary test scenario pairs corresponding to each capability boundary form a continuous boundary in the test scene space, or when at least one test dimension reaches the spatial limit of the test scene space, it is determined that the capability boundary of the autonomous driving system has been explored, and an autonomous driving capability boundary scenario set is formed based on the determined capability boundary of the autonomous driving system.

6. The determination method according to claim 1, characterized in that: The boundary exploration model is trained in the following way: Setting the state space, action space and state transfer function of the boundary exploration model; wherein each optional state included in the state space is a set of test scenario pairs; randomly selecting an initial state from the state space; Selecting an execution action from the action space according to a test result of the autonomous driving system under the test scenario pair corresponding to the initial state; Generate a transition state according to the initial state, the execution action and the state transition function; generating a reward function according to a test result of the autonomous driving system under the current test scenario pair corresponding to the transfer state, and updating parameters of the boundary exploration model according to the reward function; Determine whether the current test scenario pair corresponding to the transfer state is a capability boundary; If it is not a capability boundary, returning to the step of selecting an execution action from the action space until it is determined that the test scenario pair corresponding to the regenerated transition state is a capability boundary; If it is the capability boundary, determining whether the boundary exploration model has converged; If it has not converged, return to the step of randomly selecting an initial state from the state space until the boundary exploration model converges, thereby obtaining the trained boundary exploration model.

7. The determination method according to claim 6, characterized in that: Generate a reward function according to a test result of the autonomous driving system under the current test scenario pair corresponding to the transfer state, including: Determining whether test results of the autonomous driving system in two current test scenarios included in the current test scenario pair are the same; If they are the same, determining whether the current test scenario pair is the first test scenario pair with the same test result; if so, determining the reward function to be 0; if not, re-determining the current test scenario pair; If the test results of the autonomous driving system in two current test scenarios included in the current test scenario pair are different, determining whether the current test scenario pair is a test scenario pair where different test results occur for the first time; If so, the reward function is determined to be the initial positive reward function; If not, determining the change trend of the current spatial distance between the two test scenes and the spatial distance between the two test scenes corresponding to the previous transfer state; If the current spatial distance is shortened and is not less than the preset distance threshold, determining that the reward function is a positive increasing reward function; If the current spatial distance is less than the preset distance threshold, the reward function is determined to be a maximum positive reward function.

8. The determination method according to claim 6, characterized in that: The boundary exploration model is a dual-depth Q network; the main network is used to use - A greedy strategy is used, and according to the test results of the autonomous driving system under the test scenario, an execution action is selected from the action space; the target network is used to generate a Q value for the main network.

9. A device for determining the capability boundary of an autonomous driving system, characterized in that: The determining device comprises: A definition module, used to define a test scenario space of the autonomous driving system to be tested; wherein the test scenario space includes multiple test dimensions, and the scenario parameters under each test dimension have a corresponding parameter value range; a test scenario is formed by combining the parameter values ​​of the scenario parameters under each test dimension; A search module is used to perform multiple rounds of iterative searches in the test scene space using a pre-trained boundary exploration model, and determine the capability boundary of the autonomous driving system according to the test results of the autonomous driving system in each test scene searched; wherein each capability boundary is a group of boundary test scene pairs, and the test results of the autonomous driving system in the first boundary test scene in each group of boundary test scene pairs are different from the test results in the second boundary test scene, and the spatial distance between the first boundary test scene and the second boundary test scene is less than a preset distance threshold; the boundary exploration model adjusts the exploration strategy according to the test results obtained from the iterative search that has been performed.

10. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to perform the steps of the method for determining the capability boundary of the autonomous driving system as described in any one of claims 1 to 8.

Images