Interactive testing methods, equipment, media and products for autonomous driving models

Abstract

The present application discloses an interactive testing method, equipment, medium and product for an autonomous driving model, which relates to the field of autonomous driving. The method includes obtaining the status data of each vehicle in the test scene, including the position, speed and following distance, determining the status data of the vehicle of interest at the current moment and the status data of the corresponding surrounding vehicles, and inputting them into a behavior decision model based on reinforcement learning to obtain lateral and longitudinal behavior decisions; if the decision is to maintain lane driving, the following trajectory planning model considering driving style is used to plan the following trajectory; if the decision is to change lanes, the lane change trajectory is planned using a lane change trajectory planning model considering driving intention; the action of the vehicle of interest at the next moment is determined according to the following trajectory or lane change trajectory. Lateral and longitudinal behavior decisions are made based on the status data of the vehicle of interest and the surrounding vehicles, and the vehicle driving style is considered when planning the trajectory, thereby improving the authenticity and interactivity of the test scene.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to an interactive testing method, equipment, medium and product for an autonomous driving model. Background Art

[0002] In recent years, autonomous driving technology, as an important part of the intelligent transportation system, has shown broad development prospects in improving driving safety and traffic efficiency. However, there are still some defects in the research on the testing methods of autonomous driving models at home and abroad, especially the lack of ability to build strong interactive mixed traffic test scenarios.

[0003] The existing testing methods for autonomous driving models mainly include testing methods based on real data and testing methods based on control models. Among them, most of the testing methods based on real data use data playback to construct test scenarios, which results in relatively fixed test scenarios and lack of interactivity, making it difficult to meet the testing requirements of autonomous driving models in highly interactive scenarios. Most of the testing methods based on control models use simple rule models or learning models to construct test scenarios, which do not fully consider real driving data, resulting in the difficulty of deploying autonomous driving models with good test results in simulation environments to real vehicles. It can be seen that the existing testing methods for autonomous driving models face the problem of weak interactivity and authenticity in the constructed test scenarios. Summary of the invention

[0004] The purpose of this application is to provide an interactive testing method, equipment, medium and product for an autonomous driving model, which can improve the authenticity and interactivity of the test scenario.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides an interactive testing method for an autonomous driving model, comprising:

[0007] Acquire the status data of each vehicle at the current moment in the test scene, and determine the status data of the vehicle of interest at the current moment and the status data of the surrounding vehicles corresponding to the vehicle of interest based on the status data of each vehicle at the current moment; the vehicle of interest is the vehicle interacting with the tested vehicle at the current moment in the test scene; the tested vehicle is a vehicle driven by an automatic driving model; the surrounding vehicles are various types of vehicles interacting with the vehicle of interest, including the tested vehicle; the status data includes position, driving speed and following distance;

[0008] Inputting the state data of the vehicle of interest at the current moment and the state data of the surrounding vehicles corresponding to the vehicle of interest into the trained driving behavior decision model based on reinforcement learning, obtaining the driving behavior decision result of the vehicle of interest at the current moment; the driving behavior decision result is a lateral and longitudinal behavior decision; the lateral and longitudinal behavior decisions include changing lanes to the left and overtaking, changing lanes to the left and following the vehicle, changing lanes to the right and overtaking, changing lanes to the right and following the vehicle, or keeping the current lane and following the vehicle;

[0009] When the driving behavior decision result is to maintain the current lane and follow the car, a following trajectory planning model considering the driving style is started to output the following trajectory;

[0010] When the driving behavior decision result is changing lanes to the left and overtaking, changing lanes to the left and following a vehicle, changing lanes to the right and overtaking, or changing lanes to the right and following a vehicle, starting a lane change trajectory planning model that takes driving intention into consideration and outputting a lane change trajectory;

[0011] According to the following trajectory or lane-changing trajectory, determine the execution action of the vehicle of interest at the next moment in the test scenario.

[0012] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the interactive testing method of the autonomous driving model described above.

[0013] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the interactive testing method of the autonomous driving model described above.

[0014] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned interactive testing method of the autonomous driving model.

[0015] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0016] The present application provides an interactive testing method, device, medium and product for an autonomous driving model. The present application obtains the status data of each vehicle at the current moment in the test scene, and determines the status data of the vehicle of interest at the current moment and the status data of the surrounding vehicles corresponding to the vehicle of interest based on the status data of each vehicle at the current moment; the vehicle of interest is the vehicle interacting with the tested vehicle at the current moment in the test scene; the tested vehicle is a vehicle driven by an autonomous driving model; the surrounding vehicles are various types of vehicles interacting with the vehicle of interest, including the tested vehicle; the status data includes position, driving speed and following distance; the status data of the vehicle of interest at the current moment and the status data of the surrounding vehicles corresponding to the vehicle of interest are input into the trained driving behavior decision model based on reinforcement learning to obtain the driving behavior decision result of the vehicle of interest at the current moment. It can be seen that the driving behavior decision result obtained by using the driving behavior decision model based on reinforcement learning comprehensively considers the status data of the vehicle of interest and its corresponding surrounding vehicles, thereby improving the interactivity of the test scene.

[0017] Moreover, according to the following trajectory or lane-changing trajectory, the execution action of the vehicle of interest at the next moment in the test scenario is determined. By considering the following trajectory planning model of driving style, the real driving behavior can be better simulated, and the authenticity of the test scenario is enhanced. By combining the consideration of driving style and driving intention, the following trajectory or lane-changing trajectory is obtained, and the execution action of the vehicle of interest at the next moment in the test scenario is determined, so that the trajectory planning model simulates the decision-making process of human drivers in complex traffic environments, and enhances the authenticity of the test scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is an application environment diagram of an interactive testing method for an autonomous driving model in one embodiment of the present application;

[0020] Figure 2 A flowchart of an interactive testing method for an autonomous driving model provided in one embodiment of the present application;

[0021] Figure 3 A schematic diagram of a process for training a driving behavior decision model based on reinforcement learning provided in one embodiment of the present application;

[0022] Figure 4A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0025] The interactive testing method of the autonomous driving model provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the status data of each vehicle at the current moment in the test scene to the server 104. After the server 104 receives the status data of each vehicle at the current moment in the test scene, the server 104 uses the trained driving behavior decision model based on reinforcement learning to make a driving behavior decision, obtains the driving behavior decision result, and performs a following trajectory or a lane change trajectory planning according to the driving behavior decision result to determine the execution action of the vehicle at the next moment. The server 104 can feedback the status data of each vehicle at the next moment in the test scene to the terminal 102. In addition, in some embodiments, the interactive testing method of the automatic driving model can also be implemented separately by the server 104 or the terminal 102, such as the terminal 102 can directly process the status data of each vehicle in the test scene, or the server 104 can obtain the status data of each vehicle in the test scene from the data storage system and process it.

[0026] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, and IoT devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0027] In an exemplary embodiment, Figure 2As shown, an interactive testing method for an autonomous driving model is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps 201 to 204. Among them:

[0028] Step 201, obtain the status data of each vehicle at the current moment in the test scene, and determine the status data of the vehicle of interest at the current moment and the status data of the surrounding vehicles corresponding to the vehicle of interest based on the status data of each vehicle at the current moment; the vehicle of interest is the vehicle interacting with the tested vehicle at the current moment in the test scene; the tested vehicle is a vehicle driven by an automatic driving model; the surrounding vehicles are various types of vehicles interacting with the vehicle of interest, including the tested vehicle; the status data includes position, driving speed and following distance.

[0029] Step 202, input the status data of the vehicle being focused on at the current moment and the status data of the surrounding vehicles corresponding to the vehicle being focused on into the trained driving behavior decision model based on reinforcement learning, and obtain the driving behavior decision result of the vehicle being focused on at the current moment; the driving behavior decision result is a lateral and longitudinal behavior decision; the lateral and longitudinal behavior decisions include changing lanes to the left and overtaking, changing lanes to the left and following the vehicle, changing lanes to the right and overtaking, changing lanes to the right and following the vehicle, or keeping the current lane and following the vehicle.

[0030] Furthermore, the determination process of the trained driving behavior decision model based on reinforcement learning is as follows:

[0031] a. Obtain the real status data of each vehicle; the real status data includes the position, driving speed and following distance;

[0032] b. Determine the driving style of each vehicle according to the real state data of each vehicle, and take each vehicle as a focus vehicle in turn, determine the real state data of each focus vehicle and the real state data and driving style of surrounding vehicles corresponding to each focus vehicle; the driving styles include aggressive, conservative and general;

[0033] Specifically, in order to improve the safety, efficiency and stability of autonomous vehicles in complex interactive scenarios, it is crucial to understand and infer the driving style of surrounding vehicles. Therefore, this application obtains the average driving speed and average following time distance of each vehicle based on the real state data of each vehicle, and uses the average driving speed and average following time distance of vehicles in a fixed section of road as feature attributes, and uses the K-Means algorithm to cluster the real state data, and divides the driving style of each vehicle into three categories: aggressive, conservative and general, and constructs data sets corresponding to different driving styles, so as to use the driving style of the vehicle as the basis for training the driving behavior decision model based on reinforcement learning.

[0034] c. Constructing a first training data set; the first training data set includes a plurality of first sample data; the first sample data includes the real state data of the vehicle of interest and the real state data and driving style of surrounding vehicles corresponding to the vehicle of interest.

[0035] d. An initial driving behavior decision model based on reinforcement learning is constructed using an Actor-Critic method based on reinforcement learning; the initial driving behavior decision model based on reinforcement learning includes a strategy network and a value network.

[0036] The first sample data at the current moment is input into the strategy network to predict the action vector at the current moment; the first sample data at the current moment is defined as:

[0037] ;

[0038] in, represents the position matrix of the vehicle of interest, shows the position matrix of the surrounding vehicles, and Represent the velocity matrices of the vehicle of interest and surrounding vehicles, Refers to the driving style matrix of surrounding vehicles;

[0039] The action vector is defined as:

[0040] ;

[0041] in, and They respectively indicate changing lanes to the left and overtaking, changing lanes to the left and following the vehicle. and They respectively indicate changing lanes to the right and overtaking, changing lanes to the right and following the vehicle. Indicates maintaining the current lane and following the vehicle.

[0042] The reward value at the current moment is calculated according to the action vector at the current moment and the first driving behavior prediction result at the current moment; the first driving behavior prediction result is obtained after the first sample data is input into the trained driving behavior prediction model.

[0043] e. Input the current real state data, the current action vector, the current reward value and the first sample data of the next moment into the value network to obtain the value function.

[0044] f. Update the policy network according to the value function to obtain the trained policy network, and use it as the trained reinforcement learning-based driving behavior decision model.

[0045] Furthermore, the determination process of the trained driving behavior prediction model is as follows:

[0046] a. Construct a second training data set; the second training data set includes multiple second sample data; the second sample data includes the real state data of the vehicle of interest, the real state data of the surrounding vehicles corresponding to the vehicle of interest, and label data; the label data is the initial lateral and longitudinal behavior decisions.

[0047] b. Construct an initial driving behavior prediction model based on the long short-term memory network; Recurrent neural networks (RNN) are a classic method for predicting sequence data, but RNN can only process limited historical data, and there are problems such as gradient vanishing and gradient explosion caused by long-term dependence. To solve this problem, the long short-term memory network (LSTM) came into being and became the mainstream method for sequence data prediction. Therefore, the present application uses LSTM to train and predict multiple second sample data, and then outputs the lateral and longitudinal behavior decisions of the vehicle of interest based on the real state data of the vehicle of interest and the real state data of the surrounding vehicles. By training the model with each vehicle entering the road section as a vehicle of interest in turn, the data utilization efficiency and training effect can be improved.

[0048] c. Based on the plurality of second sample data, the initial driving behavior prediction model is trained to obtain a trained driving behavior prediction model.

[0049] The purpose of constructing the driving behavior prediction model in the above process is to provide a decision basis for the driving behavior decision model based on reinforcement learning. Specifically, the first driving behavior prediction result is used as part of the reward function of the driving behavior decision model based on reinforcement learning to encourage the concerned vehicle to make decisions consistent with human-driven vehicles in the real state data scenario when facing a test scenario similar to the real state data scenario, but it does not require strict consistency with human driving decisions, and reserves room for the concerned vehicle to make autonomous adjustments and autonomous interactions. The flowchart of using the driving behavior prediction results to train the driving behavior decision model based on reinforcement learning is as follows: Figure 3 shown.

[0050] It can be seen that this application is to realize interactive modeling of test playback data, based on the first driving behavior prediction result output by the driving behavior prediction model based on real state data, to generate the lateral and longitudinal decisions of the vehicle of interest in the test scene for different driving styles. Moreover, corresponding to the test data sets under three different driving styles, three reward functions corresponding to different driving styles are designed respectively, and three different types of driving behavior decision models based on reinforcement learning can be trained, thereby realizing the random switching of vehicle driving styles in the test scene.

[0051] Furthermore, the calculation formula of the reward value at the current moment is:

[0052] ;

[0053] in, Represents the reward value at the current moment, s is the first sample data at the current moment, a is the action vector at the current moment, o is the first driving behavior prediction result at the current moment, collision or no valid trajectory Indicates that the lateral and longitudinal behavior decisions cannot solve a valid trajectory or cause a collision. v t To focus on the current speed of the vehicle, v l is the expected driving speed corresponding to different driving styles, v gain To focus on the vehicle's current speed gain compared to the speed before lane change, g is the speed gain threshold, w 0 , w 1 and w 2 is the weight coefficient, KFTo keep following the current lane. The specific meaning of the reward function is: if the lateral and longitudinal behavior decisions cannot solve the effective trajectory or cause a collision, a fixed penalty is given; if the action vector is consistent with the first driving behavior prediction result, a fixed reward is given; if the action selected is to keep following the current lane, a reward is given based on the difference between the driving speed and the expected speed; if the action selected is to change lanes, a reward or penalty is given based on the speed gain before and after the lane change and the difference between the driving speed and the expected speed.

[0054] In each training round, the rewards for each time step are accumulated as the total reward for a state-action pair in reinforcement learning:

[0055] ;

[0056] in, is the total number of time steps included in this training round.

[0057] Step 2031: When the driving behavior decision result is to maintain the current lane and follow the car, a following trajectory planning model considering the driving style is started to output a following trajectory.

[0058] Furthermore, the calculation formula of the following trajectory planning model is:

[0059] ;

[0060] in, is the acceleration of the vehicle; is the speed of the vehicle; is the expected driving speed; is the acceleration index; is the following distance; The expected following distance is: , is the minimum safety clearance, To focus on the speed difference between the vehicle and the vehicle ahead, is the expected following distance, is the expected acceleration, Deceleration for comfort.

[0061] There are 5 parameters to be calibrated in the car-following trajectory planning model, which are expressed as , The parameter calibration part uses genetic algorithm for iterative solution. The number of genetic algorithm population is set to 5, corresponding to 5 parameters to be calibrated. The iterative equation is designed according to the principle of IDM algorithm:

[0062] ;

[0063] in, and represents the speed and position of the vehicle of interest generated by the car-following trajectory planning model, In order to focus on the distance between the vehicle and the preceding vehicle in the following trajectory planning model, and Indicates the position and speed of the vehicle in front of the vehicle of interest in the status data, is the body length of the front vehicle, Indicates the current moment, Represents the time interval between two adjacent moments. represents the attention vehicle generated by the following trajectory planning model The acceleration of time.

[0064] Objective function of genetic algorithm Designed based on minimizing the following distance error, specifically:

[0065] ;

[0066] in, In the state data, the distance between the vehicle and the preceding vehicle is the focus. The number of trajectory points generated for the model.

[0067] According to the status data of three different driving styles, three sets of parameter calibration are performed to form three sets of car-following trajectory planning models with different interaction strategies.

[0068] Step 2032: When the driving behavior decision result is changing lanes to the left and overtaking, changing lanes to the left and following, changing lanes to the right and overtaking, or changing lanes to the right and following, a lane changing trajectory planning model considering the driving intention is started and a lane changing trajectory is output.

[0069] Furthermore, the lane change trajectory planning model includes a driving intention recognition unit of the target lane vehicle and a lane change trajectory planning unit; when the driving behavior decision result is to change lanes to the left and overtake, change lanes to the left and follow the vehicle, change lanes to the right and overtake, or change lanes to the right and follow the vehicle, the lane change trajectory planning model considering the driving intention is started and the lane change trajectory is output, which specifically includes:

[0070] Construct a driving intention recognition unit for the target lane vehicle based on the long short-term memory network;

[0071] Inputting the state data of the vehicle of interest, the state data of the vehicle in the target lane and the driving style of the surrounding vehicles into the driving intention recognition unit of the vehicle in the target lane, and obtaining the expected acceleration of the vehicle in the target lane, that is, the driving intention of the vehicle in the target lane; the target lane is determined according to the lateral decision part;

[0072] The longitudinal endpoint of the vehicle of interest in the target lane is determined according to the expected acceleration of the vehicle in the target lane, and the lateral endpoint of the vehicle of interest in the target lane is determined according to the lateral decision part in the driving behavior decision result; specifically, the 50-meter area forward of the current position of the vehicle of interest is divided into several equally spaced longitudinal sampling points, and the sampling points are screened based on the overtaking or following decision of the vehicle of interest and the driving intention of the vehicle in the target lane, and the result is obtained. The lateral trajectory endpoint is the lateral center position of the target lane determined by the lateral decision part in the driving behavior decision result.

[0073] A lane-changing trajectory is obtained according to the longitudinal trajectory end point, the lateral trajectory end point and the lane-changing trajectory planning unit.

[0074] The lane-changing trajectory planning unit is specifically:

[0075] ;

[0076] in, is the lane-changing trajectory at the current moment, a 0 , a 1 , a 2 , a 3 , a 4 , a 5 are the parameters of each order, t is the current moment; the coefficients of each order are based on the initial state of the vehicle and the vehicle's terminal state Definitely got it.

[0077] The initial state Includes the vehicle status data in the current test scenario, including position, speed and acceleration, and the terminal state Includes the longitudinal trajectory end point, lateral trajectory end point, expected driving speed and expected acceleration.

[0078] Given an initial state and the terminal state , the solution of each order parameter can be expressed in matrix form:

[0079] ;

[0080] in, is the coefficient matrix, are the parameter matrices of various orders to be solved, t s is the initial state time,t e The end state time.

[0081] Therefore, the parameter matrix can be obtained by matrix inversion:

[0082] .

[0083] Furthermore, the optimal trajectory is determined iteratively by designing the objective function. The objective function is designed from the perspective of trajectory optimization, taking into account trajectory smoothness, time efficiency, and deviation from the desired end point:

[0084] ;

[0085] ;

[0086] in, represents the objective function, is the smoothness cost of the trajectory, is the time from the initial state to the final state, is the deviation of the trajectory end point from the expected end point, , and is the weight coefficient, and the trajectory smoothness is determined by the driving impact Representation.

[0087] Step 204 : determining the execution action of the vehicle of interest at the next moment in the test scenario according to the following trajectory or the lane changing trajectory.

[0088] The present application also provides an application scenario, which applies the above-mentioned interactive testing method of the autonomous driving model. Specifically: The interactive testing method of the autonomous driving model provided in this embodiment can be applied in the scenario of testing the autonomous driving model. The scenario of testing the autonomous driving model includes data acquisition and data processing links; the data acquisition link is used to obtain the status data of each vehicle at the current moment in the test scenario, and the data processing link is used to determine the status data of each vehicle at the next moment in the test scenario.

[0089] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 4As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store vehicle status data and process data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an interactive testing method for an autonomous driving model is implemented.

[0090] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0091] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0092] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0093] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0094] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0095] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0096] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0097] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. An interactive testing method for an autonomous driving model, characterized in that: The interactive testing method of the autonomous driving model includes: Acquire the status data of each vehicle at the current moment in the test scene, and determine the status data of the vehicle of interest at the current moment and the status data of the surrounding vehicles corresponding to the vehicle of interest based on the status data of each vehicle at the current moment; the vehicle of interest is the vehicle interacting with the tested vehicle at the current moment in the test scene; the tested vehicle is a vehicle driven by an automatic driving model; the surrounding vehicles are various types of vehicles interacting with the vehicle of interest, including the tested vehicle; the status data includes position, driving speed and following distance; Inputting the state data of the vehicle of interest at the current moment and the state data of the surrounding vehicles corresponding to the vehicle of interest into the trained driving behavior decision model based on reinforcement learning, obtaining the driving behavior decision result of the vehicle of interest at the current moment; the driving behavior decision result is a lateral and longitudinal behavior decision; the lateral and longitudinal behavior decisions include changing lanes to the left and overtaking, changing lanes to the left and following the vehicle, changing lanes to the right and overtaking, changing lanes to the right and following the vehicle, or keeping the current lane and following the vehicle; When the driving behavior decision result is to maintain the current lane and follow the car, a following trajectory planning model considering the driving style is started to output the following trajectory; When the driving behavior decision result is changing lanes to the left and overtaking, changing lanes to the left and following a vehicle, changing lanes to the right and overtaking, or changing lanes to the right and following a vehicle, starting a lane-changing trajectory planning model that takes driving intention into consideration and outputting a lane-changing trajectory; According to the following trajectory or lane-changing trajectory, determine the execution action of the vehicle of interest at the next moment in the test scenario.

2. The interactive testing method of the autonomous driving model according to claim 1, characterized in that: The determination process of the trained driving behavior decision model based on reinforcement learning is as follows: Acquire the real status data of each vehicle; the real status data includes the position, driving speed and following distance; Determine the driving style of each vehicle according to the real state data of each vehicle, and take each vehicle as a focus vehicle in turn, determine the real state data of each focus vehicle and the real state data and driving style of surrounding vehicles corresponding to each focus vehicle; the driving styles include aggressive, conservative and general; Constructing a first training data set; the first training data set includes a plurality of first sample data; the first sample data includes the real state data of the vehicle of interest and the real state data and driving style of surrounding vehicles corresponding to the vehicle of interest; An initial driving behavior decision model based on reinforcement learning is constructed based on the Actor-Critic method of reinforcement learning; the initial driving behavior decision model based on reinforcement learning includes a strategy network and a value network; Input the first sample data at the current moment into the policy network, predict the action vector at the current moment, and calculate the reward value at the current moment based on the action vector at the current moment and the first driving behavior prediction result at the current moment; The first driving behavior prediction result is obtained after inputting the first sample data into the trained driving behavior prediction model; Input the first sample data at the current moment, the action vector at the current moment, the reward value at the current moment, and the first sample data at the next moment into the value network to obtain a value function; The policy network is updated according to the value function to obtain a trained policy network, which is used as a trained reinforcement learning-based driving behavior decision model.

3. The interactive testing method of the autonomous driving model according to claim 2, characterized in that: The determination process of the trained driving behavior prediction model is as follows: Constructing a second training data set; the second training data set includes a plurality of second sample data; the second sample data includes the real state data of the vehicle of interest, the real state data of surrounding vehicles corresponding to the vehicle of interest, and label data; the label data is the initial lateral and longitudinal behavior decisions; Construct an initial driving behavior prediction model based on long short-term memory network; Based on the plurality of second sample data, the initial driving behavior prediction model is trained to obtain a trained driving behavior prediction model.

4. The interactive testing method of the autonomous driving model according to claim 2, characterized in that: The calculation formula of the reward value at the current moment is: ； in, Represents the reward value at the current moment, s is the first sample data at the current moment, a is the action vector at the current moment, o is the first driving behavior prediction result at the current moment, collision or no valid trajectory Indicates that the lateral and longitudinal behavior decisions cannot solve a valid trajectory or cause a collision. v t To focus on the current speed of the vehicle, v l is the expected driving speed corresponding to different driving styles, v gain To focus on the vehicle's current speed gain compared to the speed before lane change, g is the speed gain threshold, w 0 , w 1 and w 2 is the weight coefficient, KF To maintain the current lane.

5. The interactive testing method of the autonomous driving model according to claim 1, characterized in that: The calculation formula of the following trajectory planning model is: ； in, is the acceleration of the vehicle; is the speed of the vehicle; is the expected driving speed; is the acceleration index; is the following distance; The expected following distance is: , is the minimum safety clearance, To focus on the speed difference between the vehicle and the vehicle ahead, is the expected following distance, is the expected acceleration, Deceleration for comfort.

6. The interactive testing method of the autonomous driving model according to claim 1, characterized in that: The lane change trajectory planning model includes a driving intention recognition unit of the target lane vehicle and a lane change trajectory planning unit; when the driving behavior decision result is to change lanes to the left and overtake, change lanes to the left and follow the vehicle, change lanes to the right and overtake, or change lanes to the right and follow the vehicle, start the lane change trajectory planning model considering the driving intention and output the lane change trajectory, specifically including: Construct a driving intention recognition unit for the target lane vehicle based on the long short-term memory network; Inputting the state data of the vehicle of interest, the state data of the vehicle in the target lane and the driving style of the surrounding vehicles into the driving intention recognition unit of the vehicle in the target lane to obtain the expected acceleration of the vehicle in the target lane; Determine a longitudinal trajectory endpoint of the vehicle of interest in the target lane according to the expected acceleration of the vehicle in the target lane, and determine a lateral trajectory endpoint of the vehicle of interest in the target lane according to a lateral decision part in the driving behavior decision result; A lane-changing trajectory is obtained according to the longitudinal trajectory end point, the lateral trajectory end point and the lane-changing trajectory planning unit.

7. The interactive testing method of the autonomous driving model according to claim 6, characterized in that: The lane-changing trajectory planning unit is specifically: ； in, is the lane-changing trajectory at the current moment, a 0 , a 1 , a 2 , a 3 , a 4 , a 5 are the parameters of each order, t is the current moment; the various order parameters are determined according to the initial state of the vehicle and the final state of the vehicle; The initial state includes the state data of the vehicle in the current stage test scene, and the terminal state includes the longitudinal trajectory end point, the lateral trajectory end point, the expected driving speed and the expected acceleration.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an interactive testing method for the autonomous driving model described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the interactive testing method of the autonomous driving model described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the interactive testing method of the autonomous driving model described in any one of claims 1 to 7.

Images