Driving behavior model training method and natural driving environment reconstruction method and device

Abstract

A driving behavior model training method, a natural driving environment reconstruction method and device. The driving behavior model of the vehicle in the complex environment is trained by using the data-driven mode, the time series call and the space parallel call of the model are used to reconstruct the natural driving environment with time and space continuity; the background vehicle in the automatic driving system test environment is simulated by using the model-driven method, and the automatic driving system is interacted in real time, the complex real road conditions can be simulated, the car with the automatic driving system is tested in the complex interactive environment, the behavior of the background vehicle in the natural driving environment is effectively guaranteed to be close to the behavior state of the vehicle in the real environment, and the accuracy of the test is improved.

Description

Technical Field

[0001] This disclosure relates to autonomous driving technology, and more specifically, to a driving behavior model training method, a natural driving environment reconstruction method and device, and a medium. Background Technology

[0002] The development of autonomous driving currently faces serious safety issues. Autonomous vehicles developed by companies such as Tesla and Waymo have all been involved in serious traffic accidents while operating on real roads. These frequent safety problems fundamentally hinder the large-scale application and commercialization of autonomous vehicles.

[0003] Therefore, it is urgent to test and evaluate the safety of autonomous vehicles. Summary of the Invention

[0004] One embodiment of this disclosure provides a method for training a driving behavior model, wherein the driving behavior model is configured to output a predicted state value of a target vehicle in the scene at the next moment based on scene data input at a given time. The method includes:

[0005] Multiple fragmented scenes were collected based on real road environments and used as a training set. Each fragmented scene includes scene data at multiple times. The scene data includes: a scene map, and the state values ​​of a target vehicle and its background vehicles in the scene.

[0006] The driving behavior model is trained multiple times based on the training set; each training session includes: inputting scene data into the driving behavior model, the input scene data including a segmented scene T selected for this training. i The scene data at time T is obtained from the output of the driving behavior model to determine the target vehicle's position in the scene. j Conditional probability distribution P of the state prediction value at time t. θ Based on this fragmented scenario T j Scene data at time T j The state value of the target vehicle at time t and the natural probability distribution P of the acquired state value; and, according to P and P θ The divergence between the values ​​determines the loss, and the model parameters are adjusted based on the loss.

[0007] Among them, T j The moment is T i The Kth time after time point K is a preset positive integer, which is greater than or equal to 1.

[0008] An embodiment of this disclosure also provides a vehicle state prediction method, including:

[0009] The scene data is input into the driving behavior model, which is trained by the driving behavior model training method described in any embodiment of this disclosure.

[0010] The predicted state of the target vehicle at the next moment is obtained from the output of the driving behavior model;

[0011] The input scene data includes: a scene map, and state data of the target vehicle and its background vehicles in the scene at a given time, wherein the state data includes the values ​​of the vehicle's state parameters; the target vehicle is a designated vehicle among all vehicles in the scene.

[0012] One embodiment of this disclosure also provides a driving behavior model training device, such as... Figure 3 As shown, it includes: a memory and a processor, wherein the memory stores a computer program; wherein the processor is configured to execute the computer program in the memory to implement the driving behavior model training method described in any embodiment of this disclosure.

[0013] The embodiments of this disclosure can train the driving behavior model multiple times using scene data from fragmented scenarios collected from real road environments, thereby predicting the driving behavior of the target vehicle at the next moment and obtaining a state prediction value. This model can more accurately predict the driving behavior of the vehicle.

[0014] One embodiment of this disclosure provides a method for reconstructing a natural driving environment, including:

[0015] Acquire initial scene data for reconstructing the natural driving environment. The initial scene data includes: a scene map, and state data of one test vehicle and N first background vehicles in the scene at time T0.

[0016] The initial scene data is input into N pre-trained driving behavior models, and different first background vehicles are assigned as target vehicles for different driving behavior models. The test vehicle and the other first background vehicles are all used as second background vehicles for the target vehicle. The state prediction values ​​of the N first background vehicles at time T1 are obtained from the output of the N driving behavior models.

[0017] Based on the predicted state values ​​of the N first background vehicles at time T1 and the scene map, the natural driving environment of the test vehicle at time T1 is reconstructed.

[0018] An embodiment of this disclosure also provides a testing method for an autonomous driving system, including:

[0019] The test vehicle equipped with an autonomous driving system is tested based on the reconstructed natural driving environment to obtain test data of the test vehicle; wherein the natural driving environment is reconstructed using the natural driving environment reconstruction method described in any embodiment of this disclosure.

[0020] An embodiment of this disclosure also provides a natural driving environment reconstruction apparatus, including: a memory and a processor, wherein the memory stores a computer program; wherein the processor is configured to execute the computer program in the memory to implement the natural driving environment reconstruction method described in any embodiment of this disclosure.

[0021] The embodiments of this disclosure can simulate the driving behavior of various background vehicles (first background vehicles) of the test vehicle using a trained driving behavior model. Combined with a scene map, a spatially continuous natural driving environment can be reconstructed. The obtained prediction results can be used as input to drive each driving behavior model multiple times, obtaining the predicted driving behavior of each first background vehicle at subsequent time points, thereby achieving the temporal reconstruction of the natural driving environment. Therefore, the embodiments of this disclosure can construct a spatiotemporally continuous natural driving environment, enabling the tested autonomous vehicle to undergo safety testing of its behavioral capabilities in a closed-loop, high-dimensional, spatiotemporally complex natural driving environment, thereby obtaining the required test data.

[0022] One embodiment of this disclosure provides a computer storage medium storing a computer program. When the computer program is executed by a processor, it implements the driving behavior model training method, the vehicle state prediction method, the natural driving environment reconstruction method, or the autonomous driving system testing method according to any embodiment of this disclosure.

[0023] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the application. Other advantages of this application can be realized and obtained by means of the solutions described in the description and the accompanying drawings. Attached Figure Description

[0024] The accompanying drawings are used to provide an understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0025] Figure 1 This is a flowchart of a driving behavior model training method according to an embodiment of the present disclosure;

[0026] Figure 2This is a schematic diagram of a training process when using a deep neural network based on the Transformer structure as a driving behavior model according to an embodiment of this disclosure;

[0027] Figure 3 This is a flowchart of a vehicle state prediction method according to an embodiment of the present disclosure;

[0028] Figure 4 This is a flowchart of a natural driving environment reconstruction method according to an embodiment of the present disclosure;

[0029] Figure 5 This is a schematic diagram illustrating the use of a driving behavior model to predict the state of a first background vehicle in a test scenario according to an embodiment of this disclosure.

[0030] Figure 6 This is a flowchart of a testing method for an autonomous driving system according to an embodiment of the present disclosure;

[0031] Figure 7 This is a flowchart illustrating the process from data acquisition to autonomous driving system testing according to an embodiment of this disclosure;

[0032] Figure 8 This is a schematic diagram of a driving behavior model training device according to an embodiment of the present disclosure. Detailed Implementation

[0033] This application describes several embodiments, but these descriptions are exemplary and not restrictive, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0034] This application includes and contemplates combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed in this application can also be combined with any conventional features or elements to form unique inventive solutions. Any feature or element of any embodiment can also be combined with features or elements from other inventive solutions to form another unique inventive solution. Therefore, it should be understood that any feature shown and / or discussed in this application can be implemented individually or in any suitable combination. Therefore, the embodiments are not limited except by the limitations imposed by the appended claims and their equivalents. Furthermore, various modifications and changes can be made within the scope of the appended claims.

[0035] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that the method or process does not depend on the specific order of steps described herein. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation of the claims. Moreover, the claims relating to the method and / or process should not be limited to the steps performed in the order written, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments disclosed herein.

[0036] Currently, the main framework for testing the safety of autonomous vehicles is "determining the design operating domain - conducting scenario-based testing - evaluating results." The conventional method for testing the safety of autonomous vehicles is scenario-based testing, which uses data from a large number of real-world road scenarios over a limited time to test the autonomous driving system. However, this testing framework, which primarily relies on scenario-level testing, struggles to test the behavioral capabilities of autonomous vehicles in complex, highly interactive environments such as intersections, road sections, and urban areas—that is, the autonomous driving system's ability to handle external hazards in the driving environment.

[0037] Due to the high cost of road testing, technicians typically use simulation testing environments to test autonomous vehicles. In conventional simulation testing environments, in addition to the autonomous vehicle being tested, there are numerous background vehicles that interact with it. Conventional methods of controlling these background vehicles by defining traffic rules and control rules have two limitations. First, during testing, it's impossible to dynamically adjust the behavior of other road users based on the autonomous vehicle's performance, making it difficult to simulate complex real-world road conditions and thus failing to describe the highly interactive environments of the real world. Second, controlling the behavior of background vehicles based on traffic rules and control rules can lead to discrepancies between the background vehicle's state and behavior distribution and the real world, resulting in probabilities of most events that differ significantly from real-world statistics. This lack of scientific rigor and accuracy makes it difficult to accurately evaluate the behavioral capabilities of autonomous vehicles.

[0038] This disclosure proposes a method for reconstructing a natural driving environment for use in testing autonomous driving systems. It uses a driving behavior model to predict the driving behavior of each background technology vehicle and performs multiple data-driven operations on the driving behavior model to construct a spatiotemporally continuous natural driving environment. This allows the autonomous vehicle under test to conduct safety tests on its behavioral capabilities in a closed-loop, high-dimensional, spatiotemporally complex natural driving environment, thereby obtaining the required test data.

[0039] This disclosure provides a flowchart of a driving behavior model training method according to one embodiment. The trained driving behavior model can be used to reconstruct a natural driving environment. The driving behavior model is configured to output a predicted state of the target vehicle in the scene at the next moment (the moment after the given moment) based on the scene data input at a given moment. Figure 1 As shown, the training method may include the following steps:

[0040] S101. Multiple fragmented scenes were collected based on real road environments and used as a training set;

[0041] Each fragmented scene can include scene data from multiple time points. Scene data can include: a scene map, and the state values ​​of a target vehicle and its background vehicles within the scene. Multiple fragmented scenes can form a training dataset. Scene data from multiple time points within the same fragmented scene can use the same scene map.

[0042] S102. Train the driving behavior model multiple times based on the training set;

[0043] Each training session may include: inputting scene data into the driving behavior model, the input scene data including a segmented scene T selected for this training session. i The scene data at time T is obtained from the output of the driving behavior model to determine the target vehicle's position in the scene. j Conditional probability distribution P of the state prediction value at time t. θ Based on this fragmented scenario T j Scene data at time T j The state value of the target vehicle at time t and the natural probability distribution P of the acquired state value; and, according to P and P θ The divergence between the two values ​​determines the loss, and the model parameters are adjusted based on the loss. Wherein, T... j The moment is T i The Kth time point after time point P, where K is a preset positive integer, greater than or equal to 1. θ The divergence between them can be expressed using KL divergence (Kullback-Leibler Divergence), but is not limited to this.

[0044] In this embodiment of the disclosure, when training the driving behavior model, T is selected in a fragmented scene for each training session. i The scene data at time step T can be scene data from any time step within that fragmented scene, except for the scene data from the last time step. The scene data selected for input to the model during multiple training iterations can come from different fragmented scenes or from different times within the same fragmented scene. The target vehicle at time step T... j Conditional probability distribution P of the state prediction value at time t. θThis is used as the output during model training, while based on T j Conditional probability distribution P of the state prediction value at time t. θ The target vehicle can be directly determined at T. j The predicted state value at each moment, for example, for each state parameter, can be the predicted state value with the highest probability as the final predicted state value for that state parameter. After training, the driving behavior model can output only the predicted state value for the next moment. The vehicle's state value can have multiple values, such as speed and acceleration, which are detected by sensors and can be considered as true values. The above T... j The state value of the target vehicle at any given time is a sequence obtained from the training data, and the natural probability distribution P is the natural probability distribution of this sequence.

[0045] In one exemplary embodiment, the scene data may further include: scene type, vehicle shape, size, and type, as well as weather conditions, etc. Since the above information takes into account a range of data that may affect driving behavior in the real environment, this data can be integrated with the scene data... Figure 1 The scene data, serving as static data of the scenario, is used as input data when training and using the driving behavior model. Scene data containing the above information allows the model's output state predictions to more closely approximate the vehicle's state values ​​in the real environment. Furthermore, this application allows for separate data collection and model training for different scenario types, such as highways, ordinary urban roads, and mountain roads.

[0046] Different people have different driving habits, which can be categorized into several types, some more aggressive and some more conservative, each with varying risk levels. In this embodiment, to meet different testing needs, drivers with different driving habits can be selected to drive the data collection vehicle during the training phase to collect data. For example, in one example, a driver with a more aggressive driving habit can be selected to drive the data collection vehicle during training. The trained driving behavior model, when used for testing autonomous driving systems, is used to predict the driving behavior of background vehicles equipped with autonomous driving systems, posing a significant challenge to the autonomous driving system. However, this disclosure is not limited to this.

[0047] This embodiment of the disclosure can train a driving behavior model multiple times using scene data from fragmented scenarios collected in real road environments. The model then predicts the driving behavior of the target vehicle at the next moment, obtaining a state prediction value. Based on the natural probability distribution of the target vehicle's state value and the conditional probability distribution of the state prediction value, the model's loss is determined, and the model parameters are adjusted based on this loss. Since the training data all come from real road environments, this model can more accurately simulate vehicle driving behavior and accurately predict the target vehicle's state at the next moment.

[0048] In one exemplary embodiment, the multiple fragmented scenarios in this application can be obtained based on data collected by onboard sensors in the Operational Design Domain (ODD) of the vehicle. The ODD can be associated with a scenario type; depending on the type of test scenario for the autonomous driving system vehicle, data can be collected from an ODD with that type of scenario, thereby training a driving behavior model more suited to testing requirements. The state values ​​of the target vehicle and its background vehicles are the true values ​​of the vehicle's state parameters, which can include any one or more of speed, acceleration, position, and attitude. In other words, the fragmented scenarios can be obtained from real road data collected by the vehicle within a certain area and over a certain period.

[0049] For example, key intersections and road sections are selected based on the designed operating domain to determine the data collection areas for the driving behavior model. Data is collected by setting up roadside sensors at these intersections and road sections, or by deploying sensors on manually driven vehicles, to obtain raw natural driving data within the designed operating domain. This raw natural driving data then undergoes data cleaning and calibration. Data cleaning may include deleting invalid data (errors caused by sensor malfunctions, etc.), and data calibration may involve establishing a coordinate system for the raw data. Through data cleaning and calibration, a natural driving dataset can be obtained. (where: X) R，i Let N be the total number of data instances in the Natural Driving dataset, and R represent a scenario type. This Natural Driving dataset... This serves as the source of scene data for training the model in this application.

[0050] In some embodiments, data collected by onboard sensors can be used as the standard mode. Data instances obtained after data cleaning and calibration are treated as fragmented scenarios, the specific format of which is shown below:

[0051] • Collect the state values ​​of the vehicle and its surrounding vehicles at multiple times. The interval between adjacent times is a time step Δt, where Δt is a pre-set parameter. The state values ​​of all vehicles at each time can be combined with the scene map to form the scene data for that time.

[0052] • Collect map information about the area surrounding the vehicle, which can be represented as a set of M points. Each point has information such as location and category (e.g., double yellow lines, solid white lines). This information may influence the driving behavior of vehicles in the scene. This map information can be collected from external sources such as map service providers or municipal departments.

[0053] In one exemplary embodiment, the natural probability distribution of vehicle state values ​​at multiple moments in a fragmented scene can be described as: P(X) = P(X0, X...Δt , ..., X T , ...), where Δt is the length of the time step, that is, the time interval between two adjacent moments. Here, it is assumed that the initial moment is 0. The fragmented scene has a total of T time steps. Then, the subsequent moments after the initial moment can be represented as: Δt, 2Δt, ..., T-ΔT, T.

[0054] To train the model, the predicted vehicle state values ​​from the driving behavior model can be fitted based on the collected vehicle state values. This allows for equivalence to the natural probability distribution of vehicle state values ​​in fragmented scenes, i.e., P(X) = P(X0)P(X0). Δt |X0)...P(X T |X0, ..., X T-Δt Taking the target vehicle as an example, it can be used... The natural probability distribution of the target vehicle's state value at time T in the corresponding fragmented scene. Let X0, ..., X represent the state values ​​of the target vehicle at time T. T-Δt This represents the state values ​​of the target vehicle and its background at times preceding time T. The conditional probability distribution of the predicted state value of the target vehicle at time T output by the driving behavior model can be expressed as follows: in This represents the predicted state of the target vehicle at time T. In actual training, it's not necessary to use the state values ​​of the target vehicle and its background vehicles from all times prior to time T as model input to estimate the predicted state of the target vehicle at time T. In one example, a time before time T (T-Δt) can be used as a given time. The state values ​​of the target vehicle and its background vehicles at that given time are used as input to the driving behavior model. The model output can then yield the predicted state of the target vehicle at the next time step (T) and its conditional probability distribution. In some examples, the state values ​​of the target vehicle and its background vehicles at two or more time points can be used as input data for the model. Taking the state values ​​of the target vehicle and its background vehicles at a given time (T-Δt) and the previous time point (T-2Δt) as input data as an example, the predicted state value of the target vehicle at the next time point (T) and its conditional probability distribution can be obtained from the model output.

[0055] In one exemplary embodiment, T j The moment is T iThe first time after the given moment. That is, during a single training iteration of the model, the driving behavior model is driven only once based on the scene data at a given moment in the training set, yielding the predicted state of the target vehicle at the next moment. In this case, the data collection vehicle in the scene can be designated as the target vehicle; this vehicle is equipped with onboard sensors to collect training data. The state value of the data collection vehicle can be directly read from the driving system, which is more accurate, and the driver of the data collection vehicle can be selected as needed, such as choosing a driver with a certain driving habit, facilitating the training of the desired driving behavior model.

[0056] In one exemplary embodiment, T j The moment is T i At the K'th time point after time point T, K'≥2. Inputting the scene data into the driving behavior model involves taking a segment of the selected scene T from this training session. i The scene data at each time point is input into M driving behavior models, and different target vehicles are specified for different driving behavior models. M is the total number of vehicles in the scene data.

[0057] Among them, the target vehicle in the scene at T is obtained from the output of the driving behavior model. j Conditional probability distribution P of the state prediction value at time t. θ It can include:

[0058] Step 1: Obtain the values ​​of all M vehicles in scenario T from the outputs of the M driving behavior models. i+1 The predicted state value at time;

[0059] Step two, determine whether all vehicles in the scene have been obtained at T. j-1 If the predicted state value at time 1 is true, proceed to step 4; otherwise, proceed to step 3.

[0060] Step 3: Input the state prediction value of each vehicle in the latest scene and the scene map of the fragmented scene into one of the driving behavior models, and assign different target vehicles to different driving behavior models; obtain the state prediction value of all vehicles in the scene at the next moment from the output of the driving behavior model, and return to step 2;

[0061] Step 4: Place all vehicles in the scene at T j-1 The predicted state value at time T and the scene map of the fragmented scene are input into a driving behavior model, with the collected vehicle in the scene as the target vehicle; the output of the driving behavior model is used to obtain the target vehicle's state at time T. j Conditional probability distribution P of the state prediction value at time t. θ ;

[0062] The step of adjusting the model parameters according to the loss means simultaneously adjusting the parameters of all driving behavior models used in training based on the loss.

[0063] In one example of this embodiment, it is assumed that a fragmented scene includes scene data at five time points (T1-T5), and there are five vehicles in the scene. When training the driving behavior model, the scene data at time T1 is used as input to the five driving behavior models. The output of the five driving behavior models yields the predicted state values ​​of all vehicles in the scene at time T2. Then, the predicted state values ​​of all vehicles at time T2 and the scene map are input into the five driving behavior models to obtain the predicted state values ​​of all vehicles at time T3, and so on. After obtaining the predicted state values ​​of all vehicles at time T4 from the model output, the sampled vehicle can be designated as the target vehicle. The predicted state values ​​of all vehicles at time T4 and the scene map are input into one of the driving behavior models. The output of this driving behavior model yields the predicted state value of the target vehicle (e.g., the sampled vehicle) at time T5. At this point, the conditional probability distribution P of the predicted state value of the target vehicle at time T5 is used. θ The training loss is calculated using the natural probability distribution P of the target vehicle's state value at time T5 in the training set. The parameters of all five driving behavior models are then updated synchronously based on the calculated loss.

[0064] In this embodiment, the driving behavior model is driven multiple times during a single training process using scene data or prediction results, resulting in predicted vehicle state values ​​at multiple time points. When calculating the loss, the sampled vehicle can be considered as the target vehicle, and only the target vehicle at time T is considered. j Conditional probability distribution P of the state prediction value at time t. θ The divergence between the state value and the natural probability distribution P, and P at previous times. θ The divergence between P and T can also be reflected in T. j Time P θ The divergence between P and P. However, the embodiments of this disclosure are based on P. θ The divergence between P and T determines the loss, but this does not mean that the target vehicle can only be used at T. j Conditional probability distribution P of the state prediction value at time t. θ The loss can be calculated using the natural probability distribution P of the state values; for example, in other examples, it can be calculated based on T. i+1 Time to T j The loss is calculated by the average or cumulative value of the divergence between the conditional probability distribution of the predicted state value of the target vehicle at any given time and the natural probability distribution of the state value.

[0065] In this embodiment, during a training process, if one or more vehicles drive outside the preset target area at a certain moment, such as being too far away from the data collection vehicle, the state of these vehicles at subsequent moments can no longer be predicted, and the number of driving behavior models used can be reduced accordingly.

[0066] In one exemplary embodiment, the driving behavior model is a deep neural network model, which may employ the Transformer architecture. However, this disclosure is not limited to this, and other machine learning models may also be used.

[0067] In one exemplary embodiment, the training method may further include: verifying the prediction accuracy of the driving behavior model based on a validation set, provided that the training of the driving behavior model satisfies a preset convergence condition; and obtaining the trained driving behavior model upon successful verification. The convergence condition may include: based on P and P... θ The loss determined by the divergence between the parameters is less than a preset loss threshold. The validation set here can be a fragmented scene collected from a real road environment, or a validation set used in some standards; this disclosure is not limited to this. If the convergence condition is not met, training continues based on scene data from subsequent time steps of the current fragmented scene or scene data from a new fragmented scene, with each training iteration using different scene data. If the convergence condition is still not met after reaching the set maximum number of training iterations, the model structure or initialization parameters can be adjusted, and training can be restarted.

[0068] The Transformer architecture is a deep learning architecture that can include modules such as encoders and decoders. Specifically, they can work closely together through an attention mechanism. The encoder processes the input of the source language (such as the correlation data between the target vehicle and surrounding background vehicles) and generates semantic vectors, while the decoder uses these semantic vectors to generate the output of the target language (the predicted state value of the target vehicle).

[0069] Figure 2 This illustration shows a schematic diagram of the training process when the driving behavior model in this embodiment of the present disclosure uses a deep neural network based on the Transformer architecture. The driving behavior model may include: a first mapping module, a second mapping model, an encoding module, and a decoding module. Wherein:

[0070] The first mapping module 10 is configured to map the input scene data into latent variables (corresponding feature vectors). The input scene data includes the state data of all vehicles in the scene at a given time and the scene map. Here, the state data of the vehicles can be the collected state values ​​of the vehicles, the predicted state values ​​of the vehicles output by the model, or other values ​​of the state parameters, such as set values. This disclosure does not limit this.

[0071] The second mapping module 20 is configured to map the state data of the target vehicle in the input scene data to latent variables; for all vehicles in the scene, any one of them can be specified as the target vehicle.

[0072] Transformer encoding module 30 is configured to encode (feature extraction) the latent variables output by the first mapping module and output a semantic vector (context representation);

[0073] The Transformer decoding module 40 is configured to decode the semantic vector output by the encoding module and output the predicted state value of the target vehicle at the next time step. During training, it also outputs the conditional probability distribution of the predicted state value of the target vehicle at the next time step.

[0074] The aforementioned neural network model sets up two mapping modules. One mapping is used to convert the sequence of state data of all vehicles into latent variables for encoding, while the other mapping is used specifically to convert the state data of the target vehicle into latent variables for encoding. In this way, the target vehicle can be specified, and the simulation can be more effective in predicting the target vehicle.

[0075] The driving behavior model training method of the above embodiments of this disclosure can train the driving behavior model of a vehicle in a complex environment based on a training set collected from a real road environment, thereby effectively ensuring that the behavior of the background vehicle in the natural driving environment is close to the behavior state of the vehicle in the real environment.

[0076] Figure 3 This is a flowchart of a vehicle state prediction method according to an embodiment of the present disclosure. This embodiment predicts the vehicle state based on the driving behavior model trained as described above. The prediction method may include the following steps:

[0077] S301, Input the scene data into the driving behavior model, which is trained by the driving behavior model training method of any embodiment of the present disclosure;

[0078] The input scene data includes: a scene map, and state data of the target vehicle and its background vehicles in the scene at a given time. The state data includes the values ​​of the vehicle's state parameters. As mentioned above, the values ​​of the state parameters can be detected values, i.e., state values, or state prediction values ​​output by the model, or initial values ​​set for the background vehicles in the test, etc. This disclosure does not limit them.

[0079] S302, obtain the predicted state value of the target vehicle at the next moment from the output of the driving behavior model. The target vehicle is a designated vehicle among all vehicles in the scenario.

[0080] In different applications, different vehicles can be designated as target vehicles. For example, when training a driving behavior model, the vehicle in the scene being collected can be designated as the target vehicle, or a background vehicle of the collected vehicle can be designated as the target vehicle; during the testing of an autonomous driving system, the background vehicle of the vehicle being tested can be designated as the target vehicle, so that the driving behavior model can be used to simulate the driving behavior of the background vehicle.

[0081] When simulating and testing an autonomous driving system, the vehicle equipped with the system under test can be called the vehicle under test. Other vehicles in the test scenario besides the vehicle under test are called background vehicles, to distinguish them from the background vehicles of the target vehicle in the driving behavior model. In this embodiment, the background vehicles of the vehicle under test in the test scenario are called the first background vehicles, while the background vehicles of the target vehicle specified in the scene data input to the driving behavior model are called the second background vehicles. It is easy to understand that during testing, the first background vehicle can be designated as the target vehicle for which the driving behavior model needs to make predictions, while the vehicle under test can serve as the second background vehicle for the target vehicle.

[0082] The embodiments disclosed herein are not limited to applications in the testing of autonomous driving systems. In situations such as testing the vehicle under test, such as crash tests, the natural driving environment can also be reconstructed based on the methods of the embodiments disclosed herein.

[0083] One embodiment of this disclosure provides a method for reconstructing a natural driving environment, which reconstructs the natural driving environment based on a trained driving behavior model, such as... Figure 4 As shown, the reconstruction method includes:

[0084] S401, Obtain initial scene data for reconstructing the natural driving environment.

[0085] The initial scene data may include: a scene map, and the state data of one test vehicle and N background vehicles at time T0. In other words, the total number of vehicles in the initial scene is N+1. The input initial scene data can be collected from a real road environment, or it can be data set by the tester through the simulation system during simulation testing, or data automatically generated by the simulation system, etc.

[0086] S402, input the initial scene data into the N pre-trained driving behavior models respectively, assign different first background vehicles as target vehicles for different driving behavior models, and use the test vehicle and the other first background vehicles as the second background vehicles of the corresponding target vehicles. Obtain the state prediction values ​​of N first background vehicles at time T1 from the output of the N driving behavior models.

[0087] This step takes time T0 as the given time, inputs the scene data at the given time into the driving behavior model, and obtains the predicted state value of the target vehicle at the next time (i.e., time T1).

[0088] S403 reconstructs the natural driving environment of the test vehicle at time T1 based on the predicted state values ​​of N first background vehicles at time T1 and the scene map.

[0089] For example, the initial scenario might include a test vehicle and 10 background vehicles. Each of the 10 driving behavior models corresponds to one of these background vehicles, and one background vehicle is designated as the target vehicle for each of the 10 models. For instance, in the first driving behavior model, if background vehicle A is designated as the target vehicle, then the test vehicle and the other 9 background vehicles act as secondary background vehicles in that model. Similarly, in the second driving behavior model, if background vehicle B is designated as the target vehicle, then the test vehicle and the other 9 background vehicles (including background vehicle A) act as secondary background vehicles in that model, and so on.

[0090] By simulating all the first background vehicles in the test scenario, the natural driving environment can be reconstructed in a continuous space. This embodiment can also use data such as the state prediction values ​​of the first background vehicles output by the driving behavior model at the previous moment to drive the driving behavior model multiple times, thereby reconstructing the natural driving environment in continuous time.

[0091] In one exemplary embodiment, this application can be based on the reconstructed T in the following manner. l Reconstructing the natural driving environment at all times (T) l+1 Time (T) l The natural driving environment at the next time step (time), where l is an integer greater than or equal to 1:

[0092] Get T l Scene data at any given time, including: data obtained from the test vehicle at time T. l State data at time T, N' first background vehicles at time T l Predicted state values ​​and scene map at any given time.

[0093] T l Scene data at time T is input into N' driving behavior models. Different first background vehicles are assigned as target vehicles for different driving behavior models. The test vehicle and all other first background vehicles besides the target vehicle are used as second background vehicles for the target vehicle. The outputs of the N' driving behavior models yield N' first background vehicles at time T. l+1 The predicted state value at time T is based on N' first background vehicles at time T.l+1 The predicted state value at the time and the scene map are used to reconstruct the time T of the tested vehicle l+1 natural driving environment at the moment.

[0094] During the test, some background vehicles may drive to a position too far away from the tested vehicle and thus drive out of the area requiring attention. At this time, these background vehicles may no longer be simulated, and the number N' of the first background vehicles in the subsequent driving simulation is less than N to simplify the calculation. However, in other embodiments, all the first background vehicles may be simulated all the time, or when no first background vehicle drives out of the area requiring attention, N'=N.

[0095] Figure 5 shows a schematic diagram of the process of using 10 trained driving behavior models to simultaneously predict the states of 10 first background vehicles. T l The state data of each target vehicle (i.e., each first background vehicle) and the surrounding second background vehicles (including the tested vehicle) at the moment are respectively input into the corresponding driving behavior model to perform T l+1 The state of each target vehicle at the moment is predicted to obtain T l+1 The predicted state value of each target vehicle at the moment, and then based on the 10 first background vehicles at T l+1 The predicted state value at the time and the scene map are used to reconstruct the time T of the tested vehicle l+1 natural driving environment at the moment.

[0096] In an example of this embodiment, each of the N driving behavior models is a driving behavior model trained by the driving behavior model training method according to the embodiment of the present disclosure. The above state data may include values of state parameters of the vehicle, and the state parameters include any one or more of speed, acceleration, position and attitude. The values of the state parameters in the state data may be state values in a real environment, state predicted values output by the model, or values set or generated by a simulation system, which is not limited in the present disclosure. However, in another example of this embodiment, when reconstructing the spatiotemporally continuous natural driving environment, the used driving behavior model may not be the driving behavior model trained by the driving behavior model training method according to the embodiment of the present disclosure, and other driving behavior models may also be used to still reconstruct the spatiotemporally continuous natural driving environment.

[0097] The natural driving environment reconstruction method of the above embodiments of this disclosure reconstructs a spatiotemporally continuous natural driving environment by serially calling the driving behavior model in time and in parallel calling it in space. By using a model-driven method to simulate the background vehicle in the test environment of the autonomous driving system and interact with the autonomous driving system under test in real time, it can simulate complex real road conditions and test cars equipped with autonomous driving systems in complex interactive environments. This effectively ensures that the behavior of the background vehicle in the natural driving environment is close to the behavior of the vehicle in the real environment, thus improving the accuracy of the test.

[0098] This disclosure also provides a testing method for an autonomous driving system, such as... Figure 6 As shown, the test method includes:

[0099] Step S601: Reconstruct the natural driving environment, wherein the natural driving environment is reconstructed using the natural driving environment reconstruction method of any embodiment of this disclosure;

[0100] Step S602: Test the test vehicle equipped with an autonomous driving system based on the reconstructed natural driving environment to obtain test data of the test vehicle.

[0101] See Figure 7 This diagram illustrates the overall flow of a test process for an autonomous vehicle, from data acquisition in a real-world environment to testing, according to an embodiment of this disclosure. The process includes:

[0102] The first step is data collection;

[0103] This step can be based on roadside sensors or vehicle-mounted sensors to collect vehicle and road-related data.

[0104] The second step is to obtain the natural driving dataset;

[0105] This step can process the collected data, including cleaning invalid data, standardizing data format, aligning data coordinate systems, etc., to obtain a natural driving dataset.

[0106] The third step is to obtain scene data for the fragmented scene;

[0107] This step can fragment the centralized data of the natural driving dataset to obtain scene data for multiple fragmented scenarios. Each fragmented scenario can include scene data from multiple time points.

[0108] The fourth step is to learn the natural driving distribution based on the driving behavior model;

[0109] This step uses scene data from fragmented scenarios as input to train the driving behavior model, enabling it to learn the natural driving distribution. For example, it simulates and collects the driving behavior of vehicles, so that the conditional probability distribution of the target vehicle's state prediction value at the next moment output by the model is close to the natural probability distribution of the target vehicle's state value at the next moment.

[0110] The fifth step is to conduct a distribution consistency assessment;

[0111] This step can use a subset of fragmented scenes from the natural driving dataset as a validation set to evaluate the distribution consistency of the trained driving behavior model. For example, the KL divergence between the natural probability distribution of the target vehicle's state value at a certain moment and the conditional probability distribution of the predicted state value at that moment can be used as an indicator of distribution consistency.

[0112] The sixth step, if the evaluation is passed and the training is completed, is to conduct simulation tests of the autonomous driving system based on the test platform. The driving behavior model can be used to reconstruct the natural driving environment of the test vehicle.

[0113] Step 7: Obtain the test results;

[0114] The eighth step is to evaluate the behavioral capabilities of the autonomous driving system based on the test results.

[0115] Therefore, this application utilizes a data-driven approach to obtain a driving behavior model of a vehicle in a complex environment. By serially calling the model in time and concurrently calling it in space, a spatiotemporally continuous natural driving environment is reconstructed. The background vehicle in the natural driving environment interacts with the autonomous driving system under test in real time through a model-driven method, simulating complex real road conditions. The autonomous vehicle is tested in a complex interactive environment, which effectively ensures that the behavior of the background vehicle in the natural driving environment is close to the behavior of the vehicle in the real environment, thus improving the accuracy of the test.

[0116] One embodiment of this disclosure also provides a driving behavior model training device, such as... Figure 8 As shown, it includes: a memory 50 and a processor 60, wherein the memory 50 stores a computer program; wherein the processor 60 is configured to execute the computer program in the memory to implement the driving behavior model training method described in any embodiment of this disclosure.

[0117] One embodiment of this disclosure also provides a vehicle state prediction device, see [link to relevant documentation]. Figure 8 The system includes a memory and a processor, wherein the memory stores a computer program; and the processor is configured to execute the computer program in the memory to implement the vehicle state prediction method according to any embodiment of the present disclosure.

[0118] One embodiment of this disclosure also provides a natural driving environment reconstruction device, see [link to relevant documentation]. Figure 8 The system includes a memory and a processor, wherein the memory stores a computer program; wherein the processor is configured to execute the computer program in the memory to implement the natural driving environment reconstruction method according to any embodiment of the present disclosure.

[0119] This disclosure also provides a testing apparatus for an autonomous driving system, see [link to relevant documentation]. Figure 8 The system includes a memory and a processor, wherein the memory stores a computer program; wherein the processor is configured to execute the computer program in the memory to implement the testing method of the autonomous driving system according to any embodiment of the present disclosure.

[0120] An embodiment of this disclosure also provides a storage medium storing computer-executable commands for executing the driving behavior model training method, vehicle state prediction method, natural driving environment reconstruction method, or autonomous driving system testing method as described in any of the above embodiments.

[0121] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term "computer storage medium" includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0122] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0123] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, and substitutions to the above embodiments within the scope of this application.

Claims

1. A method for training a driving behavior model, wherein the driving behavior model is configured to output a predicted state value of a target vehicle in a given scene at the next moment, based on scene data at a given time, the method comprising: Multiple fragmented scenes were collected from real road environments and used as a training set; Each fragmented scene includes scene data at multiple moments, and the scene data includes: a scene map, and the state values ​​of a target vehicle and its background vehicles in the scene; The driving behavior model is trained multiple times based on the training set; each training session includes: inputting scene data into the driving behavior model, the input scene data including a segmented scene T selected for this training. i The scene data at time T is obtained from the output of the driving behavior model to determine the target vehicle's position in the scene. j Conditional probability distribution P of the state prediction value at time t. θ Based on this fragmented scenario T j Scene data at time T j The state value of the target vehicle at time t and the natural probability distribution P of the acquired state value; and, according to P and P θ The divergence between the values ​​determines the loss, and the model parameters are adjusted based on the loss. Among them, T j The moment is T i The Kth time after time point K is a preset positive integer, which is greater than or equal to 1.

2. The method according to claim 1, characterized in that: The multiple fragmented scenarios are obtained based on data collected by onboard sensors in the designed operating domain. The state value is the actual value of the vehicle's state parameters, which include any one or more of velocity, acceleration, position, and attitude.

3. The method according to claim 2, characterized in that: T j The moment is T i The first moment after the previous moment; The step of inputting scene data into the driving behavior model involves selecting a fragmented scene T for this training. i The scene data at any given time is input into one driving behavior model, with the collected vehicle in the scene as the target vehicle.

4. The method according to claim 2, characterized in that: T j The moment is T i At the K'th time after time, K'≥2; The step of inputting scene data into the driving behavior model involves selecting a fragmented scene T for this training. i The scene data at each moment is input into M driving behavior models, and different target vehicles are specified for different driving behavior models, where M is the total number of vehicles in the scene data; The target vehicle in the scenario is obtained from the output of the driving behavior model at T. j Conditional probability distribution P of the state prediction value at time t. θ ,include: Step 1: Obtain the outputs of the M driving behavior models for all M vehicles in scenario T. i+1 The predicted state value at time; Step two, determine whether all vehicles in the scene have been obtained at T. j-1 If the predicted state value at time 1 is true, proceed to step 4; otherwise, proceed to step 3. Step 3: Input the state prediction value of each vehicle in the latest scene and the scene map of the fragmented scene into one of the driving behavior models, and assign different target vehicles to different driving behavior models; obtain the state prediction value of all vehicles in the scene at the next moment from the output of the driving behavior model, and return to step 2; Step 4: Place all vehicles in the scene at T j-1 The predicted state value at time T and the scene map of the fragmented scene are input into a driving behavior model, with the collected vehicle in the scene as the target vehicle; the output of the driving behavior model is used to obtain the target vehicle at time T. j Conditional probability distribution P of the state prediction value at time t. θ ; The step of adjusting the model parameters according to the loss means simultaneously adjusting the parameters of all driving behavior models used in training based on the loss.

5. The method according to any one of claims 1 to 4, characterized in that: The driving behavior model is a deep neural network model, and the deep neural network model adopts the Transformer architecture; The method further includes: if the training of the driving behavior model satisfies a preset convergence condition, verifying the prediction accuracy of the driving behavior model based on a fragmented scenario used for verification; after successful verification, a trained driving behavior model is obtained; the convergence condition includes: based on P and P θ The loss determined by the divergence between them is less than the preset loss threshold.

6. A vehicle state prediction method, comprising: The scene data is input into the driving behavior model, which is trained by the driving behavior model training method as described in any one of claims 1 to 5. The predicted state of the target vehicle at the next moment is obtained from the output of the driving behavior model; The input scene data includes: a scene map, and state data of the target vehicle and its background vehicles in the scene at a given time. The state data includes the values ​​of the vehicle's state parameters. The target vehicle is a designated vehicle among all the vehicles in the scene.

7. A method for reconstructing a natural driving environment, comprising: Acquire initial scene data for reconstructing the natural driving environment. The initial scene data includes: a scene map, and state data of one test vehicle and N first background vehicles in the scene at time T0. The initial scene data is input into N pre-trained driving behavior models, and different first background vehicles are assigned as target vehicles for different driving behavior models. The test vehicle and the other first background vehicles are all used as second background vehicles for the corresponding target vehicles. The state prediction values ​​of the N first background vehicles at time T1 are obtained from the output of the N driving behavior models. Based on the predicted state values ​​of the N first background vehicles at time T1 and the scene map, the natural driving environment of the test vehicle at time T1 is reconstructed.

8. The method as described in claim 7, characterized in that, Also includes: Based on the reconstructed T in the following manner l Reconstructing the natural driving environment at all times (T) l+1 Natural driving environment at all times: Get T l Scene data at any given time, including: data obtained from the test vehicle at time T. l The state data at time T1, the predicted state values ​​of N' first background vehicles at time T1, and the scene map, where N'≤N; T l The scene data at each moment is input into the N' driving behavior models respectively, and different first background vehicles are assigned as target vehicles for different driving behavior models. The test vehicle and the other first background vehicles are all used as the second background vehicles of the target vehicle. The N' first background vehicles at T are obtained from the outputs of the N' driving behavior models. l+1 The predicted state value at time T is based on the N' first background vehicles at time T. l+1 Based on the predicted state values ​​at time T and the scene map, the test vehicle is reconstructed at time T. l+1 A natural driving environment at all times.

9. The method as described in claim 7, characterized in that, Also includes: The N trained driving behavior models all adopt the trained driving behavior model as described in claim 5; The state data includes the values ​​of the vehicle's state parameters, which include any one or more of velocity, acceleration, position, and attitude.

10. A testing method for an autonomous driving system, comprising: The test vehicle equipped with an autonomous driving system is tested based on the reconstructed natural driving environment to obtain test data of the test vehicle; wherein the natural driving environment is reconstructed using the method described in any one of claims 7 to 9.

11. A driving behavior model training device, comprising: A memory and a processor, wherein the memory stores a computer program; wherein the processor is configured to execute the computer program in the memory to implement the driving behavior model training method as described in any one of claims 1-5.

12. A natural driving environment reconstruction device, comprising: A memory and a processor, wherein the memory stores a computer program; wherein the processor is configured to execute the computer program in the memory to implement the natural driving environment reconstruction method as described in any one of claims 7-9.

13. A computer storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the driving behavior model training method as described in any one of claims 1-5, the vehicle state prediction method as described in claim 6, the natural driving environment reconstruction method as described in any one of claims 7-9, or the autonomous driving system testing method as described in claim 10.

Images