Automatic driving simulation method and model training method and device

Abstract

The invention provides an automatic driving simulation method and device, relates to the technical field of automatic driving, and can be applied to an automatic driving simulation scene. According to the specific implementation scheme, the method comprises the steps of obtaining first driving information of a target simulation vehicle and second driving information of a background simulation vehicle; determining a first to-be-controlled vehicle from the background simulation vehicles according to the first driving information and the second driving information; and controlling the first vehicle to be controlled. According to the method and the device, the background vehicle can generate reactive behaviors aiming at the main vehicle, so that the accuracy of an automatic driving simulation test and the fineness of vehicle control in an automatic driving simulation scene are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of autonomous driving technology, and in particular to an autonomous driving simulation method, device, equipment and storage medium, which can be applied to the scenario of autonomous driving simulation. Background Art

[0002] Autonomous driving simulation testing is the process of using computer simulation environments and virtual vehicles to test the performance, safety and stability of autonomous driving systems.

[0003] At present, when conducting autonomous driving simulation tests, various test cases can be designed based on road data, and then the designed test cases can be run in a simulation environment to test the autonomous driving system.

[0004] However, current autonomous driving simulation testing methods are not very accurate. Summary of the invention

[0005] The present disclosure provides an autonomous driving simulation method and a model training method and device, which can enable background vehicles to produce reactive behaviors towards a main vehicle, thereby improving the accuracy of autonomous driving simulation tests and the precision of vehicle control in autonomous driving simulation scenarios.

[0006] According to a first aspect of the present disclosure, an autonomous driving simulation method is provided, the method comprising: obtaining first driving information of a target simulated vehicle and second driving information of a background simulated vehicle in an autonomous driving simulation environment; determining a first vehicle that affects the autonomous driving simulation from the background simulated vehicles based on the first driving information and the second driving information, and changing the driving information of the first vehicle to obtain an updated autonomous driving simulation environment; and performing autonomous driving simulation on the target simulated vehicle based on the updated autonomous driving simulation environment.

[0007] In some possible implementations, based on the first driving information and the second driving information, a first vehicle that affects the automatic driving simulation is determined from the background simulation vehicles, and the driving information of the first vehicle is changed to obtain an updated automatic driving simulation environment, including: based on the first driving information and the second driving information, a second vehicle that affects the driving of the target simulation vehicle is determined from the background simulation vehicles, and the second driving information corresponding to the second vehicle is changed to obtain the changed driving information corresponding to the second vehicle; a third vehicle that affects the driving of the vehicle in the background simulation vehicles that has changed the second driving information is determined, and the second driving information corresponding to the third vehicle is changed until there are no more vehicles in the background simulation vehicles that affect the driving of other vehicles, to obtain an updated automatic driving simulation environment; wherein, the vehicle that has changed the second driving information includes the second vehicle, and the first vehicle includes the vehicle that has changed the second driving information.

[0008] In some possible implementations, the first driving information includes a first historical trajectory and first driving data, the second driving information includes a second historical trajectory and second driving data, and a second vehicle that affects the driving of the target simulated vehicle is determined from background simulated vehicles based on the first driving information and the second driving information, including: predicting a first driving trajectory based on the first historical trajectory and the first driving data; predicting a second driving trajectory based on the second historical trajectory and the second driving data; and determining the second vehicle from background simulated vehicles based on the first driving trajectory and the second driving trajectory.

[0009] In some possible implementations, based on the first driving information and the second driving information, a first vehicle that affects the autonomous driving simulation is determined from background simulated vehicles, and the driving information of the first vehicle is changed to obtain an updated autonomous driving simulation environment, including: determining a target point where a bounding box of the first vehicle overlaps with a bounding box of a target simulated vehicle; determining a target path for the first vehicle to reach the target point, and according to the target path, changing the driving information of the first vehicle to obtain an updated autonomous driving simulation environment.

[0010] In some possible implementations, based on the first driving information and the second driving information, the first vehicle that affects the automatic driving simulation is determined from the background simulated vehicles, and the driving information of the first vehicle is changed to obtain an updated automatic driving simulation environment, including: obtaining the driving data of the first vehicle in the first time period; inputting the driving data of the first vehicle in the first time period into the strategy model, and outputting the driving data of the first vehicle in the second time period through the strategy model to change the driving information of the first vehicle to obtain an updated automatic driving simulation environment, the first time period is the previous time period of the second time period, and the strategy model has the function of outputting the driving data of the vehicle in the second time period according to the input driving data of the vehicle in the first time period.

[0011] In some possible implementations, based on the first driving information and the second driving information, the first vehicle that affects the autonomous driving simulation is determined from the background simulated vehicles, and the driving information of the first vehicle is changed to obtain an updated autonomous driving simulation environment, including: based on the first driving information and the second driving information, the first vehicle that affects the autonomous driving simulation is determined from the background simulated vehicles, and the driving information of the first vehicle is changed within a preset time period to obtain an updated autonomous driving simulation environment.

[0012] In some possible implementations, predicting a first driving trajectory based on a first historical trajectory and first driving data includes: taking the first historical trajectory and the first driving data as input, and outputting the first driving trajectory through a prediction model, wherein the prediction model includes an encoder, a dynamic encoder, an interactive relationship module, and a decoder, and the prediction model has the function of predicting the vehicle's driving trajectory based on the input vehicle's historical trajectory and driving data.

[0013] In some possible implementations, the first historical trajectory and the first driving data are taken as input, and the first driving trajectory is output through a prediction model, including: inputting the first historical trajectory and the first driving data into a dynamic encoder, and outputting a target tensor through the dynamic encoder; inputting the target tensor into an interaction relationship module, and outputting an interaction relationship matrix through the interaction relationship module; performing data processing on the interaction relationship matrix to obtain action variables; encoding the first historical trajectory through an encoder to obtain a target feature vector; and inputting the target feature vector and the action variable into a decoder to obtain the first driving trajectory.

[0014] The first aspect of the present disclosure has at least the following beneficial effects: by obtaining the first driving information of the target simulated vehicle and the second driving information of the background simulated vehicle in the autonomous driving simulation environment; then according to the first driving information and the second driving information, determining the first vehicle that affects the autonomous driving simulation from the background simulated vehicles, and changing the driving information of the first vehicle to obtain an updated autonomous driving simulation environment; finally, according to the updated autonomous driving simulation environment, performing autonomous driving simulation on the target simulated vehicle. The background vehicle can generate reactive behavior against the main vehicle, thereby improving the accuracy of the autonomous driving simulation test and the precision of vehicle control in the autonomous driving simulation scenario, and resolving potential conflicts in the traffic simulation scenario.

[0015] According to a second aspect of the present disclosure, a model training method is provided, the method comprising:

[0016] Acquire driving data of the first target vehicle to be tested in a third time period and driving data of the first target vehicle to be tested in a fourth time period; use the driving data of the first target vehicle to be tested in the third time period as input, use the driving data of the first target vehicle to be tested in the fourth time period as output, train the neural network, and obtain a strategy model, the third time period is the previous time period of the fourth time period, and the strategy model has the function of outputting the driving data of the vehicle in the fourth time period according to the input driving data of the vehicle in the third time period.

[0017] In some possible implementations, the method further includes: updating the network parameters of the policy model by using a trust region policy optimization algorithm.

[0018] According to a third aspect of the present disclosure, a model training method is provided, the method comprising:

[0019] Obtaining the historical trajectory, driving data and driving trajectory of the second target vehicle to be tested;

[0020] The historical trajectory and driving data of the second target vehicle to be tested are taken as input, and the driving trajectory of the second target vehicle to be tested is taken as output, and the neural network is trained to obtain a prediction model. The prediction model includes an encoder, a dynamic encoder, an interactive relationship module, and a decoder. The prediction model has the function of predicting the vehicle's driving trajectory based on the input vehicle's historical trajectory and driving data.

[0021] According to a fourth aspect of the present disclosure, an autonomous driving simulation device is provided, the device comprising: an acquisition unit, a determination unit, and a simulation unit.

[0022] An acquisition unit, used to acquire first driving information of a target simulated vehicle and second driving information of a background simulated vehicle in an automatic driving simulation environment;

[0023] a determining unit, configured to determine, from the background simulation vehicles, a first vehicle that affects the automatic driving simulation according to the first driving information and the second driving information, and to change the driving information of the first vehicle to obtain an updated automatic driving simulation environment;

[0024] The simulation unit is used to perform autonomous driving simulation on a target simulation vehicle according to an updated autonomous driving simulation environment.

[0025] Optionally, the determination unit is specifically used to: determine, from the background simulation vehicles, a second vehicle that affects the driving of the target simulation vehicle based on the first driving information and the second driving information, and change the second driving information corresponding to the second vehicle to obtain the changed driving information corresponding to the second vehicle; determine a third vehicle that affects the driving of the vehicle that has changed the second driving information in the background simulation vehicles, and change the second driving information corresponding to the third vehicle until there are no more vehicles in the background simulation vehicles that affect the driving of other vehicles, and an updated automatic driving simulation environment is obtained; wherein, the vehicle that has changed the second driving information includes the second vehicle, and the first vehicle includes the vehicle that has changed the second driving information.

[0026] Optionally, the first driving information includes a first historical trajectory and first driving data, the second driving information includes a second historical trajectory and second driving data, and the determination unit is specifically used to: predict a first driving trajectory based on the first historical trajectory and the first driving data; predict a second driving trajectory based on the second historical trajectory and the second driving data; and determine a second vehicle from background simulated vehicles based on the first driving trajectory and the second driving trajectory.

[0027] Optionally, the determination unit is specifically used to: determine a target point where the bounding box of the first vehicle overlaps with the bounding box of the target simulation vehicle; determine a target path for the first vehicle to reach the target point, and according to the target path, change the driving information of the first vehicle to obtain an updated autonomous driving simulation environment.

[0028] Optionally, the determination unit is specifically used to: obtain driving data of the first vehicle in a first time period; input the driving data of the first vehicle in the first time period into a strategy model, and output the driving data of the first vehicle in a second time period through the strategy model to change the driving information of the first vehicle and obtain an updated autonomous driving simulation environment, wherein the first time period is the previous time period of the second time period, and the strategy model has the function of outputting the driving data of the vehicle in the second time period based on the input driving data of the vehicle in the first time period.

[0029] Optionally, the determination unit is specifically used to: determine the first vehicle that affects the automatic driving simulation from the background simulation vehicles based on the first driving information and the second driving information, and change the driving information of the first vehicle within a preset time period to obtain an updated automatic driving simulation environment.

[0030] Optionally, the determination unit is specifically used to: take the first historical trajectory and the first driving data as input, and output the first driving trajectory through a prediction model. The prediction model includes an encoder, a dynamic encoder, an interactive relationship module, and a decoder. The prediction model has the function of predicting the vehicle's driving trajectory based on the input vehicle's historical trajectory and driving data.

[0031] Optionally, the determination unit is specifically used to: input the first historical trajectory and the first driving data into a dynamic encoder, and output a target tensor through the dynamic encoder; input the target tensor into an interaction relationship module, and output an interaction relationship matrix through the interaction relationship module; perform data processing on the interaction relationship matrix to obtain action variables; encode the first historical trajectory through the encoder to obtain a target feature vector; input the target feature vector and the action variable into a decoder to obtain a first driving trajectory.

[0032] According to a fifth aspect of the present disclosure, a model training device is provided, comprising: an acquisition unit and a training unit.

[0033] The acquisition unit is used to acquire the driving data of the first target vehicle to be tested in the third time period and the driving data in the fourth time period.

[0034] The training unit is used to take the driving data of the first target vehicle to be tested in the third time period as input and the driving data of the first target vehicle to be tested in the fourth time period as output, train the neural network and obtain a strategy model. The third time period is the previous time period of the fourth time period. The strategy model has the function of outputting the driving data of the vehicle in the fourth time period based on the input driving data of the vehicle in the third time period.

[0035] Optionally, the device further includes: an updating unit.

[0036] The updating unit is used to update the network parameters of the policy model through the trust region policy optimization algorithm.

[0037] According to a sixth aspect of the present disclosure, a model training device is provided, the device comprising: an acquisition unit and a training unit.

[0038] The acquisition unit is used to acquire the historical track, driving data and driving track of the second target vehicle to be tested.

[0039] The training unit is used to take the historical trajectory and driving data of the second target vehicle to be tested as input and the driving trajectory of the second target vehicle to be tested as output, train the neural network, and obtain a prediction model. The prediction model includes an encoder, a dynamic encoder, an interactive relationship module, and a decoder. The prediction model has the function of predicting the vehicle's driving trajectory based on the input vehicle's historical trajectory and driving data.

[0040] According to the seventh aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the methods described in the first to third aspects.

[0041] According to an eighth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method according to the first to third aspects.

[0042] According to a ninth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method according to the first to third aspects.

[0043] The beneficial effects of the second to ninth aspects of the present disclosure can refer to the beneficial effects of the first aspect and will not be repeated here.

[0044] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0046] Figure 1 A schematic diagram of a flow chart of an autonomous driving simulation method provided in an embodiment of the present disclosure;

[0047] Figure 2 The embodiments of the present disclosure provide Figure 1 A schematic diagram of an implementation process of S102;

[0048] Figure 3 The embodiments of the present disclosure provide Figure 2 A schematic diagram of an implementation process of S201;

[0049] Figure 4 The embodiments of the present disclosure provide Figure 1 Another implementation flow diagram of S102;

[0050] Figure 5 The embodiments of the present disclosure provide Figure 1 Another implementation flow diagram of S102;

[0051] Figure 6 Another schematic diagram of a process flow of the autonomous driving simulation method provided by an embodiment of the present disclosure;

[0052] Figure 7 A flowchart of a model training method provided in an embodiment of the present disclosure;

[0053] Figure 8 Another schematic diagram of a flow chart of a model training method provided in an embodiment of the present disclosure;

[0054] Fig. 9 A schematic diagram of the composition of an autonomous driving simulation device provided in an embodiment of the present disclosure;

[0055] Fig.10 A schematic diagram of the composition of a model training device provided in an embodiment of the present disclosure;

[0056] Fig.11 Another schematic diagram of the components of the model training device provided in the embodiment of the present disclosure;

[0057] Fig.12 1 is a schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION

[0058] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0059] It should be understood that in the embodiments of the present disclosure, the character " / " generally indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", etc. are used only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features.

[0060] Autonomous driving simulation testing is the process of using computer simulation environments and virtual vehicles to test the performance, safety and stability of autonomous driving systems.

[0061] At present, when conducting autonomous driving simulation tests, various test cases can be designed based on road data, and then the designed test cases can be run in a simulation environment to test the autonomous driving system.

[0062] However, current autonomous driving simulation testing methods are not very accurate.

[0063] For example, in many autonomous driving simulation tests, road data playback (Logsim) is a commonly used method, which regenerates traffic flow by extracting vehicle trajectory information, map road network information, etc. from real traffic road data. However, Logsim has some significant disadvantages. First, because it relies on real-world traffic data, it has high requirements for the accuracy and completeness of obtaining these data. In addition, in some new road environments and extreme traffic conditions, it is difficult to obtain enough actual data to create a simulation environment, that is, it is difficult to fully cover the extreme conditions in the test scenario. Secondly, the Logsim method cannot simulate complex interactive behaviors well. Real-world traffic data is difficult to reflect various driving behaviors and relationships. In Logsim, background vehicles (non-player characters, NPCs) other than the main vehicle are played back according to road data. When testing the main vehicle equipped with an autonomous driving system, the NPC vehicle cannot generate interactive behaviors for the autonomous driving system to be tested, thereby generating unrealistic collision events and affecting the performance of the test autonomous driving system. Finally, information loss may occur during data processing, thereby affecting the accuracy of the simulation.

[0064] Against this background technology, the present disclosure provides an autonomous driving simulation method that can enable background vehicles to react to a main vehicle, thereby improving the accuracy of autonomous driving simulation tests and the precision of vehicle control in autonomous driving simulation scenarios.

[0065] Exemplarily, the autonomous driving simulation method and model training method provided by the present disclosure can be applied to the scenario of autonomous driving simulation.

[0066] For example, the execution subject of the autonomous driving simulation method and model training method provided in the embodiments of the present disclosure may be a computer or a server, or may be other devices with data processing capabilities. For example, the execution subject may be a background server of an autonomous driving vehicle, and the execution subject of the method is not limited here.

[0067] In some embodiments, the server may be a single server, or a server cluster composed of multiple servers. In some implementations, the server cluster may also be a distributed cluster. The present disclosure does not limit the specific implementation of the server.

[0068] Figure 1 The following is a flow chart of the automatic driving simulation method provided by the embodiment of the present disclosure. Figure 1 As shown, the method may include S101-S103.

[0069] S101, obtaining first driving information of a target simulated vehicle and second driving information of a background simulated vehicle in an automatic driving simulation environment.

[0070] Exemplarily, the first driving information may include the first historical trajectory and first driving data of the target simulated vehicle, and the first driving data may include the position, speed, acceleration, heading angle and other related parameters of the target simulated vehicle. The second driving information may include the second historical trajectory and second driving data of the background simulated vehicle, and the second driving data may include the position, speed, acceleration, heading angle and other related parameters of the background simulated vehicle. The target simulated vehicle may be understood as the main vehicle, and the background simulated vehicle may be understood as the NPC vehicle, and the first driving information of the target simulated vehicle and the second driving information of the background simulated vehicle may be obtained from the playback of the road acquisition data.

[0071] S102. Determine a first vehicle that affects the automatic driving simulation from background simulated vehicles based on the first driving information and the second driving information, and change the driving information of the first vehicle to obtain an updated automatic driving simulation environment.

[0072] S103. Perform autonomous driving simulation on the target simulation vehicle according to the updated autonomous driving simulation environment.

[0073] For example, after the first driving information and the second driving information are acquired, the first vehicle may be determined from the background simulation vehicles according to the first driving information and the second driving information, wherein the first vehicle is a vehicle that affects the automatic driving simulation.

[0074] For example, when the target simulated vehicle is vehicle A and the background simulated vehicles are vehicle B, vehicle C, vehicle D, and vehicle E, the driving information of vehicle A, vehicle B, vehicle C, vehicle D, and vehicle E can be obtained, and the vehicle that will have an impact on the autonomous driving simulation is determined from the background simulated vehicles B, vehicle C, vehicle D, and vehicle E according to the driving information of each vehicle. The vehicle that will have an impact on the autonomous driving simulation is determined from the background simulated vehicles B, vehicle C, vehicle D, and vehicle E as the first vehicle, and then the driving information of the first vehicle is changed to obtain an updated autonomous driving simulation environment. For example, when it is determined that vehicles B, vehicle C, and vehicle D in the background simulated vehicles will have an impact on the autonomous driving simulation according to the driving trajectory of each vehicle, vehicles B, vehicle C, and vehicle D in the background simulated vehicles are the first vehicles, and the driving information of vehicles B, vehicle C, and vehicle D can be changed to obtain an updated autonomous driving simulation environment, and the target simulated vehicle is simulated for autonomous driving according to the updated autonomous driving simulation environment.

[0075] In some possible embodiments, the first vehicle can also be determined based on the positions of the target simulated vehicle and the background simulated vehicle, and then the driving information of the first vehicle can be changed to obtain an updated autonomous driving simulation environment; based on the updated autonomous driving simulation environment, the target simulated vehicle can be simulated for autonomous driving.

[0076] The disclosed embodiment obtains the first driving information of the target simulated vehicle and the second driving information of the background simulated vehicle in the autonomous driving simulation environment; then, according to the first driving information and the second driving information, determines the first vehicle that affects the autonomous driving simulation from the background simulated vehicles, and changes the driving information of the first vehicle to obtain an updated autonomous driving simulation environment; finally, according to the updated autonomous driving simulation environment, performs autonomous driving simulation on the target simulated vehicle. The background simulated vehicle can generate reactive behavior against the main vehicle, thereby improving the accuracy of the autonomous driving simulation test and the precision of vehicle control in the autonomous driving simulation scenario, and resolving potential conflicts in the traffic simulation scenario.

[0077] Figure 2 The embodiments of the present disclosure provide Figure 1 A schematic diagram of an implementation flow of S102 is shown in FIG. Figure 2 As shown, Figure 1 The illustrated S102 may include S201 - S202 .

[0078] S201. Determine, from the background simulated vehicles, a second vehicle that affects the driving of the target simulated vehicle according to the first driving information and the second driving information, and change the second driving information corresponding to the second vehicle to obtain the changed driving information corresponding to the second vehicle.

[0079] Exemplarily, the second vehicle can be understood as a vehicle that will directly affect the target simulated vehicle, for example, the second vehicle will collide with the target simulated vehicle when it continues to travel at the current travel trajectory and travel speed. The second vehicle that affects the travel of the target simulated vehicle can be determined from the background simulated vehicles based on the first travel information of the target simulated vehicle and the second travel information of the background simulated vehicle. After the second vehicle is determined, the second travel information corresponding to the second vehicle can be changed to obtain the changed travel information corresponding to the second vehicle, thereby avoiding the second vehicle from colliding with the target simulated vehicle.

[0080] S202. Determine a third vehicle that affects the driving of the vehicle whose second driving information has been changed in the background simulation vehicles, and change the second driving information corresponding to the third vehicle until there are no more vehicles in the background simulation vehicles that affect the driving of other vehicles, thereby obtaining an updated automatic driving simulation environment.

[0081] Exemplarily, the vehicle whose second driving information has been changed includes the second vehicle, and the first vehicle includes the vehicle whose second driving information has been changed. The third vehicle can be understood as a vehicle that will indirectly affect the target simulation vehicle. When the driving information of the second vehicle is changed, the third vehicle that will affect the driving of the vehicle whose second driving information has been changed in the background simulation vehicle can be determined. Then, the second driving information corresponding to the third vehicle is changed until there is no longer a vehicle in the background simulation vehicle that affects the driving of other vehicles, and an updated automatic driving simulation environment is obtained.

[0082] For example, assuming that the target simulated vehicle is vehicle A, and the first vehicle includes vehicle B, vehicle C, vehicle D, and vehicle E, when it is determined based on the first driving information and the second driving information that vehicle B will have an impact on vehicle A, vehicle B is the second vehicle, and the second driving information corresponding to vehicle B can be changed to obtain the changed driving information corresponding to vehicle B, thereby avoiding the impact of vehicle B on vehicle A. After the second driving information corresponding to vehicle B is changed, vehicle B can travel with the changed driving information. At this time, vehicle C and vehicle D will have an impact on vehicle B traveling with the changed driving information, and vehicle C and vehicle D are the third vehicle. The second driving information corresponding to vehicle C and vehicle D can be changed until there are no more vehicles in the background simulated vehicles that affect the driving of other vehicles, so that there is no potential conflict between the target simulated vehicle and the background simulated vehicles, that is, no collision will occur between the target simulated vehicle and the background simulated vehicles.

[0083] This embodiment determines the second vehicle that affects the driving of the target simulated vehicle from the background simulated vehicles based on the first driving information and the second driving information, and changes the second driving information corresponding to the second vehicle to obtain the changed driving information corresponding to the second vehicle; then determines the third vehicle that affects the driving of the vehicle that has changed the second driving information in the background simulated vehicles, and changes the second driving information corresponding to the third vehicle, until there are no more vehicles in the background simulated vehicles that affect the driving of other vehicles, and obtains an updated autonomous driving simulation environment. The precision of vehicle control in the autonomous driving simulation scenario can be further improved, and the process of iteratively controlling the autonomous driving simulation vehicle can be used to resolve all potential conflicts in the autonomous driving simulation scenario.

[0084] Figure 3 The embodiments of the present disclosure provide Figure 2 A schematic diagram of an implementation process of S201 is shown in FIG. Figure 3 As shown, Figure 2 The illustrated S201 may include S301 - S303 .

[0085] S301: Predict a first driving trajectory according to a first historical trajectory and first driving data.

[0086] Exemplarily, the first driving data may include the position, speed, acceleration, heading angle and other related parameters of the target simulated vehicle. The first historical trajectory of the target simulated vehicle may be obtained from the log data, and then the first driving trajectory of the target simulated vehicle may be predicted based on the first historical trajectory of the target simulated vehicle and the position, speed, acceleration, heading angle and other related parameters of the target simulated vehicle. It may also be understood that the future driving trajectory of the target simulated vehicle may be predicted based on the first historical trajectory of the target simulated vehicle and the position, speed, acceleration, heading angle and other related parameters of the target simulated vehicle.

[0087] For example, at time t, the driving trajectory of the target simulated vehicle from time t to time t+5 can be predicted based on the historical trajectory of the target simulated vehicle from time t-5 to time t and related parameters such as the position, speed, acceleration, and heading angle of the target simulated vehicle.

[0088] S302: Predict a second driving trajectory according to the second historical trajectory and the second driving data.

[0089] Exemplarily, the second driving data may include the position, speed, acceleration, heading angle and other related parameters of the background simulation vehicle. The second historical trajectory of the background simulation vehicle may be obtained from the log data, and then the second driving trajectory of the background simulation vehicle may be predicted based on the second historical trajectory of the background simulation vehicle and the position, speed, acceleration, heading angle and other related parameters of the background simulation vehicle. It may also be understood that the future driving trajectory of the background simulation vehicle may be predicted based on the second historical trajectory of the background simulation vehicle and the position, speed, acceleration, heading angle and other related parameters of the background simulation vehicle.

[0090] For example, at time t, the driving trajectory of the background simulated vehicle from time t to time t+5 can be predicted based on the historical trajectory of the background simulated vehicle from time t-5 to time t and related parameters such as the position, speed, acceleration, and heading angle of the background simulated vehicle.

[0091] S303: Determine a second vehicle from background simulation vehicles according to the first driving trajectory and the second driving trajectory.

[0092] For example, after predicting the first driving track of the target simulated vehicle and the second driving track of the background simulated vehicle, the second vehicle can be determined from the background simulated vehicles according to the first driving track and the second driving track. For example, a vehicle in the background simulated vehicle whose driving track intersects with the first driving track is determined as the second vehicle.

[0093] This embodiment predicts a first driving trajectory based on the first historical trajectory and the first driving data; predicts a second driving trajectory based on the second historical trajectory and the second driving data; and then determines a second vehicle from background simulation vehicles based on the first driving trajectory and the second driving trajectory. The second vehicle can be determined based on the driving trajectory in the driving information, further improving the accuracy of controlling the second vehicle.

[0094] In some embodiments, the first historical trajectory and the first driving data may be used as inputs, and the first driving trajectory may be output through a prediction model.

[0095] Exemplarily, the prediction model may include an encoder, a dynamic encoder, an interactive relationship module, and a decoder, and the prediction model has the function of predicting the vehicle's driving trajectory based on the input vehicle's historical trajectory and driving data. The historical trajectory, driving data, and driving trajectory of a certain vehicle may be obtained, and then the historical trajectory and driving data of the vehicle are used as input, and the driving trajectory of the vehicle is used as output to train the neural network to obtain a prediction model, and the trained prediction model may include an encoder, a dynamic encoder, an interactive relationship module, and a decoder.

[0096] This embodiment uses the first historical trajectory and the first driving data as input and outputs the first driving trajectory through a prediction model, which can improve the accuracy and efficiency of the predicted trajectory, thereby improving the accuracy of conflict detection.

[0097] Figure 4 The embodiments of the present disclosure provide Figure 1 Another implementation flow diagram of S102 is shown in FIG. Figure 4 As shown, Figure 1 The illustrated S102 may include S401 - S402 .

[0098] S401: Determine a target point where a bounding box of a first vehicle overlaps with a bounding box of a target simulated vehicle.

[0099] S402: Determine a target path for the first vehicle to reach a target point, and change driving information of the first vehicle according to the target path to obtain an updated autonomous driving simulation environment.

[0100] Exemplarily, the bounding box of the first vehicle and the bounding box of the target simulated vehicle can be determined, and then the target point where the bounding box of the first vehicle overlaps with the bounding box of the target simulated vehicle can be determined based on the driving information of the first vehicle and the driving information of the target simulated vehicle. Then, a graph search algorithm is used to select a path to the target point, and a Bezier curve is used to roughly fit a smooth trajectory. This smooth trajectory is used as the target path. According to the target path, the driving information of the first vehicle is changed to obtain an updated autonomous driving simulation environment. At the same time, more sophisticated kinematic constraints are implemented when controlling the first vehicle, including lateral and longitudinal control. The conflict can be resolved by changing the speed of the first vehicle and imposing additional constraints on the time required for the first vehicle to reach the target point, so that the first vehicle arrives at the expected target later or earlier than in the log playback.

[0101] In this embodiment, a target point where the bounding box of the first vehicle overlaps with the bounding box of the target simulation vehicle is determined; then a target path for the first vehicle to reach the target point is determined, and the driving information of the first vehicle is changed according to the target path to obtain an updated autonomous driving simulation environment. The driving information of the first vehicle can be changed according to the target path to obtain an updated autonomous driving simulation environment, thereby ensuring that potential conflicts in the traffic simulation scenario are resolved.

[0102] Figure 5 The embodiments of the present disclosure provide Figure 1 Another implementation flow diagram of S102 is shown in FIG. Figure 5 As shown, Figure 1 The illustrated S102 may include S501 - S502 .

[0103] S501. Acquire driving data of a first vehicle in a first time period.

[0104] S502. Input the driving data of the first vehicle in the first time period into the strategy model, and output the driving data of the first vehicle in the second time period through the strategy model to change the driving information of the first vehicle and obtain an updated autonomous driving simulation environment.

[0105] Exemplarily, the first time period is the previous time period of the second time period, and the policy model has the function of outputting the driving data of the vehicle in the second time period according to the input driving data of the vehicle in the first time period. The driving data of the first vehicle in the k time period can be used as input, and the driving data of the first vehicle in the k+1 time period can be used as output to train the neural network to obtain the policy model. Then, the driving data of the first vehicle in the first time period is obtained, and the driving data of the first vehicle in the first time period is input into the policy model. The driving data of the first vehicle in the second time period is output through the policy model to change the driving information of the first vehicle and obtain an updated automatic driving simulation environment.

[0106] This embodiment obtains the driving data of the first vehicle in the first period; then inputs the driving data of the first vehicle in the first period into the strategy model, and outputs the driving data of the first vehicle in the second period through the strategy model to change the driving information of the first vehicle, thereby obtaining an updated automatic driving simulation environment. The driving information of the vehicle can be changed through the strategy model to achieve control of the vehicle, thereby further improving the accuracy and efficiency of vehicle control.

[0107] In some embodiments, a first vehicle that affects the autonomous driving simulation can be determined from background simulated vehicles based on the first driving information and the second driving information, and the driving information of the first vehicle can be changed within a preset time period to obtain an updated autonomous driving simulation environment.

[0108] For example, a time period may be preset according to actual scene requirements, and then the driving information of the first vehicle may be changed within the preset time period. For example, an 8-second time period may be preset, and the driving information of the first vehicle may be changed within 8 seconds. Alternatively, after determining the target point where the bounding box of the first vehicle overlaps with the bounding box of the target simulation vehicle, after controlling the first vehicle to reach the target point, the control strategy of the first vehicle may be transitioned to Logsim, and the remaining playback may be continued.

[0109] This embodiment can reduce resource consumption and improve the efficiency of vehicle control by determining the first vehicle that affects the automatic driving simulation from the background simulated vehicles based on the first driving information and the second driving information, and changing the driving information of the first vehicle within a preset time period.

[0110] Figure 6 Another flowchart of the automatic driving simulation method provided by the embodiment of the present disclosure is shown in FIG. Figure 6As shown, the method may include S601-S605.

[0111] S601: Input a first historical trajectory and first driving data into a dynamic encoder, and output a target tensor through the dynamic encoder.

[0112] Exemplarily, the prediction model obtained by the above training may include an encoder, a dynamic encoder, an interaction module, and a decoder. The first historical trajectory and the first driving data of the target simulated vehicle may be obtained, the first historical trajectory and the first driving data may be input into the dynamic encoder, and the target tensor may be output through the dynamic encoder.

[0113] For example, at each time step t, the observation value of the target simulated vehicle at this moment can be obtained, and the observation value can be a four-dimensional tensor, which includes the historical trajectory, position, speed, heading angle and other related parameters of the target simulated vehicle. Among them, x and y can be used to represent the horizontal and vertical coordinates respectively, and the coordinate value has completed the coordinate conversion of the historical coordinates with time t as the origin of the coordinate; v is used to represent the longitudinal speed of the target simulated vehicle at time t, and yaw is used to represent the heading angle of the target simulated vehicle at time t. In a specific training process and experiment, v is the longitudinal speed of the target simulated vehicle along the lane line, and yaw is the observation value obtained by rotating counterclockwise with the positive direction of the y-axis as the starting point, ranging from [0, 360], in degrees. The dynamic encoder can be a common feedforward neural network, whose input layer and output layer are simple linear layers, and the activation function (ReLU) can be used as the hidden layer of the feedforward neural network. The target simulated vehicle can convert its state (such as position, speed, heading angle) into a high-dimensional tensor (such as a 128-dimensional tensor) through a dynamic encoder, and this high-dimensional tensor is the target tensor. Similarly, the tensor corresponding to each simulated vehicle can also be output through a dynamic encoder.

[0114] S602: Input the target tensor into the interaction relationship module, and output the interaction relationship matrix through the interaction relationship module.

[0115] For example, the output of the dynamic encoder calculated at time t can be used as the input of the interactive relationship module, that is, the target tensor can be used as the input of the interactive relationship module, which can reduce the computational cost. The state of the target simulated vehicle is projected into a high-dimensional space, and then the attention weight matrix of the target simulated vehicle relative to its neighboring vehicles is calculated through the attention mechanism. The input of the multi-head attention mechanism can be expressed as formula (1).

[0116] Q l =K l =V l =f Dynamic Enc (x, y, v, yaw) formula (1)

[0117] In formula (1), Q l Indicates Query, K l Indicates Key, V l represents Value, x represents the horizontal coordinate of the spatial position, y represents the vertical coordinate of the spatial position, v represents the speed, and yaw represents the heading angle. From formula (1), we can see that Q l Indicates Query, K l Indicates Key, V l Indicates that Value has the same origin, and both are tensors obtained by encoding the spatial position coordinates, speed, and heading angle of the target simulation vehicle at time t through the dynamic encoder. l , K l 、V l The parameter matrix of can be learned in each attention head.

[0118] The attention weight matrix can be calculated first. The attention weight matrix can be determined by formula (2). The attention weight matrix is ​​Q l , K l 、V l The dot product of .

[0119]

[0120] In formula (2), attention1 represents the attention weight matrix, Q l , K l 、V l For the attention mechanism l ), softmax represents data normalization, d k is the dimension of the hidden layer, divided by d k This is to prevent the input value from being too large, causing the partial derivative to approach 0.

[0121] After determining the attention weight matrix, the interaction relationship matrix can be determined according to the attention weight matrix according to formula (3).

[0122] M=Concat(attention 1 , attention 2 ,......attention l )W O Formula (3)

[0123] In formula (3), M is the interaction matrix, which can represent the complex interaction relationship between different subsets and indicates the information that each simulated vehicle should pay attention to. O is a learnable parameter matrix.

[0124] For example, the attention mechanism is used to capture long-range spatiotemporal dependencies. The attention module can explicitly model the interaction between the target simulated vehicle and the background simulated vehicles within a preset range, and select the surrounding background simulated vehicles to pay attention to when calculating the future trajectory of the target simulated vehicle. l The influence of the interaction between each simulated vehicle based on the relative state can be represented, which tells the predicted target simulated vehicle to pay attention to the surrounding vehicles and ignore the vehicles that will not affect its future state. A high-order interaction module based on multi-head attention can be used to preserve different types of spatiotemporal relationships. The use of the attention mechanism is inspired by the Transformer architecture. A learned attention feature is mainly focused on a subset of interconnected agents, which may represent an aspect of the spatiotemporal relationship that may occur near the predicted simulated vehicle. In order to extend the attention to higher-order interactions, l different Q l , K l 、V l , we can learn the interaction relationships at different levels and represent them with attention weight matrices. We connect the above attention weight matrices and dynamically weight them to extract the complex interaction relationships between different subsets and represent them with interaction relationship matrices.

[0125] S603: Perform data processing on the interaction relationship matrix to obtain action variables.

[0126] S604: Encode the first historical trajectory through an encoder to obtain a target feature vector.

[0127] Exemplarily, the interaction relationship matrix can be processed to obtain action variables, which can represent driving behaviors and modes, such as changing lanes to the left or right, and simulating conservative or aggressive modes of vehicle behavior. The encoder of the prediction model can encode the first historical trajectory using a recurrent neural network (RNN) to obtain a target feature vector.

[0128] For example, the coordinates of each simulated vehicle at each time frame can be represented as two-dimensional coordinates (x, y) or three-dimensional coordinates (x, y, z). Then, an RNN can be instantiated for each simulated vehicle. These RNNs share the same parameters between multiple simulated vehicles, which means that the algorithm can combine batches of inputs to effectively perform joint predictions, making the algorithm scalable to any number of simulated vehicles in range. RNN introduces a deterministic hidden unit h at each time step t t , as the encoded feature vector, h t This summarizes all observations before and after time step t. At each time step t, the RNN transforms the observation x t and the hidden layer h t-1 As input, and calculate the update h t =f rnn (x t ,h t-1 ), where frnn represents the RNN encoder.

[0129] S605: Input the target feature vector and the action variable into a decoder to obtain a first driving trajectory.

[0130] Exemplarily, the above target feature vector and action variable are input into the decoder, and the output It consists of a five-dimensional vector that controls the bivariate normal distribution: μ x , μ y , σ x , σ y and correlation coefficient ρ, predicted value y t From the RNN decoder y t =f dec (h t ) calculation, and output the probability distribution of the future trajectory of each target simulated vehicle in each mode, thereby obtaining the first driving trajectory of the target simulated vehicle.

[0131] In this embodiment, the first historical trajectory and the first driving data are input into a dynamic encoder, and the target tensor is output through the dynamic encoder; the target tensor is input into an interactive relationship module, and the interactive relationship module outputs an interactive relationship matrix, and then the interactive relationship matrix is ​​processed to obtain an action variable; the first historical trajectory is encoded by the encoder to obtain a target feature vector; finally, the target feature vector and the action variable are input into a decoder to obtain the first driving trajectory. The multi-modal prediction task of the simulated vehicle in the trajectory space can be completed through the functions of different modules of the prediction model, thereby further improving the accuracy of trajectory prediction.

[0132] Figure 7 The following is a flow chart of the model training method provided in the embodiment of the present disclosure. Figure 7As shown, the method may include S701-S702.

[0133] S701. Acquire driving data of a first target vehicle to be tested in a third time period and driving data of a first target vehicle to be tested in a fourth time period.

[0134] S702: Taking the driving data of the first target vehicle to be tested in the third time period as input and taking the driving data of the first target vehicle to be tested in the fourth time period as output, training the neural network to obtain a strategy model.

[0135] Exemplarily, a strategy model that can perceive other simulated vehicles can be trained, and the modeling of the strategy model is completed by using the imitation learning method. Then, the driving data of the first target vehicle to be tested in the third time period and the driving data in the fourth time period are obtained, and the driving data of the first target vehicle to be tested in the third time period is used as input and the driving data of the first target vehicle to be tested in the fourth time period is used as output, and the neural network is trained to obtain the strategy model. Among them, the third time period is the previous time period of the fourth time period. The driving data of the first target vehicle to be tested may include relevant parameters such as the speed, steering wheel angle, longitudinal acceleration and lateral acceleration of the first target vehicle to be tested.

[0136] For example, the control policy can be viewed as a sequential decision-making task, and the control policy can be trained from real data to achieve continuous dynamic control of the simulated vehicle. For example, highway driving is viewed as a sequential decision-making task, and the driver follows a random policy π(a|s) that maps the observed road conditions s to the distribution of driving behaviors a. Given a class of policies π θ , parameterized by θ, and seeks the strategy that best imitates human driving behavior. The imitation learning method is used to learn from a series of state-action pairs (s t , a t ) to infer human driving behavior strategies, where imitation learning can use behavior cloning or reinforcement learning. The policy model consists of a multilayer perceptron (MLP), which can map the observed state into a high-dimensional vector. The policy model can output the decision-making behavior of the simulated vehicle at the next time step, such as steering wheel angle, longitudinal acceleration, and lateral acceleration. During the training process of the policy model, imitation learning can be combined with reinforcement learning using reward enhancement. Reward enhancement helps to improve the interaction and state space exploration of the learning simulated vehicle, and the output of the policy model can be distinguished from the expert data through the evaluation module, providing a valuable reward signal for the policy model, thereby improving the performance of the policy model.

[0137] Optionally, different data sets can be used to train policy models with different driving styles, such as a conservative style data set, in which there are more vehicle following behaviors and no sudden acceleration or deceleration, etc.; an aggressive driving style data set, in which there are more vehicle lane changing behaviors and frequent sudden acceleration or deceleration, etc. By using different proportions of aggressive driving style data and conservative style data, policy models with diverse styles can be trained. After the interactive closed-loop test framework identifies potential future conflicts, the trained policy models with different styles are selected to control the NPC vehicles, so that the vehicles in the traffic flow scene are not of a single driving style, thereby ensuring the high fidelity of the traffic flow scene and the high likelihood of the policy model.

[0138] This embodiment obtains the driving data of the first target vehicle to be tested in the third period and the driving data in the fourth period; then the driving data of the first target vehicle to be tested in the third period is used as input, and the driving data of the first target vehicle to be tested in the fourth period is used as output to train the neural network to obtain a strategy model. The driving information of the simulated vehicle can be changed through the strategy model to obtain an updated automatic driving simulation environment, thereby further improving the accuracy and efficiency of vehicle control.

[0139] In some embodiments, the network parameters of the policy model may be updated by a trust region policy optimization algorithm.

[0140] Illustratively, during the training process, a trust region policy optimization algorithm may be used to update policy parameters until the evaluation module cannot distinguish between the expert data and the output of the policy model.

[0141] For example, policy gradient algorithms are a class of stochastic policy search algorithms that are particularly effective reinforcement learning techniques for optimizing differentiable policies, including neural networks. As with standard backpropagation, gradient-based updates are used to optimize network parameters, but the gradients can only be approximated by simulated deployments of the policy interacting with the environment. Such empirical gradient estimates often exhibit large variance. In practice, this variance can lead to parameter updates that do not improve or may even degrade performance. Human driving policies can be learned using trust region policy optimization (TRPO). TRPO updates policy parameters through a constrained optimization process that enforces that the policy cannot change too much in a single update, thus limiting the damage that can be caused by noisy gradient estimates. Although the true reward function that governs the behavior of any particular human driver is unknown, domain knowledge can be used to craft a simulated vehicle reward function. Drivers avoid collisions and veer off the road, but also prefer to drive smoothly and minimize lane deviations. If these features can be combined into a reward function that closely matches the true reward function for human driving, then driver behavior can be modeled.

[0142] This embodiment updates the network parameters of the policy model through the trust region policy optimization algorithm, which can improve the convergence of the policy model and further enhance the performance of the policy model.

[0143] Figure 8 Another flowchart of the model training method provided by the embodiment of the present disclosure; Figure 8 As shown, the method may include S801-S802.

[0144] S801. Obtain the historical track, driving data and driving track of the second target vehicle to be tested.

[0145] S802: Taking the historical trajectory and driving data of the second target vehicle to be tested as input and the driving trajectory of the second target vehicle to be tested as output, training a neural network to obtain a prediction model.

[0146] Exemplarily, the above-mentioned prediction model may include an encoder, a dynamic encoder, an interactive relationship module, and a decoder, and the prediction model has the function of predicting the vehicle's driving trajectory based on the input vehicle's historical trajectory and driving data. The historical trajectory, driving data, and driving trajectory of the second target vehicle to be tested can be obtained, and then the historical trajectory of the second target vehicle to be tested is normalized with time step t as the origin to achieve the coordinate transformation of the second target vehicle to be tested. The historical trajectory and driving data of the second target vehicle to be tested after coordinate normalization are used as input, and the driving trajectory of the second target vehicle to be tested is used as output, and the neural network is trained to obtain a prediction model.

[0147] For example, the historical trajectory and driving data of the second target vehicle to be tested in time period k can be used as input, and the driving trajectory of the second target vehicle to be tested in time period k+1 can be used as output to train the neural network and obtain a prediction model.

[0148] In this embodiment, the historical trajectory, driving data and driving trajectory of the second target vehicle to be tested are obtained; the historical trajectory and driving data of the second target vehicle to be tested are used as input, and the driving trajectory of the second target vehicle to be tested is used as output to train the neural network to obtain a prediction model. The prediction model can be used to predict the future driving trajectory of the simulated vehicle, thereby improving the accuracy and efficiency of predicting the future driving trajectory of the simulated vehicle.

[0149] In an exemplary embodiment, the present disclosure also provides an autonomous driving simulation device, which can be used to implement the autonomous driving simulation method as described in the aforementioned embodiment. Fig. 9 The following is a schematic diagram of the composition of the automatic driving simulation device provided in the embodiment of the present disclosure. Fig. 9 As shown, the device may include: an acquisition unit 901, a determination unit 902, and a simulation unit 903.

[0150] The acquisition unit 901 is used to acquire first driving information of a target simulated vehicle and second driving information of a background simulated vehicle in an automatic driving simulation environment.

[0151] The determination unit 902 is used to determine the first vehicle that affects the automatic driving simulation from the background simulation vehicles according to the first driving information and the second driving information, and change the driving information of the first vehicle to obtain an updated automatic driving simulation environment.

[0152] The simulation unit 903 is used to perform autonomous driving simulation on the target simulation vehicle according to the updated autonomous driving simulation environment.

[0153] Optionally, the determination unit 902 is specifically used to: determine, based on the first driving information and the second driving information, a second vehicle that affects the driving of the target simulated vehicle from the background simulated vehicles, and change the second driving information corresponding to the second vehicle to obtain the changed driving information corresponding to the second vehicle; determine a third vehicle that affects the driving of the vehicle that has changed the second driving information in the background simulated vehicles, and change the second driving information corresponding to the third vehicle until there are no more vehicles in the background simulated vehicles that affect the driving of other vehicles, and an updated autonomous driving simulation environment is obtained; wherein, the vehicle that has changed the second driving information includes the second vehicle, and the first vehicle includes the vehicle that has changed the second driving information.

[0154] Optionally, the first driving information includes a first historical trajectory and first driving data, and the second driving information includes a second historical trajectory and second driving data. The determination unit 902 is specifically used to: predict a first driving trajectory based on the first historical trajectory and the first driving data; predict a second driving trajectory based on the second historical trajectory and the second driving data; and determine a second vehicle from background simulated vehicles based on the first driving trajectory and the second driving trajectory.

[0155] Optionally, the determination unit 902 is specifically used to: determine a target point where the bounding box of the first vehicle overlaps with the bounding box of the target simulation vehicle; determine a target path for the first vehicle to reach the target point, and according to the target path, change the driving information of the first vehicle to obtain an updated autonomous driving simulation environment.

[0156] Optionally, determination unit 902 is specifically used to: obtain driving data of the first vehicle in a first time period; input the driving data of the first vehicle in the first time period into a strategy model, and output the driving data of the first vehicle in a second time period through the strategy model to change the driving information of the first vehicle, and obtain an updated autonomous driving simulation environment, wherein the first time period is the previous time period of the second time period, and the strategy model has the function of outputting the driving data of the vehicle in the second time period based on the input driving data of the vehicle in the first time period.

[0157] Optionally, the determination unit 902 is specifically used to: determine the first vehicle that affects the automatic driving simulation from the background simulation vehicles based on the first driving information and the second driving information, and change the driving information of the first vehicle within a preset time period to obtain an updated automatic driving simulation environment.

[0158] Optionally, the determination unit 902 is specifically used to: take the first historical trajectory and the first driving data as input, and output the first driving trajectory through a prediction model. The prediction model includes an encoder, a dynamic encoder, an interactive relationship module, and a decoder. The prediction model has the function of predicting the vehicle's driving trajectory based on the input vehicle's historical trajectory and driving data.

[0159] Optionally, the determination unit 902 is specifically used to: input the first historical trajectory and the first driving data into a dynamic encoder, and output a target tensor through the dynamic encoder; input the target tensor into an interaction relationship module, and output an interaction relationship matrix through the interaction relationship module; perform data processing on the interaction relationship matrix to obtain action variables; encode the first historical trajectory through the encoder to obtain a target feature vector; input the target feature vector and the action variable into a decoder to obtain a first driving trajectory.

[0160] In an exemplary embodiment, the present disclosure also provides a model training device, which can be used to implement the model training method as described in the above embodiment. Fig.10Schematic diagram of the composition of the model training device provided in the embodiment of the present disclosure. Fig.10 As shown, the device may include: an acquisition unit 1001 and a training unit 1002.

[0161] The acquisition unit 1001 is used to acquire the driving data of the first target vehicle to be tested in the third time period and the driving data in the fourth time period.

[0162] The training unit 1002 is used to take the driving data of the first target vehicle to be tested in the third time period as input, and take the driving data of the first target vehicle to be tested in the fourth time period as output, to train the neural network and obtain a strategy model. The third time period is the previous time period of the fourth time period, and the strategy model has the function of outputting the driving data of the vehicle in the fourth time period based on the input driving data of the vehicle in the third time period.

[0163] Alternatively, if Fig.10 As shown, the device may further include: an updating unit 1003.

[0164] The updating unit 1003 is used to update the network parameters of the policy model by using a trust region policy optimization algorithm.

[0165] Fig.11 Another schematic diagram of the model training device provided in the embodiment of the present disclosure. Fig.11 As shown, the device may include: an acquisition unit 1101 and a training unit 1102.

[0166] The acquisition unit 1101 is used to acquire the historical track, driving data and driving track of the second target vehicle to be tested.

[0167] The training unit 1102 is used to take the historical trajectory and driving data of the second target vehicle to be tested as input and the driving trajectory of the second target vehicle to be tested as output, to train the neural network and obtain a prediction model. The prediction model includes an encoder, a dynamic encoder, an interactive relationship module, and a decoder. The prediction model has the function of predicting the vehicle's driving trajectory based on the input vehicle's historical trajectory and driving data.

[0168] The beneficial effects of the above-mentioned autonomous driving simulation device and model training device can be referred to the beneficial effects of the autonomous driving simulation method and model training method described in the aforementioned embodiments, and will not be repeated here.

[0169] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device. The electronic device may be a server, computer or other device described in the aforementioned embodiment, and can be used to implement the autonomous driving simulation method and model training method provided in the embodiment of the present disclosure.

[0170] In an exemplary embodiment, the electronic device may include: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the autonomous driving simulation method and model training method as described in the above embodiments.

[0171] For example, Fig.12 1 is a schematic block diagram of an example electronic device 1200 that can be used to implement an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0172] like Fig.12 As shown, the electronic device 1200 may include a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit 1208 into a random access memory (RAM). In RAM 1203, various programs and data required for the operation of the electronic device 1200 can also be stored. The computing unit 1201, ROM 1202, and RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface is also connected to the bus 1204.

[0173] Multiple components in the electronic device 1200 are connected to the I / O interface 1205, including: an input unit 1206, such as a keyboard, a mouse, etc.; an output unit 1207, such as various types of displays, speakers, etc.; a storage unit 1208, such as a disk, an optical disk, etc.; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1209 allows the electronic device 1200 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0174] The computing unit 1201 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), any appropriate processors, controllers, microcontrollers, etc. The computing unit 1201 performs the various methods and processes described above, such as the autonomous driving simulation method and the model training method. For example, in some embodiments, the autonomous driving simulation method and the model training method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1208.

[0175] In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1200 via the ROM 1202 and / or the communication unit 1209. When the computer program is loaded into the RAM 1203 and executed by the computing unit 1201, one or more steps of the autonomous driving simulation method and the model training method described above may be performed.

[0176] Alternatively, in other embodiments, the computing unit 1201 may be configured to execute the autonomous driving simulation method and the model training method in any other suitable manner (e.g., by means of firmware).

[0177] According to an embodiment of the present disclosure, the present disclosure also provides a readable storage medium and a computer program product.

[0178] In an exemplary embodiment, the readable storage medium may be a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to the above embodiments.

[0179] In an exemplary embodiment, a computer program product includes a computer program, and when the computer program is executed by a processor, the method according to the above embodiments is implemented.

[0180] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0181] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0182] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0183] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0184] The systems and techniques described herein can be implemented in computing systems that include back-end components (e.g., as data servers), computing systems that include middleware components (e.g., application servers), computing systems that include front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), and computing systems that include any combination of such back-end components, middleware components, and front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area networks (LANs), wide area networks (WANs), and the Internet.

[0185] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0186] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0187] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An automatic driving simulation method, the method comprising: Acquire first driving information of a target simulated vehicle and second driving information of a background simulated vehicle in an autonomous driving simulation environment; Determining a first vehicle that affects the automatic driving simulation from the background simulation vehicles according to the first driving information and the second driving information, and changing the driving information of the first vehicle to obtain an updated automatic driving simulation environment; According to the updated autonomous driving simulation environment, autonomous driving simulation is performed on the target simulation vehicle.

2. The method according to claim 1, wherein the first vehicle that affects the automatic driving simulation is determined from the background simulation vehicles according to the first driving information and the second driving information, and the driving information of the first vehicle is changed to obtain an updated automatic driving simulation environment, comprising: Determine, from the background simulated vehicles, a second vehicle that affects the driving of the target simulated vehicle according to the first driving information and the second driving information, and change the second driving information corresponding to the second vehicle to obtain the changed driving information corresponding to the second vehicle; Determine a third vehicle that affects the driving of the vehicle whose second driving information is changed in the background simulation vehicles, and change the second driving information corresponding to the third vehicle until there is no vehicle in the background simulation vehicles that affects the driving of other vehicles, thereby obtaining an updated automatic driving simulation environment; The vehicle whose second driving information is changed includes the second vehicle, and the first vehicle includes the vehicle whose second driving information is changed.

3. The method according to claim 2, wherein the first driving information includes a first historical track and first driving data, and the second driving information includes a second historical track and second driving data, and the determining, from the background simulation vehicles, a second vehicle that affects the driving of the target simulation vehicle according to the first driving information and the second driving information, comprises: Predicting the first driving trajectory according to the first historical trajectory and the first driving data; Predicting the second driving trajectory according to the second historical trajectory and the second driving data; The second vehicle is determined from the background simulation vehicles according to the first driving trajectory and the second driving trajectory.

4. The method according to claim 1, wherein the first vehicle that affects the autonomous driving simulation is determined from the background simulation vehicles according to the first driving information and the second driving information, and the driving information of the first vehicle is changed to obtain an updated autonomous driving simulation environment, comprising: Determining a target point where a bounding box of the first vehicle overlaps a bounding box of the target simulated vehicle; Determine a target path for the first vehicle to reach the target point, change the driving information of the first vehicle according to the target path, and obtain the updated autonomous driving simulation environment.

5. The method according to claim 1, wherein the first vehicle that affects the automatic driving simulation is determined from the background simulation vehicles according to the first driving information and the second driving information, and the driving information of the first vehicle is changed to obtain an updated automatic driving simulation environment, comprising: Acquiring driving data of the first vehicle in a first time period; The driving data of the first vehicle in the first time period is input into the strategy model, and the driving data of the first vehicle in the second time period is output through the strategy model to change the driving information of the first vehicle and obtain an updated autonomous driving simulation environment. The first time period is the previous time period of the second time period, and the strategy model has the function of outputting the driving data of the vehicle in the second time period according to the input driving data of the vehicle in the first time period.

6. The method according to claim 1, wherein the first vehicle that affects the autonomous driving simulation is determined from the background simulation vehicles according to the first driving information and the second driving information, and the driving information of the first vehicle is changed to obtain an updated autonomous driving simulation environment, comprising: According to the first driving information and the second driving information, a first vehicle that affects the automatic driving simulation is determined from the background simulation vehicles, and the driving information of the first vehicle is changed within a preset time period to obtain an updated automatic driving simulation environment.

7. The method according to claim 3, predicting the first driving trajectory according to the first historical trajectory and the first driving data, comprises: The first historical trajectory and the first driving data are taken as input, and the first driving trajectory is output through a prediction model. The prediction model includes an encoder, a dynamic encoder, an interactive relationship module, and a decoder. The prediction model has a function of predicting the vehicle's driving trajectory based on the input vehicle's historical trajectory and driving data.

8. The method according to claim 7, wherein the step of taking the first historical trajectory and the first driving data as input and outputting the first driving trajectory through a prediction model comprises: Inputting the first historical trajectory and the first driving data into the dynamic encoder, and outputting a target tensor through the dynamic encoder; Input the target tensor into the interaction relationship module, and output the interaction relationship matrix through the interaction relationship module; Performing data processing on the interaction relationship matrix to obtain action variables; Encoding the first historical trajectory by the encoder to obtain a target feature vector; The target feature vector and the action variable are input into the decoder to obtain the first driving trajectory.

9. A model training method, the method comprising: Acquire the driving data of the first target vehicle to be tested in the third time period and the driving data of the first target vehicle to be tested in the fourth time period; The driving data of the first target vehicle to be tested in the third time period is used as input, and the driving data of the first target vehicle to be tested in the fourth time period is used as output, and the neural network is trained to obtain a strategy model. The third time period is the previous time period of the fourth time period, and the strategy model has the function of outputting the driving data of the vehicle in the fourth time period based on the input driving data of the vehicle in the third time period.

10. A model training method, the method comprising: Obtaining the historical trajectory, driving data and driving trajectory of the second target vehicle to be tested; The historical trajectory and driving data of the second target vehicle to be tested are used as input, and the driving trajectory of the second target vehicle to be tested is used as output, and a neural network is trained to obtain a prediction model. The prediction model includes an encoder, a dynamic encoder, an interactive relationship module, and a decoder. The prediction model has the function of predicting the vehicle's driving trajectory based on the input vehicle's historical trajectory and driving data.

Images