Driving scene simulation method, model and related device

Abstract

The invention discloses a driving scene simulation method, a model and a related device. The driving scene simulation method comprises the following steps: adding at least one destination different from a current position to a specified vehicle in a scene simulation model, generating a first prediction trajectory of the specified vehicle from the current position to the at least one destination in sequence, and indicating the specified vehicle to move according to the first prediction trajectory. Through the method, the destination can be added to the specified vehicle in the scene simulation model, so that the specified vehicle in the scene can move according to the predicted trajectory of the destination, landing of a more targeted simulation scene can be realized, and testing of a driving system in a complex driving scene is facilitated.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a driving scene simulation method, model and related devices. Background Art

[0002] The simulation of autonomous driving mainly includes the main vehicle simulation part and the driving scene simulation part. The main vehicle simulation is used to simulate the driving behavior of the main vehicle equipped with the autonomous driving system, while the driving scene simulation part is used to simulate the road traffic conditions faced by the autonomous driving vehicle and the behaviors of other vehicles, pedestrians and obstacles in the scene, so as to test the effectiveness, flexibility and reliability of the autonomous driving system in different driving scenarios.

[0003] Existing driving scene simulation methods can make the corresponding simulated vehicles and pedestrians move according to preset rules, so that the vehicles and pedestrians in the scene can drive in compliance with traffic rules in the traffic flow. However, the vehicles in these simulation scenes are mainly generated or controlled based on preset rules or fixed state parameters, such as by setting a specific range of activities and driving preferences, setting the state change of the vehicle through a fixed algorithm simulated motion curve fitted by sensor data and perception algorithms, and setting the simulation perception of a specific vehicle motion posture through technologies such as 3D modeling and animation. However, the motion state of the vehicle in the driving scene generated by the existing driving scene simulation method is mechanical and rigid, and there may be sudden jumps in the motion state, lacking the dynamic realism similar to that of human driving a vehicle.

[0004] At the same time, in the existing driving scene simulation methods, the behavior of simulated vehicles and pedestrians cannot be controlled purposefully, and it is impossible to specify vehicles or pedestrians to reach a specified destination or make them perform specific traffic behaviors. The simulated vehicles and pedestrians can only achieve simple random movements without clear goal orientation. It is difficult to create targeted scenes and cannot be applied to the implementation of specific simulation scenarios such as merging and exiting intersections, overtaking, and emergency stops. Summary of the invention

[0005] The present application mainly provides a driving scene simulation method, model and related devices, aiming to solve the problem that vehicle motion in existing simulated driving scenes lacks target orientation.

[0006] In order to solve the above technical problems, the technical solution adopted by the present application is: to provide a driving scene simulation method. The driving scene simulation method includes: adding at least one destination different from the current position to a designated vehicle in a scene simulation model; generating a first predicted trajectory for the designated vehicle to sequentially reach the at least one destination from the current position; and instructing the designated vehicle to move according to the first predicted trajectory.

[0007] In some embodiments, before adding at least one destination different from the current location to the designated vehicle in the scenario simulation model, it also includes: training the scenario simulation model so that the scenario simulation model has the ability to generate the first predicted trajectory and instruct the designated vehicle to move along the first predicted trajectory.

[0008] In some embodiments, the training of the scene simulation model includes: adding true main vehicle data and a loss function to the scene simulation model, wherein the true main vehicle data includes multiple frames of true trajectory points; sequentially selecting a training starting point and at least one training target point from the multiple frames of true trajectory points; generating a second predicted trajectory from the training starting point to the at least one training target point in sequence; wherein the second predicted trajectory includes multiple predicted trajectory points; calculating the loss of the predicted trajectory points based on the loss function; and adjusting the performance parameters of the scene simulation model based on the loss.

[0009] In some embodiments, the loss includes a destination loss; the calculation of the loss of the predicted trajectory point based on the loss function includes: taking the point among the predicted trajectory points that is closest to each of the training target points as the predicted target point corresponding to the training target point; calculating the cross entropy between the point set of the predicted target points and the point set of the corresponding training target points; calculating the average distance between each of the predicted target points and the corresponding training target point; and taking the sum of the cross entropy and the distance average as the destination loss.

[0010] In some embodiments, the loss includes kinematic loss; the true value trajectory point and the predicted trajectory point include corresponding data of multiple kinematic indicators; the loss of the predicted trajectory point calculated based on the loss function includes: calculating the difference between the data of each kinematic indicator corresponding to the predicted trajectory point and the corresponding true value trajectory point; and taking the average of each difference as the kinematic loss.

[0011] In some embodiments, the kinematic metric includes at least one of front-to-back frame velocity, acceleration, acceleration derivative, steering angle, and steering angle derivative.

[0012] In some embodiments, the expected arrival time between the current location and the first destination in the sequence of destinations, and the arrival time between the previous and next destinations in the sequence of destinations are both lower than a preset time threshold.

[0013] In order to solve the above technical problems, another technical solution adopted in the present application is: to provide a driving scenario simulation model, which includes: an input end, used to obtain the current position of a specified vehicle and receive at least one destination different from the current position; a processing end, connected to the input end, used to generate a first predicted trajectory for the specified vehicle to arrive at the at least one destination in sequence from the current position, and instruct the specified vehicle to move according to the first predicted trajectory.

[0014] In order to solve the above technical problems, another technical solution adopted in the present application is: providing a storage medium, on which program data is stored, characterized in that when the program data is executed by a processor, the steps of the driving scene simulation method as described above are implemented.

[0015] In order to solve the above technical problems, another technical solution adopted in the present application is: to provide a computer device, which includes a processor and a memory connected to each other, the memory stores a computer program, and when the processor executes the computer program, the steps of the driving scene simulation method as mentioned above are implemented.

[0016] The beneficial effects of the present application are as follows: Different from the prior art, the present application discloses a driving scene simulation method, model and related device. By adding a destination to a designated vehicle in the scene simulation model, the present application enables the designated vehicle in the scene to move according to its predicted trajectory to the destination, so that the behavior of the vehicle in the simulation scene has a clear goal orientation, and can simulate a specific driving scene by specifying a destination, and can more accurately simulate various driving scenes that an autonomous driving vehicle may face in road traffic, realize more targeted testing of the performance of the autonomous driving system, realize the implementation of more targeted simulation scenes, and improve the flexibility, effectiveness and practicality of driving scene simulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0018] Figure 1 It is a flowchart of an embodiment of a driving scene simulation method provided by the present application;

[0019] Figure 2 yes Figure 1 A schematic flow chart of an embodiment of step 10 in the embodiment;

[0020] Figure 3 yes Figure 2 A schematic flow chart of an embodiment of step 14 in the embodiment;

[0021] Figure 4 yes Figure 2 A flow chart of another embodiment of step 14 in the embodiment;

[0022] Figure 5 It is a structural schematic diagram of an embodiment of a driving scene simulation model provided by the present application;

[0023] Figure 6 It is a structural schematic diagram of an embodiment of a storage medium provided by the present application;

[0024] Figure 7 It is a structural diagram of an embodiment of a computer device provided by the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The terms "first", "second", "third" in the embodiments of the present application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the feature defined as "first", "second", "third" can expressly or implicitly include at least one of the features. In the description of the present application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically limited. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0027] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0028] This application provides a driving scene simulation method, see Figure 1 , Figure 1 1 is a flow chart of an embodiment of a driving scene simulation method provided by the present application, and the driving scene simulation method comprises:

[0029] Step 20: Add at least one destination different from the current location to the designated vehicle in the scenario simulation model.

[0030] The scenario simulation model is a model used to simulate the driving scenario of the vehicle. The simulation of the required driving scenario can be achieved by inputting the corresponding map features and the state features of the elements in the corresponding scene into the scenario simulation model. The simulated driving scenario usually includes the map of the scene and the elements such as vehicles, pedestrians, roads, obstacles and traffic signs in the scene. Each element is supported by corresponding state data, such as position, speed, direction and size, to clarify the relative position, relative speed and passability of each element, so as to simulate the scenario faced by the vehicle driving. The output of the scenario simulation model can be visualized directly or indirectly, so that the corresponding simulated scene can be displayed through the display device, and the data features of the elements in the scene during the movement process can be fed back to help the relevant scenario makers or testers intuitively understand the specific situation of the simulated driving scene.

[0031] Preferably, the scene simulation model uses Smart Agent technology to control the elements in the scene. Through Smart Agent technology, each element in the map can realize autonomous understanding and planning decisions, so that the vehicles and pedestrians therein can follow the traffic flow, comply with traffic rules to move, and perform conventional obstacle avoidance and random lane flow selection to simulate the driving decisions and state changes of non-main vehicles during driving. Similarly, pedestrians and obstacles can also realize their autonomous control through Smart Agent technology, so as to simulate a more dynamic driving scene faced by the automatic driving system vehicle. The behavior of the vehicles and pedestrians controlled by the Smart Agent technology is unpredictable for the automatic driving system, which is consistent with the fact that the automatic driving vehicle or the manually driven vehicle in actual traffic cannot predict the dynamic changes and specific destinations of other vehicles or pedestrians. The application of the Smart Agent technology in the scene simulation model can make the vehicles and pedestrians in the model autonomous, without the need for additional program control, and can effectively test the adaptability and processing efficiency of the automatic driving system, which is more in line with the traffic flow in the actual road scene.

[0032] However, the existing scenario simulation model can only make the elements in the scene follow the traffic flow and make random decisions. It is impossible to set the model with purpose-oriented settings, and it is difficult to create targeted scenarios, so it cannot be applied to the implementation of simulation scenarios. For example, in the test of merging or other vehicles inserting in front of the main vehicle in a large traffic flow, the current scenario simulation model, even with the combination of Smart Agent technology, cannot control the traffic flow in which the main vehicle is located to merge or insert in front of the main vehicle as expected, and cannot force other vehicles to change lanes and insert and other purposeful driving operations, and cannot meet the needs of complex test scenarios.

[0033] To meet the needs of these targeted scenarios, the present application adds at least one destination different from the current position to the specified vehicle in the scenario simulation model. These destinations are different from the current position of the vehicle and represent the target points that the vehicle needs to reach. The setting of the destination can be set according to the specific driving task or test task. For example, in the process of changing lanes, the destination can be set to the road that needs to change lanes or a point on the road. When continuous lane changes are required, the destination can be set to the target lane or a point in the target lane, or the destination can be set on multiple lanes including the target lane, and the destination set in the target lane can be reached through each destination in sequence.

[0034] There are also many ways to add a destination to a designated vehicle. For example, you can manually send a control command to determine the designated vehicle and specify its destination, or after selecting the corresponding driving task, the scenario simulation model automatically selects the designated vehicle and automatically specifies the destination of the vehicle, so as to achieve faster selection of designated vehicles and destinations. This process can be implemented on a graphical user interface, by calling a program interface, or by relying on the internal mechanism of the model.

[0035] Optionally, after adding at least one destination different from the current position to the designated vehicle in the scenario simulation model, the method further includes: feature encoding the added destination to obtain destination features.

[0036] Destinations are added manually or automatically, and the added destinations may be in different formats, such as natural language data, structured text data, or encoded data. Some data cannot be understood by the model, so it is necessary to feature encode the added destinations to ensure that these different information formats can be converted into the same data format that the model can understand. This feature encoding process involves converting the information of different forms of destinations into numerical or vector form, which can usually be achieved through a specific destination encoder combined with natural language processing, geocoding, or specific encoding rules.

[0037] The destination feature at least includes location information, which can be coordinates in a specific world coordinate system, or map coordinates determined by map features input into the model. In addition, the destination feature can optionally include feature information such as location information, type information, priority information, path constraint information, and destination arrival speed. After obtaining the destination feature, the scenario simulation model can use this information to simulate the driving behavior of the specified vehicle more accurately, so that the simulation result is closer to the actual driving situation.

[0038] Step 30: Generate a first predicted trajectory for the designated vehicle to sequentially reach at least one destination from the current position.

[0039] After adding a destination to the designated vehicle, the corresponding possible trajectory can be predicted based on the current position of the designated vehicle and the destination. In the process of generating the preset trajectory, it is necessary to determine it in combination with the state characteristics of the vehicle at the current position, such as speed, acceleration and steering angle. The generation of the first predicted trajectory can be achieved by a specific path planning algorithm or trajectory generation algorithm to ensure that the generated first predicted trajectory complies with traffic regulations and can meet the driving behavior characteristics of the vehicle to reach the set destination. These algorithms can be independent of the model setting, but are preferably integrated into the model, and the corresponding first predicted trajectory can be generated by the model. The model is trained based on specific data so that it can generate the corresponding first predicted trajectory after determining the designated vehicle and receiving the added destination, and can directly control the corresponding vehicle to move according to the first predicted trajectory in the future.

[0040] The first predicted trajectory combines the state characteristics of the specified vehicle, and when the corresponding algorithm or model can reasonably perform trajectory prediction, it can usually make the specified vehicle drive to the destination in accordance with the control logic of the state characteristics of the real driving vehicle. Therefore, in order to make the corresponding algorithm or model more consistent with the driving logic and kinematic characteristics, and to reach the corresponding destination accurately and effectively, the algorithm or model can be trained and optimized in advance based on real driving data to achieve more accurate, reasonable and effective trajectory prediction.

[0041] Step 40: Instruct the designated vehicle to move according to the first predicted trajectory.

[0042] After the corresponding first predicted trajectory is generated, the corresponding designated vehicle can be instructed to move according to the first predicted trajectory. Since the first predicted trajectory is predicted based on the state characteristics of the designated vehicle at its current position, it is usually achievable, so that the designated vehicle follows the corresponding first predicted trajectory to reach the destination by adjusting the speed, acceleration, steering angle and other states. Other non-designated vehicles will not be affected by the change in the state of the designated vehicle because they can follow the traffic flow, comply with traffic rules and avoid conventional obstacles, thus ensuring the stability and effectiveness of the driving scene.

[0043] Since the designated vehicle is in the scene simulated by the scene simulation model, based on data support, by processing the corresponding data in the model, the corresponding designated vehicle can be forced to move according to the first predicted trajectory, so that the driving behavior of the designated vehicle can be traced and can meet the needs of targeted scenes, thereby facilitating the implementation of more simulation scenes. Different from the existing scene simulation methods where vehicles and pedestrians lack goal orientation, the method of this application can effectively solve the problem that the existing scene simulation methods cannot force other vehicles to perform purposeful driving operations such as changing lanes and interweaving, and cannot meet the needs of complex test scenes.

[0044] Through the above steps, by adding a destination to the designated vehicle in the scene simulation model, the designated vehicle in the scene can move according to its predicted trajectory to the destination, so that the behavior of the simulated vehicle has a clear goal orientation, and more targeted driving scenarios can be simulated by specifying the destination. It can more accurately simulate various driving scenarios that autonomous driving vehicles may face in road traffic, realize more targeted testing of the performance of the autonomous driving system, and realize the implementation of more targeted simulation scenarios, thereby improving the flexibility, effectiveness and practicality of driving scene simulation.

[0045] Optionally, the expected arrival time between the current location and the first destination in the sequence of destinations, and the arrival time between the previous and next destinations in the sequence of destinations are both lower than a preset time threshold.

[0046] Since the initial trajectory is supported by the status data of the initial position, i.e. the current position, in the process of trajectory prediction, it usually has a strong accuracy. In the subsequent process, since the predicted data is generated based on the previously predicted data, the longer the prediction time is, the easier it is to produce deviations, and the long prediction process will also generate a large amount of data, which requires higher model performance. Therefore, by setting a preset time threshold, the current position and each destination can be reached in sequence within the preset time threshold, thereby reducing deviations and lowering the requirements for model performance. The preset time can be determined based on factors such as model performance, actual driving experience, traffic rules and road conditions. The preset time threshold is usually set to a few seconds, such as 3 seconds, 5 seconds, 8 seconds or 10 seconds.

[0047] Optionally, before adding at least one destination different from the current location to the designated vehicle in the scenario simulation model, the method further includes:

[0048] Step 10: Train the scenario simulation model so that the scenario simulation model has the ability to generate a first predicted trajectory and instruct the designated vehicle to move along the first predicted trajectory.

[0049] After adding a destination to the designated vehicle in the scenario simulation model, if the scenario simulation model can be used to execute the subsequent tasks of generating a first predicted trajectory for the designated vehicle to arrive at at least one destination in sequence from the current position, and instructing the designated vehicle to move along the first predicted trajectory, the efficiency of the scenario simulation can be further improved, and the corresponding scenario simulation can be easier to deploy and implement.

[0050] In order to enable the corresponding scenario simulation model to have the ability to generate a first predicted trajectory and instruct the specified vehicle to move along the first predicted trajectory, this can be achieved by training the scenario simulation model. There are many methods for training the scenario simulation model. For example, the trajectory data predicted multiple times can be analyzed and compared with the trajectory data of real driving, and the corresponding parameters and hyperparameters set in the model can be optimized to make the data predicted by the model gradually conform to the real driving trajectory. For another example, a large amount of driving data can be used to train the model so that it can learn the best trajectory from the current position to different destinations, and can adjust the trajectory according to the real-time status characteristics of the vehicle.

[0051] During the model training process, different characteristics of its trajectory prediction can also be optimized. For example, the accuracy of the model trajectory prediction can be optimized based on the degree of fit between the predicted trajectory and the actual trajectory, the authenticity of the model trajectory prediction can be optimized based on the speed, acceleration, steering angle and other motion state data corresponding to the predicted trajectory and the actual trajectory, and the rationality of the model trajectory prediction can be optimized based on the specific road conditions faced by the predicted trajectory and the actual trajectory. In addition, during the model training process, different driving scenarios, traffic rules and road conditions can be set to enable the model to adapt to various complex driving environments and improve the authenticity of the scene simulation.

[0052] Optionally, see Figure 2 , to train the scene simulation model, you can follow the following steps:

[0053] Step 11: Add true host vehicle data and loss function to the scene simulation model, wherein the true host vehicle data includes multiple frames of true trajectory points.

[0054] Step 12: Select a training starting point and at least one training target point in sequence from the multi-frame true value trajectory points.

[0055] Step 13: Generate a second predicted trajectory from the training starting point to at least one training target point in sequence; wherein the second predicted trajectory includes a plurality of predicted trajectory points.

[0056] Step 14: Calculate the loss of the predicted trajectory points based on the loss function.

[0057] Step 15: Adjust the performance parameters of the scenario simulation model based on the losses.

[0058] The true value main vehicle data is a series of data actually generated during human driving. These data usually exist in the form of multiple frames of true value trajectory points. The true value trajectory points of each frame can contain the vehicle's position, speed, acceleration, steering angle and other state information at a specific time point. These data are more consistent with the kinematic characteristics and trajectory characteristics of human driving behavior. Therefore, by applying the true value main vehicle data to the training of the scene simulation model, the movement of the corresponding designated vehicle can be more consistent with the behavioral characteristics of human driving and more consistent with the situations that the autonomous driving system may face when applied to real vehicles, thereby achieving more effective and reliable driving scene simulation. The loss function is an indicator used to measure the difference between the model prediction results and the real data. The loss function is usually a function of comprehensive evaluation of multiple difference indicators. Through specific mean, square root, power, weighted sum and difference calculation methods, the evaluated loss results can better reflect the difference between the prediction results and the real data.

[0059] In the training of the model, the multi-frame true value trajectory points can be divided into a training set and a verification set to construct a training sample for the scene simulation model. The training set includes a selected training starting point and at least one training target point, and the selection process can be random or selected according to a preset selection rule. The training starting point is used as the starting point of the specified vehicle in the model, and the state of the specified vehicle is set to be consistent with the state of the training starting point, and then the model is enabled to predict the trajectory from the training starting point to each training target point in sequence to obtain the second predicted trajectory.

[0060] The second predicted trajectory can be represented by a corresponding plurality of predicted trajectory points. In addition to corresponding to each training target point, the predicted trajectory points may also correspond to other true value trajectory points, that is, true value trajectory points in the validation set that have not been selected. For example, the true value trajectory points include 1, 2, 3, 4, 5, 6, 7, 8, and 9. The true value trajectory point 2 is selected as the training starting point, and 3, 5, and 8 are selected as the training target points. Then, the points of the validation set in the corresponding trajectory include 4, 6, and 7. Therefore, the second predicted trajectory may include a plurality of predicted trajectory points corresponding to the true value trajectory points 2, 3, 4, 5, 6, 7, and 8. In the subsequent specific loss calculation process, the preset trajectory points corresponding to the validation set may be selected as the predicted trajectory points for loss testing, or the preset trajectory points corresponding to the validation set plus the training target points may be selected for loss calculation, or the loss may be calculated only based on the training target points and the corresponding true value trajectory points to characterize the accuracy and effectiveness of the model prediction.

[0061] After determining the predicted trajectory and the predicted trajectory points corresponding to the corresponding true trajectory points including the training target points, the loss of the scene simulation model can be calculated based on the corresponding loss function. The loss can be calculated based on one or more loss functions to obtain one or more loss values ​​to represent the difference between the predicted trajectory points and the true trajectory points in different performance indicators.

[0062] Based on the calculated loss, the performance parameters of the corresponding scene simulation model can be adjusted. The adjustment process is usually implemented through the back propagation algorithm, that is, according to the size of the loss and the direction of the loss value change, the weights and biases in the model are updated and optimized, so that the model can be more consistent with the actual trajectory points in the next prediction. Through multiple iterative training of the scene simulation model, the prediction ability of the model will gradually improve until it meets certain accuracy requirements. Among them, whether the accuracy requirements are met can be determined by setting the corresponding loss threshold according to the accuracy. When the calculated losses are smaller than the threshold, it means that the trained model meets the accuracy requirements.

[0063] The above steps achieve efficient training of the scene simulation model by introducing true value main vehicle data, setting corresponding training sets and verification sets, and evaluating the performance of the model by calculating the loss and adjusting the performance parameters. The scene simulation model can learn the trajectory rules in real driving and has the ability to generate accurate and reasonable predicted trajectories, which not only improves the prediction accuracy and reliability of the model, but also makes the scene simulation closer to the real driving situation. After the training process, by adding a destination to the designated vehicle in the model, the corresponding scene simulation model can be equipped with the ability to generate a first predicted trajectory and instruct the designated vehicle to move along the first predicted trajectory, thereby making the vehicles in the simulation scene more goal-oriented and adaptable to more targeted driving scenarios, which is conducive to further improving the authenticity and effectiveness of the scene simulation.

[0064] Optionally, the losses include destination losses.

[0065] Although the trained scenario simulation model can receive the destination added to the designated vehicle, there are still deviations in its predicted trajectory and the process of instructing the designated vehicle to move according to the predicted trajectory. For example, there are jumps in speed, acceleration and steering angle in the predicted trajectory, or interference from other elements in the traffic flow during the driving process, which makes it impossible for the designated vehicle to achieve or unrealistically achieve the corresponding driving. It is impossible to ensure that the designated vehicle can arrive at the added destination accurately in sequence. Therefore, the destination loss function can be added to evaluate the model prediction trajectory in training and whether the specific driving trajectory of the designated vehicle can accurately and effectively reach the destination.

[0066] The destination loss can be calculated based on different destination loss functions. The destination loss function can be calculated by taking into account various factors that lead to the deviation between the predicted trajectory point and the destination, and can be calculated by a specific function or algorithm. For example, the distance between the predicted trajectory point and the destination can be considered to determine whether the predicted trajectory point accurately passes through each destination, and the order in which the predicted trajectory point passes through or approaches each destination can be considered to determine whether the order of the destinations in the actual true value trajectory point is met. Based on the differences in various factors, the differences can be processed by specific methods such as mean, square root, power or weighting in the destination loss function to determine the corresponding destination loss.

[0067] Optionally, see Figure 3 , based on the loss function, the loss of the predicted trajectory point can be calculated as follows:

[0068] Step 141: taking the point in the predicted trajectory that is closest to each training target point as the predicted target point corresponding to the training target point.

[0069] Step 142: Calculate the cross entropy between the point set of the predicted target point and the point set of the corresponding training target point.

[0070] Step 143: Calculate the average distance between each predicted target point and the corresponding training target point.

[0071] Step 144: The sum of the cross entropy and the distance average is taken as the destination loss.

[0072] In this scheme, the selected training target points are used as the destinations in training, so the predicted trajectory points closest to each training target point can be used as the predicted target points for subsequent loss calculations, where the distance can be determined by the point distance formula based on the point coordinates, or the predicted trajectory points can be fitted into a trajectory line, and the point-line distance between the training target point and the trajectory line is calculated to determine the closest distance and the corresponding predicted target point. The predicted target point corresponds to the training target point one by one, and is used to determine the deviation between the destination reached by the trajectory and the actual destination.

[0073] After obtaining the predicted target point corresponding to the training target point, the difference in distribution between the training target point and the predicted target point can be evaluated through cross entropy to determine whether the predicted target point corresponds to the training target point in the order of arrival, that is, to determine whether the predicted trajectory can reach the corresponding destination in order. The average distance value is an indicator used to quantify the degree of deviation between the predicted target point and the training target point reached by the predicted trajectory. The larger the average distance value, the greater the degree of deviation between the predicted target point and the training target point, and the corresponding destination cannot be accurately reached. Therefore, based on the calculation of cross entropy and distance average value, the accuracy and effectiveness of the trajectory prediction of the scene simulation model can be effectively judged. Subsequently, the model can be targeted to adjust parameters or optimize the function algorithm to reduce cross entropy and distance average value to ensure that the model has the ability to predict the trajectory and instruct the specified vehicle to arrive at the added destination accurately in order.

[0074] Optionally, the losses comprise kinematic losses.

[0075] In the process of a specified vehicle in a scene simulation model moving according to a predicted trajectory, multiple kinematic state parameters are adjusted and changed. However, the state change characteristics of a specified vehicle in the traditional way are determined by the sensor data and perception algorithm outside the vehicle, which can only determine the appearance of its state changes such as position, speed, acceleration, etc., with limited accuracy and unable to detect its specific operating characteristics. The movement of the simulated vehicle usually does not conform to the kinematic logic and lacks the realism of human driving. Therefore, the kinematic loss can be calculated and continuously reduced to make the movement of the vehicle simulated by the scene simulation model more in line with the kinematic logic.

[0076] The kinematic loss can be calculated based on different kinematic loss functions. The calculation of the kinematic loss can comprehensively consider the differences between the various kinematic parameters of the predicted trajectory points and the kinematic parameters corresponding to the true trajectory points. Subsequently, the model is targetedly adjusted in parameters or the function algorithm is optimized to reduce the differences in the corresponding kinematic parameters to ensure that the model has the ability to make the specified vehicle move in a manner that conforms to the kinematic logic.

[0077] Optionally, the true value trajectory points and the predicted trajectory points include data of corresponding multiple kinematic indicators, and the kinematic indicators include at least one of front and rear frame speed, acceleration, acceleration derivative, steering angle and steering angle derivative.

[0078] In the scenario simulation model, the movement of the vehicle involves not only simple position changes, but also a series of complex kinematic parameters. These parameters not only describe the basic motion state of the vehicle, but also reflect the dynamic characteristics of the vehicle during the motion and the driver's operating intention. By deeply analyzing these kinematic indicators, we can better understand the driving behavior and optimize the scenario simulation model based on the kinematic parameters, so that the predicted trajectory predicted by the scenario simulation model and the movement of the specified vehicle are closer to the actual driving situation.

[0079] The frame speed describes the change in the vehicle's speed over a continuous period of time, which reflects the acceleration, deceleration, or constant speed of the vehicle. By comparing the frame speeds of the true and predicted trajectory points, the accuracy of the model in simulating the change in vehicle speed can be evaluated.

[0080] Acceleration represents how quickly the vehicle's speed changes. It reveals the driver's control over the accelerator and brakes. Changes in acceleration are not only related to road conditions and traffic conditions, but also closely related to the driver's driving habits and vehicle performance.

[0081] The acceleration derivative is the rate of change of acceleration, which reflects the changing trend of the vehicle's acceleration. In real driving, the change of acceleration is often continuous and smooth to avoid discomfort to passengers. Therefore, by comparing the acceleration derivatives of the true value trajectory points and the predicted trajectory points, the smoothness of the model in simulating the change of vehicle acceleration can be evaluated.

[0082] The steering angle and steering angle derivative describe the steering state and steering speed of the vehicle. During driving, the driver needs to adjust the steering angle and steering speed of the vehicle according to the road conditions and traffic signals to achieve safe and comfortable driving. Therefore, by comparing the steering angle and steering angle derivative of the true value trajectory point and the predicted trajectory point, the accuracy of the model in simulating vehicle steering can be evaluated.

[0083] By comprehensively considering the above-mentioned kinematic indicators such as front and rear frame speed, acceleration, acceleration derivative, steering angle and steering angle derivative, the performance of the scene simulation model in simulating real driving scenes can be more comprehensively evaluated. This not only helps to improve the prediction accuracy and reliability of the model, but also helps to improve the authenticity and effectiveness of the simulation scene, which is conducive to better testing of the autonomous driving system.

[0084] Optionally, see Figure 4 , based on the loss function, the loss of the predicted trajectory point can also be calculated as follows:

[0085] Step 145: Calculate the difference between the data of various kinematic indicators corresponding to the predicted trajectory point and the corresponding true value trajectory point.

[0086] Step 146: The average of the differences is taken as the kinematic loss.

[0087] By calculating the difference between the predicted trajectory points and the true trajectory points in various kinematic indicators and taking the average of these differences as the kinematic loss, a quantitative evaluation can be provided, which is helpful to understand the accuracy of the predicted trajectory in simulating real driving behavior.

[0088] Kinematic indicators include the aforementioned front and rear frame speed, acceleration, acceleration derivative, steering angle and steering angle derivative, etc. The difference between these kinematic indicators reflects the degree of deviation between the predicted trajectory and the actual trajectory in terms of dynamic characteristics. The average value of each difference is used as the kinematic loss, which provides a global evaluation perspective on the dynamic characteristics reflected by each parameter. By continuously reducing this kinematic loss, the scene simulation model can be more consistent with the kinematic logic in simulating vehicle motion, thereby improving the authenticity and effectiveness of the simulation scene.

[0089] According to the different components of kinematic loss, the scene simulation model can be optimized more finely in the future. For example, if it is found that there is a large deviation in the simulation of speed change, the speed prediction algorithm in the model can be adjusted in a targeted manner; if it is found that the simulation of steering is not accurate enough, the steering control logic in the model can be optimized. Through the calculation of kinematic loss, the performance of the scene simulation model can be continuously improved, making it closer to the real driving scene, providing a more reliable simulation environment for the testing of the autonomous driving system. The optimization based on kinematic loss can effectively improve the prediction accuracy and reliability of the model, help improve the authenticity and effectiveness of the simulation scene, and provide more powerful support for the testing of the autonomous driving system.

[0090] Optionally, see Figure 5 , Figure 5 It is a structural schematic diagram of an embodiment of a driving scene simulation model provided by the present application.

[0091] The driving scenario simulation model 50 includes: an input end 51, used to obtain the current position of the specified vehicle and receive at least one destination different from the current position; a processing end 52, connected to the input end 51, used to generate a first predicted trajectory for the specified vehicle to arrive at at least one destination in sequence from the current position, and instruct the specified vehicle to move according to the first predicted trajectory.

[0092] The input terminal 51 can interact with relevant simulation test personnel through an interactive interface, and can also select a designated vehicle and a destination through the call of a program interface. The input terminal 51 can select a designated vehicle and indicate the current position of the designated vehicle, and then select the corresponding destination of the designated vehicle through an interactive interface or a program interface.

[0093] The processing end 52 is a network or algorithm mechanism within the model. Through advance training or presetting of corresponding processing algorithms, the driving scene simulation model 50 can generate a first predicted trajectory for a specified vehicle to arrive at at least one destination in sequence from the current position through the processing end 52, and instruct the specified vehicle to move according to the first predicted trajectory.

[0094] The method functions implemented by the driving simulation model can be found in Figure 1 The steps and description of the described driving simulation method will not be repeated here.

[0095] Optionally, the driving scene simulation model 50 is composed of Figures 2 to 4 The steps described in the training are obtained. The specific steps of the training to obtain the driving simulation scene can be found in Figures 2 to 4 The described steps and their optional solutions are not repeated here.

[0096] The aforementioned driving scene simulation method can be integrated through the driving scene simulation model 50. The driving scene simulation model 50 can more accurately simulate various driving scenes that an autonomous driving vehicle may face in road traffic, achieve more targeted testing of the performance of the autonomous driving system, and implement more targeted simulation scenes, thereby improving the flexibility, effectiveness, and practicality of driving scene simulation.

[0097] See also Figure 6 , Figure 6 It is a structural diagram of an embodiment of the storage medium provided by the present application.

[0098] The storage medium 60 stores program data 61. When the program data 61 is executed by the processor, the following is achieved: Figures 1 to 5 The described driving scenario simulation method.

[0099] The program data 61 is stored in a storage medium 60, and includes a number of instructions for enabling a network device (such as a router, a personal computer, a server, or other network device) or a processor to execute all or part of the steps of the method described in each embodiment of the present application.

[0100] Optionally, the storage medium 60 may be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, or other media that can store the program data 61 .

[0101] See also Figure 7 , Figure 7 It is a structural schematic diagram of an embodiment of a computer device provided by the present application.

[0102] The computer device 70 includes a processor 72 and a memory 71 connected to each other. The memory 71 stores a computer program. When the processor 72 executes the computer program, the following is achieved: Figures 1 to 5 The driving scene simulation method described herein. The memory 71 may include the storage medium 60, or may be other independently developed memory.

[0103] Different from the prior art, the present application discloses a driving scene simulation method, model and related device. By adding a destination to a designated vehicle in the scene simulation model, the designated vehicle in the scene can move according to its predicted trajectory to the destination, so that the behavior of the vehicle in the simulation scene has a clear goal orientation, and a specific driving scene can be simulated by specifying a destination, which can more accurately simulate various driving scenes that autonomous driving vehicles may face in road traffic, realize more targeted testing of the performance of the autonomous driving system, and realize the implementation of more targeted simulation scenes, thereby improving the flexibility, effectiveness and practicality of driving scene simulation.

[0104] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the model embodiment, storage medium and computer device embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0105] The present application can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0106] In the several embodiments provided in this application, it should be understood that the disclosed methods, models, storage media, and computer devices can be implemented in other ways. For example, the device implementation described above is only illustrative, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0107] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A driving scene simulation method, characterized in that: include: adding at least one destination different from the current location to a specified vehicle in the scenario simulation model; generating a first predicted trajectory of the designated vehicle from the current position to the at least one destination in sequence; The designated vehicle is instructed to move according to the first predicted trajectory.

2. The driving scene simulation method according to claim 1, characterized in that: Before adding at least one destination different from the current location to the designated vehicle in the scenario simulation model, the method further includes: The scenario simulation model is trained so that the scenario simulation model has the ability to generate the first predicted trajectory and instruct the designated vehicle to move along the first predicted trajectory.

3. The driving scene simulation method according to claim 2, characterized in that: The training of the scene simulation model comprises: Adding true host vehicle data and a loss function to the scene simulation model, wherein the true host vehicle data includes multiple frames of true trajectory points; Selecting a training starting point and at least one training target point in sequence from the true value trajectory points of the multiple frames; Generate a second predicted trajectory from the training starting point to the at least one training target point in sequence; wherein the second predicted trajectory includes a plurality of predicted trajectory points; Calculating the loss of the predicted trajectory point based on the loss function; A performance parameter of the scenario simulation model is adjusted based on the loss.

4. The driving scene simulation method according to claim 3, characterized in that: The losses mentioned include destination losses; The calculating the loss of the predicted trajectory point based on the loss function includes: The point in the predicted trajectory point that is closest to each of the training target points is used as the predicted target point corresponding to the training target point; Calculating the cross entropy between the point set of the predicted target point and the point set of the corresponding training target point; Calculating the average distance between each predicted target point and the corresponding training target point; The sum of the cross entropy and the distance average is taken as the destination loss.

5. The driving scene simulation method according to claim 3 or 4, characterized in that: The loss includes kinematic loss; the true value trajectory point and the predicted trajectory point include data of multiple corresponding kinematic indicators; The calculating the loss of the predicted trajectory point based on the loss function includes: Calculating the difference between the data of each of the kinematic indicators corresponding to the predicted trajectory point and the corresponding true value trajectory point; The average of the differences is taken as the kinematic loss.

6. The driving scene simulation method according to claim 5, characterized in that: The kinematic indicators include at least one of front-to-back frame velocity, acceleration, acceleration derivative, steering angle, and steering angle derivative.

7. The driving scene simulation method according to claim 1, characterized in that: The expected arrival time between the current position and the first destination in the sequence of destinations, and the arrival time between the previous and next destinations in the sequence of destinations are both lower than a preset time threshold, and the first predicted trajectory is within the preset time.

8. A driving scene simulation model, characterized in that: include: An input terminal, for obtaining a current position of a specified vehicle and receiving at least one destination different from the current position; The processing end is connected to the input end, and is used to generate a first predicted trajectory for the designated vehicle to sequentially arrive at the at least one destination from the current position, and instruct the designated vehicle to move according to the first predicted trajectory.

9. A storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the steps of the driving scenario simulation method according to any one of claims 1 to 7 are implemented.

10. A computer device, characterized in that: The method comprises a processor and a memory connected to each other, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the driving scene simulation method according to any one of claims 1 to 7 are implemented.

Images