Automatic driving trajectory planning method and device based on diffusion model and electronic equipment

Abstract

The application provides an automatic driving trajectory planning method and device based on a diffusion model and electronic equipment, wherein the method comprises the following steps: acquiring vehicle surrounding information and road information; based on a pre-trained trajectory planning model, predicting self-vehicle trajectory planning and surrounding vehicle trajectory according to the vehicle surrounding information and the road information; wherein the trajectory planning model comprises an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on a diffusion model, and the trajectory planning model is trained and optimized based on a training sample set composed of a surrounding vehicle information sample, a road information sample, a navigation information sample, a noise sample and a real driving trajectory. The method can accurately predict the automatic driving trajectories of the self-vehicle and the surrounding vehicles according to the real-time road driving conditions by applying the diffusion model to the automatic driving, not only improves the safety of the automatic driving system, but also can predict and adapt to complex traffic scenes, and further improves the user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to an automatic driving trajectory planning method and device based on a diffusion model and an electronic device. BACKGROUND

[0002] In recent years, automatic driving technology has developed rapidly, and trajectory planning, as a core technology of automatic driving, has also received extensive attention. Traditional trajectory planning methods mainly rely on a set of explicitly specified rules, such as behavior decision trees, vehicle collision avoidance, etc. Although these methods are effective in simple scenarios, they lack flexibility and are difficult to cope with complex and variable road conditions, and are prone to rule conflicts that lead to uncertain states of the system.

[0003] In order to improve the flexibility and robustness of the automatic driving system, some researchers have attempted to use machine learning methods for trajectory planning. These methods usually treat the trajectory planning problem as a prediction problem, training a model to learn the mapping relationship from sensor data to trajectory sequences.

[0004] However, existing machine learning-based methods usually rely on a large amount of complete and labeled data for training, and in complex and multi-modal data distribution, they are prone to insufficient fitting capacity, resulting in imperfect or unsafe planned paths.

[0005] Therefore, there is an urgent need to develop an automatic driving trajectory planning method that can fully utilize the flexibility of machine learning methods while effectively dealing with complex multi-modal data, in order to improve the robustness and safety of the system and meet the needs of future development of automatic driving technology. SUMMARY

[0006] The present application provides an automatic driving trajectory planning method and device based on a diffusion model to overcome the shortcomings of existing automatic driving trajectory planning methods in fitting complex scene multi-modal data distribution, resulting in low accuracy of automatic driving trajectory prediction. The present application can accurately predict the automatic driving trajectory of the ego vehicle and surrounding vehicles according to real-time road driving conditions.

[0007] In one aspect, the present application provides an automatic driving trajectory planning method based on a diffusion model, comprising: obtaining surrounding vehicle information and road information; based on a pre-trained trajectory planning model, predicting ego vehicle trajectory planning and surrounding vehicle trajectory according to the surrounding vehicle information and road information; wherein the trajectory planning model comprises an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on a diffusion model, and the trajectory planning model is trained and optimized based on a training sample set composed of surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories.

[0008] Further, the predicting, based on the surrounding vehicle information and the road information, the ego vehicle trajectory planning and the surrounding vehicle trajectories by using the pre-trained trajectory planning model comprises: encoding the surrounding vehicle information and the road information by using a pre-trained environment encoder to obtain an environment encoding vector; and predicting, based on the environment encoding vector, current navigation information, Gaussian distribution noise and a noise-added time step, the ego vehicle trajectory planning and the surrounding vehicle trajectories by using a pre-trained trajectory planning decoder.

[0009] Further, the environment encoder comprises a first self-attention module, a second self-attention module and a third self-attention module; and correspondingly, the encoding, by using the pre-trained environment encoder, the surrounding vehicle information and the road information to obtain the environment encoding vector comprises: inputting the surrounding vehicle information into the first self-attention module to obtain a surrounding vehicle feature vector; inputting the road information into the second self-attention module to obtain a road feature vector; splicing the surrounding vehicle feature vector and the road feature vector, and inputting the spliced feature vector into the third self-attention module to obtain the output environment encoding vector.

[0010] Further, the training and optimization of the environment encoder comprises: collecting surrounding vehicle information samples and road information samples; taking the masked surrounding vehicle information samples and the masked road information samples as input, taking reconstructed surrounding vehicle vectors and reconstructed road vectors as output, taking the difference between the reconstructed surrounding vehicle vectors and the surrounding vehicle information samples and the difference between the reconstructed road vectors and the road information samples as training loss, and iteratively optimizing the environment encoder.

[0011] Further, the training and optimization of the trajectory planning decoder comprises: obtaining environment encoding vector samples based on the pre-trained environment encoder and the collected surrounding vehicle information samples and road information samples; constructing a training sample dataset based on the collected navigation information samples, noise samples and real driving trajectories, and the environment encoding vector samples; taking the environment encoding vector samples, the navigation information samples, noise-added time steps and noise-added real driving trajectories as input, taking de-noised driving trajectories as output, iteratively optimizing the trajectory planning decoder by using diffusion model loss and preset trajectory optimization loss; guiding and correcting the de-noised driving trajectories by using a classifier based on the pre-trained trajectory planning decoder and a preset trajectory optimization function; and the real driving trajectories comprise ego vehicle real driving trajectories and surrounding vehicle real driving trajectories.

[0012] Further, in constructing the training sample data set, the real driving trajectory of the ego vehicle is subjected to data augmentation processing, specifically including: constructing a trajectory parameter equation using a quintic polynomial according to the position coordinates of the ego vehicle; determining initial and final state boundary conditions of the real driving trajectory of the ego vehicle, the initial and final state boundary conditions including the position, orientation angle, speed, acceleration and angular velocity of the starting point, and the position, orientation angle, speed, acceleration and angular velocity of the terminal point; determining a coefficient equation set according to the initial and final state boundary conditions; solving the trajectory parameter equation according to the coefficient equation set to obtain an interpolation trajectory, which is the real driving trajectory of the ego vehicle after data augmentation.

[0013] Further, the preset trajectory optimization function includes one or more of a collision loss, an out-of-bound loss and a comfort loss in a weighted combination.

[0014] In a second aspect, the present application further provides an automatic driving trajectory planning device based on a diffusion model, comprising: an information acquisition module for acquiring vehicle information and road information; a trajectory planning module for predicting ego vehicle trajectory planning and surrounding vehicle trajectories based on a pre-trained trajectory planning model according to the vehicle information and road information; wherein the trajectory planning model comprises an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on a diffusion model, and the trajectory planning model is trained and optimized based on a training sample set composed of surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories.

[0015] In a third aspect, the present application further provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the automatic driving trajectory planning method based on the diffusion model as described above.

[0016] In a fourth aspect, the present application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the automatic driving trajectory planning method based on the diffusion model as described above.

[0017] The application provides an automatic driving trajectory planning method based on a diffusion model, which comprises the following steps: acquiring surrounding vehicle information and road information, and predicting self-vehicle trajectory planning and surrounding vehicle trajectory based on the surrounding vehicle information and the road information based on a pre-trained trajectory planning model; wherein the trajectory planning model comprises an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on a diffusion model, and the trajectory planning model is trained and optimized based on a training sample set composed of surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories. The method can accurately predict the automatic driving trajectories of the self-vehicle and the surrounding vehicles according to real-time road driving conditions by applying the diffusion model to automatic driving, thereby improving the safety of the automatic driving system, predicting and adapting to complex traffic scenes, and further improving the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0019] Figure 1 is a flowchart of the automatic driving trajectory planning method based on the diffusion model provided by the embodiment of the present application.

[0020] Figure 2 is a whole flowchart of the automatic driving trajectory planning method based on the diffusion model provided by the embodiment of the present application.

[0021] Figure 3 is a deployment diagram of the environment encoder provided by the embodiment of the present application.

[0022] Figure 4 is a deployment diagram of the trajectory planning decoder provided by the embodiment of the present application.

[0023] Figure 5 is a training and optimization diagram of the environment encoder provided by the embodiment of the present application.

[0024] Figure 6 is a training and optimization diagram of the trajectory planning decoder provided by the embodiment of the present application.

[0025] Figure 7 is a flowchart of the collision trajectory optimization provided by the embodiment of the present application.

[0026] Figure 8 is a flowchart of the combined trajectory optimization provided by the embodiment of the present application.

[0027] Figure 9A data augmentation schematic diagram of a real driving trajectory of a vehicle is provided by an embodiment of the present application.

[0028] Figure 10 A structural schematic diagram of an automatic driving trajectory planning device based on a diffusion model is provided by an embodiment of the present application.

[0029] Figure 11 A physical structure schematic diagram of an electronic device is provided by an embodiment of the present application. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0031] It should be noted that the existing automatic driving trajectory planning method relies on a clearly specified rule set, and has insufficient flexibility, and is prone to rule conflicts, which leads the system to an uncertain state. In order to improve the flexibility and robustness of the automatic driving system, some research works attempt to get rid of the constraints of rules through machine learning methods, but there is a problem of poor fitting ability for complex scene multi-modal data distribution.

[0032] In view of this, the present application proposes an automatic driving trajectory planning method based on a diffusion model, specifically, Figure 1 A flow schematic diagram of the automatic driving trajectory planning method based on the diffusion model provided by an embodiment of the present application is shown.

[0033] As Figure 1 shown, the method comprises steps S110-S120, and the following will describe the steps S110-S120 and related steps in detail.

[0034] S110, acquiring vehicle surrounding information and road information.

[0035] It is easy to understand that in the field of automatic driving, trajectory planning refers to specifying a safe and reasonable driving route for an autonomous vehicle to achieve a specific purpose, such as reaching a destination. This process usually involves a series of complex technologies and strategies, and acquiring surrounding environment information is a very key part.

[0036] In this step, the surrounding vehicle information and road information need to be obtained. The surrounding vehicle information refers to the historical information of surrounding vehicles within a set historical period, including but not limited to the position, orientation angle, speed, length and width, and type of surrounding vehicles. The road information refers to various data and knowledge related to the road, including but not limited to geographic information, lane information, traffic signs, traffic signals, intersection information, and traffic flow, etc.

[0037] Among them, the geographic information includes the basic physical characteristics of the road such as geographic position, length, width, slope, and curvature. The lane information includes the number of lanes, the position of lane lines, and the separation between lanes (such as solid lines, dashed lines, central medians, etc.). Traffic signs include speed limit signs, prohibition signs, warning signs, etc., which provide important information about road rules. Traffic signals include traffic lights, pedestrian crossing signals, etc., which affect the driving decisions of vehicles. Intersection information includes the number of intersections, intersection types (such as roundabouts, crossroads, etc.), traffic rules (such as right turn, left turn, straight ahead, etc.). Traffic flow includes vehicle density, congestion, accidents, etc., which affect driving speed and path selection.

[0038] In addition, road information can also include road facilities, road conditions, and weather conditions. Among them, road facilities include shoulders, curbs, guardrails, bridges, tunnels, etc. Road conditions include wet and slippery degree, icing, potholes, construction areas, etc., which affect the driving safety of vehicles. Weather conditions include rain, snow, fog, etc. Special weather conditions may cause reduced visibility, affecting driving decisions.

[0039] In a specific embodiment, the surrounding vehicle information includes the position, orientation angle, speed, length and width, and type of surrounding vehicles, and the road information includes road points, boundary points, traffic light information, and speed limit information, which can be obtained through sensors such as laser radar and camera, etc.

[0040] On the basis of obtaining the surrounding vehicle information and road information in step S110, further, step S120 is executed.

[0041] S120, based on the pre-trained trajectory planning model, according to the surrounding vehicle information and road information, predicting the trajectory planning of the ego vehicle and the surrounding vehicles; wherein the trajectory planning model includes an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on a diffusion model, and the trajectory planning model is trained and optimized based on a training sample set composed of surrounding vehicle information samples, road information samples, navigation information samples, noise samples, and real driving trajectories.

[0042] It is easy to understand that the embodiment can use the surrounding vehicle information and the road information to predict the trajectories of the ego vehicle (the automatic driving vehicle itself) and the surrounding vehicles based on the pre-trained trajectory planning model. The trajectory planning model in the embodiment includes an environment encoder and a trajectory planning decoder. The environment encoder is used to process the input surrounding vehicle information and road information, and the trajectory planning decoder is used to generate the predicted trajectory planning.

[0043] Specifically, the environment encoder is used to encode the input surrounding vehicle information and road information, and convert the surrounding vehicle information and road information into a form more suitable for subsequent processing. For example, the environment encoder can encode the data from the sensors (such as lidar data and image data) into a feature vector representing the surrounding environment, which may include information such as the position, speed, orientation angle, size of the vehicle, and the geometry, traffic signs and other related elements of the road. The trajectory planning decoder is used to receive the output of the environment encoder and generate the predicted ego vehicle trajectory planning and surrounding vehicle trajectory in combination with other information, such as navigation information, noise information, etc.

[0044] It is worth mentioning that the trajectory planning decoder in the trajectory planning model is based on a diffusion model, which is a probabilistic model that can generate continuous and smooth trajectories. The diffusion model simulates the real trajectory distribution by adding random noise step by step, so as to obtain more natural and diverse trajectory planning.

[0045] In order to train the trajectory planning model, a training sample set containing surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories needs to be prepared. These training samples can come from real-world data collection or be generated by simulation, which is not limited here.

[0046] During training, the predicted trajectory of the model is compared with the real driving trajectory by using the back propagation algorithm, and the loss function (such as mean square error) is calculated. Then, the gradient descent method is used to update the model parameters, so that they gradually fit the real trajectory. Repeat this training process until the model reaches the predetermined performance indicators.

[0047] After the training is completed, the trajectory planning model can predict the trajectory planning of the ego vehicle and the surrounding vehicles according to the surrounding vehicle information and the road information, which is crucial for decision-making such as avoidance and overtaking. By predicting the behavior of other vehicles, measures can be taken in advance to prevent collisions.

[0048] It should be noted that the surrounding vehicle information obtained in the embodiment is historical, the road information is objective, and the predicted ego vehicle trajectory planning and surrounding vehicle trajectory are future. As for how much of the past and how much of the future, it can be set according to actual needs, which is not limited here.

[0049] For example, in a specific embodiment, the historical time is taken as the past 2 seconds, and the future time is taken as the future 8 seconds.

[0050] In addition, Figure 2 The overall flowchart of the diffusion model-based automatic driving trajectory planning method provided by the embodiment of the application is shown. As shown in the figure, Figure 2 As shown, the surrounding vehicle information and the road information are first obtained, and then the surrounding vehicle information and the road information are encoded by using a pre-trained encoder, and the encoding output is taken as one of the inputs of a trajectory planning decoder, which, in combination with other inputs (which will be described in detail in the following embodiments), can predict and output the ego vehicle trajectory planning and the surrounding vehicle trajectory.

[0051] In the embodiment, the ego vehicle trajectory planning and the surrounding vehicle trajectory are predicted based on the pre-trained trajectory planning model according to the surrounding vehicle information and the road information, wherein the trajectory planning model includes an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on a diffusion model, and the trajectory planning model is trained and optimized based on a training sample set composed of surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories. The method can accurately predict the automatic driving trajectories of the ego vehicle and the surrounding vehicles according to the real-time road driving conditions by applying the diffusion model to the automatic driving, which not only improves the safety of the automatic driving system, but also predicts and adapts to complex traffic scenarios, thereby improving the user experience.

[0052] On the basis of the above-mentioned embodiments, further, the process of predicting the ego vehicle trajectory planning and the surrounding vehicle trajectory by the trajectory planning model according to the surrounding vehicle information and the road information will be described in detail.

[0053] Based on the pre-trained trajectory planning model, the ego vehicle trajectory planning and the surrounding vehicle trajectory are predicted according to the surrounding vehicle information and the road information, including: based on the pre-trained environment encoder, the surrounding vehicle information and the road information are encoded to obtain an environment encoding vector; based on the pre-trained trajectory planning decoder, the ego vehicle trajectory planning and the surrounding vehicle trajectory are predicted according to the environment encoding vector, the current navigation information, the Gaussian distribution noise and the noise time step.

[0054] It is easy to understand that the process of the trajectory planning model on the surrounding vehicle information and the road information includes the process of the environment encoder and the process of the trajectory planning decoder.

[0055] Figure 3 The deployment schematic diagram of the environment encoder provided by the embodiment of the application is shown.

[0056] As Figure 3As shown, the environment encoder includes a first self-attention module, a second self-attention module, and a third self-attention module.

[0057] Correspondingly, based on the pre-trained environment encoder, the surrounding vehicle information and the road information are encoded to obtain an environment encoding vector, including: inputting the surrounding vehicle information (including the information of A surrounding vehicles in T time steps) into the first self-attention module to obtain a surrounding vehicle feature vector; inputting the road information (including the information of N coordinate points in B lanes) into the second self-attention module to obtain a road feature vector; splicing the surrounding vehicle feature vector and the road feature vector, and inputting the spliced feature vector into the third self-attention module to obtain the output environment encoding vector.

[0058] Further, Figure 4 A deployment schematic diagram of the trajectory planning decoder provided by the embodiment of the application is shown.

[0059] As Figure 4 shown, the input of the trajectory planning decoder includes four parts, i.e., the information after encoding of the environment encoder (i.e., the environment encoding vector, also known as Senario Condition), the current navigation information (i.e., Navigation Condition), Gaussian distribution noise x t and the noise time step t The output of the trajectory planning decoder is the predicted self-vehicle trajectory planning and surrounding vehicle trajectory at the t time step.

[0060] The trajectory planning decoder includes an LN (Layer Norm), a scale and shift module, a self-attention mechanism module, a cross-attention mechanism module, a multi-head attention mechanism module, a feedforward network module (FFN), and a multi-layer perceptron (MLP).

[0061] The self-attention mechanism module is used to help the model focus on different parts of the input sequence and give different weights according to the context. The output of the self-attention mechanism module will be scaled and LN.

[0062] The cross-attention mechanism module is used to fuse the output of the self-attention mechanism module with the information after encoding of the environment encoder to obtain more rich information. Similarly, the output of the cross-attention mechanism module will also be scaled and LN. In particular, Figure 4The structural details of the cross-attention mechanism module and the self-attention mechanism module are given in detail, and in particular, both of the two attention mechanism modules adopt multi-head attention for information fusion, and further process the information using a feedforward network (FFN).

[0063] The feedforward network module is a fully connected network for further processing the outputs of the self-attention mechanism module and the cross-attention mechanism module.

[0064] The trajectory planning decoder specifically processes the information encoded by the environment encoder (i.e., the environment encoding vector), the current navigation information, the Gaussian distribution noise x t and the noise-added time step t The specific processing process of the trajectory planning decoder can be referred to Figure 4 , which will not be described one by one here.

[0065] In this embodiment, based on the pre-trained environment encoder, the surrounding vehicle information and the road information are encoded to obtain an environment encoding vector, and based on the pre-trained trajectory planning decoder, the ego vehicle trajectory planning and the surrounding vehicle trajectory are predicted according to the environment encoding vector, the current navigation information, the Gaussian distribution noise and the noise-added time step. This method can accurately predict the automatic driving trajectories of the ego vehicle and the surrounding vehicles according to the real-time road driving conditions by applying the diffusion model to automatic driving, not only improving the safety of the automatic driving system, but also predicting and adapting to complex traffic scenarios, thereby improving the user experience.

[0066] On the basis of the above embodiment, further, the training and optimization process of the environment encoder in the trajectory planning model will be described in detail below.

[0067] The training and optimization of the environment encoder specifically includes: collecting surrounding vehicle information samples and road information samples; taking the masked surrounding vehicle information samples and road information samples as inputs, taking the reconstructed surrounding vehicle vector and the reconstructed road vector as outputs, taking the difference between the reconstructed surrounding vehicle vector and the surrounding vehicle information sample and the difference between the reconstructed road vector and the road information sample as training loss, and iteratively optimizing the environment encoder.

[0068] It can be understood that the environment encoder is optimized in a self-supervised training manner in this embodiment. Specifically, Figure 5 The training and optimization of the environment encoder is shown in the schematic diagram of the environment encoder provided by the embodiment of the present application.

[0069] As Figure 5 shown, the collected surrounding vehicle information samples and road information samples are masked, and then the masked surrounding vehicle information samples and the masked road information samples are taken as two inputs of the training environment encoder respectively to obtain two corresponding output vectors.

[0070] Further, the two output vectors respectively pass through a linear network, which can be a fully connected layer or a multi-layer perceptron, for further compressing or converting the output vectors of the environment encoder, so as to obtain the reconstructed surrounding vehicle vector and the reconstructed road vector.

[0071] Then, the difference between the reconstructed surrounding vehicle vector and the surrounding vehicle information sample, and the difference between the reconstructed road vector and the road information sample are taken as the training loss, and the environment encoder is iteratively optimized for multiple rounds, so as to obtain the trained environment encoder.

[0072] It can be known that the purpose of training and optimizing the environment encoder is to make the reconstructed surrounding vehicle vector output by the environment encoder more approximate to the actual surrounding vehicle information sample, and to make the reconstructed road vector output by the environment encoder more approximate to the actual road information sample.

[0073] It should be noted that the environment encoder provided in the embodiment can be constructed based on multiple convolutional layers and pooling layers, or other network structures capable of achieving the same function, which are not limited here.

[0074] In the embodiment, the environment encoder is iteratively optimized by taking the masked surrounding vehicle information sample and the road information sample as input, taking the reconstructed surrounding vehicle vector and the reconstructed road vector as output, and taking the difference between the reconstructed surrounding vehicle vector and the surrounding vehicle information sample and the difference between the reconstructed road vector and the road information sample as the training loss. Then, the surrounding vehicle information and the road information are encoded based on the pre-trained environment encoder to obtain the environment encoding vector, and the ego vehicle trajectory and the surrounding vehicle trajectory are predicted based on the pre-trained trajectory planning decoder according to the environment encoding vector, the current navigation information, the Gaussian distribution noise and the noise time step. The method can accurately predict the automatic driving trajectories of the ego vehicle and the surrounding vehicles according to the real-time road driving conditions by applying the diffusion model to the automatic driving, which not only improves the safety of the automatic driving system, but also can predict and adapt to complex traffic scenes, thereby improving the user experience.

[0075] On the basis of the above embodiment, further, the training and optimization process of the trajectory planning decoder in the trajectory planning model will be described in detail.

[0076] The training optimization trajectory planning decoder specifically comprises: based on a pre-trained environment encoder, obtaining environment encoding vector samples according to collected vehicle information samples and road information samples; constructing a training sample dataset according to collected navigation information samples, noise samples and real driving trajectories, and the environment encoding vector samples; taking the environment encoding vector samples, the navigation information samples, the noise-added time steps and the noise-added real driving trajectories as inputs, and taking the de-noised driving trajectories as outputs, iteratively optimizing the trajectory planning decoder through diffusion model loss; guiding and correcting the de-noised driving trajectories through a classifier according to a pre-trained trajectory planning decoder and a preset trajectory optimization function; wherein the real driving trajectories comprise self-vehicle real driving trajectories and surrounding vehicle real driving trajectories.

[0077] Figure 6 A training optimization schematic diagram of the trajectory planning decoder provided by the embodiment of the application is shown.

[0078] According to Figure 6 It can be seen that when training the trajectory planning decoder, the input thereof comprises four parts, namely, environment encoder encoded information (i.e., environment encoding vector samples, such as green round dots in Figure 6 , navigation information samples (such as red round dots in Figure 6 , Gaussian distribution noise x t (such as self-vehicle future trajectories and surrounding vehicle future trajectories after noise addition at t steps in Figure 6 ) and noise-added time steps t , and the output of the trajectory planning decoder is predicted x t-1 , i.e., de-noised driving trajectories.

[0079] It should be noted that even the self-vehicle future trajectories and surrounding vehicle future trajectories after noise addition at t steps also need to be given a certain prompt, such as the current time state of the self-vehicle and surrounding vehicles.

[0080] It should also be noted that the network structure of the trajectory planning decoder and the functions of the related modules have been described in detail in the above embodiment, and will not be expanded here.

[0081] It is worth mentioning that diffusion model loss is used when iteratively optimizing the trajectory planning decoder, and specifically, the diffusion model loss corresponds to a diffusion model loss function expression as follows.

[0082] (1).

[0083] In formula (1), represents, represents the self-vehicle future trajectories (self-vehicle real driving trajectories) and surrounding vehicle future trajectories (surrounding vehicle real driving trajectories) in the dataset, represents a Gaussian noise, represents a number of added noises, represents a model output, represents a self-vehicle future trajectory and a surrounding vehicle future trajectory after adding noises for t steps.

[0084] In addition, in a specific embodiment, on the basis of pre-training the trajectory planning decoder through the diffusion model loss, a preset trajectory optimization function is used for classifier-guided trajectory optimization for the output planning trajectory (the de-noised driving trajectory), and the preset trajectory optimization function includes one or more weighted combinations of collision loss, out-of-bound loss, and comfort loss.

[0085] Taking the preset trajectory optimization function as the collision loss, Figure 7 a flowchart of collision trajectory optimization provided by the embodiment of the application is shown.

[0086] As shown in Figure 7 after completing the pre-training of the trajectory planning decoder, the de-noised driving trajectory output by the trajectory planning decoder is directly estimated according to the de-noised driving trajectory x t-1 , and collision detection is performed thereon to calculate the collision loss. Then, the de-noising process of the trajectory planning decoder is corrected by using the classifier guidance according to the collision loss, so as to reduce the probability of generating unsafe trajectories by the model and improve the safety performance.

[0087] Figure 7 The collision detection module in the collision detection module is used for performing collision detection according to the de-noised driving trajectory. Specifically, the de-noised driving trajectory includes the driving trajectory of the ego vehicle and the driving trajectory of the surrounding vehicle, from which the predicted distance between the ego vehicle and the surrounding vehicle can be calculated. If the predicted distance is higher than the first distance threshold, it means that there is a collision risk between the ego vehicle and the surrounding vehicle, and the collision loss is large at this time. Conversely, it means that there is no collision risk between the ego vehicle and the surrounding vehicle, and the collision loss is small at this time. Of course, the collision risk here is not limited to the ego vehicle and the surrounding vehicle, but can also be the ego vehicle and other obstacles on the road, which is not limited here.

[0088] The first distance threshold can be set according to the actual situation, which is not limited here.

[0089] Taking the preset trajectory optimization loss as a weighted combination of the collision loss, the out-of-bound loss, and the comfort loss as an example, Figure 8 a flowchart of combined trajectory optimization provided by the embodiment of the application is shown.

[0090] As shown in Figure 8 , in addition to the collision loss, the preset trajectory optimization loss also includes the out-of-bound loss and the comfort loss.

[0091] Correspondingly, the out-of-bound loss is calculated by the out-of-bound detection module, and the comfort loss is calculated by the comfort calculation module, and the calculation process is the same as the calculation process of the collision loss. Specifically, according to the denoised driving trajectory, the predicted distance between the ego vehicle and the road boundary can be obtained. If the predicted distance is higher than the second distance threshold, it means that there is an out-of-bound risk between the ego vehicle and the surrounding vehicle, and the out-of-bound loss is larger. On the contrary, it means that there is no out-of-bound risk between the ego vehicle and the surrounding vehicle, and the out-of-bound loss is smaller.

[0092] Similarly, the comfort loss is calculated by the comfort calculation module. Specifically, the speed, acceleration and angular velocity of the ego vehicle are obtained, and the speed, acceleration and angular velocity of the ego vehicle are compared with the corresponding threshold, respectively. If it is higher than the threshold, it means that it is not comfortable, and the comfort loss is larger. On the contrary, it means that it is more comfortable, and the comfort loss is smaller.

[0093] Wherein, the second distance threshold can be set according to the actual situation, which is not limited here.

[0094] After calculating the collision loss, the out-of-bound loss and the comfort loss, different weights are given according to the importance degree of each loss to perform weighted combination. Then, the combined loss and the diffusion model loss are back propagated to minimize the combined loss and the diffusion model loss, so as to update and optimize the model parameters.

[0095] In this embodiment, the environment encoding vector sample, the navigation information sample, the noise time step and the real driving trajectory after noise are input, and the denoised driving trajectory is output. The trajectory planning decoder is iteratively optimized by the diffusion model loss and the preset trajectory optimization loss, and then based on the pre-trained trajectory planning decoder, the ego vehicle trajectory planning and the surrounding vehicle trajectory are predicted according to the environment encoding vector, the current navigation information, the Gaussian distribution noise and the noise time step. This method can accurately predict the automatic driving trajectory of the ego vehicle and the surrounding vehicle by applying the diffusion model to the automatic driving according to the real-time road driving condition, not only improves the safety of the automatic driving system, but also can predict and adapt to complex traffic scenes, and further improves the user experience.

[0096] In a specific embodiment, when constructing the training sample data set, the real driving trajectory of the ego vehicle is subjected to data augmentation processing.

[0097] Specifically, according to the position coordinates of the ego vehicle, a trajectory parameter equation is constructed using a quintic polynomial; initial and final state boundary conditions of the real driving trajectory of the ego vehicle are determined, the initial and final state boundary conditions including the position, orientation angle, velocity, acceleration and angular velocity of the starting point, and the position, orientation angle, velocity, acceleration and angular velocity of the terminal point; a coefficient equation set is determined according to the initial and final state boundary conditions; the trajectory parameter equation is solved according to the coefficient equation set to obtain an interpolation trajectory, the interpolation trajectory being the real driving trajectory of the ego vehicle after data augmentation.

[0098] First, according to the position coordinates of the ego vehicle, a trajectory parameter equation is constructed using a quintic polynomial, the trajectory parameter equation being as follows (2)-(3).

[0099] (2).

[0100] (3).

[0101] In the equations (2)-(3), , denotes the current position coordinates of the ego vehicle, , , , , , is a coefficient, , , , , , is a coefficient.

[0102] Then, initial and final state boundary conditions of the real driving trajectory of the ego vehicle are determined, and a coefficient equation set is determined according to the initial and final state boundary conditions, the coefficient equation set specifically referring to the following equations (4)-(9).

[0103] (4).

[0104] (5).

[0105] (6).

[0106] (7).

[0107] (8).

[0108] (9).

[0109] In the equations (4)-(9), , denotes the position coordinates of the starting point 0, a velocity of the starting point 0, an orientation angle of the starting point 0, an acceleration of the starting point 0, an angular velocity of the starting point 0, a position coordinate of the ending point T, a velocity of the ending point T, an orientation angle of the ending point T, an acceleration of the ending point T, an angular velocity of the ending point T.

[0110] According to the coefficient equation set (4)-(9), the trajectory parameter equations (2)-(3) are solved, and a curve, that is, the interpolated trajectory, that is, the data-augmented real driving trajectory of the ego vehicle, is obtained. The interpolated trajectory represents the trajectory of the vehicle from the starting position to the target position.

[0111] Figure 9 A data augmentation schematic diagram of the real driving trajectory of the ego vehicle provided by the embodiment of the present application is shown. As Figure 9 shown, the blue curve represents the real driving trajectory of the vehicle, and the yellow curve represents the interpolated trajectory obtained by polynomial interpolation. The yellow curve can be adjusted according to the actual situation to adapt to the new target position and direction.

[0112] In this embodiment, by using a quintic polynomial to construct a trajectory parameter equation according to the position coordinates of the ego vehicle, and determining the initial and final state boundary conditions of the real driving trajectory of the ego vehicle, and then determining a coefficient equation set according to the initial and final state boundary conditions, an interpolated trajectory is obtained by solving the trajectory parameter equation according to the coefficient equation set, and the interpolated trajectory is the data-augmented real driving trajectory of the ego vehicle. This method performs data augmentation on the real driving trajectory of the ego vehicle when constructing the training sample data set, expands the size of the data set, enables the trajectory planning decoder to learn more diversified driving behaviors, and improves the generalization ability and the ability to cope with unknown situations of the model.

[0113] Corresponding to the automatic driving trajectory planning method based on the diffusion model described in the above embodiments, the present application also provides an automatic driving trajectory planning device based on the diffusion model. Specifically, Figure 10 A structure schematic diagram of the automatic driving trajectory planning device based on the diffusion model provided by the embodiment of the present application is shown.

[0114] As Figure 10 ​As shown, the device comprises: an information acquisition module 1010, configured to acquire vehicle information and road information; a trajectory planning module 1020, configured to predict self-vehicle trajectory planning and surrounding vehicle trajectory based on a pre-trained trajectory planning model according to the vehicle information and road information; wherein the trajectory planning model comprises an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on a diffusion model, and the trajectory planning model is obtained by training and optimization based on a training sample set composed of vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories.

[0115] In the present embodiment, the vehicle information and road information are acquired by the information acquisition module 1010, and the self-vehicle trajectory planning and surrounding vehicle trajectory are predicted based on a pre-trained trajectory planning model by the trajectory planning module 1020 according to the vehicle information and road information; wherein the trajectory planning model comprises an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on a diffusion model, and the trajectory planning model is obtained by training and optimization based on a training sample set composed of vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories. By applying the diffusion model to automatic driving, the device can accurately predict the automatic driving trajectory of the self-vehicle and surrounding vehicles according to the real-time road driving conditions, not only improving the safety of the automatic driving system, but also predicting and adapting to complex traffic scenarios, thereby improving the user experience.

[0116] It should be noted that the automatic driving trajectory planning device based on the diffusion model provided by the present embodiment can be mutually corresponding to the automatic driving trajectory planning method based on the diffusion model described in the above embodiments, and will not be described here.

[0117] Figure 11 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 11As shown, the electronic device can include a processor 1110, a communications interface 1120, a memory 1130, and a communications bus 1140, wherein the processor 1110, the communications interface 1120, and the memory 1130 complete mutual communication through the communications bus 1140. The processor 1110 can invoke a logical instruction in the memory 1130 to execute the automatic driving trajectory planning method based on the diffusion model, which includes: obtaining vehicle information and road information; based on a pre-trained trajectory planning model, predicting self-vehicle trajectory planning and surrounding vehicle trajectories according to the vehicle information and the road information; wherein the trajectory planning model includes an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on a diffusion model, and the trajectory planning model is trained and optimized based on a training sample set composed of vehicle information samples, road information samples, navigation information samples, noise samples, and real driving trajectories.

[0118] In addition, the logical instruction in the memory 1130 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0119] On the other hand, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, which is executed by a processor to implement the automatic driving trajectory planning method based on the diffusion model provided by the above-mentioned method, which includes: obtaining vehicle information and road information; based on a pre-trained trajectory planning model, predicting self-vehicle trajectory planning and surrounding vehicle trajectories according to the vehicle information and the road information; wherein the trajectory planning model includes an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on a diffusion model, and the trajectory planning model is trained and optimized based on a training sample set composed of vehicle information samples, road information samples, navigation information samples, noise samples, and real driving trajectories.

[0120] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0121] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, and the computer software products can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0122] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An autonomous driving trajectory planning method based on a diffusion model, characterized in that, include: Obtain vehicle and road information; Based on a pre-trained trajectory planning model, the vehicle trajectory planning and the surrounding vehicle trajectory are predicted according to the surrounding vehicle information and road information. The trajectory planning model includes an environmental encoder and a trajectory planning decoder. The trajectory planning decoder is constructed based on a diffusion model. The trajectory planning model is trained and optimized based on a training sample set consisting of surrounding vehicle information samples, road information samples, navigation information samples, noise samples, and real driving trajectories. The environmental encoder includes a first self-attention module, a second self-attention module, and a third self-attention module; Based on a pre-trained environment encoder, the surrounding vehicle information and road information are encoded to obtain an environment encoding vector, including: The surrounding vehicle information is input into the first self-attention module to obtain the surrounding vehicle feature vector; The road information is input into the second self-attention module to obtain the road feature vector; The vehicle feature vector and the road feature vector are concatenated, and the concatenated feature vector is input into the third self-attention module to obtain the output environment encoding vector; Training and optimizing the trajectory planning decoder specifically includes: Based on a pre-trained environment encoder, environment encoding vector samples are obtained from collected vehicle information samples and road information samples. A training sample dataset is constructed based on the collected navigation information samples, noise samples, real driving trajectories, and environmental coding vector samples. The trajectory planning decoder is iteratively optimized by using the environmental coding vector sample, navigation information sample, noisy time step and noisy real driving trajectory as input, and the denoising driving trajectory as output. Based on the pre-trained trajectory planning decoder and the preset trajectory optimization function, the classifier guides and corrects the denoised driving trajectory. The actual driving trajectory includes the actual driving trajectory of the vehicle itself and the actual driving trajectory of the surrounding vehicles.

2. The autonomous driving trajectory planning method based on a diffusion model according to claim 1, characterized in that, The pre-trained trajectory planning model predicts the vehicle trajectory and the surrounding vehicle trajectory based on the surrounding vehicle information and road information, including: Based on a pre-trained environment encoder, the surrounding vehicle information and road information are encoded to obtain an environment encoding vector. Based on the pre-trained trajectory planning decoder, the vehicle trajectory planning and the surrounding vehicle trajectory are predicted according to the environment encoding vector, current navigation information, Gaussian distributed noise and noisy time step.

3. The autonomous driving trajectory planning method based on a diffusion model according to claim 1, characterized in that, Training and optimizing the encoder environment specifically includes: Collect samples of vehicle information and road information; The environment encoder is iteratively optimized by taking the masked vehicle information samples and road information samples as inputs, the reconstructed vehicle vectors and road vectors as outputs, and the differences between the reconstructed vehicle vectors and vehicle information samples, and the differences between the reconstructed road vectors and road information samples as training losses.

4. The autonomous driving trajectory planning method based on the diffusion model according to claim 1, characterized in that, When constructing the training sample dataset, data augmentation processing is performed on the actual driving trajectories of the vehicle, specifically including: Based on the vehicle's position coordinates, a fifth-order polynomial is used to construct the trajectory parametric equations. The initial and final state boundary conditions of the actual driving trajectory of the vehicle are determined. The initial and final state boundary conditions include the position, orientation angle, velocity, acceleration and angular velocity of the starting point, and the position, orientation angle, velocity, acceleration and angular velocity of the ending point. Based on the initial and final state boundary conditions, determine the system of coefficient equations; Based on the set of coefficient equations, the trajectory parameter equations are solved to obtain the interpolated trajectory, which is the actual driving trajectory of the vehicle after data augmentation.

5. The autonomous driving trajectory planning method based on a diffusion model according to claim 1, characterized in that, The preset trajectory optimization function includes one or more weighted combinations of collision loss, out-of-bounds loss, and comfort loss.

6. An autonomous driving trajectory planning device based on a diffusion model, characterized in that, include: The information acquisition module is used to acquire surrounding vehicle information and road information; The trajectory planning module is used to predict the vehicle trajectory and the surrounding vehicle trajectory based on the pre-trained trajectory planning model and the surrounding vehicle information and road information. The trajectory planning model includes an environmental encoder and a trajectory planning decoder. The trajectory planning decoder is constructed based on a diffusion model. The trajectory planning model is trained and optimized based on a training sample set consisting of surrounding vehicle information samples, road information samples, navigation information samples, noise samples, and real driving trajectories. The environmental encoder includes a first self-attention module, a second self-attention module, and a third self-attention module; Based on a pre-trained environment encoder, the surrounding vehicle information and road information are encoded to obtain an environment encoding vector, including: The surrounding vehicle information is input into the first self-attention module to obtain the surrounding vehicle feature vector; The road information is input into the second self-attention module to obtain the road feature vector; The vehicle feature vector and the road feature vector are concatenated, and the concatenated feature vector is input into the third self-attention module to obtain the output environment encoding vector; Training and optimizing the trajectory planning decoder specifically includes: Based on a pre-trained environment encoder, environment encoding vector samples are obtained from collected vehicle information samples and road information samples. A training sample dataset is constructed based on the collected navigation information samples, noise samples, real driving trajectories, and environmental coding vector samples. The trajectory planning decoder is iteratively optimized by using the environmental coding vector sample, navigation information sample, noisy time step and noisy real driving trajectory as input, and the denoising driving trajectory as output. Based on the pre-trained trajectory planning decoder and the preset trajectory optimization function, the classifier guides and corrects the denoised driving trajectory. The actual driving trajectory includes the actual driving trajectory of the vehicle itself and the actual driving trajectory of the surrounding vehicles.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the autonomous driving trajectory planning method based on the diffusion model as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the autonomous driving trajectory planning method based on the diffusion model as described in any one of claims 1 to 5.

Images