Automatic driving track planning method and device based on diffusion model and electronic equipment

Abstract

The invention provides an automatic driving track planning method and device based on a diffusion model, and electronic equipment. The method comprises the following steps: acquiring surrounding vehicle information and road information; on the basis of a pre-trained trajectory planning model, according to the surrounding vehicle information and the road information, predicting own vehicle trajectory planning and surrounding vehicle trajectories; wherein the trajectory planning model comprises an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on a diffusion model, and the trajectory planning model is obtained by training and optimizing a training sample set formed by surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories. According to the method, the diffusion model is applied to automatic driving, the automatic driving tracks of the vehicle and surrounding vehicles can be accurately predicted according to the real-time road driving condition, the safety of an automatic driving system is improved, complex traffic scenes can be predicted and adapted, and then the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an autonomous driving trajectory planning method, device and electronic equipment based on a diffusion model. Background Art

[0002] In recent years, autonomous driving technology has developed rapidly, and trajectory planning, as a core technology of autonomous driving, has also received widespread attention. Traditional trajectory planning methods mainly rely on clearly defined rule sets, such as behavior-based decision trees, vehicle collision avoidance, etc. Although these methods are effective in simple scenarios, they lack flexibility and are difficult to cope with complex and changing road conditions. They are also prone to rule conflicts that cause the system to fall into an uncertain state.

[0003] In order to improve the flexibility and robustness of autonomous driving systems, some researchers have tried to use machine learning methods for trajectory planning. These methods usually regard the trajectory planning problem as a prediction problem and train the model to learn the mapping relationship from sensor data to trajectory sequences.

[0004] However, existing machine learning-based methods usually rely on large amounts of complete and labeled data for training. Under complex and multimodal data distributions, they are prone to insufficient fitting capabilities, resulting in incomplete planning paths or safety risks.

[0005] Therefore, there is an urgent need to develop an autonomous driving trajectory planning method that can fully utilize the flexibility of machine learning methods while effectively coping with complex multimodal data, so as to improve the robustness and safety of the system and meet the needs of future autonomous driving technology development. Summary of the invention

[0006] The present invention provides an autonomous driving trajectory planning method, device and electronic device based on a diffusion model, which are used to overcome the defect that the existing autonomous driving trajectory planning method has poor fitting ability for multimodal data distribution of complex scenes, resulting in low accuracy of autonomous driving trajectory prediction, and can accurately predict the autonomous driving trajectories of the vehicle and surrounding vehicles according to real-time road driving conditions.

[0007] On the one hand, the present invention provides an automatic driving trajectory planning method based on a diffusion model, comprising: obtaining surrounding vehicle information and road information; based on a pre-trained trajectory planning model, predicting the own vehicle trajectory planning and the surrounding vehicle trajectory according to the surrounding vehicle information and road information; wherein the trajectory planning model comprises an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on the diffusion model, and the trajectory planning model is trained and optimized based on a training sample set consisting of surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories.

[0008] Furthermore, the trajectory planning model based on pre-training predicts the self-vehicle trajectory planning and the surrounding vehicle trajectory according to the surrounding vehicle information and the road information, including: encoding the surrounding vehicle information and the road information based on a pre-trained environment encoder to obtain an environment coding vector; based on a pre-trained trajectory planning decoder, predicting the self-vehicle trajectory planning and the surrounding vehicle trajectory according to the environment coding vector, current navigation information, Gaussian distribution noise and noise-added time step.

[0009] Furthermore, the environment encoder includes a first self-attention module, a second self-attention module and a third self-attention module; accordingly, the pre-trained environment encoder encodes the surrounding vehicle information and road information to obtain an environment encoding vector, including: inputting the surrounding vehicle information into the first self-attention module to obtain a surrounding vehicle feature vector; inputting the road information into the second self-attention module to obtain a road feature vector; splicing the surrounding vehicle feature vector and the road feature vector, and inputting the spliced ​​feature vector into the third self-attention module to obtain the output environment encoding vector.

[0010] Furthermore, training and optimizing the environment encoder specifically includes: collecting surrounding vehicle information samples and road information samples; taking the masked surrounding vehicle information samples and road information samples as input, reconstructing the surrounding vehicle vector and reconstructing the road vector as output, taking the difference between the reconstructed surrounding vehicle vector and the surrounding vehicle information sample, and the difference between the reconstructed road vector and the road information sample as training loss, and iteratively optimizing the environment encoder.

[0011] Furthermore, training and optimizing the trajectory planning decoder specifically includes: based on a pre-trained environment encoder, obtaining environment coding vector samples according to the collected surrounding vehicle information samples and road information samples; constructing a training sample data set according to the collected navigation information samples, noise samples and real driving trajectories, and environment coding vector samples; taking the environment coding vector samples, navigation information samples, noisy time steps and the real driving trajectories after noisy as inputs, and taking the de-noised driving trajectories as outputs, iteratively optimizing the trajectory planning decoder through diffusion model loss and preset trajectory optimization loss; according to the pre-trained trajectory planning decoder and the preset trajectory optimization function, guiding and correcting the de-noised driving trajectory through a classifier; wherein the real driving trajectory includes the real driving trajectory of the own vehicle and the real driving trajectory of the surrounding vehicles.

[0012] Furthermore, when constructing a training sample data set, data augmentation processing is performed on the actual driving trajectory of the ego vehicle, specifically including: constructing a trajectory parameter equation using a quintic polynomial according to the position coordinates of the ego vehicle; determining the initial and final state boundary conditions of the actual driving trajectory of the ego vehicle, the initial and final state boundary conditions including the position, orientation angle, velocity, acceleration and angular velocity of the starting point, and the position, orientation angle, velocity, acceleration and angular velocity of the ending point; determining a coefficient equation group according to the initial and final state boundary conditions; solving the trajectory parameter equation according to the coefficient equation group to obtain an interpolated trajectory, which is the actual driving trajectory of the ego vehicle after data augmentation.

[0013] Furthermore, the preset trajectory optimization function includes a weighted combination of one or more of collision loss, out-of-bounds loss and comfort loss.

[0014] In a second aspect, the present invention also provides an automatic driving trajectory planning device based on a diffusion model, comprising: an information acquisition module for acquiring surrounding vehicle information and road information; a trajectory planning module for predicting the self-vehicle trajectory planning and the surrounding vehicle trajectory based on the surrounding vehicle information and road information based on a pre-trained trajectory planning model; wherein the trajectory planning model comprises an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on the diffusion model, and the trajectory planning model is trained and optimized based on a training sample set consisting of surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories.

[0015] In a third aspect, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, an autonomous driving trajectory planning method based on a diffusion model as described above is implemented.

[0016] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an autonomous driving trajectory planning method based on a diffusion model as described in any one of the above.

[0017] The automatic driving trajectory planning method based on the diffusion model provided by the present invention obtains surrounding vehicle information and road information, and predicts the trajectory planning of the self-vehicle and the surrounding vehicle trajectory based on the pre-trained trajectory planning model according to the surrounding vehicle information and road information; wherein the trajectory planning model includes an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on the diffusion model, and the trajectory planning model is trained and optimized based on a training sample set consisting of surrounding vehicle information samples, road information samples, navigation information samples, noise samples, and real driving trajectories. By applying the diffusion model to automatic driving, the method can accurately predict the automatic driving trajectory of the self-vehicle and surrounding vehicles according to real-time road driving conditions, which not only improves the safety of the automatic driving system, but also can predict and adapt to complex traffic scenarios, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flow chart of an autonomous driving trajectory planning method based on a diffusion model provided in an embodiment of the present invention.

[0020] Figure 2 It is a schematic diagram of the overall process of the autonomous driving trajectory planning method based on the diffusion model provided in an embodiment of the present invention.

[0021] Figure 3 It is a deployment diagram of the environment encoder provided by an embodiment of the present invention.

[0022] Figure 4 It is a deployment diagram of a trajectory planning decoder provided in an embodiment of the present invention.

[0023] Figure 5 It is a schematic diagram of training optimization of an environment encoder provided by an embodiment of the present invention.

[0024] Figure 6 1 is a schematic diagram of training optimization of a trajectory planning decoder provided in an embodiment of the present invention.

[0025] Figure 7 It is a schematic diagram of the process of collision trajectory optimization provided by an embodiment of the present invention.

[0026] Figure 8 It is a schematic diagram of the process of combined trajectory optimization provided by an embodiment of the present invention.

[0027] Fig. 9It is a schematic diagram of data augmentation of the actual driving trajectory of the vehicle provided by an embodiment of the present invention.

[0028] Fig.10 It is a structural schematic diagram of an automatic driving trajectory planning device based on a diffusion model provided in an embodiment of the present invention.

[0029] Fig.11 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] It should be noted that the existing autonomous driving trajectory planning methods rely on a clearly defined set of rules, which are not flexible enough and prone to rule conflicts, causing the system to fall into an uncertain state. In order to improve the flexibility and robustness of the autonomous driving system, some research works have tried to get rid of the constraints of rules through machine learning methods, but there is a problem of poor fitting ability for complex scene multimodal data distribution.

[0032] In view of this, the present invention proposes an automatic driving trajectory planning method based on a diffusion model. Specifically, Figure 1 A flow chart of an autonomous driving trajectory planning method based on a diffusion model provided in an embodiment of the present invention is shown.

[0033] like Figure 1 As shown, the method includes steps S110-S120, and steps S110-S120 and related steps will be described in detail below.

[0034] S110, obtaining surrounding vehicle information and road information.

[0035] It is easy to understand that in the field of autonomous driving, trajectory planning refers to specifying a safe and reasonable driving route for an autonomous vehicle to achieve a specific purpose, such as reaching a destination. This process usually involves a series of complex technologies and strategies, among which obtaining information about the surrounding environment is a very critical part.

[0036] In this step, it is necessary to obtain surrounding vehicle information and road information. Surrounding vehicle information refers to the historical information of surrounding vehicles within a set historical period, including but not limited to the position, orientation angle, speed, length, width and type of surrounding vehicles. Road information refers to various data and knowledge related to roads, including but not limited to geographic information, lane information, traffic signs, traffic signals, intersection information and traffic flow, etc.

[0037] Among them, geographic information includes the geographical location, length, width, slope, curvature and other basic physical characteristics of the road. Lane information includes the number of lanes, the location of lane lines, and the dividers between lanes (such as solid lines, dotted lines, central isolation strips, etc.). Traffic signs include speed limit signs, prohibition signs, warning signs, etc., which provide important information about road rules. Traffic signals include traffic lights, pedestrian crossing signals, etc., which affect the driving decisions of vehicles. Intersection information includes the number of intersections, intersection types (such as roundabouts, crossroads, etc.), and traffic rules (such as right turn, left turn, straight, etc.). Traffic flow includes vehicle density, congestion, accidents, etc., which affect driving speed and path selection.

[0038] In addition, road information may also include road facilities, road conditions and weather conditions. Road facilities include shoulders, curbs, guardrails, bridges, tunnels, etc. Road conditions include slippery, icy, potholes, construction areas, etc., which affect the driving safety of vehicles. Weather conditions include rain, snow, fog and other special weather conditions that may reduce visibility and affect driving decisions.

[0039] In a specific embodiment, the surrounding vehicle information includes the position, heading angle, speed, length, width and type of surrounding vehicles, and the road information includes road points, boundary points, traffic light information and speed limit information, which can be obtained through sensors such as lidar and cameras.

[0040] Based on the acquisition of surrounding vehicle information and road information in step S110, step S120 is further executed.

[0041] S120, based on a pre-trained trajectory planning model, predict the own vehicle trajectory planning and the surrounding vehicle trajectory according to the surrounding vehicle information and road information; wherein the trajectory planning model includes an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on a diffusion model, and the trajectory planning model is trained and optimized based on a training sample set consisting of surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories.

[0042] It is easy to understand that this embodiment can predict the trajectory of the self-driving vehicle (the autonomous driving vehicle itself) and surrounding vehicles (surrounding vehicles) based on the pre-trained trajectory planning model using surrounding vehicle information and road information. The trajectory planning model in this embodiment includes an environment encoder and a trajectory planning decoder. The environment encoder is used to process the input surrounding vehicle information and road information, and the trajectory planning decoder is used to generate a predicted trajectory plan.

[0043] Specifically, the environment encoder is used to encode the input surrounding vehicle information and road information, and convert the surrounding vehicle information and road information into a form that is more suitable for subsequent processing. For example, the environment encoder can encode data from sensors (such as lidar data and image data) into feature vectors representing the surrounding environment. These features may include information such as the vehicle's position, speed, orientation angle, size, and the geometry of the road, traffic signs, and other related elements. The trajectory planning decoder is used to receive the output of the environment encoder and combine other information, such as navigation information, noise information, etc., to generate the predicted self-vehicle trajectory planning and surrounding vehicle trajectory.

[0044] It is worth mentioning that the trajectory planning decoder in the trajectory planning model is built based on the diffusion model, which is a probabilistic model that can generate continuous and smooth trajectories. The diffusion model simulates the real trajectory distribution by gradually adding random noise, thereby obtaining more natural and diverse trajectory planning.

[0045] In order to train the trajectory planning model, it is necessary to prepare a training sample set containing surrounding vehicle information samples, road information samples, navigation information samples, noise samples, and real driving trajectories. These training samples can be collected from real-world data or generated through simulation, which is not specifically limited here.

[0046] During training, the model's predicted trajectory is compared with the actual driving trajectory through the back propagation algorithm, and the loss function (such as mean square error) is calculated. Then, the model parameters are updated using the gradient descent method to gradually fit the actual trajectory. This training process is iterated repeatedly until the model reaches the predetermined performance index.

[0047] After training, the trajectory planning model can predict the trajectory planning of the vehicle and surrounding vehicles based on the surrounding vehicle information and road information, which is crucial for decision-making such as avoidance and overtaking. By predicting the behavior of other vehicles, countermeasures can be taken in advance to prevent collisions.

[0048] It should be noted that the vehicle information obtained in this embodiment is historical, the road information is objective, and the predicted output of the vehicle trajectory planning and vehicle trajectory is future. As for the historical time and future time, they can be set according to actual needs and are not specifically limited here.

[0049] For example, in a specific embodiment, the historical moment is taken as 2 seconds in the past, and the future moment is taken as 8 seconds in the future.

[0050] in addition, Figure 2 FIG. 1 is a schematic diagram showing the overall process of the automatic driving trajectory planning method based on the diffusion model provided by an embodiment of the present invention. Figure 2 As shown, firstly, the surrounding vehicle information and road information are obtained, and then the surrounding vehicle information and road information are encoded using a pre-trained encoder, and the encoded output is used as one of the inputs of the trajectory planning decoder. Combined with other inputs (which will be expanded in detail in the embodiments below), the trajectory planning decoder can predict and output the own vehicle trajectory planning and the surrounding vehicle trajectory.

[0051] In this embodiment, by obtaining surrounding vehicle information and road information, and based on a pre-trained trajectory planning model, the trajectory planning of the self-vehicle and the surrounding vehicle trajectory are predicted according to the surrounding vehicle information and road information; wherein the trajectory planning model includes an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on a diffusion model, and the trajectory planning model is trained and optimized based on a training sample set consisting of surrounding vehicle information samples, road information samples, navigation information samples, noise samples, and real driving trajectories. By applying the diffusion model to autonomous driving, this method can accurately predict the autonomous driving trajectory of the self-vehicle and surrounding vehicles according to real-time road driving conditions, which not only improves the safety of the autonomous driving system, but also can predict and adapt to complex traffic scenarios, thereby improving the user experience.

[0052] On the basis of the above embodiments, the process of the trajectory planning model predicting the trajectory planning of the vehicle and the trajectory of the surrounding vehicle according to the surrounding vehicle information and the road information will be described in detail below.

[0053] Based on a pre-trained trajectory planning model, the ego vehicle trajectory planning and the surrounding vehicle trajectory are predicted according to the surrounding vehicle information and the road information, including: based on a pre-trained environment encoder, the surrounding vehicle information and the road information are encoded to obtain an environment encoding vector; based on a pre-trained trajectory planning decoder, the ego vehicle trajectory planning and the surrounding vehicle trajectory are predicted according to the environment encoding vector, current navigation information, Gaussian distribution noise and noise-added time step.

[0054] It is easy to understand that the process of the trajectory planning model processing the surrounding vehicle information and road information includes the processing of the environment encoder and the processing of the trajectory planning decoder.

[0055] Figure 3 A schematic diagram of the deployment of an environment encoder provided by an embodiment of the present invention is shown.

[0056] like Figure 3As shown, the environment encoder includes a first self-attention module, a second self-attention module, and a third self-attention module.

[0057] Correspondingly, based on the pre-trained environment encoder, the surrounding vehicle information and the road information are encoded and processed to obtain an environment encoding vector, including: inputting the surrounding vehicle information (including the information of A surrounding vehicles in T time steps) into the first self-attention module to obtain a surrounding vehicle feature vector; inputting the road information (including the information of N coordinate points in B lanes) into the second self-attention module to obtain a road feature vector; splicing the surrounding vehicle feature vector and the road feature vector, and inputting the spliced ​​feature vector into the third self-attention module to obtain an output environment encoding vector.

[0058] Further, Figure 4 A schematic diagram of the deployment of a trajectory planning decoder provided in an embodiment of the present invention is shown.

[0059] like Figure 4 As shown in Figure 1, the input of the trajectory planning decoder consists of four parts, namely, the information encoded by the environment encoder (i.e., the environment encoding vector, also known as Senario Condition), the current navigation information (i.e., Navigation Condition), and Gaussian distribution noise. x t And the noise time step t , the output of the trajectory planning decoder is the predicted ego-vehicle trajectory planning and surrounding vehicle trajectory at time step t.

[0060] The trajectory planning decoder includes LN (Layer Norm, layer normalization), scale and shift module (Scale andShift), self-attention mechanism module (Self Atten), cross attention mechanism module (Cross Attn), multi-head attention mechanism module (Multi-Head Attention), feedforward network module (FFN) and multi-layer perceptron (MLP).

[0061] Among them, the self-attention mechanism module is used to help the model focus on different parts of the input sequence and assign different weights according to the context. The output of the self-attention mechanism module will be scaled and LN.

[0062] The cross-attention module is used to fuse the output of the self-attention module with the information encoded by the environment encoder to obtain richer information. Similarly, the output of the cross-attention module will also be scaled and LN. In particular, Figure 4The structural details of the cross-attention mechanism module and the self-attention mechanism module are given in detail. In particular, both attention mechanism modules use multi-head attention for information fusion and use feedforward network (FFN) to further process the information.

[0063] The feed-forward network module is a fully connected network that is used to further process the outputs of the self-attention mechanism module and the cross-attention mechanism module.

[0064] The trajectory planning decoder is used for the information encoded by the environment encoder (i.e., the environment encoding vector), the current navigation information, and the Gaussian distribution noise. x t And the noise time step t The specific processing process can be found in Figure 4 , I will not expand on them one by one here.

[0065] In this embodiment, based on the pre-trained environment encoder, the surrounding vehicle information and road information are encoded to obtain the environment encoding vector, and based on the pre-trained trajectory planning decoder, the self-vehicle trajectory planning and the surrounding vehicle trajectory are predicted according to the environment encoding vector, current navigation information, Gaussian distribution noise and noise time step. By applying the diffusion model to autonomous driving, this method can accurately predict the autonomous driving trajectory of the self-vehicle and surrounding vehicles according to real-time road driving conditions, which not only improves the safety of the autonomous driving system, but also can predict and adapt to complex traffic scenarios, thereby improving the user experience.

[0066] On the basis of the above embodiments, the training optimization process of the environment encoder in the trajectory planning model will be described in detail below.

[0067] Training and optimizing the environment encoder specifically includes: collecting surrounding vehicle information samples and road information samples; taking the masked surrounding vehicle information samples and road information samples as input, taking the reconstructed surrounding vehicle vector and the reconstructed road vector as output, taking the difference between the reconstructed surrounding vehicle vector and the surrounding vehicle information sample, and the difference between the reconstructed road vector and the road information sample as training loss, and iteratively optimizing the environment encoder.

[0068] It is understandable that this embodiment uses a self-supervised training method to optimize the environment encoder. Specifically, Figure 5 A schematic diagram of training optimization of an environment encoder provided by an embodiment of the present invention is shown.

[0069] like Figure 5 As shown, the collected surrounding vehicle information samples and road information samples are masked, and then the masked surrounding vehicle information samples and the masked road information samples are used as two inputs of the training environment encoder respectively to obtain two corresponding output vectors.

[0070] Furthermore, the two output vectors are respectively passed through a linear network, which can be a fully connected layer or a multi-layer perceptron, for further compressing or converting the output vector of the environment encoder, thereby obtaining a reconstructed vehicle vector and a reconstructed road vector.

[0071] Then, the difference between the reconstructed surrounding vehicle vector and the surrounding vehicle information sample, as well as the difference between the reconstructed road vector and the road information sample are used as training losses. After multiple rounds of iterative optimization, the trained environment encoder can be obtained.

[0072] From this, it can be seen that the purpose of training and optimizing the environment encoder is to make the reconstructed surrounding vehicle vector output by the environment encoder closer to the actual surrounding vehicle information sample, and to make the reconstructed road vector output by the environment encoder closer to the actual road information sample.

[0073] It should be noted that the environment encoder provided in this embodiment can be constructed based on multiple convolutional layers and pooling layers, or it can be other network structures that can achieve the same function, which is not specifically limited here.

[0074] In this embodiment, by taking masked surrounding vehicle information samples and road information samples as input, reconstructing surrounding vehicle vectors and reconstructing road vectors as output, taking the difference between the reconstructed surrounding vehicle vector and the surrounding vehicle information sample, and the difference between the reconstructed road vector and the road information sample as training loss, iteratively optimizing the environment encoder, and then encoding the surrounding vehicle information and road information based on the pre-trained environment encoder to obtain the environment encoding vector, and based on the pre-trained trajectory planning decoder, according to the environment encoding vector, current navigation information, Gaussian distribution noise and noise-added time step, predicting the self-vehicle trajectory planning and the surrounding vehicle trajectory. By applying the diffusion model to autonomous driving, this method can accurately predict the autonomous driving trajectory of the self-vehicle and surrounding vehicles according to real-time road driving conditions, which not only improves the safety of the autonomous driving system, but also can predict and adapt to complex traffic scenes, thereby improving the user experience.

[0075] On the basis of the above embodiments, the training optimization process of the trajectory planning decoder in the trajectory planning model will be described in detail below.

[0076] The training and optimization trajectory planning decoder specifically includes: based on a pre-trained environment encoder, obtaining environment coding vector samples according to collected surrounding vehicle information samples and road information samples; constructing a training sample data set according to collected navigation information samples, noise samples and real driving trajectories, and environment coding vector samples; taking environment coding vector samples, navigation information samples, noisy time steps and real driving trajectories after noisy as inputs, and taking de-noised driving trajectories as outputs, iteratively optimizing the trajectory planning decoder through diffusion model loss; according to the pre-trained trajectory planning decoder and the preset trajectory optimization function, guiding and correcting the de-noised driving trajectory through a classifier; wherein the real driving trajectory includes the real driving trajectory of the own vehicle and the real driving trajectory of the surrounding vehicles.

[0077] Figure 6 A schematic diagram of training optimization of a trajectory planning decoder provided in an embodiment of the present invention is shown.

[0078] according to Figure 6 It can be seen that when training the trajectory planning decoder, its input includes four parts, namely, the information encoded by the environment encoder (i.e., the environment encoding vector sample, such as Figure 6 green dots in the image), navigation information samples (such as Figure 6 red dots in the figure), Gaussian noise x t (like Figure 6 The future trajectory of the ego vehicle and the surrounding vehicle after adding noise t steps) and the noise adding time step t , the output of the trajectory planning decoder is the predicted x t-1 , which is the driving trajectory after noise removal.

[0079] It should be noted that even the future trajectory of the ego vehicle and the surrounding vehicles after t steps of noise addition still need to give certain hints, such as the current state of the ego vehicle and the surrounding vehicles.

[0080] It should also be noted that the network structure of the trajectory planning decoder and the functions of related modules have been described in detail in the above embodiments, which will not be expanded here.

[0081] It is worth mentioning that the diffusion model loss is used when iteratively optimizing the trajectory planning decoder. Specifically, the diffusion model loss function expression corresponding to the diffusion model loss is as follows (1).

[0082] (1).

[0083] In formula (1), express, represents the future trajectory of the ego vehicle (real driving trajectory of the ego vehicle) and the future trajectory of the surrounding vehicle (real driving trajectory of the surrounding vehicle) in the dataset, represents Gaussian noise, represents the number of noise adding steps, represents the model output, It represents the future trajectory of the ego vehicle and the surrounding vehicles after adding noise t steps.

[0084] In addition, in a specific embodiment, based on the pre-training trajectory planning decoder through the diffusion model loss, a preset trajectory optimization function is used for the output planned trajectory (driving trajectory after denoising) to perform classifier-guided trajectory optimization, and the preset trajectory optimization function includes one or more weighted combinations of collision loss, out-of-bounds loss and comfort loss.

[0085] Taking the collision loss as an example, Figure 7 A schematic diagram of the process of collision trajectory optimization provided by an embodiment of the present invention is shown.

[0086] like Figure 7 As shown in the figure, after completing the pre-trained trajectory planning decoder, the de-noised driving trajectory output by the trajectory planning decoder is x t-1 , the real driving trajectory without noise is directly estimated based on the de-noised driving trajectory, and collision detection is performed based on this, and the collision loss is calculated. Then, the de-noising process of the trajectory planning decoder is corrected based on the collision loss using a classifier to reduce the probability of the model generating unsafe trajectories and improve safety performance.

[0087] Figure 7 The collision detection module in is used to perform collision detection according to the driving trajectory after noise removal. Specifically, the driving trajectory after noise removal includes the driving trajectory of the self-vehicle and the driving trajectory of the surrounding vehicles, thereby calculating the predicted distance between the self-vehicle and the surrounding vehicles. If the predicted distance is higher than the first distance threshold, it means that there is a collision risk between the self-vehicle and the surrounding vehicles, and the collision loss is large at this time; otherwise, it means that there is no collision risk between the self-vehicle and the surrounding vehicles, and the collision loss is small at this time. Of course, the collision risk here is not limited to the self-vehicle and the surrounding vehicles, but can also be the self-vehicle and other obstacles on the road, which is not specifically limited here.

[0088] The first distance threshold may be set according to actual conditions and is not specifically limited here.

[0089] Taking the preset trajectory optimization loss as a weighted combination of collision loss, out-of-bounds loss, and comfort loss as an example, Figure 8 A schematic diagram of the process of combined trajectory optimization provided by an embodiment of the present invention is shown.

[0090] like Figure 8 As shown, in addition to the collision loss, the preset trajectory optimization loss also includes out-of-bounds loss and comfort loss.

[0091] Correspondingly, the out-of-bounds loss is calculated by the out-of-bounds detection module, and the comfort loss is calculated by the comfort calculation module. The calculation process is the same as the calculation process of the collision loss. Specifically, the predicted distance between the vehicle and the road boundary can be obtained according to the de-noised driving trajectory. If the predicted distance is higher than the second distance threshold, it means that there is an out-of-bounds risk between the vehicle and the surrounding vehicles, and the out-of-bounds loss is large at this time; otherwise, it means that there is no out-of-bounds risk between the vehicle and the surrounding vehicles, and the out-of-bounds loss is small at this time.

[0092] Similarly, the comfort loss is calculated by the comfort calculation module. Specifically, the speed, acceleration and angular velocity of the vehicle are obtained, and the speed, acceleration and angular velocity of the vehicle are compared with the corresponding thresholds. If they are higher than the thresholds, it means that the vehicle is uncomfortable and the comfort loss is large. Otherwise, it means that the vehicle is relatively comfortable and the comfort loss is small.

[0093] The second distance threshold may be set according to actual conditions and is not specifically limited here.

[0094] After calculating the collision loss, out-of-bounds loss, and comfort loss, different weights are assigned to each loss according to its importance for weighted combination. Then, the combined loss is back-propagated with the diffusion model loss to minimize the combined loss and the diffusion model loss, thereby achieving the effect of updating and optimizing the model parameters.

[0095] In this embodiment, by taking the environment coding vector samples, navigation information samples, noise time steps and the real driving trajectory after noise as input, and taking the de-noised driving trajectory as output, the trajectory planning decoder is iteratively optimized through the diffusion model loss and the preset trajectory optimization loss, and then based on the pre-trained trajectory planning decoder, according to the environment coding vector, current navigation information, Gaussian distribution noise and noise time steps, the self-vehicle trajectory planning and the surrounding vehicle trajectory are predicted. By applying the diffusion model to autonomous driving, this method can accurately predict the autonomous driving trajectory of the self-vehicle and the surrounding vehicles according to the real-time road driving conditions, which not only improves the safety of the autonomous driving system, but also can predict and adapt to complex traffic scenes, thereby improving the user experience.

[0096] In a specific embodiment, when constructing the training sample data set, data augmentation processing is performed on the actual driving trajectory of the vehicle.

[0097] Specifically, according to the position coordinates of the ego vehicle, a trajectory parameter equation is constructed using a quintic polynomial; the initial and final state boundary conditions of the actual driving trajectory of the ego vehicle are determined, and the initial and final state boundary conditions include the position, orientation angle, velocity, acceleration and angular velocity of the starting point, and the position, orientation angle, velocity, acceleration and angular velocity of the ending point; according to the initial and final state boundary conditions, a group of coefficient equations is determined; according to the group of coefficient equations, the trajectory parameter equation is solved to obtain an interpolated trajectory, which is the actual driving trajectory of the ego vehicle after data augmentation.

[0098] First, according to the position coordinates of the ego vehicle, a quintic polynomial is used to construct the trajectory parameter equation. The trajectory parameter equation is as follows (2)-(3).

[0099] (2).

[0100] (3).

[0101] In formulas (2)-(3), , Indicates the current position coordinates of the vehicle, , , , , , is the coefficient, , , , , , is the coefficient.

[0102] Then, the initial and final state boundary conditions of the actual driving trajectory of the vehicle are determined, and the coefficient equation group is determined based on the initial and final state boundary conditions. The coefficient equation group can be specifically referred to in the following equations (4)-(9).

[0103] (4).

[0104] (5).

[0105] (6).

[0106] (7).

[0107] (8).

[0108] (9).

[0109] In equations (4)-(9), , Indicates the position coordinates of the starting point 0, Indicates the velocity at the starting point 0, Indicates the orientation angle of the starting point 0, represents the acceleration at the starting point 0, represents the angular velocity of the starting point 0, , Represents the position coordinates of the end point T, represents the speed of the end point T, represents the orientation angle of the end point T, represents the acceleration of the end point T, represents the angular velocity of the end point T.

[0110] According to the coefficient equations (4)-(9), solving the trajectory parameter equations (2)-(3) can obtain a curve, namely the interpolated trajectory, which is also the actual driving trajectory of the vehicle after data augmentation. The interpolated trajectory represents the trajectory of the vehicle from the starting position to the target position.

[0111] Fig. 9 FIG. 2 shows a schematic diagram of data augmentation of the actual driving trajectory of the vehicle provided by an embodiment of the present invention. Fig. 9 As shown in Figure 1, the blue curve represents the actual driving trajectory of the vehicle, while the yellow curve represents the interpolated trajectory obtained by polynomial interpolation. The yellow curve can be adjusted according to the actual situation to adapt to the new target position and direction.

[0112] In this embodiment, the trajectory parameter equation is constructed using a quintic polynomial according to the position coordinates of the vehicle, and the initial and final state boundary conditions of the actual driving trajectory of the vehicle are determined, and then the coefficient equation group is determined according to the initial and final state boundary conditions, so that the trajectory parameter equation is solved according to the coefficient equation group to obtain the interpolated trajectory, which is the actual driving trajectory of the vehicle after data augmentation. This method expands the scale of the data set by augmenting the actual driving trajectory of the vehicle when constructing the training sample data set, so that the trajectory planning decoder can learn more diverse driving behaviors, and improve the generalization ability of the model and the ability to cope with unknown situations.

[0113] Corresponding to the automatic driving trajectory planning method based on the diffusion model described in the above embodiments, the present invention also proposes an automatic driving trajectory planning device based on the diffusion model. Specifically, Fig.10 A schematic diagram of the structure of an automatic driving trajectory planning device based on a diffusion model provided in an embodiment of the present invention is shown.

[0114] like Fig.10As shown, the device includes: an information acquisition module 1010, used to obtain surrounding vehicle information and road information; a trajectory planning module 1020, used to predict the self-vehicle trajectory planning and the surrounding vehicle trajectory based on the surrounding vehicle information and road information based on a pre-trained trajectory planning model; wherein the trajectory planning model includes an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on a diffusion model, and the trajectory planning model is trained and optimized based on a training sample set consisting of surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories.

[0115] In this embodiment, the surrounding vehicle information and road information are obtained through the information acquisition module 1010, and the trajectory planning module 1020 predicts the trajectory planning of the self-vehicle and the surrounding vehicle trajectory based on the pre-trained trajectory planning model according to the surrounding vehicle information and road information; wherein the trajectory planning model includes an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on the diffusion model, and the trajectory planning model is trained and optimized based on the training sample set consisting of surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories. By applying the diffusion model to automatic driving, the device can accurately predict the automatic driving trajectory of the self-vehicle and surrounding vehicles according to the real-time road driving conditions, which not only improves the safety of the automatic driving system, but also can predict and adapt to complex traffic scenes, thereby improving the user experience.

[0116] It should be noted that the autonomous driving trajectory planning device based on the diffusion model provided in the embodiment of the present invention and the autonomous driving trajectory planning method based on the diffusion model described in the above embodiments can be referred to each other, and will not be described in detail here.

[0117] Fig.11 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.11As shown, the electronic device may include: a processor 1110, a communication interface 1120, a memory 1130 and a communication bus 1140, wherein the processor 1110, the communication interface 1120 and the memory 1130 communicate with each other through the communication bus 1140. The processor 1110 may call the logic instructions in the memory 1130 to execute the automatic driving trajectory planning method based on the diffusion model, the method comprising: obtaining surrounding vehicle information and road information; based on the pre-trained trajectory planning model, predicting the self-vehicle trajectory planning and the surrounding vehicle trajectory according to the surrounding vehicle information and road information; wherein the trajectory planning model comprises an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on the diffusion model, and the trajectory planning model is trained and optimized based on a training sample set consisting of surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories.

[0118] In addition, the logic instructions in the above-mentioned memory 1130 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0119] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the automatic driving trajectory planning method based on the diffusion model provided by the above-mentioned methods, the method comprising: obtaining surrounding vehicle information and road information; based on a pre-trained trajectory planning model, predicting the self-vehicle trajectory planning and the surrounding vehicle trajectory according to the surrounding vehicle information and road information; wherein the trajectory planning model comprises an environment encoder and a trajectory planning decoder, the trajectory planning decoder is constructed based on the diffusion model, and the trajectory planning model is trained and optimized based on a training sample set consisting of surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories.

[0120] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0121] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic driving trajectory planning method based on a diffusion model, characterized in that: include: Get weekly vehicle information and road information; Based on the pre-trained trajectory planning model, predict the trajectory planning of the ego vehicle and the trajectory of the surrounding vehicles according to the surrounding vehicle information and road information; Among them, the trajectory planning model includes an environment encoder and a trajectory planning decoder. The trajectory planning decoder is constructed based on a diffusion model. The trajectory planning model is trained and optimized based on a training sample set consisting of surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories.

2. The automatic driving trajectory planning method based on the diffusion model according to claim 1, characterized in that: The trajectory planning model based on the pre-training predicts the trajectory planning of the vehicle and the trajectory of the surrounding vehicle according to the surrounding vehicle information and the road information, including: Based on a pre-trained environment encoder, the surrounding vehicle information and the road information are encoded to obtain an environment encoding vector; Based on the pre-trained trajectory planning decoder, the ego vehicle trajectory planning and the surrounding vehicle trajectory are predicted according to the environment encoding vector, the current navigation information, the Gaussian distribution noise and the noise added time step.

3. The automatic driving trajectory planning method based on the diffusion model according to claim 2, characterized in that: The environment encoder includes a first self-attention module, a second self-attention module and a third self-attention module; Correspondingly, the pre-trained environment encoder encodes the surrounding vehicle information and road information to obtain an environment encoding vector, including: Inputting the surrounding vehicle information into the first self-attention module to obtain a surrounding vehicle feature vector; Inputting the road information into a second self-attention module to obtain a road feature vector; The surrounding vehicle feature vector and the road feature vector are concatenated, and the concatenated feature vector is input into the third self-attention module to obtain the output environment encoding vector.

4. The automatic driving trajectory planning method based on the diffusion model according to claim 1, characterized in that: Train and optimize the environment encoder, including: Collecting samples of vehicle information and road information; The environment encoder is iteratively optimized using masked surrounding vehicle information samples and road information samples as input, reconstructed surrounding vehicle vectors and reconstructed road vectors as output, and the difference between the reconstructed surrounding vehicle vector and the surrounding vehicle information samples, and the difference between the reconstructed road vector and the road information samples as training losses.

5. The automatic driving trajectory planning method based on the diffusion model according to claim 4 is characterized in that: Train and optimize the trajectory planning decoder, including: Based on the pre-trained environment encoder, an environment encoding vector sample is obtained according to the collected surrounding vehicle information samples and road information samples; A training sample data set is constructed based on the collected navigation information samples, noise samples, real driving trajectories, and environment coding vector samples; Taking environment coding vector samples, navigation information samples, noisy time steps and real driving trajectories after noisy as inputs, and taking de-noised driving trajectories as outputs, the trajectory planning decoder is iteratively optimized through diffusion model loss; According to the pre-trained trajectory planning decoder and the preset trajectory optimization function, the de-noised driving trajectory is corrected through the classifier guidance; The real driving trajectory includes the real driving trajectory of the own vehicle and the real driving trajectory of surrounding vehicles.

6. The automatic driving trajectory planning method based on the diffusion model according to claim 5, characterized in that: When constructing the training sample data set, data augmentation is performed on the actual driving trajectory of the vehicle, including: According to the position coordinates of the ego vehicle, the trajectory parameter equation is constructed using a quintic polynomial; Determine the initial and final state boundary conditions of the actual driving trajectory of the vehicle, wherein the initial and final state boundary conditions include the position, orientation angle, speed, acceleration and angular velocity of the starting point, and the position, orientation angle, speed, acceleration and angular velocity of the ending point; Determine the coefficient equation group according to the initial and final state boundary conditions; According to the coefficient equation group, the trajectory parameter equation is solved to obtain an interpolated trajectory, where the interpolated trajectory is the actual driving trajectory of the vehicle after data augmentation.

7. The automatic driving trajectory planning method based on the diffusion model according to claim 5, characterized in that: The preset trajectory optimization function includes a weighted combination of one or more of collision loss, out-of-bounds loss and comfort loss.

8. An automatic driving trajectory planning device based on a diffusion model, characterized in that: include: Information acquisition module, used to obtain surrounding vehicle information and road information; A trajectory planning module, used for predicting the trajectory planning of the vehicle and the trajectory of the surrounding vehicles based on the pre-trained trajectory planning model and the surrounding vehicle information and the road information; Among them, the trajectory planning model includes an environment encoder and a trajectory planning decoder. The trajectory planning decoder is constructed based on a diffusion model. The trajectory planning model is trained and optimized based on a training sample set consisting of surrounding vehicle information samples, road information samples, navigation information samples, noise samples and real driving trajectories.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the autonomous driving trajectory planning method based on the diffusion model according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the autonomous driving trajectory planning method based on a diffusion model as described in any one of claims 1 to 7 is implemented.

Images