An automatic driving car diffusion navigation method based on visual language model guidance

Abstract

The application discloses a kind of automatic driving car diffusion navigation methods based on visual language model guide, it is related to the field of automatic driving, including: obtaining the image sequence and natural language instruction of car navigation, the semantic color block diagram of each frame is obtained by carrying out semantic mapping to each frame image in image sequence;The semantic color block diagram is spliced to form a time continuous semantic graph sequence;Through convolutional neural network, spatial structure feature extraction and splicing are carried out to semantic graph sequence, then encoding is carried out through Transformer network, and spatiotemporal fusion feature is obtained;Spatiotemporal fusion feature is used as conditional embedding vector, input pre-trained conditional denoising diffusion probability model to carry out trajectory prediction, according to the output trajectory point sequence, vehicle is navigated control.The application fuses image statistics priori and language prompt to generate semantic region map, guides diffusion model to generate future trajectory in line with instruction semantics, realizes end-to-end language driven control.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, more particularly to an automatic driving car diffusion navigation method based on visual language model guidance. BACKGROUND

[0002] In the automatic driving navigation of a car, especially in the implementation of natural language guided autonomous navigation in a semi-structured environment, it is of great significance to improve the naturalness of human-computer interaction, reduce the dependence on high-precision maps, and enhance the adaptability of the system to complex environments.

[0003] Existing methods usually separate the language understanding and trajectory generation modules, and the language information usually only acts on the perception stage, failing to achieve integrated modeling of cross-modal semantics. At the same time, the trajectory prediction process responds to the dynamic changes of language and environmental state with a lag, making it difficult to achieve high consistency with task semantics. In addition, natural language expressions themselves have ambiguity, which makes it uncertain when aligning with the spatial semantic regions in the image, and trajectory control has strict requirements for real-time performance, further exacerbating the coupling difficulty between language understanding and motion decision-making.

[0004] Therefore, how to generate a future trajectory that conforms to the semantic of the instruction based on image and language information for navigation is a problem that needs to be solved by those skilled in the art. SUMMARY

[0005] Therefore, the present application provides an automatic driving car diffusion navigation method based on visual language model guidance, aiming to accurately analyze high-level semantic intent from language instructions and generate navigation trajectories that are consistent with semantics and have dynamic feasibility in combination with the information of the perception scene.

[0006] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0007] The present application discloses an automatic driving car diffusion navigation method based on visual language model guidance, comprising:

[0008] Step 1: obtaining an image sequence and a natural language instruction for car navigation, and performing semantic mapping on each frame of image in the image sequence to obtain a semantic color block diagram of each frame;

[0009] Step 2: concatenating the semantic color block diagrams to form a time-continuous semantic diagram sequence;

[0010] Step 3: performing spatial structure feature extraction and concatenation on the semantic diagram sequence through a convolutional neural network, and then encoding through a Transformer network to obtain spatio-temporal fusion features;

[0011] Step 4: input the spatio-temporal fusion feature as a conditional embedding vector into a pre-trained conditional denoising diffusion probability model for trajectory prediction, and control the vehicle according to the output trajectory point sequence.

[0012] Further, the step 1 specifically comprises:

[0013] Step 1.1: for any frame of original image, extract high-dimensional feature map through visual encoder;

[0014] Step 1.2: for all pixels in each channel of the high-dimensional feature map, calculate the mean and variance, and splice the mean and variance of all channels to obtain the statistical feature vector of the high-dimensional feature map;

[0015] Step 1.3: project the statistical feature vector to a vector space with the same dimension as the text embedding through a nonlinear projection network to obtain a semantic prompt vector;

[0016] Step 1.4: splice the natural language instruction and the semantic prompt vector to obtain an enhanced language input;

[0017] Step 1.5: input the enhanced language input and the original image into a visual language model, and map the semantics to obtain a semantic color block map;

[0018] Step 1.6: repeat steps 1.1 to 1.5 to obtain semantic color block maps of each frame.

[0019] Further, the step 3 specifically comprises:

[0020] Step 3.1: for each frame of semantic color block map in the semantic map sequence, extract spatial structure features through a convolutional neural network to obtain embedded tensors after region division of each frame, and encode into a spatial feature sequence;

[0021] Step 3.2: splice the spatial feature sequence in the time dimension to obtain a spatio-temporal feature sequence;

[0022] Step 3.3: input the spatio-temporal feature sequence after introducing position encoding into the Transformer network, and after L-layer Transformer network coding, obtain the spatio-temporal fusion feature.

[0023] Further, the Transformer network is a multi-head self-attention Transformer network stacked by L layers; the multi-head attention mechanism in the first layer Transformer network is represented as:

[0024] ;

[0025] wherein, is the output of the layer multi-head attention mechanism, denotes a self-attention operation function, denotes the output of the layer Transformer network, denotes a concatenation operation, denotes an output projection matrix, is the total number of attention heads, is the output of the th attention head, and the calculation formula is:

[0026] ;

[0027] wherein, is a normalized attention weight function, is a query vector, is a query weight matrix; is a key vector, is a key weight matrix; is a value vector, is a value weight matrix;

[0028] In the layer Transformer network, the residual connection and the feedforward network are represented as:

[0029] ;

[0030] ;

[0031] wherein, is a layer normalization function, denotes a feedforward neural network, is an intermediate quantity, denotes the output of the layer Transformer network.

[0032] Further, the conditional denoising diffusion probability model is obtained by training in the following manner:

[0033] Obtain a plurality of real driving trajectories and navigation image sequence data of automobiles, and determine a 0-point time and a diffusion time step;

[0034] Based on the image sequence data within the length of time before the 0-point time, the conditional embedding vector of each sample is obtained by processing using the method of steps 1 to 3;

[0035] Based on the image sequence data within the The driving trajectory data within the time length is used to construct a noisy sample of each sample by adding Gaussian noise;

[0036] The condition embedding vector and the noisy sample of the trajectory are used as a training sample, and the noise prediction network is trained and learned by using a plurality of training samples, and a denoising model is obtained according to the trained noise prediction network.

[0037] Further, in the training process, the noise prediction network takes the minimum mean square error of the predicted noise and the real added noise as the loss function, and the formula is:

[0038] ;

[0039] Among them, is a denoising loss function, is a joint distribution expectation, is a driving trajectory sequence after 0 point, is a diffusion time step, is a real noise, is a predicted noise, is a noisy sample, is a condition embedding vector.

[0040] Further, the conditional denoising diffusion probability model performs trajectory prediction, specifically: taking Gaussian noise as a sampling initial trajectory, taking the condition embedding vector of the current vehicle as a condition, using the denoising model to perform reverse denoising according to the diffusion time step, and obtaining a predicted trajectory;

[0041] In the reverse denoising, the standard sampling update is performed times of step-by-step updates, and the calculation formula of each step is as follows:

[0042] ;

[0043] Among them, and are the retention ratio and the cumulative retention ratio at , respectively, controls the noise residual intensity, is the resampling noise.

[0044] Further, in the reverse denoising, a jump strategy is introduced to accelerate the sampling update and improve efficiency; in the accelerated sampling update, only times of inference updates are performed on one sub-sampling sequence , and the specific sampling update formula is as follows:

[0045] ;

[0046] Among them, Control sampling randomness, For Noise intensity coefficient, For sampling randomness control coefficient, Indicate the noise prediction network.

[0047] Through the above technical solutions, compared with the prior art, the application provides an automatic driving car diffusion navigation method based on visual language model guidance, which has the following technical effects:

[0048] 1. The application introduces an image statistical guidance mechanism, enhances the structural alignment capability between language and image through semantic statistical prompt vectors, improves the analysis accuracy of natural language instructions in complex scenes, and realizes more stable and controllable semantic guidance.

[0049] 2. The application models the semantic evolution process by constructing a historical multi-frame semantic region map sequence and introducing a space-time Transformer model, realizes dynamic perception of scene changes and language context, and generates trajectories that meet semantic constraints by embedding diffusion models as conditions, which has good continuity, diversity and adjustability.

[0050] 3. The application adopts a skip sampling strategy in the reasoning stage, which significantly reduces the sampling cost while ensuring the quality of the trajectory, and meets the requirements of real-time and low delay for vehicle control. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only embodiments of the application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0052] Figure 1 The overall flowchart provided by the application;

[0053] Figure 2 The semantic color block sequence generation framework based on visual language model is shown in the figure;

[0054] Figure 3 The conditional embedding acquisition process of space-time fusion features is shown in the figure;

[0055] Figure 4 The conditional denoising diffusion model training process is shown in the figure;

[0056] Figure 5 The future trajectory reasoning and control output process based on diffusion model is shown in the figure. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0058] The embodiments of the present application disclose an automatic driving car diffusion navigation method based on visual language model guidance, as shown in the formula (I), comprising: Figure 1

[0059] Step 1: obtaining an image sequence of car navigation and a natural language instruction, performing semantic mapping on each frame image in the image sequence to obtain a semantic color block diagram of each frame;

[0060] Step 2: splicing the semantic color block diagram to form a time-continuous semantic diagram sequence;

[0061] Step 3: performing spatial structure feature extraction and splicing on the semantic diagram sequence through a convolutional neural network, and then performing encoding through a Transformer network to obtain a spatio-temporal fusion feature;

[0062] Step 4: taking the spatio-temporal fusion feature as a conditional embedding vector, inputting a pre-trained conditional denoising diffusion probability model to perform trajectory prediction, and performing navigation control on the vehicle according to the output trajectory point sequence.

[0063] In a specific embodiment, step 1 specifically comprises:

[0064] Step 1.1: for any original image frame, extracting a high-dimensional feature map through a visual encoder;

[0065] Step 1.2: for all pixels in each channel of the high-dimensional feature map, calculating the mean and variance, and splicing the mean and variance of all channels to obtain a statistical feature vector of the high-dimensional feature map;

[0066] Step 1.3: projecting the statistical feature vector to a vector space with the same dimension as the text embedding through a nonlinear projection network to obtain a semantic prompt vector;

[0067] Step 1.4: splicing the natural language instruction and the semantic prompt vector to obtain an enhanced language input;

[0068] Step 1.5: inputting the enhanced language input and the original image into a visual language model to obtain a semantic color block diagram through semantic mapping;

[0069] Step 1.6: repeating steps 1.1 to 1.5 to obtain a semantic color block diagram of each frame.

[0070] ​Specifically, the application mainly relies on the visual perception module to extract environmental information, such as Figure 2 As shown in the figure, first, the image structure features are encoded into the language guide by statistical analysis of the image visual feature map, thereby enhancing the guiding ability of language to the visual scene.

[0071] At the time step , the original image from the front-view camera is acquired and recorded as , and are the height and width of the original image; the high-dimensional feature map of the image is extracted using a visual encoder , wherein represents the number of channels, and are the height and width of the feature map.

[0072] For the feature map , the mean value and the variance of all pixels in each channel are calculated, and the formula is as follows:

[0073] ;

[0074] ;

[0075] The mean values and variances of all channels are spliced into a statistical feature vector , which is represented as:

[0076] ;

[0077] The application designs a learnable nonlinear projection network to project the statistical vector to a vector space with the same dimension as the text embedding, thereby obtaining a semantic prompt vector , and the formula is as follows: The nonlinear projection network is composed of a group of fully connected layers and has a nonlinear activation function to enhance the expression ability. The text embedding refers to a high-dimensional vector representation of the semantic coding of the natural language instruction, which is used to represent the language semantic information in the vector space. The text embedding is extracted by a language encoder (such as Transformer) and is used for alignment with the visual features to realize multi-modal fusion with consistent semantics.

[0078] The original natural language instruction and the statistical embedding are spliced to form an enhanced language input , which is represented as: .

[0079] The following semantic mapping is performed using a visual language model: ;in, The output single-channel semantic color block map is labeled with the passable area indicated by the current language instruction, visualizing it as a semantically coherent target region. Each pixel value represents the probability that the location belongs to the semantic target in the natural language instruction. The visual language model adopts a multimodal joint encoding framework (such as CLIPSeg), and calculates the pixel-level semantic similarity distribution by aligning visual features with language features, thereby outputting the semantic color block map.

[0080] At time step At any given moment, the current frame and its preceding frames will be used. Image sequence of frames Combined with language instructions This model establishes the evolutionary relationship between language and visual semantics over time. Semantic mapping is performed on all input image frames, and these frames are then concatenated to form a temporally continuous semantic graph sequence. ;sequence Reflecting at any moment arrive During this period, the semantic map of the frame-by-frame matching results between language commands and the environment in the scene observed by the vehicle maintains the same resolution as the input image in space.

[0081] In one specific embodiment, step 3 specifically includes:

[0082] Step 3.1: For each frame of semantic color block image in the semantic graph sequence, spatial structure features are extracted using a convolutional neural network to obtain the embedding tensor after region division of each frame, and encoded into a spatial feature sequence;

[0083] Step 3.2: Concatenate the spatial feature sequences along the time dimension to obtain the spatiotemporal feature sequences;

[0084] Step 3.3: After introducing position encoding into the spatiotemporal feature sequence, input it into the Transformer network. After encoding by the L-layer Transformer network, the spatiotemporal fusion feature is obtained.

[0085] Specifically, such as Figure 3 As shown, the conditional input of the diffusion model is obtained through the sequence of color block semantic maps. Spatial features are extracted from the semantic map of each frame, and the sequence is then processed. Semantic graph of each frame The input is fed into a lightweight convolutional neural network for spatial structure feature extraction, yielding the embedding tensor after region segmentation: ;in, It is a convolutional neural network. The number of regions to divide the semantic graph. For each region's feature dimensions, each of represents the frame the semantic embedding of the region.

[0086] After convolutional neural network encoding, the spatial feature sequence of the moment frame is obtained: The region features of all frames are spliced in the time dimension to form a spatio-temporal fusion feature tensor: ; wherein, is the spatio-temporal feature sequence, is a splicing function, is the length of the time frame.

[0087] To maintain the spatio-temporal order information, position encoding is introduced: ; wherein fuses the embedding of the time frame position and the spatial block position, and is realized by sinusoidal encoding. Input into a multi-head self-attention Transformer network stacked by layers to model spatio-temporal interaction.

[0088] In one specific embodiment, the Transformer network is a multi-head self-attention Transformer network stacked by layers; the layer Transformer network is represented as:

[0089] ;

[0090] wherein, is the output result of the layer multi-head attention mechanism, represents a self-attention operation function, represents the output of the layer Transformer network, represents a splicing operation, represents an output projection matrix, is the total number of attention heads, is the output of the attention head, and the calculation formula is:

[0091] ;

[0092] wherein, is a normalized attention weight function, is a query vector, is a query weight matrix; is a key vector, is a key weight matrix; is a value vector, is a value weight matrix;

[0093] The first In the layer Transformer network, the residual connection and the feedforward network are represented as:

[0094] ;

[0095] ;

[0096] wherein, is a layer normalization function, denotes a feedforward neural network, is an intermediate quantity, denotes the output of the layer Transformer network. After the layer Transformer encoding, the final spatio-temporal semantic fusion feature is obtained:

[0097] ; the vector is the conditional embedding representation of the diffusion model trajectory generation process. In one specific embodiment, the conditional denoising diffusion probability model is obtained by training in the following way:

[0098] Obtain a plurality of real driving trajectory and navigation image sequence data of a car, and determine the 0 point time and the diffusion time step;

[0099] Based on the image sequence data within the time length before the 0 point time, the conditional embedding vector of each sample is obtained by using the method of steps 1 to 3;

[0100] Based on the driving trajectory data within the time length after the 0 point time, the noisy trajectory sample of each sample is constructed by adding Gaussian noise; Use the conditional embedding vector and the noisy trajectory sample as a training sample, use a plurality of training samples to train and learn the noise prediction network, and obtain the denoising model according to the trained noise prediction network.

[0101] Specifically, the present application introduces a conditional denoising diffusion probability model to predict the future trajectory distribution, as shown in

[0102] First, a forward process from data distribution to Gaussian distribution is established by gradually adding Gaussian noise to disturb the trajectory. The target prediction trajectory (i.e. based on the time length after the 0 point time) is defined as

[0103] Specifically, the present application introduces a conditional denoising diffusion probability model to predict the future trajectory distribution, as shown in Figure 4 First, a forward process from data distribution to Gaussian distribution is established by gradually adding Gaussian noise to disturb the trajectory. The target prediction trajectory (i.e. based on the time length after the 0 point time) is defined as ​​The driving trajectory data within a given time period is a sequence of two-dimensional coordinate points: ;in, Indicates the vehicle's future... The two-dimensional planar position of the step. Define the discrete-time step. , for trajectory Adding Gaussian noise to create noisy samples The following diffusion process is satisfied: ;in, For conditional probability distribution, It follows a Gaussian distribution. The mean, For covariance, The noise intensity coefficient, It is an identity matrix. To simplify sampling, a specific moment is constructed directly from the original trajectory. Noisy samples: ;in, , Then, a conditional reverse process is learned to gradually reconstruct the true trajectory.

[0104] The goal of training is to learn a parameterized neural network. Embedded under given conditions In this case, noise on the predicted trajectory , represented as: ;in, To predict noise values, For noise prediction networks, the parameters are: This network is implemented based on the UNet architecture, and receives the current diffusion step as input. Current diffusion time step and conditional semantic representation Noise estimation with output and input of the same dimension .

[0105] In one specific embodiment, during training, the noise prediction network uses the minimum mean square error between the predicted noise and the actual added noise as its loss function, as shown in the formula:

[0106] ;

[0107] in, Let be the denoising loss function. For the joint distribution expectation, This is the sequence of driving trajectories after 0:00. For diffusion time step, This is real noise. To predict noise, For noisy samples, , To condition embedding vectors, keep corresponding to target trajectories during training.

[0108] In one specific embodiment, after the model training is completed, the learned conditional denoising diffusion probability model is used for trajectory prediction. As shown in Figure 5 , the process starts from a standard Gaussian noise trajectory, and under the guidance of the conditional semantic representation , a reverse denoising process is performed step by step. The initial trajectory is sampled from a standard normal distribution , which represents the trajectory under the strongest noise condition, and then the real trajectory is gradually restored through the trained denoising network. In the reverse denoising process of the diffusion model, it starts from a pure Gaussian noise, and the diffusion model training is essentially training a denoising model to gradually denoise the pure Gaussian noise according to the conditional input, and finally restore the desired data, so the initial trajectory is a random trajectory.

[0109] The conditional denoising diffusion probability model is used for trajectory prediction, specifically: taking Gaussian noise as the sampling initial trajectory, taking the current vehicle's condition embedding vector as the condition, using the denoising model to perform reverse denoising according to the diffusion time step, and obtaining the predicted trajectory;

[0110] In reverse denoising, standard sampling update is performed times step-by-step update, and the calculation formula of each step is as follows:

[0111] ;

[0112] wherein, and are the retention ratio and cumulative retention ratio at , respectively, controls the noise residual intensity, is the resampling noise.

[0113] In one specific embodiment, in reverse denoising, a jump strategy is introduced to accelerate sampling update and improve efficiency; in accelerated sampling update, only inference updates are performed on a sub-sampling sequence , and the specific sampling update formula is as follows:

[0114] ;

[0115] wherein, controls the sampling randomness, is the noise intensity coefficient at , is the sampling randomness control coefficient, denotes the noise prediction network. In After being generated, the complete trajectory point sequence is obtained under the condition

[0116]

[0117] The trajectory will be decoded by the downstream control module as a vehicle low-level control variable signal, realizing consistent real-time navigation control of the instruction.

[0118] The various embodiments described in the specification are progressive in nature, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be mutually referred to. For the apparatus disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0119] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.​​

Claims

1.A method for diffusion navigation of an autonomous vehicle based on visual language model guidance, characterized in that, The application relates to a method for vehicle navigation based on natural language instructions and image sequences. The method comprises the following steps: Step 1: obtaining an image sequence of vehicle navigation and a natural language instruction, and performing semantic mapping on each frame of the image sequence to obtain a semantic color block diagram of each frame; Step 2: splicing the semantic color block diagrams to obtain a time-continuous semantic diagram sequence; Step 3: performing spatial structure feature extraction and splicing on the semantic diagram sequence through a convolutional neural network, and then performing coding through a Transformer network to obtain a spatio-temporal fusion feature; Step 4: inputting the spatio-temporal fusion feature as a conditional embedding vector into a pre-trained conditional denoising diffusion probability model to perform trajectory prediction, and performing navigation control on the vehicle according to an output trajectory point sequence. The step 1 specifically comprises the following steps: Step 1.1: extracting a high-dimensional feature map through a visual encoder for any original frame image; Step 1.2: calculating the mean and variance of all pixels in each channel of the high-dimensional feature map, and splicing the mean and variance of all channels to obtain a statistical feature vector of the high-dimensional feature map; Step 1.3: projecting the statistical feature vector into a vector space with the same dimension as a text embedding through a nonlinear projection network to obtain a semantic prompt vector; Step 1.4: splicing the natural language instruction and the semantic prompt vector to obtain an enhanced language input; Step 1.5: inputting the enhanced language input and the original image into a visual language model to perform semantic mapping and obtain a semantic color block diagram; 2.The diffusion navigation method for an automatic driving vehicle based on a visual language model guidance according to claim 1, wherein, Step 1.6: repeating steps 1.1 to 1.5 to obtain a semantic color block diagram of each frame. The step 3 specifically comprises the following steps: Step 3.1: performing spatial structure feature extraction on each frame of the semantic color block diagram in the semantic diagram sequence through a convolutional neural network to obtain an embedded tensor after regional division of each frame, and coding the embedded tensor into a spatial feature sequence; Step 3.2: splicing the spatial feature sequence in the time dimension to obtain a spatio-temporal feature sequence; 3.The diffusion navigation method for an automatic driving car based on a visual language model guidance according to claim 2, wherein, The Transformer network is a multi-head self-attention Transformer network stacked by a multi-head self-attention Transformer network; first The multi-head attention mechanism in the Transformer network is represented as: ； wherein, is the output of the layer multi-head attention mechanism, denotes a self-attention operation function, denotes the output of the layer Transformer network, denotes a concatenation operation, denotes an output projection matrix, is the total number of attention heads, is the output of the th attention head, calculated as: ； wherein, is a normalized attention weight function, is a query vector, is a query weight matrix; is a key vector, is a key weight matrix; is a value vector, is a value weight matrix; No. In a multilayer Transformer network, the residual connections and feedforward network are represented as follows: ； ； wherein, is a layer normalization function, denotes a feed-forward neural network, is an intermediate quantity, denotes the first output of the layer Transformer network. 4.The diffusion navigation method for an automatic driving vehicle based on a visual language model guide according to claim 1, wherein, Step 3.3: inputting the spatio-temporal feature sequence into the Transformer network after introducing position encoding, and obtaining the spatio-temporal fusion feature after L-layer Transformer network coding. The conditional denoising diffusion probability model is trained in the following manner: Based on the image sequence data within a time length before the 0th moment The conditional embedding vectors of each sample are obtained by using the steps 1 to 3. Based on the time after 0 o'clock The driving trajectory data within a time period is used to construct noisy trajectory samples for each sample by adding Gaussian noise; Obtaining a plurality of real vehicle driving trajectories and navigation image sequence data, and determining a 0-time point and a diffusion time step; 5.The method of claim 4, wherein, Using the conditional embedding vector and the trajectory noisy sample as a training sample, training and learning a noise prediction network using a plurality of training samples, and obtaining a denoising model according to the trained noise prediction network. ； wherein, is a denoising loss function, is a joint distribution expectation, is a driving trajectory sequence after time 0, is a diffusion time step, is a real noise, is a prediction noise, is a noisy sample, is a conditional embedding vector. 6.The method of claim 5, wherein, During the training process, the noise prediction network takes the minimum mean square error between the predicted noise and the real added noise as the loss function, and the formula is as follows: The conditional denoising diffusion probability model performs trajectory prediction in the following manner: taking Gaussian noise as a sampling initial trajectory, taking the conditional embedding vector of the current vehicle as a condition, and performing reverse denoising on the denoising model according to the diffusion time step to obtain a predicted trajectory. In the reverse denoising, the standard sampling update is performed and the calculation formula of each step is as follows: ； wherein and are the retention ratio and the cumulative retention ratio, respectively, when control the residual strength of the noise, is a resampling noise. 7.The method of claim 6, wherein, In the reverse denoising, a skip strategy is introduced to accelerate sampling update and improve efficiency; in the accelerated sampling update, only one sub-sampling sequence is updated at a time, and the specific sampling update formula is as follows: ； wherein, controlling sampling randomness, for a noise intensity coefficient at the time, is a sampling randomness control coefficient, denotes a noise prediction network.

Images