Multi-modal trajectory prediction method based on conditional diffusion model

Abstract

The invention discloses a multi-modal trajectory prediction method based on a conditional diffusion model, and belongs to the technical field of trajectory prediction. According to the method, on the basis of a non-autoregression conditional diffusion model framework, a trajectory prediction task is conceptualized into a guide inverse process of motion uncertainty diffusion, and conversion of ship motion from an uncertainty state to a deterministic state is clearly simulated by gradually eliminating uncertainty of a marine area; multi-modal mixing of interaction information and physical information is used as a generation condition to guide each stage of a back diffusion process so as to capture inherent complexity and uncertainty in marine behaviors, and then trajectory prediction performance is improved. In addition, the trade-off between the diversity and the certainty of the generated trajectory sample can be realized by adjusting the length of the Markov chain.

Description

Technical Field

[0001] The present invention relates to the technical field of trajectory prediction, and in particular to a multimodal trajectory prediction method based on a conditional diffusion model. Background Art

[0002] Trajectory prediction is critical for maritime safety systems, facilitating functions such as maritime traffic monitoring, collision avoidance, anomaly detection, and threat assessment. However, due to the complexity of human behavior and the various factors that affect ship motion, developing accurate prediction models is extremely challenging. Traditional physics-based models have been shown to be effective for short-term predictions, but struggle to achieve long-term accuracy due to their idealized assumptions. To address this limitation, the shift toward long short-term memory networks (LSTMs) and related variants has become a representative approach for ship trajectory prediction. However, the prediction of ship trajectories is inherently probabilistic and multimodal, emphasizing the importance of accurately predicting an unbiased distribution that encompasses a variety of potential future paths. Existing studies have utilized deep generative models (GANs, CVAEs) to traverse the probability distribution over all possible future trajectories. However, these models exploit latent variables rather than explicitly modeling the uncertainty inherent in the data distribution, introducing limitations in capturing the full complexity of ship motion.

[0003] Furthermore, ship motion is significantly affected by the surrounding scene environment, including physical constraints and different motion patterns in different geographical areas. In this regard, it is necessary to incorporate physical priors into the modeling process to avoid potential collision risks. In addition, spatial interactions between adjacent ships should also be taken into account. Even if a ship has a relatively certain long-term goal, its actual motion will be affected by the complex interaction dynamics between different ships, especially during close contact. Therefore, this patent proposes a multimodal trajectory prediction method based on a conditional diffusion model, which significantly improves the performance of ship trajectory prediction based on a non-autoregressive diffusion model framework and conditional modeling. This integration is crucial for developing realistic ocean scenarios that can accurately reflect the different conditions and navigation behaviors encountered in different ocean environments. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that the inherent uncertainty in the data distribution is not explicitly modeled in the existing deep generative models, which limits the model's ability to capture the full complexity of ship motion; and the existing methods do not consider the significant impact of physical constraints and spatial interactions on ship trajectories. Therefore, based on the conditional diffusion framework, the present invention uses a mixture of interactive information and physical information as a guiding factor for trajectory generation to guide each stage of the reverse diffusion process. This includes using the past trajectories of the target ship and surrounding ships and the scene context to provide a series of diverse and reasonable future trajectories. Compared with other deep generative models based on latent variables, the present invention explicitly simulates the evolution of ship motion from an uncertain state to a certain state, while capturing the inherent complexity and uncertainty in maritime behavior, thereby solving the problems mentioned in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The present invention provides a multimodal trajectory prediction method based on a conditional diffusion model, comprising the following steps:

[0007] Step S10: Future trajectory of the target ship Perform forward diffusion processing and convert the trajectory into a white Gaussian noise vector by gradually adding Gaussian noise Where k is the number of diffusion steps;

[0008] Step S20: construct a conditional encoder ψ to generate a conditional embedding that guides the back-diffusion step;

[0009] Step S201: Set the target ship observation history x -L+1：0 Encoded as high-dimensional conditional feature Z en ;

[0010] Step S202: Modeling the serial characteristics of the ship itself and its neighbors Get the high-dimensional encoding representation Z of the spatial interaction information neigh :

[0011] Step S203: Create pixel-level alignment, using the scene context image and observed 2D coordinate trajectory to encode the physical scene information into high-dimensional features Z map ;

[0012] Step S30: The mixture of the conditional encoding representations obtained in steps S201, S202 and S203 is used as a condition to guide the back-diffusion learning process, expressed as

[0013] Step S40: Define the reverse diffusion denoising network θ, using the diffused future trajectory The number of denoising steps k and the conditional embedding c are taken as input to generate the denoised trajectory;

[0014] Step S401: Calculate the time embedding k emb =[β k ,sin(β k ), cos(β k )] to encode the time dimension information of the diffusion process, where β k From fixed sample values;

[0015] Step S402: explicitly upsampling the features through multiple fully connected layers and element-by-element multiplication to obtain feature aggregation representation;

[0016] Step S403: Use the decoder to capture complex spatiotemporal dependencies and downsample the output sequence to the trajectory dimension to form a Noise prediction of the same dimension

[0017] Step S50: In the training phase, the loss function is defined as the minimum mean square error loss to update the model parameters;

[0018] Step S60: In the inference phase, a non-Markov denoising strategy DDIM is used to reconstruct the original data from the noisy state and then deterministically estimate the direction to the next target timestamp, iteratively applying the denoising steps with reduced timestamps to obtain the denoised trajectory

[0019] In the above scheme, the specific steps of step S10 are as follows:

[0020] Step S101, the forward diffusion process, gradually introduces Gaussian noise into the clean future trajectory sequence , is converted into a white Gaussian noise vector through k diffusion steps The process is represented as follows:

[0021]

[0022] in, represents the parameters of the kth diffusion step, and ∈ represents Gaussian noise.

[0023] In the above scheme, the specific steps of step S20 are as follows:

[0024] Step S201, the history information encoder converts the target ship observation history x -L+1：0 Encoding is performed by using an LSTM network to capture the temporal dependencies between all states and encode them into high-dimensional features, as shown below:

[0025] Z en =LSTMen (x -L+1：0 )

[0026] x -L+1：0 =[x -L+1 ,x -L+1 , …, x0]

[0027] where x -L+1：0 Represents the observed historical trajectory of the target ship, indicating the -L+1 To all historical trajectory points of x0, LSTM en represents the long short-term memory network;

[0028] Step S202: The task of the spatial interaction information encoder is to model the trajectory information of neighboring ships around the ship. First, the scene of each timestamp t is abstracted into a spatiotemporal directed graph G = (V, E) to model the neighbor information of each ship, where nodes represent ships and edges represent their spatial interactions. The Euclidean distance l2 is calculated as a factor to determine whether the ships affect each other, where the threshold of the influencing distance is set as a hyperparameter:

[0029] l2=||(x i -x j ) 2 +(y i -y j ) 2 ||

[0030] Where (x i , x j ), (x j ,y j ) represent the coordinates of two points, ||*|| represents the Euclidean norm of the vector, that is, the length of the vector;

[0031] Since the number of neighbors of each ship changes over time, it is necessary to summarize their past trajectories. Given a set of neighboring ship trajectories, in represents the i-th trajectory, including the -L+1 , to L historical trajectory points from x0, i represents the index of the trajectory, from 1 to N, L represents the number of time steps of backtracking, N is the total number of trajectories in the set, and the adjacent ship trajectory set The aggregate state of is calculated as:

[0032]

[0033] Among them, element-wise addition is applied to preserve the fixed structure of the encoder input and count information, processing the concatenated features of itself and neighbors to obtain a more informative representation:

[0034]

[0035] Step S203, in order to create pixel-level alignment between different modalities to learn a rich meaningful context representation that effectively utilizes scene context (similar to images) and observation trajectory (2D coordinates), first, the past trajectory x is transformed into -L+1：0 =[x -L+1 , x -L+1 , …, x0] is converted into a trajectory heat map H, which is calculated as follows:

[0036]

[0037] Where σ is the standard deviation of the Gaussian kernel, (x t ,y t ) represents the longitude and latitude coordinates at timestamp t. If the two Gaussians overlap, the element-wise summation is adopted to ensure the intensity and smoothness of the trajectory heat map. 2D-CNN is used to extract joint features from the scene context and past trajectories, and the trajectory heat map is fused with the image as follows:

[0038]

[0039] Among them, α is a hyperparameter, is a scene image, ⊙ denotes an element-wise multiplication operation, and H is a trajectory heatmap.

[0040] In the above scheme, the specific steps of step S30 are as follows:

[0041] In step S301, the conditional aggregator performs concatenation and feature transformation on the features obtained from S201, S202, and S203 through the MLP layer, as shown below:

[0042]

[0043] In the above solution, the specific steps of step S40 are as follows:

[0044] Step S401, in essence, the back diffusion network is a noise prediction network that uses the diffused future trajectory The denoising step k and the conditional embedding c are taken as input. First, the diffusion trajectories obtained at different time steps are feature encoded by calculating the time embedding representation, which is expressed as follows:

[0045] k emb =[β k ,sin(β k ), cos(β k )]

[0046] Among them, β k From fixed sampling, concatenate with the condition c along the variable dimension to generate a new condition c k ,enter Transform the diffuse future trajectory embedding by linear projection

[0047] Step S402, apply several fully connected layers and use element-by-element multiplication to explicitly embed the input in the feature dimension and the new condition c k Upsample to obtain an aggregated representation of the features

[0048] ω=sigmoid(FC1(c k ))

[0049] b=FC2(c k )

[0050]

[0051] Among them, FC1, FC2, and FC3 represent fully connected layers, and sigmoid is the activation function. Broadcast by timestamp to form right Broadcast the same timestamp to get

[0052] Step S403, before the decoder layer, position embedding is introduced in the implementation to inject information about the relative positions of different timestamps. Subsequently, the Transformer-based decoder consists of multiple self-attention layers to model the complex spatiotemporal dependencies of the fused features, where the self-attention module promotes enhanced interaction between timestamps by leveraging the derived query, key, and value sequence representations, which are expressed as follows:

[0053]

[0054] Among them, SelfAttention represents the self-attention module, and PositionEncoding represents the position encoding module

[0055]

[0056] FFN represents a feedforward neural network, which transforms the output sequence e through several fully connected layers with the same aggregation operation. k Downsampled to the trajectory dimension to form Noise prediction of the same dimension The denoised output is calculated in a deterministic manner and is expressed as:

[0057]

[0058] in is the denoised trajectory, represents the estimated trajectory at step ks, ∈θ(*) represents the noise prediction function, which is based on the current diffusion trajectory and denoising step k and conditional embedding c to predict the noisy part of the trajectory, is a hyperparameter obtained by fixed sampling, s is the step size reduction used to accelerate the back diffusion process, is an estimate of the original trajectory state at step k, calculated as:

[0059]

[0060] in is a hyperparameter obtained by fixed sampling, is the denoised trajectory, representing the trajectory estimate at denoising step k.

[0061] In the above scheme, the specific steps of step S50 are as follows:

[0062] In step S501, the training loss function is defined as the mean square error loss. Both diffusion and back diffusion are modeled as Markov chains with Gaussian transitions. Trajectory prediction is performed by modeling the joint distribution. The modeling is expressed as follows:

[0063]

[0064] in, It means that under given conditions c, from the initial Gaussian noise trajectory To the final trajectory The joint probability distribution of represents the initial Gaussian noise trajectory The probability distribution of θ represents a conditional probability distribution, Indicates from arrive The probability distribution of each intermediate trajectory of , K is the total number of steps in the forward diffusion process, It means that the initial trajectory obeys a Gaussian distribution with a mean of 0 and a unit matrix I as the covariance. c is the conditional embedding obtained in step S301, which includes the observed trajectories of the target ship itself and its neighboring ships, as well as conditional information such as the multimodal scene context. At the same time:

[0065]

[0066] in, This means that the conditional probability is subject to is the mean, is the normal distribution with variance, represents the mean estimated output controlled by the neural network parameter θ under given condition c, depending on the denoising step k, represents the variance obtained by fixing the sampled values.

[0067] Since both the forward diffusion process and the backward diffusion process are Gaussian distributions, the training objective is naturally converted to calculating the difference between the means:

[0068]

[0069] in represents the approximate posterior mean estimate at the denoising step k following the reverse diffusion process, with c as a given condition, It represents the mean estimate given by the noise prediction network with θ as parameter at denoising step k, given c.

[0070] Finally, a conditional denoiser is learned by applying the following simplified loss function at each step k, which leads to better generative properties as follows:

[0071]

[0072] 7. The multimodal trajectory prediction research method based on the conditional diffusion model according to claim 1 is characterized in that: the specific steps of step S60 are as follows:

[0073] Step S601, the back diffusion process uses a non-Markov denoising strategy DDIM, which starts by reconstructing the original data from the noisy state and then deterministically estimates the direction to the next target timestamp. Once the encoder ψ and decoder θ are trained, a reasonable trajectory can be generated through the denoising step. It is worth noting that the initial input is Gaussian noise The number of iterations is K / s. The denoised trajectory is obtained by iteratively applying the denoising step with decreasing timestamps as the final prediction.

[0074] The beneficial effects of the present invention are:

[0075] 1. The present invention conceptualizes the trajectory prediction task as a technical means of guiding the inverse process of the diffusion of motion pattern uncertainty, explicitly simulates the transition of motion from uncertainty to determinism, solves the problem that existing deep generative models are unable to explicitly model the inherent uncertainty in data distribution, and achieves the effect of improving the model's ability to fully capture the dynamics and complexity of trajectory motion, effectively dealing with future uncertainties in dynamic scenes.

[0076] 2. The present invention encodes the historical trajectories of the target ship and the neighboring ships into state embedding by multimodally fusing external condition information such as neighboring spatial interactions and physical constraints (steps 20 and 30), and combines the observed trajectories with the two-dimensional pixel-level heat map formed by the static scene context, thereby ensuring the effective integration of physical prior information in the modeling process, solving the problem that the existing technology is difficult to model the surrounding scene environmental factors, achieving the effect of being able to generate spatially sensitive and physically reasonable future trajectories, and significantly improving the accuracy of the prediction.

[0077] 3. The present invention uses a non-autoregressive trajectory generation method (step 60) to sample noise trajectories from a predefined Gaussian distribution and gradually derive accurate future trajectories using a back diffusion process, thereby solving the problems of error accumulation and weak resistance to noise data in existing methods, thereby achieving the effect of reducing prediction errors and improving model robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 The figure is a flow chart of the steps of the method of the present invention. DETAILED DESCRIPTION

[0079] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0080] See also Figure 1 , the present invention provides a technical solution:

[0081] The present invention provides the following technical solution: a multimodal trajectory prediction method based on a conditional diffusion model. The specific steps are to use the ship trajectory prediction in the Denmark Strait area as an example. The specific steps of applying the trajectory prediction method to the ship trajectory prediction in the Denmark Strait area are as follows:

[0082] Step 1: Future trajectory of the target ship Perform forward diffusion processing and convert the trajectory into a white Gaussian noise vector by gradually adding Gaussian noise Where k is the number of diffusion steps;

[0083] Step 2: Construct a conditional encoder ψ to generate conditional embeddings that guide the back-diffusion step.

[0084] Step 2-1: Use the history information encoder to convert the target ship observation history x -L+1：0Encoding is performed by using an LSTM network to capture the temporal dependencies between all states and encode them into high-dimensional features, as shown below:

[0085] Z en =LSTM en (x -L+1：0 )

[0086] x -L+1：0 =[x -L+1 ,x -L+1 , …, x0]

[0087] where x -L+1：0 Represents the observed historical trajectory of the target ship, indicating the -L+1 To all historical trajectory points of x0, LSTM en represents the long short-term memory network;

[0088] Step 2-2, the task of the spatial interaction information encoder is to model the trajectory information of neighboring ships around the ship. First, the scene of each timestamp t is abstracted into a spatiotemporal directed graph G = (V, E) to model the neighbor information of each ship, where nodes represent ships and edges represent their spatial interactions. The Euclidean distance l2 is calculated as a factor to determine whether the ships affect each other, where the threshold of the impact distance is set as a hyperparameter:

[0089] l2=||(x i -x j ) 2 +(y i -y j ) 2 ||

[0090] Where (x i , x j ), (x j ,y j ) represent the coordinates of two points, ||*|| represents the Euclidean norm of the vector, that is, the length of the vector;

[0091] Since the number of neighbors of each ship changes over time, it is necessary to summarize their past trajectories. Given a set of neighboring ship trajectories, in represents the i-th trajectory, including the -L+1 , to L historical trajectory points from x0, i represents the index of the trajectory, from 1 to N, L represents the number of time steps of backtracking, N is the total number of trajectories in the set, and the adjacent ship trajectory set The aggregate state of is calculated as:

[0092]

[0093] Among them, element-wise addition is applied to preserve the fixed structure of the encoder input and count information, processing the concatenated features of itself and neighbors to obtain a more informative representation:

[0094]

[0095] Step 2-3, for scene context modeling, our implementation is to create pixel-level alignment between different modalities to learn a rich and meaningful context representation that effectively utilizes scene context (similar to images) and observation trajectory (2D coordinates). First, the past trajectory x is transformed into -L+1：0 =[x -L+1 , x -L+1 , …, x0] is converted into a trajectory heat map H, which is calculated as follows:

[0096]

[0097] Where σ is the standard deviation of the Gaussian kernel, (x t，yt ) represents the longitude and latitude coordinates at timestamp t. If the two Gaussians overlap, the element-wise summation is adopted to ensure the intensity and smoothness of the trajectory heat map. 2D-CNN is used to extract joint features from the scene context and past trajectories, and the trajectory heat map is fused with the image as follows:

[0098]

[0099] Where α is a hyperparameter, is a scene image, ⊙ denotes an element-wise multiplication operation, and H is a trajectory heatmap.

[0100] Step 3: The mixture of conditional encoding representations obtained in steps S201, S202 and S203 is used as the condition to guide the back-diffusion learning process, and the features are concatenated and transformed through the conditional aggregator MLP, which is expressed as follows:

[0101]

[0102] Step 4: Define the reverse diffusion denoising network θ, using the diffused future trajectory Denoising step number k and conditional embedding c As input, denoised trajectory is generated;

[0103] Step 4-1, by calculating the time embedding representation, the diffusion trajectories obtained at different time steps are feature encoded, which is expressed as follows:

[0104] k emb =[β k ,sin(β k ), cos(β k )]

[0105] Among them, β k From fixed sampling, concatenate with the condition c along the variable dimension to generate a new condition c k ,enter Transform the diffuse future trajectory embedding by linear projection

[0106] Step 4-2, apply several fully connected layers and use element-wise multiplication to explicitly embed the input in the feature dimension and the new condition c k Upsample to obtain an aggregated representation of the features

[0107] ω=sigmoid(FC1(c k ))

[0108] b=FC2(c k )

[0109]

[0110] Among them, FC1, FC2, and FC3 represent fully connected layers, and sigmoid is the activation function. Broadcast by timestamp to form right Broadcast the same timestamp to get

[0111] In step 4-3, before the decoder layer, position embedding is introduced in the implementation to inject information about the relative positions of different timestamps. Subsequently, the Transformer-based decoder consists of multiple self-attention layers to model the complex spatiotemporal dependencies of the fused features, where the self-attention module promotes enhanced interactions between timestamps by leveraging the derived query, key, and value sequence representations, as shown below:

[0112]

[0113] Among them, SelfAttention represents the self-attention module, and PositionEncoding represents the position encoding module

[0114]

[0115] FFN represents a feedforward neural network, which transforms the output sequence e through several fully connected layers with the same aggregation operation. k Downsampled to the trajectory dimension to form Noise prediction of the same dimension The denoised output is calculated in a deterministic manner and is expressed as:

[0116]

[0117] in is the denoised trajectory, representing the estimated trajectory at step ks, ∈ θ (*) represents the noise prediction function, which is based on the current diffusion trajectory and denoising step k and conditional embedding c to predict the noisy part of the trajectory, is a hyperparameter obtained by fixed sampling, s is the step size reduction used to accelerate the back diffusion process, is an estimate of the original trajectory state at step k, calculated as:

[0118]

[0119] in is a hyperparameter obtained by fixed sampling, is the denoised trajectory, representing the trajectory estimate at denoising step k.

[0120] Step 5: In the training phase, the model parameters are updated by defining the loss function as mean square error loss;

[0121] In step 5-1, the training loss function is defined as the mean square error loss. Specifically, both diffusion and back diffusion are modeled as Markov chains with Gaussian transitions. Trajectory prediction is performed by modeling the joint distribution. The modeling is expressed as follows:

[0122]

[0123] in, It means that under given conditions c, from the initial Gaussian noise trajectory To the final trajectory The joint probability distribution of represents the initial Gaussian noise trajectory The probability distribution of θ represents a conditional probability distribution, Indicates from arrive The probability distribution of each intermediate trajectory of , K is the total number of steps in the forward diffusion process, It means that the initial trajectory obeys a Gaussian distribution with a mean of 0 and a unit matrix I as the covariance. c is the conditional embedding obtained in step S301, which includes the observed trajectories of the target ship itself and its neighboring ships, as well as conditional information such as the multimodal scene context. At the same time:

[0124]

[0125] in, This means that the conditional probability is subject to is the mean, is the normal distribution with variance, represents the mean estimated output controlled by the neural network parameter θ under given condition c, depending on the denoising step k, represents the variance obtained by fixing the sampled values.

[0126] Since both the forward diffusion process and the backward diffusion process are Gaussian distributions, the training objective is naturally converted to calculating the difference between the means:

[0127]

[0128] in represents the approximate posterior mean estimate at the denoising step k following the reverse diffusion process, with c as a given condition, It represents the mean estimate given by the noise prediction network with θ as parameter at denoising step k, given c.

[0129] Finally, a conditional denoiser is learned by applying the following simplified loss function at each step k, which leads to better generative properties as follows:

[0130]

[0131] Step 6: In the inference phase, a non-Markov denoising strategy DDIM is adopted to obtain the denoised trajectory by reconstructing the original data from the noisy state and then deterministically estimating the direction to the next target timestamp, iteratively applying the denoising steps with reduced timestamps

[0132] Step 6-1, in the inference phase, the back diffusion process uses a non-Markov denoising strategy DDIM, which starts by reconstructing the original data from the noisy state and then deterministically estimates the direction to the next target timestamp. Once the encoder ψ and decoder θ are trained, a reasonable trajectory can be generated through the denoising step. It is worth noting that the initial input is Gaussian noise The number of iterations is K / s. The denoised trajectory is obtained by iteratively applying the denoising step with decreasing timestamps As a final prediction.

[0133] The multimodal trajectory prediction research method based on the conditional diffusion model includes a forward diffusion module, a conditional encoder module ψ, and a reverse diffusion denoising network θ, specifically:

[0134] In the forward diffusion phase, Gaussian noise is gradually added to the clean future trajectory. , is transformed into a white Gaussian noise vector in k diffusion steps

[0135] In the conditional encoder module, the mixture of conditional coded representations is used as the condition to guide the back-diffusion learning process, and the features are concatenated and transformed through MLP;

[0136] In the reverse diffusion denoising network θ, the diffused future trajectory is used The denoising step k and the conditional embedding c are taken as input to iteratively generate denoised trajectories;

[0137] The method of the present invention takes the trajectory prediction method as the research object, and conceptualizes the trajectory prediction task as the guided inverse process of the diffusion of motion uncertainty based on the non-autoregressive diffusion model framework. By gradually eliminating the uncertainty in the ocean area, the transition of the ship motion from uncertainty to certainty is clearly simulated; the multimodal mixture of interactive information and physical information is used as a generation condition to guide each stage of the reverse diffusion process to capture the inherent complexity and uncertainty in maritime behavior, thereby improving the trajectory prediction performance. In addition, the trade-off between the diversity and certainty of the generated trajectory samples can be achieved by adjusting the length of the Markov chain. Although the present invention has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned embodiments, or to replace some of the technical features therein by equivalents. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multimodal trajectory prediction method based on a conditional diffusion model, characterized in that: The following steps are involved: step S10: Future trajectory of target ship Perform forward diffusion processing and convert the trajectory into a white Gaussian noise vector by gradually adding Gaussian noise Where k is the number of diffusion steps; Step S20: construct a conditional encoder ψ to generate a conditional embedding that guides the back-diffusion step; Step S201: Set the target ship observation history x -L+1：0 Encoded as high-dimensional conditional feature Z en ; Step S202: Modeling the serial characteristics of the ship itself and its neighbors Get the high-dimensional encoding representation Z of the spatial interaction information neigh ; Step S203: Create pixel-level alignment, using the scene context image and observed 2D coordinate trajectory to encode the physical scene information into high-dimensional features Z map ; Step S30: The mixture of the conditional encoding representations obtained in steps S201, S202 and S203 is used as a condition to guide the back-diffusion learning process, expressed as Step S40: Define the reverse diffusion denoising network θ, using the diffused future trajectory The number of denoising steps k and the conditional embedding c are taken as input to generate the denoised trajectory; Step S401: Calculate the time embedding k emb =[β k ,sin(β k ), cos(β k )] to encode the time dimension information of the diffusion process, where β k From fixed sample values; Step S402: explicitly upsampling the features through multiple fully connected layers and element-by-element multiplication to obtain feature aggregation representation; Step S403: Use the decoder to capture complex spatiotemporal dependencies and downsample the output sequence to the trajectory dimension to form a Noise prediction of the same dimension Step S50: In the training phase, the loss function is defined as the minimum mean square error loss to update the model parameters; Step S60: In the inference phase, a non-Markov denoising strategy DDIM is used to reconstruct the original data from the noisy state and then deterministically estimate the direction to the next target timestamp, iteratively applying the denoising steps with reduced timestamps to obtain the denoised trajectory 2. The multimodal trajectory prediction research method based on the conditional diffusion model according to claim 1 is characterized by: The specific steps of step S10 are as follows: Step S101, the forward diffusion process, gradually introduces Gaussian noise into the clean future trajectory sequence , is converted into a white Gaussian noise vector through k diffusion steps The process is represented as follows: in, represents the parameters of the kth diffusion step, and ∈ represents Gaussian noise.

3. The multimodal trajectory prediction research method based on the conditional diffusion model according to claim 1 is characterized by: The specific steps of step S20 are as follows: Step S201, the history information encoder converts the target ship observation history x -L+1：0 Encoding is performed by using an LSTM network to capture the temporal dependencies between all states and encode them into high-dimensional features, as shown below: Z en =LSTM en (x -L+1：0 ) x -L+1：0 =[x -L+1 ,x -L+1 ,…, x0] where x -L+1：0 Represents the observed historical trajectory of the target ship, indicating the -L+1 To all historical trajectory points of x0, LSTM en represents the long short-term memory network; Step S202: The task of the spatial interaction information encoder is to model the trajectory information of neighboring ships around the ship. First, the scene of each timestamp t is abstracted into a spatiotemporal directed graph G = (V, E) to model the neighbor information of each ship, where nodes represent ships and edges represent their spatial interactions. The Euclidean distance l2 is calculated as a factor to determine whether the ships affect each other, where the threshold of the influencing distance is set as a hyperparameter: l2=||(x i -x j ) 2 +(and i -and j ) 2 || Where (x i , x j ), (x j ,y j ) represent the coordinates of two points, ||*|| represents the Euclidean norm of the vector, that is, the length of the vector; Since the number of neighbors of each ship changes over time, it is necessary to summarize their past trajectories. Given a set of neighboring ship trajectories, in represents the i-th trajectory, including the -L+1 , to L historical trajectory points from x0, i represents the index of the trajectory, from 1 to N, L represents the number of time steps of backtracking, N is the total number of trajectories in the set, and the adjacent ship trajectory set The aggregate state of is calculated as: Among them, element-wise addition is applied to preserve the fixed structure of the encoder input and count information, processing the concatenated features of itself and neighbors to obtain a more informative representation: Step S203, in order to create pixel-level alignment between different modalities to learn a rich meaningful context representation that effectively utilizes scene context (similar to images) and observation trajectory (2D coordinates), first, the past trajectory x is transformed into -L+1：0 =[x -L+1 , x -L+1 , …, x0] is converted into a trajectory heat map H, which is calculated as follows: Where σ is the standard deviation of the Gaussian kernel, (x t ,y t ) represents the longitude and latitude coordinates at timestamp t. If the two Gaussians overlap, the element-wise summation is adopted to ensure the intensity and smoothness of the trajectory heat map. 2D-CNN is used to extract joint features from the scene context and past trajectories, and the trajectory heat map is fused with the image as follows: Where α is a hyperparameter, is a scene image, ⊙ denotes an element-wise multiplication operation, and H is a trajectory heatmap.

4. The multimodal trajectory prediction research method based on the conditional diffusion model according to claim 1 is characterized by: The specific steps of step S30 are as follows: In step S301, the conditional aggregator performs concatenation and feature transformation on the features obtained from S201, S202, and S203 through the MLP layer, as shown below:

5. The multimodal trajectory prediction research method based on conditional diffusion model according to claim 1 is characterized by: The specific steps of step S40 are as follows: Step S401, in essence, the back diffusion network is a noise prediction network that uses the diffused future trajectory The denoising step k and the conditional embedding c are taken as input. First, the diffusion trajectories obtained at different time steps are feature encoded by calculating the time embedding representation, which is expressed as follows: k emb =[β k ,sin(β k ),cos(β k )] Among them, β k From fixed sampling, concatenate with the condition c along the variable dimension to generate a new condition c k ,enter Transform the diffuse future trajectory embedding by linear projection Step S402, apply several fully connected layers and use element-by-element multiplication to explicitly embed the input in the feature dimension and the new condition c k Upsample to obtain the aggregated representation of features ω=sigmoid(FC1(c k )) b=FC2(c k ) Among them, FC1, FC2, and FC3 represent fully connected layers, and sigmoid is the activation function. Broadcast by timestamp to form right Broadcast the same timestamp to get Step S403, before the decoder layer, position embedding is introduced in the implementation to inject information about different time periods. The information of the relative positions of the timestamps is then used. The Transformer-based decoder consists of multiple self-attention layers to model the complex spatiotemporal dependencies of the fused features, where the self-attention module promotes enhanced interaction between timestamps by leveraging the derived query, key, and value sequence representations, as shown below: Among them, SelfAttention represents the self-attention module, and PositionEncoding represents the position encoding module FFN represents a feedforward neural network, which transforms the output sequence e through several fully connected layers with the same aggregation operation. k Downsampled to the trajectory dimension to form Noise prediction of the same dimension The denoised output is calculated in a deterministic manner and is expressed as: in is the denoised trajectory, represents the estimated trajectory at step ks, ∈θ(*) represents the noise prediction function, which is based on the current diffusion trajectory and denoising step k and conditional embedding c to predict the noisy part of the trajectory, is a hyperparameter obtained by fixed sampling, s is the step size reduction used to accelerate the back diffusion process, is an estimate of the original trajectory state at step k, calculated as: in is a hyperparameter obtained by fixed sampling, is the denoised trajectory, representing the trajectory estimate at denoising step k.

6. The multimodal trajectory prediction research method based on the conditional diffusion model according to claim 1 is characterized by: The specific steps of step S50 are as follows: In step S501, the training loss function is defined as the mean square error loss. Both diffusion and back diffusion are modeled as Markov chains with Gaussian transitions. Trajectory prediction is performed by modeling the joint distribution. The modeling is expressed as follows: in, It means that under given conditions c, from the initial Gaussian noise trajectory To the final trajectory The joint probability distribution of represents the initial Gaussian noise trajectory The probability distribution of θ represents a conditional probability distribution, Indicates from arrive The probability distribution of each intermediate trajectory of , K is the total number of steps in the forward diffusion process, It means that the initial trajectory obeys a Gaussian distribution with a mean of 0 and a unit matrix I as the covariance. c is the conditional embedding obtained in step S301, which includes the observed trajectories of the target ship itself and its neighboring ships, as well as conditional information such as the multimodal scene context. At the same time: in, This means that the conditional probability is subject to is the mean, is the normal distribution with variance, represents the mean estimated output controlled by the neural network parameter θ under given condition c, depending on the denoising step k, represents the variance obtained by fixing the sampling value; Since both the forward diffusion process and the backward diffusion process are Gaussian distributions, the training objective is naturally converted to calculating the difference between the means: in represents the approximate posterior mean estimate at the denoising step k following the reverse diffusion process, with c as a given condition, represents the mean estimate given by the noise prediction network with θ as parameter at denoising step k, given c; Finally, a conditional denoiser is learned by applying the following simplified loss function at each step k, which leads to better generative properties as follows:

7. The multimodal trajectory prediction research method based on the conditional diffusion model according to claim 1, Features: The specific steps of step S60 are as follows: Step S601, the back diffusion process uses a non-Markov denoising strategy DDIM, which starts by reconstructing the original data from the noisy state, and then deterministically estimates the direction to the next target timestamp. Once the encoder ψ and decoder θ are trained, a reasonable trajectory can be generated through the denoising step. The initial input is Gaussian noise The number of iterations is K / s, and the denoised trajectory is obtained by iteratively applying the denoising steps with decreasing timestamps. as the final prediction.

Images