Vehicle track complementing method and system fused with road topological map

Abstract

According to the invention, based on a low-line-number road side laser radar, real-time extraction and optimization of multi-vehicle tracks are researched, and a track completion method and system combined with a road topological map are provided for solving the problem of track interruption caused by mutual shielding of vehicle targets in a complex environment. According to the method, track completion is carried out by obtaining a full-time motion mode, context information and environment characteristics of a vehicle in a scene and combining a road topological map. Specifically, the topological map is coded by adopting a gating loop unit and a map attention network, and missing trajectory data is generated by utilizing a diffusion probability model based on trajectory context and map condition constraints. By integrating point cloud data acquired by a roadside laser radar, a vehicle track is complemented, and a high-precision continuous track conforming to road constraints, traffic rules and vehicle kinematics characteristics is generated. The trajectory completion method has important significance in improving the intelligent traffic decision-making level and promoting the application of the intelligent driving technology.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision and deep learning, and particularly to the technology of vehicle trajectory completion in road scenes of intelligent transportation systems. More specifically, the present invention relates to a vehicle trajectory completion method integrating a road topological map. Background Art

[0002] Complete trajectory data is the basis for traffic state analysis such as traffic flow statistics and congestion detection, and also plays a key role in fields such as path planning, traffic flow prediction, and the formulation of emergency response strategies. How to obtain predicted trajectories and complete them to obtain continuous trajectory information has become a key task in the current field of computer vision. However, in roadside LiDAR perception systems, there is a problem of missing target point clouds caused by target occlusion, which further leads to problems such as target loss and trajectory interruption during target tracking. Currently, common trajectory prediction methods can be divided into rule-based methods and deep learning-based methods.

[0003] In rule-based methods, Yi et al. proposed an adaptive cruise control (ACC) vehicle trajectory prediction algorithm, which accurately predicts vehicle trajectories based on the yaw rate of change and the curvature rate of change to solve adverse situations in the ACC system; Chen et al. proposed a speed prediction method based on a scene-based hidden Markov model (HMM) to predict the speed of the vehicle in front to predict the best trajectory for vehicle lane change. However, these methods only focus on vehicle motion modeling and ignore the scene constraints of trajectory prediction, resulting in inaccurate predictions in dynamic urban environments and lacking multi-modal trajectory prediction capabilities. In the trajectory completion task, some statistical completion methods use the average or mean value to replace missing values. In addition, alternative methods such as linear fitting, the K-nearest neighbor algorithm, and the expectation maximization algorithm are also used. However, these methods use rigid prior assumptions, and when the data distribution changes or the data has a complex structure, the generalization ability of these methods will be limited.

[0004] To overcome the shortcomings of rule-based methods, the research on deep learning-based trajectory prediction methods has been continuously deepened. Among them, most studies are based on the encoder-decoder architecture, encoding the historical trajectories and other features of the target, and decoding the hidden features based on LSTM or its variants in the time dimension to generate the predicted trajectories of each object. For example, the Transformer framework proposed by Salzmann et al. encodes the multi-output historical trajectories using a multi-head attention model after vectorizing the trajectories through a multi-layer perceptron (MLP), thus achieving accurate prediction of future multi-modal trajectories; Liu et al. designed a network architecture based on stacked Transformers to establish multimodality at the feature level through a set of fixed independent candidate regions, called mmTransformer. However, these methods perform poorly in the trajectory completion task because autoregressive methods may not be able to provide enough global information to accurately fill in the missing data due to their locality. Summary of the Invention

[0005] An object of the present invention is to provide a vehicle trajectory completion method that integrates a road topological map, and proposes a map-conditioned trajectory diffusion generation model (MCTD). The full-time motion pattern, context information, and environmental features of the scenario are used to fuse and encode the feature information of the topological map, and a diffusion probability model based on trajectory context and map condition constraints is used to generate the missing trajectories. Experimental results show that the trajectory completion network proposed by the present invention can accurately complete the target position and motion state during the interruption period, and effectively recover the trajectory missing caused by occlusion, sensor failure, etc. It is of great significance to improve the level of intelligent transportation decision-making and promote the implementation of intelligent driving applications.

[0006] To solve the above technical problems, the present invention provides a vehicle trajectory completion method that integrates a road topological map, and proposes a map-conditioned trajectory diffusion generation model (MCTD), including designing a "time-sequence - space" two-stream encoder to enhance the model's understanding of complex road network topological relationships, and improving the noise prediction network in the CSDI model by adding a lane context interaction module for fusing lane and trajectory features to the diffusion model noise prediction network; specifically including the following steps:

[0007] Step 1, construct a trajectory completion dataset;

[0008] Step 2, construct a topological map encoding module, input the lane node sequence features, and after node feature embedding, time-sequence feature encoding, topological relationship modeling, and graph attention aggregation processing, output the lane node features that fuse the road structure for input to the noise prediction network;

[0009] Step 3: Construct a noise prediction network by adding a Transformer module for lane context that fuses lane and trajectory features to the existing CSDI;

[0010] Step 4: Incorporate the noise prediction network into the diffusion model network, and learn the data distribution through a progressive noise addition and denoising process to output the completed trajectory features;

[0011] Step 5: Use the trajectory completion dataset to train the trajectory completion model formed by Steps 2 - 4, and use the trained model to achieve vehicle trajectory completion.

[0012] Furthermore, the trajectory completion dataset in Step 1 includes the actually observed trajectory points X0 and its mask matrix M0, lane nodes A and its lane mask matrix M lane , lane node adjacency matrix A next ; Standardize the trajectory points and lane nodes with the same mean and standard deviation. To simulate the trajectory missing scenario, the trajectory points X0 also include a set of trajectory points X1 with a missing rate of K%, and its mask matrix M.

[0013] Furthermore, the processing process of node feature embedding is as follows:

[0014] Take the vector topological map G(V, E) as the input, where the lane nodes V are the centerlines of all roads within a fixed area around the target vehicle, and the edges E include successor edges E suc and adjacent edges E prox two categories; Divide the lane centerline into small segments of fixed length, and each segment, that is, the lane node, is discretized into N pose points. The feature vector of each node is expressed as where the feature of each pose point includes position, yaw angle, and traffic control element marker; Perform linear transformation and non - linear mapping on the original lane node features:

[0015] f (0) = LeakyReLU(W e f + b e )

[0016] Among them, define as the embedding matrix, d in is the dimension of the node of the input feature, d emb is the dimension of the node feature after embedding encoding, f is the original feature of the lane node, LeakyReLu(.) is the activation function, is the embedding representation of the lane feature.

[0017] Furthermore, the processing process of topological relationship modeling is as follows:

[0018] When constructing the adjacency matrix of lane nodes, first initialize it with an identity matrix, that is, add an edge pointing to itself for each node. To further capture the spatial topology of the lane network, construct the adjacency matrix based on the road connection relationship.

[0019]

[0020] To maintain the bidirectionality of relationship propagation, convert the adjacency matrix into an undirected graph form:

[0021]

[0022] where, ∨ represents the element-wise OR operation of two matrices.

[0023] Furthermore, the processing process of graph attention aggregation is as follows:

[0024] Assume that the feature of node i in the l-th layer is The set of neighbor nodes of node i is For each head m ∈ {1, 2,..., M}, generate a query through the projection matrix key value

[0025]

[0026] where, is the learnable matrix for the m-th head, h is the total dimension of the output of this attention layer, and h / M is the single-head dimension;

[0027] Calculate the attention score of node i to neighbor j, and only allow valid connections marked by the adjacency matrix to participate in the calculation:

[0028]

[0029]

[0030] where, represents the original correlation score between node i and neighbor j, reflecting the similarity of the features of the two nodes, represents filtering the original attention calculation range and only calculating the part where the two nodes are connected;

[0031] is the probability distribution of the similarity of each node, representing the final attention score between nodes; for each head m, aggregate the value vectors of neighbor nodes:

[0032]

[0033] z idenotes the fused feature obtained by aggregating all attention heads;

[0034] At this time, the output dimension of each head is h / M, and then the outputs of all heads are concatenated to restore the total dimension h, and then the information is integrated through a linear projection:

[0035]

[0036] W o denotes the linear mapping matrix used to restore the feature dimension of all attention heads obtained by concatenation to the total output hidden layer dimension;

[0037] Finally, the aggregated neighborhood node features are added to the input features to achieve residual connection:

[0038]

[0039] where denotes the feature of node i in the (l + 1)-th layer.

[0040] Furthermore, the input of the noise prediction network includes the conditional observations constructed from the trajectory point X0 and the interpolation target diffusion step t, the map encoding G(V, E), the time and space information features from the trajectory data; and

[0041] After extracting features from and respectively and then concatenating them, they are fed into the temporal Transformer and the feature Transformer to further extract temporal and spatial features. The output is used as the input of the lane context Transformer, and then concatenated with the time embedding and the feature embedding. Finally, the output of the noise prediction network is obtained after extracting features through 2 residual blocks.

[0042] First, the self-attention operation takes the trajectory feature X as the input, and the processing process is expressed as:

[0043] X (1) = LayerNorm(Attention(Q X , K X , V X ) + X)

[0044] where X (1) denotes the feature output by self-attention, and the key Q X , the value K X , the query VX All come from trajectory features. LayerNorm(·) represents layer normalization; Attention(·) represents self-attention operation. Assume the dimension of each head is d head , then the calculation formula is:

[0045]

[0046] Subsequently, taking the lane feature H as context information, perform cross-attention operation:

[0047] X (2) = LayerNorm(Cross-Attention(Q X , K H , V H ) + X (1) )

[0048] Among them, X (2) represents the output of cross-attention. The values and queries come from lane features, while the keys come from trajectory features;

[0049] Finally, further process the fused feature X( 2 ) with a feed-forward network; the feed-forward network consists of two fully connected layers, and the Gaussian error linear unit GELU activation function is applied between the two fully connected layers. The calculation process of the feed-forward network is as follows:

[0050] X (3) = LayerNorm(Linear2(GELU(Linear1(X (2) ))) + X (2) )

[0051] Among them, X (3) represents the output of the feed-forward network. Linear1 and Linear2 are two fully connected layers.

[0052] Furthermore, during the training process, the input data is the vehicle trajectory data X0 with missing values and the topological structure G(V, E) of the vectorized topological map, where the lane nodes V are the centerlines of all roads within a fixed area around the target vehicle, and the edges E include successor edges E suc and adjacent edges E prox of two types; adopt a random sampling strategy to divide the observed values X0 into conditional observed values and imputation targets Specifically, randomly draw an integer r from (0, 100), and randomly select r% from the observed values X0 as conditional observed values The remaining part is the imputation target To mark which positions in X0 are selected as conditional observed values introduce a conditional mask mco ∈ {0, 1}, m co The position where m m co The position where m Assume the encoded map is G enc , obtained from Step 2. The goal of the trajectory completion model is to estimate the true conditional data distribution through the model distribution Estimate the true conditional data distribution During the forward diffusion process, for each diffusion step t, gradually add noise to the interpolation target In the reverse denoising process, extend the parameterization method in the diffusion model DDPM; and train the noise prediction network ∈ by minimizing the loss function θ :

[0053]

[0054] Among them, Denotes taking the expectation of the joint distribution of three random variables. X0 ∼ q(X0) means the input data X0 follows the training set distribution q(X0), Denotes that the noise ∈ follows the standard normal distribution;

[0055] Thus, given the observed value X0 and the trained noise prediction network ∈ θ , through the diffusion model adding and removing noise, output the predicted interpolation target

[0056] Furthermore, after training, the task in the inference stage is to complete the missing part X1 in the observed value X0. Let the conditional mask be equal to the missing observed value mask, that is, m co = X. At this time, the conditional observed value Where ⊙ represents element-wise multiplication. Input the conditional observed value at this time into the trained model, and calculate the result as the final result of the trajectory completion and output it.

[0057] The present invention also provides a vehicle trajectory completion system integrating a road topology map, including:

[0058] A processor and a memory. The memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the vehicle trajectory completion method integrating the road topology map as described in the above technical solution.

[0059] The present invention has innovated and optimized the structural design in the field of trajectory completion. The proposed trajectory completion method based on road topological maps not only surpasses traditional methods in terms of accuracy but also achieves significant improvements in computational efficiency and resource utilization. This method can effectively address the problem of trajectory loss caused by occlusion, sensor failures, etc., providing a more accurate and efficient solution for fields such as intelligent transportation, autonomous driving, and path planning. It has at least the following beneficial effects:

[0060] 1. The completion accuracy of this application in combination with road topological maps is significantly improved: The trajectory completion method proposed by the present invention combines the spatio-temporal motion laws of lane nodes with the constraints of road topological maps and uses gated recurrent units (GRUs) and graph attention networks (GATs) for effective trajectory completion. During the completion process, the topological structure and traffic rules of the road are considered, avoiding unreasonable trajectories caused by complex road conditions such as intersections or traffic control areas. Compared with existing methods, the method of the present invention has a significant improvement in trajectory completion accuracy and can more accurately restore the target position and motion state due to occlusion or failure.

[0061] 2. This application introduces a diffusion probability model to improve the stability and feasibility of trajectory generation: The present invention adopts a denoising diffusion probability model (DDPM), and gradually restores the target trajectory through the processes of forward diffusion and reverse denoising. This method can avoid the problems of unstable training and mode collapse existing in traditional generation models. Especially when dealing with trajectory completion with long time spans and high continuity, it shows superior robustness. This model not only surpasses traditional methods in terms of accuracy but also effectively avoids unreasonable trajectories during the generation process, conforming to traffic rules and road layouts.

[0062] 3. This application uses multi-modal information fusion to enhance the robustness of the model: The trajectory completion method of the present invention realizes high-precision trajectory restoration during the reverse denoising process by cross-modal fusion of road topological maps and trajectory information. The adopted Transformer structure can effectively handle the complex relationship between lane features and trajectory sequences, enabling the model to fully understand the dynamic interaction between roads and vehicles when facing trajectory completion tasks in complex urban environments. Compared with existing methods, the present invention further improves the accuracy and robustness of trajectory completion through multi-modal information fusion.

[0063] 4. This application uses a self-supervised learning strategy to improve training efficiency: The present invention adopts a self-supervised training strategy, enabling effective training of the model and achieving trajectory completion even in the case of partial semantic label loss. Especially for the missing trajectories in the dataset, by generating pseudo-labels for other basic datasets and combining the idea of multi-task learning, the model can effectively conduct trajectory completion training without labeled data. Compared with traditional methods, the method of the present invention has higher data utilization efficiency during the training process, further improving the overall performance.

[0064] 5. This application has high calculation accuracy and strong practical application feasibility: Experimental results show that the trajectory completion method proposed by the present invention exhibits excellent accuracy on multiple datasets. Compared with other traditional methods, on the Argoverse dataset, the ADE and FDE are 0.3200 and 0.2352 respectively, and on the Lyft Level 5 dataset, the ADE and FDE are 0.3200 and 0.2350 respectively, showing higher prediction accuracy. This makes the method of the present invention have strong advantages in application scenarios with high accuracy requirements, such as real-time path planning and obstacle avoidance in intelligent transportation systems and autonomous driving vehicles.

[0065] In summary, by innovatively combining road topology information and diffusion probability models, the present invention proposes an efficient and accurate trajectory completion method. Its advantages are not only reflected in accuracy but also in solving the deficiencies of existing methods in complex environments by improving calculation efficiency and resource utilization rate. This method provides more accurate and efficient depth perception capabilities for fields such as intelligent transportation and autonomous driving, and has important practical application value and prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a schematic diagram of the noise prediction network structure of the present invention;

[0067] Figure 2 It is a schematic diagram of the network structure of the present invention;

[0068] Figure 3 It is the trajectory completion result of the Argoverse dataset in the embodiment of the present invention;

[0069] Figure 4 It is the trajectory completion result of the Lyft Level 5 dataset in the embodiment of the present invention;

[0070] Figure 5 It is the trajectory completion result of the nuScenes dataset in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0071] To better understand the purpose, structure, and function of the present invention, the following provides a more detailed description of the present invention in conjunction with the accompanying drawings, so that those skilled in the art can implement it with reference to the text of the specification.

[0072] It should be noted that, unless otherwise specified, the experimental methods described in the following implementation schemes are all conventional methods, and the reagents and materials, unless otherwise specified, can all be obtained from commercial channels; in the description of the present invention, the orientation or positional relationships indicated by the terms "horizontal", "longitudinal", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationships shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as a limitation to the present invention.

[0073] This application decomposes the vehicle trajectory completion problem into a three-stage module. In the first stage, a topological map encoding module is studied to enhance the model's understanding of the topological relationships of complex road networks; in the second stage, a noise prediction network is studied, which fuses the lane context feature information of lane and trajectory features; and in the third stage, the DDMP is used to simulate the forward diffusion process and the reverse process to learn complex spatio-temporal dependencies and generate high-fidelity trajectory data. The accompanying drawings show the network structure studied in this application.

[0074] This application discloses a vehicle trajectory completion method that fuses a road topological map, including the following steps:

[0075] Step 1: Construction of a trajectory completion dataset. In order to use the nuScenes, Argoverse, and Lyft datasets for the trajectory completion task, this application modifies the original motion prediction data and constructs a trajectory completion dataset containing a vectorized high-definition map. It includes: the actually observed trajectory points X0 and its mask matrix M0, the lane nodes A and its lane mask matrix X lane , the lane node adjacency matrix A next . The trajectory points and lane nodes are standardized with the same mean and standard deviation. To simulate the trajectory missing scenario, the trajectory points X0 also include a trajectory point X1 with a missing rate of K% (K takes 30, 50, and 70 respectively in the test), and its mask matrix M.

[0076] Among them, the trajectory points X0 and the mask matrix M0 are used as the input of the vehicle trajectory and mask during the training in Step 5, the lane nodes A and its lane mask matrix M lane are used as the input of the node embedding in Step 2, and the lane node adjacency matrix A next is used as the input of the topological relationship modeling layer in Step 2. The missing trajectory points X1 are used as the true value to evaluate the model.

[0077] Step 2: Construct a topological map encoding module. To enhance the model's understanding of the complex road network topological relationship, this method combines the advantages of sequence processing and graph neural networks, and proposes a "time-series - space" two-stream encoder: First, use a gated recurrent unit to analyze the changing pattern of lane nodes over time. The input of the gated recurrent unit is the sequence feature of lane nodes, and the output is the lane node feature fused with time series. The output is directly input to the dynamic graph attention network. Then, use the dynamic graph attention network to understand the connection structure between roads, and output the lane node feature fused with road structure. The above steps include a node feature embedding layer, a time-series feature encoding layer, a topological relationship modeling layer, and a graph attention aggregation layer.

[0078] (1) Node feature embedding

[0079] Take the vector topological map G(V, E) as the input, where the lane node V is the center line of all roads within a fixed area around the target vehicle, and the edge E includes successor edges E suc and adjacent edges E prox There are two categories. The successor edge E suc is the edge connecting consecutive nodes on the same lane. Multiple successors are allowed at a fork intersection, and multiple predecessors are allowed at a merge intersection, reflecting the natural driving direction of vehicles within the lane; the adjacent edge E prox connects adjacent lanes within the distance threshold. To prevent incorrect connection of lanes in different directions at intersections, a heading angle difference threshold is set, simulating the situation of legal lane changes of vehicles. The lane center line is divided into small segments of fixed length, and each segment (i.e., lane node) is discretized into N pose points. The feature vector of each node is expressed as where each pose point feature includes content such as position, heading angle, traffic control element marker, etc. Perform linear transformation and non-linear mapping on the original lane node features (such as coordinates, direction angle, traffic rules, etc.):

[0080] f (0) = LeakyReLU(W e f + b e )

[0081] Among them, define as the embedding matrix, d in is the dimension of the node of the input feature, d emb is the dimension of the node feature after embedding encoding, f is the original feature of the lane node, LeakyReLu(.) is the activation function, is the embedding representation of the lane feature.

[0082] (2) Time-series feature encoding layer

[0083] Regarding the sequential characteristics of lane nodes, a gated recurrent unit (GRU) is adopted to capture the temporal dependence relationship.

[0084] h (0) = GRU(f (0) )

[0085] where h (0) is the lane feature embedding representation that mixes temporal features;

[0086] (3) Topological relationship modeling layer

[0087] When constructing the adjacency matrix A of lane nodes, it is first initialized with the identity matrix, that is, an edge pointing to itself is added to each node. This self-loop connection ensures that each node retains its own features during aggregation and avoids over-reliance on neighbor nodes. To further capture the spatial topological structure of the lane network, the adjacency matrix is constructed based on the road connection relationship

[0088]

[0089] To maintain the bidirectionality of relationship propagation, the adjacency matrix is transformed into an undirected graph form:

[0090]

[0091] where ∨ represents the element-wise OR operation of two matrices.

[0092] (4) Graph attention aggregation layer

[0093] To model the complex spatial dependence relationship between lane nodes, a multi-layer graph attention network is adopted for feature fusion. Its core is to learn the interaction weights between nodes through the multi-head attention mechanism. Assume that the feature of node i in the l-th layer is The set of neighbor nodes of node i is For each head m ∈ {1, 2,..., M}, queries are generated through the projection matrix keys values

[0094]

[0095] where is the learnable matrix of the m-th head, h is the total dimension of the output of this attention layer, and h / M is the single-head dimension.

[0096] Calculate the attention score of node i to neighbor j, and only allow valid connections marked by the adjacency matrix (A ij = 1) to participate in the calculation:

[0097]

[0098] Among them, represents the original correlation score between node i and neighbor j, reflecting the similarity of the features of the two nodes. represents filtering the original attention calculation range and only calculating the connected part of the two nodes.

[0099] is the probability distribution of the similarity of each node, representing the final attention score between nodes. is an intermediate step, and finally it is to use to calculate

[0100] For each head m, aggregate the value vectors of neighboring nodes:

[0101]

[0102] z i represents the fused feature obtained by aggregating all attention heads;

[0103] At this time, the output dimension of each head is h / M. In order to integrate the features learned by each head, the outputs of all heads are concatenated, that is, the total dimension h is restored. Subsequently, information is integrated through linear projection:

[0104]

[0105] W o represents the linear mapping matrix, which is used to restore the feature dimension of all attention heads obtained by concatenation to the total output hidden layer dimension;

[0106] Finally, add the aggregated neighboring node features to the input features to achieve residual connection:

[0107]

[0108] Among them, represents the feature of node i in the l+1 layer;

[0109] Step 3: Construct a noise prediction network ∈ θ , input the observed trajectory and map encoding, and predict the noise added at each step in the forward diffusion process. By accurately estimating the noise through the noise prediction network, the diffusion model can gradually remove the noise in the reverse process and recover the target trajectory from the random Gaussian noise. This application is improved based on the noise prediction network in CSDI, and a Transformer module for fusing lane and trajectory features, namely lane context, is added, as shown in Figure 1 . Among them, the input of the noise prediction network includes the conditional observation value constructed from the trajectory point X0 and the interpolation target Diffusion step \(t\), map encoding \(G(V, E)\), temporal and spatial information features from trajectory data. and After separately extracting features from the diffusion step \(t\) and concatenating them, the data is fed into a temporal Transformer and a feature Transformer to further extract temporal and spatial features. The output is used as the input to the lane context Transformer, and then concatenated with a time embedding and a feature embedding. After the above process extracts features through 2 residual blocks, the output of the noise prediction network is obtained.

[0110] In the lane context Transformer module, deep fusion of lane features and trajectory sequences is achieved through multiple cascaded Transformer blocks. Self-attention, cross-attention, and feed-forward networks are executed in sequence, and residual connections and normalization are applied at each step to ensure the stability and effectiveness of the model.

[0111] First, the self-attention operation takes the trajectory feature \(X\) (the trajectory feature is from the feature extraction of the original trajectory sequence) as the input, aiming to enable the model to dynamically focus on the correlations between different moments in the trajectory sequence.

[0112] \(X\) (1) = LayerNorm(Attention(Q X , K X , V X )) + K

[0113] where \(X\) (1) represents the feature output by self-attention, and the key \((Q\) X ), value \((K\) X ), and query \((V\) X ) all come from the trajectory feature. LayerNorm(·) represents layer normalization. Attention(·) represents the self-attention operation. Assuming the dimension of each head is \(d\) head , the calculation formula is:

[0114]

[0115] Subsequently, taking the lane feature \(H\) (the lane feature is from the output of step 2) as the context information, the cross-attention operation is performed:

[0116] \(X\) (2) = LayerNorm(Cross - Attention(Q X , K H , V H )) + \(X\) (1) )

[0117] where \(X\) (2)Represents the output of cross-attention, where the values and queries come from lane features and the keys come from trajectory features. In this step, deep interaction is achieved between lane features and trajectory features, enabling effective transmission of lane context information in the trajectory sequence.

[0118] Finally, the fused feature X is further processed by a feed-forward network (2) . The feed-forward network consists of two fully connected layers, and the Gaussian Error Linear Unit (GELU) activation function is applied between the two fully connected layers. The GELU activation function can effectively capture non-linear relationships and enhance the expressive power of the model. The calculation process of the feed-forward network is as follows:

[0119] X (3) = LayerNorm(Linear2(GELU(Linear1(X (2) )))+X (2) )

[0120] where X (3) represents the output of the feed-forward network, and Linear1 and Linear2 are two fully connected layers.

[0121] Each operation step in the lane context Transformer is accompanied by a residual connection and layer normalization. The residual connection ensures that the flow of information in the network is not blocked, and normalization helps to accelerate the training process and prevent gradient explosion or disappearance. Through the deep fusion of self-attention, cross-attention, and the feed-forward network, the model can capture complex dependencies between trajectories and lane features, thereby improving the ability to understand and reconstruct vehicle motion patterns in a dynamic environment.

[0122] In the noise network, the diffusion step number T is set to 50, the diffusion embedding dimension is 128, the minimum noise level β0 is set to 0.0001, and the maximum noise level β T is set to 0.5. The noise level β at any diffusion step is generated by linear interpolation t . The number of residual layers is set to 4, the number of channels in each layer is 64, and 8 heads are used for the calculation of the multi-head attention mechanism. The time embedding dimension of the model is 128, the feature embedding dimension is 16, and context information is generated by concatenating the time embedding and the feature embedding. In the lane context Transformer module, a lane node encoding with a dimension of 32 is used as the context condition, and a single-layer Transformer structure is adopted. Each head of the attention mechanism processes 16-dimensional subspace features. The output dimension of the 1D convolutional layer for processing trajectory data is 64.

[0123] Step 4: Incorporate the noise network into the diffusion model DDPM and learn the data distribution through a progressive noise addition and denoising process. The output of this step is the completed trajectory features.

[0124] 1) Forward diffusion process

[0125] Assume the initial time-series data is x0. The forward diffusion process gradually destroys x0 into Gaussian noise by progressively adding noise. Its essence is a Markov chain. For the time-series data x0, each step of the diffusion process is defined as:

[0126]

[0127] where β t ∈(0,1) represents the variance of the preset random noise, which usually increases monotonically with the time step t. x t represents the noisy data at the t-th step. From q(x t ∣x t-1 ), the joint probability distribution from the initial data x0 to x1, x2,..., x T can be inferred:

[0128]

[0129] By recursively expanding the Markov chain and letting α t =1 - β t , the direct mapping from x0 to x t can be deduced:

[0130]

[0131] where α t represents the cumulative signal retention coefficient controlling the first t steps. As t approaches T, α t approaches 0, and the data x t gradually degenerates into pure noise Therefore, x t at any time step follows a Gaussian distribution:

[0132]

[0133] (2) Reverse denoising process

[0134] The goal of the reverse process is to gradually recover the original data distribution q(x0) from the Gaussian noise θ (x t-1 ∣x t ) by learning the conditional distribution p . Its core lies in constructing a reversible Markov chain and parameterizing the reverse transition probability through a neural network to approximate the true posterior distribution q(x t-1|x t )。

[0135] According to Bayes' theorem and the Markov property of the diffusion process, the true posterior distribution q(x t-1 |x t ) can be written as:

[0136]

[0137] However, the marginal distribution q(x t ) cannot be calculated directly. The diffusion model simplifies the problem by assuming that the transition distribution at each step of the reverse process is Gaussian, i.e.:

[0138]

[0139] where μ θ is associated with the noise prediction network ∈ θ through the reparameterization trick. Specifically, by reversing the forward diffusion process, we get:

[0140]

[0141] Substituting this formula into the transition distribution q(x t-1 |x t , x0) of the forward process, the true posterior mean and standard deviation can be derived as:

[0142]

[0143] Expressing x0 in terms of x t and ∈, we can obtain:

[0144]

[0145] In this way, the goal of the network is transformed into accurately predicting the noise ∈ added at each step, rather than directly predicting the original data x0 or the mean μ θ .

[0146] Step 5: Train the entire network overall:

[0147] During the training process, the input data is the vehicle trajectory data X0 with missing values and the topological structure G(V, E) of the vectorized topological map. The observed values X0 are divided into conditional observed values and imputation targets Specifically, an integer r is randomly drawn from (0, 100), and r% of the observed values X0 are randomly selected as conditional observed values The remaining part is the imputation target To mark which positions in X0 are selected as conditional observed values a conditional mask m is introducedco ∈ {0, 1} K×L , where K represents the feature dimension of the trajectory nodes, and L represents the length of the time series. m co The position where m m co = 0 is Assume the encoded map is G enc , and the goal of the trajectory completion model (i.e., the overall network model composed of steps 2 - 4) is to estimate the true conditional data distribution through the model distribution Estimate the true conditional data distribution Then the forward and backward processes of the diffusion model are as follows:

[0148]

[0149] Among them, In the forward diffusion process, for each diffusion step t, gradually add noise to the interpolation target Add noise:

[0150]

[0151] Train the noise prediction network ∈ by minimizing the loss function θ :

[0152]

[0153] Among them, θ represents the model parameters, denotes taking the expectation of the joint distribution of three random variables. X0 ∼ q(X o ): It means that the input data X0 follows the training set distribution q(X0). denotes that the noise ∈ follows the standard normal distribution. t: Represents the number of diffusion steps.

[0154] In the reverse denoising process, extend the parameterization method in DDPM:

[0155]

[0156] Among them, μ DDPM and σ DDPM are the mean and standard deviation in DDPM. Thus, given the observed value X0 and the trained noise prediction network ∈ θ , through the diffusion model adding and removing noise, output the predicted interpolation target Thereby updating the network. Attached Figure 2 shows the network structure studied in this application.

[0157] The algorithm flow in the training stage is as follows:

[0158] Algorithm: MCTD Training Algorithm

[0159]

[0160] After training is completed, the task in the inference stage is to complete the missing part X1 in the observation value X0. Let the conditional mask be equal to the missing observation mask, i.e., m co = M. At this time, the conditional observation value where ⊙ represents element-wise multiplication. Input the conditional observation value at this time into the model, and the calculated result is output as the final result of trajectory completion, and compared with the trajectory point X1 to evaluate the algorithm performance.

[0161] The effects of the present invention are illustrated by the following experiments:

[0162] (1) Dataset

[0163] To verify the effectiveness of the method, we evaluate on the public datasets Argoverse, Lyft Level 5, and nuScenes.

[0164] The Argoverse dataset was launched by the American autonomous driving company Argo AI in 2019, covering tasks such as motion prediction, 3D object detection, and target tracking. The Argoverse motion prediction dataset is a subset of the Argoverse dataset specifically designed for the trajectory prediction task. In approximately 333,000 scenes, each scene contains the historical trajectory of a target object in the past 2 seconds (10 frames) and the future trajectory in the next 3 seconds (30 frames). The trajectory data includes information such as the category (e.g., vehicle, pedestrian), size, speed, and orientation angle of the target object. In addition, Argoverse also provides vectorized high-precision map data, accurately recording the starting point, ending point of each lane and its connection relationship with other lanes, and also providing other semantic information such as drivable areas, crosswalks, stop signs, and traffic lights. The map data can be accessed using the MapAPI provided by Argoverse official.

[0165] The Lyft Level 5 dataset was collected by Lyft's autonomous vehicles in areas such as Palo Alto, California, USA. It contains over 1000 hours of driving records, covering 170,000 scenarios and over 2500 kilometers of complex road data. The data is segmented into 30 - second segments and efficiently stored in the zarr format for fast reading and writing. Each data segment contains data from multiple sensors such as LiDAR, cameras, and GPS / IMU, providing a detailed description of the surrounding environment. Lyft Level 5 provides a manually annotated high - definition semantic map that includes information such as road structure, lane lines, traffic signs, and the status changes of dynamic traffic lights. In addition, a high - resolution aerial map of the Palo Alto area with a resolution of 8 centimeters per pixel is provided by NearMap. In the experiment, we cut the continuous dataset into five - second segments, with 10 trajectory points per second.

[0166] The nuScenes dataset was launched by automotive manufacturer Aptiv in 2019 and is a large - scale publicly available dataset widely used in the fields of autonomous driving perception, localization, and map construction. The vehicle collecting data is equipped with 6 cameras with a 360° field of view, 1 LiDAR, 5 millimeter - wave radars, and GPS / IMU devices. The data covers a variety of driving environments (urban roads, highways) and weather conditions in Boston, USA and Singapore. The core data of the nuScenes dataset in the trajectory prediction task includes object trajectory data and scene semantic data. The object trajectory data includes information such as the position, speed, acceleration, and object category of all dynamic objects (agents) in each scene within 20 seconds (sampled at 2Hz, a total of 40 frames). The scene semantic data is a vectorized high - definition map and traffic light status data. Among them, the high - definition map describes the topological relationship of the road in a hierarchical structure, divided into independent layers such as lane, road_segment, and ped_crossing.

[0167] (2) Evaluation Metrics

[0168] The Average Displacement Error (ADE) and Final Displacement Error (FDE) are used to evaluate the deviation between the completed trajectory and the ground - truth trajectory.

[0169] ADE is defined as the average of the Euclidean distances between all time steps of the completed trajectory and the corresponding points of the ground - truth trajectory. It reflects the global consistency between the completed trajectory and the ground - truth trajectory. The calculation formula is as follows:

[0170]

[0171] Among them, \(T\) is the total number of time steps of the trajectory, is the completed trajectory coordinates output by the algorithm,

[0172] is the true trajectory coordinates.

[0173] FDE is defined as the Euclidean distance between the end point (the last time step) of the completed trajectory and the end point of the true trajectory, which reflects the long-term trend of the algorithm to infer the trajectory. The calculation formula is as follows:

[0174]

[0175] (3) Experimental evaluation

[0176] To verify the trajectory completion performance of the trajectory completion network proposed in this paper under different missing rates, the completion accuracies with missing rates of 30%, 50%, and 70% were tested on the Argoverse, Lyft Level 5, and nuScenes datasets respectively.

[0177] Table 1 Experimental results of trajectory completion under different missing rates

[0178]

[0179] The experimental results in Table 1 show that among the three datasets of Argoverse, Lyft Level 5, and nuScenes, as the missing rate increases from 30% to 70%, the accuracy of trajectory completion gradually decreases, specifically manifested as an upward trend in both the average displacement error and the final displacement error. This phenomenon indicates that in the case of fewer missing trajectories, the model can more effectively learn the potential patterns of the trajectories, thereby more accurately inferring the missing information from the complete data. Further analysis reveals that there are certain differences in the completion accuracy between different datasets: the overall accuracy of the nuScenes dataset is the lowest, while the overall accuracy of the Lyft Level 5 dataset is the highest, and the accuracy of the Argoverse dataset is between the two. This difference can be attributed to the trajectory lengths of each dataset: the trajectory length of nuScenes is the shortest (not exceeding 25 points), the trajectory length of Lyft Level 5 is the longest (not exceeding 50 points), and the trajectory length of Argoverse is in the middle (not exceeding 40 points). The experimental results show that longer continuous trajectories can provide richer spatio-temporal pattern information for the model, thereby improving the accuracy of trajectory completion.

[0180] Figures 3 to 5The trajectory completion effects of the method in this paper on three public datasets, Argoverse, Lyft Level 5, and nuScenes, are shown. To comprehensively evaluate the model performance, six time-series trajectory samples are randomly selected from each dataset, and the error distributions in the x and y directions are respectively shown. The red crosses in the figure represent the observed values, and the blue dots represent the true values of the missing parts. To quantify the uncertainty of the interpolation results, 100 sets of interpolation results are generated based on the probability model, and the median is used as the interpolation result (green solid line), while the 5% and 95% quantiles are shown as the confidence intervals (green shaded areas).

[0181] In addition, we selected some commonly used deep learning models in the field of trajectory completion to compare with the method proposed in this paper. According to the validation set accuracy, the method proposed in this application is superior to the vast majority of methods and shows comparable performance on other evaluation metrics. The comparison results with other existing methods are shown in Table 2 below. And the average running time of this algorithm is only 0.14 s, with relatively high efficiency, further proving its feasibility in practical applications.

[0182] All baseline models use the same hyperparameters as MCTD and are experimented on the Argoverse and Lyft Level 5 datasets with a missing rate of 50%. The experimental results are shown in Table 2:

[0183] Table 2 Comparison Experimental Results

[0184]

[0185] The above method CSDI is cited from Tashiro Y, Song J, Song Y, et al. Csdi: Conditional score-based diffusion models for probabilistic time series imputation[J] / / Advances in neural information processing systems, 2021, 34: 24804-24816.

[0186] The above method Bi-LSTM is cited from Zhou P, Shi W, Tian J, et al. Attention-based bidirectional long short-term memory networks for relation classification[C] / / Proceedings of the 54th annual meeting of the association for computational linguistics(volume 2:Short papers). 2016:207-212.

[0187] The above method CS-LSTM is cited from Deo N, Trivedi M M. Convolutional social pooling for vehicle trajectory prediction[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 2018:1468-1476.

[0188] The above method ResNet-50 is cited from He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2016:770-778.

[0189] The above method STGAT is cited from Huang Y, Bi H, Li Z, et al. Stgat: Modeling spatial-temporal interactions for human trajectory prediction[C] / / Proceedings of the IEEE / CVF international conference on computer vision. 2019:6272-6281.

[0190] The above method NAOMI is cited from Liu Y, Yu R, Zheng S, et al. Naomi: Non-autoregressive multiresolution sequence imputation[J]. Advances in neural information processing systems, 2019, 32.

[0191] The above method LaneGCN is cited from Liang M, Yang B, Hu R, et al. Learning lane graph representations for motion forecasting[C] / / Computer Vision - ECCV 2020: 16th European Conference, Glasgow, UK, August 23 - 28, 2020, Proceedings, Part II 16. Springer International Publishing, 2020: 541 - 556.

[0192] The above method TrajGAT is cited from Yao D, Hu H, Du L, et al. Trajgat: A graph-based long-term dependency modeling approach for trajectory similarity computation[C] / / Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. 2022: 2275 - 2285.

[0193] On the other hand, the embodiment of the present invention further provides a vehicle trajectory completion system integrating a road topological map, including:

[0194] A processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the vehicle trajectory completion method integrating the road topological map as described in the above technical solution.

[0195] It will be understood that the present invention is described by way of some embodiments, and those skilled in the art will know that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the present invention. Although the embodiments of the present invention have been disclosed as above, it is not limited to the applications listed in the specification and embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily made. Therefore, without departing from the general concept defined by the claims and the equivalent scope, the present invention is not limited to the specific details and the examples shown and described herein.

Claims

1. A vehicle trajectory completion method integrating a road topology map, characterized in that: The steps include: Step 1: Build a trajectory completion dataset; Step 2: Build a topological map encoding module, input lane node sequence features, and after node feature embedding, temporal feature encoding, topological relationship modeling, and graph attention aggregation processing, output lane node features that integrate the road structure for input into the noise prediction network. Step 3: Build a noise prediction network by adding a lane context Transformer module to the existing CSDI for fusing lane and trajectory features. Step 4: Integrate the noise prediction network into the diffusion model network, learn the data distribution through the progressive noise addition and denoising process, and output the completed trajectory features; Step 5: Use the trajectory completion dataset to train the trajectory completion model constructed in steps 2-4, and use the trained model to complete the vehicle trajectory.

2. The vehicle trajectory completion method integrating road topology map as claimed in claim 1, characterized in that: The trajectory completion dataset in step 1 includes the actually observed trajectory point X0 and its mask matrix M0, lane node A and its lane mask matrix M lane , Lane node adjacency matrix A next ; The trajectory points and lane nodes are standardized with the same mean and standard deviation. In order to simulate the trajectory missing scenario, the trajectory point X0 also includes a trajectory point X1 with a missing rate of K%, and its mask matrix M.

3. The vehicle trajectory completion method integrating road topology map as claimed in claim 1, characterized in that: The process of node feature embedding is as follows: The vector topological map G(V,E) is used as input, where the lane node V is the center line of all roads in a fixed area around the target vehicle, and the edge E includes the successor edge E suc and the adjacent edge E prox Two categories; the lane centerline is divided into small segments of fixed length, each segment, i.e., lane node, is discretized into N pose points, and the feature vector of each node is expressed as Each pose point feature includes position, yaw angle, and traffic control element mark; the original lane node features are linearly transformed and nonlinearly mapped: f (0) =LeakyReLU(W e f+b e ) Among them, the definition is the embedding matrix, d in It is the dimension of the node of the input feature, d emb is the dimension of the node feature after embedding encoding, f is the original feature of the lane node, LeakyReLu(.) is the activation function, is the embedded representation of lane features.

4. The vehicle trajectory completion method integrating road topology map as claimed in claim 1, characterized in that: The process of topological relationship modeling is as follows: When constructing the adjacency matrix of lane nodes, we first initialize it with a unit matrix, that is, add an edge pointing to each node. To further capture the spatial topological structure of the lane network, we construct an adjacency matrix based on the road connection relationship. In order to maintain the bidirectionality of relationship propagation, the adjacency matrix is ​​converted into an undirected graph: Here, ∨ indicates that two matrices are element-wise ORed.

5. The vehicle trajectory completion method integrating road topology map as claimed in claim 4, characterized in that: The processing process of graph attention aggregation is as follows: Assume that the feature of node i in layer l is The set of neighboring nodes of node i is For each head m∈{1,2,...,M}, the query is generated by the projection matrix key value in, is the learnable matrix of the mth head, h is the total dimension of the attention layer output, and h / M is the single head dimension; Calculate the attention score of node i to neighbor j, allowing only valid connections marked by the adjacency matrix to participate in the calculation: in, represents the original correlation score between node i and neighbor j, reflecting the similarity of the features of the two nodes. Indicates filtering the original attention calculation range and only calculating the part where two nodes are connected; is the probability distribution of the similarity of each node, indicating the final attention score between nodes; for each head m, the value vector of the aggregated neighboring nodes: z i Represents the fusion features obtained by aggregating all attention heads; At this point, the output dimension of each head is h / M, and then the outputs of all heads are concatenated to restore the total dimension h, and then the information is integrated through linear projection: W o represents the linear mapping matrix, which is used to restore the feature dimensions of all concatenated attention heads to the total dimension of the output hidden layer; Finally, the aggregated neighborhood node features are added to the input features to achieve residual connection: in, Represents the features of node i at the l+1th layer.

6. The vehicle trajectory completion method integrating road topology map as claimed in claim 1, characterized in that: The input of the noise prediction network includes the conditional observation constructed by the trajectory point X0 and interpolation target Diffusion step t, map encoding G(V,E), which is the temporal and spatial information features of trajectory data, where V is the lane node and E is the edge; and After extracting features in the diffusion step t, they are concatenated and sent to the time series Transformer and feature Transformer to further extract time series and spatial features. The output is used as the input of the lane context Transformer, and then concatenated with time embedding and feature embedding. Finally, after extracting features through two residual blocks, the output of the noise prediction network is obtained.

7. The vehicle trajectory completion method of integrating road topology map according to claim 1 or 6, characterized in that: In the lane context Transformer module, the deep fusion of lane features and trajectory sequences is achieved through multiple cascaded Transformer blocks, which sequentially execute self-attention, cross-attention and feed-forward networks; First, the self-attention operation takes the trajectory feature X as input, and the processing process is expressed as: X (1) =LayerNorm(Attention(Q X ,K X ,V X )+X) Among them, X (1) Represents the features of the self-attention output, key Q X , value K X 、Query V X All are derived from trajectory features, LayerNorm(·) represents layer normalization; Attention(·) represents self-attention operation, assuming that the dimension of each head is d head , the calculation formula is: Then, the lane feature H is used as context information to perform a cross-attention operation: X (2) =LayerNorm(Cross-Attention(Q X ,K H ,V H )+X (1) ) Among them, X (2) Represents the output of cross attention, where the value and query come from lane features, while the key comes from trajectory features; Finally, the fused feature X( 2 ); The feedforward network consists of two fully connected layers, and the Gaussian error linear unit GELU activation function is applied between the two fully connected layers. The calculation process of the feedforward network is as follows: X (3) =LayerNorm(Linear2(GELU(Linear1(X (2) )))+X (2) ) Among them, X (3) Represents the output of the feedforward network, Linear1 and Linear2 are two fully connected layers.

8. The vehicle trajectory completion method integrating road topology map as claimed in claim 2, characterized in that: During the training process, the input data is the vehicle trajectory data X0 with missing values ​​and the topological structure G(V,E) of the vectorized topological map, where V is the lane node and E is the edge; the observation value X0 is divided into conditional observation values ​​using a random sampling strategy and interpolation target Specifically, an integer r is randomly selected from (0,100), and r% of the observations X0 are randomly selected as conditional observations. The remaining part is the interpolation target To mark which positions in X0 are selected as conditional observations Introducing conditional mask m co ∈{0,1},m co =1 is the position m co =0 is the position Assume that the encoded map is G enc , obtained from step 2, the goal of the trajectory completion model is to distribute Estimate the true conditional data distribution In the forward diffusion process, for each diffusion step t, the interpolation target Add noise, in the reverse denoising process, expand the parameterization method in the diffusion model DDPM; and train the noise prediction network ∈ by minimizing the loss function θ : Among them, θ is the model parameter, It means taking the expectation of the joint distribution of three random variables, X0~q(X0) means that the input data X0 follows the distribution of the training set Indicates that the noise ∈ follows the standard normal distribution; Thus, given the observation value X0 and the trained noise prediction network ∈ θ , through the diffusion model denoising, output prediction interpolation target 9. The vehicle trajectory completion method integrating road topology map as claimed in claim 8, characterized in that: After training is completed, the task of the inference phase is to complete the missing part X1 in the observation value X0, and let the conditional mask equal the missing observation mask, that is, m co =M, then the conditional observation value Where ⊙ represents element-by-element multiplication. The conditional observation value at this time is input into the trained model, and the calculation result is output as the final result of trajectory completion.

10. The vehicle trajectory completion system integrating road topology map is characterized by: include: A processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the vehicle trajectory completion method for integrating road topology maps as described in any one of claims 1 to 9.

Images