Track prediction method based on dual-view fusion
Patent Information
- Application Number
- CN202511037554.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-28
- Publication Date
- 2025-11-14
AI Technical Summary
Existing trajectory prediction methods only construct the map from one perspective, making it difficult to capture remote or irregular relationships in complex scenes, resulting in insufficient accuracy in interactive modeling.
A trajectory prediction method based on dual-view fusion is adopted to acquire road and proxy data from global and local perspectives respectively, construct sparse graph structure and fully connected graph structure, and perform feature-level integration through a multi-stage interactive fusion mechanism to predict multiple candidate endpoints and generate predicted trajectories.
It improves the accuracy of interactive modeling, enhances the ability to perceive dynamic changes over time, and improves the accuracy of trajectory prediction, especially performing well in multi-objective decision-making tasks.
Smart Images
Figure CN120953933A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of trajectory prediction technology, specifically relating to a trajectory prediction method based on dual-view fusion. Background Technology
[0002] With the rapid development of intelligent driving, autonomous delivery robots, and multi-agent collaborative systems, higher demands are being placed on the ability to predict the future trajectories of dynamic agents. The core objective of trajectory prediction technology is to accurately predict the motion trends of multiple agents over several future time steps, given known historical trajectories and environmental information. This technology is widely used in key tasks such as autonomous driving decision-making and planning, traffic risk warning, and intelligent monitoring behavior analysis, and represents a core research problem in the field of intelligent systems.
[0003] In real-world, complex traffic environments, the behavior of intelligent agents is highly uncertain, and their decision-making process is influenced by a variety of factors, including: the behavior of other vehicles or pedestrians in the vicinity; the road network structure (lane direction, drivability, intersections, etc.); and the current behavioral intention (whether to turn, whether to accelerate, etc.).
[0004] To address this, researchers have proposed end-to-end trajectory prediction methods based on deep learning in recent years, which can be mainly categorized as follows: interactive modeling methods based on single-view graph neural networks, fully connected graph modeling methods based on Transformer, and map-guided trajectory prediction methods.
[0005] The main problem with current algorithms is the lack of a single interactive graph structure: most methods only construct the graph from one perspective, making it difficult to capture remote or irregular relationships in complex scenes.
[0006] To address the aforementioned problems, this invention proposes a trajectory prediction method based on a dual-view structural modeling and multi-stage interactive fusion mechanism. Summary of the Invention
[0007] In view of this, the present invention provides a trajectory prediction method based on dual-view fusion to solve the problem that existing methods only construct the image from one viewpoint and are difficult to capture remote or irregular relationships in complex scenes.
[0008] To achieve the above objectives, the present invention provides the following technical solution:
[0009] This invention provides a trajectory prediction method based on dual-view fusion, comprising the following steps:
[0010] Obtain road data and proxy data from both global and local perspectives;
[0011] Global road node features and global proxy node features are obtained based on road data and proxy data from a global perspective, and local road node features and local proxy node features are obtained based on road data and proxy data from a local perspective.
[0012] Based on the sparse graph structure, the time-sensitive features of the proxy nodes are obtained by splicing the local road node features and the local proxy node features.
[0013] By filtering global road node features and global proxy node features, the target road node features and target proxy node features are obtained;
[0014] By fusing time-sensitive features, target road node features, and target proxy node features, the deep features of the proxy node are obtained.
[0015] Multiple candidate endpoints are predicted based on the depth features of the proxy nodes, and the predicted trajectory is obtained by concatenating the depth features of the proxy nodes and the multiple candidate endpoints.
[0016] As an embodiment of the present invention, global road node features are obtained based on road data from a global perspective, including:
[0017] The road data from the global perspective is projected into a high-dimensional space through linear processing to obtain the initial features, as shown below:
[0018] x lane (0) =ReLU(LN(Wx) lane ))
[0019] In the formula, W represents the learnable parameters used to perform the linear transformation, ReLU() represents the activation function, LN() represents the layer normalization, and x lane This represents road data from a global perspective, x lane (0) Indicates initial features;
[0020] The initial features are processed using an MLP to obtain enhanced features, as shown below:
[0021]
[0022] In the formula, f() represents a multilayer perceptron. This represents the concatenation operation, x′ lane Indicates enhanced features;
[0023] The road data and enhanced features are added together and then normalized to obtain the final encoding result, as shown below:
[0024] x lane (1) =Norm(x lane +x′lane )
[0025] In the formula, x lane (1) This represents the final encoding result, and Norm() represents normalization;
[0026] Global max pooling is performed on all final encoding results to obtain global road node features, as shown below:
[0027]
[0028] In the formula, The road node features are represented by and max() represents the max pooling operation.
[0029] As an embodiment of the present invention, global proxy node characteristics are obtained based on proxy node data from a global perspective, including:
[0030] By combining a feature pyramid network with a one-dimensional convolutional neural network, feature extraction is performed on the agent data from a global perspective to obtain the global agent node features, as shown below:
[0031]
[0032] In the formula, Let FPN() represent the feature pyramid network, Res1D() represent one-dimensional convolution, l represent the scale, and x represent the surrogate node features. agent Represents proxy node data, x agent (0) This indicates the initial characteristics of the proxy node.
[0033] As an embodiment of the present invention, based on a sparse graph structure, local road node features and local proxy node features are spliced together to obtain time-sensitive features of the proxy nodes, including:
[0034] Based on a pre-defined sparse graph construction strategy, a sparse graph structure is constructed, and each edge of the sparse graph structure is determined. Each edge of the sparse graph structure represents the relative positional relationship between two nodes at each historical moment in each frame, as shown below:
[0035]
[0036] In the formula, This represents the relative positional relationship between nodes i and j at time t. θ represents the distance between nodes i and j, θ represents the first orientation angle between nodes i and j, and t represents time. This represents the second orientation angle between nodes i and j;
[0037] Encoding is performed based on a multi-level sine and cosine time coding mechanism, and the encoded result is shown below:
[0038]
[0039] In the formula, This represents the value of the 2m-th encoded dimension at time t. This represents the value of the (2m+1)th encoding dimension at time t, where m represents a set of indices for the encoding dimension pairs, and d represents the total dimension of the encoding vector.
[0040] By concatenating the encoded result with each edge of the sparse graph structure, enhanced edge features are obtained. These features are then processed through linear transformation and layer normalization to obtain the edge embedding vector.
[0041] Based on the sparse graph structure and edge embedding vectors, the time-sensitive features of the proxy nodes are obtained as follows:
[0042]
[0043] In the formula, H represents the local proxy node features, GAT() represents the graph attention mechanism, ReLU() represents the activation function, and LayerNorm() represents layer normalization. This indicates the relative positional relationship after it has been encoded by the time encoder. This indicates a time-sensitive characteristic.
[0044] As an embodiment of the present invention, global road node features and global proxy node features are filtered to obtain target road node features and target proxy node features, including:
[0045] Based on the attention mechanism, global road node features are selected from all global road node features and global proxy node features to obtain initial global road node features;
[0046] Based on the attention mechanism, global road node features are filtered from all initial global road node features to obtain target global road node features;
[0047] Based on the attention mechanism, the global proxy node features are selected from all target global road node features and global proxy node features to obtain the initial global proxy node features;
[0048] Based on the attention mechanism, global proxy node features are filtered out from all initial global proxy node features to obtain target global proxy node features.
[0049] As an embodiment of the present invention, the proxy node deep features are obtained by fusing time-sensitive features, target road node features, and target proxy node features, including:
[0050] By fusing the target global proxy node features and the time-sensitive features of the proxy nodes, the proxy node fusion result is obtained, as shown below:
[0051]
[0052] In the formula, H fuison This indicates the fusion result of the proxy nodes. The target global proxy node features are represented by MHA(), which represents the multi-head attention mechanism.
[0053] By fusing the proxy node fusion results and the target global road node features, the deep features of the proxy nodes are obtained, as shown below:
[0054]
[0055] In the formula, Indicates the depth feature of the proxy node. This represents the global road node features of the target.
[0056] As an embodiment of the present invention, predicting candidate endpoints based on the depth features of proxy nodes, and concatenating the depth features of proxy nodes and candidate endpoints to obtain a predicted trajectory includes:
[0057] Based on the attention mechanism, the deep features of all agent nodes are sequentially filtered three times and then processed by a single layer to obtain enhanced deep features.
[0058] The enhanced deep features are input into the MLP to predict multiple candidate endpoints. Each candidate endpoint is then used by the MLP to predict its offset, resulting in multiple target candidate endpoints, as shown below:
[0059]
[0060] In the formula, E represents the candidate endpoint. This indicates the candidate endpoint for the target, and Flatten() indicates that the dimensions are flattened. This indicates enhanced deep features;
[0061] After concatenating multiple target candidate endpoints with the deep features of proxy nodes, the input is given to the MLP to predict the relative offset sequence and obtain the predicted offset sequence.
[0062] The predicted offset sequence is concatenated with the corresponding candidate endpoint to obtain the predicted trajectory.
[0063] As an embodiment of the present invention, the attention mechanism is as follows:
[0064]
[0065] In the formula, Q, K, and V all represent inputs, ATTENTION() represents the attention mechanism, and Softmax() represents the activation function. Indicates the dimension of the Key vector.
[0066] The beneficial effects of this invention are as follows: It achieves higher interaction modeling accuracy by employing a dual-view graph modeling structure, simultaneously constructing a sparse graph structure centered on agent nodes and a fully connected graph structure centered on the global scene, and introducing a cross-view fusion module for feature-level integration. This solves the information bias risk inherent in single-view graph building schemes, such as incomplete adjacency relationships or redundant connections. By introducing two types of complementary graph structures, this invention achieves synergistic optimization of local behavior modeling and global semantic modeling, significantly improving the accuracy of interaction modeling between agents.
[0067] The model exhibits enhanced perception of temporal and sequential dynamic changes. During graph modeling between agent nodes, a Relative Position Encoding (RPE) based on multi-frame positional relationships is constructed, and multi-frequency sine and cosine temporal embeddings are used to enhance edge features. Subsequently, an edge-aware graph attention mechanism is introduced to model node relationships. This addresses the issue of traditional graph neural networks neglecting the evolution of interactions over time, leading to decreased prediction performance in complex scenarios. By constructing time-sensitive edge embeddings, the model demonstrates significant advantages in perceiving the intensity and directionality of dynamic interactions.
[0068] This method achieves higher trajectory prediction accuracy and covers multimodal targets: It employs a target-point-guided trajectory decoder structure, first generating multiple endpoint candidates, and then generating the complete trajectory based on the endpoint guidance. Furthermore, it enhances the discriminative ability between multimodal output branches by designing a maximum marginal margin classification loss. This addresses the problem of traditional decoding methods lacking structured control over trajectory endpoints, which easily leads to predicted trajectory divergence or distortion. By structurally predicting endpoints and separating the modeling of endpoints from the trajectory, this method improves the consistency between the trajectory and the target, making it particularly suitable for multi-objective decision-making tasks.
[0069] Other advantages, objectives, and features of the invention will be set forth in the following description and will be apparent to those skilled in the art in some respects, or may be learned by practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description
[0070] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration:
[0071] Figure 1 This is a general structural diagram of the present invention;
[0072] Figure 2 This is a schematic diagram of a map encoder designed for processing road data according to the present invention;
[0073] Figure 3 This is a schematic diagram of the trajectory encoder designed according to the present invention for processing proxy data;
[0074] Figure 4 This is a schematic diagram of the relative position encoding parameters of the present invention;
[0075] Figure 5 This is a schematic diagram of the process of the present invention. Detailed Implementation
[0076] like Figures 1-5 As shown, this invention provides a trajectory prediction method based on dual-view fusion, comprising the following steps:
[0077] S1: Obtain road data and proxy data from both the global and local perspectives;
[0078] S2: Obtain global road node features and global proxy node features based on road data and proxy data from a global perspective, and obtain local road node features and local proxy node features based on road data and proxy data from a local perspective;
[0079] S3: Based on the sparse graph structure, splice the local road node features and local proxy node features to obtain the time-sensitive features of the proxy nodes;
[0080] S4: Filter global road node features and global proxy node features to obtain target road node features and target proxy node features;
[0081] S5: Merge time-sensitive features, target road node features, and target proxy node features to obtain proxy node deep features;
[0082] S6: Predict multiple candidate endpoints based on the depth features of the proxy nodes, and concatenate the depth features of the proxy nodes and the multiple candidate endpoints to obtain the predicted trajectory.
[0083] The working principle of the above technical solution is as follows: In the actual prediction process, firstly, road data and proxy data are acquired from both global and local perspectives. The input size of the road data is (N, 12), where N represents the number of road nodes, and 12 represents: 1) center point coordinates (x, y); 2) local direction vectors (x direction, y direction); 3) turning signs (left turn, right turn); 4) whether traffic control is in place; 5) whether there is an intersection; 6) whether there is a left adjacent lane; 7) whether there is a right adjacent lane, etc. The input size of the proxy nodes is (K, 40), where K represents the number of proxy nodes, and 40 represents 2×20, i.e., 20 steps of historical trajectory data, each containing (x, y) coordinates. It should be noted that the difference between the road data and proxy data from the global and local perspectives lies in the different coordinates within the data, i.e., local and global coordinates respectively. The local coordinates are respectively... Each proxy node is used as the midpoint, and the coordinates of other nodes are transformed accordingly. Then, global road node features and global proxy node features are obtained from the road data and proxy data from a global perspective, respectively; local road node features and local proxy node features are obtained from the road data and proxy data from a local perspective, with the same processing method for both. Next, based on a sparse graph structure, the local road node features and local proxy node features are concatenated to obtain the time-sensitive features of the proxy nodes. Then, based on a fully connected graph structure, the global road node features and global proxy node features are filtered to obtain the target road node features and target proxy node features. Finally, the time-sensitive features, target road node features, and target proxy node features are fused to obtain the proxy node depth features. Multiple candidate endpoints are predicted based on the proxy node depth features, and the predicted trajectory is obtained by concatenating the proxy node depth features and multiple candidate endpoints. Vehicles, pedestrians, etc., are collectively referred to as proxies.
[0084] The beneficial effects of the above technical solution are as follows: By adopting a dual-view graph modeling structure with higher interaction modeling accuracy, a sparse graph structure centered on agent nodes (Agent-Centric Graph) and a fully connected graph structure centered on the global scene (Scene-Centric Graph) are constructed simultaneously, and a cross-view fusion module is introduced for feature-level integration. This solves the problem that existing methods only construct graphs from one viewpoint, making it difficult to capture remote or irregular relationships in complex scenes. At the same time, it avoids the information bias risk of single-view graph construction schemes, such as incomplete adjacency relationships or redundant connections. By introducing two types of complementary graph structures, this invention achieves synergistic optimization of local behavior modeling and global semantic modeling, significantly improving the accuracy of interaction modeling between agents.
[0085] In one embodiment, global road node features are obtained based on road data from a global perspective, including:
[0086] The road data from the global perspective is projected into a high-dimensional space through linear processing to obtain the initial features, as shown below:
[0087] x lane (0) =ReLU(LN(Wx) lane ))
[0088] In the formula, W represents the learnable parameters used to perform the linear transformation, ReLU() represents the activation function, LN() represents the layer normalization, and x lane This represents road data from a global perspective, x lane (0) Indicates initial features;
[0089] The initial features are processed using an MLP to obtain enhanced features, as shown below:
[0090]
[0091] In the formula, f() represents a multilayer perceptron. This represents the concatenation operation, x′ lane Indicates enhanced features;
[0092] The road data and enhanced features are added together and then normalized to obtain the final encoding result, as shown below:
[0093] x lane (1) =Norm(x lane +x′ lane )
[0094] In the formula, x lane (1) This represents the final encoding result, and Norm() represents normalization;
[0095] Global max pooling is performed on all final encoding results to obtain global road node features, as shown below:
[0096]
[0097] In the formula, The road node features are represented by and max() represents the max pooling operation.
[0098] The working principle of the above technical solution is as follows: Global road data processing is mainly divided into two stages: Stage 1, structure preservation and initial feature enhancement; linear processing (i.e., fully connected layers) is used to project the original input features into a high-dimensional space to obtain the initial features; such as... Figure 2As shown, an MLP is then used to map the initial features, while retaining the structural features after global max pooling, to obtain enhanced features. Stage two is semantic enhancement and fusion normalization; that is, the road data from the global perspective in the previous stage is added to the enhanced features and normalized to form the final encoding result. Finally, global max pooling is performed on all nodes to output the global road node features within the scene range. The size of the processed road node is (N, 128). It should be noted that the same processing method is used to process the road data from the local perspective to obtain local road node features. The specific process will not be elaborated here.
[0099] In one embodiment, obtaining global proxy node characteristics based on proxy node data from a global perspective includes:
[0100] By combining a feature pyramid network with a one-dimensional convolutional neural network, feature extraction is performed on the agent data from a global perspective to obtain the global agent node features, as shown below:
[0101]
[0102] In the formula, Let FPN() represent the feature pyramid network, Res1D() represent one-dimensional convolution, l represent the scale, and x represent the surrogate node features. agent Represents proxy node data, x agent (0) This indicates the initial characteristics of the proxy node.
[0103] The working principle of the above technical solution is as follows: a Feature Pyramid Network (FPN) is used as the backbone structure, combined with a one-dimensional convolutional neural network (1D CNN) to perform preliminary feature extraction on proxy data with a small global perspective, thereby achieving effective encoding of multi-scale temporal features in historical trajectories; in the above formula, at each scale l, a one-dimensional residual convolutional module Res1D() is used to extract temporal features, which are then fused by the FPN at multiple scales to finally generate multi-scale global proxy node features; it should be noted that the same processing method is used to process proxy data at a local perspective to obtain local road node features, the specific process of which will not be elaborated here.
[0104] In one embodiment, based on a sparse graph structure, local road node features and local proxy node features are concatenated to obtain time-sensitive features of the proxy nodes, including:
[0105] Based on a pre-defined sparse graph construction strategy, a sparse graph structure is constructed, and each edge of the sparse graph structure is determined. Each edge of the sparse graph structure represents the relative positional relationship between two nodes at each historical moment in each frame, as shown below:
[0106]
[0107] In the formula, This represents the relative positional relationship between nodes i and j at time t. θ represents the distance between nodes i and j, θ represents the first orientation angle between nodes i and j, and t represents time. This represents the second orientation angle between nodes i and j;
[0108] Encoding is performed based on a multi-level sine and cosine time coding mechanism, and the encoded result is shown below:
[0109]
[0110] In the formula, This represents the value of the 2m-th encoded dimension at time t. This represents the value of the (2m+1)th encoding dimension at time t, where m represents a set of indices for the encoding dimension pairs, and d represents the total dimension of the encoding vector.
[0111] By concatenating the encoded result with each edge of the sparse graph structure, enhanced edge features are obtained. These features are then processed through linear transformation and layer normalization to obtain the edge embedding vector.
[0112] Based on the sparse graph structure and edge embedding vectors, the time-sensitive features of the proxy nodes are obtained as follows:
[0113]
[0114] In the formula, H represents the local proxy node features, GAT() represents the graph attention mechanism, ReLU() represents the activation function, and LayerNorm() represents layer normalization. This indicates the relative positional relationship after it has been encoded by the time encoder. This indicates a time-sensitive characteristic; it is worth noting that, H (0) and Both represent quantities related to the reasoning process and have no specific meaning;
[0115] The working principle of the above technical solution is as follows: by processing the features of local proxy nodes and local road nodes from a local perspective, the two are spliced together to obtain the total node, which has a size of (N, 128), where N is the total number of nodes.
[0116] like Figure 4As shown, when constructing the sparse graph, edge feature design is performed. Each edge of the sparse graph represents the relative positional relationship between two nodes at each historical moment in each frame, including: relative Euclidean distance; orientation difference (angle); relative direction (orientation vector difference); and RPE relative position encoding. The sparse graph structure construction strategy is to use each agent node as a reference center, calculate its average Euclidean distance to other nodes throughout the entire observation time window, and retain the K nearest neighbors with the smallest distance for each node, constructing a K-nearest neighbor (KNN) graph structure to form a sparse interaction graph. This method simulates the nearest neighbor attention mechanism of human driving, reduces noise caused by redundant edge connections, and improves the model. To improve computational efficiency, a multi-frequency sine and cosine time coding mechanism is introduced through a time encoder to encode the time steps and obtain the coding result. The coding result is concatenated with the relative position features to form enhanced edge features, which are input into a multi-head attention module. Through linear transformation and layer normalization, edge embedding vectors are obtained. Then, graph attention modeling is used to apply an edge-aware graph attention network on the sparse graph, and edge embedding is used to adjust the attention weights. A multi-layer stacked structure + residual connection + layer normalization is adopted to output the time-sensitive features of the proxy nodes. Finally, the time-sensitive features of each proxy node are obtained, with a size of (N, 128), where N represents the number of proxy nodes and 128 is the number of features.
[0117] The beneficial effects of the above technical solution are as follows: By enhancing the ability to perceive dynamic changes over time, a relative position encoding (RPE) based on multi-frame positional relationships is constructed during the graph modeling process between agent nodes. Multi-frequency sine and cosine time embedding is used to enhance edge features, and then an edge-aware graph attention mechanism is introduced to model node relationships. Compared with traditional graph neural networks that often ignore the evolution of interactions over time, leading to a decrease in prediction performance in complex scenarios (such as turning and merging), this technical solution, by constructing time-sensitive edge embeddings, gives the model a significant advantage in perceiving the intensity and directionality of dynamic interactions.
[0118] In one embodiment, global road node features and global proxy node features are filtered to obtain target road node features and target proxy node features, including:
[0119] Based on the attention mechanism, global road node features are selected from all global road node features and global proxy node features to obtain initial global road node features;
[0120] Based on the attention mechanism, global road node features are filtered from all initial global road node features to obtain target global road node features;
[0121] Based on the attention mechanism, the global proxy node features are selected from all target global road node features and global proxy node features to obtain the initial global proxy node features;
[0122] Based on the attention mechanism, global proxy node features are filtered out from all initial global proxy node features to obtain target global proxy node features;
[0123] The attention mechanism is as follows:
[0124]
[0125] In the formula, Q, K, and V all represent inputs, ATTENTION() represents the attention mechanism, and Softmax() represents the activation function. Indicates the dimension of the Key vector.
[0126] The working principle and beneficial effects of the above technical solution are as follows: This technical solution processes the global proxy node features and global road node features under global coordinates. Specifically, it mainly relies on a multi-head attention module. First, the two types of nodes are processed by the attention module, and then the result is processed by the attention module again to obtain the target road node features, with a size of (M, 128), where M is the number of road nodes. The obtained road nodes and proxy nodes are processed by the attention module again, and then the result is processed by the attention module for the last time to obtain the target proxy node features, with a size of (N, 128), where N is the number of proxy nodes. Specifically, during the filtering process through the attention mechanism, for example, the input and output of the four attention modules are as follows: [(road node features, proxy node features, proxy node features), road node feature 1], [(road node features 1, road node features 1, road node features 1), target road node features], [(proxy node features, target road node features, target road node features), proxy node features 1], [(proxy node features 1, proxy node features 1, proxy node features 1), target proxy node features. The target road node feature size is (M, 128), where M is the number of road node features; the target proxy node feature size is (N, 128), where N is the number of proxy node features.
[0127] In one embodiment, the deep features of the proxy node are obtained by fusing time-sensitive features, target road node features, and target proxy node features, including:
[0128] By fusing the target global proxy node features and the time-sensitive features of the proxy nodes, the proxy node fusion result is obtained, as shown below:
[0129]
[0130] In the formula, H fuison This indicates the fusion result of the proxy nodes. This indicates the characteristics of the target global proxy node; what does MHA() represent?
[0131] By fusing the proxy node fusion results and the target global road node features, the deep features of the proxy nodes are obtained, as shown below:
[0132]
[0133] In the formula, Indicates the depth feature of the proxy node. This represents the global road node features of the target.
[0134] The working principle and beneficial effects of the above technical solution are as follows: Features from a local perspective focus on behavioral details, while features from a global perspective focus on structural stability. A single perspective may result in information loss and bias, so it is necessary to fuse the two types of features. The entire fusion process is divided into two steps: the first stage is to fuse the local output and the global output; the second stage is to interact and model the fusion result with the global map nodes to further improve the agent-map relationship modeling and obtain the agent's deep features.
[0135] In one embodiment, predicting candidate endpoints based on the depth features of proxy nodes, and concatenating the depth features of proxy nodes and candidate endpoints to obtain the predicted trajectory includes:
[0136] Based on the attention mechanism, the deep features of all agent nodes are sequentially filtered three times and then processed by a single layer to obtain enhanced deep features.
[0137] The enhanced deep features are input into the MLP to predict multiple candidate endpoints. Each candidate endpoint is then used by the MLP to predict its offset, resulting in multiple target candidate endpoints, as shown below:
[0138]
[0139] In the formula, E represents the candidate endpoint. This indicates the candidate endpoint for the target, and Flatten() indicates that the dimensions are flattened. This indicates enhanced deep features;
[0140] After concatenating multiple target candidate endpoints with the deep features of proxy nodes, the input is given to the MLP to predict the relative offset sequence and obtain the predicted offset sequence.
[0141] The predicted offset sequence is concatenated with the corresponding candidate endpoint to obtain the predicted trajectory.
[0142] The working principle of the above technical solution is as follows: This technical solution enhances the temporal context awareness by processing the fused proxy node deep features through three self-attention modules and a single-layer normalization process; the attention mechanism in this embodiment is the same as the attention mechanism described above; then, through the provided target point prediction module, the enhanced representation is flattened and fed into the MLP to predict multiple candidate endpoints, and each candidate endpoint is further fine-tuned by the MLP to adjust the predicted offset; finally, through the trajectory generation module, the endpoint and proxy representation are concatenated and input into the MLP to predict the relative offset sequence, and then the endpoint coordinates are concatenated to restore the complete predicted trajectory. The final output result is (N, 60), where N is the host of the proxy node, 60 is the predicted trajectory for 30 frames, and each trajectory contains (x, y) coordinates;
[0143] The beneficial effects of the above technical solution are as follows: The solution achieves higher trajectory prediction accuracy and covers multimodal targets. It employs a target-point-guided trajectory decoder structure, first generating multiple endpoint candidates, and then generating a complete trajectory based on the endpoint guidance. Furthermore, it enhances the discriminative ability between multimodal output branches by designing a maximum marginal margin classification loss. This avoids the lack of structured control over trajectory endpoints in traditional decoding methods, which can easily lead to trajectory divergence or distortion. This method improves the consistency between the trajectory and the target by structurally predicting the endpoint and separating the modeling endpoint from the trajectory, making it particularly suitable for multi-target decision-making tasks.
[0144] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.
Claims
1. A trajectory prediction method based on dual-view fusion, characterized in that, Includes the following steps: Obtain road data and proxy data from both global and local perspectives; Global road node features and global proxy node features are obtained based on road data and proxy data from a global perspective, and local road node features and local proxy node features are obtained based on road data and proxy data from a local perspective. Based on the sparse graph structure, the time-sensitive features of the proxy nodes are obtained by splicing the local road node features and the local proxy node features. By filtering global road node features and global proxy node features, the target road node features and target proxy node features are obtained; By fusing time-sensitive features, target road node features, and target proxy node features, the deep features of the proxy node are obtained. Multiple candidate endpoints are predicted based on the depth features of the proxy nodes, and the predicted trajectory is obtained by concatenating the depth features of the proxy nodes and the multiple candidate endpoints.
2. The trajectory prediction method based on dual-view fusion according to claim 1, characterized in that, Global road node features are obtained from road data from a global perspective, including: The road data from the global perspective is projected into a high-dimensional space through linear processing to obtain the initial features, as shown below: x lane (0) =ReLU(LN(Wx lane )) In the formula, W represents the learnable parameters used to perform linear transformation, ReLU() represents the activation function, LN() represents layer normalization, and x lane This represents road data from a global perspective, x lane (0) Indicates initial features; The initial features are processed using an MLP to obtain enhanced features, as shown below: In the formula, f() represents a multilayer perceptron. This represents the concatenation operation, x′ lane Indicates enhanced features; The road data and enhanced features are added together and then normalized to obtain the final encoding result, as shown below: x lane (1) =Norm(x lane +x′ lance ) In the formula, x lane (1) This represents the final encoding result; Norm() represents normalization. Global max pooling is performed on all final encoding results to obtain global road node features, as shown below: In the formula, This represents the characteristics of road nodes, and max() represents the max pooling operation.
3. The trajectory prediction method based on dual-view fusion according to claim 1, characterized in that, Global proxy node characteristics are derived from proxy node data from a global perspective, including: By combining a feature pyramid network with a one-dimensional convolutional neural network, feature extraction is performed on the agent data from a global perspective to obtain the global agent node features, as shown below: x agent (0) =x agent In the formula, The surrogate node features are represented by FPN(), the feature pyramid network is represented by Res1D(), and the scale is represented by l. gent Represents proxy node data, x agent (0) This indicates the initial characteristics of the proxy node.
4. The trajectory prediction method based on dual-view fusion according to claim 1, characterized in that, Based on the sparse graph structure, local road node features and local proxy node features are concatenated to obtain the time-sensitive features of the proxy nodes, including: Based on a pre-defined sparse graph construction strategy, a sparse graph structure is constructed, and each edge of the sparse graph structure is determined. Each edge of the sparse graph structure represents the relative positional relationship between two nodes at each historical moment in each frame, as shown below: In the formula, This represents the relative positional relationship between nodes i and j at time t. θ represents the distance between nodes i and j, θ represents the first orientation angle between nodes i and j, and t represents time. This represents the second orientation angle between nodes i and j; Encoding is performed based on a multi-level sine and cosine time coding mechanism, and the encoded result is shown below: In the formula, This represents the value of the 2m-th encoded dimension at time t. This represents the value of the (2m+1)th encoding dimension at time t, where m represents a set of indices for the encoding dimension pairs, and d represents the total dimension of the encoding vector. By concatenating the encoded result with each edge of the sparse graph structure, enhanced edge features are obtained. These features are then processed through linear transformation and layer normalization to obtain the edge embedding vector. Based on the sparse graph structure and edge embedding vectors, the time-sensitive features of the proxy nodes are obtained as follows: In the formula, H represents the local proxy node features, GAT() represents the graph attention mechanism, ReLU() represents the activation function, and LayerNorm() represents layer normalization. This indicates the relative positional relationship after it has been encoded by the time encoder. This indicates a time-sensitive characteristic.
5. The trajectory prediction method based on dual-view fusion according to claim 1, characterized in that, By filtering global road node features and global proxy node features, target road node features and target proxy node features are obtained, including: Based on the attention mechanism, global road node features are selected from all global road node features and global proxy node features to obtain initial global road node features; Based on the attention mechanism, global road node features are filtered from all initial global road node features to obtain target global road node features; Based on the attention mechanism, the global proxy node features are selected from all target global road node features and global proxy node features to obtain the initial global proxy node features; Based on the attention mechanism, global proxy node features are filtered out from all initial global proxy node features to obtain target global proxy node features.
6. The trajectory prediction method based on dual-view fusion according to claim 1, characterized in that, By fusing time-sensitive features, target road node features, and target proxy node features, the deep features of the proxy node are obtained, including: By fusing the target global proxy node features and the time-sensitive features of the proxy nodes, the proxy node fusion result is obtained, as shown below: In the formula, H fuison This indicates the fusion result of the proxy nodes. This indicates the characteristics of the target global proxy node, and MHA() represents the multi-head attention mechanism; By fusing the proxy node fusion results and the target global road node features, the deep features of the proxy nodes are obtained, as shown below: In the formula, Indicates the depth feature of the proxy node. This represents the global road node features of the target.
7. The trajectory prediction method based on dual-view fusion according to claim 1, characterized in that, Candidate endpoints are predicted based on the depth features of proxy nodes. The predicted trajectory is obtained by concatenating the depth features of proxy nodes and the candidate endpoints, including: Based on the attention mechanism, the deep features of all agent nodes are sequentially filtered three times and then processed by a single layer to obtain enhanced deep features. The enhanced deep features are input into the MLP to predict multiple candidate endpoints. Each candidate endpoint is then used by the MLP to predict its offset, resulting in multiple target candidate endpoints, as shown below: In the formula, E represents the candidate endpoint. This indicates the candidate endpoint for the target, and Flatten() indicates that the dimensions are flattened. This indicates enhanced deep features; After concatenating multiple target candidate endpoints with the deep features of proxy nodes, the input is given to the MLP to predict the relative offset sequence and obtain the predicted offset sequence. The predicted offset sequence is concatenated with the corresponding candidate endpoint to obtain the predicted trajectory.
8. A trajectory prediction method based on dual-view fusion according to claim 6 or 7, characterized in that, The attention mechanism is as follows: In the formula, Q, K, and V all represent the inputs, ATTENTION() represents the attention mechanism, and Softmax() represents the activation function. Indicates the dimension of the Key vector.