A single-step trajectory prediction method based on Mamba-attention spatiotemporal decoupling

Abstract

This invention provides a single-step trajectory prediction method based on Mamba-attention spatiotemporal decoupling, belonging to the technical field of single-step trajectory prediction methods. This invention involves: Mamba-based trajectory temporal feature extraction; fusion of attention mechanisms and Mamba-based trajectory spatial feature extraction; dynamic fusion of spatiotemporal features based on cross-attention; trajectory offset prediction; and finally, prediction position calculation. This invention reduces computational overhead; captures spatial relationships such as relative positions and speeds of multiple vehicles; decouples the complex relationship between vehicle motion state and spatial position; achieves deep fusion and dynamic optimization of spatiotemporal features; and provides a high-precision, low-latency trajectory prediction solution for autonomous driving.

Description

Technical Field

[0001] This invention relates to a single-step trajectory prediction method based on Mamba-attention spatiotemporal decoupling, belonging to the technical field of single-step trajectory prediction methods. Background Technology

[0002] In autonomous driving systems, single-step trajectory prediction is a crucial step for achieving safe and efficient driving. By accurately predicting the vehicle's next position, it provides continuous positioning support for real-time tracking and offers core basis for dynamic obstacle avoidance and path planning decisions, serving as a vital bridge connecting environmental perception and control execution. However, in autonomous driving scenarios, vehicles often operate in complex environments with frequent multi-vehicle interactions, frequent acceleration and deceleration, and high-speed driving. These complexities have prompted mainstream deep learning-based trajectory prediction methods to improve accuracy through network structure design, but the resulting surge in computational complexity makes it impossible to simultaneously meet the dual demands of real-time performance and accuracy required by autonomous driving. While research on lightweight spatiotemporal networks aims to balance performance and efficiency, their sequential feature processing mechanism using temporal-spatial modules lacks explicit decoupling of spatiotemporal semantics, leading to confusion between motion details and spatial interaction features. Some methods utilize global pooling to achieve feature fusion between two graphs, but fail to adaptively capture the interaction weights between spatiotemporal features, impacting prediction performance.

[0003] In the field of single-step trajectory prediction technology for autonomous driving, existing mainstream deep learning-based methods suffer from difficulties in achieving both high-precision prediction and real-time computation due to their complex network structures. Lightweight spatiotemporal networks, on the other hand, suffer from insufficient decoupling of spatiotemporal features and weak adaptive learning capabilities of interaction weights. This results in poor dynamic adaptability of the algorithms in complex scenarios such as frequent multi-vehicle interactions and high-speed driving, failing to meet the dual requirements of real-time performance and accuracy for autonomous driving. Therefore, to address the problems of fuzzy feature representation and insufficient dynamic adaptability in existing technologies, this invention proposes a spatiotemporal attention decoupling and dynamic attention fusion method, combined with the linear complexity of Mamba, to effectively solve the aforementioned defects while ensuring that the algorithm possesses real-time computational performance that meets the needs of practical applications. Summary of the Invention

[0004] The purpose of this invention is to solve the problems existing in the prior art and to provide a single-step trajectory prediction method based on Mamba-attention spatiotemporal decoupling.

[0005] By leveraging the efficient long-sequence modeling capabilities and linear complexity of the Mamba model, this method replaces traditional structures to enhance the extraction of temporal dynamic features of vehicle motion trajectories, reducing computational overhead. Through the collaborative design of temporal Mamba modeling and spatial cross-attention mechanisms, a refined spatial interaction information representation system is constructed to capture spatial relationships such as the relative positions and speeds of multiple vehicles. An adaptive attention fusion network is designed to dynamically couple the temporal features extracted by Mamba with spatial interaction features, decoupling the complex relationship between vehicle motion state and spatial position, achieving deep fusion and dynamic optimization of spatiotemporal features, and providing a high-precision, low-latency trajectory prediction solution for autonomous driving.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] A single-step trajectory prediction method based on Mamba-attention spatiotemporal decoupling includes the following steps:

[0008] Step 1: Trajectory temporal feature extraction based on Mamba;

[0009] In the extraction of trajectory temporal features T, the dependency relationship of the target trajectory is formalized as a learning problem of conditional probability distribution; then, a target temporal motion feature network is constructed. This network, through a stacked architecture of three Mamba blocks, realizes the modeling and capture of the temporal dependency relationship of the data and encodes the temporal dynamic information of the target temporal motion features.

[0010] Step 2: Integrate attention mechanism and Mamba trajectory space feature extraction;

[0011] Based on the trajectory space characteristics, a target space feature extraction network is constructed. This network consists of three X-Mamba blocks. Each X-Mamba block is composed of a self-attention mechanism and a Mamba block. After three layers of X-Mamba block feature extraction processing, the target space features S are finally accurately captured.

[0012] Step 3: Dynamic fusion of spatiotemporal features based on cross-attention;

[0013] Using the temporal feature T from step one as the query and the spatial feature S from step two as the key and value, the trajectory temporal feature extraction based on Mamba in step one and the trajectory spatial feature extraction based on Mamba in step two are combined. By dynamically calculating the attention weight of temporal to spatial, the decoupled fusion of temporal state-dominated spatial interaction is achieved: after calculating the decoupled fusion of spatial interaction, the original temporal features are preserved through residual connection to obtain the fused feature TSF;

[0014] Step 4: Trajectory offset prediction;

[0015] The fused TSF obtained in step 3 is passed through a multilayer perceptron network to finally obtain the predicted offset of the trajectory bounding box.

[0016] Step 5: Predicted location calculation;

[0017] Based on the predicted offset of the trajectory bounding box obtained in step four, and combined with the position of the last frame of the trajectory, the final single-step predicted position result is obtained.

[0018] Preferably, the specific process of formalizing the dependency relationship of the target trajectory into a conditional probability distribution learning problem in step one is as follows:

[0019] A single trajectory given input ,in, Indicates the first Four-dimensional feature vectors at each time step , The coordinates of the center point of the normalized target. The goal of the time series model is to estimate the distribution of the normalized target bounding box's length and width. And decompose it into a product of conditional probabilities:

[0020]

[0021] in, Indicates the current time step Previously Historical trajectory sequence data.

[0022] Preferably, the specific process of constructing the target temporal motion feature network in step one is as follows:

[0023] Within the deep learning framework, by learning the mapping function Implementation, where hidden states Satisfying the recursive relation:

[0024]

[0025] in, The parameter is nonlinear functions, The nonlinear function represented is constructed through a target temporal motion feature network; This represents the hidden state at time step t. This represents the hidden state at time step t-1. The hidden state learned by the network encodes the temporal dynamic information of the target's temporal motion features.

[0026] Preferably, the specific process of constructing the target space feature extraction network in step two is as follows:

[0027] Trajectories of multiple targets are input into the target spatial feature extraction network. Where N is the number of trajectories, L is the length of the input trajectory, and 4 is the feature representation dimension of the target bounding box;

[0028] First, it needs to be flattened out. Each target trajectory is implemented according to the following rules:

[0029]

[0030] in, ;

[0031] The query Q, key K, and value V of the self-attention layer all come from the same input. It is obtained through linear transformation; the embedding dimension is The number of attention heads is Then the dimension of each head is ;

[0032]

[0033] in, , , It is a learnable weight matrix. Then, the dot product between the query and the key is calculated to obtain the attention score. The attention scores are then subjected to a softmax operation to obtain the attention weights. Multiply the attention weights by the values ​​to obtain the attention output attnoutput;

[0034]

[0035]

[0036]

[0037] in, , , , obtained Obtained through matrix transformation , The trajectory features after self-attention processing will be... Input is fed into the Mamba layer, and output is obtained. Then Flattened The shape undergoes further processing, and after three layers of X-Mamba block feature extraction, the spatial features of the target are finally accurately captured.

[0038] Preferably, the specific process of dynamically calculating the attention weights of the space in step three is as follows:

[0039]

[0040] in, The output of cross-attention is the dynamically weighted result of spatial features guided by temporal features. To query the projection matrix corresponding to Q, key K, and value V, where L is the length of the input trajectory. , It refers to the number of heads to focus on.

[0041] Preferably, in step three, after calculating the decoupling and fusion of spatial interactions, the original temporal features are preserved through residual connections to obtain the fused feature TSF, as shown in the following formula:

[0042]

[0043] Wherein, TSF is the final fused feature after spatiotemporal dynamic weighting, S is the spatial feature, and T is the temporal feature.

[0044] Preferably, the fused TSF feature described in step four is processed through a multilayer perceptron network (MLP) to obtain the predicted offset of the trajectory bounding box at time t.

[0045]

[0046] in, This represents the offset of the trajectory bounding box predicted by the multilayer perceptron network. This represents the offset of the center point's coordinates relative to the trajectory of the last historical frame, i.e., the position at time t-1. This represents the aspect ratio of the trajectory relative to the last historical frame.

[0047] Preferably, in step five, based on the predicted trajectory offset obtained in step four and combined with the position of the last frame of the trajectory, the formula for obtaining the final single-step predicted position result is as follows:

[0048]

[0049]

[0050]

[0051]

[0052] in, , , , Used to characterize the trajectory in The position at time, which is determined by the normalized coordinates of the center point. , Add normalized length and width , It is composed of historical references for predicting the position at time t.

[0053] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0054] This invention proposes a spatiotemporal attention decoupling and attention dynamic fusion mechanism. By explicitly separating spatiotemporal semantics and adaptively learning feature weights, it improves prediction accuracy while reducing computational complexity, achieving high-precision and low-time-consuming real-time prediction. This effectively enhances the algorithm's adaptability in complex dynamic scenarios and provides a reliable trajectory prediction solution for autonomous driving systems.

[0055] This invention innovatively introduces a cross-attention mechanism at the spatiotemporal feature fusion level. Guided by temporal features, it constructs a dynamic weight replication strategy to perform refined weight allocation for spatial interaction features. This mechanism, through an adaptive dynamic adjustment paradigm, dynamically regulates the weights of spatiotemporal feature interactions based on the dynamic evolution of vehicle kinematic parameters and spatial interaction relationships in complex driving scenarios. This process effectively avoids the information redundancy or loss problems caused by fixed weight allocation patterns in existing methods. By strengthening the expression of key features, it achieves deep fusion and optimization of spatiotemporal semantics, significantly improving the ability of vehicle trajectory prediction to represent complex scenarios.

[0056] In terms of temporal feature extraction, this invention fully leverages the linear complexity advantage and long sequence modeling capability of the Mamba model by designing a fusion network of Mamba and attention mechanisms. Through deep coupling at the architectural level, it achieves order-of-magnitude optimization of computational complexity while maintaining the complete extraction of feature information. Attached Figure Description

[0057] Figure 1 This is a schematic diagram illustrating the basic principle of the method in an embodiment of the present invention.

[0058] Figure 2 This is an example of a single-step prediction network for autonomous driving trajectory based on Mamba-attention spatiotemporal decoupling in this invention. Detailed Implementation

[0059] The present invention will be further described in detail below with reference to the accompanying drawings: This embodiment is implemented under the premise of the technical solution of the present invention, and detailed implementation methods are given, but the protection scope of the present invention is not limited to the following embodiments.

[0060] like Figure 1 and Figure 2As shown, the trajectory single-step prediction method based on Mamba-attention spatiotemporal decoupling involved in this embodiment includes the following steps:

[0061] Step 1: Trajectory Temporal Feature Extraction Based on Mamba

[0062] Trajectory temporal features are a set of features in trajectory data that are closely related to the time dimension. They are used to describe the patterns, regularities, or attributes of the trajectory's position, motion state (such as velocity and direction) as they change over time. In trajectory temporal feature extraction T, the dependencies of the target trajectory are formalized as a learning problem of conditional probability distributions; given a single trajectory as input... ,in Indicates the first Four-dimensional feature vectors at each time step ,in The coordinates of the center point of the normalized target. Given the normalized target bounding box dimensions, the goal of the time series model is to estimate the joint probability distribution. And decompose it into a product of conditional probabilities:

[0063]

[0064] in, Indicates the current time step Previously All historical trajectory sequence data. In a deep learning framework, this is typically achieved by learning a mapping function. Implementation, where hidden states Satisfying the recursive relation:

[0065]

[0066] in, The parameter is Nonlinear functions, specifically, The nonlinear function represented is constructed through a target temporal motion feature network. This represents the hidden state at time step t. This represents the hidden state at time step t-1. The network uses a stacked architecture of three Mamba blocks to model and capture temporal dependencies in the data. The hidden states learned by the network encode the temporal dynamic information of the target's temporal motion features. The Mamba block is a novel state-space model that discretizes the original spatial state model to adapt to the time-series input. It then selectively applies the discretized spatial state model to capture dynamic sequence dependencies. Its specific principle is as follows:

[0067] State-space model. The state-space model is a general mathematical framework for modeling dynamic systems, which uses hidden state vectors. Input sequence Mapping to output response From a mathematical perspective, the dynamics of a system can be described by a set of first-order differential equations:

[0068]

[0069] Among them, matrix Indicates the state transition parameters. , and These are the projection parameters. R represents the set of real numbers, meaning that all elements of the vector or matrix are real numbers; M is the hidden state vector. The dimension of the hidden state is an M-dimensional real vector, where 1 represents the input. Dimensions (because) (i.e., the input is a one-dimensional real number), therefore the dimension of matrix B is... The same applies to C and D.

[0070] Discretization. For processing discrete sequences such as time series and natural language. The SSM (State Space Model) needs to be discretized from continuous-time form to discrete-time form:

[0071]

[0072] in, For the first The hidden state at each time step This is the hidden state from the previous time step; For the first The input sequence at each time step; and The discretized projection parameters are obtained by transforming continuous parameters using the zero-order preservation rule.

[0073]

[0074] in, This represents the time interval (time step) of discretization, i.e., the continuous time t is discretized into t=0,Δ,2Δ,... express The product of the time interval and parameter B in the continuous-time model, i.e. .

[0075] Selective state-space model. The inherent linear time-invariant property of SSM depends on matrices A, B, C, and The consistency across different inputs limits its ability to filter and understand contextual details within different input sequences. Mamba addresses this by using B, C, and... By treating the parameters as dynamic and input-dependent, this limitation is overcome, thus transforming the SSM into a time-varying model. This modification enables the model to adapt more effectively to different input contexts, enhancing its ability to capture relevant temporal dynamics and resulting in a more accurate and efficient representation of the input sequence.

[0076] Step 2: Integrating attention mechanisms and Mamba for trajectory space feature extraction

[0077] Trajectory spatial features are the sum of various spatial relationships between different trajectories, and can be used to explore the influence mechanisms and laws governing the interaction between trajectories. A target spatial feature extraction network is constructed, consisting of three X-Mamba blocks, each comprising a self-attention mechanism and a Mamba block. The network is input with the trajectories of multiple targets. Where N is the number of trajectories, L is the length of the input trajectory, and 4 is the feature representation dimension of the target bounding box. First, it needs to be flattened into... Each target trajectory is implemented according to the following rules:

[0078]

[0079] in, This formula represents the relationship between the value at each position in X after it is flattened and the value at the corresponding position before it is flattened.

[0080] In the self-attention layer, the query Q, key K, and value V all come from the same input. It is obtained through linear transformation. The embedding dimension is... The number of attention heads is Then the dimension of each head is .

[0081]

[0082] in, , , It is a learnable weight matrix. Then, the dot product between the query and the key is calculated to obtain the attention score. Apply a softmax operation to the attention scores to obtain the attention weights. Multiply the attention weights by the values ​​to obtain the attention output attnoutput.

[0083]

[0084]

[0085]

[0086] in, , , The result Obtained through matrix transformation , The trajectory features after self-attention processing will be... Input is fed into the Mamba layer, and output is obtained. Then... Flattened The shape undergoes further processing. The flattening rules and formulas for Y are the same as those for X, so the specific process is omitted. However, X and Y differ in that X serves as the input to the spatial feature extraction network, while Y is the output after passing through an X-Mamba block. After three layers of X-Mamba block feature extraction processing, the target spatial features are finally accurately captured.

[0087] Step 3: Dynamic Fusion of Spatiotemporal Features Based on Cross-Attention

[0088] Using temporal features T as the query and spatial features S as the key and value, the decoupled fusion of temporal state-dominated spatial interactions is achieved by dynamically calculating the attention weight of temporal features on spatial features.

[0089]

[0090] in, The output of cross-attention is the dynamically weighted result of spatial features guided by temporal features. To query the projection matrix corresponding to Q, key K, and value V, where L is the length of the input trajectory. , This refers to the number of attention points. Here, temporal features are used as queries to find other related interaction trajectories and assign them high weights, consistent with the normal understanding of attention interacting with the trajectory itself. After calculating the spatial fusion, the original temporal features are preserved through residual connections to obtain TSF (Temporal-Spatial Fusion), avoiding spatial interaction noise from obscuring the motion trend. The fused feature TSF formula is as follows:

[0091]

[0092] Wherein, TSF is the final fused feature after spatiotemporal dynamic weighting, S is the spatial feature, and T is the temporal feature.

[0093] Step 4: Trajectory Offset Prediction

[0094] The TSF feature is fused and then passed through a multilayer perceptron network (MLP). The MLP is an existing network design; in specific applications, the number of layers and related parameters of the MLP can be adjusted as needed. Finally, the predicted offset of the trajectory bounding box at time t is obtained, as follows:

[0095]

[0096] in, This represents the offset of the trajectory bounding box predicted by the multilayer perceptron network. This represents the offset of the center point's coordinates relative to the trajectory of the last historical frame, i.e., the position at time t-1. The aspect ratio relative to the trajectory of the last historical frame;

[0097] Step 5: Predicted Location Calculation

[0098] After obtaining the predicted offset, the final single-step prediction result is calculated using the following formula, combined with the position of the last frame of the trajectory. It is worth noting that because the predicted offset has been normalized, it needs to be multiplied by the width and height of the image during calculation to restore its actual scale.

[0099]

[0100]

[0101]

[0102]

[0103] in, , , , Used to characterize the trajectory in The position at time, which is determined by the normalized coordinates of the center point. , Add normalized length and width , It is composed of historical references for predicting the position at time t.

[0104] The above description is merely a preferred embodiment of the present invention. These specific embodiments are different implementations based on the overall concept of the present invention, and the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A single-step trajectory prediction method based on Mamba-attention spatiotemporal decoupling, characterized in that, Includes the following steps: Step 1: Trajectory temporal feature extraction based on Mamba; In the extraction of trajectory temporal features T, the dependency relationship of the target trajectory is formalized as a learning problem of conditional probability distribution; then, a target temporal motion feature network is constructed. This network, through a stacked architecture of three Mamba blocks, realizes the modeling and capture of the temporal dependency relationship of the data and encodes the temporal dynamic information of the target temporal motion features. Step 2: Integrate attention mechanism and Mamba trajectory space feature extraction; Based on the trajectory space characteristics, a target space feature extraction network is constructed. This network consists of three X-Mamba blocks. Each X-Mamba block is composed of a self-attention mechanism and a Mamba block. After three layers of X-Mamba block feature extraction processing, the target space features S are finally accurately captured. Step 3: Dynamic fusion of spatiotemporal features based on cross-attention; Using the temporal feature T from step one as the query and the spatial feature S from step two as the key and value, the trajectory temporal feature extraction based on Mamba in step one and the trajectory spatial feature extraction based on Mamba in step two are combined. By dynamically calculating the attention weight of temporal to spatial, the decoupled fusion of temporal state-dominated spatial interaction is achieved: after calculating the decoupled fusion of spatial interaction, the original temporal features are preserved through residual connection to obtain the fused feature TSF; Step 4: Trajectory offset prediction; The fused TSF obtained in step 3 is passed through a multilayer perceptron network to finally obtain the predicted offset of the trajectory bounding box. The fused TSF feature described in step four is processed through a multilayer perceptron network (MLP) to obtain... The predicted offset of the time-trajectory bounding box. in, This represents the offset of the trajectory bounding box predicted by the multilayer perceptron network. The coordinates of the center point relative to the trajectory of the last historical frame, i.e. The offset of the position at any given time. The aspect ratio relative to the trajectory of the last historical frame; Step 5: Predicted location calculation; Based on the predicted offset of the trajectory bounding box obtained in step four, and combined with the position of the last frame of the trajectory, the final single-step predicted position result is obtained; the formula is as follows: in, , , , Used to characterize the trajectory in The position at time, which is determined by the normalized coordinates of the center point. , Add normalized length and width , Composition, as a prediction Historical reference for time and location.

2. The trajectory single-step prediction method based on Mamba-attention spatiotemporal decoupling according to claim 1, characterized in that, The specific process of formalizing the dependency relationship of the target trajectory into a learning problem of conditional probability distribution as described in step one is as follows: A single trajectory given input ,in, Indicates the first Four-dimensional feature vectors at each time step , Representing dimension, The coordinates of the center point of the normalized target. The goal of the time series model is to estimate the joint probability distribution, given the normalized dimensions of the target bounding box. And decompose it into a product of conditional probabilities: in, Indicates the current time step Previously Historical trajectory sequence data.

3. The trajectory single-step prediction method based on Mamba-attention spatiotemporal decoupling according to claim 2, characterized in that, The specific process of constructing the target temporal motion feature network in step one is as follows: Within the deep learning framework, by learning the mapping function Implementation, where hidden states Satisfying the recursive relation: in, The parameter is nonlinear functions, The nonlinear function represented is constructed through a target temporal motion feature network; express Hidden states under time step, express The hidden states at each time step, learned by the network, encode the temporal dynamic information of the target's temporal motion features.

4. The trajectory single-step prediction method based on Mamba-attention spatiotemporal decoupling according to claim 1, characterized in that, The specific process of constructing the target space feature extraction network in step two is as follows: Trajectories of multiple targets are input into the target spatial feature extraction network. ∈ Where N is the number of trajectories, L is the length of the input trajectory, and 4 is the feature representation dimension of the target bounding box; First, it needs to be flattened out. Each target trajectory is implemented according to the following rules: in, ; The query Q, key K, and value V of the self-attention layer all come from the same input. It is obtained through linear transformation; the embedding dimension is The number of attention heads is Then the dimension of each head is ; in, , , It is a learnable weight matrix. Then, the dot product between the query and the key is calculated to obtain the attention score. The attention scores are then subjected to a softmax operation to obtain the attention weights. Multiply the attention weights by the values ​​to obtain the attention output attnoutput; in, , , , obtained Obtained through matrix transformation , For trajectory features after self-attention processing, Input is fed into the Mamba layer, and output is obtained. Then Flattened The shape undergoes further processing, and after three layers of X-Mamba block feature extraction, the spatial features of the target are finally accurately captured.

5. The trajectory single-step prediction method based on Mamba-attention spatiotemporal decoupling according to claim 1, characterized in that, The specific process of dynamically calculating the attention weights of the space in step three is as follows: in, The output of cross-attention is the dynamically weighted result of spatial features guided by temporal features. For spatial features, As a time series feature, To query the projection matrix corresponding to Q, key K, and value V, where L is the length of the input trajectory. , It refers to the number of heads to focus on.

6. The trajectory single-step prediction method based on Mamba-attention spatiotemporal decoupling according to claim 5, characterized in that, In step three, after calculating the decoupling and fusion of spatial interactions, the original temporal features are preserved through residual connections to obtain the fused feature TSF, as shown in the following formula: Among them, TSF is the final fusion feature after spatiotemporal dynamic weighting processing.

Images