A vehicle front target trajectory prediction method based on fusion of transformer and LSTM

Abstract

This invention discloses a method for predicting the trajectory of a target ahead of a vehicle based on the fusion of Transformer and LSTM. The method includes: acquiring an onboard video stream ahead of the vehicle; identifying targets and obtaining bounding box information using a target detection algorithm based on a YOLO architecture; performing multi-target tracking using a Kalman filter and constructing a historical trajectory sequence; embedding features into the historical trajectories using a Transformer encoder to capture long-range spatiotemporal dependencies; gradually generating an initial prediction sequence based on encoded features using an autoregressive decoder; extracting local temporal features using an LSTM temporal adjustment module to smooth and correct the initial prediction; and outputting the final trajectory prediction result through a weighted fusion strategy. This invention, through a hybrid architecture, balances global dependency modeling and local temporal smoothing, effectively solving the problems of complex multi-target interactions and variable motion patterns in the vehicle operating environment, and significantly improving the accuracy and robustness of trajectory prediction.

Description

Technical Field

[0001] This invention relates to the fields of intelligent transportation systems and computer vision technology, and in particular to a method for predicting the trajectory of a target ahead of a vehicle based on the fusion of Transformer and LSTM. Background Technology

[0002] With the rapid development of intelligent transportation systems and autonomous driving technology, vehicle driving safety has become a focus of social concern. Vehicles typically travel in complex road environments, mixing with other vehicles, pedestrians, and non-motorized vehicles, creating a highly variable operating environment. Frequent incidents such as pedestrians trespassing, vehicles illegally changing lanes, and obstacles obstructing traffic pose significant challenges to vehicle driving safety.

[0003] Currently, active safety technologies for vehicles primarily rely on target detection algorithms. While existing improved YOLO algorithms (such as STA-YOLO) can effectively address issues like dense target occlusion and scale diversity, improving detection accuracy and real-time performance, simply identifying the target's location is insufficient. Therefore, accurately predicting the future trajectory of targets ahead and identifying potential hazards are crucial for ensuring that autonomous driving or assisted driving systems make correct decisions.

[0004] In the field of trajectory prediction, existing technologies mainly face the following challenges:

[0005] 1) Difficulty in capturing long-range dependencies: Traditional recursive prediction models (such as RNN and LSTM) are prone to gradient vanishing when processing long sequences, making it difficult to effectively capture long-distance spatiotemporal dependencies, resulting in inaccurate judgment of the intention of distant targets.

[0006] 2) Poor local temporal smoothness: Although models based on global attention mechanisms (such as Transformer) are good at capturing global context information, the trajectories they generate often lack temporal continuity constraints, and are prone to local jitter or jumps, which do not conform to the physical inertial laws of object motion in the vehicle operating environment.

[0007] 3) Difficulty in multi-scale feature fusion: Existing hybrid models often struggle to find the optimal balance between global contextual features and local temporal features, lacking effective fusion strategies to simultaneously ensure both long-term accuracy and short-term reasonableness of predictions.

[0008] Therefore, there is an urgent need for a vehicle forward target trajectory prediction method that can simultaneously take into account the long-range spatiotemporal dependency capture accuracy and the physical smoothness and rationality of the generated trajectory in complex scenarios. Summary of the Invention

[0009] The purpose of this invention is to provide a vehicle forward target trajectory prediction method based on the fusion of Transformer and LSTM, which solves the problems of low trajectory prediction accuracy, insufficient long-range dependency capture and poor trajectory smoothness in the existing technology under complex traffic scenarios.

[0010] To achieve the above objectives, the present invention provides the following technical solution: a method for predicting the trajectory of a vehicle's forward target based on the fusion of Transformer and LSTM, comprising the following steps:

[0011] Step 1: Acquire the onboard video stream in front of the vehicle, adjust the video frame images to a uniform resolution and perform normalization processing;

[0012] Step 2: Process the single-frame image using a target detection algorithm based on the YOLO architecture to identify targets in front and obtain their category, confidence score, and bounding box coordinates; execute a non-predefined category labeling strategy, that is, for targets that do not belong to the preset safety category but have a detection confidence score higher than the safety threshold, forcibly label them as the other category;

[0013] Step 3: Use the Kalman filter algorithm to estimate and associate the states of all detected targets, and construct a historical trajectory sequence containing the coordinates of the target center point;

[0014] Step 4: Use the Transformer encoder to encode the features of the historical observation coordinate sequence, capture the long-range spatiotemporal dependencies between trajectory points through a multi-head self-attention mechanism, and output a memory matrix containing global context information.

[0015] Step 5: Utilize the autoregressive decoder to perform decoding and prediction based on the memory matrix, and gradually generate the initial predicted trajectory sequence for the future through masked self-attention mechanism and cross-attention mechanism;

[0016] Step 6: Extract local temporal features of the trajectory using the LSTM timing adjustment module, map the initial predicted trajectory sequence to a high-dimensional space and input it into the module, use the LSTM gating mechanism to filter high-frequency jitter noise in the initial prediction, and output a locally corrected trajectory sequence that conforms to physical inertia.

[0017] Step 7: Using a weighted fusion strategy, the initial predicted trajectory sequence generated in Step 5 and the local corrected trajectory sequence output in Step 6 are linearly fused according to preset weights to calculate the final future trajectory prediction result.

[0018] Furthermore, in step 2, the target detection algorithm built based on the YOLO architecture includes the following steps:

[0019] Step 2-1: Input the standardized image into the object detection network for forward inference, and output a prediction tensor containing object category probability, confidence level and bounding box coordinate information;

[0020] Step 2-2: Set a confidence threshold to filter targets with a confidence level below the threshold;

[0021] Steps 2-3 involve using the Non-Maximum Suppression (NMS) algorithm to remove duplicates from the multiple overlapping bounding boxes. The degree of overlap between two bounding boxes is calculated using the Intersection over Union (IoU) formula as follows:

[0022]

[0023] in, As the reference detection frame, The detection boxes to be compared. Let be the area of ​​the intersection region between the two frames. The area of ​​the union region of the two bounding boxes is denoted as IoU. If the IoU is greater than the preset NMS threshold, the bounding boxes with higher overlap are deleted, and the optimal detection result is retained.

[0024] Furthermore, in step 2, the specific logic of the non-predefined category labeling strategy is as follows:

[0025] Set a predefined set of categories Set confidence threshold For the detected target, if its category And confidence level Then reset the target category to ;

[0026] The other category targets share the same bounding box regression and subsequent tracking logic with other predefined category targets to provide geometric safety redundancy.

[0027] Furthermore, in step 3, the Kalman filter algorithm maintains the target state through a "prediction-update" loop, specifically including:

[0028] State prediction phase: Based on the target state at the previous moment and control input The prior state at the current time is calculated using the following state prediction equation. :

[0029]

[0030] in, Here is the state transition matrix. This is the control matrix.

[0031] Data association matching: Calculate the intersection-over-union (IoU) between the target detection box output in step 2 and the current tracking box. If the IoU value is greater than a preset tracking threshold, it is determined that the currently detected target and the tracking target have successfully matched, and the detection box is used as the observation value at the current time. ;

[0032] State update phase: using the observed values The prior state estimate is corrected, and the Kalman gain is calculated. Posterior state estimation and the updated covariance matrix The specific formula is as follows:

[0033]

[0034]

[0035]

[0036] in, The observation matrix; To observe the noise covariance matrix; It is the identity matrix. To predict the covariance matrix; the updated state This refers to the trajectory point information of the target at the current moment.

[0037] Furthermore, in step 4, the Transformer encoder is composed of multiple stacked encoder modules, and the specific structure of each encoder module includes:

[0038] Self-attention mechanism layer: Employs a multi-head self-attention mechanism to map the input sequence into a query matrix. Key matrix Sum matrix The attention output is calculated using the following formula to capture the long-range spatiotemporal dependency between any two points in the sequence:

[0039]

[0040] in, For the attention mechanism layer, It is a normalized exponential function;

[0041] Feedforward network layer: Composed of fully connected layers, used to integrate the output information of the self-attention mechanism layer. Its calculation process is as follows:

[0042]

[0043] in, As the input to the feedforward network layer, , This is the weight matrix. , For bias vectors, Represents the ReLU activation function;

[0044] Residual connection and layer normalization module: Located at the output of the self-attention mechanism layer and the feedforward network layer, it is used to perform residual connection and normalization processing on the input and output of each layer to alleviate the gradient vanishing problem during training.

[0045] Furthermore, in step 5, the autoregressive decoder is composed of multiple stacked decoder modules, and the specific structure of each decoder module includes:

[0046] Masked Self-Attention Layer: Used to capture the dependencies between target trajectory points. It introduces a masking strategy when calculating attention weights to ensure that the model depends only on the current and previous trajectory points when predicting the current position; for position... Query and location The attention weights assigned to the key values ​​satisfy the following causal constraint:

[0047]

[0048] This is to strictly prevent future information leaks and ensure the causality of trajectory generation;

[0049] Encoder-decoder cross-attention layer: Used to guide target trajectory generation using contextual information from the encoder output; its calculation formula is as follows:

[0050]

[0051] in, This represents the query matrix derived from the current hidden state of the decoder. and Both represent the key matrix and value matrix derived from the output memory matrix of the Transformer encoder. This formula is used to calculate the correlation between the features of the decoder and the encoder, thereby achieving adaptive fusion of historical trajectory information.

[0052] Feedforward neural network: Consists of two linear transformation layers and an intermediate ReLU activation function; its dimensionality transformation path is as follows. Among them, the intermediate layer dimension Set as model dimension Four times that of the previous model, this design of first expanding and then compressing enhances the model's ability to nonlinearly map acceleration, deceleration, and turning motion patterns.

[0053] Furthermore, in step 6, the LSTM timing adjustment module specifically includes:

[0054] Gated filtering outlier suppression unit: Based on the forget gate, input gate and output gate inside the Long Short-Term Memory (LSTM) network, it selectively discards noise information in the cell state at the previous time step through the forget gate, and updates the trajectory features at the current time step to the cell state through the input gate, thereby smoothing the instantaneous abnormal fluctuations in the output trajectory of the Transformer decoder.

[0055] Hierarchical adjustment and multi-scale optimization structure: A multi-layer stacked LSTM network structure is adopted, in which the bottom LSTM layer is used to capture the short-term high-frequency motion features of the trajectory, and the high-level LSTM layer is used to integrate the long-term low-frequency trends. Multi-scale optimization of the trajectory is achieved through layer-by-layer feature extraction.

[0056] Forward adjustment and causal consistency guarantee logic: according to time steps The trajectory points are processed sequentially to ensure that the timing is accurate. The adjusted output depends only on time. Input and The hidden state before the moment strictly follows the causal relationship of time to prevent future information from interfering with the correction of the current trajectory.

[0057] Furthermore, in step 7, the fusion output module adopts a weighted fusion strategy, and the fusion formula is:

[0058]

[0059] in, The output of the Transformer The output is the result of LSTM optimization, where α is the weighting coefficient. The weighting coefficient is set to 0.7.

[0060] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described above.

[0061] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the above-described method.

[0062] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0063] This invention employs a dual-branch fusion architecture: the Transformer branch handles long-sequence dependencies, addressing intent prediction in complex scenarios; the LSTM branch handles local temporal jitter, conforming to the physical inertia of object motion. This invention fully considers the complexity of vehicle operation scenarios and addresses the contradiction between long-term prediction and short-term smoothing in a single model by improving overall robustness through a weighted fusion strategy. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of the overall architecture of the trajectory prediction model provided in this embodiment of the invention, showing the complete process from inputting historical trajectories to Transformer encoding, autoregressive decoding, LSTM refinement and final fusion.

[0065] Figure 2 This is a schematic diagram of the target tracking process based on Kalman filtering in an embodiment of the present invention.

[0066] Figure 3 This is a schematic diagram of the LSTM timing adjustment module in an embodiment of the present invention.

[0067] Figure 4 This is a visual diagram of the trajectory prediction results in a typical turning scenario according to an embodiment of the present invention.

[0068] Figure 5 This is a visualization diagram of trajectory prediction results in complex scenarios according to an embodiment of the present invention. Detailed Implementation

[0069] This invention proposes a method for predicting the trajectory of a target ahead of a vehicle based on the fusion of Transformer and LSTM.

[0070] The input data for this method comes from an onboard vision sensor. Its core processing logic encompasses multiple stages, including data preprocessing, object detection, multi-object tracking, global feature encoding, autoregressive decoding, temporal adjustment, and fusion output. Specifically: data preprocessing is responsible for frame extraction and standardization of the video stream; object detection is responsible for identifying obstacles ahead and outputting bounding boxes based on an improved YOLO architecture; multi-object tracking is responsible for constructing object identity associations and historical trajectory sequences using Kalman filtering; global feature encoding uses a Transformer encoder to capture the long-range spatiotemporal dependencies of the trajectory; autoregressive decoding is responsible for generating the initial predicted trajectory; temporal adjustment uses LSTM to extract local temporal features and smooth the trajectory; and fusion output combines global and local features to output the final prediction result.

[0071] like Figure 1As shown, this method employs a dual-branch fusion architecture: the Transformer branch handles long-sequence dependency issues, addressing intent prediction in complex scenarios; the LSTM branch handles local temporal jitter, conforming to the physical inertia of object motion. This invention fully considers the complexity of vehicle operation scenarios and addresses the contradiction between long-term prediction and short-term smoothing in a single model by improving overall robustness through a weighted fusion strategy.

[0072] Specifically, it includes the following steps:

[0073] Step 1: Acquire the onboard video stream in front of the vehicle, adjust the video frame images to a uniform resolution and perform normalization processing;

[0074] Step 2: Process the single-frame image using a target detection algorithm based on the YOLO architecture to identify targets in front and obtain the target's category, confidence score, and bounding box coordinates. During this process, execute a non-predefined category labeling strategy, that is, for targets that do not belong to the preset safety category but have a detection confidence score higher than the safety threshold, forcibly label them as the "other" category.

[0075] Step 3: Use the Kalman filter algorithm to estimate and associate the state of all detected targets (including preset categories and other categories) to construct a historical trajectory sequence containing the coordinates of the target center point;

[0076] Step 4: Use the Transformer encoder to encode the features of the historical observation coordinate sequence, capture the long-range spatiotemporal dependencies between trajectory points through a multi-head self-attention mechanism, and output a memory matrix containing global context information.

[0077] Step 5: Utilize the autoregressive decoder to perform decoding and prediction based on the memory matrix, and gradually generate the initial predicted trajectory sequence for the future through masked self-attention mechanism and cross-attention mechanism;

[0078] Step 6: Extract local temporal features of the trajectory using the LSTM timing adjustment module, map the initial predicted trajectory sequence to a high-dimensional space and input it into the module, use the LSTM gating mechanism to filter high-frequency jitter noise in the initial prediction, and output a locally corrected trajectory sequence that conforms to physical inertia.

[0079] Step 7: Using a weighted fusion strategy, the initial predicted trajectory sequence generated in Step 5 and the local corrected trajectory sequence output in Step 6 are linearly fused according to preset weights to calculate the final future trajectory prediction result.

[0080] The method of the present invention will now be described in detail with reference to the accompanying drawings:

[0081] Example

[0082] Combination Figure 1 A method for predicting the trajectory of a vehicle's forward target based on the fusion of Transformer and LSTM includes the following steps:

[0083] Step 1: The data preprocessing module acquires the video stream collected by the vehicle's front-mounted visual sensor (such as a wide-angle camera). In order to adapt to the input requirements of the neural network, the video frame images are adjusted to a uniform resolution (e.g., 640*640 pixels), and the pixel values ​​are divided by 255 for normalization so that they are distributed in the range of [0, 1].

[0084] The second step: The target detection module uses a detection network based on the YOLO architecture to perform inference on a single frame image, identify the target in front and obtain the target's category, confidence score and bounding box coordinate information, and uses non-maximum suppression (NMS) to remove redundant boxes.

[0085] Non-maximum suppression (NMS): For the overlapping bounding box problem, the intersection-over-union ratio (IoU) is calculated. The IoU calculation formula is as follows:

[0086]

[0087] in, As the reference detection frame, These are the detection boxes to be compared. If the IoU is greater than a set threshold, redundant boxes with high overlap are removed, and the best detection result is retained.

[0088] Non-predefined category (Other) marking strategy: For unknown obstacles that may appear on open roads (such as fallen rocks or spilled goods), the following logic is executed: Set a predefined set of safety categories. and safety confidence threshold If the detected target category But its confidence level If the target is not found in the target, it will be forcibly marked as "other". This strategy ensures that all obstacles with physical entities can enter the subsequent trajectory prediction process.

[0089] Step 3: The multi-target tracking module uses the Kalman filter algorithm to associate the detected targets, establish the target identity index between consecutive frames, and extract historical motion trajectories to construct a historical observation coordinate sequence.

[0090] Combination Figure 2 A Kalman filter is used to establish inter-frame correlation. To address the uncertainty of the motion of the target in front of the vehicle, the specific implementation steps are as follows:

[0091] Step 3-1: State Prediction. Based on the target state at the previous time step. and control input The prior state at the current time is calculated using the state prediction equation. :

[0092]

[0093] in, This represents the state vector, which includes the coordinates of the target center. and speed ; This is the state transition matrix, used to describe the state evolution of the target under the uniform velocity model; For control matrix; To control the input vector.

[0094] Step 3-2: State Update. Utilize current observations. (Detection box coordinates) Correct the predicted value and calculate the Kalman gain. and posterior state estimation :

[0095]

[0096]

[0097] in, For the observation matrix, To observe the noise covariance, This is the error covariance matrix. Through continuous prediction and updating, historical trajectory sequence data of the target is constructed.

[0098] Step 4: The global feature encoding module uses the Transformer encoder to embed features into the historical observation coordinate sequence, captures the long-range spatiotemporal dependencies between trajectory points through a multi-head self-attention mechanism, and outputs a memory matrix.

[0099] The encoder consists of multiple stacked encoder layers, with a core multi-head self-attention mechanism. This mechanism maps the input sequence to queries (Q), keys (K), and values ​​(V), calculated using the following formula:

[0100]

[0101] in: They represent the query matrix, key matrix, and value matrix, respectively. This represents the dimension of the key vector. As a scaling factor; This is the normalized exponential function. Using this formula, the model can focus on the correlation between any two time steps in the historical trajectory, capturing long-range dependencies.

[0102] Step 5: The autoregressive decoding module, based on the memory matrix, gradually generates the initial predicted trajectory sequence for the future through mask self-attention mechanism and cross-attention mechanism.

[0103] The autoregressive decoding module uses the decoder to gradually generate the initial prediction trajectory. The decoder introduces a masked attention mechanism, which uses a mask matrix... This ensures that the prediction of the current location relies only on the current and previous trajectory points, preventing the leakage of future information.

[0104]

[0105] Meanwhile, the decoder reads the encoder's output through a cross-attention layer, enabling the interaction between historical information and future predictions.

[0106]

[0107] in, From the current state of the decoder, The memory matrix from the encoder output.

[0108] Step 6: The timing adjustment module uses an LSTM network to extract local timing features of the trajectory and smooths and corrects the initial predicted trajectory sequence.

[0109] Combination Figure 3 The timing adjustment module calls the LSTM network to... Fine-tuning is performed. This is based on the initial prediction sequence generated by the Transformer decoder. First, it is mapped to a high-dimensional space through a feature embedding layer to obtain the sequence. Subsequently, the sequence is fed into the LSTM timing adjustment module. At each time step... The LSTM unit is based on the current input The hidden state of the previous time step Update its cell state Output the current hidden state. LSTM filters high-frequency noise through a gating mechanism. The specific implementation steps are as follows:

[0110] Step 6-1: Forget Gate Calculation. This determines the cell state from the previous time step. What information is discarded:

[0111]

[0112] Step 6-2: Input Gate and Cell State Update. Calculate the new information at the current time step and update the cell state. :

[0113]

[0114]

[0115]

[0116] Step 6-3: Output gate calculation. Calculate the current hidden state based on the updated cell state. That is, the corrected trajectory points:

[0117]

[0118]

[0119] in, It is the Sigmoid activation function. , These are trainable parameters (weight matrix and bias vector). This indicates element-wise multiplication.

[0120] Finally, the hidden state sequence output by the last LSTM layer The adjusted predicted trajectory is obtained by mapping back to the trajectory coordinate space through a linear output layer. The sequence output by the LSTM is denoted as .

[0121] Step 7: The fusion output module fuses the global features generated by the Transformer with the local features extracted by the LSTM according to preset weights to calculate the final future trajectory prediction result.

[0122] The fusion output module executes a weighted fusion algorithm to combine the outputs of the Transformer branches. Output of LSTM branch By performing a linear combination, the final prediction result is obtained. :

[0123]

[0124] Experiments have verified that when the weighting coefficient When the value is 0.7, the model achieves optimal performance in terms of mean squared error (MSE), average displacement error (ADE), and final displacement error (FDE).

[0125] Through the above steps, this method demonstrates excellent performance under various complex road conditions. To verify the effectiveness of this method, the prediction results were visualized and analyzed. Figure 4 , Figure 5 The solid blue line and dots represent historical states, the solid green line and asterisks represent the actual future trajectory, and the dashed red line and dots represent the predicted trajectory of this method.

[0126] like Figure 4 The figure illustrates the prediction performance of the method of this invention in a single-objective nonlinear motion scenario. In this scenario, the target vehicle is performing a lane change or a slight turn, and its trajectory exhibits a distinct arc. As can be seen from the figure, the red predicted trajectory generated by this method highly overlaps with the green true future trajectory, and no significant divergence is observed even at the end of the trajectory (the key point of the FDE index). This demonstrates that after introducing the LSTM time-series adjustment module, the model can well follow the physical inertia of the object's motion, generating a smooth curve that conforms to the vehicle's kinematics, effectively eliminating the jagged jitter that may be produced by a single Transformer model.

[0127] like Figure 5 The figure illustrates the prediction performance of the method of this invention in a complex multi-target concurrent scenario. This scenario contains at least five different moving targets, each with different directions of motion (some to the left, some to the right) and varying motion patterns (including straight-line movement and turns with different curvatures). As can be seen from the figure, the method of this invention can accurately capture the motion intentions of all targets simultaneously, with the red predicted trajectory closely following the green ground truth trajectory of each target. This demonstrates that the present invention, combining YOLO target detection and Kalman filtering in front-end processing with the global attention mechanism of Transformer, can effectively handle the multi-target interaction problem in complex traffic flows, maintaining extremely high prediction accuracy and robustness even with a large number of targets and significant differences in their motion characteristics.

[0128] The method of the present invention can be executed on an electronic device, which includes a processor and a memory. The memory is used to store computer programs and historical trajectory data, and the processor is used to execute the programs in the memory to achieve the functions of steps 1 to 7 described above.

[0129] The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. All components not explicitly stated in this embodiment can be implemented using existing technology.

Claims

1. A method for predicting the trajectory of a vehicle's forward target based on the fusion of Transformer and LSTM, characterized in that, Includes the following steps: Step 1: Acquire the onboard video stream in front of the vehicle, adjust the video frame images to a uniform resolution and perform normalization processing; Step 2: Process the single-frame image using a target detection algorithm based on the YOLO architecture to identify targets in front and obtain their category, confidence score, and bounding box coordinates; execute a non-predefined category labeling strategy, that is, for targets that do not belong to the preset safety category but have a detection confidence score higher than the safety threshold, forcibly label them as the other category; Step 3: Use the Kalman filter algorithm to estimate and associate the states of all detected targets, and construct a historical trajectory sequence containing the coordinates of the target center point; Step 4: Use the Transformer encoder to encode the features of the historical observation coordinate sequence, capture the long-range spatiotemporal dependencies between trajectory points through a multi-head self-attention mechanism, and output a memory matrix containing global context information. Step 5: Utilize the autoregressive decoder to perform decoding and prediction based on the memory matrix, and gradually generate the initial predicted trajectory sequence for the future through masked self-attention mechanism and cross-attention mechanism; Step 6: Extract local temporal features of the trajectory using the LSTM timing adjustment module, map the initial predicted trajectory sequence to a high-dimensional space and input it into the module, use the LSTM gating mechanism to filter high-frequency jitter noise in the initial prediction, and output a locally corrected trajectory sequence that conforms to physical inertia. Step 7: Using a weighted fusion strategy, the initial predicted trajectory sequence generated in Step 5 and the local corrected trajectory sequence output in Step 6 are linearly fused according to preset weights to calculate the final future trajectory prediction result.

2. The method according to claim 1, characterized in that, In step 2, the target detection algorithm based on the YOLO architecture includes the following steps: Step 2-1: Input the standardized image into the object detection network for forward inference, and output a prediction tensor containing object category probability, confidence level and bounding box coordinate information; Step 2-2: Set a confidence threshold to filter targets with a confidence level below the threshold; Steps 2-3 involve using a non-maximum suppression algorithm to remove duplicates from overlapping bounding boxes. The degree of overlap between two bounding boxes is calculated using the following intersection-union ratio (IU / R) formula: （1） in, As the reference detection frame, The detection boxes to be compared. Let be the area of ​​the intersection region between the two frames. The area of ​​the union region of the two bounding boxes is used; if the IoU is greater than the preset NMS threshold, the bounding boxes with higher overlap are deleted, and the best detection result is retained.

3. The method according to claim 1, characterized in that, In step 2, the specific logic of the non-predefined category labeling strategy is as follows: Set a predefined set of categories Set confidence threshold For the detected target, if its category And confidence level Then reset the target category to Other categories of targets share the same bounding box regression and subsequent tracking logic with other predefined categories of targets to provide geometric safety redundancy.

4. The method according to claim 1, characterized in that, In step 3, the Kalman filter algorithm maintains the target state through a "prediction-update" loop, specifically including: State prediction phase: Based on the target state at the previous moment and control input The prior state at the current time is calculated using the following state prediction equation. : （2） in, Here is the state transition matrix. For control matrix; Data association matching: Calculate the intersection-over-union (IoU) ratio between the target detection bounding box output in step 2 and the current tracking bounding box. If the IoU value is greater than the preset tracking threshold, it is determined that the currently detected target and the tracking target have successfully matched, and the detection bounding box is used as the observation value at the current time. ; State update phase: using the observed values The prior state estimate is corrected, and the Kalman gain is calculated. Posterior state estimation and the updated covariance matrix The specific formula is as follows: （3） （4） （5） in, The observation matrix; To observe the noise covariance matrix; It is the identity matrix. To predict the covariance matrix; the updated state This refers to the trajectory point information of the target at the current moment.

5. The method according to claim 1, characterized in that, In step 4, the Transformer encoder is composed of multiple stacked encoder modules, and the specific structure of each encoder module includes: Self-attention mechanism layer: Employs a multi-head self-attention mechanism to map the input sequence into a query matrix. Key matrix Sum matrix The attention output is calculated using the following formula to capture the long-range spatiotemporal dependency between any two points in the sequence: （6） in, For the attention mechanism layer, It is a normalized exponential function; Feedforward network layer: Composed of fully connected layers, used to integrate the output information of the self-attention mechanism layer. Its calculation process is as follows: （7） in, As the input to the feedforward network layer, , This is the weight matrix. , For bias vectors, Represents the ReLU activation function; Residual connection and layer normalization module: Located at the output of the self-attention mechanism layer and the feedforward network layer, it is used to perform residual connection and normalization processing on the input and output of each layer to alleviate the gradient vanishing problem during training.

6. The method according to claim 1, characterized in that, In step 5, the autoregressive decoder is composed of multiple stacked decoder modules, and the specific structure of each decoder module includes: Masked Self-Attention Layer: Used to capture the dependencies between target trajectory points. It introduces a masking strategy when calculating attention weights to ensure that the model depends only on the current and previous trajectory points when predicting the current position; for position... Query and location The attention weights assigned to the key values ​​satisfy the following causal constraint: （8） This is to strictly prevent future information leaks and ensure the causality of trajectory generation; Encoder-decoder cross-attention layer: Used to guide target trajectory generation using contextual information from the encoder output; its calculation formula is as follows: （9） in, This represents the query matrix derived from the current hidden state of the decoder. and Both represent the key matrix and value matrix derived from the output memory matrix of the Transformer encoder. This formula is used to calculate the correlation between the features of the decoder and the encoder, thereby achieving adaptive fusion of historical trajectory information. Feedforward neural network: Consists of two linear transformation layers and an intermediate ReLU activation function; its dimensionality transformation path is as follows. Among them, the intermediate layer dimension Set as model dimension Four times that of the previous model, this design of first expanding and then compressing enhances the model's ability to nonlinearly map acceleration, deceleration, and turning motion patterns.

7. The method according to claim 1, characterized in that, In step 6, the LSTM timing adjustment module specifically includes: Gated filtering outlier suppression unit: It is constructed based on the forget gate, input gate and output gate inside the long short-term memory network. The forget gate selectively discards the noise information in the cell state at the previous time step, and the input gate updates the trajectory features at the current time step to the cell state, thereby smoothing the instantaneous abnormal fluctuations in the output trajectory of the Transformer decoder. Hierarchical adjustment and multi-scale optimization structure: A multi-layer stacked LSTM network structure is adopted, in which the bottom LSTM layer is used to capture the short-term high-frequency motion features of the trajectory, and the high-level LSTM layer is used to integrate the long-term low-frequency trends. Multi-scale optimization of the trajectory is achieved through layer-by-layer feature extraction. Forward adjustment and causal consistency guarantee logic: according to time steps The trajectory points are processed sequentially to ensure that the timing is accurate. The adjusted output depends only on time. Input and The hidden state before the moment strictly follows the causal relationship of time to prevent future information from interfering with the correction of the current trajectory.

8. The method according to claim 1, characterized in that, In step 7, the fusion output module adopts a weighted fusion strategy, and the fusion formula is as follows: （10） in, The output of the Transformer The output is the optimized LSTM output, and α is the weight coefficient.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

Images