An individual behavior trajectory synthesis and simulation method fusing coding-decoding attention

Abstract

The application discloses a kind of individual behavior trajectory synthesis and simulation method of fusion encoding-decoding attention, belong to artificial intelligence behavior analysis and dynamic simulation field.Model is constructed by image semantic segmentation, three-layer stacked ConvLSTM is used to extract space-time characteristics;Attention mechanism is introduced when decoding to fuse features;Sparse convolution and modal attention weight are used to integrate multi-modal information;Kalman filter adjustment, Gaussian mixture model sampling and non-maximum suppression are used to screen path;And carry out multiple index evaluation.The application effectively solves the precision and diversity problem of trajectory synthesis in complex scene, provides efficient trajectory synthesis and simulation method for multiple fields.

Description

Technical Field

[0001] A method for synthesizing and simulating individual behavioral trajectories by integrating encoding-decoding attention, belonging to the field of artificial intelligence behavior analysis and dynamic simulation. Background Technology

[0002] With the rapid development of intelligent transportation systems and virtual reality technology, the accurate synthesis and dynamic simulation of individual behavior trajectories have become key technical requirements in fields such as public safety early warning, urban behavior analysis, and human-computer interaction.

[0003] However, traditional trajectory data processing methods are limited by single data sources and static rule models, revealing significant shortcomings when dealing with sparse spatiotemporal data scenarios: linear interpolation and other methods distort the trajectory due to the assumption of uniform motion; rule-based path planning ignores real-time traffic conditions and scene context, making it difficult to reflect dynamic uncertainty and environmental adaptability. Although models such as RNN / LSTM, GAN, and VAE improve trajectory prediction accuracy, RNN / LSTM struggles to capture long-term spatiotemporal dependencies due to gradient vanishing, GAN is prone to pattern collapse, and VAE suffers from insufficient trajectory diversity due to mismatched distribution assumptions. Furthermore, traditional self-attention mechanisms have high computational complexity, and while sparse attention reduces computation, rigid window settings lead to the loss of key local features. Existing methods fail to effectively integrate heterogeneous data, resulting in a lack of physical consistency in simulated trajectories; generated trajectories are disconnected from real traffic flow dynamics and lack real-time environmental adaptive correction mechanisms, making it difficult to meet the requirements of high-precision scene reconstruction and dynamic simulation.

[0004] Therefore, this invention proposes a method for synthesizing and simulating individual behavior trajectories by fusing encoder-decoder attention. It constructs a heterogeneous traffic object tensor and pedestrian pose keypoint model through image semantic segmentation, extracts spatiotemporal features using a three-layer stacked ConvLSTM, and introduces a dynamic attention mechanism to enhance feature correlation. It fuses multimodal data through sparse convolution, and combines Kalman filtering and Gaussian mixture models to achieve dynamic trajectory adjustment and multimodal sampling. Finally, through a simulation-completion collaborative optimization framework, it ensures that the generated trajectory conforms to physical laws and traffic flow dynamics characteristics, providing a highly reliable solution for behavior analysis in complex scenarios. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a method for synthesizing and simulating individual behavioral trajectories that integrates encoding and decoding attention.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] S1. Perform semantic segmentation on the original image using Equation 1 to extract multi-category obstacle information. Construct a heterogeneous traffic object tensor using Equation 2. Model pedestrian pose key points using Equation 3. Perform min-max normalization using Equation 4.

[0008] Formula 1:

[0009]

[0010] Formula 2:

[0011]

[0012] Formula 3:

[0013]

[0014] Formula 4:

[0015]

[0016] in, For the original image, To segment network parameters, For the segmentation result, c∈{1,2,...,13} represents the traffic object category, and (x,y) represents the 7×7 spatial grid coordinates. Let k be a 13×7×7 tensor, and k∈{1,2,...,18} be the keypoint indices. Let X be an 18×7×7 tensor, where X represents the original sensor data. and These are the minimum and maximum values ​​of the data column, respectively. The data is the normalized interval [0,1].

[0017] S2. Using Equation 5, three stacked ConvLSTM layers are used to extract spatiotemporal features layer by layer. Equation 6 is used to transfer features between layers. Equation 7 is used to aggregate the hidden states.

[0018] Formula 5:

[0019]

[0020] Formula 6:

[0021]

[0022] Formula 7:

[0023]

[0024] in, Let be the hidden state at the current time t. Let t be the cell state at the current time t. Let t be the input tensor of the t-th frame. This is the hidden state from the previous moment. This represents the cell state at the previous moment. The encoder can learn the weight matrix. This represents the hidden state at time t in layer l, where l is the encoder layer number. Let be the learnable weight matrix of the convolution in layer l, * represent the convolution operation, and σ be the sigmoid activation function. Let l be the bias vector of the l-th layer. For the spatiotemporal feature tensor after splicing, This represents the hidden state at time T in layer 3;

[0025] S3. Input the decoder using Equation 8, perform state updates using ConvLSTM using Equation 9, and generate the trajectory displacement using Equation 10:

[0026] Formula 8:

[0027]

[0028] Formula 9:

[0029]

[0030] Formula 10:

[0031]

[0032] in, This is the encoder output from the previous moment. For the deconvolution weights of the decoder, The feature map after deconvolution. Let be the hidden state of the encoder at the current time t. Let be the cell state of the encoder at the current time t. Let be the input tensor of the encoder in frame t. This represents the hidden state of the encoder in the previous time step. This represents the cell state of the encoder at the previous time step. The encoder can learn the weight matrix. For the output layer convolution weights, For the predicted two-dimensional displacement, This is the bias vector for the output layer;

[0033] S4. The similarity score of the encoder is calculated using cross-correlation or Euclidean distance in Equation 11. The attention module is introduced in Equation 12 to calculate the attention weights of the encoder's hidden state. The attention-weighted context tensor is generated in Equation 13. The attention-weighted context tensor is fused with the decoder output in Equation 14 to generate the final feature.

[0034] Formula 11:

[0035]

[0036] Formula 12:

[0037]

[0038] Formula 13:

[0039]

[0040] Formula 14:

[0041]

[0042] in, and Let i be the hidden state of the encoder at time i and j. The encoder hides the state at time i. transpose, Let be the attention weights of the encoder state at time t. Let be the hidden state of the encoder at time t. This is the query vector of the decoder at the current moment. For the attention-weighted context tensor, and These are the fusion weights for the context and decoder state, respectively. Let be the hidden state of the encoder at the current time t. For the final feature tensor after fusion, Here, ReLU is the bias vector during the fusion process, and ReLU is the modified linear unit activation function.

[0043] S5. The heterogeneous traffic tensor and pose tensor are concatenated along the channel dimension using Equation 15 to generate the input tensor. Local features are extracted using sparse convolution in Equation 16, and multimodal features are fused using the attention mechanism in Equation 17.

[0044] Formula 15:

[0045]

[0046] Formula 16:

[0047]

[0048] Formula 17:

[0049]

[0050] in, To splice along the channel dimension, It is a 16×7×7 tensor. To compute only non-zero elements in a sparse convolution operation, For sparse convolution kernel weights, yes Features after sparse convolution and For modal attention weights, As a characteristic of sparse traffic, For sparse pose features Features of fusion;

[0051] S6, Equation 18 based on fusion features The predicted trajectory is then adjusted using Kalman filtering according to Equation 19:

[0052] Formula 18:

[0053]

[0054] Formula 19:

[0055]

[0056] in, To predict the weights of the head fully connected layer, As a feature of fusion, To predict the bias vector of the head fully connected layer, These are the initial trajectory prediction values. Let be the optimal estimated state vector at time t after Kalman filtering adjustment. Let be the predicted value of the state at time t based on information from time t-1 and earlier. Here is the Kalman gain matrix. These are real-time sensor measurements. This is the observation matrix that maps the state to the measurement space;

[0057] S7. Use a Gaussian mixture model to sample and generate multimodal trajectories;

[0058] Furthermore, step S7 includes the following specific steps:

[0059] S71. Initialize the parameter π of the Gaussian mixture model;

[0060] S72. Optimize model parameters using the Expectation-Maximization (EM) algorithm;

[0061] S73. Define the trajectory sampling area as a circular area with a radius of 5 meters around the pedestrian;

[0062] S74. Generate multiple candidate trajectories within the sampling area;

[0063] S75. Use Non-Maximum Suppression (NMS) to select the optimal trajectory;

[0064] S8. Calculate the average displacement error (ADE) using Equation 20, and the final displacement error (FDE) using Equation 21, to evaluate the trajectory generation results:

[0065] Formula 20:

[0066]

[0067] Equation 21:

[0068]

[0069] in, For the number of test samples, To predict the time step, Let be the predicted coordinates of the i-th sample at time t. Let be the true coordinates of the i-th sample at time t. For the i-th sample Predicted endpoint coordinates at time [time]. For the i-th sample The true endpoint coordinates of the moment;

[0070] S9. Calculate the trajectory coverage (TC) using Equation 22:

[0071] Equation 22:

[0072]

[0073] in, Let I be the number of test samples, and I be the indicator function. Let be the predicted coordinates of the i-th sample at time t. Let be the true coordinates of the i-th sample at time t, and ε be the threshold.

[0074] S10. Visualize the predicted trajectory, the actual trajectory, and the attention weight distribution using heatmaps and trajectory lines.

[0075] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0076] 1. This invention innovatively captures the long-term spatiotemporal dependence and spatial correlation of individual behavior through a three-layer stacked ConvLSTM and dynamic attention mechanism, solving the trajectory distortion problem caused by the uniform velocity assumption or simple interpolation in traditional methods. The average displacement error (ADE) and final displacement error (FDE) are significantly reduced, improving the accuracy of trajectory prediction in complex scenarios.

[0077] 2. This invention combines a heterogeneous traffic object tensor constructed by image semantic segmentation with a pedestrian pose key point model. It achieves efficient fusion of traffic scene and human behavior features through sparse convolution. With dynamic adjustment of modal attention weights, it ensures that the generated trajectory conforms to the collision avoidance logic and vehicle interaction rules in the real environment, thereby improving the physical credibility of the simulation results.

[0078] 3. This invention innovatively employs Kalman filtering to adjust the predicted trajectory in real time, combined with Gaussian mixture model to adaptively sample and generate multimodal candidate trajectories within a 5-meter radius, and uses non-maximum suppression (NMS) to screen the optimal path. This not only ensures trajectory diversity, but also achieves dynamic adaptation to real-time traffic conditions through a simulation-completion collaborative framework, significantly improving the real-time performance and computational efficiency of trajectory simulation in complex urban scenarios. Attached Figure Description

[0079] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0080] Figure 1 This is a flowchart illustrating the overall technical process of an individual behavior trajectory synthesis and simulation method that integrates encoder-decoder attention, as proposed in this invention.

[0081] Figure 2 Flowchart of a method for generating multimodal trajectories by sampling a Gaussian mixture model. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0083] Example 1, referring to Figure 1 , 2 The overall flowchart of this invention is shown below. Figure 1 The steps of the method for generating multimodal trajectories using Gaussian mixture model sampling according to the present invention are as follows: Figure 2 :

[0084] First, semantic segmentation of the original image is performed using Equation 1 to extract multi-category obstacle information. Then, a heterogeneous traffic object tensor is constructed using Equation 2. Next, pedestrian pose key points are modeled using Equation 3, and min-max normalization is performed using Equation 4.

[0085] Formula 1:

[0086]

[0087] Formula 2:

[0088]

[0089] Formula 3:

[0090]

[0091] Formula 4:

[0092]

[0093] in, For the original image, To segment network parameters, For the segmentation result, c∈{1,2,...,13} represents the traffic object category, and (x,y) represents the 7×7 spatial grid coordinates. Let k be a 13×7×7 tensor, and k∈{1,2,...,18} be the keypoint indices. Let X be an 18×7×7 tensor, where X represents the original sensor data. and These are the minimum and maximum values ​​of the data column, respectively. The data is the normalized interval [0,1].

[0094] Equation 5 uses a three-layer stacked ConvLSTM to extract spatiotemporal features layer by layer, Equation 6 performs inter-layer feature transfer, and Equation 7 aggregates the hidden states:

[0095] Formula 5:

[0096]

[0097] Formula 6:

[0098]

[0099] Formula 7:

[0100]

[0101] in, Let be the hidden state at the current time t. Let t be the cell state at the current time t. Let t be the input tensor of the t-th frame. This is the hidden state from the previous moment. This represents the cell state at the previous moment. The encoder can learn the weight matrix. This represents the hidden state at time t in layer l, where l is the encoder layer number. Let be the learnable weight matrix of the convolution in layer l, * represent the convolution operation, and σ be the sigmoid activation function. Let l be the bias vector of the l-th layer. For the spatiotemporal feature tensor after splicing, This represents the hidden state at time T in layer 3;

[0102] The decoder input is obtained through Equation 8, the state is updated using ConvLSTM through Equation 9, and the trajectory displacement is generated through Equation 10.

[0103] Formula 8:

[0104]

[0105] Formula 9:

[0106]

[0107] Formula 10:

[0108]

[0109] in, This is the encoder output from the previous moment. For the deconvolution weights of the decoder, The feature map after deconvolution. Let be the hidden state of the encoder at the current time t. Let be the cell state of the encoder at the current time t. Let be the input tensor of the encoder in frame t. This represents the hidden state of the encoder in the previous time step. This represents the cell state of the encoder at the previous time step. The encoder can learn the weight matrix. For the output layer convolution weights, For the predicted two-dimensional displacement, This is the bias vector for the output layer;

[0110] Equation 11 calculates the similarity score of the encoder using cross-correlation or Euclidean distance. Equation 12 introduces an attention module to calculate the attention weights of the encoder's hidden state. Equation 13 generates the attention-weighted context tensor. Equation 14 fuses the attention-weighted context tensor with the decoder output to generate the final feature.

[0111] Formula 11:

[0112]

[0113] Formula 12:

[0114]

[0115] Formula 13:

[0116]

[0117] Formula 14:

[0118]

[0119] in, and Let i be the hidden state of the encoder at time i and j. The encoder hides its state at time i. transpose, Let be the attention weights of the encoder state at time t. Let be the hidden state of the encoder at time t. This is the query vector of the decoder at the current moment. For the attention-weighted context tensor, and These are the fusion weights for the context and decoder state, respectively. Let be the hidden state of the encoder at the current time t. For the final feature tensor after fusion, Here, ReLU is the bias vector during the fusion process, and ReLU is the modified linear unit activation function.

[0120] Equation 15 concatenates the heterogeneous traffic tensor and the pose tensor along the channel dimension to generate the input tensor. Equation 16 uses sparse convolution to extract local features, and Equation 17 uses the attention mechanism to fuse multimodal features.

[0121] Formula 15:

[0122]

[0123] Formula 16:

[0124]

[0125] Formula 17:

[0126]

[0127] in, To splice along the channel dimension, It is a 16×7×7 tensor. To compute only non-zero elements in a sparse convolution operation, For sparse convolution kernel weights, yes Features after sparse convolution and For modal attention weights, As a characteristic of sparse traffic, For sparse pose features Features of fusion;

[0128] Based on fusion features, Equation 18 The predicted trajectory is then adjusted using Kalman filtering according to Equation 19:

[0129] Formula 18:

[0130]

[0131] Formula 19:

[0132]

[0133] in, To predict the weights of the head fully connected layer, As a feature of fusion, To predict the bias vector of the head fully connected layer, These are the initial trajectory prediction values. Let be the optimal estimated state vector at time t after Kalman filtering adjustment. Let be the predicted value of the state at time t based on information from time t−1 and earlier. Here is the Kalman gain matrix. These are real-time sensor measurements. This is the observation matrix that maps the state to the measurement space;

[0134] Multimodal trajectories are generated using a Gaussian mixture model for sampling.

[0135] Furthermore, the method of generating multimodal trajectories using a Gaussian mixture model includes the following specific steps:

[0136] 1) Initialize the parameter π of the Gaussian mixture model;

[0137] 2) The Expectation-Maximization (EM) algorithm is used to optimize the model parameters;

[0138] 3) Define the trajectory sampling area as a circular area with a radius of 5 meters around the pedestrian;

[0139] 4) Generate multiple candidate trajectories within the sampling area;

[0140] 5) Use Non-Maximum Suppression (NMS) to select the optimal trajectory;

[0141] The average displacement error (ADE) is calculated using Equation 20, and the final displacement error (FDE) is calculated using Equation 21. The trajectory generation results are then evaluated.

[0142] Formula 20:

[0143]

[0144] Equation 21:

[0145]

[0146] in, For the number of test samples, To predict the time step, Let be the predicted coordinates of the i-th sample at time t. Let be the true coordinates of the i-th sample at time t. For the i-th sample Predicted endpoint coordinates at time [time]. For the i-th sample The true endpoint coordinates of the moment;

[0147] Calculate trajectory coverage (TC) using Equation 22:

[0148] Equation 22:

[0149]

[0150] in, Let I be the number of test samples, and I be the indicator function. Let be the predicted coordinates of the i-th sample at time t. Let be the true coordinates of the i-th sample at time t, and ε be the threshold.

[0151] The predicted trajectory, the actual trajectory, and the attention weight distribution are visualized using heatmaps and trajectory lines.

Claims

1. A method for synthesizing and simulating individual behavioral trajectories by fusing encoder-decoder attention, characterized in that, Includes the following steps: S1. Perform semantic segmentation on the original image using Equation 1 to extract multi-category obstacle information. Construct a heterogeneous traffic object tensor using Equation 2. Model pedestrian pose key points using Equation 3. Perform min-max normalization using Equation 4. Formula 1: Formula 2: Formula 3: Formula 4: in, For the original image, To segment network parameters, For the segmentation result, c∈{1,2,...,13} represents the traffic object category, and (x,y) represents the 7×7 spatial grid coordinates. Let k be a 13×7×7 tensor, and k∈{1,2,...,18} be the keypoint indices. Let X be an 18×7×7 tensor, where X represents the original sensor data. and These are the minimum and maximum values ​​of the data column, respectively. The data is the normalized interval [0,1]. S2. Using Equation 5, three stacked ConvLSTM layers are used to extract spatiotemporal features layer by layer. Equation 6 is used to transfer features between layers. Equation 7 is used to aggregate the hidden states. Formula 5: Formula 6: Formula 7: in, Let be the hidden state at the current time t. Let t be the cell state at the current time t. Let t be the input tensor of the t-th frame. This is the hidden state from the previous moment. This represents the cell state at the previous moment. The encoder can learn the weight matrix. Let be the hidden state at time t in layer l, where l is the encoder layer number. Let be the learnable weight matrix of the convolution in layer l, * represent the convolution operation, and σ be the sigmoid activation function. Let l be the bias vector of the l-th layer. For the spatiotemporal feature tensor after splicing, This represents the hidden state at time T in layer 3; S3. Input the decoder using Equation 8, perform state updates using ConvLSTM using Equation 9, and generate the trajectory displacement using Equation 10: Formula 8: Formula 9: Formula 10: in, This is the encoder output from the previous moment. For the deconvolution weights of the decoder, The feature map after deconvolution. Let be the hidden state of the encoder at the current time t. Let be the cell state of the encoder at the current time t. Let be the input tensor of the encoder in frame t. This represents the hidden state of the encoder in the previous time step. This represents the cell state of the encoder at the previous time step. The encoder can learn the weight matrix. For the output layer convolution weights, For the predicted two-dimensional displacement, This is the bias vector for the output layer; S4. The similarity score of the encoder is calculated using cross-correlation or Euclidean distance in Equation 11. The attention module is introduced in Equation 12 to calculate the attention weights of the encoder's hidden state. The attention-weighted context tensor is generated in Equation 13. The attention-weighted context tensor is fused with the decoder output in Equation 14 to generate the final feature. Formula 11: Formula 12: Formula 13: Formula 14: in, and Let i be the hidden state of the encoder at time i and j. The encoder hides the state at time i. transpose, Let be the attention weights of the encoder state at time t. Let be the hidden state of the encoder at time t. This is the query vector of the decoder at the current moment. For the attention-weighted context tensor, and These are the fusion weights for the context and decoder state, respectively. Let be the hidden state of the encoder at the current time t. For the final feature tensor after fusion, Here, ReLU is the bias vector during the fusion process, and ReLU is the modified linear unit activation function. S5. The heterogeneous traffic tensor and pose tensor are concatenated along the channel dimension using Equation 15 to generate the input tensor. Local features are extracted using sparse convolution in Equation 16, and multimodal features are fused using the attention mechanism in Equation 17. Formula 15: Formula 16: Formula 17: in, To splice along the channel dimension, It is a 16×7×7 tensor. To compute only non-zero elements in a sparse convolution operation, For sparse convolution kernel weights, for Features after sparse convolution and For modal attention weights, As a characteristic of sparse traffic, For sparse pose features Features of fusion; S6, Equation 18 based on fusion features The predicted trajectory is then adjusted using Kalman filtering according to Equation 19: Formula 18: Formula 19: in, To predict the weights of the head fully connected layer, As a feature of fusion, To predict the bias vector of the head fully connected layer, These are the initial trajectory prediction values. Let be the optimal estimated state vector at time t after Kalman filtering adjustment. Let be the predicted value of the state at time t based on information from time t-1 and earlier. Here is the Kalman gain matrix. These are real-time sensor measurements. This is the observation matrix that maps the state to the measurement space; S7. Use a Gaussian mixture model to sample and generate multimodal trajectories; S8. Calculate the average displacement error (ADE) using Equation 20, and the final displacement error (FDE) using Equation 21, to evaluate the trajectory generation results: Formula 20: Equation 21: in, For the number of test samples, To predict the time step, Let be the predicted coordinates of the i-th sample at time t. Let be the true coordinates of the i-th sample at time t. For the i-th sample Predicted endpoint coordinates at time [time]. For the i-th sample The true endpoint coordinates of the moment; S9. Calculate the trajectory coverage (TC) using Equation 22: Equation 22: in, Let I be the number of test samples, and I be the indicator function. Let be the predicted coordinates of the i-th sample at time t. Let be the true coordinates of the i-th sample at time t, and ε be the threshold. S10. Visualize the predicted trajectory, the actual trajectory, and the attention weight distribution using heatmaps and trajectory lines.

2. The method for synthesizing and simulating individual behavioral trajectories by fusing encoder-decoder attention as described in claim 1, characterized in that, The S7 step includes the following specific steps: 1) Initialize the parameter π of the Gaussian mixture model; 2) The Expectation-Maximization (EM) algorithm is used to optimize the model parameters; 3) Define the trajectory sampling area as a circular area with a radius of 5 meters around the pedestrian; 4) Generate multiple candidate trajectories within the sampling area; 5) Use nonmaximum suppression (NMS) to select the optimal trajectory.

Images