Multi-scene adaptive pedestrian trajectory prediction method based on attention mechanism

Abstract

A multi-scene adaptive pedestrian trajectory prediction method based on an attention mechanism realizes trajectory prediction in multiple scenes by constructing an adaptive pedestrian trajectory prediction model, and the model design comprises the steps of extracting historical trajectory features by using a multi-head attention mechanism, and capturing interaction among pedestrians by a social attention module by using a graph convolutional network. The track alignment module further compares and aligns track characteristics in different scenes; and finally, through a sample weighted enhanced training strategy, the learning ability of the model to the complex trajectory is optimized. According to the method, the accuracy of pedestrian trajectory prediction can be remarkably improved, and the overall prediction performance and robustness are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of pedestrian trajectory prediction, and in particular to a multi-scenario adaptive pedestrian trajectory prediction method based on an attention mechanism. Background Art

[0002] With the rapid advancement of urbanization and the continuous increase in population density, urban traffic has become more complex and pedestrian safety issues have become increasingly prominent. In this context, the importance of pedestrian trajectory prediction technology in the fields of intelligent transportation systems, robot navigation, autonomous vehicles and video surveillance has continued to increase. Pedestrian trajectory prediction technology aims to provide strong support for traffic management and safety measures by analyzing and predicting the future movement paths of pedestrians. In addition, pedestrian trajectory prediction technology can significantly improve traffic safety and reduce the occurrence of traffic accidents by accurately predicting the movement paths of pedestrians. By improving traffic signal control and driving route planning, it can also improve the operating efficiency of the overall transportation system. In addition, this technology supports intelligent transportation systems and autonomous driving technologies, helping autonomous vehicles make safer and more effective decisions. In robot navigation and video surveillance, pedestrian trajectory prediction improves navigation accuracy and public safety. Pedestrian trajectory prediction methods can not only promote the construction of smart cities, but also improve the level of intelligent urban management, ultimately bringing significant social and economic benefits.

[0003] Existing pedestrian trajectory prediction technologies mainly include models based on recurrent neural networks (RNNs), generative adversarial networks (GANs), and attention mechanisms. Pedestrian trajectory prediction methods based on LSTM (Long Short-Term Memory Networks), such as S-LSTM, Group-LSTM, and StarNet, perform well in specific scenarios by processing sequence data and capturing long-term dependencies. GAN-based prediction methods, such as SGAN, Sophie, and Social Way, generate more realistic pedestrian trajectory samples through adversarial games. Although these models have shown excellent performance in specific application scenarios, they still have limitations when dealing with diverse and complex scenarios. For example, existing models usually assume that pedestrian behavior is uniform, ignoring the spatiotemporal differences in crowd distribution, and ultimately it is difficult to fully capture the diversity of individual behaviors in different scenarios. In addition, traditional methods rely on the general feature representation of a single sample, which easily ignores key individual-level fine-grained features, resulting in insufficient adaptability and accuracy when dealing with trajectory prediction in complex environments.

[0004] Therefore, traditional trajectory prediction methods often rely on simple motion models and are unable to accurately capture complex spatiotemporal relationships and the interaction of multiple factors. Therefore, how to effectively integrate multi-source data and improve the accuracy and adaptability of prediction models has become an important technical issue that needs to be overcome in pedestrian trajectory prediction methods. Summary of the invention

[0005] In order to overcome the problems of limited pedestrian trajectory prediction accuracy in diversified scenarios, neglect of the spatiotemporal differences in crowd distribution, and insufficient capture of pedestrian spatial interaction information in the existing technology, the present invention provides a multi-scenario adaptive pedestrian trajectory prediction method based on attention mechanism. Trajectory prediction in multiple scenarios is achieved by constructing an adaptive pedestrian trajectory prediction model. The model design includes the use of a multi-head attention mechanism to extract historical trajectory features, and the social attention module uses a graph convolutional network to capture the interaction between pedestrians, while the trajectory alignment module further compares and aligns trajectory features in different scenarios; finally, through the sample-weighted reinforcement training strategy, the model's learning ability for complex trajectories is optimized, and the robustness and accuracy of the prediction are improved.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A multi-scenario adaptive pedestrian trajectory prediction method based on an attention mechanism, the method comprising the following steps:

[0008] (1) Obtaining pedestrian historical position information: Collecting image information in the scene and extracting historical observation positions at a set time step as basic position information for pedestrian trajectory prediction;

[0009] (2) Design a social attention module to capture the interactive information of pedestrians, use a graph convolutional network to further learn the motion trends, and output spatiotemporal features;

[0010] (3) Design a trajectory alignment module to fully capture the diversity of individual behaviors in different scenarios, calculate the differences and align them to obtain the contrast difference features of pedestrian trajectories, and finally integrate them into the backbone network to obtain the prediction results of pedestrian trajectories;

[0011] (4) Pedestrian trajectory prediction backbone network model design: Based on the multi-head attention mechanism, the feature information of the pedestrian's historical trajectory sequence is captured, and combined with the recurrent neural network to predict the pedestrian's trajectory position in the future time step, the model is finally able to adapt to trajectory prediction tasks in different scenarios;

[0012] (5) Pedestrian trajectory prediction model training: A sample-weighted reinforcement training strategy is adopted to enhance the learning of complex trajectories. A weight is assigned to each training sample, and these weights are dynamically updated during the training process. For samples with poor pedestrian trajectory prediction accuracy, their training weights are increased so that they receive more attention in subsequent training, thereby improving the model's prediction accuracy for complex trajectory samples.

[0013] The technical concept of the present invention is as follows: First, by introducing a multi-head attention mechanism and a social attention module, the ability to extract features of pedestrian trajectories and capture interactive information is significantly improved. Second, the trajectory alignment module solves the challenge of trajectory comparison between different scenes and enhances the model's adaptability in diverse scenes. Finally, the sample-weighted reinforcement training strategy is adopted to effectively improve the model's learning ability for complex trajectory samples, and enhance the robustness and accuracy of prediction.

[0014] The beneficial effects of the present invention are mainly manifested as follows: the present invention can significantly improve the accuracy of pedestrian trajectory prediction, accurately capture spatiotemporal features through a multi-head attention mechanism and a social attention module, and adaptively compare trajectory differences between different scenes through a trajectory alignment module; ultimately, the model only needs a small amount of sample data of the target scene to achieve high-precision pedestrian trajectory prediction in the scene, so the method proposed in the present invention can better adapt to trajectory prediction tasks in different scenes. At the same time, the sample weighted reinforcement training strategy enhances the model's ability to learn complex trajectories and improves overall prediction performance and robustness. The weighted fusion method in the model training and optimization process enables the prediction model to quickly adapt to complex scenes, which can effectively reduce potential collision risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is the implementation flow chart of the multi-scenario adaptive pedestrian trajectory prediction method based on the attention mechanism;

[0016] Figure 2 This is the overall model structure diagram of the multi-scenario adaptive pedestrian trajectory prediction method based on the attention mechanism;

[0017] Figure 3 This is a model reinforcement training flowchart for the multi-scenario adaptive pedestrian trajectory prediction method based on the attention mechanism. DETAILED DESCRIPTION

[0018] The present invention will be further described below in conjunction with the accompanying drawings.

[0019] Reference Figure 1 to Figure 3 , a multi-scenario adaptive pedestrian trajectory prediction method based on attention mechanism, the method comprising the following steps:

[0020] (1) Obtaining pedestrian historical position information: Collecting image information in the scene and extracting historical observation positions at a set time step as basic position information for pedestrian trajectory prediction;

[0021] The process of step (1) is as follows:

[0022] (1.1) Collect image information in the scene and extract the time step t1 = [1, T obs] is used as the basic position information for pedestrian trajectory prediction, namely:

[0023]

[0024] Among them, T obs Set according to the actual situation, the reference is 3.6 seconds, p i represents the basic position information of the ith pedestrian, including the position of obs ]’s pedestrian two-dimensional plane position information is as follows:

[0025]

[0026] in, and Respectively represent the two-dimensional horizontal and vertical position coordinate information of the i-th pedestrian at time step t1;

[0027] (1.2) Collect and extract the i-th pedestrian at time step t2∈[T obs +1,T pre ] is the trajectory position information of p i* :

[0028]

[0029] Among them, T pre Set according to the actual situation and Tobs, the reference is 4.8 seconds, and Respectively represent the two-dimensional horizontal and vertical position coordinate information of the i-th pedestrian at time step t2;

[0030] (2) Design a social attention module to capture the interactive information of pedestrians, use a graph convolutional network to further learn the motion trends, and output spatiotemporal features;

[0031] The process of step (2) is as follows:

[0032] (2.1) Node representation design for pedestrian trajectory prediction. In the social attention module, the movement trajectory of pedestrians is described by a graph. The interaction between pedestrians in each time step is abstractly defined as a graph representation G t1 :

[0033] G t1 =(V t1 ,E t1 ,F t1 )

[0034] in, is the vertex set of pedestrians in the graph, representing all pedestrians at time t1, that is, the vertex set of N pedestrians; The value of is the coordinate position observed by the i-th pedestrian at time t1 Edge Set represents the relationship between the i-th and j-th pedestrians, and a relationship edge is constructed between every two pedestrians; and the feature set Defined as:

[0035]

[0036] Where W is the weight and σ(*) is the ReLU activation function;

[0037] (2.2) Calculate the social soft attention function. Since the interactions between pedestrians are complex and diverse, these interactions are roughly divided into three categories according to the actual situation. These three types of interactions include walking towards each other, walking away from each other, and walking side by side. In order to better analyze and simulate the pedestrian interaction mode, a new social soft attention function is proposed to construct the adjacency matrix represents the attention weight between the i-th and j-th pedestrians, which is calculated as follows:

[0038]

[0039] in, and are the velocity vectors of pedestrian i and pedestrian j at time t1; α and β represent the velocity vector and and the angle between the two pedestrian connection paths, and Respectively represent the position coordinates of pedestrian i and pedestrian j at time t1;

[0040] In addition, when i=j, that is, the two pedestrian nodes are the same, γ is used to represent the self-attention value of the node, and the value of γ is calculated according to the adjacency matrix A t1 The maximum value of the elements in is set to 0.1;

[0041] ① When two pedestrians walk towards each other, the angles α and β formed by their velocity vectors are both acute angles; at this time, cosα and cosβ are both greater than zero. If the result is greater than 0, it indicates that there is a significant impact between the two people, and both pedestrians need to pay attention to each other to avoid a head-on collision.

[0042] ② When two pedestrians walk in opposite directions, the angles α and β formed by their velocity vectors are both obtuse angles; at this time, cosα and cosβ are both less than zero. The calculation result of is limited to 0, which means that there is no direct mutual influence between the two pedestrians leaving in opposite directions;

[0043] ③ When two pedestrians walk side by side, that is, pedestrian i is in front and pedestrian j is behind, the angle α formed by the velocity vectors of the two is an obtuse angle, while β is an acute angle; if the moving speed of pedestrian j is significantly higher than that of pedestrian i, there is a possibility of future collision. At this time, the output of the attention function is greater than 0, indicating a potential collision risk; on the contrary, if the speed of pedestrian j is lower than that of pedestrian i, then there is no collision risk and the output of the attention function will be set to 0.

[0044] In order to better reflect the magnitude of the collision risk, a natural function form is used to represent the nonlinear exponential changes between pedestrians due to the influence of Euclidean distance and relative speed. This processing method ensures that when the distance between pedestrians is close, the collision risk is higher, the attention weight between pedestrians is also larger, and the network can better extract and understand the characteristics of social interaction.

[0045] (2.3) Trajectory domain feature output, using graph convolutional network (GCN) to further extract spatiotemporal features, and perform graph convolution operation at time step t1, that is:

[0046]

[0047] Among them, A t1 is the adjacency matrix at time step t1, D t1 is the submatrix of the graph, E is the identity matrix, and the time step T1 to T obs The adjacency matrix of is stacked as:

[0048]

[0049] The matrix stack is:

[0050]

[0051] At the same time, the feature matrix of the lth layer in the GCN layer is stacked as:

[0052]

[0053] Therefore, calculate the GCN output F after the lth layer l+1 for:

[0054]

[0055] Among them, W l is the learnable parameter of the lth layer, and a total of three GCN network layers are used to extract the spatiotemporal feature representation of the observation trajectory, then l∈[1,3]; and the feature representation obtained by the last layer of GCN network is defined as Where N is the number of pedestrians in the trajectory domain, D f and L obs is the data dimension of the extracted features;

[0056] (3) Design a trajectory alignment module to fully capture the diversity of individual behaviors in different scenarios, calculate the differences and align them to obtain the contrast difference features of pedestrian trajectories, and finally integrate them into the backbone network to obtain the prediction results of pedestrian trajectories;

[0057] Since there are differences between trajectory domains in different scenarios, the present invention designs a trajectory alignment module to fully capture the diversity of individual behaviors in different scenarios, ultimately enabling the model to better adapt to trajectory prediction tasks in different scenarios.

[0058] Specifically, the pedestrian trajectory information in different scenarios is firstly used to obtain the trajectory features of the corresponding trajectory domain through the social attention module designed in step (2); then the trajectory alignment module is used for deep refinement, comparison and running-in; finally, the pedestrian trajectory comparison difference feature dif is obtained. i , integrated into the backbone network to alleviate the misalignment between different pedestrians and different sample spaces; the process of step (3) is as follows:

[0059] (3.1) Acquisition of source trajectory and target trajectory information: Collect the location information of pedestrian trajectories in the source scene and the target scene, and obtain the graph representation G of the pedestrian trajectories in the source scene respectively. t1 (source) = {V t1 (source),E t1 (source),F t1 (source)}, and the graph representation of pedestrian trajectories in the target scene G t1 (target)=

[0060] {V t1 (target),E t1 (target),F t1 (target)};

[0061] (3.2) G t1 (source) and G t1 (target) are used as the input of the social attention module designed in step (2), and the feature representations of the source trajectory F(source) and the target trajectory F(target) are obtained respectively. These feature representations contain the spatiotemporal information of each pedestrian in different trajectory domains;

[0062] (3.3) Calculation of attention score in trajectory domain. Since the same feature vector may have different performances in different trajectory domains, in order to determine the representativeness of the pedestrian’s feature vector in the trajectory domain, the attention mechanism is used to learn the relative correlation between the feature vector and the trajectory domain. The attention scores of the i-th pedestrian in the source trajectory domain and the target trajectory domain are ρ respectively. i(source) and ρ i (target), which is used to measure the importance of each pedestrian feature in a specific trajectory domain, is calculated as:

[0063]

[0064] Among them, α and W h is a learnable weight parameter, tanh(*) is a tangent function, and f i (source) and f i (target) are the position features of the i-th pedestrian in the source trajectory domain and the target trajectory domain respectively;

[0065] (3.4) Pedestrian trajectory comparison difference feature dif i Calculate the attention score value ρ based on step (3.3) i (source) and ρ i (target), respectively calculate the final feature representation c of the source trajectory domain and the target trajectory domain i (source) and c i (target):

[0066] c i (source) = ρ i (source)·f i (source)

[0067] c i (target) = ρ i (target)·f i (target)

[0068] After that, the individual-level differences between the final feature representations of the two trajectory domains are calculated, and the calculated differences are mapped to the feature vector h through a fully connected layer. i Data features with the same dimensions, namely:

[0069] dif i =FL(c i (source)-c i (target),W fl )

[0070] Among them, W fl is a learnable weight matrix, FL(*,*) is a fully connected layer, and finally, the calculated pedestrian trajectory is compared with the difference feature dif i Integrate it into the backbone network model to achieve pedestrian trajectory prediction with comprehensive feature analysis;

[0071] (4) Pedestrian trajectory prediction backbone network model design: Based on the multi-head attention mechanism, the feature information of the pedestrian's historical trajectory sequence is captured, and combined with the recurrent neural network to predict the pedestrian's trajectory position in the future time step, the model is finally able to adapt to trajectory prediction tasks in different scenarios;

[0072] The processing process of the pedestrian trajectory prediction backbone network model is as follows:

[0073] (4.1) Pedestrian historical trajectory position First, the neural network embedding layer is used to encode the position and obtain the encoded feature data

[0074]

[0075] in, is the embedding layer; W e represents the weight coefficient of the embedding layer, which is obtained by step-by-step iterative optimization through network training; and t1∈[1,T obs ], iv[1,N];

[0076] (4.2) Since pedestrian motion patterns vary in different scenarios, the pedestrian trajectories obtained in step (3) under different scenarios are compared with the difference features dif i Introduced into the backbone network to solve the shift problem of contrast between different trajectory domains, that is, the fused features for:

[0077]

[0078] Among them, concat(*) represents the data channel concatenation operation;

[0079] (4.3) Use the multi-head attention mechanism to encode the position feature data Projected into the query matrix, key matrix and value matrix, for the i-th pedestrian sequence, calculate and The matrix is:

[0080]

[0081] Among them, f Q (*), f K (*) and f V (*) Corresponding query, key and value functions, and They represent the query, key and value matrices of the i-th pedestrian under the j-th attention head respectively. The total number of multi-head attention heads is M, then j∈[1,M];

[0082] (4.4) Attention coefficient of multi-head attention And the feature vector calculation based on multi-head attention:

[0083]

[0084] Among them, softmax(*) is the softmax normalization function, d k is the normalization factor; h is the feature calculation result of the i-th pedestrian in the j-th attention head; i is the feature vector result of the i-th pedestrian after multi-head attention processing, which contains time dependency. concat(*) is the data channel concatenation operation;

[0085] Therefore, the entire data operation based on the multi-head attention mechanism is:

[0086]

[0087] Among them, Multihead(*) is the multi-head attention operation function;

[0088] (4.5) Construct a trajectory decoder based on a recurrent neural network. In order to enhance the robustness of pedestrian trajectory sequence prediction, a random vector noise that follows a Gaussian distribution is transformed into i Integrate into the trajectory decoder, then the input feature vector v of the trajectory decoder i for:

[0089] v i =concat(h i ,noise i )

[0090] Secondly, a trajectory decoder is constructed based on the multi-head attention mechanism and recurrent neural network, and combined with the fully connected layer and ReLU activation function, the prediction result of the pedestrian trajectory position is finally obtained, namely:

[0091] p′ i* =Decoder(v i ,W p )

[0092] Among them, p′ i* is the trajectory prediction result of the i-th pedestrian, Decoder(*) is the trajectory decoder processing function, W p is the network weight coefficient of the trajectory decoder;

[0093] (5) Pedestrian trajectory prediction model training: A sample-weighted reinforcement training strategy is adopted to enhance the learning of complex trajectories. A weight is assigned to each training sample, and these weights are dynamically updated during the training process. For samples with poor pedestrian trajectory prediction accuracy, their training weights are increased so that they receive more attention in subsequent training, thereby improving the model's prediction accuracy for complex trajectory samples.

[0094] The process of step (5) is as follows:

[0095] (5.1) Trajectory prediction loss function design, the goal is to predict the pedestrian in the future time step t2∈[T obs +1,T pre ]’s trajectory position, therefore, the loss function loss is defined as:

[0096]

[0097] Among them, p i* and p′ i* They represent pedestrian i at time step [T obs +1,T pre ]’s true position vector and predicted position vector, MSE(*,*) is the mean square error, and N is the total number of people;

[0098] (5.2) Parameter initialization. The total number of samples in the training data set is M, so the initial weights of all samples are initialized:

[0099]

[0100] Among them, m∈[1,M], and the initial network parameters of the pedestrian trajectory prediction model are initialized to θ0;

[0101] (5.3) Iteratively train the pedestrian trajectory prediction model and calculate the loss value of each sample in the training dataset:

[0102]

[0103] in, represents the initialization weight of the mth training sample, loss m represents the direct loss value of the mth training sample calculated using step (5.1), L m Represents the weighted loss value of the mth sample;

[0104] (5.4) Model parameter update, calculate the average loss Loss1 of the first iteration:

[0105]

[0106] Based on the loss calculation result Loss1, the gradient descent method is used to optimize the model parameters, and the parameters θ1 of the pedestrian trajectory prediction model after the first iteration are obtained;

[0107] (5.5) Update the sample training weights, and calculate the loss1, loss2, ..., loss according to the loss function of each sample in the first iteration M , update the weighted value of each set of training sample data, that is:

[0108]

[0109] in, is the training weight of the mth sample after the first iteration update, μ is the learning rate of the preset sample training weight, and μ>0; softmax(*) is the softmax function; since the loss value corresponding to complex samples is larger during the training process, this method can further increase the training weight of complex samples and reduce the training weight of simple samples; this enhanced training strategy enables the model to dynamically adjust the importance of each training sample during the iterative training process, so that the training samples with poor pedestrian trajectory prediction effect can get more attention. Ultimately, this improves the robustness of the pedestrian trajectory prediction model in complex situations;

[0110] (5.6) Repeat steps (5.3) to (5.5) for a total of T iterations, and obtain the total loss calculation values ​​Loss1, Loss2, ..., Loss for T iterations respectively. T , and the network model parameters θ1,θ2,...,θ of the pedestrian trajectory prediction model T ;

[0111] (5.7) Based on the principle that a larger loss function value indicates a poorer prediction performance of the sample, and vice versa, a better prediction performance, the model parameter fusion method is used to obtain the final pedestrian trajectory prediction model. First, the fusion factor τ of each model in the T-times trained model is obtained. k :

[0112]

[0113] Among them, τ k is the fusion factor of the pedestrian trajectory prediction model obtained by the kth iteration training, Loss k represents the total loss value of the kth iteration training, and k∈[1,T];

[0114] (5.8) The parameters of T pedestrian trajectory prediction models are weighted fused based on the fusion factor:

[0115]

[0116] Among them, θ mix is the fused pedestrian trajectory prediction model parameter, θ k is the parameter of the model at the kth iteration. Finally, θ mix The network parameters of the pedestrian trajectory prediction model are applied to it and used as the final trained pedestrian trajectory prediction model.

[0117] The multi-scenario adaptive pedestrian trajectory prediction method based on the attention mechanism of this embodiment is implemented in the following steps:

[0118] Step 1: Deploy camera sensors above multiple target monitoring areas for real-time monitoring, which are defined as source areas and target areas. Shoot a large number of dynamic videos of pedestrians walking in the source area and a small number of video data in the target area in real time, and collect image data at a fixed frequency, with a time step of t1 = [1, T obs ], and the pedestrian image history data of the two regions are obtained, which are expressed as:

[0119]

[0120] Among them, T obs Set according to the actual situation, the reference is 3.6 seconds, p i represents the basic position information of the ith pedestrian, including the position of obs ] of the pedestrian’s two-dimensional plane position information, specifically:

[0121]

[0122] in, and Respectively represent the two-dimensional horizontal and vertical position coordinate information of the i-th pedestrian at time step t1;

[0123] Step 2: Continue to sample image data at the same frequency in the two regions, collecting pedestrians at time step t2∈[T obs +1,T pre ] is the trajectory position information of p i* , as the real trajectory data in the dataset:

[0124]

[0125] Among them, T pre Set according to the actual situation and Tobs, the reference is 4.8 seconds, and Respectively represent the two-dimensional horizontal and vertical position coordinate information of the i-th pedestrian at time step t2;

[0126] Step 3: Collect the pedestrian historical trajectory information p from the source area and the target areai The nodes in each time step are defined as graph representation G t1 :

[0127] G t1 =(V t1 ,E t1 ,F t1 )

[0128] in, is the vertex set of pedestrians in the graph, representing all pedestrians at time t1, that is, the vertex set of N pedestrians; The value of is the coordinate position observed by the i-th pedestrian at time t1 Edge Set represents the relationship between the i-th and j-th pedestrians, and a relationship edge is constructed between every two pedestrians; and the feature set Defined as:

[0129]

[0130] Where W is the weight and σ(*) is the ReLU activation function;

[0131] Step 4: Calculate the social soft attention function and construct the adjacency matrix represents the attention weight between the i-th and j-th pedestrians, which is calculated as follows:

[0132]

[0133] in, and are the velocity vectors of pedestrian i and pedestrian j at time t1; α and β represent the velocity vector and and the angle between the two pedestrian connection paths, and They represent the position coordinates of pedestrian i and pedestrian j at time t1, and the value of γ is set to 0.1;

[0134] Step 5: Integrate information through the graph convolutional neural network (GCN) and perform graph convolution operations at time step t1, namely:

[0135]

[0136] Among them, A t1 is the adjacency matrix at time step t1, D t1 is the submatrix of the graph, E is the identity matrix, and the time step T1 to T obs The adjacency matrix of is stacked as:

[0137]

[0138] The matrix stack is:

[0139]

[0140] At the same time, the feature matrix of the lth layer in the GCN layer is stacked as:

[0141]

[0142] Therefore, calculate the GCN output F after the lth layer l+1 for:

[0143]

[0144] Among them, W l is the learnable parameter of the lth layer, and a total of three GCN network layers are used to extract the spatiotemporal feature representation of the observation trajectory, then l∈[1,3]; and the feature representation obtained by the last layer of GCN network is defined as Where N is the number of pedestrians in the trajectory domain, D f and L obs is the data dimension of the extracted features, and the feature representation of the source trajectory F(source) and the feature representation of the target trajectory F(target) are obtained;

[0145] Step 6: Calculate the attention score for the source track and the target track, which is calculated as:

[0146]

[0147] Among them, α and W h is a learnable weight parameter, tanh(*) is a tangent function, and f i (source) and f i (target) are the position features of the i-th pedestrian in the source trajectory domain and the target trajectory domain respectively;

[0148] Step 7: Based on the attention scores calculated in step 6, calculate the final feature representation c of the source trajectory domain and the target trajectory domain respectively i (source) and c i (target):

[0149] c i (source) = ρ i (source)·f i (source)

[0150] c i (target) = ρ i(target)·f i (target)

[0151] After that, the individual-level differences between the final feature representations of the two trajectory domains are calculated, and the calculated differences are mapped to the feature vector h through a fully connected layer. i Data features with the same dimensions, namely:

[0152] dif i =FL(c i (source)-c i (target),W fl )

[0153] Among them, W fl is the learnable weight matrix, FL(*,*) is the fully connected layer;

[0154] Step 8: Collect the historical trajectory information of pedestrians in the source area Integrate into the backbone network for position encoding and obtain the encoded feature data

[0155]

[0156] in, is the embedding layer; W e represents the weight coefficient of the embedding layer, which is obtained by step-by-step iterative optimization through network training; and t1∈[1,T obs ], i∈[1,N];

[0157] Step 9: Compare the pedestrian trajectories obtained in step 7 with the difference feature dif i And the position encoding feature obtained in the eighth step Features after channel splicing and fusion for:

[0158]

[0159] Among them, concat(*) represents the data channel concatenation operation;

[0160] Step 10: Use the multi-head attention mechanism in the backbone network to perform multi-head attention feature fusion on the spliced ​​data. The operation is as follows:

[0161]

[0162] Among them, Multihead(*) is the multi-head attention operation function;

[0163] Step 11: Add additional noise to the multi-head attention fusion features obtained in step 10, and input them into the trajectory decoder of the backbone network. Combined with the fully connected layer and the ReLU activation function, the prediction result of the pedestrian trajectory position is finally obtained, namely:

[0164] p′ i* =Decoder(v i ,W p )

[0165] Among them, p′ i* is the trajectory prediction result of the i-th pedestrian, Decoder(*) is the trajectory decoder processing function, W p is the network weight coefficient of the trajectory decoder;

[0166] Step 12: Based on the predicted results of the pedestrian's future trajectory in step 11 and the actual pedestrian trajectory in the training data set in step 2, a loss function based on mean square error (MSE) is designed to calculate the difference between the predicted value and the actual value. The loss function loss is defined as:

[0167]

[0168] Among them, p i* and p′ i* They represent pedestrian i at time step [T obs +1,T pre ]’s true position vector and predicted position vector, MSE(*,*) is the mean square error, and N is the total number of people;

[0169] Step 13: Initialize a fixed training weight for each training sample:

[0170]

[0171] Among them, m∈[1,M], and the initial network parameters of the pedestrian trajectory prediction model are initialized to θ0;

[0172] Step 14: Calculate the total loss value of the current iteration using the loss function designed in step 12 and the weighted training value of the sample;

[0173] Step 15: Update the network parameters of the pedestrian trajectory prediction model and update the weighted training weights of all samples;

[0174] Step 16: loop through steps 14 and 15 until the preset number of iterations is reached; then perform weighted fusion of model parameters based on the total loss calculation results of each iteration, and finally obtain a trained pedestrian trajectory prediction model;

[0175] Step 17: Deploy the trained pedestrian trajectory prediction model to the server, and transmit the data collected by the camera sensors in each area to the server respectively. The pedestrian trajectory prediction model can finally realize the pedestrian trajectory prediction at future moments in different area scenes.

[0176] The contents described in the embodiments of this specification are merely enumerations of implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A multi-scenario adaptive pedestrian trajectory prediction method based on attention mechanism, characterized in that: The method comprises the following steps: (1) Obtaining pedestrian historical position information: Collecting image information in the scene and extracting historical observation positions at a set time step as basic position information for pedestrian trajectory prediction; (2) Design a social attention module to capture the interactive information of pedestrians, use a graph convolutional network to further learn the motion trends, and output spatiotemporal features; (3) Design a trajectory alignment module to fully capture the diversity of individual behaviors in different scenarios, calculate the differences and align them to obtain the contrast difference features of pedestrian trajectories, and finally integrate them into the backbone network to obtain the prediction results of pedestrian trajectories; (4) Pedestrian trajectory prediction backbone network model design: Based on the multi-head attention mechanism, the feature information of the pedestrian's historical trajectory sequence is captured, and combined with the recurrent neural network to predict the pedestrian's trajectory position in the future time step, the model is finally able to adapt to trajectory prediction tasks in different scenarios; (5) Pedestrian trajectory prediction model training: A sample-weighted reinforcement training strategy is adopted to enhance the learning of complex trajectories. A weight is assigned to each training sample, and these weights are dynamically updated during the training process. For samples with poor pedestrian trajectory prediction accuracy, their training weights are increased so that they receive more attention in subsequent training, thereby improving the model's prediction accuracy for complex trajectory samples.

2. The multi-scenario adaptive pedestrian trajectory prediction method based on attention mechanism as claimed in claim 1, characterized in that: The process of step (1) is as follows: (1.1) Collect image information in the scene and extract the time step t1 = [1, T obs ] is used as the basic position information for pedestrian trajectory prediction, namely: Among them, T obs Set according to the actual situation, p i represents the basic position information of the ith pedestrian, including the position of obs ]’s pedestrian two-dimensional plane position information is as follows: in, and Respectively represent the two-dimensional horizontal and vertical position coordinate information of the i-th pedestrian at time step t1; (1.2) Collect and extract the i-th pedestrian at time step t2∈[T obs +1,T pre ] is the trajectory position information of p i* : Among them, T pre Set according to the actual situation and Tobs, and They respectively represent the two-dimensional horizontal and vertical position coordinate information of the i-th pedestrian at time step t2.

3. A multi-scenario adaptive pedestrian trajectory prediction method based on attention mechanism as claimed in claim 1 or 2, characterized in that: The process of step (2) is as follows: (2.1) Node representation design for pedestrian trajectory prediction. In the social attention module, the movement trajectory of pedestrians is described by a graph. The interaction between pedestrians in each time step is abstractly defined as a graph representation G t1 : G t1 =(V t1 ,E t1 ,F t1 ) in, is the vertex set of pedestrians in the graph, representing all pedestrians at time t1, that is, the vertex set of N pedestrians; The value of is the coordinate position observed by the i-th pedestrian at time t1 Edge Set Represents the relationship between the i-th and j-th pedestrians, and constructs a relationship edge between every two pedestrians; The feature set Defined as: Where W is the weight and σ(*) is the ReLU activation function; (2.2) Calculate the social soft attention function. The interactions between pedestrians include walking towards each other, walking away from each other, and walking side by side. A social soft attention function is proposed to construct the adjacency matrix represents the attention weight between the i-th and j-th pedestrians, which is calculated as follows: in, and are the velocity vectors of pedestrian i and pedestrian j at time t1; α and β represent the velocity vector and and the angle between the two pedestrian connection paths, and Respectively represent the position coordinates of pedestrian i and pedestrian j at time t1; In addition, when i=j, that is, the two pedestrian nodes are the same, γ is used to represent the self-attention value of the node, and the value of γ is calculated according to the adjacency matrix A t1 The maximum value of the elements in is set to 0.1; ① When two pedestrians walk towards each other, the angles α and β formed by their velocity vectors are both acute angles; at this time, cosα and cosβ are both greater than zero. If the result is greater than 0, it indicates that there is a significant impact between the two people, and both pedestrians need to pay attention to each other to avoid a head-on collision. ② When two pedestrians walk in opposite directions, the angles α and β formed by their velocity vectors are both obtuse angles; at this time, cosα and cosβ are both less than zero. The calculation result of is limited to 0, which means that there is no direct mutual influence between the two pedestrians leaving in opposite directions; ③ When two pedestrians walk side by side, that is, pedestrian i is in front and pedestrian j is behind, the angle α formed by the velocity vectors of the two is an obtuse angle, while β is an acute angle; if the moving speed of pedestrian j is significantly higher than that of pedestrian i, there is a possibility of future collision, and the output of the attention function is greater than 0, indicating a potential collision risk; on the contrary, if the speed of pedestrian j is lower than that of pedestrian i, then there is no collision risk, and the output of the attention function will be set to 0; The magnitude of the collision risk adopts a natural function form to represent the nonlinear exponential variation between pedestrians affected by the Euclidean distance and relative speed; (2.3) Trajectory domain feature output, using the graph convolutional network GCN to further extract spatiotemporal features, and perform graph convolution operations at time step t1, namely: Among them, A t1 is the adjacency matrix at time step t1, D t1 is the submatrix of the graph, E is the identity matrix, and the time step T1 to T obs The adjacency matrix of is stacked as: The matrix stack is: At the same time, the feature matrix of the lth layer in the GCN layer is stacked as: Therefore, calculate the GCN output F after the lth layer l+1 for: Among them, W l is the learnable parameter of the lth layer, and a total of three GCN network layers are used to extract the spatiotemporal feature representation of the observation trajectory, then l∈[1,3]; and the feature representation obtained by the last layer of GCN network is defined as Where N is the number of pedestrians in the trajectory domain, D f and L obs is the data dimension of the extracted features.

4. A multi-scenario adaptive pedestrian trajectory prediction method based on attention mechanism as claimed in claim 1 or 2, characterized in that: The process of step (3) is as follows: (3.1) Acquisition of source trajectory and target trajectory information: Collect the location information of pedestrian trajectories in the source scene and the target scene, and obtain the graph representation G of the pedestrian trajectories in the source scene respectively. t1 (source) = {V t1 (source),E t1 (source),F t1 (source)}, and the graph representation of pedestrian trajectories in the target scene G t1 (target)= {V t1 (target),E t1 (target),F t1 (target)}; (3.2) G t1 (source) and G t1 (target) are used as the input of the social attention module designed in step (2), and the feature representations of the source trajectory F(source) and the target trajectory F(target) are obtained respectively. These feature representations contain the spatiotemporal information of each pedestrian in different trajectory domains; (3.3) Calculation of attention score in trajectory domain. Since the same feature vector may have different performances in different trajectory domains, in order to determine the representativeness of the pedestrian’s feature vector in the trajectory domain, the attention mechanism is used to learn the relative correlation between the feature vector and the trajectory domain. The attention scores of the i-th pedestrian in the source trajectory domain and the target trajectory domain are ρ respectively. i (source) and ρ i (target), which is used to measure the importance of each pedestrian feature in a specific trajectory domain, is calculated as: Among them, α and W h is a learnable weight parameter, tanh(*) is a tangent function, and f i (source) and f i (target) are the position features of the i-th pedestrian in the source trajectory domain and the target trajectory domain respectively; (3.4) Pedestrian trajectory comparison difference feature dif i Calculate the attention score value ρ calculated based on step (3.3) i (source) and ρ i (target), respectively calculate the final feature representation c of the source trajectory domain and the target trajectory domain i (source) and c i (target): c i (source)=ρ i (source)·f i (source) c i (target)=ρ i (target)·f i (target) After that, the individual-level differences between the final feature representations of the two trajectory domains are calculated, and the calculated differences are mapped to the feature vector h through a fully connected layer. i Data features with the same dimensions, namely: dif i =FL(c i (source)-c i (target),W fl ) Among them, W fl is a learnable weight matrix, FL(*,*) is a fully connected layer, and finally, the calculated pedestrian trajectory is compared with the difference feature dif i Integrate it into the backbone network model to achieve pedestrian trajectory prediction based on comprehensive feature analysis.

5. A multi-scenario adaptive pedestrian trajectory prediction method based on attention mechanism as claimed in claim 1 or 2, characterized in that: In step (4), the processing process of the pedestrian trajectory prediction backbone network model is as follows: (4.1) Pedestrian historical trajectory position First, the neural network embedding layer is used to encode the position and obtain the encoded feature data in, is the embedding layer; W e represents the weight coefficient of the embedding layer, which is obtained by step-by-step iterative optimization through network training; and t1∈[1,T obs ], i∈[1,N]; (4.2) Since pedestrian motion patterns vary in different scenarios, the pedestrian trajectories obtained in step (3) under different scenarios are compared with the difference features dif i Introduced into the backbone network to solve the shift problem of contrast between different trajectory domains, that is, the fused features for: Among them, concat(*) represents the data channel concatenation operation; (4.3) Use the multi-head attention mechanism to encode the position feature data Projected into the query matrix, key matrix and value matrix, for the i-th pedestrian sequence, calculate and The matrix is: Among them, f Q (*), f K (*) and f V (*) Corresponding query, key and value functions, and They represent the query, key and value matrices of the i-th pedestrian under the j-th attention head respectively. The total number of multi-head attention heads is M, then j∈[1,M]; (4.4) Attention coefficient of multi-head attention And the feature vector calculation based on multi-head attention: Among them, softmax(*) is the softmax normalization function, d k is the normalization factor; h is the feature calculation result of the i-th pedestrian in the j-th attention head; i is the feature vector result of the i-th pedestrian after multi-head attention processing, which contains time dependency. concat(*) is the data channel concatenation operation; Therefore, the entire data operation based on the multi-head attention mechanism is: Among them, Multihead(*) is the multi-head attention operation function; (4.5) Construct a trajectory decoder based on a recurrent neural network. In order to enhance the robustness of pedestrian trajectory sequence prediction, a random vector noise that follows a Gaussian distribution is transformed into i Integrate into the trajectory decoder, then the input feature vector v of the trajectory decoder i for: v i =concat(h i ,noise i ) Secondly, a trajectory decoder is constructed based on the multi-head attention mechanism and recurrent neural network, and combined with the fully connected layer and ReLU activation function, the prediction result of the pedestrian trajectory position is finally obtained, namely: p′ i* =Decoder(v i ,W p ) Among them, p′ i* is the trajectory prediction result of the i-th pedestrian, Decoder(*) is the trajectory decoder processing function, W p is the network weight coefficient of the trajectory decoder.

6. A multi-scenario adaptive pedestrian trajectory prediction method based on attention mechanism as claimed in claim 1 or 2, characterized in that: The process of step (5) is as follows: (5.1) Trajectory prediction loss function design, the goal is to predict the pedestrian in the future time step t2∈[T obs +1,T pre ]’s trajectory position, therefore, the loss function loss is defined as: Among them, p i* and p′ i* They represent pedestrian i at time step [T obs +1,T pre ]’s true position vector and predicted position vector, MSE(*,*) is the mean square error, and N is the total number of people; (5.2) Parameter initialization. The total number of samples in the training data set is M, so the initial weights of all samples are initialized: Among them, m∈[1,M], and the initial network parameters of the pedestrian trajectory prediction model are initialized to θ0; (5.3) Iteratively train the pedestrian trajectory prediction model and calculate the loss value of each sample in the training dataset: in, represents the initialization weight of the mth training sample, loss m represents the direct loss value of the mth training sample calculated using step (5.1), L m Represents the weighted loss value of the mth sample; (5.4) Model parameter update, calculate the average loss Loss1 of the first iteration: Based on the loss calculation result Loss1, the model parameters are optimized using the gradient descent method, and the parameters θ1 of the pedestrian trajectory prediction model after the first iteration are obtained; (5.5) Update the sample training weights, and calculate the loss1, loss2, ..., loss according to the loss function of each sample in the first iteration M , update the weighted value of each set of training sample data, that is: in, is the training weight of the mth sample after the first iteration update, μ is the learning rate of the preset sample training weight, and μ>0; softmax(*) is the softmax function; (5.6) Repeat steps (5.3) to (5.5) for a total of T iterations, and obtain the total loss calculation values ​​Loss1, Loss2, ..., Loss for T iterations respectively. T , and the network model parameters θ1,θ2,...,θ of the pedestrian trajectory prediction model T ; (5.7) The final pedestrian trajectory prediction model is obtained by using the model parameter fusion method. First, the fusion factor τ of each model in the T-times trained model is obtained. k : Among them, τ k is the fusion factor of the pedestrian trajectory prediction model obtained by the kth iteration training, Loss k represents the total loss value of the kth iteration training, and k∈[1,T]; (5.8) The parameters of T pedestrian trajectory prediction models are weighted fused based on the fusion factor: Among them, θ mix is the fused pedestrian trajectory prediction model parameter, θ k is the parameter of the model at the kth iteration, and finally θ mix It is applied to the network parameters of the pedestrian trajectory prediction model and used as the final trained pedestrian trajectory prediction model.

Images