An adaptive pedestrian trajectory prediction method based on space-time fusion
By combining graph neural networks and multi-head attention mechanisms, the problem of insufficient generalization ability of pedestrian trajectory prediction models across different environments is solved, enabling effective handling of short-term and long-term behaviors and improving prediction accuracy and adaptability.
Patent Information
- Application Number
- CN202411965570.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-11-28
- Estimated Expiration
- 2044-12-30
AI Technical Summary
Existing pedestrian trajectory prediction models lack generalization ability across different environments and struggle to handle both short-term and long-term behavioral changes simultaneously, resulting in significant prediction errors.
A spatiotemporal fusion method based on graph neural networks is adopted. Scene features are extracted through graph convolutional networks, and deep fusion is performed using multi-head attention mechanism. Adaptive technology and recursive loop are combined to capture long-term temporal dependencies, thereby achieving cross-domain trajectory prediction.
It improves the model's generalization ability in different scenarios, enabling it to accurately predict pedestrian movement trajectories, adapt to potential differences in different environments, and capture long-term movement patterns.
Smart Images
Figure CN119962721B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer vision, and in particular to a self-adaptive pedestrian trajectory prediction method based on space-time fusion. BACKGROUND
[0002] Under the background of rapid urbanization, pedestrians, as an important part of urban transportation system, their behavior patterns and movement trajectories have a profound impact on traffic safety, urban management, etc. With the continuous growth of urban population and the gradual popularization of intelligent technology, urban transportation is facing increasingly serious challenges. How to effectively predict the movement trajectory of pedestrians to improve traffic safety and optimize urban management has become an important topic in traffic engineering and intelligent system research. Through accurate trajectory prediction, in-depth understanding of pedestrian behavior can be achieved, and necessary data support and decision basis can be provided for various traffic equipment and systems.
[0003] Although some progress has been made in recent years of literature research, there are still problems in the field differences. Existing methods assume that pedestrian behavior is consistent in different environments, ignoring the influence of space-time feature differences in different environments on behavior. For example, pedestrians in urban centers usually follow traffic rules, move faster and have straight paths; while in open spaces, pedestrians often act more randomly. Ignoring the potential differences between different environments will limit the generalization ability of the model in new environments or diverse environments, resulting in significant prediction errors.
[0004] Some recent research has tried to improve the cross-domain adaptability of trajectory prediction models to solve this problem. Some studies use recursive least squares algorithm for real-time adjustment, and some methods use graph neural networks to extract domain-invariant features by designing loss functions. However, the above methods still have certain limitations: recursive least squares algorithm is suitable for capturing short-term behavior changes, but it is difficult to handle long-term dependencies; while graph neural networks perform well in local interaction modeling, but in high-dimensional scenarios with complex trajectory data and frequent interactions, it is difficult to grasp the global behavior patterns.
[0005] How to improve the generalization ability of the model to different scenarios and be able to handle both short-term and long-term behavior changes has become a topic in this field. SUMMARY
[0006] In order to overcome the shortcomings of the prior art, the present application provides a kind of adaptive pedestrian trajectory prediction method based on space-time fusion, and the model obtains the motion characteristics of pedestrians in different scenes through graph neural network, and the space-time difference characteristics between domains are calculated using domain adaptation technology, which are used as different elements with pedestrian time series through multi-head attention mechanism for deep space-time fusion, to effectively cope with cross-domain trajectory prediction task, then, long-term time dependence is captured through the recursive cycle of main network, and the model can realize accurate and contextually appropriate prediction of pedestrian motion in different environments.
[0007] The technical scheme adopted by the present application to solve its technical problems is:
[0008] A kind of adaptive pedestrian trajectory prediction method based on space-time fusion, comprising the following steps:
[0009] S1: extract the time position feature vector of pedestrians in different scenes, pre-process information and build the graph structure of source scene and target scene;
[0010] S2: realize the alignment of domain features, use graph convolution network GCN and domain adaptation technology, adjust and integrate the space-time feature changes between source trajectory domain and target trajectory domain, so as to complete feature alignment;
[0011] S3: build time backbone network, encode time information, and domain space-time feature difference is used as an important element for prediction, deep fusion of space-time features is realized using multi-head attention mechanism, and high-precision pedestrian trajectory prediction is realized through autoregressive iteration.
[0012] Further, in the S1, if the scene contains N pedestrians, the position of each pedestrian at any time t is represented as Wherein is the two-dimensional coordinate of pedestrian i at time t recorded from the bird's eye view, and the total sequence length in the whole time range is defined as T seq , the observed time period is t=0 to t=T obs , the observation data of each time step t is abstracted as a graph G t =(P t ,E t ,F t ), which contains vertices, edges and related features, and for different trajectory scenes, i.e. source domain and target domain, exclusive graphs G(s) and G(t) are constructed to depict the interaction characteristics of pedestrians in a specific domain;
[0013] In the graph G t , the vertex set represents the position of N pedestrians at time t, and the vertex records the spatial position of the individual, which is the basis for analyzing its trajectory and motion pattern, and the edge set Et ={e ij |i=1,2,...,N;i≠j} describes the interaction between pedestrians i and j through the edge e ij The model can reveal the mutual influence of pedestrian motion in a group environment, and at the same time, the model defines a feature vector for each pedestrian, which constitutes a feature set F t ={f i t |i=1,2,...,N}, the feature vector f i t represents the motion characteristics of pedestrian i at time t, and its calculation formula is:
[0014]
[0015] where W is a weight matrix, is a ReLU activation function, the interaction between pedestrians is described by an adjacency matrix , and the element in the matrix is represented as:
[0016]
[0017] The interaction intensity is normalized according to the spatial distance between the two.
[0018] Further, the process of S2 is as follows:
[0019] S2.1: A graph convolutional network is used to extract the spatio-temporal features in the trajectory graph;
[0020] For the source trajectory domain and the target trajectory domain, the spatial GCN is used to aggregate the information of the adjacent nodes to generate a preliminary feature representation. The input trajectory graph is source domain G(s) and target domain G(t). At any time t, the trajectory graph G t It consists of two key parts: the normalized adjacency matrix and the submatrix defined as follows:
[0021]
[0022] where A t is the weighted adjacency matrix, D t is the diagonal submatrix of A t , and E is the identity matrix; within the observation period T obs , the adjacency matrix and the submatrix are stacked in time sequence:
[0023]
[0024] At the lth layer of the GCN, the form of the feature matrix is:
[0025]
[0026] where N is the number of pedestrians, D f represents the feature dimension, T obs is the number of time steps of observation, Next, the GCN updates the features of each layer using the following propagation rule:
[0027]
[0028] where W l represents the learnable weight matrix of the l-th layer, and σ(·) is a nonlinear activation function. Each layer gradually integrates larger spatial neighborhoods and longer temporal scales of contextual information based on the features extracted from the previous layer. Subsequently, the TCN is used to model the temporal dependencies of pedestrians. In order to stabilize feature propagation, a residual connection mechanism is added to combine the optimized features with the original input. Finally, the domain features F(s) and F(t) of the source domain and the target domain are generated.
[0029] S2.2: Align the generated features of the two domains to reduce the impact of the differences between the domains on prediction;
[0030] For each pedestrian i in the domain r e {s, t}, the attention score θ i (r) is calculated, which allows the model to weight the features based on their importance in the domain. The calculation formula of the attention score is as follows:
[0031]
[0032] where, represents the feature vector of pedestrian i in domain r, and parameters W h and a are learnable parameters used to adaptively adjust the attention score of a specific domain. After obtaining the attention score, the features in each domain are weighted and aggregated. The aggregated features of domain r are defined as:
[0033]
[0034] By weighting and aggregating the correlation of the feature vectors, the core features of each domain are extracted. Subsequently, to align the feature representations of the source domain and the target domain, the model compares the difference between the aggregated feature representations d(s) and d(t), and outputs the domain-specific feature difference Align i .
[0035] Further, the process of S3 is as follows:
[0036] S3.1: Time series encoding;
[0037] The observed pedestrian trajectory is encoded, and the input trajectory representation is a time series of spatial positions where each position corresponds to the two-dimensional spatial coordinates of pedestrian i at time t, a subsequent embedding layer is applied, which converts as follows:
[0038]
[0039] where, is an embedding function, W e is a learnable weight matrix, to emphasize the relative motion characteristics of pedestrians in the observation sequence, each position is offset by subtracting the last observation position to obtain the encoded time series
[0040] S3.2: Multi-head attention fusion;
[0041] A multi-head self-attention mechanism is used, and after the encoding step, the input sequence is mapped to query and key matrix representations through a learnable linear transformation, and the spatio-temporal difference feature Align i is mapped to a value matrix representation:
[0042]
[0043] where, and represent the corresponding query, key and value functions, respectively, considering the time dependence problem of subsequent prediction, the dot product between query and value is performed to calculate the attention coefficient for the weighted value, and the corresponding output is provided, and the calculation formula of the multi-head self-attention mechanism is as follows:
[0044]
[0045] where, d k is the dimension of the key vector, which is used as a scaling factor to stabilize the gradient in the optimization process, and the Softmax function ensures that the attention weights are correctly normalized across the entire sequence;
[0046] The output of the multi-head self-attention mechanism is then passed to the temporal feature optimization network to finally generate a series of feature vectors
[0047] S3.3: Decoding prediction;
[0048] The decoding process obtains the feature vector through a series of weights W pA parameterized linear layer, the decoder generates the predicted position of the next time step t=n+1 by adding the spatio-temporal difference feature Align in the prediction process i to enhance the field adaptability again:
[0049]
[0050] wherein, represents the decoding function defined by the parameter W p The decoding function defined by the parameter W
[0051] The beneficial effects of the present application mainly manifest in:
[0052] 1. The present application simulates the social interaction and movement trend of pedestrians by using graph structure, uses adaptive learning technology to deeply mine the trajectory feature difference between different scenes, and effectively improves the field generalization ability of the model.
[0053] 2. The present application uses self-attention recurrent neural network to capture the time dependence in the pedestrian trajectory, effectively models the long-term motion pattern and realizes the long-term evolution of the predicted trajectory.
[0054] 3. The present application adopts a new pedestrian trajectory prediction method, which combines various technical means to deeply integrate spatio-temporal difference features and time features, so that the model can adaptively adjust to cope with the potential trajectory difference between different scenes. BRIEF DESCRIPTION OF DRAWINGS
[0055] Figure 1 The adaptive pedestrian trajectory prediction method based on spatio-temporal fusion provided by the present application is a whole flow chart.
[0056] Figure 2 The field alignment part structure diagram of the present application.
[0057] Figure 3 The time backbone network structure diagram of the present application.
[0058] Figure 4 The training comparison diagram of the present application and the existing pedestrian trajectory prediction method on the ETH-UCY data set. DETAILED DESCRIPTION
[0059] The present application will be further described below in conjunction with the drawings.
[0060] Referring to Figures 1-4 , an adaptive pedestrian trajectory prediction method based on spatio-temporal fusion, comprising the following steps:
[0061] S1: Extract the time position feature vector of the pedestrian in different scenes, pre-process the information and construct the graph structure of the source scene and the target scene, the process is as follows:
[0062] If the scene contains N pedestrians, the position of each pedestrian at any time t can be represented as where is the two-dimensional coordinate of pedestrian i at time t recorded from the bird's eye view, and the total sequence length in the entire time range is defined as T seq , the observed time period is t=0 to t=T obs , the observation data at each time step t is abstracted as a graph G t =(P t ,E t ,F t ), which contains vertices, edges and related features, and the graphs G(s) and G(t) are constructed respectively for different trajectory scenes, i.e. source domain and target domain, to characterize the pedestrian interaction characteristics in a specific domain.
[0063] In the graph G t , the vertex set represents the positions of N pedestrians at time t, and the vertex records the spatial position of the individual, which is the basis for analyzing its trajectory and motion pattern, the edge set E t ={e ij |i=1,2,...,N; i≠j} describes the interaction between pedestrians i and j, through the edge e ij , the model can reveal the mutual influence of pedestrian motion in group environment, at the same time, the model defines a feature vector for each pedestrian, which constitutes the feature set F t ={f i t |i=1,2,...,N}, the feature vector f i t represents the motion characteristics of pedestrian i at time t, and its calculation formula is:
[0064]
[0065] where W is the weight matrix, is the ReLU activation function, the interaction between pedestrians is described by the adjacency matrix , and the element in the matrix is represented as:
[0066]
[0067] The interaction strength is normalized according to the spatial distance between them;
[0068] S2: alignment of field characteristics, using graph convolution network (GCN) and field adaptation technology, adjusting and integrating the spatio-temporal feature changes between the source trajectory field and the target trajectory field, so as to complete the feature alignment, the structure diagram of the method is as shown in Figure 2 The process is as follows:
[0069] S2.1: a graph convolution network is used to extract the spatio-temporal features in the trajectory graph;
[0070] For the source trajectory field and the target trajectory field, the spatial GCN is used to aggregate the information of the adjacent nodes to generate a preliminary feature representation, the input trajectory graph is source domain G(s) and target domain G(t), at any time t, the trajectory graph G t It consists of two key parts: normalized adjacency matrix And the sub-matrix The definitions are as follows:
[0071]
[0072] Where A t is the weighted adjacency matrix, D t is the diagonal sub-matrix of A t , and E is the identity matrix; within the observation period T obs , the adjacency matrix and the sub-matrix are stacked in time sequence:
[0073]
[0074] At the lth layer of the GCN, the form of the feature matrix is:
[0075]
[0076] Where N is the number of pedestrians, D f represents the feature dimension, T obs is the number of observation time steps, then the GCN updates the features of each layer using the following propagation rule:
[0077]
[0078] Where W l represents the learnable weight matrix of the lth layer, and σ(·) is a nonlinear activation function (such as PReLU), the GCN is designed as a three-layer structure (l=1, 2, 3), each layer gradually integrates larger range of spatial neighborhood and longer time scale of context information based on the features extracted by the previous layer, then TCN is used to model the time dependence of pedestrians, in order to stabilize the feature propagation, residual connection mechanism is added to combine the optimized features with the original input, finally, the domain features F(s) and F(t) of the source field and the target field are generated;
[0079] S2.2: Align the generated two domain features to reduce the prediction impact caused by the difference between domains;
[0080] Compute the attention score θ i (r) for each pedestrian i in domain r e {s, t}, which allows the model to weight the features based on their importance in the domain. The attention score is computed as follows:
[0081]
[0082] where, is the feature vector of pedestrian i in domain r, and parameter W h and a are learnable parameters that adaptively adjust the attention score for a specific domain. After obtaining the attention score, the features in each domain are weighted and aggregated. The aggregated features for domain r are defined as:
[0083]
[0084] By weighting and aggregating the correlation of the feature vectors, the core features of each domain are extracted. Subsequently, to align the feature representations of the source and target domains, the model compares the aggregated feature representations d(s) and d(t) and outputs the domain-specific feature difference Align i ;
[0085] S3: Build a temporal backbone network to encode temporal information, which, together with the domain spatiotemporal feature difference, is an important element for prediction. Utilize the multi-head attention mechanism to achieve deep fusion of spatiotemporal features, and implement high-precision pedestrian trajectory prediction through autoregressive iteration. The structural diagram of the method is shown in Figure 3 , and the process is as follows:
[0086] S3.1: Temporal encoding;
[0087] Encode the observed pedestrian trajectory. The input trajectory representation is a time series containing spatial positions where each position corresponds to the two-dimensional spatial coordinates of pedestrian i at time t. Then apply an embedding layer, whose conversion process is as follows:
[0088]
[0089] where, is the embedding function, and W e is a learnable weight matrix. To emphasize the relative motion characteristics of the pedestrians in the observation sequence, each position is offset by subtracting the last observed position to obtain the encoded time series
[0090] S3.2: Multi-head attention fusion;
[0091] The input sequence is fused by a multi-head self-attention mechanism after the encoding step The spatio-temporal discrepancy feature Align i is mapped to a value matrix representation by a learnable linear transformation:
[0092]
[0093] where, and denote the corresponding query, key and value functions, respectively. To consider the temporal dependency problem of subsequent prediction, the dot product between query and value is performed to calculate the attention coefficient for weighting value and provide the corresponding output. The calculation formula of the multi-head self-attention mechanism is as follows:
[0094]
[0095] where, d k is the dimension of the key vector, which is used as a scaling factor to stabilize the gradient in the optimization process, and the Softmax function ensures that the attention weight is correctly normalized in the entire sequence,
[0096] The output of the multi-head self-attention mechanism is then passed to the temporal feature optimization network to finally generate a series of feature vectors
[0097] S3.3: Decoding prediction;
[0098] The decoding process obtains the feature vectors through the self-attention mechanism, and the decoder generates the predicted position of the next time step t = n + 1 through a series of linear layers parameterized by weights W p , which adds the spatio-temporal discrepancy feature Align i again to enhance the field adaptation ability in the prediction process:
[0099]
[0100] where, denotes the decoding function defined by the parameters W p , and the output generates the future trajectory position of the pedestrian.
[0101] In this embodiment, the experiment is carried out on pytorch, and the network training and testing are carried out on NVIDIA RTX4080 GPU. During the experiment, the model is trained and evaluated on the ETH and UCY public data sets, and compared with other methods using the same data set.
[0102] Specifically, the following steps are included:
[0103] Step 1, data preprocessing
[0104] For the data set, we use the downsampling operation to only calculate the number of trajectories whose time step length is greater than or equal to 20.
[0105] Step 2, graph structure construction
[0106] By constructing the graph structure G(s) and G(t), the interaction behavior of pedestrians in the source trajectory field and the target trajectory field is simulated respectively, and the graph of each time step t contains a set of vertexes P t , a set of edges E t reflecting the relationship between pedestrians, and a set of node features F t describing motion characteristics.
[0107] Step 3, trajectory field feature extraction
[0108] Based on the initial feature set F 0 , GCN is used to extract spatio-temporal features. GCN aggregates the information of adjacent nodes to generate preliminary feature representation. In this process, the features are processed by TCN. In order to stabilize feature propagation, a residual connection mechanism is added to combine the optimized features with the original features F 0 . If the input and output dimensions are inconsistent, adjust through a 1x1 convolutional layer. Finally, generate the features of the target field and the source field F(s) and F(t) through PReLU activation.
[0109] Step 4, field feature alignment
[0110] The extracted features F(s) and F(t) will be further aligned to reduce the difference between the source field and the target field. First, the features are reduced through a linear layer and adjusted through a Tanh activation function to improve stability. Then, another linear layer generates an attention weight vector, which is normalized as θ i (s) and θ i (t). Based on these weights, the features are weighted and summed to obtain the field aggregation features d(s) and d(t). Subsequently, by calculating the difference between the two, the field difference feature Align i is obtained, which contains key spatio-temporal change information.
[0111] Step 5, time series encoding
[0112] Observed input sequence First, the generated features are projected into a feature space by a linear layer, and then passed through a ReLU activation function and a dropout process to obtain the encoded sequence
[0113] Step 6, multi-head attention fusion
[0114] Time series Align with domain difference features i As a prediction element, the multi-head attention mechanism is used to capture the time dependence across time steps by calculating the relationship between features at different time points to generate attention output. To ensure the stability of the model when the network depth increases, the attention output is combined with the original input through a residual connection Subsequently, the output is further stabilized by layer normalization to improve the convergence during training. The normalized output is further optimized by a feedforward neural network. The features are first projected into a high-dimensional space by a linear layer, and then passed through a ReLU activation function to introduce nonlinearity. Subsequently, a second linear layer maps the features back to the original dimension. To preserve the original input feature information, the network output is combined with the normalized input through a second residual connection and further processed by additional layer normalization to generate the encoded features
[0115] Step 7, decoding prediction
[0116] Encoded features are used to predict future positions in an autoregressive manner. In each step, the features are combined with the domain difference features i to improve cross-domain adaptability. The combined features generate predicted positions through a linear layer The predicted positions are combined with the observed input sequence to form the input for the next iteration, enabling the decoder to recursively predict future positions across multiple time steps.
[0117] Step 8, comparative test
[0118] The proposed method is evaluated on two widely used pedestrian trajectory prediction datasets, ETH and UCY, the ETH dataset consists of two scenarios, ETH and HOTEL, and the UCY dataset consists of three subsets, UNIV, ZARA1 and ZARA2, the datasets provide human motion data in real-world scenarios, each trajectory point is captured at a frequency of 2.5Hz (i.e. once every 0.4 seconds), during training and evaluation, each observed trajectory is 8 frames (3.2 seconds) and the predicted trajectory is 12 frames (4.8 seconds), and the results are compared with other advanced methods, we use two evaluation indicators:
[0119]
[0120] ADE is used to measure the average L2 distance between the predicted trajectory and the real trajectory at all time steps. FDE calculates the L2 distance between the final position of the predicted trajectory and the real trajectory at the end of the prediction time range, by Figure 4 It can be seen that we have the best results.
[0121] The embodiments of the present specification are only enumerations of implementation forms of the inventive concept, and are only used for the purpose of description. The protection scope of the present application should not be regarded as being limited to the specific forms presented in the present embodiments, and the protection scope of the present application also extends to equivalent technical means that can be thought of by those skilled in the art according to the inventive concept.
Claims
1. A spatio-temporal fusion based adaptive pedestrian trajectory prediction method, characterized in that, The method comprises the following steps: S1: extracting the time position feature vector of pedestrians in different scenes, preprocessing information, and constructing a graph structure of the source scene and the target scene; S2: realizing alignment of domain features, adjusting and integrating the spatiotemporal feature changes between the source trajectory domain and the target trajectory domain by using a graph convolution network (GCN) and a domain adaptation technology, thereby completing feature alignment; S3: building a time backbone network, encoding time information, and using the domain spatiotemporal feature difference as an important element for prediction, realizing deep fusion of spatiotemporal features by using a multi-head attention mechanism, and realizing high-precision pedestrian trajectory prediction through self-recurrent iteration; In the S1, if the scene contains N pedestrians, the position of each pedestrian at any time t is represented as where is the two-dimensional coordinate of pedestrian i at time t recorded from the bird's eye view, and the total sequence length over the entire time range is defined as The observed time period is to The observation data at each time step t is abstracted as a graph which contains vertices, edges and related features. For different trajectory scenarios, i.e., the source domain and the target domain, exclusive graphs and are constructed to depict the pedestrian interaction characteristics in a specific domain. Figure In this paper, the set of vertices represents the position of N pedestrians at time t, the vertex records the spatial position of individuals, which is the basis for analyzing their trajectories and movement patterns, the set of edges describes the interaction between pedestrians i and j, through the edge , the model can reveal the mutual influence of pedestrian movement in group environment, at the same time, the model defines a feature vector for each pedestrian, which constitutes the feature set , the feature vector represents the movement characteristics of pedestrian i at time t, and its calculation formula is: ； where W is a weight matrix, is a ReLU activation function, the interaction relationship between pedestrians is described by an adjacency matrix The elements in the matrix are represented as: ； The interaction intensity is normalized according to the spatial distance between the two; The process of S2 is as follows: S2.1: using a graph convolution network to extract spatiotemporal features in a trajectory graph; For source trajectory domain and target trajectory domain, the preliminary feature representation is generated by using spatial GCN to aggregate information of respective neighboring nodes, and the input trajectory graph is respectively and target domain At any time t, the trajectory graph is composed of two key parts: the normalized adjacency matrix and the submatrix , which are defined as follows: ； ； wherein is a weighted adjacency matrix, is a diagonal submatrix of E is the identity matrix; within an observation period adjacency matrix and the submatrix are stacked in chronological order: ； ； At the first layer of the GCN, the feature matrix has the form: ； where N is the number of pedestrians, denotes the feature dimension, is the number of observed time steps, Next, the GCN updates the features of each layer using the following propagation rule: ； wherein, represents a learnable weight matrix of the l-th layer, is a nonlinear activation function, and the GCN is designed as a three-layer structure, each layer gradually integrates larger spatial neighborhoods and longer temporal scale context information based on the features extracted by the previous layer, then the TCN is used to model the temporal dependencies of pedestrians, in order to stabilize the feature propagation, a residual connection mechanism is added to combine the optimized features and the original input, finally, the domain features of the source domain and the target domain are generated and ; S2.2: aligning the generated two domain features to reduce the prediction influence caused by the difference between the domains; For each pedestrian i in the field The attention score is calculated Let the model be able to weight the features based on their importance in the field, and the calculation formula of the attention score is as follows: ； wherein, represents the feature vector of pedestrian i in domain r, parameters and are learnable parameters for adaptively adjusting the attention score of a specific domain, after obtaining the attention score, the features in each domain are weighted aggregated, the aggregated features of domain r are defined as: ； By weighting and aggregating the correlation of the feature vectors, the core features of each domain are extracted, and then, for the alignment of the feature representations of the source domain and the target domain, the model will aggregate the feature representations and perform difference comparison, and output domain-specific feature differences .
2. The spatio-temporal fusion based adaptive pedestrian trajectory prediction method of claim 1, wherein, The process of S3 is as follows: S3.1: time sequence encoding; The observed pedestrian trajectories are encoded, input trajectory representation is a time series of spatial positions where each position corresponds to the two-dimensional spatial coordinates of pedestrian i at time t, followed by an embedding layer which transforms as follows: ； in, For embedded functions, To create a learnable weight matrix, and to emphasize the relative motion characteristics of pedestrians in the observation sequence, each position... Subtract the last observation position To perform the offset, obtain the encoded time series. ; S3.2: multi-head attention fusion; Using multi-head self-attention mechanism, the input sequence is mapped into a sequence of vectors The spatio-temporal discrepancy features are mapped into a value matrix representation by a learnable linear transformation: The spatio-temporal discrepancy features are mapped into a value matrix representation by a learnable linear transformation: ； where, , and denote the corresponding query, key and value functions, respectively; considering the temporal dependency issue of subsequent prediction, the dot product between query and value is performed to calculate the attention coefficient for weighted value, and the corresponding output is provided, and the calculation formula of multi-head self-attention mechanism is as follows: ； where, is the dimension of the key vector, used as a scaling factor to stabilize the gradients during optimization, and the Softmax function ensures that the attention weights are properly normalized across the sequence. The output of the multi-head self-attention mechanism is then passed to a temporal feature optimization network, which ultimately generates a series of feature vectors ; S3.3: decoding prediction; Decoding processes the feature vectors obtained by the self-attention mechanism The decoder generates the predicted position for the next time step by a series of linear layers parameterized by weights The decoder is enhanced again by adding spatio-temporal difference features during the prediction process to enhance the domain adaptation capability: ； wherein, represents a decoding function defined by parameters that outputs the future trajectory positions of the pedestrian.