A pedestrian trajectory prediction method based on hierarchical envelope domain adaptation

Abstract

The application discloses a pedestrian trajectory prediction method based on hierarchical envelope domain adaptation, and comprises the following steps: obtaining source domain original trajectory samples and target domain original trajectory samples; constructing a pedestrian trajectory prediction model based on hierarchical envelope domain adaptation, namely, a MESC-HEDA model; training the MESC-HEDA model by using the source domain original trajectory samples and the target domain original trajectory samples, and predicting pedestrian trajectories in the target domain original trajectory samples, wherein the MESC-HEDA model comprises a two-stage envelope sample construction module, a global hierarchical envelope sample construction module and a hierarchical envelope domain adaptation module, wherein the two-stage envelope sample construction module comprises a local hierarchical envelope sample construction submodule, an individual hierarchical envelope sample construction submodule and a two-stage envelope sample feature fusion submodule; and the hierarchical envelope domain adaptation module comprises a local hierarchical envelope domain adaptation submodule and a global hierarchical envelope domain adaptation submodule. The application effectively solves the domain shift problem in the pedestrian trajectory scene.

Description

Technical Field

[0001] This invention belongs to the field of trajectory prediction technology, specifically relating to a pedestrian trajectory prediction method based on hierarchical envelope domain adaptation. Background Technology

[0002] Pedestrian trajectory prediction in complex environments is extremely challenging, influenced by numerous complex factors, including pedestrian movement status, destination, age, gender, and interactions between multiple pedestrians. Early research largely relied on knowledge models such as social force models and constant velocity models; however, the nonlinearity of pedestrian interactions and the complexity of these factors resulted in poor generalizability and significant limitations. In recent years, pedestrian trajectory prediction models based on machine learning, especially deep learning, have become a research hotspot, offering improved prediction accuracy compared to traditional algorithms. However, most deep learning-based methods often assume consistent marginal probability distributions between training and test sets, focusing on learning uniform interaction patterns between pedestrians and between pedestrians and the geographical environment, while neglecting the domain offset between training and test sets, leading to poor model generalization performance. For example, in densely populated areas like amusement parks or shopping malls, pedestrian interactions are very frequent, often requiring constant avoidance, detours, or waiting to avoid collisions and blockages, resulting in tortuous, complex, and constantly changing trajectories. In contrast, on streets, collisions and interactions between pedestrians are less frequent, and pedestrians tend to follow the planned road direction, with trajectories mostly in straight lines. Ignoring the distribution differences between the above scenarios will inevitably affect the accuracy of pedestrian trajectory prediction when a model built from one domain is applied to another domain.

[0003] Performance analyses of pedestrian trajectory prediction models are mostly based on five trajectory scenarios: ETH, HOTEL, UNIV, ZARA1, and ZARA2. Quantitative evaluations using metrics such as sample size, number of pedestrians, walking speed, and acceleration reveal significant differences between domains. The same metric varies considerably across different scenarios, a phenomenon commonly referred to as the pedestrian trajectory scenario domain shift problem. Common solutions include treating the training scenario as the source domain and the test scenario as the target domain, and using domain adaptation methods to achieve distribution alignment. While recent pedestrian trajectory prediction models based on domain adaptation have emerged, they only consider local social level samples and local-level domain adaptation, limiting their ability to handle the multi-layered complexity and uncertainty of interactions between pedestrians and between pedestrians and the environment. In fact, pedestrian trajectories are influenced by three factors: individual pedestrian attributes, the social characteristics of neighboring pedestrians, and the characteristics of the global geographical environment. Therefore, a pedestrian trajectory prediction method with domain adaptation is necessary. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a pedestrian trajectory prediction method based on hierarchical envelope domain adaptation.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A pedestrian trajectory prediction method based on hierarchical envelope domain adaptation includes:

[0007] Obtain the original trajectory samples of the source domain and the original trajectory samples of the target domain;

[0008] Construct a pedestrian trajectory prediction model based on hierarchical envelope domain adaptation, namely the MESC-HEDA model;

[0009] The MESC-HEDA model is trained using original trajectory samples from the source domain and the target domain, and pedestrian trajectories in the original trajectory samples from the target domain are predicted.

[0010] The MESC-HEDA model comprises: a two-level envelope sample construction module, a global-level envelope sample construction module, and a hierarchical envelope domain adaptation module. The two-level envelope sample construction module includes a local-level envelope sample construction submodule, an individual-level envelope sample construction submodule, and a two-level envelope sample feature fusion submodule. The hierarchical envelope domain adaptation module includes a local-level envelope domain adaptation submodule and a global-level envelope domain adaptation submodule.

[0011] The local hierarchical envelope sample construction submodule is used to generate local hierarchical envelope samples based on the pedestrian adjacency relationships in the sample space hierarchy, including local hierarchical envelope samples of the source domain and local hierarchical envelope samples of the target domain.

[0012] The individual-level envelope sample construction submodule is used to generate individual-level envelope samples based on the pedestrian relationships at the individual level in the samples, including individual-level envelope samples from the source domain and individual-level envelope samples from the target domain.

[0013] The bi-level envelope sample feature fusion submodule is used to generate bi-level envelope samples based on local-level envelope samples and individual-level envelope samples, including bi-level envelope samples from the source domain and bi-level envelope samples from the target domain.

[0014] The global hierarchical envelope sample construction module is used to generate global hierarchical envelope samples based on the spatiotemporal feature distribution of the two-level envelope samples, including global hierarchical envelope samples of the source domain and global hierarchical envelope samples of the target domain.

[0015] The local hierarchical envelope domain adaptation submodule is used to determine the local hierarchical domain distribution alignment loss based on the local hierarchical envelope samples of the source and target domains using an attention mechanism.

[0016] The global hierarchical envelope domain adaptation submodule is used to determine the global hierarchical domain distribution alignment loss based on the global hierarchical envelope samples of the source and target domains.

[0017] Furthermore, based on the pedestrian adjacency relationships in the sample space hierarchy, local hierarchical envelope samples are generated, including:

[0018] For any time t of all observed trajectories, i.e. t∈{T1, ..., T} obs}, T1, T2, ..., T obs Indicate the observation trajectory time, execute S21-S27:

[0019] S21: Obtain the original trajectory sequence samples of N pedestrians at time t, i.e., the coordinates of N pedestrians at time t. tr t i Let represent the coordinates of pedestrian i at time t, where i∈{1,2,...,N};

[0020] S22: Will Processing as a graph representation G loc =(V loc E loc ),in, G represents loc The vertex set, G represents loc edge set, The edge formed by the coordinates of pedestrian i and pedestrian j is determined by the local hierarchical pedestrian adjacency relation R. loc and the local hierarchical adjacency matrix A loc express;

[0021] S23: Determine the feature vectors F of N pedestrians at time t. loc ,in, This represents the velocity vector of pedestrian i at time t; and Let x and y represent the components of the coordinates of pedestrian i at time t, respectively. and Let x and y represent the components of the coordinates of pedestrian i at time t-1, respectively.

[0022] S24: Through the local hierarchical envelope mapping function F lhe Determine the local hierarchical pedestrian adjacency relation R between any two pedestrians among all pedestrians. loc , in, tr represents the velocity of pedestrian j at time t. t j Let represent the coordinates of pedestrian j at time t,

[0023]

[0024] and This represents the x and y components of the coordinates of pedestrian j at time t; This represents the Euclidean distance between pedestrians i and j after filtering at time t; r1 and r2 represent the two distances set.

[0025] Describes the velocity vector of pedestrian i at time t. The angle between the line connecting the current position of pedestrian j and the line connecting the current position of pedestrian j, and the angle between the lines connecting the current position and the line connecting the current position of pedestrian j. Taking the cosine function, we get

[0026] This represents the combined metric of the angle between the velocities of pedestrians i and j at time t.

[0027]

[0028] S25: By introducing a threshold hyperparameter η∈[0,1] to adjust R loc Perform local-level envelope sample interaction strength filtering to generate a local-level mask M. loc ,in, It is an indicator function, when R loc Output 1 if ≥η, otherwise output 0;

[0029] S26: Element-wise multiplication operation on R loc The selection process is performed, and the identity matrix I is superimposed to generate a local hierarchical adjacency matrix A. loc A loc =M loc ⊙R loc +I, ⊙ indicates element-wise multiplication;

[0030] S27: Apply the Zero-Softmax method to the local hierarchical adjacency matrix A loc Normalization is performed to generate a normalized local hierarchical adjacency matrix A'. loc Thus, the local hierarchical envelope sample [F] corresponding to time t is obtained. loc A′ loc ];

[0031] For all observed trajectory times corresponding to F loc and A' loc Stacking is performed to obtain local hierarchical envelope samples [F] Loc A Loc ], where F Loc ={F1, F2, ..., F obs}, F t F represents the value corresponding to time t. loc A Loc={A′1,A′2,...,A′ obs}, A′ t A' represents the value of A' at time t. loc , t∈{T1,...,T obs}

[0032] Furthermore, based on the pedestrian relationships at the individual level in the sample, an individual-level envelope sample is generated, including:

[0033] For any pedestrian i among all pedestrians, i ∈ {1, 2, ..., N}, where N represents the total number of pedestrians, execute S31-S37:

[0034] S31: Obtain all observation trajectory times T1, T2, ..., T obs Coordinates of insider i in, tr t i Let represent the coordinates of pedestrian i at time t, where t∈{T1, ..., T} obs};

[0035] S32: Will Processing as a graph representation G ind =(V ind E ind ),in, For G ind The vertex set, It is G ind The edge set, any edge Pedestrian adjacency relationship R at the individual level ind And the individual hierarchical adjacency matrix A ind express;

[0036] S33: Pedestrian i during time period T1 to T2 obs The corresponding eigenvector F ind , This represents the velocity vector of pedestrian i at time t;

[0037] S34: Using self-attention mechanisms, determine individual-level pedestrian adjacency relationships R. ind ,in,

[0038]

[0039] W Q and W K It is a learnable parameter; Q ind and K ind These represent the query and key in the self-attention mechanism, respectively. Indicates K indTranspose; D represents the scaling factor; F′ ind =(F ind +PE)W ind Represents a high-dimensional tensor with positional encoding information; PE = {fraction e} t |t = T1, ..., T obs} represents the trajectory of a single pedestrian. t i In time period T l To T obs Position-encoded tensor within the frame; t W represents the number of time frames at time t; ind Indicates learnable parameters;

[0040] S35: By setting the threshold hyperparameter ψ∈[0,1] for R ind Perform interactive filtering of individual-level envelope samples to generate an individual-level mask M. ind ,in, It is an indicator function, when R ind Output 1 if the value is greater than or equal to ψ, otherwise output 0.

[0041] S36: Mask the individual hierarchy M ind Adjacency relationship with individual pedestrians R ind The fusion process yields the individual-level adjacency matrix A. ind A ind =M ind ⊙R ind , ⊙ indicates element-wise multiplication;

[0042] S37: Using the Zero-Softmax method to process the individual hierarchical adjacency matrix A ind Normalization is performed to generate a normalized individual hierarchical adjacency matrix A′. ind Thus, the individual-level envelope sample [F] corresponding to pedestrian i is obtained. ind A′ ind ];

[0043] For all pedestrians, F loc and A′ loc Stacking is performed to obtain individual hierarchical envelope samples [F] Ind A Ind ], where F Ind ={F1, F2, ..., F N}, F i F represents the value corresponding to pedestrian i ind A Ind ={A′1,A′2,...,A′ N}, A′ iA′ represents the pedestrian i. ind , i∈{1,2,...,N}.

[0044] Furthermore, based on the local-level envelope samples and the individual-level envelope samples, a two-level envelope sample is generated, including:

[0045] Graph convolution is performed on the graph structure of the local hierarchical envelope samples to obtain the local hierarchical feature representation F′ of the local hierarchical envelope samples. Loc , where F′ Loc =σ(A Loc F Loc W Loc ), where σ represents the activation function ReLU(); W Loc Indicates learnable parameters;

[0046] Graph convolution is performed on the graph structure of the individual-level envelope samples to obtain the individual-level feature representation F′ of the individual-level envelope samples. Ind , where F′ Ind =σ(A Ind F Ind W Ind ), where σ represents the activation function ReLU(); W Ind Indicates learnable parameters;

[0047] Representing local hierarchical features as F′ Loc Individual hierarchical feature representation F′ Ind By splicing and fusion, the feature representation F of the bipolar envelope sample is obtained. tef ,in, This indicates that the data is spliced ​​and fused according to the feature dimensions; L represents the set of real numbers; obs represents the total number of all observation times; D represents the feature dimension of the local hierarchical envelope sample and the individual hierarchical envelope sample.

[0048] Furthermore, based on the spatiotemporal feature distribution of the two-level envelope samples, global hierarchical envelope samples are generated, including:

[0049] The feature representation F of the bipolar envelope sample tef The input is a multi-layer temporal convolutional network (TCN) with l layers. The output of the previous layer is used as the input of the next layer to obtain the predicted trajectory. For the l-th layer of the TCN:

[0050]

[0051] in, W represents the input of the l-th layer TCN. (l) For the learnable parameters of the l-th layer TCN, This represents the output of the l-th layer, i.e., the predicted output of a multi-layer TCN;

[0052] The output of the lth layer Five parameters F of the bivariate Gaussian distribution of the predicted trajectory coordinates are determined by a linear layer. final ,Right now Where Linear() represents linear layer processing, t∈[T] obs+1 T obs+2 , ..., T pre ], T obs+1 T obs+2 , ..., T pre Represents the time frame of the predicted trajectory. The parameter representing the mean trajectory value of pedestrian i at time t in the predicted trajectory time frame includes the mean trajectory values ​​in both the x and y directions. and Let represent the standard deviation component of pedestrian i's trajectory at time t, which includes the standard deviations of the trajectory in both the x and y directions. and Time represents the correlation coefficient of pedestrian i's trajectory at time t;

[0053] Based on the average trajectory parameter of each pedestrian Calculate the linear velocity of the pedestrian at time t. Right now

[0054] Predict the time period T obs+1 To T pre The linear velocity forms the vector U i ,Right now Among them, L pre Indicates the predicted time period T obs+1 To T pre The total number of predicted trajectory moments;

[0055] U of all pedestrians i By splicing the samples, we obtain the global hierarchical envelope samples. Right now

[0056] Furthermore, based on the local hierarchical envelope samples of the source and target domains, an attention mechanism is used to determine the local hierarchical domain distribution alignment loss, including:

[0057] Obtain local hierarchical feature representations of local hierarchical envelope samples from the source and target domains. and Where, N s and N t These represent the number of pedestrians in the local hierarchical envelope samples of the source and target domains, respectively.

[0058] The local hierarchical features of the source and target domains are represented by F′. Loc(s) and F′ Loc(t) Break it down according to pedestrians, that is

[0059] in,

[0060] For each local hierarchical envelope sample in the source domain

[0061] Will Flattened into D c The size of the feature vector, where D c =L obs ×D,

[0062] Determine the interaction weights of different pedestrians in the local hierarchical envelope samples of the source domain. Right now in, express transpose,

[0063] The summation of interpersonal relationship interactions in the local hierarchical envelope samples of the source domain is determined by J. (s) ,Right now

[0064] J corresponding to all local hierarchical envelope samples in the source domain (s) Stacking the features yields a local hierarchical representation M of the pedestrian attention features in the source domain. (s) ,Right now Among them, K (s) This indicates the number of local hierarchical envelope samples in the source domain;

[0065] For each local layer envelope sample in the target domain

[0066] Will Flattened into D c The size of the feature vector, where D c =L obs ×D,

[0067] Determine the interaction weights of different pedestrians in the local hierarchical envelope samples of the target domain. Right now in, express transpose,

[0068] The summation of interpersonal relationship interactions in the local hierarchical envelope samples of the target domain is expressed as J. (t) ,Right now

[0069] J corresponding to all local hierarchical envelope samples in the target domain(t) Stacking is performed to obtain the local hierarchical pedestrian attention feature set representation M of the target domain. (t) ,Right now Among them, K (t) This indicates the number of local hierarchical envelope samples in the target domain;

[0070] M is represented by local hierarchical pedestrian attention feature sets in the source and target domains. (s) and M (t) Determine the local hierarchical domain distribution alignment loss L loc L loc =Dist1(M (s) M (t) ), where Dist1() represents the first distance function.

[0071] Furthermore, based on the global hierarchical envelope samples of the source and target domains, the global hierarchical domain distribution alignment loss is determined, including:

[0072] Obtain the global hierarchical envelope samples F of the source and target domains. glo(s) and F glo(t) ;

[0073] Determine the linear velocity difference ratio between the source domain trajectory and the target domain trajectory. Let i represent the linear velocity of pedestrian i in the source domain at time t. N represents the linear velocity of pedestrian j in the target domain at time t. s N represents the total number of pedestrians in the source domain of the training set. t This represents the total number of pedestrians in the target domain of the validation set;

[0074] Determine the global hierarchical domain distribution alignment loss L glo L glo =Dist2(F glo(s) ,κ×F glo(t) ), where Dist2() represents the second distance function.

[0075] Furthermore, the objective function L for training the MESC-HEDA model using the original trajectory samples from the source domain and the original trajectory samples from the target domain is:

[0076] L = L pre +λ1L loc +λ2L glo ,

[0077] Among them, L pre Let λ1 and λ2 represent the prediction loss, and N represent the total number of pedestrians in the source domain.

[0078]

[0079] Represents the probability density function. The parameter representing the mean trajectory value of pedestrian i at time t in the predicted trajectory time frame includes the mean trajectory values ​​in both the x and y directions. and Let represent the standard deviation component of pedestrian i's trajectory at time t, which includes the standard deviations of the trajectory in both the x and y directions. and Time represents the correlation coefficient of pedestrian i's trajectory at time t.

[0080] The beneficial effects of this invention are:

[0081] This invention constructs a bipolar envelope sample by designing individual hierarchical envelope samples and combining them with local hierarchical envelope samples. Furthermore, based on the spatiotemporal characteristics of the bipolar envelope samples, a global envelope sample is generated, thereby constructing a multi-level envelope sample. This allows for the acquisition of richer spatiotemporal feature representations of pedestrian trajectory data at different levels.

[0082] This invention designs a local hierarchical domain adaptation module and a global hierarchical domain adaptation module between the source domain and the target domain for samples at all levels of envelope, thereby realizing hierarchical envelope domain adaptation. Through transfer learning via the hierarchical envelope domain adaptation module, it can better adapt to the multi-level complexity and uncertainty between pedestrians and between pedestrians and the environment.

[0083] This invention further enhances the transfer learning capability of the model's multi-level cross-domain spatiotemporal features by weighted fusion of the domain distribution differences of local and global level feature representations, thereby achieving pedestrian trajectory prediction under domain distribution alignment.

[0084] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0085] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0086] Figure 1 This is a flowchart of a pedestrian trajectory prediction method based on hierarchical envelope domain adaptation;

[0087] Figure 2 This is a block diagram of the MESC-HEDA model;

[0088] Figure 3 This is a schematic diagram of the local hierarchical envelope sample construction module;

[0089] Figure 4 This is a schematic diagram of the individual-level envelope sample construction module. Detailed Implementation

[0090] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the preferred embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0091] This invention proposes a pedestrian trajectory prediction model (MESC-HEDA) based on Hierarchical Envelope Domain Adaptation (HEDA). MESC stands for Multi-level Envelope Sample Construction. First, a local-level envelope sample construction module is designed by constructing local-level pedestrian adjacency relationships, and an individual-level envelope sample construction module is designed based on individual-level pedestrian relationships. These two modules are then merged to form a two-stage envelope sample construction (TESC) module. Next, based on the two-stage envelope sample construction module, the spatiotemporal feature distribution of all pedestrian trajectories is calculated, thereby constructing a global-level envelope sample. Then, based on attention mechanisms and cross-domain distribution alignment, a Local Envelope Domain Adaptation (LEDA) module and a Global Envelope Domain Adaptation (GEDA) module are designed respectively. A weighted prediction loss function is then constructed to fuse the two modules and perform joint optimization. This model exhibits low and stable error and performs well in cross-domain pedestrian trajectory prediction.

[0092] In a given scenario, N pedestrians are in a time period from T1 to T2. obs observation trajectory The learnable parameter is W * The model f(·) (i.e., the MESC-HEDA model) can predict a relatively short future time period T. obs+1 To T pre trajectory in Let represent the two-dimensional spatial coordinates of the i-th pedestrian at time frame t, as follows:

[0093]

[0094] Figure 1This is a flowchart of a pedestrian trajectory prediction method based on hierarchical envelope domain adaptation. (Example:) Figure 1 As shown, the method includes:

[0095] Obtain the original trajectory samples of the source domain and the original trajectory samples of the target domain;

[0096] Obtain a pedestrian trajectory prediction model based on hierarchical envelope domain adaptation, namely the MESC-HEDA model;

[0097] The MESC-HEDA model is trained using original trajectory samples from the source domain and the original trajectory samples from the target domain, and pedestrian trajectories in the original trajectory samples from the target domain are predicted.

[0098] Figure 2 This is a block diagram of the MESC-HEDA model. (Example) Figure 2 As shown, the MESC-HEDA model includes: a two-level envelope sample construction module, a global hierarchical envelope sample construction module, and a hierarchical envelope domain adaptation module. The two-level envelope sample construction module includes a local hierarchical envelope sample construction sub-module (also called "local hierarchical envelope sample construction module"), an individual hierarchical envelope sample construction sub-module (also called "individual hierarchical envelope sample construction module"), and a two-level envelope sample feature fusion sub-module (also called "two-level envelope sample feature fusion module"). The hierarchical envelope domain adaptation module includes a local hierarchical envelope domain adaptation sub-module (also called "local hierarchical envelope domain adaptation module") and a global hierarchical envelope domain adaptation sub-module (also called "global hierarchical envelope domain adaptation module").

[0099] The local hierarchical envelope sample construction submodule is used to generate local hierarchical envelope samples based on the pedestrian adjacency relationship in the sample space hierarchy. These samples include local hierarchical envelope samples of the source domain and local hierarchical envelope samples of the target domain. Specifically, when the local hierarchical envelope sample construction submodule processes samples from the source domain, it generates local hierarchical envelope samples of the source domain; when the local hierarchical envelope sample construction submodule processes samples from the target domain, it generates local hierarchical envelope samples of the source domain.

[0100] The individual-level envelope sample construction submodule is used to generate individual-level envelope samples based on the pedestrian relationships at the individual level in the samples. These samples include individual-level envelope samples from the source domain and individual-level envelope samples from the target domain. Specifically, when the individual-level envelope sample construction submodule processes samples from the source domain, it generates individual-level envelope samples from the source domain; when it processes samples from the target domain, it generates individual-level envelope samples from the source domain.

[0101] The bi-level envelope sample feature fusion module is used to generate bi-level envelope samples based on local-level envelope samples and individual-level envelope samples. These samples include bipolar envelope samples of the source domain and bipolar envelope samples of the target domain. Specifically, when the bi-level envelope sample feature fusion module processes the local-level envelope samples and individual-level envelope samples of the source domain, it generates bipolar envelope samples of the source domain; when the bi-level envelope sample feature fusion module processes the local-level envelope samples and individual-level envelope samples of the target domain, it generates bipolar envelope samples of the target domain.

[0102] The global hierarchical envelope sample construction module is used to generate global hierarchical envelope samples based on the spatiotemporal feature distribution of bipolar envelope samples. These samples include global hierarchical envelope samples of the source domain and global hierarchical envelope samples of the target domain. Specifically, when the global hierarchical envelope sample construction module processes the bipolar envelope samples of the source domain, it generates global hierarchical envelope samples of the source domain; when the global hierarchical envelope sample construction module processes the bipolar envelope samples of the target domain, it generates global hierarchical envelope samples of the target domain.

[0103] The Local Hierarchical Envelope Domain Adaptation Submodule is used to determine the Local Hierarchical Domain Distribution Alignment Loss based on the local hierarchical envelope samples of the source and target domains using an attention mechanism.

[0104] The global hierarchical envelope domain adaptation submodule is used to determine the global hierarchical domain distribution alignment loss based on the global hierarchical envelope samples of the source and target domains.

[0105] Existing deep learning approaches primarily rely on modeling based on raw data samples, automatically extracting deep features reflecting the original data samples. However, the envelope sample learning mechanism proposed in this invention aims to mine the correlation information between raw samples, transforming it into new samples. This allows the original samples to be mapped into multi-layered envelope samples, representing the target object from multiple levels. Research shows that compared to existing methods directly based on raw samples, the hierarchical envelope sample modeling method helps improve accuracy. The envelope sample learning mechanism is briefly described as follows: Given a dataset X = [x1, x2, ..., x...]... n ] T For any sample x i For each x ∈ X, its nearest neighbor samples are found through the operator η1 and formed into a nearest neighbor sample set N(x). i ), Then x i With N(x) i ) merge into a new sample set X c =[x i N(x) i )] T Simultaneously construct the adjacency matrix A(X) c Expressing their adjacency relationships yields the sample set [X]. c , A(X cThe process of [X] is the sample envelope process; secondly, by transforming operator ψ1, the sample set [X] is... c , A(X c Transformation X e =ψ1([X c , A(X c If the generated sample set X is obtained by ]), then the new sample set X is called X. e For envelope samples. Transform from X to X e The process is called envelope sample transformation. Multi-level envelope sample sets. Together with the existing sample set X, they form a hierarchical envelope sample space. in Therefore, the model H(X) based on the original samples is transformed into a model based on multi-level envelope samples. This achieves the construction of multi-level envelope samples, where L represents the total number of envelope sample transformation operations. After L envelope sample transformation operations, L hierarchical envelope sample sets are obtained.

[0106] Figure 3 This is a schematic diagram of a local hierarchical envelope sample construction module. For example... Figure 3 As shown, based on the original pedestrian trajectory sequence samples in the scene (which can be either the source domain or the target domain), and utilizing the spatial relationships between each pedestrian, a local hierarchical envelope mapping function F is used. lhe The adjacency relationships of the original trajectory samples at the spatial level are mined to construct a series of spatial graph structures with adjacency matrices; then, local hierarchical envelope samples are obtained through appropriate interaction strength filtering and normalization operations [F]. loc A′ loc This enables envelope sample transformation.

[0107] Specifically, in conjunction with the description in "Module 1" below, the local hierarchical envelope sample construction module generates local hierarchical envelope samples based on the pedestrian adjacency relationships in the sample space hierarchy, including:

[0108] For any time t of all observed trajectories, i.e. t∈{T1, ..., T} obs}, T1, T2, ..., T obs Indicate the observation trajectory time, execute S21-S27:

[0109] S21: Obtain the original trajectory sequence samples of N pedestrians at time t (i.e., the trajectory data of each pedestrian in the same scene at the same time), that is, the coordinates of N pedestrians at time t. tr t i Let represent the coordinates of pedestrian i at time t, where i∈{1,2,...,N};

[0110] S22: Will Processing as a graph representation G loc =(V loc E loc ),in, G represents loc The vertex set, G represents loc edge set, Let R represent the edges formed by the coordinates of pedestrians i and j, where each edge is determined by the local hierarchical pedestrian adjacency relation R. loc and the local hierarchical adjacency matrix A loc express;

[0111] S23: Determine the feature vectors F of N pedestrians at time t. loc ,in, This represents the velocity vector of pedestrian i at time t; and Let x and y represent the components of the coordinates of pedestrian i at time t, respectively. and Let x and y represent the components of the coordinates of pedestrian i at time t-1, respectively.

[0112] S24: Through the local hierarchical envelope mapping function F lhe Determine the local hierarchical pedestrian adjacency relation R between any two pedestrians among all pedestrians. loc R loc Taking into account the local hierarchical interaction relationships of pedestrian groups (i.e., the relationships between all pedestrians at the same time in the same scene), two local hierarchical pedestrian group envelopes with radii r1 and r2 are defined. When the distance between other pedestrians and the target pedestrian is within the range of the two circles, these pedestrians can be regarded as local hierarchical envelope samples. This represents the Euclidean distance between pedestrians i and j after the filtering process at time t, i.e.

[0113]

[0114] and This represents the x and y components of the coordinates of pedestrian j at time t; This represents the Euclidean distance between pedestrians i and j after filtering at time t; r1 and r2 represent the two distances set. Describes the velocity vector of pedestrian i at time t. The angle between the line connecting the current position of pedestrian j and the line connecting the current position of pedestrian j, and the angle between the lines connecting the current position and the line connecting the current position of pedestrian j. Taking the cosine function, we get

[0115]

[0116] This represents the combined metric of the angle between the velocities of pedestrians i and j at time t.

[0117] definition, In summary, the local hierarchical envelope mapping function F lhe () includes various factors such as the speed of pedestrians in the group, the distance between them and their neighbors, and the angle between their relative directions of motion, representing the local level pedestrian adjacency relationship R. loc The structure is as follows:

[0118]

[0119] in, tr represents the velocity of pedestrian j at time t. t j Let represent the coordinates of pedestrian j at time t,

[0120] S25: By introducing a threshold hyperparameter η∈[0,1] to adjust R loc Perform local-level envelope sample interaction strength filtering to generate a local-level mask M. loc ,Right now

[0121]

[0122] in, It is an indicator function, when R loc Output 1 if ≥η, otherwise output 0;

[0123] S26: Element-wise multiplication operation on R loc The selection process is performed, and the identity matrix I is superimposed to generate a local hierarchical adjacency matrix A. loc ,Right now:

[0124] A loc =M loc ⊙R loc +I, (6)

[0125] Where ⊙ represents element-wise multiplication;

[0126] S27: Apply the Zero-Softmax method to the local hierarchical adjacency matrix A loc Normalization is performed to generate a normalized local hierarchical adjacency matrix A′. loc Thus, the local hierarchical envelope sample [F] corresponding to time t is obtained. loc A′ loc ];

[0127] For all observed trajectory times corresponding to F loc and A′ loc Stacking is performed to obtain local hierarchical envelope samples [F]loc A Loc ], where F Loc ={F1, F2, ..., F obs}, F t F represents the value corresponding to time t. loc A Loc ={A′1,A′2,...,A′ obs}, A′ t Denotes A′ corresponding to time t loc , t∈{T1,...,T obs}

[0128] In step S27, the local hierarchical adjacency matrix A is... loc Each element in Normalization can be expressed as:

[0129]

[0130] Where ∈ is a minimal constant to ensure numerical stability.

[0131]

[0132]

[0133] Figure 4 This is a schematic diagram of the individual-level envelope sample construction module. For example... Figure 4 As shown, based on the original pedestrian trajectory sequence samples in the scene (which can be either the source domain or the target domain), the adjacency relationships of the original trajectory samples at the temporal level are mined by utilizing the temporal correlation of the same pedestrian, thereby constructing a series of temporal graph structures with adjacency matrices; then, individual-level envelope samples are obtained through appropriate interaction strength filtering and normalization operations [F]. ind A′ ind This enables envelope sample transformation.

[0134] Specifically, in conjunction with the description in "Module 2" below, the individual-level envelope sample construction module generates individual-level envelope samples based on the pedestrian relationships at the individual level in the samples, including:

[0135] For any pedestrian i among all pedestrians, i ∈ {1, 2, ..., N}, where N represents the total number of pedestrians, execute S31-S37:

[0136] S31: Obtain all observation trajectory times T1, T2, ..., T obs Coordinates of insider i in, tr t iLet represent the coordinates of pedestrian i at time t, where t∈{T1, ..., T} obs};

[0137] S32: Will Processing as a graph representation G ind =(V ind E ind ),in, For G ind The vertex set, It is G ind The edge set, any edge Pedestrian adjacency relationship R at the individual level ind And the individual hierarchical adjacency matrix A ind express;

[0138] S33: Pedestrian i during time period T1 to T2 obs The corresponding eigenvector F ind , This represents the velocity vector of pedestrian i at time t;

[0139] S34: Using self-attention mechanisms, determine individual-level pedestrian adjacency relationships R. ind Frame t Let t represent the time frame number at time t. Assuming data is taken every 0.4s, then the time frame number corresponding to time t is equivalent to: 0.4s (T1) corresponds to time frame 1, 0.8s (T2) corresponds to time frame 2, 1.2s (T3) corresponds to time frame 3, and so on. That is, T1 is equivalent to 0.4s, T2 is 0.8s, T3 is 1.2s, and so on until T... obs (Assuming there are 8 sampling times) is 3.2s, while the Frame t It is the frame number, an integer, and can be 1, 2, ..., 8.

[0140] PE = {Frame t |t = T1, ..., T obs} represents the trajectory of a single pedestrian. t i During time period T1 to T obs The positional encoding tensor within; in the original individual-level feature vector F ind Based on this, a position encoding tensor PE is added, and the learnable parameter W is used. ind Perform a linear transformation to obtain a high-dimensional tensor F′ with positional encoding information. ind ,Right now:

[0141] F′ ind =(F ind +PE)W ind (8)

[0142] Through learnable parameters W Q and W K The query and key for the self-attention mechanism are obtained separately. Finally, the product of query and key is scaled using a scaling factor D to maintain stability, and normalized using the Softmax function to obtain the individual-level envelope sample pedestrian adjacency relation R. ind

[0143]

[0144] Among them, W Q and W K It is a learnable parameter; Q ind and K ind These represent the query and key in the self-attention mechanism, respectively. Indicates K ind Transpose;

[0145] S35: By setting the threshold hyperparameter ψ∈[0,1] for R ind Perform interactive filtering of individual-level envelope samples to generate an individual-level mask M. ind ,Right now:

[0146]

[0147] in, It is an indicator function, when R ind Output 1 if the value is greater than or equal to ψ, otherwise output 0.

[0148] S36: Mask the individual hierarchy M ind Adjacency relationship with individual pedestrians R ind The fusion process yields the individual-level adjacency matrix A. ind ,Right now:

[0149] A ind =M ind ⊙R ind (11)

[0150] Where ⊙ represents element-wise multiplication;

[0151] S37: Using the Zero-Softmax method to process the individual hierarchical adjacency matrix A ind Normalization is performed to generate a normalized individual hierarchical adjacency matrix A′. ind Thus, the individual-level envelope sample [F] corresponding to pedestrian i is obtained. ind A′ ind ];

[0152] For all pedestrians, F loc and A′ locStacking is performed to obtain individual hierarchical envelope samples [F] Ind A Ind ], where F Ind ={F1, F2, ..., F N}, F i F represents the value corresponding to pedestrian i ind A Ind ={A′1,A′2,...,A′ N}, A′ i A′ represents the pedestrian i. ind , i∈{1,2,...,N}.

[0153] In S37, the individual hierarchical adjacency matrix A ind After normalization, it can be expressed as:

[0154] A′ ind =Zero-Softmax(A ind (11)

[0155]

[0156] The following describes the bipolar envelope sample feature fusion module.

[0157] Based on the description in "Module 3" below, the bipolar envelope sample feature fusion module generates bipolar envelope samples based on local-level envelope samples and individual-level envelope samples, including:

[0158] Graph convolution is performed on the graph structure of the local hierarchical envelope samples to obtain the local hierarchical feature representation F′ of the local hierarchical envelope samples. Loc ,Right now:

[0159] F′ Loc =σ(A Loc F Loc W Loc (12)

[0160] Where σ represents the activation function ReLU(); W Loc Indicates learnable parameters;

[0161] Graph convolution is performed on the graph structure of the individual-level envelope samples to obtain the individual-level feature representation F′ of the individual-level envelope samples. Ind ,Right now:

[0162] F′ Ind =σ(A Ind F Ind W Ind (13)

[0163] Where σ represents the activation function ReLU(); W Ind Indicates learnable parameters;

[0164] Representing local hierarchical features as F′ Loc Individual hierarchical feature representation F′ Ind By splicing and fusion, the feature representation F of the bipolar envelope sample is obtained. tef ,Right now:

[0165]

[0166] in, This indicates that the data is spliced ​​and fused according to the feature dimensions; L represents the set of real numbers; obs represents the total number of all observation times; D represents the feature dimension of the local hierarchical envelope sample and the individual hierarchical envelope sample.

[0167]

[0168] The following describes the global hierarchical envelope sample construction module.

[0169] The global hierarchical envelope sample construction module generates global hierarchical envelope samples based on the spatiotemporal feature distribution of the two-level envelope samples, including:

[0170] The feature representation F of the bipolar envelope sample tef The input is a multi-layer Temporal Convolutional Network (TCN) with l layers. The output of the previous layer is used as the input of the next layer to obtain the predicted trajectory. For the l-th layer of the TCN:

[0171]

[0172] in, W represents the input of the l-th layer TCN. (l) For the learnable parameters of the l-th layer TCN, This represents the output of the l-th layer, i.e., the predicted output of a multi-layer TCN; the number of layers in a TCN can be arbitrary, such as 3, 4, 5, etc.

[0173] The output of the l-th layer (i.e., the last layer) Five parameters F of the bivariate Gaussian distribution of the predicted trajectory coordinates are determined by a linear layer. final ,Right now:

[0174]

[0175] Where Linear() represents linear layer processing, t∈[T] obs+1 T obs+2, ..., T pre ], T obs+1 T obs+2 , ..., T pre This represents the time frame for predicting the trajectory. Through this linear layer, trajectory prediction can be performed for all prediction times. The parameter representing the mean trajectory value of pedestrian i at time t in the predicted trajectory time frame includes the mean trajectory values ​​in both the x and y directions. and Let represent the standard deviation component of pedestrian i's trajectory at time t, which includes the standard deviations of the trajectory in both the x and y directions. and Time represents the correlation coefficient of pedestrian i's trajectory at time t;

[0176] Then, the global hierarchical envelope samples are determined according to the following formula:

[0177]

[0178] Specifically, based on the trajectory mean parameter of each pedestrian Calculate the linear velocity of the pedestrian at each time frame t. Right now

[0179] Predict the time period T obs+1 To T pre The linear velocity forms the vector U i ,Right now Among them, L pre Indicates the predicted time period T obs+1 To T pre The total number of predicted trajectory moments;

[0180] U of all pedestrians i By splicing the samples, we obtain the global hierarchical envelope samples. Right now

[0181] To address the inconsistency in trajectory feature distribution between the source and target domains, this paper constructs a multi-level hierarchical envelope domain adaptation module. A significant difference from existing domain adaptation methods is that a single "sample" in pedestrian trajectory prediction is composed of combinations of different numbers of pedestrian trajectories, exhibiting varying scales and dimensions. Extracting uniform features from samples at different scales leads to an inability to accurately represent sample features, resulting in information loss. On one hand, this paper employs an attention-based sample feature extraction method, which can more effectively adapt to different sample scales, avoiding inaccuracies in local and global domain adaptation caused by feature scale representation errors. On the other hand, the combination of local and global envelope samples encompasses all pedestrian motion information, effectively resolving the domain offset problem.

[0182] This invention designs a hierarchical envelope domain adaptation module, which includes a local hierarchical envelope domain adaptation submodule and a global hierarchical envelope domain adaptation submodule. The functionality of the hierarchical envelope domain adaptation module is described below in conjunction with the description in "Module 4".

[0183] The local hierarchical envelope domain adaptation submodule can determine the local hierarchical domain distribution alignment loss based on the local hierarchical envelope samples of the source and target domains using an attention mechanism, including:

[0184] Obtain local hierarchical feature representations of local hierarchical envelope samples from the source and target domains. and Where, N s and N t These represent the number of pedestrians in the local hierarchical envelope samples of the source and target domains, respectively.

[0185] The local hierarchical features of the source and target domains are represented by F′. Loc(s) and F′ Loc(t) Breaking it down by pedestrian, that is:

[0186]

[0187] in,

[0188] For each local hierarchical envelope sample in the source domain

[0189] Will Flattened into D c The size of the feature vector, where D c =L obs ×D,

[0190] Determine the interaction weights of different pedestrians in the local hierarchical envelope samples of the source domain. Right now:

[0191]

[0192] in, express The transpose of the sample, through different pedestrian features in the same local layer envelope samples. The product of these factors represents the interaction weights, and the cumulative sum of the effects of all pairwise pedestrian interactions is calculated. The attention score is the proportion of a specific pair of pedestrians within this sum; obviously, the larger the proportion, the stronger the interaction, and vice versa. The number of features D... c It plays a normalization role.

[0193] The summation of interpersonal relationship interactions in the local hierarchical envelope samples of the source domain is determined by J. (s) ,Right now:

[0194]

[0195] J corresponding to all local hierarchical envelope samples in the source domain (s) Stacking the features yields a local hierarchical representation M of the pedestrian attention features in the source domain. (s) ,Right now:

[0196]

[0197] Among them, K (s) This indicates the number of local hierarchical envelope samples in the source domain;

[0198] For each local layer envelope sample in the target domain

[0199] Will Flattened into D c The size of the feature vector, where D c =L obs ×D,

[0200] Determine the interaction weights of different pedestrians in the local hierarchical envelope samples of the target domain. Right now:

[0201]

[0202] in, express The transpose of the sample, through different pedestrian features in the same local layer envelope samples. The product of these factors represents the interaction weights, and the cumulative sum of the effects of all pairwise pedestrian interactions is calculated. The attention score is the proportion of a specific pair of pedestrians within this sum; obviously, the larger the proportion, the stronger the interaction, and vice versa. The number of features D... c It plays a normalization role.

[0203] The summation of interpersonal relationship interactions in the local hierarchical envelope samples of the target domain is expressed as J. (t) ,Right now:

[0204]

[0205] J corresponding to all local hierarchical envelope samples in the target domain (t) Stacking is performed to obtain the local hierarchical pedestrian attention feature set representation M of the target domain. (t) ,Right now:

[0206]

[0207] Among them, K (t) This indicates the number of local hierarchical envelope samples in the target domain;

[0208] M is represented by local hierarchical pedestrian attention feature sets in the source and target domains. (s) and M (t) Determine the local hierarchical domain distribution alignment loss L loc ,Right now:

[0209] L loc =Dist1(M (s) M (t) ), (twenty two)

[0210] Here, Dist1() represents the first distance function.

[0211] The global hierarchical domain adaptation submodule determines the global hierarchical domain distribution alignment loss based on the global hierarchical envelope samples of the source and target domains, including:

[0212] Obtain the global hierarchical envelope samples F of the source and target domains. glo(s) and F glo(t) ;

[0213] Determine the linear velocity difference ratio between the source domain trajectory and the target domain trajectory. Let i represent the linear velocity of pedestrian i in the source domain at time t. N represents the linear velocity of pedestrian j in the target domain at time t. s Nt represents the total number of pedestrians in the source domain of the training set, and Nt represents the total number of pedestrians in the target domain of the validation set.

[0214] Determine the global hierarchical domain distribution alignment loss L glo ,Right now:

[0215] L glo =Dist2(F glo(s) ,κ×F glo(t) ), (twenty three)

[0216] Here, Dist2() represents the second distance function.

[0217] There are various choices for distance functions Dist1() and Dist2(), such as MMD loss, CORAL loss, GFK loss, and L2 distance. This invention further enhances the network's ability to transfer and learn multi-level cross-domain spatiotemporal features by jointly calculating the distribution differences of local and global hierarchical features in the target and source domains.

[0218]

[0219]

[0220] Assuming the pedestrian coordinates are... It follows a binary Gaussian distribution. This represents the mean, which includes the mean values ​​of the trajectories in both the x and y directions. and This represents the standard deviation, which includes the standard deviations of the trajectory in both the x and y directions. and Let L represent the correlation coefficient, and N represent the total number of pedestrians in the source domain. Therefore, the prediction loss L for predicting the trajectory... pre This can be transformed into minimizing the negative log-likelihood loss:

[0221]

[0222] in, This represents the probability density function.

[0223] The objective function L for training the MESC-HEDA model using original trajectory samples from the source domain and the target domain is:

[0224] L = L pre +λ1L loc +λ2L glo (25)

[0225] Where λ1 and λ2 represent weights used to balance these three losses, and the predicted loss L pre It is only relevant to samples in the source domain and not to samples in the target domain. The model minimizes the trajectory prediction loss L by jointly minimizing the samples. pre Local hierarchical domain distribution alignment loss L in the source and target domains loc Alignment loss L with global hierarchical domain distribution glo Conduct training.

[0226] The advantages of this invention will be illustrated by specific simulation experiments.

[0227] Datasets: The experiment selected two representative public real-world datasets: ETH and UCY. These two datasets have been widely used in numerous studies on pedestrian trajectory prediction methods. The datasets contain social interactions between pedestrians and their surroundings, as well as interactions between pedestrians and their environment, such as crowd gathering and dwelling, collision avoidance, and group movement. The ETH dataset consists of two scenes: ETH and HOTEL, while the UCY dataset consists of three scenes: UNIV, ZARA1, and ZARA2. The UCY dataset scenes tend to be densely populated, often containing more complex pedestrian social interactions, while the ETH dataset scenes tend to be sparsely populated, implying less social interaction between pedestrians.

[0228] Experimental Setup: Following the accepted paradigm of existing domain-adaptation-based trajectory prediction research, each scene is considered a trajectory domain. The model is first trained on one domain, then validated on validation sets in the other four domains, and finally tested on the remaining four domains. The effectiveness of the model is verified by comparing its prediction accuracy. In this invention, A, B, C, D, and E represent the pedestrian trajectory domains of five scenes: ETH, HOTEL, UNIV, ZARA1, and ZARA2, respectively; thus, there are a total of 20 trajectory prediction tasks: A→B / C / D / E, B→A / C / D / E, C→A / B / D / E, D→A / B / C / E, E→A / B / C / D. During the training phase, the model of this invention can only access the trajectories observed in the validation set; that is, the validation set is only used for domain adaptation.

[0229] Evaluation Metrics: The model performance is evaluated using the Average Displacement Error (ADE) and the Final Displacement Error (FDE). ADE is used to evaluate the average prediction accuracy of the predicted trajectory, while FDE is used to evaluate the prediction accuracy of the trajectory endpoints in the last frame. Currently, since Social-LSTM, Social-STGCNN, and Social-GAN all use the evaluation criterion that the model output follows a bivariate Gaussian distribution, this invention also follows this unified criterion for performance comparison.

[0230] Parameter settings: The experimental software platform was PyTorch 2.3, and the hardware platform consisted of an i9-13900H CPU, an RTX A3000 GPU, and 64GB of memory. The observed pedestrian historical trajectory lasted 3.2 seconds (8 time frames), and the predicted trajectory lasted 4.8 seconds (12 time frames). The GCN layer was 1, the TCN layer was 5, and the feature dimension was 16. During training, the batch size was set to 16, the local hierarchical domain distribution alignment loss weight was set to 9, the global hierarchical domain distribution alignment loss weight was set to 5, the local hierarchical envelope sample thresholding hyperparameter was set to 0.05, and the individual hierarchical envelope sample thresholding hyperparameter was set to 0.7. The entire model was trained for 200 epochs, with an initial learning rate of 0.01, which was changed to 0.005 after 100 epochs. All parameter settings unrelated to the algorithm were based on generally accepted experimental paradigms from relevant research.

[0231] Under the same experimental setup as described above, five related models, STGAT, Transformer, Social-STGCNN

[23] , SGCN, and T-GNN, were compared with the MESC-HEDA model proposed in this invention. Tables 1 and 2 show the ADE and FDE metrics of the model in 20 scenario task combinations, respectively.

[0232] Table 1. Comparison of ADE between the MESC-HEDA model and the reference model

[0233]

[0234]

[0235] Table 2. Comparison of FDE between the MESC-HEDA model and the reference model

[0236]

[0237] As shown in Tables 1 and 2, the average ADE and FDE scores of the MESC-HEDA model proposed in this invention consistently outperform the other five reference models. Overall, the MESC-HEDA model improves the ADE score by 25.0% and 22.7% compared to the SGCN and T-GNN models, respectively, and the MESC-HEDA model improves the FDE score by 23.3% and 19.8% compared to the SGCN and T-GNN models, respectively. The MESC-HEDA model achieves the best ADE score in 13 out of 20 tasks and the best FDE score in 12 out of 20 tasks. This may be because the hierarchical envelope domain adaptation model acquires richer spatiotemporal feature representations of spatiotemporal sequence data at different levels by constructing multi-level envelope samples. Simultaneously, it achieves domain distribution alignment by learning multi-level spatiotemporal feature information transferable from the source domain to the target domain, further enhancing the model's multi-level cross-domain spatiotemporal feature transfer learning capability.

[0238] Therefore, most existing pedestrian trajectory prediction models do not consider cross-domain adaptation and fail to address the problem of pedestrian trajectory scene domain offset. A very small number of domain-adaptation-based pedestrian trajectory prediction models only consider local social level samples and their domain adaptation, neglecting the construction of individual and global level samples and the building of domain adaptation models. This invention constructs multi-level hierarchical samples and integrates them with domain adaptation to build a hierarchical domain-adaptive trajectory prediction model, effectively overcoming the shortcomings of the aforementioned methods in addressing the problem of large differences in trajectory data distribution between different pedestrian trajectory scenes. Specifically, this invention proposes a pedestrian trajectory prediction method based on hierarchical envelope domain adaptation. This method constructs multi-level envelope samples and builds a hierarchical envelope domain-adaptive trajectory prediction model based on them, thereby fully constructing the spatiotemporal feature representation of pedestrian trajectory data at different levels and effectively addressing the problem of large distribution differences between trajectory domains in different scenes. Finally, multiple sets of experiments were conducted, and the experimental results show that the multi-level envelope samples and hierarchical envelope domain adaptation proposed in this invention have superior performance compared to other representative algorithms and can effectively address the multi-level complexity of interactions between pedestrians and between pedestrians and the environment. Furthermore, the multi-level envelope sample learning module and hierarchical domain adaptation module proposed in this invention can be combined with other methods, which has good reference value for related research.

[0239] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A pedestrian trajectory prediction method based on hierarchical envelope domain adaptation, characterized in that, include: Obtain the original trajectory samples of the source domain and the original trajectory samples of the target domain; Construct a pedestrian trajectory prediction model based on hierarchical envelope domain adaptation, namely the MESC-HEDA model; The MESC-HEDA model is trained using original trajectory samples from the source domain and the target domain, and pedestrian trajectories in the original trajectory samples from the target domain are predicted, where: The MESC-HEDA model includes: a two-level envelope sample construction module, a global hierarchical envelope sample construction module, and a hierarchical envelope domain adaptation module. The two-level envelope sample construction module includes a local hierarchical envelope sample construction submodule, an individual hierarchical envelope sample construction submodule, and a two-level envelope sample feature fusion submodule. The hierarchical envelope domain adaptation module includes a local hierarchical envelope domain adaptation submodule and a global hierarchical envelope domain adaptation submodule. The local hierarchical envelope sample construction submodule is used to generate local hierarchical envelope samples based on the pedestrian adjacency relationship in the sample space hierarchy, including local hierarchical envelope samples of the source domain and local hierarchical envelope samples of the target domain. The individual-level envelope sample construction submodule is used to generate individual-level envelope samples based on the pedestrian relationships at the individual level in the samples, including individual-level envelope samples of the source domain and individual-level envelope samples of the target domain. The bi-level envelope sample feature fusion submodule is used to generate bi-level envelope samples based on local hierarchical envelope samples and individual hierarchical envelope samples, including bipolar envelope samples of the source domain and bipolar envelope samples of the target domain. The global hierarchical envelope sample construction module is used to generate global hierarchical envelope samples based on the spatiotemporal feature distribution of the bilevel envelope samples, including global hierarchical envelope samples of the source domain and global hierarchical envelope samples of the target domain. The local hierarchical envelope domain adaptation submodule is used to determine the local hierarchical domain distribution alignment loss based on the local hierarchical envelope samples of the source and target domains using an attention mechanism. The global hierarchical envelope domain adaptation submodule is used to determine the global hierarchical domain distribution alignment loss based on the global hierarchical envelope samples of the source and target domains. Based on the pedestrian adjacency relationships in the sample space, local hierarchical envelope samples are generated, including: For any time of all observed trajectories ,Right now , Indicate the observation trajectory time, execute S21-S27: S21: Acquisition time The original trajectory sequence samples of each pedestrian, i.e. time Coordinates of a pedestrian , , express Pedestrians at all times coordinates ; S22: Will Processed as a graphical representation ,in, express The vertex set, express edge set, pedestrian and pedestrians The edges formed by the coordinates are determined by the local hierarchical pedestrian adjacency relationships. and local hierarchical adjacency matrix express; S23: Confirm time Feature vector of a pedestrian ,in, , express Pedestrians at all times The velocity vector; and They represent Pedestrians at all times In the coordinates and Components in two directions; and They represent Pedestrians at all times In the coordinates and Components in two directions; S24: Through local hierarchical envelope mapping function Determine the local hierarchical pedestrian adjacency relationship between any two pedestrians among all pedestrians. , ,in, express Pedestrians at all times speed, They represent Pedestrians at all times coordinates ， and express Pedestrians at all times In the coordinates and Components in two directions; express Pedestrians who are screened at all times and pedestrians Euclidean distance; and This indicates the two distances set; express Pedestrians at all times velocity vector with pedestrians The angle between the lines connecting the current positions, and the angle between the lines. Taking the cosine function, we get ; ,express Pedestrians at all times and pedestrians A comprehensive measure of the velocity angle; ； S25: By introducing a threshold hyperparameter right Perform local-level envelope sample interaction strength filtering to generate local-level masks. ,in, , It is an indicator function, when Output 1 if the condition is met, otherwise output 0. S26: Element-wise multiplication pairs Perform filtering and simultaneously overlay an identity matrix. Generate a local hierarchical adjacency matrix ,Right now , This indicates element-wise multiplication; S27: Applying the Zero-Softmax method to the local hierarchical adjacency matrix Perform normalization to generate a normalized local hierarchical adjacency matrix. Thus obtaining Local hierarchical envelope samples corresponding to time 1 ; For all observed trajectory times corresponding to and Stacking is performed to obtain local hierarchical envelope samples. ,in, , Indicates and Time corresponding , , Indicates and Time corresponding , ; Based on the pedestrian relationships at the individual level in the sample, generate individual-level envelope samples, including: For any pedestrian among all pedestrians ,Right now , This indicates the total number of pedestrians. Execute steps S31-S37: S31: Obtain all observation trajectory times insider coordinates ,in, , pedestrian exist Coordinates of time, ; S32: Will Processed as a graphical representation ,in, for The vertex set, yes The edge set, any edge Pedestrian adjacency relationships at the individual level and individual hierarchical adjacency matrix express; S33: Pedestrian In time period arrive The corresponding eigenvectors , , pedestrian exist The velocity vector at any given moment; S34: Utilize self-attention mechanisms to determine pedestrian adjacency relationships at the individual level. ,in, ， and These are learnable parameters; and These represent the query and key in the self-attention mechanism, respectively. Indicates to Transpose; Indicates the scaling factor; This represents a high-dimensional tensor with positional encoding information; Represents the trajectory of a single pedestrian In time period arrive The positional encoding tensor within; express The number of time frames at a given moment; Indicates learnable parameters; S35: By setting the threshold hyperparameter right Perform interactive filtering of individual-level envelope samples to generate individual-level masks. ,in, , It is an indicator function, when Output 1 if the condition is met, otherwise output 0. S36: Mask the individual hierarchy Adjacency relationship with individual pedestrians By merging, an individual-level adjacency matrix is ​​obtained. ,Right now , This indicates element-wise multiplication; S37: Using the Zero-Softmax method to analyze the individual hierarchical adjacency matrix Normalization is performed to generate a normalized individual hierarchical adjacency matrix. Thus, to obtain information about pedestrians Corresponding individual hierarchical envelope samples ; For all pedestrians and Stacking is performed to obtain individual hierarchical envelope samples. ,in, , Indicates to pedestrians corresponding , , Indicates to pedestrians corresponding , .

2. The pedestrian trajectory prediction method based on hierarchical envelope domain adaptation according to claim 1, characterized in that, Based on the local-level envelope samples and the individual-level envelope samples, a two-level envelope sample is generated, including: Graph convolution is performed on the graph structure of the local hierarchical envelope samples to obtain the local hierarchical feature representation of the local hierarchical envelope samples. ,in, , This represents the activation function ReLU(); Indicates learnable parameters; Graph convolution is performed on the graph structure of the individual-level envelope samples to obtain the individual-level feature representation of the individual-level envelope samples. ,in, , This represents the activation function ReLU(); Indicates learnable parameters; Representing local hierarchical features Individual-level feature representation By splicing and fusion, the feature representation of the bipolar envelope sample is obtained. ,in, , This indicates that the data is spliced ​​and fused according to the feature dimensions; Represents the set of real numbers; This represents the total number of all observed times; The feature dimensions represent the local hierarchical envelope samples and the individual hierarchical envelope samples.

3. The pedestrian trajectory prediction method based on hierarchical envelope domain adaptation according to claim 2, characterized in that, Based on the spatiotemporal feature distribution of the two-level envelope samples, a global hierarchical envelope sample is generated, including: Feature representation of bipolar envelope samples The input has A multi-layer temporal convolutional network (TCN) uses the output of the previous layer as the input of the next layer to obtain the predicted trajectory. Specifically, for the ... Layer TCN: ， in, Indicates the first The input of the layer TCN, For the first Learnable parameters of layer TCN Indicates the first The output of the layer, i.e. the prediction output of the multi-layer TCN; The first Layer output Five parameters of the bivariate Gaussian distribution of the predicted trajectory coordinates are determined through a linear layer. ,Right now ,in, Indicates linear layer processing. , Represents the time frame of the predicted trajectory. Representing the time frame of the predicted trajectory Pedestrians at all times The trajectory mean parameter includes and The average of the trajectories in both directions, i.e. and , express Pedestrians at all times The trajectory standard deviation component, which includes and The standard deviation of the trajectory in both directions, i.e. and , Time indicates Carved pedestrians Trajectory correlation coefficient; Based on the average trajectory parameter of each pedestrian Calculate pedestrians linear velocity at time t ,Right now ; Predicted time period arrive Linear velocity forming vector ,Right now ,in, Indicates the predicted time period arrive The total number of predicted trajectory moments; All pedestrians By splicing the samples, we obtain the global hierarchical envelope samples. ,Right now .

4. The pedestrian trajectory prediction method based on hierarchical envelope domain adaptation according to claim 2, characterized in that, Based on the local hierarchical envelope samples of the source and target domains, an attention mechanism is used to determine the local hierarchical domain distribution alignment loss, including: Obtain local hierarchical feature representations of local hierarchical envelope samples from the source and target domains. and ,in, and These represent the number of pedestrians in the local hierarchical envelope samples of the source and target domains, respectively. Representing the local hierarchical features of the source and target domains and Break it down according to pedestrians, that is , ,in, , , For each local hierarchical envelope sample in the source domain Will Flattened The size of the feature vector, where, , Determine the interaction weights of different pedestrians in the local hierarchical envelope samples of the source domain. ,Right now ,in, express transpose, Determine the summation of the interpersonal relationship interactions in the local hierarchical envelope samples of the source domain. ,Right now , All local hierarchical envelope samples in the source domain correspond to Stacking the features yields a local hierarchical representation of pedestrian attention features in the source domain. ,Right now ,in, This represents the number of local hierarchical envelope samples in the source domain; For each local layer envelope sample in the target domain Will Flattened The size of the feature vector, where, , Determine the interaction weights of different pedestrians in the local hierarchical envelope samples of the target domain. ,Right now ,in, express transpose, Determine the summation of the interpersonal relationship interactions in the local hierarchical envelope samples of the target domain. ,Right now , All local hierarchical envelope samples in the target domain correspond to Stacking is performed to obtain a local hierarchical representation of pedestrian attention features in the target domain. ,Right now ,in, This indicates the number of local hierarchical envelope samples in the target domain; The pedestrian attention features are represented by local hierarchical features in the source and target domains. and Determine the local hierarchical domain distribution alignment loss ,Right now , This represents the first distance function.

5. The pedestrian trajectory prediction method based on hierarchical envelope domain adaptation according to claim 4, characterized in that, Based on the global hierarchical envelope samples of the source and target domains, determine the global hierarchical domain distribution alignment loss, including: Obtain global hierarchical envelope samples of the source and target domains. and ; Determine the linear velocity difference ratio between the source domain trajectory and the target domain trajectory. , Let i represent the linear velocity of pedestrian i in the source domain at time t. This represents the linear velocity of pedestrian j in the target domain at time t. This represents the total number of pedestrians in the source domain of the training set. This represents the total number of pedestrians in the target domain of the validation set; Determine the global hierarchical domain distribution alignment loss ,Right now ,in, This represents the second distance function.

6. The pedestrian trajectory prediction method based on hierarchical envelope domain adaptation according to claim 5, characterized in that, The objective function for training the MESC-HEDA model using original trajectory samples from the source domain and the target domain. for: ， in, Indicates the predicted loss. and Indicates weight, This represents the total number of pedestrians in the source domain. ， Represents the probability density function. Representing the time frame of the predicted trajectory Pedestrians at all times The trajectory mean parameter includes and The average of the trajectories in both directions, i.e. and , express Pedestrians at all times The trajectory standard deviation component, which includes and The standard deviation of the trajectory in both directions, i.e. and , Time indicates Carved pedestrians Trajectory correlation coefficient.

Images