A pedestrian trajectory prediction method based on double-branch spatiotemporal graph convolution network fusion

Abstract

The application provides a kind of pedestrian trajectory prediction method based on double-branch space-time graph convolution network fusion, belongs to the field of mobile robot perception and decision-making, solve the technical problems that pedestrian social interaction is insufficient in traditional method, and pedestrian movement mode is difficult to capture, need to simulate complex time dependence, its technical scheme is: including the following steps: step one, using pre-training CNN extracts pedestrian scene features;Step two, in double branch, respectively construct pedestrian two two interaction graph and group interaction graph, and embed scene features in the graph;Step three, using spatial GCN extracts pedestrian spatial features;Step four, using time extrapolation convolution network processes the space-time features output by two branches to predict the Gaussian distribution parameters of trajectory.The beneficial effects of the application are that the application can better model the complex social interaction of pedestrians and capture the unique movement patterns of each pedestrian, improving the accuracy and generalization ability of the prediction.

Description

Technical Field

[0001] This invention relates to the fields of mobile robots and deep learning technology, and in particular to a method for predicting pedestrian trajectories based on a bi-branch spatiotemporal graph convolutional network. Background Technology

[0002] Pedestrian trajectory prediction attempts to predict socially acceptable future paths by analyzing past movement patterns of pedestrians. It is useful in many practical applications, such as surveillance systems. Successful prediction of future pedestrian trajectories is closely related to accurate modeling of surrounding pedestrians and physical scene influencing factors.

[0003] Despite some progress in current literature research, pedestrian trajectory prediction remains a significant challenge due to the complex social interactions among pedestrians and the uncertainty of pedestrian movement patterns.

[0004] Pioneering work in pedestrian trajectory prediction simulated interactions between pedestrians using handcrafted energy functions. However, these methods struggled to handle the complexities of crowded situations. With the development of deep learning, early research aggregated the hidden states of recurrent neural networks to model pedestrian interactions based on location information. In recent years, many graph-based models have attracted considerable attention due to their efficient representation of graph structures. In general, these methods aim to better simulate social interactions between agents and the historical motion dynamics of each agent. Previous studies have attempted to improve social interactions; however, an overemphasis on individual interactions has neglected group interactions. Furthermore, because pedestrian speeds fluctuate, it is unreasonable to consider different time steps equally when modeling pedestrian motion patterns. Therefore, performance degrades when predicting complex pedestrian trajectories.

[0005] How to solve the above-mentioned technical problems is the challenge facing this invention. Summary of the Invention

[0006] The purpose of this invention is to provide a pedestrian trajectory prediction method based on a bi-branch spatiotemporal graph convolutional network. By using a bi-branch network to simulate pairwise and intra-group interactions of pedestrians and capturing complex social interactions among pedestrians, the method incorporates scene features to improve the accuracy of the model. Furthermore, by utilizing attention-based spatiotemporal graph convolution, the method enhances the processing capability for special motion patterns, thereby achieving more accurate pedestrian trajectory prediction.

[0007] To achieve the aforementioned objectives, the present invention employs the following technical solution: a pedestrian trajectory prediction method based on a bi-branch spatiotemporal graph convolutional network, comprising the following steps:

[0008] Step 1: Extract scene visual features using a pre-trained CNN;

[0009] Step 2: Construct pairwise interaction graphs and inter-group interaction graphs for pedestrians in the two branches, and embed scene features into the graphs.

[0010] Based on the trajectory coordinates input by the pedestrian The proposed method, which processes the data into a graphical representation, consists of two branches. The pairwise interaction branch calculates the influence matrix of graph nodes based on the relative position, velocity, and direction of the pedestrian agents, and constructs a pairwise interaction graph to simulate interactions between pedestrians. The grouping branch establishes pedestrian pairs using the L2 similarity of the trajectories between pedestrians, and groups pedestrians based on similar trajectories given a learnable threshold θ. A grouping mask M is then established to process the groups. For pedestrians in the same group, a distance-based kernel function is used to calculate the influence matrix of graph nodes, and an intra-group interaction graph is constructed to simulate interactions between pedestrians. Scene features are embedded in the dual-branch graph to fully utilize scene information.

[0011] Step 3: Extract spatiotemporal features using spatiotemporal graph convolution.

[0012] On the bi-branch graph, spatial GCN is used to perform convolution operations at each time step t. After processing social interaction and scene interaction, TCN is used to model pedestrian temporal dependence. TCN receives the output of GCN, uses self-attention to calculate the influence strength of nodes at different time steps, focuses on specific time steps of pedestrians, and extracts temporal features through convolution. Spatiotemporal graph convolution captures the spatiotemporal features of pedestrians.

[0013] Step 4: The temporal extrapolation convolutional network processes the spatiotemporal features of the two branch outputs to predict the two-dimensional Gaussian distribution parameters of the trajectory.

[0014] The spatiotemporal features extracted by the dual branches are embedded into the temporal extrapolation convolutional network Txp-cnn. Txp-cnn directly operates on the time channel to predict the Gaussian distribution parameters for the next 12 time steps. The ensemble network MLP adaptively fuses the prediction parameters of the two branches to obtain the final trajectory prediction parameters. Based on the generated prediction model, multiple outputs are generated to handle the uncertainty of the future trajectory. In order to avoid a large difference between the generated samples and the ground truth, the trajectory that is closest to the true distribution is selected as the final predicted trajectory by sampling the Gaussian distribution 20 times.

[0015] As a further optimization of the pedestrian trajectory prediction method based on a dual-branch spatiotemporal graph convolutional network provided by the present invention, step one, which uses a pre-trained CNN to extract scene visual features, includes the following:

[0016] To fully utilize scene information, this invention employs a pre-trained CNN network to extract scene image features. We use a pre-trained VGG19 network as the backbone, and the extracted features are represented by VGG19. ph express:

[0017] V ph=CNN(I image W cnn )

[0018] Where CNN(.) represents a pre-trained VGG19 network, W cnn For the weights of the pre-trained network, I image This represents the scene images in the dataset.

[0019] As a further optimization of the pedestrian trajectory prediction method based on a bi-branch spatiotemporal graph convolutional network provided by this invention, in step two, pairwise interaction graphs and inter-group interaction graphs of pedestrians are constructed in the two branches respectively, and scene features are embedded in the graphs. Specifically, this invention constructs a series of graphs to represent pedestrian trajectories, and at time step t, all pedestrians are connected to form a complete graph G. t =(V t E t V t Representing node characteristics, E t Represents graph G t The edges, in the two branches, are based on G respectively. t Constructing a pair graph G t,p With group graph G t,g .

[0020] In the pairwise interaction branch, the following function is used to calculate the pairwise interactions between pedestrians:

[0021]

[0022] Let α represent the velocity of pedestrian i at time t, α and β represent the angles between the velocity vector and the medium, and l represent the distance between pedestrians.

[0023]

[0024] After weight normalization, the pedestrian adjacency matrix A is obtained. p The adjacency matrix A is obtained using this method. p Representation of spatial graph G t The node weights are used to obtain the pairwise interaction graph G of pedestrians. t,p .

[0025] In the group interaction branch, pedestrian historical trajectories are used as input, and L2 distance is used to calculate the similarity between different pedestrian trajectories:

[0026]

[0027] in Let F(.) represent the feature similarity of pedestrians ij, F(.) represent a learnable convolutional layer, and define a learnable threshold θ to match trajectories with feature similarity less than the threshold as similarity pairs.

[0028]

[0029] S represents the set of similar pedestrian pairs, and pair(i,j) represents a pair of pedestrians with similar trajectory features. All related pedestrians are assigned to the same group using the set of similar pedestrian pairs, and a group index is assigned. A group mask M is defined based on the pedestrian grouping results.

[0030]

[0031] If node i and node j are in the same group, then M ij Using 1 to indicate that the mask removes redundant edges that are irrelevant to pedestrians, for pedestrians interacting within a group, a distance-based kernel function is used to construct the pedestrian group interaction weights:

[0032]

[0033] Weight normalization yields the pedestrian adjacency matrix A g The adjacency matrix A is obtained using this method. g Representation of spatial graph G t The node weights are used to obtain the pedestrian group interaction graph G. t,g A g elements Represented as:

[0034]

[0035] Insert the scene visual features obtained in step one into the node features of the bibranch graph to incorporate the pedestrian's surrounding environment into the consideration of social interaction:

[0036]

[0037] As a further optimization of the pedestrian trajectory prediction method based on a dual-branch spatiotemporal graph convolutional network provided by this invention, in step three, spatiotemporal features are extracted using spatiotemporal graph convolution, specifically as follows:

[0038] Using GCN, convolution operations are performed at each time step t. (Graph G) t The convolution operation is represented as:

[0039]

[0040]

[0041]

[0042] A t Represents graph G t The adjacency matrix, D t It is the degree matrix of the graph at time t. It is a relational graph with added self-loops. It is a matrix The diagonal node degree matrix, σ(.) is the nonlinear activation function, and the spatial feature representation obtained after convolving the bi-branch spatial graph is:

[0043]

[0044]

[0045] V t ′ ,p V t ′ ,g This represents the spatial feature output obtained after convolution of the two branch spatial graphs. After GCN processes social and scene interactions, TCN is used to process pedestrian temporal dependencies. TCN receives the spatiotemporal features output by GCN and applies them to the time channel. To better simulate the complex movement patterns of pedestrians, an attention mechanism is executed on the node features to assign weights to each time node.

[0046]

[0047] With temporal graph node weights, similar to the spatial graph, GCN is used to extract pedestrian temporal features. The outputs of the two branches are V. n ′ ,p V n ′ ,g , which is an embedding vector containing the spatiotemporal features of pedestrians.

[0048] As a further optimization of the pedestrian trajectory prediction method based on a dual-branch spatiotemporal graph convolutional network provided by this invention, in step four, a temporal extrapolation convolutional network is used to process the spatiotemporal features output by the two branches to predict the Gaussian distribution parameters of the trajectory. The step of adaptively fusing the two branches using an ensemble network is as follows:

[0049] Temporal extrapolation convolutional networks receive spatiotemporal features and directly apply them to the temporal channel. Then, convolution operators are used to correctly infer further trajectories. After convolution, we obtain [T]. obs+1 :T pred Parameters of the bivariate Gaussian distribution of the coordinate sequence during the period in Represents the mean of the coordinates. ρ represents the coordinate covariance. t The correlation coefficient between x and y is represented by N. The outputs of the group-wise and pair-wise branches are respectively determined by N. p and N g This indicates that the trajectory table distribution generated under the branch-aggregation network is N = MLP(N p N g ).

[0050] The predicted output is obtained by sampling a two-dimensional Gaussian distribution.

[0051]

[0052] As a further optimization of the pedestrian trajectory prediction method based on a bi-branch spatiotemporal graph convolutional network provided by the present invention, in step four, the uncertainty of the future trajectory is handled by generating multiple outputs based on the generated prediction model. In order to avoid a large difference between the generated samples and the ground truth, the trajectory closest to the true distribution is selected as the final predicted trajectory by sampling the samples 20 times. We use the predicted Gaussian distribution parameters and the ground truth of the future trajectory to calculate the negative log-likelihood loss to train the model.

[0053]

[0054] Two commonly used metrics for evaluating performance are Average Displacement Error (ADE) and Final Displacement Error (FDE). ADE calculates the average L2 distance between the ground truth and the predicted trajectory, while FDE calculates the L2 distance between the ground truth endpoint and the predicted endpoint. These two metrics are defined as follows:

[0055]

[0056]

[0057] Where n represents the number of people in the row, T pred To predict the number of frames, and These are the predicted value and the ground truth value at time t, respectively.

[0058] As a further optimization scheme for the pedestrian trajectory prediction method based on a bi-branch spatiotemporal graph convolutional network provided by the present invention, the grouping threshold θ and the parameters of the spatiotemporal graph convolutional network are adjusted using the training set. During the test, 20 samples sampled using the predicted distribution are used to output the sample that is closest to the true distribution.

[0059] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0060] 1. This invention uses a graph structure to simulate pedestrian social interactions, constructing a two-branch graph spatiotemporal network. In the pair-wise branch, pedestrian movement speed, direction, and distance are calculated based on the observed pedestrian movement trajectory, and interactions between pedestrians are calculated to simulate individual behavior between pedestrians. In the group-wise branch, L2 similarity of pedestrian trajectory features is used to automatically find groups. Pedestrians interacting within a group are used to calculate interaction weights based on a distance kernel function to simulate group behavior of pedestrians. The two-branch spatiotemporal network comprehensively models complex social interactions, thereby predicting acceptable trajectories under various social norms.

[0061] 2. This invention incorporates environmental information into its scope, using a pre-trained VGG19 network to extract environmental features. To amplify the impact of the scene, it uses pedestrian coordinates and environmental features at each moment of the observation time to calculate the scene attention features of pedestrians at each moment. The extracted scene attention features are then embedded into node features, enabling the transmission of environmental information in the GCN and improving the accuracy of model prediction.

[0062] 3. This invention uses a temporal graph convolutional network with attention, which adaptively assigns corresponding weights to each node and observes specific time steps in the trajectory, thereby better modeling the unique motion patterns of pedestrians and improving the predictive performance of the model.

[0063] 4. The new pedestrian trajectory prediction method adopted in this invention combines multiple technical means to better capture the complex interactions between pedestrians and others and the environment, and can model the time dependence of the trajectory well, thereby improving the prediction accuracy and generalization ability. Attached Figure Description

[0064] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0065] Figure 1 The flowchart shows the overall process of the pedestrian trajectory prediction method based on a bi-branch spatiotemporal graph convolutional network provided by this invention.

[0066] Figure 2 for Figure 1 A schematic diagram of pedestrian grouping in a pedestrian trajectory prediction method based on a bi-branch spatiotemporal graph convolutional network.

[0067] Figure 3 This is a comparison chart of different sampling times on the ETH-UCY dataset for this invention.

[0068] Figure 4 This is a schematic diagram comparing the training of the present invention on the ETH-UCY dataset with existing deep learning schemes.

[0069] Figure 5This is a schematic diagram comparing the training of the present invention on the SDD dataset with existing deep learning schemes. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Of course, the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0071] Example 1

[0072] See Figures 1 to 5 This embodiment proposes a pedestrian trajectory prediction method based on a bi-branch spatiotemporal graph convolutional network.

[0073] In this embodiment, the experiments of the present invention were conducted on PyTorch, and network training and testing were performed on an NVIDIA 2070Ti GPU. During the experiments, the model was trained and evaluated on the ETH and UCY public datasets, and compared with other methods using the same datasets.

[0074] 1. Data Preprocessing

[0075] For the dataset, we use downsampling to count only the number of trajectories with a time step length greater than or equal to 20.

[0076] 2. Scene Feature Extraction

[0077] To fully utilize scene information and extract visual features from the scene, a pre-trained VGG19 was used as a convolutional neural network.

[0078] 3. Spatiotemporal graph convolution

[0079] At time step t, connect all pedestrians to form a complete graph G. t Figure G t The node features include world coordinate representations of pedestrians. In the pairwise interaction branches, a kernel function is used to calculate pedestrian interaction weights based on pedestrian speed, direction, and distance, resulting in a pedestrian adjacency matrix and constructing a pairwise pedestrian interaction graph. In the group interaction branches, a defined method is used to group pedestrians based on their historical trajectories, obtaining a pedestrian mask to filter out irrelevant redundant edges. A distance-based kernel function is then used to calculate the weights of pedestrians within each group, resulting in a pedestrian adjacency matrix and constructing an intra-group pedestrian interaction graph. To fully utilize scene information, the extracted visual features are embedded into graph G. tThe node features are used simultaneously in the two branches to realize the spatial transmission of node features. In order to better model the unique motion pattern of pedestrians and model the time dependence, the spatial features output by GCN are used as input to TCN and applied to the time channel to construct a time map. An attention mechanism is used to assign corresponding weights to each time node to realize the time step of the observed features and output the spatiotemporal features of the two-branch graph.

[0080] The specific pedestrian groupings are as follows:

[0081] 1) Take the historical trajectory of the pedestrian as input and use L2 distance to calculate the similarity of different pedestrian trajectories to obtain the pedestrian trajectory similarity matrix.

[0082] 2) Define a learnable threshold θ, and match trajectories in the similarity matrix whose feature similarity is less than the threshold as similar pairs.

[0083] 3) Use pedestrian pairs to assign all related pedestrians to the same group to construct an interaction mask.

[0084] 4. Prediction Model

[0085] Temporal extrapolation convolutional networks receive spatiotemporal features and directly apply them to the temporal channel. Then, convolution operators are used to correctly infer further trajectories. After convolution, we obtain [T]. obs+1 :T pred Parameters of the bivariate Gaussian distribution of the coordinate sequence during the period The outputs of the group-wise branch and the pair-wise branch are respectively determined by N. p and N g This indicates that the trajectory table distribution generated under the branch-aggregation network is N = MLP(N p N g The predicted output is obtained by sampling a two-dimensional Gaussian distribution N. To avoid a large difference between the generated samples and the ground truth, the trajectory that is closest to the true distribution is selected as the final predicted trajectory by sampling the samples 20 times.

[0086] 5. Comparative Experiment

[0087] The proposed method was evaluated on two well-known pedestrian trajectory prediction datasets, ETH and UCY. ETH contains two scenes, denoted as ETH and HOTEL, while UCY contains three scenes, denoted as ZARA1, ZARA2, and UNIV. Samples on both datasets were sampled at 0.4s / 8s. During training and evaluation, the first 3.2 seconds (8 frames) were used as the observation history, and the remaining 4.8 seconds (12 frames) were used as the ground truth for prediction. The results were compared with other state-of-the-art methods, and we used two evaluation metrics:

[0088]

[0089]

[0090] ADE is used to calculate the average L2 distance between the ground truth and the predicted trajectory, while FDE calculates the L2 distance between the ground truth endpoint and the predicted endpoint. Figure 4 It can be seen that we have the best results.

[0091] Example 2

[0092] The SDD dataset uses drones to collect trajectory interaction information for 20,000 objects in eight different scenarios during peak campus hours, presented from an overhead view. Each object's trajectory is labeled with a unique ID. The eight scenarios include multiple classes navigating the campus. A total of 19,000 objects are included, comprising 11,200 pedestrians, 6,400 cyclists, 1,300 cars, 300 skateboarders, 200 golf carts, and 100 buses. Coordinates in SDD are given in pixels.

[0093] The specific implementation schemes described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific implementation schemes of the present invention and are not intended to limit the scope of the present invention. Any equivalent changes and modifications made by those skilled in the art without departing from the concept and principles of the present invention should fall within the scope of protection of the present invention.

Claims

1. A method for predicting pedestrian trajectories based on a dual-branch spatiotemporal graph convolutional network, characterized in that, Includes the following steps: Step 1: Extract scene visual features using a pre-trained CNN; Step 2: Construct pairwise interaction graphs and inter-group interaction graphs for pedestrians in both branches, and embed scene features into the graphs based on the trajectory coordinates input by the pedestrians ( The proposed method, which processes pedestrian data into a graphical representation, consists of two branches. The pairwise interaction branch calculates the influence matrix of graph nodes based on the relative position, velocity, and direction of the pedestrian agents, constructing pairwise interaction graphs to simulate interactions between pedestrians. The grouping branch establishes pedestrian pairs using the L2 similarity of pedestrian trajectories and groups pedestrians based on a given learning threshold θ, creating a grouping mask. To handle grouping, a distance-based kernel function is used to calculate the influence matrix of graph nodes for pedestrians in the same group, constructing an intra-group interaction graph to simulate interactions between pedestrians, and embedding scene features in the bi-branch graph; In step two, pairwise pedestrian interaction graphs and inter-group interaction graphs are constructed in the two branches respectively, and scene features are embedded in the graphs, specifically: Construct a pedestrian space map at time t. , Representing node characteristics, Representation diagram For the edges, in the pairwise interaction branches, use the following function to calculate the pairwise interactions of pedestrians: ； Let represent the velocity of pedestrian i at time t, and α and β represent the angles between the velocity vector and the medium. The distance between pedestrians is represented by the pedestrian adjacency matrix after weight normalization. Use the obtained adjacency matrix Representation of spatial diagram The node weights are used to obtain a graph of pairwise interactions between pedestrians. In the group interaction branch, pedestrian historical trajectories are used as input, and L2 distance is used to calculate the similarity between different pedestrian trajectories: ； in pedestrian Feature similarity, F(.) represents a learnable convolutional layer, and a learnable threshold is defined. Trajectories with feature similarity less than a threshold are matched as similar pairs; ； S represents the set of pedestrian similarity pairs. This represents pairs of pedestrians with similar trajectory features. All related pedestrians are grouped into the same group using the pedestrian pair set, and a group index is assigned. A group mask M is defined based on the pedestrian grouping results. ； If node i and node j are in the same group, then Using 1 to indicate that the interaction mask removes redundant edges that are irrelevant to pedestrians, for pedestrians interacting within a group, a distance-based kernel function is used to construct the interaction weights within the pedestrian group: ； The weights represent the influence between pedestrian nodes interacting within a group. Normalizing these weights yields the pedestrian adjacency matrix. Use the obtained adjacency matrix Representation of spatial diagram The node weights yield the pedestrian group interaction graph. elements Represented as: ； Insert the scene visual features obtained in step one into the node features of the bibranch graph to incorporate the pedestrian's surrounding environment into the consideration of social interaction: ； Step 3: Extract spatiotemporal features using spatiotemporal graph convolution. On the graphs of the two branches, use a spatial GCN at each time step. Convolutional operations are performed on the data. After processing social and scene interactions, TCN is used to model pedestrian temporal dependencies. TCN receives the output of GCN and uses self-attention to calculate the influence strength of nodes at different time steps. It focuses on specific time steps of pedestrians and extracts temporal features through convolution. Spatiotemporal graph convolution captures the spatiotemporal features of pedestrians. Step 4: The temporal extrapolation convolutional network processes the spatiotemporal features of the two branches' outputs to predict the two-dimensional Gaussian distribution parameters of the trajectory. The spatiotemporal features extracted by the dual branches are embedded into the temporal extrapolation convolutional network Txp-cnn. Txp-cnn directly acts on the temporal channel to predict the Gaussian distribution parameters for the next 12 time steps, and the two branch prediction parameters are adaptively fused using the ensemble network MLP to obtain the final trajectory prediction parameters.

2. The pedestrian trajectory prediction method based on a bi-branch spatiotemporal graph convolutional network according to claim 1, characterized in that, In step one, extracting pedestrian scene features using a pre-trained CNN includes: Extracting visual features from pedestrian scenes using a pre-trained convolutional network: ； Where CNN(.) represents a pre-trained network, For the weights of the pre-trained network, This represents the scene images in the dataset.

3. The pedestrian trajectory prediction method based on a bi-branch spatiotemporal graph convolutional network according to claim 2, characterized in that, In step three, spatiotemporal features are extracted using spatiotemporal graph convolution, specifically as follows: Using GCN to perform convolution operations at each time step t, the convolution operation of the graph at time t is represented as: ； ； ； Representation diagram The adjacency matrix, It is the degree matrix of the graph at time t. It is a non-linear activation function. It is a relational graph with added self-loops. It is a matrix The degree matrix of the diagonal nodes, For the new node features of the graph Gt mentioned above, which embeds scene features, the spatial feature representation after the bi-branch graph convolution is obtained by the following formula: ； ； , This represents the spatial feature output obtained after convolution of the two branch spatial graphs. After GCN processes social and scene interactions, TCN is used to process pedestrian temporal dependencies. TCN receives the spatiotemporal features output by GCN and applies them to the temporal channel. The features output by GCN are then processed by one-dimensional convolution. This means that an attention mechanism is applied to the node features to assign weights to each time point: ； With temporal graph node weights, similar to the spatial graph, GCN is used to extract pedestrian temporal features. The outputs of the two branches are respectively... , , which is an embedding vector containing the spatiotemporal features of pedestrians.

4. The pedestrian trajectory prediction method based on a bi-branch spatiotemporal graph convolutional network according to claim 3, characterized in that, In step four, a temporal extrapolation convolutional network is used to process the spatiotemporal features output from the two branches to predict the Gaussian distribution parameters of the trajectory. An ensemble network is then used to adaptively fuse the two branches. Specifically: Txp-CNN receives the spatiotemporal features from the dual-branch output and directly applies them to the time channel. It uses convolution operators to infer further trajectories. After convolution by Txp-CNN, the result is […]. The coordinate sequence during the period has a bivariate Gaussian distribution parameter N( , ),in Represents the mean of the coordinates. Represents coordinate covariance. This represents the correlation coefficient between x and y. The outputs of the group-wise and pair-wise branches are respectively... and This indicates that the trajectory table distribution generated under the branch-aggregation network is N=MLP( , ); The predicted output is obtained by sampling a two-dimensional Gaussian distribution. =( ); 。 5. The pedestrian trajectory prediction method based on a bi-branch spatiotemporal graph convolutional network according to claim 4, characterized in that, The model is trained by calculating the negative log-likelihood loss using the predicted Gaussian distribution parameters and the ground truth of the future trajectory. ； Two metrics are used to evaluate performance: Average Displacement Error (ADE) and Final Displacement Error (FDE). ADE is used to calculate the average L2 distance between the ground truth and the predicted trajectory, while FDE calculates the L2 distance between the ground truth endpoint and the predicted endpoint. These two metrics are defined as follows: ； t=pred ; Where n represents the number of people in each row. To predict the number of frames, and These are the predicted value and the ground truth value at time t, respectively.

Images