A trajectory prediction method based on endpoint-guided iterative learning and one-time prediction

Abstract

This invention discloses a trajectory prediction method based on endpoint-guided iterative learning and one-time prediction. The method includes: constructing a spatiotemporal interaction module to obtain spatiotemporal interaction features based on the target pedestrian; constructing an endpoint prediction module to obtain the predicted endpoint position; constructing a trajectory prediction module, which concatenates the spatiotemporal interaction features based on the target pedestrian, multiple learnable future trajectory location tokens, and the embedded features corresponding to the predicted endpoint position, inputs the concatenation into the trajectory prediction module, and predicts the trajectory position at future moments based on the spatiotemporal interaction features of the target pedestrian; constructing a loss calculation module to calculate the final loss value and optimize the model to obtain the optimal pedestrian trajectory prediction model; and in the testing phase, using the optimal pedestrian trajectory prediction model to obtain the complete trajectory of the test target pedestrian at future moments. This invention effectively eliminates the error accumulation problem through endpoint-guided iterative learning and one-time prediction, improving computational efficiency and prediction accuracy.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, pattern recognition, digital image processing, artificial intelligence, and pedestrian trajectory prediction, and specifically relates to a trajectory prediction method based on endpoint-guided cyclic learning and one-time prediction. Background Technology

[0002] In recent years, significant progress has been made in pedestrian trajectory prediction methods based on deep learning. Among these, Xue et al. utilized a recurrent prediction method. Specifically, LSTM predicts the position at each future time step sequentially based on recurrent learning. First, it predicts the position at the first future time step based on all observed trajectory positions; then, it predicts the position at the second future time step based on all observed trajectory positions and the first predicted future time step, and so on, until the positions at all future time steps are obtained. Mohamed et al. converted the observed trajectory positions of the target pedestrian and neighboring pedestrians at each time step into a spatiotemporal graph. The first image stores the observed trajectory positions of the target pedestrian and neighboring pedestrians at the first time step, and the second image stores the observed trajectory positions of the target pedestrian and neighboring pedestrians at the second time step. Then, a graph convolutional neural network is used to process the image composed of observed trajectories at all time steps, and a one-time prediction method is used to predict the positions at all future time steps at once. Wong et al. proposed segmenting the target pedestrian's neighbors into several regions based on angles, assigning equal weights to all neighbors in each region—that is, neighbors closer to the target pedestrian and those farther away have the same weight. They then extracted the spatiotemporal interaction features of the target pedestrian within each region and predicted the trajectory for future moments based on these features. Zhao et al. introduced a three-stage network: first, predicting the endpoint position, and then, based on the predicted endpoint position, predicting the positions for other future moments sequentially.

[0003] The aforementioned methods improve pedestrian trajectory prediction performance to some extent, but still have certain limitations. These methods neglect the deep synergistic relationship between cyclic prediction and one-time prediction methods. They either use only cyclic prediction to predict the position at each future time step sequentially, which can learn the temporal dependencies between future trajectories but is time-inefficient and suffers from error accumulation; or they use one-time prediction to predict the position at all future time steps at once, which is time-efficient and avoids error accumulation, but cannot learn the dependencies between future trajectories. Meanwhile, existing pedestrian trajectory prediction methods ignore the influence of different neighbors on the target pedestrian's trajectory at different times. Therefore, a method is needed that can effectively overcome the shortcomings of cyclic and one-time prediction while effectively utilizing their advantages, and dynamically adjust neighbor weights based on spatial distance and temporal relationships. Summary of the Invention

[0004] The purpose of this invention is to combine the advantages of cyclic prediction methods and one-time prediction methods, and to dynamically adjust the weights of neighbors based on their location and temporal relationships, thereby improving the performance of pedestrian trajectory prediction. To this end, this invention provides a trajectory prediction method based on endpoint-guided cyclic learning and one-time prediction.

[0005] To achieve the aforementioned objective, the present invention proposes a trajectory prediction method based on endpoint-guided cyclic learning and one-time prediction, comprising the following steps:

[0006] Step S1: Construct a spatiotemporal interaction module to target pedestrians. The observed trajectories of the target pedestrian and its neighboring pedestrians are input into the spatiotemporal interaction module to obtain data based on the target pedestrian. The spatiotemporal interaction characteristics;

[0007] Step S2, construct the destination prediction module, and use the target pedestrian-based prediction module to predict the destination. The spatiotemporal interaction features and learnable endpoint tokens are input into the endpoint prediction module, and the predicted endpoint position is obtained with the help of the endpoint position regression head.

[0008] Step S3, construct a trajectory prediction module, and use the target pedestrian-based... The spatiotemporal interaction features, multiple learnable future trajectory location tokens, and the embedded features corresponding to the predicted endpoint location are concatenated and input into the trajectory prediction module to predict a trajectory based on the target pedestrian. The trajectory position at future moments based on the spatiotemporal interaction characteristics;

[0009] Step S4: Construct a loss calculation module, in which the target pedestrian is used. The true trajectory position at future moments and the target pedestrian obtained in step S3 The final loss value is obtained by calculating the trajectory position of the future moment of the spatiotemporal interaction features. The final loss value is then used to perform backpropagation optimization on the pedestrian trajectory prediction model composed of the spatiotemporal interaction module, the endpoint prediction module, and the trajectory prediction module. After multiple optimization iterations, the optimal pedestrian trajectory prediction model is obtained.

[0010] Step S5: During the testing phase, the optimal pedestrian trajectory prediction model is used to make a one-time prediction based on the observed trajectory of the test target pedestrian to obtain the complete trajectory of the test target pedestrian at future times. The complete trajectory of the test target pedestrian at future times includes the trajectory positions of all future times based on the spatiotemporal interaction characteristics of the test target pedestrian and the endpoint position of the future times based on the spatiotemporal interaction characteristics of the test target pedestrian.

[0011] Optionally, step S1 includes the following steps:

[0012] Step S11, move the target pedestrian The observed trajectories of the target pedestrian and its neighbors are input into the embedding layer to obtain the target pedestrian-based data. Embedding features of observed trajectories and embedding features of observed trajectories based on neighboring pedestrians;

[0013] Step S12, merge the target pedestrian-based... The embedding features of the observed trajectory and the embedding features of the observed trajectory based on neighboring pedestrians are used to obtain the target pedestrian-based observation trajectory. Merged embedding features of pedestrians and their neighbors :

[0014] Step S13: Construct a spatiotemporal interaction module to integrate the target pedestrian-based... Merged embedding features of pedestrians and their neighbors Inputting the spatiotemporal interaction module yields results based on the target pedestrian. The spatiotemporal interaction characteristics.

[0015] Optionally, the spatiotemporal interaction module includes a temporal feature extraction layer, a spatial feature extraction layer, and a spatiotemporal interaction layer, wherein the temporal feature extraction layer is used to extract the target pedestrian. The spatial feature extraction layer is used to extract the temporal features of the target pedestrian. The spatiotemporal interaction layer, composed of multiple Transformers, is used to extract spatial features based on the target pedestrian. The spatiotemporal interaction characteristics.

[0016] Optionally, step S13 includes the following steps:

[0017] Step S131, target pedestrian The observed trajectories of the target pedestrian and its neighboring pedestrians are input into the temporal feature extraction layer of the spatiotemporal interaction module, and the target pedestrian is obtained based on the temporal proximity relationship. Temporal attention mask matrix ;

[0018] Step S132, target pedestrian The observed trajectories of the target pedestrian and their neighbors are input into the spatial feature extraction layer of the spatiotemporal interaction module, and based on spatial relationships, the target pedestrian's trajectory is obtained. Spatial attention mask matrix ;

[0019] Step S133, utilizing the target pedestrian-based... Temporal attention mask matrix and the target pedestrian Spatial attention mask matrix The dot product is used to obtain the target pedestrian. Spatiotemporal attention mask matrix ;

[0020] Step S134, based on the target pedestrian Spatiotemporal attention mask matrix and the target pedestrian Merged embedding features of pedestrians and their neighbors Inputting the spatiotemporal interaction layer of the spatiotemporal interaction module yields results based on the target pedestrian. spatiotemporal interaction characteristics .

[0021] Optionally, step S2 includes the following steps:

[0022] Step S21, based on the target pedestrian The spatiotemporal interaction features and learnable endpoint tokens are concatenated to obtain a target pedestrian-based result. Input trajectory features;

[0023] Step S22: Construct an endpoint prediction module, and use the endpoint prediction module to predict the destination based on the target pedestrian. Feature extraction is performed on the input trajectory features to obtain the endpoint position features based on the input trajectory features;

[0024] Step S23: Input the endpoint location features based on the input trajectory features into the endpoint location regression head to obtain multiple predicted endpoint locations. During the training phase, a filter is used to select those that match the target pedestrian. The actual endpoint location is the closest predicted endpoint location.

[0025] Optionally, step S3 includes the following steps:

[0026] Step S31: Embedding processing is performed on the predicted endpoint position to obtain the embedded features corresponding to the predicted endpoint position, so as to match the features with the target pedestrian-based feature. The spatiotemporal interaction features and the dimension of the location token that can learn future trajectories are the same;

[0027] Step S32, based on the target pedestrian The spatiotemporal interaction features, the multiple learnable future trajectory location tokens, and the embedded features corresponding to the predicted endpoint position are cyclically concatenated to obtain a target pedestrian-based... Trajectory characteristics at future moments, excluding the endpoint;

[0028] Step S33: Construct a trajectory prediction module, which is based on the target pedestrian... The trajectory features, excluding the destination, at future moments are input into the trajectory prediction module for cyclic prediction to obtain a trajectory prediction based on the target pedestrian. The trajectory position of future moments based on the spatiotemporal interaction characteristics.

[0029] Optionally, in step S32, during the first loop stitching, the stitching based on the target pedestrian is performed. The spatiotemporal interaction features, the location token of the first learnable future trajectory, and the embedding features corresponding to the predicted endpoint position are used to obtain a target pedestrian-based... The trajectory features of the first future moment; during the second cyclic stitching, the stitching is based on the target pedestrian. The spatiotemporal interaction features, the location tokens of the first two learnable future trajectories, and the embedding features corresponding to the predicted endpoint position are used to obtain a target pedestrian-based... The trajectory characteristics of the first two future moments; and so on, the trajectory characteristics of the first two future moments. In the next iteration of the stitching process, the stitching is based on the target pedestrian. The spatiotemporal interaction characteristics, all A location token that can learn future trajectories and the embedded features corresponding to the predicted endpoint location are used to obtain a target pedestrian-based... All Trajectory characteristics at a future moment, excluding the endpoint.

[0030] Optionally, in step S33, during cyclic prediction, the input to the nth cyclic prediction is the output of the nth cyclic concatenation in step S32, and the output is based on the target pedestrian. The trajectory positions of the nth future time steps based on the spatiotemporal interaction characteristics and the nth prediction based on the target pedestrian The endpoint position of the future moment based on the spatiotemporal interaction characteristics of the target pedestrian. All spatiotemporal interaction features The trajectory position at the 1st future moment and the 2nd future moment The prediction based on the target pedestrian The future endpoint position of the spatiotemporal interaction features is composed of the target pedestrian. The trajectory position of future moments based on the spatiotemporal interaction characteristics.

[0031] Optionally, step S4 includes the following steps:

[0032] Step S41, construct the loss calculation module;

[0033] Step S42, in the loss calculation module, using the target pedestrian The actual trajectory location at a future moment, based on the target pedestrian The trajectory position at future moments of the spatiotemporal interaction characteristics, through The norm is used to calculate the final loss value. ;

[0034] Step S43, using the final loss value The pedestrian trajectory prediction model, which consists of the spatiotemporal interaction module, the endpoint prediction module, and the trajectory prediction module, is optimized by backpropagation. After multiple iterations, the optimal pedestrian trajectory prediction model is obtained.

[0035] Optionally, step S5 includes the following steps:

[0036] Step S51: In the testing phase, the observation trajectories of the test target pedestrian and its neighboring pedestrians are embedded to obtain the embedding features based on the observation trajectory of the test target pedestrian and the embedding features based on the observation trajectories of the test target pedestrian and its neighboring pedestrians. After merging, the embedding features are input into the spatiotemporal interaction module of the optimal pedestrian trajectory prediction model to obtain the spatiotemporal interaction features based on the test target pedestrian.

[0037] Step S52: Concatenate the spatiotemporal interaction features based on the test target pedestrian and the trained endpoint token to obtain the input trajectory features based on the test target pedestrian, and input them into the endpoint prediction module of the optimal pedestrian trajectory prediction model to obtain the endpoint position features based on the input trajectory features of the test target pedestrian.

[0038] Step S53: Input the endpoint position features based on the input trajectory features of the test target pedestrian into the endpoint position regression head to obtain multiple predicted endpoint positions, and perform embedding processing on each predicted endpoint position to obtain multiple embedded features based on the predicted endpoint positions.

[0039] Step S54: For each embedding feature based on the predicted endpoint location, the spatiotemporal interaction feature based on the test target pedestrian, the location token of the trained future trajectory, and the embedding feature based on the predicted endpoint location are concatenated in sequence to obtain the trajectory feature based on the test target pedestrian corresponding to the embedding feature based on the predicted endpoint location.

[0040] Step S55: Input the trajectory features based on the test target pedestrian into the trajectory prediction module of the optimal pedestrian trajectory prediction model to predict the trajectory positions of all future moments at once. This results in multiple sets of future trajectory positions and future endpoint positions based on the spatiotemporal interaction features of the test target pedestrian. Each set of future trajectory positions and future endpoint positions based on the spatiotemporal interaction features of the test target pedestrian constitutes a complete future trajectory of the test target pedestrian.

[0041] This invention effectively eliminates the problem of error accumulation by using endpoint-guided cyclic learning and one-time prediction, thereby improving computational efficiency and prediction accuracy. Attached Figure Description

[0042] Figure 1 This is a flowchart of a trajectory prediction method based on endpoint-guided cyclic learning and one-time prediction according to an embodiment of the present invention;

[0043] Figures 2A-2F This is a visualization of pedestrian trajectory prediction results on the ETH / UCY dataset obtained according to an embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0045] Figure 1 This is a flowchart of a trajectory prediction method based on endpoint-guided cyclic learning and one-time prediction according to an embodiment of the present invention. The following is an example... Figure 1 To illustrate some specific implementation processes of the present invention, we will use examples, such as... Figure 1 As shown, the trajectory prediction method based on endpoint-guided cyclic learning and one-time prediction includes the following steps:

[0046] Step S1: Construct a spatiotemporal interaction module to target pedestrians. The observed trajectories of the target pedestrian and its neighboring pedestrians are input into the spatiotemporal interaction module to obtain data based on the target pedestrian. The spatiotemporal interaction characteristics;

[0047] In one embodiment of the present invention, the target pedestrian The neighboring pedestrian refers to the target pedestrian. Pedestrians within a circle with a preset radius and centered at a given point; here, the target pedestrian. Neighboring pedestrians also include the target pedestrian. This factor will be taken into account in subsequent calculations.

[0048] Further, step S1 includes the following steps:

[0049] Step S11, move the target pedestrian The observed trajectories of the target pedestrian and its neighbors are input into the embedding layer to obtain the target pedestrian-based data. Embedding features of observed trajectories and embedding features of observed trajectories based on neighboring pedestrians;

[0050] Among them, based on the target pedestrian No. Embedding features of the observed trajectory at any time and based on neighbor pedestrians No. Embedding features of the observed trajectory at any time It can be represented as:

[0051]

[0052]

[0053] in,( Indicates the target pedestrian No. The position of the trajectory is constantly observed. Indicates neighboring pedestrians No. Observe the position of the trajectory at all times. Indicates the embedding layer. and This represents the learnable weights of the embedding layer.

[0054] In one embodiment of the present invention, the embedding layer is obtained based on a multilayer perceptron.

[0055] Step S12, merge the target pedestrian-based... The embedding features of the observed trajectory and the embedding features of the observed trajectory based on neighboring pedestrians are used to obtain the target pedestrian-based observation trajectory. Merged embedding features of pedestrians and their neighbors :

[0056]

[0057] in, Indicates the target pedestrian The number of neighboring pedestrians, as mentioned above, is the target pedestrian. Neighboring pedestrians also include the target pedestrian. Therefore, the target pedestrian The actual number of pedestrians in the neighborhood is , Indicates the target pedestrian The length of the observed trajectories of the pedestrians and their neighbors. Indicates based on target pedestrian Embedding features of the observed trajectory Indicates based on target pedestrian excluding target pedestrians Embedded features of the observed trajectories of all neighboring pedestrians other than the original pedestrian.

[0058] Step S13: Construct a spatiotemporal interaction module to integrate the target pedestrian-based... Merged embedding features of pedestrians and their neighbors Inputting the spatiotemporal interaction module yields results based on the target pedestrian. The spatiotemporal interaction characteristics.

[0059] The spatiotemporal interaction module includes a temporal feature extraction layer, a spatial feature extraction layer, and a spatiotemporal interaction layer. The temporal feature extraction layer is used to extract the target pedestrian. The spatial feature extraction layer is used to extract the temporal features of the target pedestrian. The spatiotemporal interaction layer, composed of multiple Transformers, is used to fuse spatiotemporal features and extract spatial characteristics based on the target pedestrian. The spatiotemporal interaction characteristics.

[0060] Step S13 further includes the following steps:

[0061] Step S131, target pedestrian The observed trajectories of the target pedestrian and its neighboring pedestrians are input into the temporal feature extraction layer of the spatiotemporal interaction module, and the target pedestrian is obtained based on the temporal proximity relationship. Temporal attention mask matrix :

[0062]

[0063]

[0064]

[0065] in, , Used to control the weight size. This indicates a modulo operation. This indicates that the target pedestrian is being considered. Temporal attention mask matrix No. line, number The elements of the column.

[0066] Step S132, target pedestrian The observed trajectories of the target pedestrian and their neighbors are input into the spatial feature extraction layer of the spatiotemporal interaction module, and based on spatial relationships, the target pedestrian's trajectory is obtained. Spatial attention mask matrix :

[0067]

[0068]

[0069]

[0070] in, , Used to control the weight size. Represents Euclidean distance. Indicates rounding up. This indicates that the target pedestrian is being considered. Spatial attention mask matrix No. line, number The elements of the column.

[0071] Step S133, utilizing the target pedestrian-based... Temporal attention mask matrix and the target pedestrian Spatial attention mask matrix The dot product is used to obtain the target pedestrian. Spatiotemporal attention mask matrix :

[0072]

[0073] in, This indicates the dot product operation.

[0074] Step S134, based on the target pedestrian Spatiotemporal attention mask matrix and the target pedestrian Merged embedding features of pedestrians and their neighbors Inputting the spatiotemporal interaction layer of the spatiotemporal interaction module yields results based on the target pedestrian. spatiotemporal interaction characteristics :

[0075]

[0076]

[0077]

[0078]

[0079]

[0080] Where TST stands for Spatiotemporal Interaction Layer. This represents the dot product operation. For querying the matrix, The key matrix, for The transpose of the matrix, As the normalization factor, For value matrices, This is the learnable parameter matrix.

[0081] Step S2, construct the destination prediction module, and use the target pedestrian-based prediction module to predict the destination. The spatiotemporal interaction features and learnable endpoint tokens are input into the endpoint prediction module, and the predicted endpoint position is obtained with the help of the endpoint position regression head.

[0082] Here, the learnable endpoint token refers to a learnable token vector corresponding to the endpoint position. At the beginning of training, the learnable endpoint token is a randomly initialized vector. During subsequent training, the learnable endpoint token changes as the training process progresses, eventually resulting in a single endpoint token.

[0083] Further, step S2 includes the following steps:

[0084] Step S21, based on the target pedestrian The spatiotemporal interaction features and learnable endpoint tokens are concatenated to obtain a target pedestrian-based result. Input trajectory features;

[0085] In one embodiment of the present invention, in order to ensure the target pedestrian The temporal nature of the predicted location, for the target pedestrian. When concatenating the spatiotemporal interaction features and learnable endpoint tokens, the target pedestrian-based features are concatenated first. The spatiotemporal interaction characteristics are then concatenated with the learnable endpoint token.

[0086] Step S22: Construct an endpoint prediction module, and use the endpoint prediction module to predict the destination based on the target pedestrian. Feature extraction is performed on the input trajectory features to obtain the endpoint position features based on the input trajectory features, which are used to constrain subsequent trajectory prediction.

[0087] In one embodiment of the present invention, the destination prediction module is composed of multiple Transformers, and the input is the target pedestrian-based prediction module. The input trajectory features are used to output the endpoint position features based on the input trajectory features.

[0088] Step S23: Input the endpoint location features based on the input trajectory features into the endpoint location regression head to obtain multiple predicted endpoint locations. During the training phase, a filter is used to select those that match the target pedestrian. The actual endpoint location is the closest predicted endpoint location.

[0089] In one embodiment of the present invention, the endpoint location regression head is composed of a multilayer sensing mechanism. To simulate the multimodal characteristics of pedestrian trajectories, the endpoint location regression head can simultaneously output multiple predicted endpoint locations. During the training phase, the filter selects the target pedestrian. The training process proceeds in the next stage by using the closest predicted endpoint position to the actual endpoint position.

[0090] Step S3, construct a trajectory prediction module, and use the target pedestrian-based... The spatiotemporal interaction features, multiple learnable future trajectory location tokens, and the embedded features corresponding to the predicted endpoint location are concatenated and input into the trajectory prediction module to predict a trajectory based on the target pedestrian. The trajectory position at future moments based on the spatiotemporal interaction characteristics;

[0091] Here, the learnable future trajectory location token refers to a learnable location token that is related to the target pedestrian. The token vector corresponding to each trajectory point position in the future trajectory, excluding the endpoint. Similar to the learnable endpoint token, at the start of training, the token vector corresponding to each trajectory point position in the learnable future trajectory position token is also a randomly initialized vector. During subsequent training, the learnable future trajectory position token changes as the training process progresses, ultimately training to obtain the position token of the future trajectory.

[0092] In one embodiment of the present invention, the trajectory prediction module is composed of multiple layers of Transformers. In order to fully improve the performance of the trajectory prediction module and the destination prediction module and reduce the model learning complexity, the trajectory prediction module and the destination prediction module can use the same parameters.

[0093] Further, step S3 includes the following steps:

[0094] Step S31: Embedding processing is performed on the predicted endpoint position to obtain the embedded features corresponding to the predicted endpoint position, so as to match the features with the target pedestrian-based feature. The spatiotemporal interaction features and the dimension of the location token that can learn future trajectories are the same;

[0095] Step S32, based on the target pedestrian The spatiotemporal interaction features, the multiple learnable future trajectory location tokens, and the embedded features corresponding to the predicted endpoint position are cyclically concatenated to obtain a target pedestrian-based... Trajectory characteristics at future moments, excluding the endpoint;

[0096] As mentioned above, the location token that can learn future trajectories and the target pedestrian Each trajectory point in the future trajectory corresponds to a position; therefore, the position token of the learnable future trajectory is related to the target pedestrian. The spatial-temporal interaction features correspond to the trajectory position order at each future moment, i.e., the position token of the first learnable future trajectory corresponds to the target pedestrian. The first future moment's trajectory position of the spatiotemporal interaction features, and the second learnable future trajectory position token corresponding to the target pedestrian. The trajectory position of the second future moment of the spatiotemporal interaction characteristics, and so on.

[0097] If let This indicates that the target pedestrian is being considered. The number of all trajectory locations, including the endpoint, at future moments based on the spatiotemporal interaction characteristics of the target pedestrian, after removing the endpoint, is then... The number of trajectory positions in future moments of the spatiotemporal interaction features is So, during the training phase, there are a total of In the second iteration of stitching, during the first iteration, the stitching is based on the target pedestrian. The spatiotemporal interaction features, the location token of the first learnable future trajectory, and the embedding features corresponding to the predicted endpoint position are used to obtain a target pedestrian-based... The trajectory features of the first future moment; during the second cyclic stitching, the stitching is based on the target pedestrian. The spatiotemporal interaction features, the location tokens of the first two learnable future trajectories, and the embedding features corresponding to the predicted endpoint position are used to obtain a target pedestrian-based... The trajectory characteristics of the first two future moments; and so on, the trajectory characteristics of the first two future moments. In the next iteration of the stitching process, the stitching is based on the target pedestrian. The spatiotemporal interaction characteristics, all A location token that can learn future trajectories and the embedded features corresponding to the predicted endpoint location are used to obtain a target pedestrian-based... All Trajectory characteristics at a future moment, excluding the endpoint.

[0098] Step S33: Construct a trajectory prediction module, which is based on the target pedestrian... The trajectory features, excluding the destination, at future moments are input into the trajectory prediction module for cyclic prediction to obtain a trajectory prediction based on the target pedestrian. The trajectory position of future moments based on the spatiotemporal interaction characteristics.

[0099] In this step, during the iterative prediction, the input for the nth iterative prediction is the output of the nth iterative concatenation in step S32, and the output is based on the target pedestrian. The trajectory positions of the nth future time steps based on the spatiotemporal interaction characteristics and the nth prediction based on the target pedestrian The endpoint position of the future moment of the spatiotemporal interaction features. For example, the input of the first loop prediction is the output of the first loop concatenation in step S32, that is, the concatenation of the target pedestrian-based prediction. The spatiotemporal interaction features, the location token of the first learnable future trajectory, and the embedded features corresponding to the predicted endpoint position are used to obtain the target pedestrian-based data. The trajectory features of the first future moment, the output is based on the target pedestrian. The trajectory location at the first future moment of the spatiotemporal interaction features and the first prediction based on the target pedestrian The endpoint position of the future moment of the spatiotemporal interaction features; the input of the second loop prediction is the output of the second loop concatenation in step S32, that is, the concatenation of the target pedestrian-based prediction. The spatiotemporal interaction features, the location tokens of the first two learnable future trajectories, and the embedded features corresponding to the predicted endpoint position are used to obtain the target pedestrian-based data. The trajectory features of the first two future time moments are used as the output, which is based on the target pedestrian. The trajectory positions of the first two future moments of the spatiotemporal interaction features and the second prediction based on the target pedestrian The endpoint of the future moment of the spatiotemporal interaction characteristics; and so on, the... The input for the next iteration of the prediction is the first iteration in step S32. The output of the next loop concatenation, i.e., the concatenation based on the target pedestrian The spatiotemporal interaction characteristics, all The target pedestrian is obtained by combining a learnable future trajectory location token with the embedded features corresponding to the predicted endpoint location. All The trajectory features at a future moment are used as the output, which is based on the target pedestrian. All spatiotemporal interaction features The trajectory position at the 1st future moment and the 2nd future moment The prediction based on the target pedestrian The endpoint position at a future moment based on the spatiotemporal interaction characteristics, ultimately, the target pedestrian... All spatiotemporal interaction features The trajectory position at the 1st future moment and the 2nd future moment The prediction based on the target pedestrian The future endpoint position of the spatiotemporal interaction features is composed of the target pedestrian. The trajectory position of future moments based on the spatiotemporal interaction characteristics.

[0100] Step S4: Construct a loss calculation module, in which the target pedestrian is used. The true trajectory position at future moments and the target pedestrian obtained in step S3 The final loss value is obtained by calculating the trajectory position of the future moment of the spatiotemporal interaction features. The final loss value is then used to perform backpropagation optimization on the pedestrian trajectory prediction model composed of the spatiotemporal interaction module, the endpoint prediction module, and the trajectory prediction module. After multiple optimization iterations, the optimal pedestrian trajectory prediction model is obtained.

[0101] Further, step S4 includes the following steps:

[0102] Step S41, construct the loss calculation module;

[0103] Step S42, in the loss calculation module, using the target pedestrian The actual trajectory location at a future moment, based on the target pedestrian The trajectory position at future moments of the spatiotemporal interaction characteristics, through The norm is used to calculate the final loss value. :

[0104]

[0105] in, This represents the trajectory position loss value, which can be taken as the value based on the target pedestrian, excluding the endpoint position. The future trajectory location of the spatiotemporal interaction features and the corresponding target pedestrian The actual trajectory location at a future moment Norm, This represents the loss value at the endpoint location, which can be taken as the value based on the target pedestrian. The future endpoint location of the spatiotemporal interaction features and the target pedestrian The true endpoint of a future moment Norm, This is used to balance the impact of the two loss values ​​on model performance.

[0106] Step S43, using the final loss value The pedestrian trajectory prediction model, which consists of the spatiotemporal interaction module, the endpoint prediction module, and the trajectory prediction module, is optimized by backpropagation. After multiple iterations, the optimal pedestrian trajectory prediction model is obtained.

[0107] The method of iteratively optimizing the model based on the loss value is a technique that should be mastered by those skilled in the art, and will not be described in detail in this invention.

[0108] Step S5: During the testing phase, the optimal pedestrian trajectory prediction model is used to make a one-time prediction based on the observed trajectory of the test target pedestrian to obtain the complete trajectory of the test target pedestrian at future times. The complete trajectory of the test target pedestrian at future times includes the trajectory positions of all future times based on the spatiotemporal interaction characteristics of the test target pedestrian and the endpoint position of the future times based on the spatiotemporal interaction characteristics of the test target pedestrian.

[0109] Further, step S5 includes the following steps:

[0110] Step S51: In the testing phase, the observation trajectories of the test target pedestrian and its neighboring pedestrians are embedded to obtain the embedding features based on the observation trajectory of the test target pedestrian and the embedding features based on the observation trajectories of the test target pedestrian and its neighboring pedestrians. After merging, the embedding features are input into the spatiotemporal interaction module of the optimal pedestrian trajectory prediction model to obtain the spatiotemporal interaction features based on the test target pedestrian.

[0111] Step S52: Concatenate the spatiotemporal interaction features based on the test target pedestrian and the trained endpoint token to obtain the input trajectory features based on the test target pedestrian, and input them into the endpoint prediction module of the optimal pedestrian trajectory prediction model to obtain the endpoint position features based on the input trajectory features of the test target pedestrian.

[0112] Step S53: Input the endpoint position features based on the input trajectory features of the test target pedestrian into the endpoint position regression head to obtain multiple predicted endpoint positions, and perform embedding processing on each predicted endpoint position to obtain multiple embedded features based on the predicted endpoint positions.

[0113] In one embodiment of the present invention, a complete trajectory of the test target pedestrian includes the trajectory positions at all future moments based on the spatiotemporal interaction characteristics of the test target pedestrian and the endpoint positions at future moments based on the spatiotemporal interaction characteristics of the test target pedestrian. To simulate the multimodal characteristics of pedestrian motion, the present invention uses multiple predicted endpoint positions to predict multiple complete trajectories of the test target pedestrian; that is, one predicted endpoint position is used to predict one complete trajectory of the test target pedestrian, two predicted endpoint positions are used to predict two complete trajectories of the test target pedestrian, and so on.

[0114] Step S54: For each embedding feature based on the predicted endpoint location, the spatiotemporal interaction feature based on the test target pedestrian, the location token of the trained future trajectory, and the embedding feature based on the predicted endpoint location are concatenated in sequence to obtain the trajectory feature based on the test target pedestrian corresponding to the embedding feature based on the predicted endpoint location.

[0115] In this way, multiple embedded features based on the predicted endpoint location correspond to multiple trajectory features based on the test target pedestrian.

[0116] Step S55: Input the trajectory features based on the test target pedestrian into the trajectory prediction module of the optimal pedestrian trajectory prediction model to predict the trajectory positions of all future moments at once. This results in multiple sets of future trajectory positions and future endpoint positions based on the spatiotemporal interaction features of the test target pedestrian. Each set of future trajectory positions and future endpoint positions based on the spatiotemporal interaction features of the test target pedestrian constitutes a complete future trajectory of the test target pedestrian.

[0117] Experimental results show that the trajectory prediction method based on endpoint-guided cyclic learning and one-time prediction in this invention achieves ADE / FDE ratios of 0.17 / 0.27 on the ETH / UCY dataset, 6.71 / 10.25 on the SDD dataset, and 0.65 / 0.82 on the NBA dataset. This demonstrates that the method effectively integrates the advantages of cyclic learning and one-time prediction, efficiently capturing the target pedestrian. The spatiotemporal interaction characteristics significantly improve the performance of pedestrian trajectory prediction.

[0118] Figures 2A-2F The above describes a visualization result of pedestrian trajectory prediction on the ETH / UCY dataset obtained according to an embodiment of the present invention, wherein... Figures 2A-2C This is a scene image. Figures 2D-2F The image shows the predicted trajectory of the target pedestrian, with circles representing the observed trajectory and pentagrams representing the predicted trajectory.

[0119] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A trajectory prediction method based on end-to-end guided cycle learning and one-shot prediction, characterized in that, The method comprises the following steps: Step S1, constructing a space-time interaction module, inputting the observed trajectory of the target pedestrian and the observed trajectory of the neighbor pedestrian into the space-time interaction module, obtaining a space-time interaction feature based on the target pedestrian ; Step S2, constructing an endpoint prediction module, inputting the target pedestrian based spatiotemporal interaction features and learnable endpoint tokens into the endpoint prediction module, and obtaining a predicted endpoint location by means of an endpoint location regression head; Step S3, constructing a trajectory prediction module, splicing the spatiotemporal interaction features of the target pedestrian , the position tokens of the plurality of learnable future trajectories, and the embedding features corresponding to the predicted end position, inputting the trajectory prediction module, and predicting the trajectory position of the future moment based on the spatiotemporal interaction features of the target pedestrian ; Step S4, a loss calculation module is constructed, in which the target pedestrian The real trajectory position at the future moment and the trajectory position at the future moment calculated based on the spatio-temporal interaction features of the target pedestrian obtained in step S3 are used to calculate a final loss value, and the pedestrian trajectory prediction model composed of the spatio-temporal interaction module, the end point prediction module and the trajectory prediction module is optimized by back propagation using the final loss value. After multiple optimization iterations, an optimal pedestrian trajectory prediction model is obtained. Step S5, in the test stage, a one-time prediction is performed based on the observation trajectory of the test target pedestrian by using the optimal pedestrian trajectory prediction model, to obtain a complete trajectory of the test target pedestrian at a future time, wherein the complete trajectory of the test target pedestrian at the future time comprises trajectory positions of all future times based on the spatiotemporal interaction features of the test target pedestrian and an end position at the future time based on the spatiotemporal interaction features of the test target pedestrian; The step S1 comprises the following steps: Step S11, inputting the observed trajectory of the target pedestrian and the observed trajectory of the neighbor pedestrian of the target pedestrian into an embedding layer to obtain an embedding feature based on the observed trajectory of the target pedestrian and an embedding feature based on the observed trajectory of the neighbor pedestrian. the embedding feature based on the observed trajectory of the target pedestrian and the embedding feature based on the observed trajectory of the neighbor pedestrian.​ Step S12, merging the embedding features based on the observation trajectory of the target pedestrian and the embedding features based on the observation trajectory of the neighbor pedestrian, to obtain the merged embedding features based on the target pedestrian and the neighbor pedestrian Step S13, determining the target pedestrian based on the merged embedding features Step S14, determining the target pedestrian based on the merged embedding features : Step S13, constructing a space-time interaction module, inputting the merged embedding features of the target pedestrian and the neighbor pedestrians of the target pedestrian into the space-time interaction module to obtain space-time interaction features of the target pedestrian. Step S13, constructing a space-time interaction module, inputting the merged embedding features of the target pedestrian and the neighbor pedestrians of the target pedestrian into the space-time interaction module to obtain space-time interaction features of the target pedestrian.​​ 2. The method of claim 1, wherein, The space-time interaction module comprises a time feature extraction layer, a space feature extraction layer and a space-time interaction layer, wherein the time feature extraction layer is configured to extract time features of the target pedestrian , the space feature extraction layer is configured to extract space features of the target pedestrian , and the space-time interaction layer is composed of multiple Transformers and is configured to extract space-time interaction features based on the target pedestrian .

3. The method of claim 2, wherein, The step S13 comprises the following steps: Step S131, input the observed trajectory of the target pedestrian and its neighbor pedestrians into the time feature extraction layer of the spatio-temporal interaction module, obtain a time attention mask matrix based on the target pedestrian based on the time proximity relationship; Step S132, input the observation trajectory of the target pedestrian and its neighbor pedestrians into the spatial feature extraction layer of the spatio-temporal interaction module, obtain the spatial attention mask matrix based on the target pedestrian based on the spatial relationship ; Step S133, obtain a spatio-temporal attention mask matrix based on the target pedestrian by point multiplication with the time attention mask matrix based on the target pedestrian and the space attention mask matrix based on the target pedestrian. ;​​​​​ Step S134, based on the target pedestrian Spatiotemporal attention mask matrix and the target pedestrian Merged embedding features of pedestrians and their neighbors Inputting the spatiotemporal interaction layer of the spatiotemporal interaction module yields results based on the target pedestrian. spatiotemporal interaction characteristics .

4. The method of claim 1, wherein, The step S2 comprises the following steps: Step S21, based on the target pedestrian The spatiotemporal interaction features and learnable endpoint tokens are concatenated to obtain a target pedestrian-based result. Input trajectory features; Step S22, constructing a destination prediction module, using the destination prediction module to extract features from the input trajectory features of the target pedestrian based trajectory features to obtain destination position features based on the input trajectory features; Step S23, input the end position feature based on the input trajectory feature into the end position regression head to obtain a plurality of predicted end positions. In the training stage, use the filter to screen out the predicted end position closest to the real end position of the target pedestrian .

5. The method of claim 1, wherein, The step S3 comprises the following steps: Step S31, performing embedding processing on the predicted end position to obtain an embedding feature corresponding to the predicted end position, so as to have the same dimension as the spatiotemporal interaction feature of the target pedestrian and the position token of the learnable future trajectory. the predicted end position to obtain an embedding feature corresponding to the predicted end position, so as to have the same dimension as the spatiotemporal interaction feature of the target pedestrian and the position token of the learnable future trajectory. Step S32, performing cyclic splicing on the spatiotemporal interaction feature of the target pedestrian , the position token of the plurality of learnable future trajectories, and the embedding feature corresponding to the predicted end position to obtain a trajectory feature of a future time point of the target pedestrian except the end point. Step S33, constructing a trajectory prediction module, inputting the trajectory features of the future moment of the target pedestrian except the end point into the trajectory prediction module for loop prediction, obtaining the trajectory position of the future moment based on the spatiotemporal interaction features of the target pedestrian ​ 6. The method of claim 5, wherein, In the step S32, when the first cycle splicing is performed, the splicing is performed on the space-time interaction feature of the target pedestrian , the position token of the first learnable future trajectory, and the embedding feature corresponding to the predicted end position, to obtain the trajectory feature of the first future moment of the target pedestrian . During the second iteration of stitching, the stitching is based on the target pedestrian. The spatiotemporal interaction features, the location tokens of the first two learnable future trajectories, and the embedding features corresponding to the predicted endpoint position are used to obtain a target pedestrian-based... The trajectory characteristics of the first two future moments; and so on, the trajectory characteristics of the first two future moments. In the next iteration of the stitching process, the stitching is based on the target pedestrian. The spatiotemporal interaction characteristics, all A location token that can learn future trajectories and the embedded features corresponding to the predicted endpoint location are used to obtain a target pedestrian-based... All Trajectory characteristics at a future moment, excluding the endpoint.

7. The method of claim 6, wherein, The input of the n-th loop prediction in the step S33 is the output of the n-th loop splicing in the step S32, and the output is the trajectory position of the first n future time points based on the spatio-temporal interaction feature of the target pedestrian , and the end position of the future time point based on the spatio-temporal interaction feature of the target pedestrian predicted by the n-th prediction. The trajectory position of the future time point based on the spatio-temporal interaction feature of the target pedestrian is composed of the trajectory position of all future time points based on the spatio-temporal interaction feature of the target pedestrian , and the end position of the future time point based on the spatio-temporal interaction feature of the target pedestrian predicted by the n-th prediction. ​ 8. The method of claim 1, wherein, The step S4 comprises the following steps: Step S41, a loss calculation module is constructed; Step S42, in the loss calculation module, the target pedestrian real trajectory position at future moment, the trajectory position at future moment based on the spatio-temporal interaction feature of the target pedestrian is calculated by norm to obtain the final loss value ; Step S43, using the final loss value The pedestrian trajectory prediction model composed of the space-time interaction module, the end point prediction module and the trajectory prediction module is optimized by back propagation, and after multiple iterations, an optimal pedestrian trajectory prediction model is obtained.

9. The method of claim 1, wherein, The step S5 comprises the following steps: Step S51, in the test stage, the observation trajectories of the test target pedestrian and its neighbor pedestrians are embedded to obtain embedded features based on the observation trajectory of the test target pedestrian and embedded features based on the observation trajectories of the neighbor pedestrians of the test target pedestrian, which are combined and input into a spatiotemporal interaction module of the optimal pedestrian trajectory prediction model to obtain spatiotemporal interaction features based on the test target pedestrian; Step S52, the spatiotemporal interaction features based on the test target pedestrian and the end token obtained by training are spliced to obtain input trajectory features based on the test target pedestrian, which are input into an end prediction module of the optimal pedestrian trajectory prediction model to obtain end position features of the input trajectory features based on the test target pedestrian; Step S53, the end position features of the input trajectory features based on the test target pedestrian are input into an end position regression head to obtain a plurality of predicted end positions, and each predicted end position is embedded to obtain a plurality of embedded features based on the predicted end positions; Step S54, for each embedded feature based on the predicted end position, the spatiotemporal interaction features based on the test target pedestrian, the position token of the future trajectory obtained by training, and the embedded feature based on the predicted end position are spliced in sequence to obtain trajectory features based on the test target pedestrian corresponding to the embedded feature based on the predicted end position; Step S55, the trajectory features based on the test target pedestrian are input into a trajectory prediction module of the optimal pedestrian trajectory prediction model to perform one-time prediction to obtain trajectory positions of all future times, a plurality of groups of trajectory positions of future times based on the spatiotemporal interaction features of the test target pedestrian and end positions of future times based on the spatiotemporal interaction features of the test target pedestrian are obtained, and each group of trajectory positions of future times based on the spatiotemporal interaction features of the test target pedestrian and end positions of future times based on the spatiotemporal interaction features of the test target pedestrian constitutes a complete trajectory of the test target pedestrian at a future time.

Images