A method, apparatus, device, and storage medium for predicting pedestrian trajectories.

Abstract

This invention discloses a method, apparatus, device, and storage medium for predicting pedestrian trajectories, relating to the field of computer technology. The solution first determines each sample data point and its corresponding future true trajectory. Then, for each sample data point, a pedestrian trajectory prediction model to be trained is used to determine the first predicted trajectory corresponding to that sample data. Next, the redundant error corresponding to that sample data is determined to determine the second predicted trajectory corresponding to that sample data after removing the redundant error. Finally, the pedestrian trajectory prediction model to be trained is trained with the optimization objective of minimizing the deviation between the second predicted trajectory of each sample data point and the future true trajectory. The trained pedestrian trajectory prediction model is then used to predict pedestrian trajectories. This solution improves the accuracy of model training by eliminating the redundant error introduced by random variables in the pedestrian trajectory prediction model, thereby improving the accuracy of the trained pedestrian trajectory prediction model in predicting pedestrian trajectories.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method, apparatus, device, and storage medium for predicting pedestrian trajectories. Background Technology

[0002] Currently, with the development of autonomous driving technology, one of the key issues that autonomous driving equipment needs to consider during operation is how to detect pedestrians entering traffic lanes as early as possible and identify their movement intentions and trajectories in urban traffic environments. According to the World Health Organization's "2018 Global Status Report on Road Safety," the number of deaths due to traffic accidents has increased by 1.35 million per year, meaning that on average, someone loses their life on the road every 24 seconds, with approximately 23% of these deaths involving pedestrians. The report also emphasizes that road traffic injuries are now a leading cause of death among children and young adults aged 5 to 29. Improving vehicle driving safety and reducing the number of road traffic accidents is one of the important issues that needs to be addressed. Only by accurately predicting the direction of pedestrian movement and promptly avoiding potential pedestrians can the probability of collisions between pedestrians and vehicles be reduced.

[0003] Existing technologies obtain historical trajectory features by modeling historical trajectories in the time domain and interaction features by modeling them in the spatial domain. Latent vectors are then sampled from a normal distribution to represent the multimodality of pedestrian motion, thereby predicting future trajectories based on trajectory features, interaction features, and latent vectors. However, the latent vectors themselves are unrelated to the historical trajectory features and interaction features of pedestrians. Introducing latent vectors inevitably interferes with the expression of trajectory and interaction features, introducing unnecessary redundant errors during trajectory prediction in subsequent training. This reduces the accuracy of model training, resulting in lower accuracy in predicting pedestrian trajectories after training. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, device, and storage medium for predicting pedestrian trajectories to address the aforementioned technical problems.

[0005] The following technical solution is adopted in this specification:

[0006] This specification provides a method for predicting pedestrian trajectories, including:

[0007] Obtain a pedestrian historical trajectory dataset, segment each pedestrian historical trajectory data in the dataset according to a preset time, use the historical trajectory data before the preset time as sample data, and label the historical trajectory data after the preset time as the future real trajectory corresponding to the sample data;

[0008] For each sample data, the pedestrian's historical trajectory features, pedestrian interaction features, and random variables are input into the pedestrian trajectory prediction model to be trained to determine the first predicted trajectory of the sample data; wherein, the pedestrian trajectory prediction model is a neural network model that incorporates random variables to simulate the randomness of pedestrian behavior in order to predict pedestrian trajectories.

[0009] The redundancy error of the sample data is determined based on the random variables corresponding to the sample data and the predetermined optimization features; wherein, the optimization features are obtained by multiplying the concatenated matrix of historical trajectory features and interaction features with the optimization factor;

[0010] The second predicted trajectory of the sample data is determined based on the difference between the first predicted trajectory and the redundancy error of the sample data.

[0011] The pedestrian trajectory prediction model is trained with the goal of minimizing the deviation between the second predicted trajectory of the corresponding sample data and the labeled future true trajectory. The trained pedestrian trajectory prediction model is then used to predict pedestrian trajectories.

[0012] Optionally, the determination of the pedestrian's historical trajectory features and the pedestrian's interaction features specifically includes:

[0013] The sample data is input into the encoder of the pedestrian trajectory prediction model to be trained to obtain the historical trajectory features of the pedestrians corresponding to the sample data.

[0014] The interaction between pedestrians is modeled based on their historical trajectory characteristics to determine the interaction characteristics of pedestrians corresponding to the sample data.

[0015] Optionally, the optimization factor is not less than zero and less than one, and the optimization factor is negatively correlated with the degree of redundant error constructed by the optimization feature on the random variable.

[0016] Optionally, the optimization factor is zero.

[0017] Optionally, the step of training the pedestrian trajectory prediction model to be trained with the optimization objective of minimizing the deviation between the second predicted trajectory of the corresponding sample data and the labeled future true trajectory specifically includes:

[0018] Based on the sample data, the historical displacement of the pedestrian corresponding to the sample data is determined, so as to determine the mask factor corresponding to the sample data. The mask factor is used to characterize whether the sample data is used to determine the deviation between the actual trajectory label and the future trajectory label.

[0019] Determine the gap between the second predicted trajectory corresponding to the sample data and the labeled future true trajectory, and determine the deviation corresponding to the sample data based on the product of the gap and the mask factor corresponding to the sample data;

[0020] The pedestrian trajectory prediction model to be trained is trained with the optimization objective of minimizing the sum of the deviations of each sample data.

[0021] Optionally, determining the mask factor corresponding to the sample data specifically includes:

[0022] Based on the sample data, determine whether the pedestrians corresponding to the sample data have historical displacements;

[0023] If so, then the mask factor corresponding to the sample data is set to one;

[0024] If not, then the masking factor corresponding to the sample data is determined to be zero.

[0025] Optionally, determining whether the pedestrian corresponding to the sample data has a historical displacement based on the sample data specifically includes:

[0026] Based on the sample data, determine the positional changes of pedestrians at adjacent historical moments corresponding to the sample data;

[0027] The average speed of pedestrians at each adjacent historical moment is determined based on the changes in their positions at adjacent historical moments.

[0028] Determine whether the average speed of pedestrians at each adjacent historical moment is greater than zero;

[0029] If so, then it is determined that the pedestrians corresponding to the sample data have historical displacements;

[0030] If not, then it is determined that the pedestrian corresponding to the sample data has no historical displacement.

[0031] This specification provides a pedestrian trajectory prediction device, including:

[0032] The acquisition module is used to acquire the pedestrian historical trajectory dataset, segment each pedestrian historical trajectory data in the dataset according to a preset time, use the historical trajectory data before the preset time as sample data, and label the historical trajectory data after the preset time as the future real trajectory corresponding to the sample data.

[0033] The prediction module is used to input the pedestrian's historical trajectory features, pedestrian interaction features, and random variables into the pedestrian trajectory prediction model to be trained for each sample data to determine the first predicted trajectory of the sample data; wherein, the pedestrian trajectory prediction model is a neural network model that incorporates random variables to simulate the randomness of pedestrian behavior in order to predict pedestrian trajectories.

[0034] The error construction module is used to determine the redundancy error of the sample data based on the random variables corresponding to the sample data and the pre-determined optimization features; wherein, the optimization features are obtained by multiplying the concatenated matrix of historical trajectory features and interaction features with the optimization factor;

[0035] The error removal module is used to determine the second predicted trajectory of the sample data based on the difference between the first predicted trajectory and the redundancy error of the sample data.

[0036] The training module is used to train the pedestrian trajectory prediction model to be trained with the optimization objective of minimizing the deviation between the second predicted trajectory of the corresponding sample data and the labeled future true trajectory, so as to perform pedestrian trajectory prediction through the trained pedestrian trajectory prediction model.

[0037] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described pedestrian trajectory prediction method.

[0038] This specification provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described pedestrian trajectory prediction method.

[0039] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0040] This scheme first determines each sample data point and its corresponding future true trajectory based on the pedestrian historical trajectory dataset. Then, for each sample data point, the pedestrian trajectory prediction model to be trained determines the first predicted trajectory. Next, based on pre-determined optimization features and random variables, the redundancy error corresponding to that sample data is determined. Based on the first predicted trajectory and the redundancy error, the second predicted trajectory corresponding to that sample data is determined. Finally, the pedestrian trajectory prediction model to be trained is trained with the optimization objective of minimizing the deviation between the second predicted trajectory and the future true trajectory for each sample data point. The trained pedestrian trajectory prediction model is then used to predict pedestrian trajectories. This scheme improves the accuracy of model training by eliminating the redundancy error introduced by the random variables in the pedestrian trajectory prediction model, thereby improving the accuracy of the trained pedestrian trajectory prediction model in predicting pedestrian trajectories. Attached Figure Description

[0041] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0042] Figure 1a This is a schematic diagram of an existing pedestrian trajectory prediction model provided in this specification;

[0043] Figure 1b This specification provides a schematic diagram for simultaneously predicting the trajectory of pedestrians with and without historical trajectories.

[0044] Figure 1c This specification provides a schematic diagram for trajectory prediction of pedestrians with historical trajectories only;

[0045] Figure 2 This is a flowchart illustrating a pedestrian trajectory prediction method provided in this specification;

[0046] Figure 3 This is a schematic diagram of a pedestrian trajectory prediction process provided in this specification;

[0047] Figure 4a This manual provides a comparative diagram showing the relationship between the amount of pedestrian trajectory prediction data and learning efficiency under different conditions, using ADE as an indicator;

[0048] Figure 4b This manual provides a comparative diagram showing the relationship between the amount of pedestrian trajectory prediction data and learning efficiency under different conditions, using FDE as an indicator;

[0049] Figure 5 This is a schematic diagram of a pedestrian trajectory prediction device provided in this specification;

[0050] Figure 6 This is a schematic diagram of a computer device for implementing a pedestrian trajectory prediction method, as provided in this specification. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0052] Currently, mature pedestrian trajectory prediction models typically incorporate random vectors composed of random variables to simulate the randomness of pedestrian behavior, thereby establishing a multimodal model of pedestrian movement. However, the introduced random variables themselves are not correlated with the trajectory features and interaction features obtained from the sample data. The introduction of random variables inevitably affects the expression of trajectory features and interaction features when predicting pedestrian trajectories, thus introducing redundant errors into subsequent trajectory prediction.

[0053] On the other hand, current pedestrian trajectory prediction models do not consider the irrelevance of pedestrians with zero historical trajectories to the modeling of pedestrian movement patterns. When training pedestrian trajectory prediction models, there may be individual samples in the sample data corresponding to pedestrians who have not moved at any historical time, i.e., they have always stood in place and have no historical trajectory. Since it is impossible to model the movement patterns of such people based on existing historical data, trajectory prediction for such pedestrians often has a high degree of randomness. When training the pedestrian trajectory prediction model based on the deviation between the predicted trajectory and the labeled trajectory of such people, it will reduce the model's learning effect. Naturally, this prevents the network model from maximizing the absorption of training data. Only by more accurately modeling pedestrian movement patterns and accurately capturing the correspondence between historical and future trajectories can autonomous vehicles drive more safely and thus be widely used in cities.

[0054] Figure 1a This is a schematic diagram of an existing pedestrian trajectory prediction model provided in this specification. Figure 1a As shown, existing pedestrian trajectory prediction models obtain historical trajectory features by modeling the time domain of historical trajectories, obtain interaction features by modeling the spatial domain of historical trajectories, and sample latent vectors from the normal distribution to represent the multimodality of pedestrian movement. Thus, they predict future trajectories based on trajectory features, interaction features, and latent vectors.

[0055] Figure 1b This specification provides a schematic diagram for simultaneously predicting the trajectory of pedestrians with and without historical trajectories. Figure 1c This is a schematic diagram illustrating a method for predicting the trajectory of pedestrians with historical tracks, as provided in this specification. Figure 1b neutralization Figure 1c The left side shows the observed historical trajectories of pedestrians. Through continuous model training, the final movement patterns of pedestrians are modeled, thereby predicting the future trajectories of pedestrians on the right. Among these, in... Figure 1b In one section, the modeling of pedestrian movement patterns is demonstrated simultaneously for pedestrians with and without historical trajectory information. Figure 1c The section demonstrates modeling the movement patterns of pedestrians with historical trajectory information only.

[0056] Figure 1b The example demonstrates a scene containing pedestrians with historical trajectory information and pedestrians without any historical trajectory (but potentially with future trajectories). The movement patterns of these two types of pedestrians are modeled to predict future trajectories. However, for pedestrians without any historical trajectory, their movement patterns are untraceable. Figure 1bThe right half of the diagram illustrates that the future trajectories of pedestrians without historical trajectory information are highly probabilistic and have no correlation with their historical trajectories. In other words, it's impossible to model the movement patterns of this type of pedestrian based on existing historical data. Therefore, if the network model is iteratively updated during the training phase for both types of pedestrian movement patterns, it will be unable to fully learn the correspondence between historical and future trajectories. However, in complex scenarios, the practical application of advanced autonomous driving often necessitates avoiding even the slightest possibility of accidents, especially the prediction of pedestrian trajectories in traffic scenarios, which is crucial for traffic safety.

[0057] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0058] Figure 2 This is a flowchart illustrating a pedestrian trajectory prediction method described in this specification, which specifically includes the following steps:

[0059] S101: Obtain the pedestrian historical trajectory dataset, segment each pedestrian historical trajectory data in the dataset according to a preset time, use the historical trajectory data before the preset time as sample data, and label the historical trajectory data after the preset time as the future real trajectory corresponding to the sample data.

[0060] Generally, when training a pedestrian trajectory model, the server of a business platform can first obtain a historical pedestrian trajectory dataset collected by a data collector, or it can obtain a historical pedestrian trajectory dataset from a third-party data platform. This manual does not restrict how to obtain the dataset.

[0061] After obtaining the pedestrian historical trajectory dataset, the server can use a certain moment in the dataset as the segmentation moment, take the historical trajectory data before the segmentation moment as the sample data, and take the historical trajectory data after the segmentation moment as the future real trajectory label corresponding to the sample data.

[0062] Taking pedestrian i as an example, let's assume that pedestrian i's historical trajectory dataset is from time t1 to time t. pred Up to this point, if the time of division is time t obs Then the server can display the times t1, t2, ..., t obs The trajectory data within the time frame is used as sample data, and then time t is... obs+1 , t obs+2 , ..., t predThe trajectory data within the sample is used as the label corresponding to pedestrian i's sample data. Subsequently, the pedestrian trajectory prediction model can be used to predict the trajectory by using the sample data as the input of the historical trajectory. The predicted trajectory is then trained based on the deviation between the predicted trajectory and the future real trajectory label, i.e., the real trajectory.

[0063] Therefore, the sample data to be input into the model as historical trajectories can be represented as follows:

[0064]

[0065] Among them, X i The sample data corresponding to pedestrian i consists of a series of two-dimensional coordinate points with time information, that is, from time t1 to time t obs The coordinates of each time point within the sample data are used. Similarly, the future true trajectory annotation, which serves as the sample data, can be represented as:

[0066]

[0067] Among them, Y i The annotation of the sample data corresponding to pedestrian i is based on time t. obs +1 until time t pred The coordinates of each time point within the time frame.

[0068] The specific representation of coordinate points can be determined as needed. It can be coordinate data based on the coordinate system of the data acquisition device, or data in other coordinate systems, or the relative position of a pedestrian, that is, using historical displacement instead of absolute position coordinates. This manual does not impose any restrictions on this.

[0069] The server mentioned in this manual can be a server set up on a business platform, or a device such as a desktop computer or laptop computer capable of executing the solution described in this manual. For ease of explanation, the following description will only focus on the server as the execution subject.

[0070] S102: For each sample data, the pedestrian's historical trajectory features, pedestrian interaction features, and random variables are input into the pedestrian trajectory prediction model to be trained to determine the first predicted trajectory of the sample data. The pedestrian trajectory prediction model is a neural network model that incorporates random variables to simulate the randomness of pedestrian behavior in order to predict pedestrian trajectories.

[0071] After determining each sample data and the corresponding future true trajectory label, the server can input the sample data into the pedestrian trajectory prediction model. Through the pedestrian trajectory prediction model, the historical trajectory characteristics of each sample data and the pedestrian interaction characteristics corresponding to each sample data are determined. Based on the historical trajectory characteristics, interaction characteristics, and introduced random variables, the first predicted trajectory corresponding to each sample data is determined.

[0072] The pedestrian trajectory prediction model mentioned herein can employ a neural network model that incorporates random variables to simulate the randomness of pedestrian behavior in order to predict pedestrian trajectories; this specification does not impose any limitations on this. For example, it could be the STGAT model based on RNN and GAT, or the SGAN model based on RNN and GAN, etc.

[0073] Specifically, the server can input sample data into the encoder of the pedestrian trajectory prediction model to be trained, obtain the historical trajectory features of the pedestrians corresponding to the sample data, and model the interaction between each pedestrian based on the historical trajectory features of the pedestrians through the pedestrian trajectory prediction model to be trained, thereby determining the interaction features of the pedestrians corresponding to the sample data.

[0074] Taking SGAN as an example, the server can input sample data into the SGAN model to train the generator. The SGAN generator is based on an encoder-decoder architecture. The encoder maps sample data to historical trajectory features. The SGAN model then calculates the positions of pedestrians and other pedestrians in the scene, encodes these positions using a multilayer perceptron (MLP), and stacks and fuses them with the hidden layer features output from the generator's encoder for each pedestrian. The fused features are then processed through an MLP and max pooling operation to perform feature summarization, ultimately outputting interaction features that characterize pedestrian interactions. Finally, for each sample data point, the historical trajectory features, interaction features, and random variables are used as input to the decoder to predict future trajectories. The specific process is already well-established and will not be elaborated upon here.

[0075] The first predicted trajectory for each sample data can be determined by the following formula:

[0076] h i =Decoder(m i ||g i ||z;W d )

[0077] Among them, h i Let m be the first predicted trajectory of the sample data corresponding to pedestrian i, and let m be the decoder of the pedestrian trajectory prediction model. i To correspond to the trajectory features of pedestrian i, g i Let z represent the interaction features corresponding to pedestrian i, z be a random variable introduced into the pedestrian trajectory prediction model, and || denote the concatenation operation. W d These are the weight parameters of the decoder in the pedestrian trajectory prediction model.

[0078] S103: Determine the redundancy error of the sample data based on the random variables corresponding to the sample data and the predetermined optimization features. The optimization features are obtained by multiplying the concatenated features of historical trajectory features and interaction features with the optimization factor.

[0079] As mentioned above, existing pedestrian trajectory prediction models typically incorporate random vectors composed of random variables to simulate the randomness of pedestrian behavior, thereby establishing a multimodal model of pedestrian movement. However, the introduced random variables themselves are not correlated with the historical trajectory features and interaction features obtained from the sample data. The introduction of random variables inevitably affects the expression of historical trajectory features and interaction features when predicting pedestrian trajectories, thus introducing redundant errors into subsequent trajectory prediction.

[0080] Therefore, after determining the first predicted trajectory corresponding to each sample data through the pedestrian trajectory prediction model, the server can further determine the redundant error caused by the random variable in order to eliminate the redundant error.

[0081] Specifically, for each sample data point, the server can concatenate its historical trajectory features and interaction features to obtain a concatenated feature. Then, based on the product of a preset optimization factor and this concatenated feature, the server determines the optimized feature for that sample data. This optimized feature, along with random variables, is then used by a pedestrian trajectory prediction model to determine the redundancy error corresponding to the sample data.

[0082] The optimization factor can be any preset value within the interval [0, 1). This optimization factor is negatively correlated with the degree of redundancy error constructed by the optimization feature on the random variable. Preferably, the optimization factor can be preset to zero, and the redundancy errors corresponding to each sample data can be determined by the following formula:

[0083] h i =Decoder(OF||z;W) d )

[0084] Among them, h i ' represents the redundancy error of the sample data corresponding to pedestrian i, Decoder is the decoder of the pedestrian trajectory prediction model, OF is the concatenation feature corresponding to pedestrian i, z is the random variable introduced by the pedestrian trajectory prediction model, and || represents the concatenation operation. W d These are the weight parameters of the decoder in the pedestrian trajectory prediction model.

[0085] S104: Determine the second predicted trajectory of the sample data based on the difference between the first predicted trajectory and the redundancy error.

[0086] S105: The pedestrian trajectory prediction model to be trained is trained with the optimization objective of minimizing the deviation between the second predicted trajectory of the corresponding sample data and the labeled future true trajectory, so as to predict pedestrian trajectories through the trained pedestrian trajectory prediction model.

[0087] After determining the redundancy errors corresponding to each sample data as described above, the server can determine the second prediction trajectory corresponding to each sample data based on the first prediction trajectory and the redundancy error of that sample data. The second prediction trajectory is the prediction trajectory after eliminating the redundancy error.

[0088] Specifically, the server can determine the second predicted trajectory corresponding to the sample data based on the difference between the first predicted trajectory and the redundancy error. The second predicted trajectory for each sample data can be determined using the following formula:

[0089] Y i =Output(h i -h i ′;W o )

[0090] Among them, Y i The second predicted trajectory is the sample data corresponding to pedestrian i, and Output is the output module of the pedestrian trajectory prediction model. i For the first predicted trajectory of the sample data corresponding to pedestrian i, h i 'W represents the redundancy error of the sample data corresponding to pedestrian i. o These are the weight parameters for the output module of the pedestrian trajectory prediction model.

[0091] Finally, for each sample data point, the server can determine the bias loss based on the second predicted trajectory and the future true trajectory annotations for that sample data, and train the pedestrian trajectory prediction model with the optimization objective of minimizing the bias of each sample data point. The specific loss setting can be determined as needed, and this manual does not impose any restrictions on it. After training the pedestrian trajectory prediction model to be trained through multiple iterations, the server can use the trained pedestrian trajectory prediction model to predict pedestrian trajectories.

[0092] based on Figure 2The pedestrian trajectory prediction method described here first determines each sample data and its corresponding future true trajectory based on the historical pedestrian trajectory dataset. Then, for each sample data, the pedestrian trajectory prediction model to be trained determines the first predicted trajectory for that sample data. Next, based on pre-determined optimization features and random variables, the redundancy error corresponding to that sample data is determined. Based on the first predicted trajectory and the redundancy error, the second predicted trajectory for that sample data is then determined. Finally, the pedestrian trajectory prediction model to be trained is trained with the optimization objective of minimizing the deviation between the second predicted trajectory and the future true trajectory for each sample data. The trained pedestrian trajectory prediction model is then used for pedestrian trajectory prediction. This method improves the accuracy of model training by eliminating the redundancy error introduced by the random variables in the pedestrian trajectory prediction model, thereby improving the prediction accuracy of the trained pedestrian trajectory prediction model.

[0093] When applying the pedestrian trajectory prediction method provided in this manual, it is not necessary to rely on... Figure 2 The steps shown are executed in sequence. The specific execution order of each step can be determined as needed, and this manual does not impose any restrictions on it.

[0094] Furthermore, in one or more embodiments of this specification, in step S105, when the server trains the pedestrian trajectory prediction model with the optimization objective of minimizing the deviation between the second predicted trajectory of each sample data and the corresponding label, there may be individual sample data corresponding to pedestrians who have no displacement at any historical time, that is, they have always stood in place and have no historical trajectory. Since it is impossible to model the movement pattern of such people based on existing historical data, the trajectory prediction for such pedestrians is often highly random. When training the pedestrian trajectory prediction model based on the deviation between the predicted trajectory of such people and the label, the learning effect of the model will be reduced.

[0095] Therefore, the server can first determine the historical displacement of the pedestrian corresponding to each sample data point, thus determining the mask factor for that sample data. This mask factor characterizes whether the sample data is used to determine the deviation from the future true trajectory annotation. Then, the server can determine the difference between the second predicted trajectory corresponding to that sample data and the future true trajectory annotation. The deviation corresponding to that sample data is determined by multiplying this difference by the mask factor. Finally, the pedestrian trajectory prediction model is trained with the optimization objective of minimizing the sum of the deviations of all sample data points.

[0096] Specifically, for each sample data, the server can determine whether the pedestrian corresponding to the sample data has historical displacement. If historical displacement exists, the mask factor corresponding to the sample data is set to one; if no historical displacement exists, the mask factor corresponding to the sample data is set to zero.

[0097] The deviation of each sample data can be expressed by the following formula:

[0098]

[0099] in, To account for the bias in the sample data, The difference between the second predicted trajectory and the labeled trajectory of the sample data obtained from the pedestrian trajectory prediction model. This serves as a mask for the sample data. By masking out sample data corresponding to pedestrians without historical displacement, the pedestrian trajectory prediction model no longer focuses on the deviation between the sample data and the labels for pedestrians without historical displacement during training. This allows the pedestrian trajectory prediction model to maximize the absorption of training data and improve the model's learning performance.

[0100] Furthermore, in one or more embodiments of this specification, when the server determines whether a pedestrian corresponding to each sample data has historical displacement based on the sample data, it can first determine the positional change of the pedestrian corresponding to the sample data in adjacent historical moments based on the sample data. The positional change of the pedestrian corresponding to the sample data in adjacent historical moments can be determined by the following formula:

[0101]

[0102]

[0103] in, In the historical trajectory of the sample data corresponding to pedestrian i, pedestrian i at time t k-1 At time t k lateral position change, For pedestrian i at time t k The horizontal coordinate of the location For pedestrian i at time t k-1 The horizontal coordinate of the location. For pedestrian i at time t k-1 At time t k The change in longitudinal position, For pedestrian i at time t k The vertical coordinate of the location, For pedestrian i at time t k-1 The vertical coordinate of the location.

[0104] Furthermore, the server can determine the pedestrian's average speed at each adjacent historical moment based on location changes. The average speed of the pedestrian at each adjacent historical moment can be determined by the following formula:

[0105]

[0106] in, The average speed of pedestrian i at each adjacent historical moment is given by the formula. The other formulas can be referred to the above corresponding explanations, and will not be repeated here.

[0107] Finally, the server can determine whether the average speed of the pedestrian at each adjacent historical moment is greater than zero. If it is greater than zero, it is determined that the pedestrian corresponding to the sample data has historical displacement. If it is not greater than zero, it is determined that the pedestrian corresponding to the sample data does not have historical displacement.

[0108] Of course, the determination of whether the value is greater than zero can be made according to a preset precision. Furthermore, to determine whether a pedestrian corresponding to the sample data has historical displacement, one can directly accumulate the pedestrian's position changes and take the average; this can directly determine whether the pedestrian has historical displacement based on the position changes. There are various methods for determining whether a pedestrian has historical displacement; this manual does not limit how to determine this, and this is merely an example.

[0109] Based on the above, the server uses a mask to block sample data corresponding to pedestrians who do not have historical displacement. The mask corresponding to each sample data can be determined by the following formula:

[0110]

[0111] in, To determine the function for the mask, To determine the average speed of pedestrian i at each adjacent historical moment, the mask corresponding to each sample data can be determined using this mask determination function.

[0112] Figure 3 This is a schematic diagram of a pedestrian trajectory prediction process provided in this specification. Figure 3 The upper left part corresponds to step S102, where the pedestrian trajectory prediction model determines the first predicted trajectory based on the sample data. The lower left part corresponds to step S103, where the pedestrian trajectory prediction model determines the redundancy error based on the optimization features and random variables. The difference between the outputs of the upper and lower left parts determines the second predicted trajectory corresponding to the sample data. Figure 3 The middle part is represented by a minus sign inside a circle, and the last part is... Figure 3 The right side of the middle section represents the sample data corresponding to pedestrians who do not have historical displacement, which are masked to further improve the model's learning effect.

[0113] In addition, to further illustrate this solution, the following embodiments of applying this solution are also provided in this specification.

[0114] To evaluate the training accuracy of this scheme, specifically the accuracy of pedestrian trajectory prediction after training the model using this scheme, a comparative experiment was provided. The selected pedestrian trajectory prediction models were STGAT (a pedestrian trajectory prediction model based on RNN and GAT) and SGAN (a pedestrian trajectory prediction model based on RNN and GAN). To ensure fairness in the comparative experiment, the same hyperparameters were maintained in both the application and non-application scenarios when evaluating the two models. Furthermore, to ensure reproducibility of the experimental results, the same random number seed was set for all evaluation trials. The comparative experimental results are shown in Tables 1 and 2.

[0115] Table 1 Comparison of STGAT Model Before and After Applying This Solution

[0116]

[0117] Table 2 Comparison of SGAN models before and after applying this scheme

[0118]

[0119] Table 1 compares the STGAT model provided in this manual before and after applying this scheme, and Table 2 compares the SGAN model provided in this manual before and after applying this scheme. In error evaluation, both ADE and FDE are better the lower they are. "This Scheme - STGAT" refers to the evaluation results of the trained model after applying this scheme to the STGAT pedestrian trajectory prediction model. As shown in Table 1, compared to the original model, the evaluation index of the model after applying this scheme is lower, indicating that the prediction performance of the model after training with this scheme is improved. Table 2 shows the evaluation results of the trained model after applying this scheme to the SGAN pedestrian trajectory prediction model. Similarly, it can be seen that the evaluation index of the model after applying this scheme is lower, indicating that the prediction performance of the model after training with this scheme is improved. Therefore, from Tables 1 and 2, it can be seen that this scheme can effectively improve the accuracy of model training.

[0120] Furthermore, this manual also explores the relationship between the amount of pedestrian trajectory prediction data and learning efficiency in two scenarios: applying this scheme and not applying this scheme. Specifically, this manual randomly samples different proportions of training data from the historical pedestrian trajectory dataset, namely 5%, 25%, and 50%. The training data is randomly selected and divided, and the same data is used as sample data for model training in both scenarios. The experiments are based on STGAT for comparison; one scenario is the original STGAT pedestrian trajectory prediction model without applying this scheme, and the other scenario is the STGAT pedestrian trajectory prediction model with this scheme applied. The comparison results are as follows: Figure 4a and Figure 4b As shown.

[0121] Figure 4a This manual provides a comparative diagram showing the relationship between the amount of data and learning efficiency in pedestrian trajectory prediction under different conditions, using ADE as the indicator. Figure 4b This document provides a comparative diagram illustrating the relationship between the amount of data and learning efficiency for pedestrian trajectory prediction under different conditions, using FDE as the metric. Figure 4a The paper shows a comparison of the average ADE evaluation index when the model is used to predict pedestrian trajectories after training the model under two different conditions. Figure 4b The paper shows a comparison of the average FDE evaluation index when the model is used to predict pedestrian trajectories after training the model under two different conditions.

[0122] by Figure 4a To illustrate, the horizontal axis in the graph represents the amount of training data, and the vertical axis represents the average ADE evaluation index value. The dashed line represents the model evaluation after the original STGAT pedestrian trajectory prediction was not applied, while the solid line represents the model evaluation after the STGAT pedestrian trajectory prediction model was trained using this method. Figure 4b Similarly, the evaluation index is the average FDE evaluation index.

[0123] Clearly, when using different proportions of training data, the model using this approach, regardless of whether it's based on the ADE or FDE metrics, consistently achieves higher accuracy in pedestrian trajectory prediction than the model without this approach. Furthermore, as shown in Figure 4, with increasing training data volume, the original STGAT pedestrian trajectory prediction model without this approach did not effectively absorb more training data; instead, its learning efficiency fluctuated wildly. This instability is due to interference factors in pedestrian trajectory prediction, preventing the training data from properly guiding the model's learning. This approach eliminates interference during pedestrian trajectory prediction training, improving the accuracy and learning effect of pedestrian trajectory prediction during training. This, in turn, enhances the prediction accuracy of pedestrian trajectories after training.

[0124] The above describes one or more embodiments of a pedestrian trajectory prediction method provided in this specification. Based on the same idea, this specification also provides a corresponding pedestrian trajectory prediction device, such as... Figure 5 As shown.

[0125] Figure 5 This specification provides a schematic diagram of a pedestrian trajectory prediction device, which includes:

[0126] The acquisition module 201 is used to acquire the pedestrian historical trajectory dataset, divide each pedestrian historical trajectory data in the dataset according to a preset time, use the historical trajectory data before the preset time as sample data, and label the historical trajectory data after the preset time as the future real trajectory corresponding to the sample data.

[0127] The prediction module 202 is used to input the pedestrian's historical trajectory features, pedestrian interaction features, and random variables into the pedestrian trajectory prediction model to be trained for each sample data to determine the first predicted trajectory of the sample data; wherein, the pedestrian trajectory prediction model is a neural network model that incorporates random variables to simulate the randomness of pedestrian behavior in order to predict pedestrian trajectories.

[0128] Error construction module 203 is used to determine the redundancy error of sample data based on the random variables corresponding to the sample data and the predetermined optimization features; wherein, the optimization features are obtained by multiplying the concatenation matrix of historical trajectory features and interaction features with the optimization factor;

[0129] Error removal module 204 is used to determine the second predicted trajectory of the sample data based on the difference between the first predicted trajectory of the sample data and the redundancy error.

[0130] Training module 205 is used to train the pedestrian trajectory prediction model to be trained with the optimization objective of minimizing the deviation between the second predicted trajectory of the corresponding sample data and the labeled future true trajectory, so as to perform pedestrian trajectory prediction through the trained pedestrian trajectory prediction model.

[0131] Optionally, the prediction module 202 inputs the sample data into the encoder of the pedestrian trajectory prediction model to be trained, obtains the historical trajectory features of the pedestrians corresponding to the sample data, models the interaction between each pedestrian based on the historical trajectory features of the pedestrians, and determines the interaction features of the pedestrians corresponding to the sample data.

[0132] Optionally, the optimization factor is not less than zero and less than one, and the optimization factor is negatively correlated with the degree of redundant error constructed by the optimization feature on the random variable.

[0133] Optionally, the optimization factor is zero.

[0134] Optionally, the training module 205 determines the historical displacement of the pedestrian corresponding to the sample data based on the sample data, thereby determining the mask factor corresponding to the sample data. The mask factor is used to characterize whether the sample data is used to determine the deviation between the sample data and the future true trajectory label. It determines the gap between the second predicted trajectory corresponding to the sample data and the labeled future true trajectory, and determines the deviation corresponding to the sample data based on the product of the gap and the mask factor corresponding to the sample data. The optimization objective is to minimize the sum of the deviations of each sample data, and the pedestrian trajectory prediction model to be trained is trained.

[0135] Optionally, the training module 205 determines whether the pedestrian corresponding to the sample data has historical displacement based on the sample data. If yes, the mask factor corresponding to the sample data is determined to be one; otherwise, the mask factor corresponding to the sample data is determined to be zero.

[0136] Optionally, the training module 205 determines the positional changes of the pedestrians in adjacent historical moments based on the sample data, determines the average speed of the pedestrians in each adjacent historical moment based on the positional changes of the pedestrians in adjacent historical moments, and determines whether the average speed of the pedestrians in each adjacent historical moment is greater than zero. If it is, it determines that the pedestrians in the sample data have historical displacement; if not, it determines that the pedestrians in the sample data do not have historical displacement.

[0137] Specific limitations regarding the pedestrian trajectory prediction device can be found in the limitations of the pedestrian trajectory prediction method described above, and will not be repeated here. Each module in the aforementioned pedestrian trajectory prediction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0138] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 2 The provided method for predicting pedestrian trajectories.

[0139] This instruction manual also provides Figure 6 The schematic diagram of the computer device shown is as follows: Figure 6 At the hardware level, the computer device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 2 The provided method for predicting pedestrian trajectories.

[0140] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0141] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for predicting pedestrian trajectories, characterized in that, include: Obtain a pedestrian historical trajectory dataset, segment each pedestrian historical trajectory data in the dataset according to a preset time, use the historical trajectory data before the preset time as sample data, and label the historical trajectory data after the preset time as the future real trajectory corresponding to the sample data; For each sample data, the pedestrian's historical trajectory features, pedestrian interaction features, and random variables are input into the pedestrian trajectory prediction model to be trained to determine the first predicted trajectory of the sample data; wherein, the pedestrian trajectory prediction model is a neural network model that incorporates random variables to simulate the randomness of pedestrian behavior in order to predict pedestrian trajectories. The redundancy error of the sample data is determined based on the random variables corresponding to the sample data and the predetermined optimization features; wherein, the optimization features are obtained by multiplying the concatenated features of historical trajectory features and interaction features with the optimization factor. The second predicted trajectory of the sample data is determined based on the difference between the first predicted trajectory and the redundancy error of the sample data. The pedestrian trajectory prediction model is trained with the goal of minimizing the deviation between the second predicted trajectory of the corresponding sample data and the labeled future true trajectory. The trained pedestrian trajectory prediction model is then used to predict pedestrian trajectories.

2. The pedestrian trajectory prediction method as described in claim 1, characterized in that, The determination of pedestrian historical trajectory features and pedestrian interaction features specifically includes: The sample data is input into the encoder of the pedestrian trajectory prediction model to be trained to obtain the historical trajectory features of the pedestrians corresponding to the sample data. The interaction between pedestrians is modeled based on their historical trajectory characteristics to determine the interaction characteristics of pedestrians corresponding to the sample data.

3. The pedestrian trajectory prediction method as described in claim 1, characterized in that, The optimization factor is not less than zero and less than one, and the optimization factor is negatively correlated with the degree of redundancy error constructed by the optimization feature on the random variable.

4. The pedestrian trajectory prediction method as described in claim 3, characterized in that, The optimization factor is zero.

5. The pedestrian trajectory prediction method as described in claim 1, characterized in that, The process of training the pedestrian trajectory prediction model, with the optimization objective of minimizing the deviation between the second predicted trajectory of the corresponding sample data and the labeled future true trajectory, specifically includes: Based on the sample data, the historical displacement of the pedestrian corresponding to the sample data is determined, so as to determine the mask factor corresponding to the sample data. The mask factor is used to characterize whether the sample data is used to determine the deviation between the actual trajectory label and the future trajectory label. Determine the gap between the second predicted trajectory corresponding to the sample data and the labeled future true trajectory, and determine the deviation corresponding to the sample data based on the product of the gap and the mask factor corresponding to the sample data; The pedestrian trajectory prediction model to be trained is trained with the optimization objective of minimizing the sum of the deviations of each sample data.

6. The pedestrian trajectory prediction method as described in claim 5, characterized in that, Determining the mask factor corresponding to the sample data specifically includes: Based on the sample data, determine whether the pedestrians corresponding to the sample data have historical displacements; If so, then the mask factor corresponding to the sample data is set to one; If not, then the masking factor corresponding to the sample data is determined to be zero.

7. The pedestrian trajectory prediction method as described in claim 6, characterized in that, The step of determining whether the pedestrian corresponding to the sample data has historical displacement based on the sample data specifically includes: Based on the sample data, determine the positional changes of pedestrians at adjacent historical moments corresponding to the sample data; The average speed of pedestrians at each adjacent historical moment is determined based on the changes in their positions at adjacent historical moments. Determine whether the average speed of pedestrians at each adjacent historical moment is greater than zero; If so, then it is determined that the pedestrians corresponding to the sample data have historical displacements; If not, then it is determined that the pedestrian corresponding to the sample data has no historical displacement.

8. A pedestrian trajectory prediction device, characterized in that, include: The acquisition module is used to acquire the pedestrian historical trajectory dataset, segment each pedestrian historical trajectory data in the dataset according to a preset time, use the historical trajectory data before the preset time as sample data, and label the historical trajectory data after the preset time as the future real trajectory corresponding to the sample data. The prediction module is used to input the pedestrian's historical trajectory features, pedestrian interaction features, and random variables into the pedestrian trajectory prediction model to be trained for each sample data to determine the first predicted trajectory of the sample data; wherein, the pedestrian trajectory prediction model is a neural network model that incorporates random variables to simulate the randomness of pedestrian behavior in order to predict pedestrian trajectories. The error construction module is used to determine the redundancy error of the sample data based on the random variables corresponding to the sample data and the pre-determined optimization features; wherein, the optimization features are obtained by multiplying the concatenated matrix of historical trajectory features and interaction features with the optimization factor; The error removal module is used to determine the second predicted trajectory of the sample data based on the difference between the first predicted trajectory and the redundancy error of the sample data. The training module is used to train the pedestrian trajectory prediction model to be trained with the optimization objective of minimizing the deviation between the second predicted trajectory of the corresponding sample data and the labeled future true trajectory, so as to perform pedestrian trajectory prediction through the trained pedestrian trajectory prediction model.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the pedestrian trajectory prediction method according to any one of claims 1 to 7.

10. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the pedestrian trajectory prediction method according to any one of claims 1 to 7.

Images