A pedestrian trajectory prediction method and system

Abstract

The application discloses a pedestrian trajectory prediction method and system. The method generates a speed sequence according to the historical positions of multiple pedestrians, and performs first-order Haar wavelet decomposition on the speed sequence. One group of trend coefficients and detail coefficients is enhanced by independent residual enhancement and inverse transformation to obtain enhanced speed, and the other group of original coefficients is used to generate the frequency domain motion condition of the corresponding pedestrian. Based on the enhanced speed sequence and pedestrian interaction information, an observation time sequence representation is formed, and the frequency domain motion condition is used to modulate the update gate and candidate state of the future query, so that the gate value and candidate state are independent of the previous hidden state. The gated recursion is converted into a parallel prefix scan form, forward and reverse parallel prefix scans are performed, and two-way states are fused to update the future query and form a prediction feature, and then a future position sequence is generated. The application makes homologous motion priors participate in observation coding and state updating at multiple future time points respectively.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, industrial scenarios, and time-series data processing, specifically to a method and system for predicting pedestrian trajectories by using wavelet frequency-domain motion priors of historical velocity for observation representation and future gating state updates. Background Technology

[0002] Pedestrian trajectory prediction typically infers the location at multiple subsequent time points based on two-dimensional position or velocity within a limited observation window, combined with social interactions between pedestrians. Existing methods can employ recurrent networks, temporal convolutions, graph neural networks, or attention networks to organize historical motion and spatial interaction information. These methods can be applied to complex industrial scenarios such as factory areas and industrial parks, where pedestrian movement exhibits strong dynamism and interactivity, posing higher demands on the effective representation of historical motion characteristics and the modeling of future state evolution.

[0003] Existing frequency domain trajectory prediction methods typically decompose historical position or trajectory representation into multiple frequency components, and encode, reconstruct, or use these different frequency components as conditions for generating models. In some existing schemes, frequency domain components are mainly used for input encoding, feature reconstruction, or generation conditions, but a direct computational connection has not yet been formed where "the same historical velocity original coefficients enter the enhanced velocity path and the future gated state modulation path respectively."

[0004] Furthermore, the thresholds or candidate states for conventional cyclic states typically depend on the previous hidden state, requiring sequential computation across multiple future time points. While general bidirectional cyclic prediction can incorporate information from two time directions, it differs from using the original wavelet coefficients of the same historical velocity simultaneously to enhance the observation input and modulate parallel gating recursion. Therefore, a technical solution with a clear data source and connectivity is needed, enabling the same motion prior to serve observation encoding and future state evolution along two paths respectively, while preserving pedestrian interaction information. Summary of the Invention Technical problems to be solved

[0005] The technical problem to be solved by this invention is: in multi-pedestrian trajectory prediction, how to separate and enhance the steady motion trend and local motion changes in historical velocity, and retain the frequency domain motion conditions from the same wavelet decomposition to the gated state update path at multiple future times, while taking into account pedestrian interaction and state information in two future time directions. Technical solution

[0006] To address the aforementioned technical problems, this invention provides a method for predicting pedestrian trajectories. This method acquires the two-dimensional historical positions of multiple pedestrians, generates a two-dimensional historical velocity sequence of equal length to the position sequence, and performs first-order Haar wavelet decomposition along the time dimension. Two processing paths are set for the same set of trend coefficients and detail coefficients: the first processing path uses residual enhancement for both types of coefficients and reconstructs the enhanced velocity using inverse wavelet transform; the second processing path uses the original coefficients without residual enhancement to generate the frequency domain motion conditions for the corresponding pedestrians.

[0007] Based on enhanced velocity and pedestrian interaction information, an observation time series representation is formed, and a future query sequence is constructed accordingly. For each pedestrian, the scaling parameters and offset parameters of the update gate and candidate state are generated from the pedestrian's frequency domain motion conditions, so that the gate value and candidate state are determined by the current future query without depending on the previous hidden state. The update gate is used to perform a gated combination of the previous hidden state and the candidate state to form a gated recursion. Since the update gate and the candidate state are both determined by the current future query and the frequency domain motion conditions without depending on the previous hidden state, the gated recursion at each future time is represented as an affine recursion about the previous hidden state, and further represented as a pair of recursion coefficients. The gated recursion is converted into a parallelizable prefix scan form through a binary combination operation that satisfies the associative law. Forward and reverse parallel prefix scans are performed with the zero vector as the initial state, and the two states after restoring the time order are fused to generate a future position sequence. Using the zero vector as the initial state for two-way scanning, forward and reverse parallel prefix scans are performed respectively, and the two-way states after restoring the time order are merged to generate the future position sequence.

[0008] The present invention also provides a pedestrian trajectory prediction system for implementing the above method, including a historical motion acquisition module, a wavelet decomposition module, a dual-path motion prior module, an interactive observation coding module, a frequency domain conditional bidirectional state update module, and a trajectory generation module; the modules are implemented by a processor executing programs in the memory. Beneficial effects

[0009] Compared to schemes that use wavelet components only at the input, this invention forms two traceable paths from the same set of original trend coefficients and detail coefficients: the enhancement coefficients are transformed by inverse transformation to form the enhancement velocity used by the observation node, and the non-enhanced coefficients form the future state modulation conditions of the same pedestrian, thus avoiding the disconnect between the two types of information sources.

[0010] Since frequency domain motion conditions act on the transformations of the update gate and candidate states respectively, this invention enables historical motion patterns to directly participate in state updates at each future time step. Because the gate value and candidate state do not depend on the previous hidden state, gated recursion can be converted into a parallelizable prefix scan form; the co-position fusion of forward and reverse scans provides state information in two time directions for each future time step. The physical interaction path can still have its side features calculated from the original position and original velocity, thus avoiding wavelet enhancement altering the meaning of physical quantities such as distance and relative velocity. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the overall structure of the pedestrian trajectory prediction system of the present invention. Detailed Implementation

[0012] The present invention will be further described below with reference to the accompanying drawings. The embodiments described are used to illustrate the technical implementation of the present invention and are not intended to limit the scope of protection to a specific dataset, hidden dimension, or number of candidate trajectories. System Composition and Data Definition

[0013] Reference Figure 1 The pedestrian trajectory prediction system includes a historical motion acquisition module, a wavelet decomposition module, a dual-path motion prior module, an interactive observation coding module, a frequency domain conditional bidirectional state update module, and a trajectory generation module. Figure 1 In this system, WCFM corresponds to historical motion acquisition, wavelet decomposition, and dual-path motion prior processing; PEEA corresponds to interactive observation coding; MGBR corresponds to frequency-domain conditional bidirectional state update; and Multimodal Decoder corresponds to trajectory generation. The system can be implemented using a computing device, which includes a processor and a memory; the memory stores programs and model parameters executable by the processor.

[0014] Let the observation window include Each observation time, the prediction window contains A future time-time scenario contains N valid pedestrians. The two-dimensional position of pedestrian i at observation time t is denoted as . The historical motion acquisition module will obtain the first speed. Let it be the zero vector, and for calculate

[0015] This yields a velocity sequence of the same length as the position sequence. In this embodiment The value is even so that adjacent velocities are processed in pairs during the first-level Haar decomposition. The input position can be obtained from an existing trajectory data interface; this invention does not limit the specific acquisition device for the position data. Method and Flow

[0016] The method of the present invention includes a historical motion acquisition step S201, a wavelet decomposition step S202, a dual-path processing step S203, an interactive observation coding step S204, a frequency domain conditional bidirectional scanning step S205, and a future trajectory generation step S206.

[0017] In step S201, the historical motion acquisition module receives the historical positions of multiple pedestrians and obtains a two-dimensional historical velocity sequence according to the above-mentioned difference and first term zero-padding rules.

[0018] In step S202, the Haar decomposition unit in the wavelet decomposition module receives the two-dimensional historical velocity sequence and performs a first-level Haar discrete wavelet transform on each coordinate component of each pedestrian along the time dimension, outputting the original trend coefficients and original detail coefficients. For adjacent velocity vectors, the trend coefficients and detail coefficients are respectively...

[0019] in When implementing this, the coefficient can be set to... The low-pass convolution kernel and coefficients are High-pass convolution kernels are used, with a stride of two.

[0020] Step S203: The dual-path motion prior module processes the same set of original trend coefficients and original detail coefficients in two ways. (Refer to...) Figure 1 The WCFM part of the algorithm includes a trend residual mapping unit, a detail residual mapping unit, and a Haar wavelet inverse transform unit in the first processing path. The enhancement coefficients can be expressed as...

[0021] in and A mapping consisting of mutually independent linear layers and nonlinear activations. and The residual scale is a learnable scale. In one embodiment, the two residual scales are initialized to zero. The Haar wavelet inverse transform unit performs a transpose convolutional inverse transform on the enhancement coefficients to obtain a scale whose length is still [value missing]. The enhanced velocity sequence.

[0022] The second processing path does not use enhancement coefficients. For each pedestrian, the original coefficient conditional projection unit conditions the pedestrian's... , The original trend coefficients and original detail coefficients corresponding to the coordinates are flattened and stitched along the coordinate and time dimensions, and then linearly projected to obtain the frequency domain motion conditions, i.e. Therefore, both the enhanced velocity sequence and the frequency domain motion conditions originate from the same original velocity sequence, but serve different subsequent calculation paths.

[0023] Step S204: The enhanced velocity node projection unit in the interactive observation coding module projects the enhanced velocity sequence into pedestrian node features. (Refer to...) Figure 1 In the PEEA section, the physical edge calculation unit still uses the original position and original velocity. For pedestrian i,j, define... , as well as The effective distance threshold for learnable neighborhood units is obtained through positive-valued learnable parameters. and will satisfy and Pedestrians are considered as effective neighbors.

[0024] Under constant relative velocity conditions, when Greater than numerical stability At that time, calculate ;when At that time, the collision time risk is Otherwise, set to zero; also set to zero when the relative velocity is approximately zero. The four-dimensional physical edge characteristics are determined by the normalized distance. Magnitude of relative velocity The components include velocity direction similarity and the risk quantity based on collision time, wherein... This represents the relative velocity scale within the scene.

[0025] The edge attention unit maps physical edge features to edge scores and adds the logarithm of the distance soft gate. The neighbor aggregation unit performs Softmax on valid neighbors and then aggregates the features of neighbor nodes. The gated residual fusion unit adds the aggregation result to the node features of the corresponding pedestrian. When there are no valid neighbors, the aggregation result is set to zero. The observation temporal coding unit performs subsequent temporal convolution and temporal embedding on the obtained pedestrian representation.

[0026] The interactive representation is convolved along the time dimension using one-dimensional convolution and superimposed with the observation time embedding to form an observation time series representation. The representation of the last observation time is copied to each prediction time and superimposed with the future time embedding to form the future query sequence for each person.

[0027] Step S205: The frequency domain conditional bidirectional state update module processes future queries. (Refer to...) Figure 1 In the MGBR section, the conditional mapping unit generates four sets of parameters based on the frequency domain motion conditions of the same pedestrian. , , and The gate and candidate state calculation unit calculate the normalized future query. Calculate the update gate and candidate state using the following formula:

[0028] in, and These are the learnable biases for updating the gate linear layer and the candidate state linear layer, respectively; This is a nonlinear activation function for candidate states, used to form the candidate state representations required for parallel gating recursion. The scaling parameter can be adjusted according to... Generate, so that each component is greater than and less than The offset parameter can be set as follows: Generate, so that each component is greater than and less than The threshold and candidate states depend only on the current future query and frequency domain motion conditions, without reading the previous hidden state. The recursive state satisfies...

[0029] Since the update gate and candidate states are determined by the current future query and frequency domain motion conditions without reading the previous hidden state, the gated recursion can be converted into a parallel prefix scan. The forward scan unit uses the zero vector as the initial hidden state and performs a parallel prefix scan in normal time order; the reverse scan unit uses the zero vector as the initial hidden state, performs the same scan on the flipped future query sequence, and then restores the original time order. The same-position state fusion unit averages the two-way states and follows... Update the original future query with the gated residual, where This represents the result of isotopic fusion of two states. For future queries after the update.

[0030] Optionally, time-decayed cross-attention units are used to update future queries. For queries, perform cross-attention using observation time-series representations as keys and values. Subtract the unnormalized score. ,in The time distance between the future moment and the observed moment. The learnable decay parameter is used to preserve content matching and distinguish different historical moments.

[0031] Step S206, refer to Figure 1 In the Multimodal Decoder section, the trajectory generation module directly generates predicted features from the future query updated in step S205, or generates predicted features through cross-attention and feedforward processing. These features are then mapped to a two-dimensional displacement space by a two-dimensional mapping unit and obtained through a multi-output unit. The candidate future displacement residuals are generated. The constant velocity baseline recovery unit adds back each residual using the last observed velocity as the constant velocity displacement baseline, and the absolute position accumulation unit accumulates over time from the last observed position to obtain the candidate future position sequence. The specific value is not a core limitation of this invention; in one embodiment, it is 20. Training Implementation Methods

[0032] The training samples include the two-dimensional historical positions of multiple pedestrians within the observation window and their two-dimensional true future positions within the prediction window. The first term of step S201 is zero-padding and differencing is performed on the two-dimensional historical positions, followed by the complete forward computation of steps S202 to S206, to obtain the positions of each pedestrian. Candidate future displacement residuals. Based on the last observed velocity. Construct a constant velocity displacement baseline; if the first The real frame-by-frame displacement at each future moment is The monitoring residual is

[0033] make ,in For the first Candidate displacement residuals. One embodiment uses the best-of-... Loss and average error of all candidates as auxiliary constraints:

[0034] Backpropagation is performed based on the loss function to update the network parameters. The first and second learnable residual scales corresponding to the trend residual mapping and detail residual mapping are both initialized to zero during training and updated during backpropagation; the learnable parameters involved in conditional mapping, gate and candidate state calculation, interactive observation encoding, and trajectory generation are updated simultaneously. In one embodiment, the Adam optimizer and initial learning rate are used. And train for a maximum of 150 cycles; number of candidate trajectories Set the value to 20. After training is complete, the obtained parameters will be updated and used in the prediction process described in claim 1. System Implementation Methods and Alternative Forms

[0035] The historical motion acquisition module, wavelet decomposition module, dual-path motion prior module, interactive observation coding module, frequency domain conditional bidirectional state update module, and trajectory generation module can be deployed as a sub-module of a network, or executed sequentially by multiple computational processes. The processor can be a central processing unit, a graphics processing unit, or a processing unit that supports tensor operations, and the memory is used to store programs, parameters, and intermediate tensors.

[0036] The embodiments of this invention include dual paths for original coefficients, modulation of update gates and candidate states by frequency domain conditions, calculation of gates and candidate states without prior state dependence, and two-way parallel prefix scanning. The physical edge features employ a four-dimensional physical edge feature structure composed of normalized distance, relative velocity magnitude, velocity direction similarity, and a risk quantity based on collision time. The number of first-order Haar transforms, residual mapping layers, hidden dimensions, number of attention heads, and number of candidate trajectories can be set according to the observation length and computational resources, but the same original coefficients enter the enhanced velocity path and the frequency domain condition path respectively, and the frequency domain conditions are used to modulate the future update gates and candidate states of the same pedestrian.

Claims

1. A method for predicting pedestrian trajectories, characterized in that, Includes the following steps: The two-dimensional historical positions of multiple pedestrians within the observation window are obtained, and a two-dimensional historical velocity sequence of equal length is generated based on the difference between the two-dimensional historical positions of adjacent observation times, wherein the velocity vector of the first observation time is set as the zero vector. Perform first-level Haar wavelet decomposition on each of the two-dimensional historical velocity sequences along the time dimension to obtain trend coefficients and detail coefficients; The trend coefficient and the detail coefficient are processed by a first processing path and a second processing path; the first processing path obtains the enhanced trend coefficient and the enhanced detail coefficient through independent residual mapping, and obtains an equal-length enhanced velocity sequence through inverse wavelet transform; The second processing path combines and projects the original trend coefficients and original detail coefficients without residual enhancement to obtain the frequency domain motion conditions of the corresponding pedestrian. Based on the enhanced velocity sequence and pedestrian interaction information, an observation time series representation is formed, and a future query sequence corresponding to multiple future moments is generated; For each pedestrian, the first scaling parameter and the first offset parameter used for calculating the update gate are generated by the frequency domain motion conditions, and the second scaling parameter and the second offset parameter are used for calculating the candidate state, so that the update gate and candidate state at each future time are determined by the corresponding future query and the frequency domain motion conditions and do not depend on the previous hidden state. The update gate is used to perform gated combination of the previous hidden state and the candidate state to form an affine recursion about the previous hidden state; The affine recursion at each future time point is represented as a pair of recursion coefficients, and a gated recursion form that can be parallelized by prefix scanning is formed by using binary combination operations that satisfy the associative law. Using the zero vector as the initial hidden state, perform forward and reverse parallel prefix scans on the future query sequence, and fuse the forward and reverse states corresponding to the future time after restoring the time order. Generate a sequence of future locations for each pedestrian based on the fused future states.

2. The pedestrian trajectory prediction method according to claim 1, characterized in that, The first-level Haar wavelet decomposition processes adjacent velocity vectors in the two-dimensional historical velocity sequence in pairs, obtaining trend coefficients through low-pass transformation and detail coefficients through high-pass transformation.

3. The pedestrian trajectory prediction method according to claim 1, characterized in that, The first processing path performs nonlinear mapping on the trend coefficient and the detail coefficient respectively, and fuses the mapping result with the corresponding original coefficient using a learnable residual scale; the second processing path combines the original trend coefficient and the original detail coefficient and maps them to the frequency domain motion conditions of the corresponding pedestrian. The first scaling parameter, the first offset parameter, the second scaling parameter, and the second offset parameter are generated by the frequency domain motion conditions through a learnable mapping.

4. The pedestrian trajectory prediction method according to claim 1, characterized in that, The process of forming the pedestrian interaction information includes: mapping the enhanced velocity sequence to pedestrian node features; constructing physical edge features representing the relative motion relationship between pedestrians based on the two-dimensional historical position and the two-dimensional historical velocity sequence, and determining the effective neighbors of each pedestrian based on learnable neighborhood conditions; determining the neighbor weights of each effective neighbor according to the physical edge features, aggregating the neighbor node features according to the neighbor weights, and fusing the aggregation result with the pedestrian node features of the corresponding pedestrian to form the pedestrian interaction information.

5. The pedestrian trajectory prediction method according to claim 1, characterized in that, The process of forming the observation time series representation and the future query sequence includes: performing a one-dimensional convolution along the time dimension on the pedestrian representation containing the pedestrian interaction information and superimposing the observation time embedding; copying the representation of the last observation time and superimposing the predicted time embedding of each future time to obtain the future query sequence.

6. The pedestrian trajectory prediction method according to claim 1, characterized in that, The fused future state is added to the future query at the corresponding future time using a gated residual method to obtain the updated future query.

7. The pedestrian trajectory prediction method according to claim 6, characterized in that, Cross-attention is performed using the updated future query as the query and the observation time sequence representation as the key and value. The non-negative decay amount, which is formed by the product of the absolute value of the learnable decay coefficient and the time distance between the observation time and the future time, is subtracted from the attention score before normalization. The future position sequence is generated based on the output of the cross-attention.

8. The pedestrian trajectory prediction method according to claim 6, characterized in that, The updated future query forms a predictive feature, which is then mapped to multiple candidate future displacement residual sequences. The constant velocity displacement baseline formed by the last observed velocity is added to each of the candidate future displacement residual sequences. The obtained future displacements are accumulated time-by-time starting from the last observed position to obtain multiple candidate future position sequences.

9. A pedestrian trajectory prediction system, characterized in that, include: The historical motion acquisition module is used to acquire the two-dimensional historical positions of multiple pedestrians, set the velocity vector of the first observation moment as the zero vector, and set the velocity vector of each of the remaining observation moments as the difference between the two-dimensional historical position of the observation moment and the previous observation moment, so as to generate a two-dimensional historical velocity sequence of the same length as the two-dimensional historical position sequence. The wavelet decomposition module is used to perform first-level Haar wavelet decomposition on the two-dimensional historical velocity sequence along the time dimension to obtain trend coefficients and detail coefficients. The dual-path motion prior module is used to perform two processing paths on the same set of trend coefficients and detail coefficients. One processing path is to perform residual enhancement on the two types of coefficients and obtain the enhanced velocity sequence through wavelet inverse transform. The other processing path combines the original trend coefficients and original detail coefficients without residual enhancement and projects them into frequency domain motion conditions. An interactive observation coding module is used to form an observation time series representation and a future query sequence based on the enhanced velocity sequence and the interaction information between pedestrians; The frequency-domain conditional bidirectional state update module is used to modulate the gated state update of the future query sequence based on the frequency-domain motion conditions, so that the update gate and candidate state at each future time are determined by the corresponding future query and the frequency-domain motion conditions and do not depend on the previous hidden state, and form an affine recursion about the previous hidden state; the affine recursion is converted into a pair of recursive coefficients that can be processed by binary combination operations that satisfy the associative law, and a forward and reverse parallel prefix scan is performed on the future query sequence to fuse the bidirectional states at the corresponding future time. The trajectory generation module is used to update future queries with the fused future states to form predictive features, and generate future location sequences based on the predictive features.

Images