Pedestrian trajectory prediction method combining Kalman filter and attention mechanism recurrent neural network
By combining attention-based recurrent neural networks and Kalman filters, the problems of noise suppression and interaction modeling in pedestrian trajectory prediction for service robots in dynamic environments are solved, improving prediction accuracy and robustness, and enhancing navigation capabilities in complex scenarios.
Patent Information
- Application Number
- CN202511118955.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-11
- Publication Date
- 2025-12-26
AI Technical Summary
Existing technologies for predicting pedestrian trajectories in dynamic environments for service robots suffer from insufficient sensor noise suppression, difficulty in dynamic interaction modeling, and long-term prediction bias, resulting in unstable navigation and low accuracy.
By combining attention-based recurrent neural networks (RNNs) with Kalman filters, pedestrian trajectory prediction is optimized through multimodal data perception and dynamic noise suppression. Multi-head attention mechanisms are used to capture pedestrian interaction features, and state correction is performed through Kalman filters to improve prediction accuracy and robustness.
It significantly reduces the impact of sensor noise, improves the accuracy and robustness of pedestrian trajectory prediction, achieves high-precision navigation decision support, and adapts to complex dynamic environments.
Smart Images
Figure CN121213618A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of service robot navigation technology, specifically to a pedestrian trajectory prediction algorithm and system that combines an attention mechanism recurrent neural network (RNN) with a Kalman filter, to improve the navigation safety and robustness of robots in dynamic environments. Background Technology
[0002] Autonomous navigation of service robots in dynamic environments heavily relies on pedestrian trajectory prediction, but existing methods face core challenges such as insufficient sensor noise suppression, difficulty in modeling dynamic interactions, and long-term prediction bias. While the sti-Graph framework improves prediction accuracy in dense crowds by decoupling human-robot interaction features, its fixed interaction patterns struggle to adapt to complex scene changes. The SOGM method, although capable of inferring long-term pedestrian intentions, is limited by high real-time computational requirements. In practical applications, sensor noise causes trajectory jitter and instability. Deployed LiDAR and RGB-D cameras inherently possess noise, such as sparse point clouds, depth errors, and ambient lighting interference. This leads to large fluctuations in the original detected position data, directly impacting subsequent prediction and navigation safety. Furthermore, real-world scenarios are complex, and pedestrian interactions in complex dynamic environments are difficult to model. Traditional simple physical models cannot capture these nonlinear relationships, resulting in prediction bias. In addition, prediction models based on single neural networks lack state estimation optimization mechanisms, leading to significant cumulative errors. Summary of the Invention
[0003] To address the aforementioned issues, this invention proposes an optimization framework that integrates multimodal data and dynamic noise suppression, combining an attention-based recurrent neural network (RNN) with a Kalman filter. By integrating RGB-D / LiDAR sensor data, the framework dynamically captures the spatiotemporal interaction features of pedestrians using an attention mechanism, dynamically calculates the interaction weights between different pedestrians using a multi-head attention mechanism, and introduces a Kalman filter to suppress noise and correct the initial predicted trajectory. The model input includes not only basic states such as position and velocity but also actual perception data from RGB-D / LiDAR. This enables the model to perceive the influence of obstacles and channel scene structure on pedestrian intentions, improving the predictive adaptability to real-world scenarios. This framework inherits the theoretical advantages of heterogeneous interaction modeling and temporal consistency optimization, enhancing the model's adaptability to group behavior while reducing sensor noise interference, ultimately achieving high-precision and robust trajectory prediction. Figure 1 As shown, the multimodal detection data, after being processed by an attention network, is dynamically optimized through Kalman filtering to provide a reliable decision-making basis for service robot navigation.
[0004] This invention proposes a pedestrian trajectory prediction method that combines Kalman filter and attention mechanism recurrent neural network (RNN). It aims to reduce the interference of multimodal sensor noise on trajectory prediction, improve the model's ability to model dynamic pedestrian interactions and group behavior, optimize the accuracy and robustness of long-term trajectory prediction, and provide an efficient, reliable and low-cost solution for predicting pedestrian trajectories.
[0005] To achieve the above objectives, the technical solution of the present invention is as follows: a pedestrian trajectory prediction method combining a Kalman filter and an attention mechanism recurrent neural network (RNN), the method comprising the following steps:
[0006] S1: Multimodal perception and dynamic interaction modeling
[0007] S2: Spatiotemporal interaction-driven trajectory prediction
[0008] S3: Trajectory optimization based on Kalman filtering
[0009] S4: Navigation strategy generation and verification.
[0010] Step S1, multimodal perception and dynamic interaction modeling, is detailed below.
[0011] Pedestrian motion data was acquired using multimodal sensors (RGB-D camera and LiDAR). LiDAR data was used to detect leg features via an augmentation classifier, while RGB-D data was used to detect upper body features via depth template matching. A detection-level fusion strategy was employed, and the correlation between multi-sensor observations was calculated based on the Mahalanobis distance formula: z i ,z j Let S be the observed values and S be the covariance matrix.
[0012]
[0013] Nearest neighbor matching is performed on multi-sensor observations, and real-time pedestrian status tracking is achieved by combining the interactive multiple model (IMM) algorithm with Kalman filtering; simultaneously, based on the pedestrian's relative position p... i ,p j With the direction of motion v i ,v j Calculate the social relationship weight, σ d The distance scale parameter controlling the attenuation intensity of the table;
[0014] Dynamically identify groups of people walking together.
[0015] Step S2: Spatiotemporal interaction-driven trajectory prediction, as detailed below.
[0016] A graph neural network (GNN) based on an attention mechanism is constructed, with input including pedestrian state (position, velocity) and environmental visual features; a multi-head attention mechanism is employed.
[0017]
[0018] Modeling spatiotemporal interaction relationships, where h is the number of attention heads, and W O This is the linear transformation matrix used for output fusion. Each head is defined as head. i =Attn(Q i ,K i V i (), dynamically captures heterogeneous interaction patterns among pedestrians through self-attention weights; utilizes a temporal prediction network Inferring the future trajectory of a pedestrian, where x t v represents the current position. t Given the current speed, a t For the current acceleration, intention information I t Generated by an attention mechanism; the trajectory prediction results are integrated into a proximal policy optimization (PPO) reinforcement learning framework, whose reward function R... t =α·AP+β·GA+γ·SC fusion intention prediction, where AP is the proximity penalty, GA is the goal arrival reward, SC is the social compliance constraint, and α, β, and γ are weight coefficients, driving the robot to generate a navigation strategy that conforms to the pedestrian's intention.
[0019] Step S3: Trajectory optimization based on Kalman filtering, as detailed below.
[0020] The initial trajectory prediction value output from step 2 As the input state quantity of the Kalman filter, the trajectory data is optimized through a two-stage prediction-update process:
[0021] (1) Prediction stage: Calculate the prior state estimate based on the state transition matrix A and the control matrix B. and error covariance
[0022] (2) Update phase: Based on real-time sensor observations y k Calculate Kalman gain Integrating observational data and predicted values to update the posterior state estimate and covariance By adjusting the parameter configurations of process noise covariance Q and observation noise covariance R, noise suppression and trend fidelity can be balanced.
[0023] Step S4: Navigation strategy generation and verification, as detailed below.
[0024] The optimized trajectory output from step 3 guides the service robot's navigation, enabling dynamic obstacle avoidance on the TurtleBot2 physical platform. The predictive performance is evaluated using quantitative metrics (mean square error MSE, signal-to-noise ratio SNR, and trend retention coefficient), verifying the effectiveness of the method in reducing sensor noise interference and improving long-term trajectory accuracy.
[0025] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the pedestrian trajectory prediction method combining a Kalman filter and an attention mechanism recurrent neural network.
[0026] A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the pedestrian trajectory prediction method combining a Kalman filter and an attention mechanism recurrent neural network.
[0027] Compared to existing technologies, the advantages of this invention are as follows: This method demonstrates significant advantages in both simulation and physical experiments. Through dynamic optimization of the Kalman filter, the impact of sensor noise on trajectory prediction is significantly reduced. Under the default configuration, the mean square error (MSE) is 0.032, the mean absolute error (MAE) is 0.024, and the signal-to-noise ratio (SNR) is improved to 25.8 dB. Under high process noise configuration, the MSE further decreases to 0.018, the MAE is 0.016, and the SNR reaches 32.1 dB, indicating that the algorithm can still maintain high-precision prediction even in strong noise environments.
[0028] The spatiotemporal interaction modeling module captures the consistency of pedestrian movement through a multi-head attention mechanism, and the social relationship weights are calculated as follows:
[0029]
[0030] The model effectively identifies group behavior, and experiments show that the correlation coefficient between the predicted trajectory and the actual path reaches 0.95, verifying its adaptability to dynamic interactions. Furthermore, the algorithm exhibits low latency (response time <50ms) in physical migration experiments on the TurtleBot2 platform, successfully achieving a seamless transition from simulation to real-world environments, providing a reliable solution for robot navigation in complex dynamic scenarios. Attached Figure Description
[0031] Figure 1 This is a schematic diagram of the overall framework of the present invention.
[0032] Figure 2 This is a comparison chart showing the smoothing effect of the Kalman filter under the default noise configuration. Detailed Implementation
[0033] To enhance understanding of the present invention, the embodiments will be described in detail below with reference to the accompanying drawings.
[0034] Example: See Figures 1-2 A pedestrian trajectory prediction method combining Kalman filter and attention mechanism recurrent neural network (RNN) includes the following steps:
[0035] S1: Multimodal perception and dynamic interaction modeling
[0036] S2: Spatiotemporal interaction-driven trajectory prediction
[0037] S3: Trajectory optimization based on Kalman filtering
[0038] S4: Navigation strategy generation and verification.
[0039] The implementation process of multimodal perception and dynamic interaction modeling is as follows.
[0040] A 12m×12m static pipeline scene was built in a simulation environment, and the TurtleBot3i mobile robot platform was deployed, integrating an RGB-D camera (Intel RealSense D435) and a LiDAR sensor (Hokuyo UTM-30LX). The RGB-D camera captured the upper body features of pedestrians in real time using depth template matching technology, while the LiDAR sensor used an enhanced classifier to detect leg point cloud data. Multi-source sensor data were correlated using a detection-level fusion strategy: the correlation between observations was calculated based on the Mahalanobis distance formula, where z... i z represents the historical trajectory observation value. j S represents the current detected value, and S is the covariance matrix. The specific calculation follows the formula:
[0041]
[0042] After establishing multi-sensor data association through the nearest neighbor matching algorithm, the pedestrian's position and velocity status are dynamically updated by combining the interactive multiple model algorithm (IMM) with Kalman filtering. Simultaneously, based on the relative positions p between pedestrians... i ,p j and direction of motion v i ,v j Construct a social relationship model and calculate the social relationship weights using a formula:
[0043]
[0044] Where the distance scale parameter σ d=1.2m, used to identify coordinated walking groups with consistent movement.
[0045] A graph neural network (GNN) based on a multi-head attention mechanism is constructed, with input including pedestrian state vectors (position, velocity) and environmental visual features. Heterogeneous interaction relationships are learned through a self-attention mechanism; the core computation is shown in the formula:
[0046]
[0047] Where Q (Query), K (Key), and V (Value) represent the query vector, key vector, and value vector, respectively. Furthermore, a multi-head attention model is used to expand the representation of capabilities, as shown in the formula:
[0048]
[0049] Each attention head i Independently learns different interaction modes. The temporal prediction network is based on the current state x. t ,v t ,a t and the intention information generated by attention I t Output future location prediction The prediction results are integrated into the Proximal Policy Optimization (PPO) framework, and the reward function is designed as follows:
[0050] R t =α·AP + β·GA + γ·SC
[0051] In this system, AP triggers a negative reward when the robot is less than 1 meter away from the pedestrian, GA provides a positive reward when the robot reaches the target point, and SC is based on social relationship weights w. ij Adjust the path to align with the group's movement intentions.
[0052] The initial trajectory prediction output by the spatiotemporal interaction module The input Kalman filter is dynamically optimized. During the prediction phase, prior state estimates are calculated based on the state transition matrix A and the control matrix B.
[0053]
[0054] Simultaneously update the error covariance:
[0055]
[0056] During the update phase, the Kalman gain is calculated based on the real-time sensor observations $y_k$:
[0057]
[0058] Update the posterior state by fusing observational data and predicted values:
[0059]
[0060] Final optimized error covariance:
[0061]
[0062] By adjusting the process noise covariance Q(transition) cov The observation noise covariance R$($obseration_{cov}) achieves a balance between noise suppression and trend fidelity. Experiments validate three configurations: default parameters (Q=0.1, R=1.0), high process noise (Q=0.5, R=1.0), and high observation noise (Q=0.1, R=2.0).
[0063] The optimized trajectory prediction model was deployed on the TurtleBot2 physical platform. First, 10,000 sets of data containing Gaussian noise (σ) were collected in the simulation environment. 2 The model was trained using pedestrian trajectory data with a frame rate of 0.05. After 200 iterations, the loss function of the attention RNN converged to 0.023. Real-world testing was conducted in a laboratory (5-8 people / 100㎡), with multimodal sensors simultaneously acquiring data at LiDAR (15fps) and RGB-D (30fps) frame rates. The trajectory prediction cycle was 100ms / execution, and the Kalman filter computation latency was controlled within 5ms. Performance was evaluated using three core metrics:
[0064] 1. Prediction accuracy: Measure the mean square error (MSE) and mean absolute error (MAE) between the predicted trajectory and the actual trajectory.
[0065] 2. Noise Suppression: Quantifying the data denoising effect through signal-to-noise ratio (SNR).
[0066] 3. Trend fidelity: Calculate the correlation coefficient between the predicted trajectory and the actual trajectory.
[0067] Experimental data show (see Table 1),
[0068] Table 1
[0069] Configuration Scheme MSE MAE Std SNR (dB) Trend retention Default configuration (blue line) 0.032 0.024 0.056 25.8 0.95 High process noise configuration (red line) 0.018 0.016 0.048 32.1 0.90 High observation noise configuration (green line) 0.021 0.018 0.060 28.7 0.85
[0070] Wherein, MSE (mean squared error):
[0071]
[0072] n: Number of samples
[0073] y i : True value
[0074] Predicted value;
[0075] MAE (Mean Absolute Error):
[0076]
[0077] Std (Standard Deviation):
[0078]
[0079] Data mean
[0080] SNR (Signal-to-Noise Ratio):
[0081]
[0082] P signal Effective signal power
[0083] P noise Noise power The high process noise configuration achieves optimal noise suppression (MSE = 0.018, SNR = 32.1dB) under strong interference environments, while the default configuration maintains the best trend tracking capability in dynamic scenarios (correlation coefficient 0.95). The deviation between the physical platform test results (MSE = 0.035) and the simulation results (MSE = 0.032) is less than 9%, verifying the generalization ability of the algorithm.
[0084] Figure 1 This is a schematic diagram of the hardware architecture of this experimental platform, showing the integration of the TurtleBot robot, multimodal sensors (radar, LiDAR), and control module;
[0085] Figure 2 Comparison of the smoothing effect of the Kalman filter under the default noise configuration. The default configuration (blue line) shows the process noise covariance (Q = 0.1), observation noise covariance (R = 1.0), and confidence levels between the equalization model prediction and sensor observations. Experiments show that its MSE is 0.032, MAE is 0.024, SNR reaches 25.8 dB, and trend retention coefficient (TR) is 0.95, making it suitable for real-time tracking of short-term fluctuation characteristics in dynamic environments.
[0086] It should be noted that the above embodiments are not intended to limit the scope of protection of the present invention. Equivalent transformations or substitutions made based on the above technical solutions all fall within the scope of protection of the claims of the present invention.
Claims
1. A method for predicting pedestrian trajectories by combining a Kalman filter and an attention-based recurrent neural network, characterized in that, The method includes the following steps: S1: Multimodal perception and dynamic interaction modeling S2: Spatiotemporal interaction-driven trajectory prediction S3: Trajectory optimization based on Kalman filtering S4: Navigation strategy generation and verification.
2. The pedestrian trajectory prediction method combining Kalman filter and attention mechanism recurrent neural network according to claim 1, characterized in that, Step S1: Multimodal perception and dynamic interaction modeling, as detailed below. Pedestrian motion data was acquired using multimodal sensors (RGB-D camera and LiDAR). LiDAR data was used to detect leg features via an augmentation classifier, while RGB-D data was used to detect upper body features via depth template matching. A detection-level fusion strategy was employed, and the correlation between multi-sensor observations was calculated based on the Mahalanobis distance formula: z i ,z j Let S be the observed values and S be the covariance matrix. d 2 (i,j)=(z j -z i )′S -1 (z j -z i ) Nearest neighbor matching is performed on multi-sensor observations, and real-time pedestrian status tracking is achieved by combining the interactive multiple model (IMM) algorithm with Kalman filtering; simultaneously, based on the pedestrian's relative position p... i ,p j With the direction of motion v i ,v j Calculate the social relationship weight, σ d The distance scale parameter controlling the attenuation intensity of the table; Dynamically identify groups of people walking together.
3. The pedestrian trajectory prediction method combining Kalman filter and attention mechanism recurrent neural network according to claim 1, characterized in that, Step S2: Spatiotemporal interaction-driven trajectory prediction, as detailed below. A graph neural network (GNN) based on an attention mechanism is constructed, with input including pedestrian state (position, velocity) and environmental visual features; a multi-head attention mechanism is employed. Modeling spatiotemporal interaction relationships, where h is the number of attention heads, and W O Each head is defined as a linear transformation matrix used for output fusion. i =Attn(Q i ,K i , V i) Dynamically capture heterogeneous interaction patterns among pedestrians through self-attention weights; utilize temporal prediction networks. Inferring the future trajectory of a pedestrian, where x t v represents the current position. t Given the current speed, a t For the current acceleration, intention information I t Generated by an attention mechanism; the trajectory prediction results are integrated into a proximal policy optimization (PPO) reinforcement learning framework, whose reward function R... t =α·AP+β·GA+γ·SC fusion intention prediction, where AP is the proximity penalty, GA is the goal arrival reward, SC is the social compliance constraint, and α, β, and γ are weight coefficients, driving the robot to generate a navigation strategy that conforms to the pedestrian's intention.
4. The pedestrian trajectory prediction method combining Kalman filter and attention mechanism recurrent neural network according to claim 1, characterized in that, Step S3: Trajectory optimization based on Kalman filtering, as detailed below. The initial trajectory prediction value output from step 2 As the input state quantity of the Kalman filter, the trajectory data is optimized through a two-stage prediction-update process: (1) Prediction stage: Calculate the prior state estimate based on the state transition matrix A and the control matrix B. and error covariance (2) Update phase: Based on real-time sensor observations y k Calculate Kalman gain Integrating observational data and predicted values to update the posterior state estimate and covariance By adjusting the parameter configurations of process noise covariance Q and observation noise covariance R, noise suppression and trend fidelity can be balanced.
5. The pedestrian trajectory prediction method combining Kalman filter and attention mechanism recurrent neural network according to claim 1, characterized in that, Step S4: Navigation strategy generation and verification, specifically as follows: the optimized trajectory output in step 3 is used to guide the service robot's navigation, and dynamic obstacle avoidance is achieved on the TurtleBot2 physical platform; the prediction performance is evaluated through quantitative indicators (mean square error MSE, signal-to-noise ratio SNR, trend retention coefficient), and the effectiveness of the method in reducing sensor noise interference and improving long-term trajectory accuracy is verified.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a pedestrian trajectory prediction method combining a Kalman filter and an attention mechanism recurrent neural network as described in any one of claims 1 to 5.
7. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, the computer instructions implement a pedestrian trajectory prediction method combining a Kalman filter and an attention mechanism recurrent neural network as described in any one of claims 1-5.