Pedestrian Locomotion Forecasting via Pose Stream Disentanglement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems face challenges in predicting pedestrian locomotion dynamics in self-driving scenarios due to the complexity of multimodal pedestrian locomotion distribution, occlusions, and the scarcity of annotated pedestrian pose datasets, which affects the accuracy of forecasting human movement.
Innovation Solution
A method that generates frame-level supervision for human poses, suppresses noise, and fills missing joints using a pose completion module, splitting the poses into global and local streams to forecast future human locomotion by combining these streams using quasi recurrent neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full annotations of pedestrian dynamics including egomotion and pose are obtained, then forecasting accuracy is improved, but annotation cost and time increase significantly
Solution Approach 1:
The patent uses pose estimation models (e.g., OpenPose, AlphaPose) to generate synthetic pose annotations from video frames, creating copies of ground truth data without manual annotation. This automated copying process provides dense frame-level supervision signals while avoiding the time-consuming manual annotation process, resolving the contradiction between accuracy and annotation time
Solution Approach 2:
The system performs self-supervised learning by automatically generating its own training annotations through pose estimation algorithms. The pose completion module and stream disentanglement framework enable the system to create and utilize its own supervision signals without external human annotation, making the system self-sufficient and eliminating the bottleneck of manual annotation time
2Measurement precision
If complex pedestrian locomotion distribution is modeled in detail, then prediction accuracy is improved, but model complexity increases
Solution Approach 1:
The patent segments pedestrian locomotion into two independent streams: global stream (overall body translation and orientation) and local stream (limb movements and pose changes). This segmentation simplifies the modeling task by breaking down complex multimodal locomotion distribution into manageable components that can be modeled separately and then combined, reducing overall model complexity while maintaining prediction accuracy
Solution Approach 2:
The patent applies different modeling approaches to different parts of the locomotion problem: the global stream uses trajectory prediction models for overall movement, while the local stream uses pose estimation and completion models for limb movements. This local quality approach optimizes each component for its specific function, achieving high overall accuracy without requiring a single overly complex model
3Manufacturing precision
If dense frame-level supervision is generated for all frames, then training quality is improved, but computational resources increase
Solution Approach 1:
The patent generates dense frame-level supervision for all frames (excessive action) but selectively applies different processing levels: full pose completion and stream disentanglement for key frames, and lighter processing for intermediate frames. This partial application of full supervision maintains training quality while reducing overall computational resource consumption compared to processing every frame at maximum detail
Data Source
AI summary
A method for predicting spatial positions of several key points on a human body in the near future in an egocentric setting is described. The method includes generating a frame-level supervision for human poses. The method also includes suppressing noise and filling missing joints of the human body using a pose completion module. The method further includes splitting the poses into a global stream and a local stream. Furthermore, the method includes combining the global stream and the local stream to forecast future human locomotion.