Markerless Motion Capture for Sudden Pedestrian Movement Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing motion prediction techniques, particularly for pedestrian safety in automated vehicles, struggle to accurately predict sudden changes in pedestrian movement due to their reliance on classical methods that require significant prior lateral motion data, leading to increased reaction times and potential safety hazards.
Innovation Solution
A markerless motion capture system utilizing neural networks to extract skeletal joint locations and predict future movement from video data, allowing for the prediction of immediate changes in whole-body motion vectors without the need for external markers, and extending to various animate subjects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If classical optical flow techniques are used for motion prediction, then the system can provide simple predictions of direction and speed, but the prediction accuracy deteriorates when there is insufficient prior lateral motion data, leading to increased reaction times
Solution Approach 1:
The patent replaces classical optical flow techniques with deep learning-based markerless motion capture systems. The system uses convolutional neural networks (CNNs) to extract skeletal joint locations directly from video frames, substituting traditional computer vision mechanics with neural network-based pattern recognition. This enables accurate prediction of sudden motion changes without requiring significant prior lateral motion data.
Solution Approach 2:
The patent transitions from 2D optical flow analysis to 3D skeletal joint location extraction by introducing depth estimation through multi-camera setups or monocular depth prediction networks. This dimensional enhancement allows the system to capture motion in three-dimensional space, improving prediction accuracy for sudden changes in pedestrian behavior.
2Measurement precision
If markerless motion capture with neural networks is used, then the system can extract skeletal joint locations and predict future motion accurately, but the device complexity increases compared to classical filtering methods
Solution Approach 1:
The patent extracts only the essential skeletal joint locations from full video frames using trained neural networks. Instead of processing entire video sequences or using complex marker-based systems, the system identifies and tracks key anatomical points (joints, limbs, torso) directly from images. This extraction approach maintains high prediction accuracy while reducing computational complexity compared to comprehensive motion capture systems.
Solution Approach 2:
The patent uses trained neural network models that have been pre-trained on large datasets of human motion. These pre-trained models serve as reusable copies of motion patterns that can be applied to new video inputs without requiring retraining. This allows the system to achieve high measurement precision while avoiding the computational burden of real-time training, effectively copying learned motion dynamics to new situations.
3Measurement precision
If fine-grained motion predictions for each joint position are made, then the system provides detailed relative position data, but the computational load increases and translational motion of the base reference frame is removed
Solution Approach 1:
The patent segments the motion prediction task into two distinct components: (1) extraction of skeletal joint locations relative to the pelvis using CNNs, and (2) prediction of whole-body translational motion using separate temporal convolutional networks. This segmentation allows the system to maintain detailed joint position data while efficiently computing overall motion trends, reducing redundant calculations and energy consumption.
Solution Approach 2:
The patent performs preliminary extraction of skeletal joint locations and stores these relative position data before predicting future translational motion. By pre-processing and storing joint position information, the system avoids redundant calculations during the prediction phase, reducing computational energy requirements while maintaining measurement precision for both fine-grained and whole-body motion.
Data Source
AI summary
A motion prediction system for predicting the motion of a random animate subject. A first neural network is a markerless motion capture network, trained to receive video data of the subject and to process the video data to generate a time sequence of musculoskeletal motion capture data. A second neural network is a motion prediction network, trained to receive the musculoskeletal motion capture data and to process the data to generate a prediction of the subject's location based on position change in position of joints and/or muscles.