Graph-Attention Transformer for Real-Time Body Pose and Motion Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for predicting human motion trajectories and poses are inefficient in terms of memory requirements and computational effort, making real-time applications challenging, especially for mobile robots and autonomous vehicles.
Innovation Solution
A graph-attention-based transformer model with input data transformation is used to predict 3D positions of body parts in a global coordinate system, allowing for joint prediction of motion trajectory and pose with high computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing methods for predicting human motion trajectories and poses are used, then prediction accuracy can be achieved, but memory requirements and computational effort increase significantly
Solution Approach 1:
The method separates motion trajectory prediction and pose prediction into two distinct prediction streams. The motion trajectory prediction focuses on predicting the position of a reference body part (e.g., pelvis or torso) in global coordinates, while the pose prediction focuses on predicting the relative positions of other body parts. This segmentation allows each stream to be optimized independently, reducing overall computational complexity while maintaining prediction accuracy.
Solution Approach 2:
The method extracts and processes the motion trajectory and pose information separately through different neural network branches. By taking out the reference body part position prediction from the full pose prediction process, the system can handle each prediction task with appropriate computational resources, avoiding the need to process all body parts simultaneously with full computational complexity.
2Measurement precision
If existing methods for predicting human motion trajectories and poses are used, then prediction accuracy can be achieved, but real-time application becomes challenging
Solution Approach 1:
The segmented architecture enables real-time processing by dividing the computational workload into two lighter streams rather than one heavy stream. The motion trajectory prediction stream processes global position data, while the pose prediction stream processes relative body part positions, allowing parallel computation that meets real-time requirements for mobile robot applications.
Solution Approach 2:
The method dynamically adapts the prediction process by using the predicted motion trajectory to inform the pose prediction. The system dynamically adjusts the reference frame for pose prediction based on the predicted global position, enabling efficient real-time updates as the human subject moves through different poses and positions.
Data Source
AI summary
A method for training a machine learning model to ascertain body poses and positions of a body having multiple body parts is provided. The method includes, for each training trajectory of a plurality of training trajectories, wherein each training trajectory indicates a position of each body part of the multiple body parts in a global coordinate system for each time of a given sequence of times: transforming the training trajectory into a transformed training trajectory; predicting, by means of the machine learning model, positions of the body parts at one or more times following the prediction start time; and ascertaining a loss by comparing the predicted positions with positions of the body parts indicated by the training trajectory in the training trajectory for the one or more times following the prediction start time.