Real-Time Monocular 3D Posture Estimation with Autoencoders
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for 3-D posture estimation from 2-D images are inaccurate, require depth sensors or multiple cameras, and fail to leverage temporal information effectively, leading to high resource consumption and poor posture transition.
Innovation Solution
A neural network model comprising an autoencoder and a second encoder, trained using monocular video from a single low-end device, employs residual network blocks with skip connections and 1-D convolution to estimate 3-D posture in real-time with high accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional CV-based 3-D posture monitoring techniques using depth sensors or stereovision cameras are employed, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent extracts and removes the depth sensor requirement from the conventional system. By using a standard monocular RGB camera instead of depth sensors or stereovision systems, the invention achieves 3-D posture estimation without the complex hardware while maintaining acceptable accuracy through learned temporal correlations
Solution Approach 2:
The patent replaces the mechanical/optical depth sensing mechanism with a computational approach using neural networks. Instead of using physical depth sensors or multiple cameras to geometrically compute depth, the system uses a single camera with AI algorithms that learn temporal patterns to infer 3-D posture, substituting mechanical complexity with intelligent processing
2Manufacturing precision
If parametric forms such as SMPL models are used for 3-D shape recovery, then manufacturing precision is improved, but use of energy and processing power increase
Solution Approach 1:
The patent extracts only the essential skeletal posture information needed for exercise monitoring rather than recovering the entire 3-D body shape. By focusing specifically on joint positions and body segment orientations using skeleton-based approaches rather than full parametric models, the system reduces computational load while maintaining precision for the specific application of posture monitoring
Solution Approach 2:
The patent applies partial action by recovering only the necessary 3-D posture parameters (skeletal joints and body segments) rather than the complete 3-D body surface geometry. This selective recovery approach provides sufficient information for exercise form monitoring without the excessive computational requirements of full parametric model fitting
3Measurement precision
If techniques exploiting temporal information are used for 3-D posture estimation, then measurement precision is improved, but loss of time increases due to two state computations
Solution Approach 1:
The patent applies preliminary action by pre-training the neural network on large datasets containing temporal posture sequences. The network learns temporal correlations and motion patterns during the training phase, enabling it to efficiently process real-time video frames with reduced computational overhead during actual posture estimation, thus balancing accuracy with speed
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
The disclosure relates generally to methods and systems for real time video driven human 3-dimensional (3-D) posture estimation during physical activities. Conventional techniques do not exploit temporal information, they do not give smooth transition of postures over time. Furthermore, the techniques that exploit the temporal information suffer from higher time requirements due to two state computations. The present disclosure solves the technical problems in the art with the methods and systems for real time video driven human 3-D posture estimation during physical activities. The present invention discloses a smart-phone camera based automatic posture monitoring system designed with an auto-encoder based architecture. The disclosed auto-encoder based cross-modal method uses monocular video (2-D image sequences) from a single low-end mobile device (for example, smart-phone camera) for estimating human 3-D posture in real time (~ 5 fps) with high accuracy (less than 1 cm error per joint location).