3D Pose Estimation via Hierarchical Skeleton ANN
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in accurately and efficiently performing pose estimation and body tracking, particularly for human subjects due to complexities in depth projection and variations in human body shapes, which hinders effective use in augmented reality, virtual reality, and human-computer interface applications.
Innovation Solution
The implementation of an artificial neural network (ANN) system that uses a hierarchical skeleton model, a landmark detector, a joint angle encoder, and an inverse kinematics ANN to perform pose estimation and body tracking, enabling accurate reconstruction of 3D poses from 2D skeletal landmarks and joint angles, and generating tracking images for immersive applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional pose estimation methods are used, then the system is simpler, but accuracy and reliability are insufficient due to depth projection ambiguities and body shape variations
Solution Approach 1:
The patent replaces conventional mechanical/vision-based pose estimation methods with an artificial neural network system. The ANN processes images through multiple layers (convolutional, fully connected, recurrent) to accurately estimate 3D poses and joint angles, overcoming the depth projection ambiguities and body shape variations that plague traditional methods. This substitution of computational approach directly improves measurement precision while managing system complexity through standardized ANN architecture patterns.
2Productivity
If traditional body tracking approaches are used, then implementation is easier, but speed and accuracy are insufficient for real-time applications
Solution Approach 1:
The patent implements continuous body tracking by processing sequential frames through the ANN system. The recurrent neural network components maintain temporal continuity, tracking joint positions and angles across multiple frames to produce smooth, accurate motion sequences. This continuous processing enables real-time applications while maintaining high accuracy through the ANN's ability to extract consistent pose information from each frame.
Solution Approach 2:
The patent replaces traditional tracking algorithms with an ANN-based system that processes images through convolutional layers, fully connected layers, and recurrent networks. This neural network approach simultaneously achieves high tracking speed and accuracy by learning temporal patterns and spatial relationships from training data, overcoming the speed-accuracy tradeoff inherent in conventional methods.
3Measurement precision
If complex deep learning models are used to improve accuracy, then pose estimation becomes more precise, but computational requirements and system complexity increase
Solution Approach 1:
The patent segments the pose estimation task into distinct functional components within the ANN architecture: convolutional layers for feature extraction, fully connected layers for pose regression, and recurrent layers for temporal processing. This segmentation allows each component to be optimized for its specific function, improving overall accuracy while managing computational requirements through modular design that can be deployed efficiently.
Data Source
AI summary
According to one implementation, a pose estimation and body tracking system includes a computing platform having a hardware processor and a system memory storing a software code including a tracking module trained to track motions. The software code receives a series of images of motion by a subject, and for each image, uses the tracking module to determine locations corresponding respectively to two-dimensional (2D) skeletal landmarks of the subject based on constraints imposed by features of a hierarchical skeleton model intersecting at each 2D skeletal landmark. The software code further uses the tracking module to infer joint angles of the subject based on the locations and determine a three-dimensional (3D) pose of the subject based on the locations and the joint angles, resulting in a series of 3D poses. The software code outputs a tracking image corresponding to the motion by the subject based on the series of 3D poses.