Human Pose Estimation Using Kinematic Structure and Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional large convolutional neural networks (CNNs) for human pose estimation are computationally expensive and power-intensive, making them unsuitable for devices with constrained resources, and reducing their size leads to significant accuracy loss.
Innovation Solution
A method using a kinematic structure to update features of keypoints in a CNN, where features are downscaled, arranged into sets corresponding to keypoints, and updated based on neighboring keypoints, then upsampled for accurate pose prediction, reducing the model's complexity and computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large convolutional neural networks are used for pose estimation, then accuracy is improved, but computational cost and power consumption increase
Solution Approach 1:
The patent segments the pose estimation task into two stages: a coarse estimation stage using a lightweight network and a refinement stage using a larger network only for specific regions. This segmentation allows the system to achieve high accuracy while reducing overall computational cost and power consumption by avoiding full-network processing for all regions.
Solution Approach 2:
The patent applies different network complexities to different regions of the image. A lightweight network processes the entire image for coarse pose estimation, while a larger network is applied only to specific regions of interest for refinement. This local quality approach maintains high accuracy in critical regions while reducing overall computational burden.
2Measurement precision
If large convolutional neural networks are used for pose estimation, then accuracy is improved, but device complexity and resource requirements increase
Solution Approach 1:
The patent divides the pose estimation process into sequential stages with different network complexities. The first stage uses a simple network for global coarse estimation, while the second stage uses a more complex network only for local refinement. This segmentation reduces the effective model complexity required for the overall system.
Solution Approach 2:
The patent applies a more complex network only partially to specific regions where refinement is needed, rather than applying it uniformly across the entire image. This partial action approach reduces the effective computational resources required while maintaining high accuracy in critical areas.
3Productivity
If the size of the CNN model is decreased, then computational cost is reduced, but accuracy is lost
Solution Approach 1:
The patent segments the processing into coarse estimation using a small network and refinement using a larger network. This segmentation allows the system to achieve high productivity with the small network for most regions while using the larger network only when needed for accuracy-critical regions.
Solution Approach 2:
The patent introduces an intermediary refinement stage that selectively applies additional computational complexity only where necessary. This intermediary approach allows the system to maintain high computational efficiency for most regions while achieving high accuracy in specific areas through the refinement stage.
Data Source
AI summary
According to an aspect, a method for pose estimation using a convolutional neural network includes extracting features from an image, downsampling the features to a lower resolution, arranging the features into sets of features, where each set of features corresponds to a separate keypoint of a pose of a subject, updating, by at least one convolutional block, each set of features based on features of one or more neighboring keypoints using a kinematic structure, and predicting the pose of the subject using the updated sets of features.