3D Body Posture Detection Training With Re-Projection Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The lack of high-quality labeled data in unconstrained scenes and the difficulty in model convergence hinder accurate three-dimensional body posture detection in natural scenes using existing neural network models.
Innovation Solution
A method for training neural network models using a heatmap regression network structure that incorporates re-projection in the image domain, combining first and second loss functions to achieve stable and fast convergence, utilizing monocular images with two-dimensional and three-dimensional labels to enhance training accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional three-dimensional body posture detection neural network models are used, then detection capability is provided, but high-quality labeled data is lacking and convergence difficulty increases
Solution Approach 1:
The patent uses two-dimensional image data as a copy or projection of three-dimensional posture data. By training the neural network to detect 2D keypoint locations in images and then projecting these 2D detections back into 3D space using camera parameters, the system avoids the need for difficult-to-acquire 3D labeled data while maintaining detection accuracy.
Solution Approach 2:
The patent transforms the problem from direct 3D posture detection to 2D image space detection followed by 3D reconstruction. The neural network operates in the 2D image dimension where data is easier to obtain, and then the results are mapped back to 3D space through mathematical projection, effectively solving the data acquisition difficulty.
2Measurement precision
If traditional three-dimensional body posture detection neural network models are used, then detection capability is provided, but convergence difficulty increases
Solution Approach 1:
By using 2D image data as a surrogate for 3D posture data during training, the patent enables faster convergence. The 2D detection task is computationally simpler and has clearer optimization signals, allowing the network to converge more quickly while still achieving accurate 3D posture reconstruction through projection.
3Ease of manufacture
If re-projection based on camera parameters is used, then training is simplified and data acquisition is easier, but additional processing steps are required
Solution Approach 1:
The patent introduces camera parameters as an intermediary element that connects 2D image detections to 3D posture reconstruction. This intermediary enables the use of easily acquired 2D image data for training while systematically transforming the results into 3D space, making the additional processing step a structured and manageable operation rather than a source of complexity.
Data Source
AI summary
A method for training models is provided. The method includes: inputting an image training sample corresponding to a current iteration into a current posture detection network model, and acquiring a first loss function corresponding to the current iteration; re-projecting the current output result of the current posture detection network model, and acquiring a second loss function corresponding to the current iteration; and acquiring a posture detection network model for a next iteration by performing backpropagation on the current posture detection network model, and achieving training of the posture detection network model by performing the next iteration before an iteration end condition is met.