3D Human Pose Data Augmentation for Camera Domain Shift
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D human body model estimation techniques face challenges in accurately recognizing human poses and camera positions due to domain shifts between training and test data, particularly when camera positions differ, limiting the effectiveness of domain adaptation methods like SPIN and DAPA.
Innovation Solution
A method that involves generating augmented teacher data by specifying and augmenting camera and human posture distributions based on sample images from the target domain, using a combination of domain discriminators and augmenters to create synthetic images that better represent the target environment, thereby training a model capable of recognizing human poses across varied camera positions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If domain adaptation is performed using conventional techniques (SPIN, DAPA) with sample images from target domain, then the model can adapt to some extent, but the adaptation effectiveness is limited when camera positions differ significantly from training data
Solution Approach 1:
The patent performs preliminary actions by specifying the distribution characteristics of camera positions and human postures from target domain sample images before actual domain adaptation. This preliminary distribution specification enables the model to anticipate and adapt to the target domain's characteristics in advance, improving reliability when camera positions differ significantly.
Solution Approach 2:
The patent changes parameters by explicitly modeling and adjusting the distribution parameters of camera positions (position coordinates, orientation angles) and human postures (joint angles, body dimensions). By parameterizing these distributions and performing domain adaptation on the parameters themselves, the method achieves more effective adaptation across varying camera positions compared to conventional techniques.
2Measurement precision
If new 3D teacher data is collected using special measurement systems (Motion Capture) for target domain training, then training accuracy improves, but implementation difficulty and cost increase significantly
Solution Approach 1:
The patent creates synthetic 3D teacher data by copying and transforming 3D human body models from the source domain, adjusting their parameters to match the target domain's distribution characteristics. This copying approach generates accurate training data without requiring expensive Motion Capture systems, making implementation feasible in practical applications.
Solution Approach 2:
The system performs self-service by automatically generating augmented teacher data through computational processes. Instead of requiring external specialized measurement equipment, the system uses algorithms to specify distributions, generate 3D models, and create training data autonomously from available sample images, eliminating dependency on external MoCap facilities.
3Ease of operation
If the training model is fine-tuned using only 2D skeleton information from sample images, then the process is simple, but the model cannot generalize to images captured from different camera positions
Solution Approach 1:
The patent transitions from 2D skeleton information to 3D spatial understanding by explicitly specifying and adapting the distribution of camera positions (3D coordinates and orientations) and human postures in 3D space. This dimensional elevation enables the model to generalize across different camera positions while maintaining the simplicity of fine-tuning through distribution parameter adjustment.
Data Source
AI summary
A generation method includes specifying a first distribution of postures of a person and a second distribution of positions or orientations or both of a camera based on a plurality of sample images in which the postures of the person and the positions and orientations of the camera that captures the person are different from each other augmenting the postures of the person in a range included in the first distribution augmenting the positions or orientations or both of the camera in a range included in the second distribution and generating an augmented image based on the augmented positions or orientations or both of the camera and the augmented postures of the person, by using a processor.