Infant Pose Estimation Using Synthetic Data and Domain Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer vision models struggle to accurately estimate infant poses due to significant differences in body ratios and pose complexity compared to adults, and the lack of adequate training data for infants is exacerbated by privacy concerns, leading to insufficient performance of mainstream algorithms.
Innovation Solution
A fine-tuned domain-adapted infant pose estimation model (FiDIP) that leverages transfer learning from adult pose data and incorporates a synthetic and real infant pose dataset (SyRIP) using a multi-stage invariant representation learning strategy, aligning features through domain adaptation and synthetic data augmentation to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If computer models trained on large-scale adult pose datasets are used for infant pose estimation, then the model has sufficient training data, but the estimation accuracy deteriorates due to significant differences in body ratios and pose varieties
Solution Approach 1:
The patent creates a synthetic infant pose dataset by copying and adapting adult pose data through 3D body model transformations. Synthetic infant images are generated by mapping adult pose annotations to infant-specific 3D models (SMIL), applying infant body ratios, and rendering realistic images that preserve pose information while adapting to infant characteristics. This allows leveraging large-scale adult data while achieving infant-specific estimation accuracy.
Solution Approach 2:
The patent transforms adult pose data into infant pose data by changing key parameters including body scale factors, limb length ratios, head-to-body proportions, and joint angle distributions. The 3D body model parameters are adjusted to reflect infant anatomy, and pose parameters are re-scaled to match infant movement patterns, enabling accurate infant pose estimation from adult training data.
2Measurement precision
If real infant images are collected for training, then the pose estimation accuracy improves, but privacy and security concerns worsen the availability of adequate training data
Solution Approach 1:
Instead of collecting real infant images, the patent copies pose information from adult images and synthesizes infant images using 3D models. The synthetic images preserve all necessary pose annotations while eliminating privacy concerns, as no real infant photos are required. This approach maintains data availability for training while ensuring privacy compliance.
Solution Approach 2:
The patent introduces a 3D body model as an intermediary between adult pose data and infant pose estimation. The 3D model serves as a mediator that transforms adult pose annotations into synthetic infant images with accurate pose labels, bridging the gap between available adult data and the target infant application without requiring direct infant image collection.
3Measurement precision
If RGB-D data sequences are used for infant pose estimation, then the pose estimation accuracy improves, but the device complexity increases due to difficulty in obtaining such data with regular webcams
Solution Approach 1:
The patent copies depth information and pose annotations from adult RGB-D datasets and applies them to synthetic infant images. By transferring learning from adult RGB-D data through domain adaptation, the system achieves infant pose estimation accuracy without requiring actual RGB-D sensors, maintaining compatibility with standard RGB webcam systems.
Solution Approach 2:
The patent replaces the need for physical RGB-D sensing hardware with a computational approach. Instead of using depth cameras to capture infant poses directly, the system uses 2D RGB images combined with 3D body models and synthetic data to infer 3D pose information, substituting mechanical depth sensing with algorithmic pose reconstruction.
Data Source
AI summary
Methods are provided for estimating a pose of an infant using image analysis and artificial intelligence. A classifier is trained using a dataset containing hybrid synthetic and real infant pose data. Multi-stage invariant representation machine learning strategies are employed that transfer knowledge from adjacent domains of adult poses and synthetic infant images into a fine-tuned domain-adapted infant pose estimation model.