3D Face Model Generation from 2D Landmarks via Geometric Pose Transformation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing facial landmark detection technologies require annotated data for machine learning, making it inconvenient to adapt to changes in application functions and limiting flexibility in data collection.
Innovation Solution
An image processing method that estimates landmarks using a geometric model of a standard face, allowing for the derivation of three-dimensional landmarks from two-dimensional images without pre-annotated data, and updates these landmarks based on the pose of the image to generate a three-dimensional face model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning is performed using facial data with annotated landmarks, then landmark detection accuracy is improved, but adaptability to application function changes deteriorates
Solution Approach 1:
The patent segments the landmark detection task into two independent components: (1) a machine learning model that detects landmarks present in the training data, and (2) a geometric model that estimates landmarks not present in the training data through pose transformation. This segmentation allows each component to specialize, with the ML model achieving high accuracy on known landmarks while the geometric model providing adaptability to new landmark types without requiring retraining.
Solution Approach 2:
The patent introduces a geometric model as an intermediary between the machine learning model and the final landmark estimation. This geometric model uses pose information (rotation, translation, scaling) to transform landmarks from a standard face to match the detected pose, enabling the system to estimate landmarks that were not annotated in the original training data while maintaining consistency with the ML model's detections.
2Measurement precision
If new learning data is collected for each application function change, then landmark detection accuracy is improved, but data collection time and cost increase
Solution Approach 1:
The patent performs preliminary action by pre-defining a geometric model of a standard face with known landmark positions and relationships. This geometric model is prepared in advance and can be transformed to estimate various landmark types without requiring new data collection. When application functions change, the system can estimate new landmark types by transforming the pre-prepared geometric model rather than collecting new annotated data.
Solution Approach 2:
The patent uses copying by creating transformed versions of the geometric model through pose transformations (rotation, translation, scaling). Instead of collecting new data for each application, the system copies the standard face geometric model and applies transformations to generate estimates for different landmark configurations, significantly reducing data collection requirements.
3Adaptability or versatility
If a geometric model of standard face is used to estimate landmarks, then adaptability to various applications is improved, but manufacturing complexity of the system increases
Solution Approach 1:
The patent merges two relatively simple components - a machine learning model for pose and landmark detection, and a geometric model for transformation - into a unified system. The ML model handles the complex task of detecting features in images, while the geometric model handles the mathematical transformation. This merging allows the system to achieve high adaptability without either component becoming overly complex.
Solution Approach 2:
The geometric model serves multiple functions: it stores the standard face geometry, performs pose transformation, estimates missing landmarks, and adapts to different application requirements. This multi-functionality reduces the need for separate systems for each application, thereby reducing overall system complexity while maintaining high adaptability.
4Ease of operation
If landmarks are estimated without annotated data, then ease of data collection is improved, but measurement precision of landmark positions deteriorates
Solution Approach 1:
The patent applies dynamics by making the system flexible in how it obtains landmark information. Rather than relying solely on static annotated data, the system dynamically combines ML-based detection of visible landmarks with geometric model-based estimation of missing landmarks. This dynamic approach allows the system to maintain precision by using available annotated data where possible while filling gaps through geometric transformation.
Solution Approach 2:
The patent changes parameters by transforming the geometric model using pose parameters (rotation angles, translation vectors, scaling factors) derived from the ML model's detection of visible landmarks. By adjusting these parameters based on actual image content, the system maintains measurement precision even when estimating landmarks without direct annotations, as the transformation parameters are grounded in observed facial features.
Data Source
AI summary
An image processing method includes the steps of: extracting a first two-dimensional feature point from a two-dimensional face image; on the basis of a geometric model of a standard face, deriving a three-dimensional feature point set including a first three-dimensional feature point corresponding to the first two-dimensional feature point; generating a three-dimensional face model including the three-dimensional feature point set; estimating an input pose of the two-dimensional face image to determine whether to update the three-dimensional feature point set; updating the three-dimensional feature point set by relocating the first three-dimensional feature point according to the first two-dimensional feature point, on the basis of the determination on whether to update the three-dimensional feature point set; and projecting the three-dimensional face model onto the two-dimensional face image. The present invention comprises a physical computer-readable recording medium in which a program for executing, in a computer, an image processing method is recorded.