3D Face Mesh Illumination Detection for AR Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting illumination in real scenes, especially for augmented reality applications, face challenges due to complex image acquisition processes and errors caused by varying face shapes, leading to inaccurate illumination detection.
Innovation Solution
A method and apparatus that reconstruct a deformable three-dimensional face mesh model from a standard template, matching it with a face image to determine key feature points' brightness and derive illumination information based on a pre-established relationship between predetermined brightness and illumination, trained using history face images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional image acquisition methods are used to detect illumination in real scenes, then comprehensive illumination information can be obtained, but the operations become complicated and large errors occur due to varying face shapes and reflectivity
Solution Approach 1:
The patent extracts only the necessary illumination information from face images by focusing on key feature points (eyes, nose, mouth) rather than attempting to capture comprehensive scene illumination. This extraction approach simplifies the acquisition process while maintaining detection accuracy by concentrating on discriminative features that are less affected by varying face shapes.
Solution Approach 2:
The patent applies local quality by treating different regions of the face differently - key feature points are selected based on their local characteristics and illumination patterns. Each key point is analyzed individually with its own brightness determination, allowing the system to adapt to local variations in face geometry and lighting conditions rather than applying a uniform approach to the entire face.
2Reliability
If comprehensive image acquisition operations are performed to account for geometry and reflectivity factors, then more complete illumination information can be obtained, but the detection process becomes more complex and error-prone
Solution Approach 1:
The patent segments the face into distinct key feature points (eyes, nose, mouth) and analyzes illumination at each segment independently. This segmentation allows the system to handle geometric variations and reflectivity effects locally at each feature point, improving reliability by focusing on stable, discriminative regions while keeping the overall process simple through modular analysis.
Solution Approach 2:
The patent uses a pre-trained corresponding relationship model that captures the typical illumination patterns of key face features. This copied knowledge from training data allows the system to reliably determine illumination without performing complex real-time analysis of geometric and reflectivity factors, maintaining both reliability and operational simplicity.
Data Source
AI summary
An illumination detection method comprises acquiring a face image to be detected and a three-dimensional face mesh template; deforming the three-dimensional face mesh template according to the face image to obtain a reconstructed face mesh model; according to the deformation positions, in the reconstructed face mesh model, of key feature points in the three-dimensional face mesh template, determining the brightness of feature points, corresponding to the key feature points, in the face image; and according to the relationship between the predetermined brightness of the key feature points and the illumination, and the brightness of the feature points, corresponding to the key feature points, in the face image, determining illumination information of the face image.