3D Facial Recognition with Partial Occlusion Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing facial recognition systems on mobile devices are hindered by face masks or coverings, as they are designed to recognize uncovered faces, leading to a need for a system that can learn and unlock devices based on partially covered human faces and bodies.
Innovation Solution
An AI-based computer vision system that uses a point detector, geometric feature evaluator, internal calibrator, and depth evaluator to construct a 3D model of a human face or body, capable of identifying unique features with or without face coverings, and can be trained for both facial and body recognition, allowing for secure locking and unlocking of devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional facial recognition systems are used, then device security is maintained, but the system cannot recognize faces when covered by masks or face coverings
Solution Approach 1:
The system transitions from two-dimensional facial image analysis to three-dimensional depth mapping by constructing a depth map from multiple image views. This dimensional transformation allows the system to capture facial geometry and structural relationships that remain identifiable even when facial features are partially obscured by masks, thereby maintaining recognition accuracy while adapting to covered faces
Solution Approach 2:
The system performs preliminary training by capturing multiple images of the user's face from different angles and constructing a reference depth map before actual recognition occurs. This pre-established three-dimensional facial model serves as a template that enables reliable recognition even when the user's face is covered during subsequent authentication attempts
2Ease of operation
If the system requires complete facial visibility for recognition, then recognition accuracy is high, but the system becomes unusable in public places requiring face coverings
Solution Approach 1:
By constructing three-dimensional depth maps from multiple two-dimensional images, the system recovers facial structural information that is not lost even when parts of the face are covered. The depth encoding preserves geometric relationships between visible features, allowing the system to compensate for missing information from covered areas
Solution Approach 2:
The system creates a three-dimensional digital copy or model of the user's facial structure during training. This synthetic depth representation serves as a reference that can be compared against partial views of the face during recognition, enabling the system to operate with incomplete visual information while maintaining ease of use in mask-wearing scenarios
3Device complexity
If the system uses only two-dimensional facial images for recognition, then the system is simple, but it cannot accurately represent facial features when partially covered
Solution Approach 1:
The system enhances two-dimensional image data by constructing three-dimensional depth maps through multi-view geometry and epipolar constraints. This additional dimensional information provides more robust facial feature representation that is less susceptible to occlusion, improving measurement precision without requiring fundamentally complex hardware changes
Solution Approach 2:
The same multi-image processing pipeline that captures color information also generates depth maps, making the imaging system multi-functional. The feature extraction and matching algorithms are extended to operate on both two-dimensional texture and three-dimensional geometry, allowing the system to maintain moderate complexity while achieving superior precision in covered-face scenarios
Data Source
AI summary
Systems and methods for learning and recognizing features of an image are provided. A point detector identifies points in an image where there are two-dimensional changes. A geometric feature evaluator overlays at least one mesh on the image and analyzes geometric features on the at least one mesh. An internal calibrator transforms data from the point detector and the geometric feature evaluator into a three-dimensional point figure of the image, and a depth evaluator determines a final shape of the image. A three-dimensional object model of the image is constructed. The image could be a human face or body. Exemplary systems and methods can construct and learn features of a human face based on a partial view where part of the face is covered. Systems and methods can unlock a mobile device based on recognition of the features of the user's face.