Virtual Eyewear Fitting Using 2D Face Mesh Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for selecting and fitting head-mounted wearable devices, such as eyewear, lack accuracy in virtual fitting due to not accounting for user-specific facial features, leading to potential misfit and functional issues.
Innovation Solution
A method utilizing a reference mesh generated from a large pool of user data, combined with a user-specific mesh, to accurately position a virtual frame on a user's face through a rigid transformation, adjusting for features like the sellion point, enabling precise fitting without physical try-on.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a generic reference mesh is used for virtual fitting, then the fitting process is simplified and faster, but the accuracy of the fit decreases due to lack of user-specific facial features
Solution Approach 1:
The system segments the fitting process into two distinct stages: first mapping to a generic reference mesh for rapid initial positioning, then refining to a user-specific mesh for accurate final fit. This segmentation allows the system to benefit from both the speed of generic templates and the precision of personalized models.
Solution Approach 2:
The system performs preliminary action by first mapping the virtual frame to the generic reference mesh to establish initial positions and orientations. This preliminary positioning serves as a foundation that is subsequently refined using user-specific facial features, combining efficiency with accuracy.
2Measurement precision
If a user-specific mesh is used for virtual fitting, then the fitting accuracy improves, but the complexity of the process increases
Solution Approach 1:
The complex process of creating and mapping to a user-specific mesh is segmented into manageable steps: facial landmark detection, mesh generation, rigid transformation calculation, and virtual frame positioning. This segmentation reduces perceived complexity while maintaining high accuracy.
Solution Approach 2:
The generic reference mesh serves as an intermediary between the simple image input and the complex user-specific mesh. It provides a stable intermediate representation that simplifies the transformation process while enabling accurate mapping to the final personalized model.
3Speed
If facial landmarks are not detected, then the process is faster, but the virtual frame cannot be accurately positioned on user-specific features like the sellion point
Solution Approach 1:
Facial landmark detection is performed as a preliminary action that enables subsequent precise positioning. The detected landmarks serve as reference points that guide the rigid transformation process, ensuring the virtual frame is accurately positioned relative to key facial features like the sellion point.
Solution Approach 2:
The system replaces manual trial-and-error fitting with an automated computer vision system that detects facial landmarks and calculates transformations. This substitution of mechanical/manual processes with computational methods achieves both speed and precision simultaneously.
Data Source
AI summary
A system and method for fitting a head mounted wearable device for a user based on a single two-dimensional image is provided. The image may include the face/head of the user, captured by an image sensor of a computing device, via an application executing on the computing device. A sellion node, of a plurality of nodes of a reference mesh, may be mapped to a sellion node, of a plurality of nodes, of a user mesh. The reference mesh may represent a general head mesh based on data collected from a large pool of users. The user mesh may be generated from the two-dimensional image. A positioning of a virtual frame on the two-dimensional image of the user may be adjusted based on a difference in position of the sellion node of the reference mesh and the sellion node of the user mesh.