Facial Landmark Detection via Graph Convolution and Patch Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional facial landmark detection networks are computationally expensive, preventing their real-time implementation on power-limited and area-limited devices, such as mobile devices, which struggle with high-resolution image processing due to high computational complexity and costs measured in floating point operations per second (FLOPS).
Innovation Solution
The implementation of a method using a graph convolution network for efficient facial landmark detection, which generates patches of an image, extracts patch features, and aggregates them to reduce computational load while maintaining accuracy through an attention mechanism, coarse-to-fine mechanism, and error estimation techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional facial landmark detection networks are used, then detection accuracy is maintained, but computational complexity and FLOPS increase significantly
Solution Approach 1:
The image is divided into multiple patches, and facial landmarks are detected independently for each patch rather than processing the entire image at once. This segmentation approach reduces the computational burden while maintaining detection accuracy, as each patch can be processed separately with fewer floating point operations.
Solution Approach 2:
The method extracts and processes only the necessary patch features relevant to facial landmark detection, rather than analyzing the complete image. By taking out and focusing on specific regions containing facial information, the system reduces unnecessary computations while preserving detection precision.
2Measurement precision
If high-resolution image processing is performed, then detection accuracy improves, but power consumption and computational cost increase
Solution Approach 1:
The high-resolution image is segmented into smaller patches for processing. This allows the system to achieve accurate landmark detection on high-resolution images without processing the entire image at full resolution simultaneously, thereby reducing power consumption while maintaining detection quality.
Solution Approach 2:
The method applies partial processing by focusing computational resources only on patches containing facial regions rather than the entire high-resolution image. This partial action approach maintains detection accuracy while significantly reducing the energy required for processing.
3Speed
If real-time processing is implemented, then processing speed increases, but computational resources and accuracy may be compromised
Solution Approach 1:
By segmenting the image into patches, the system can process multiple patches in parallel or sequentially with reduced computational load per patch. This enables real-time processing speed while maintaining accuracy, as each patch requires fewer computational resources than processing the full image.
Solution Approach 2:
The method performs preliminary processing by extracting patch features before aggregate processing. This preliminary action organizes the data in a way that facilitates faster subsequent processing while preserving the information needed for accurate landmark detection.
Data Source
AI summary
Embodiments of the present disclosure enable and accurate detection of facial landmarks on mobile devices in real-time. An architecture of a facial landmark detection model is provided including one or more of an attention mechanism (e.g., an attention network), a graph convolution model (e.g., a two-dimensional facial geometry graph convolution model), a multiscale coarse-to-fine mechanism, a patch-facial landmark detachment mechanism, and error estimation techniques. The attention mechanism may increase the accuracy of the facial landmark detection model by attending to meaningful patches. The graph convolution network may improve patch feature aggregation by considering the facial landmarks' geometry. The coarse-to-fine mechanism reduces a network convergence to two cycles (e.g., two facial landmark detection iterations). A patch-facial landmark detachment mechanism reduces the computation burden without significant accuracy degradation. Error estimation techniques provide accurate estimation of the regression error to the computation load and increase the accuracy of the model.