Supervised Descent Method for Facial Landmark Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for facial feature detection and tracking in images and videos lack accuracy, robustness, and are too slow for real-time applications, particularly failing to handle asymmetric expressions and deformable objects like faces effectively.
Innovation Solution
The Supervised Descent Method (SDM) formulates facial feature detection and tracking as a non-linear least squares problem, learning a sequence of descent directions to minimize the mean of NLS functions without computing the Jacobian or Hessian, enabling efficient real-time processing and long-term tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional landmark localization algorithms are used on clustered shape subspaces, then facial feature detection can be performed, but the method is slow because it has to search for each landmark along the diagonal
Solution Approach 1:
The patent pre-computes and stores descent directions and step sizes during an offline training phase. These pre-computed values are then directly applied during real-time facial feature detection without requiring iterative optimization, eliminating the slow diagonal search process while maintaining accurate landmark localization.
Solution Approach 2:
The patent divides the facial feature detection process into two distinct phases: an offline training phase where descent directions and step sizes are pre-computed, and an online detection phase where these pre-computed values are applied. This segmentation allows computationally intensive operations to be performed once rather than repeatedly during real-time processing.
2Reliability
If 2nd order descent methods are used to solve non-linear least squares problems, then robust and reliable nonlinear optimization is achieved, but the Hessian might be large and not positive definite
Solution Approach 1:
The patent pre-computes descent directions and step sizes during an offline training phase using 2nd order descent methods, storing these values for later use. This eliminates the need to compute large Hessian matrices during real-time processing while maintaining the robustness of 2nd order optimization methods.
Solution Approach 2:
The patent replaces the traditional iterative 2nd order descent optimization process (which requires computing and inverting large Hessian matrices) with a simplified process that applies pre-computed descent directions and step sizes. This substitution maintains optimization robustness while dramatically reducing computational complexity during real-time processing.
3Measurement precision
If richer and more discriminative features are added to improve detection accuracy, then measurement precision improves, but the computational load increases making real-time processing difficult
Solution Approach 1:
The patent pre-computes descent directions and step sizes using rich and discriminative features during an offline training phase. By performing this computationally intensive work in advance, the system can use sophisticated features to improve detection accuracy while maintaining real-time processing capability during online detection.
Data Source
AI summary
The present invention relates to a system for detecting and tracking facial features in images and can be used in conjunction with a camera. Given a camera, the system will detect facial landmarks in images. The present invention includes software for real time, accurate facial feature detection and tracking in unconstrained images and videos. The present invention is better, more robust and faster than existing approaches and can be implemented very efficiently allowing real-time processing, even on low-power devices, such as mobile phones.