Facial Expression Recognition via Segmented Key Point Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning-based expression recognition methods require a large number of labeled samples, are computationally intensive, and cannot be efficiently run on ordinary front-end devices or mobile terminals in real time, making them impractical for real-world human-computer interaction.
Innovation Solution
The proposed method involves a key point learning model with a backbone network and a key point recognition network to extract facial expression and feature information, using multi-layer perceptron models for emotion recognition and fully connected layers for local expression recognition, allowing for efficient feature extraction and reduced data requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning-based methods are used for expression recognition, then recognition accuracy is improved, but computational complexity and time complexity increase significantly
Solution Approach 1:
The patent segments the expression recognition task into two independent parts: emotion recognition (global facial expression) and local expression recognition (specific facial features like eyebrows, eyes, mouth). Each part uses a simplified model trained on specific data, avoiding the need for a single complex deep learning model to handle all aspects, thus reducing computational complexity while maintaining accuracy.
Solution Approach 2:
The patent extracts and processes different types of facial information through separate pathways: global facial features for emotion recognition and local key point features for local expression recognition. This extraction approach allows each pathway to use optimized simplified models rather than relying on complex deep learning architectures.
2Measurement precision
If deep learning-based methods are used for expression recognition, then recognition performance is improved, but the amount of training data and labeling cost increase
Solution Approach 1:
The patent divides the training data into two specialized datasets: one for emotion recognition and another for local expression recognition. Each dataset is optimized for its specific task, requiring fewer total samples compared to a comprehensive deep learning dataset that must cover all expression types and variations.
Solution Approach 2:
The patent extracts only the necessary features for each recognition task from the training data. For emotion recognition, only global facial features are extracted and labeled, while for local expression recognition, only relevant key point features are extracted. This selective extraction reduces the overall labeling burden and data requirements.
3Measurement precision
If deep learning-based methods are used for expression recognition, then recognition capability is improved, but real-time processing on mobile devices becomes difficult
Solution Approach 1:
The patent segments the recognition system into two lightweight modules that can be independently optimized for mobile deployment. Each module uses simplified algorithms rather than heavy deep learning architectures, enabling real-time processing on mobile devices while maintaining recognition capability.
Solution Approach 2:
The patent extracts only the essential features needed for recognition, processing them through simplified models that require minimal computational resources. This allows the system to run in real-time on mobile devices with limited processing power, memory, and energy resources.
4Measurement precision
If deep learning-based methods are used for expression recognition, then recognition accuracy is improved, but network structure complexity increases
Solution Approach 1:
The patent segments the network structure into two simple, dedicated networks: one for emotion recognition and another for local expression recognition. Each network has a straightforward architecture optimized for its specific task, avoiding the need for a single complex multi-purpose deep learning network.
Solution Approach 2:
The patent extracts and processes different feature types through separate simplified network pathways. The global feature extraction network and local key point network are both structurally simple, with each network containing only the necessary layers and operations for its specific recognition task.
Data Source
AI summary
An electronic device obtains an image that includes a face. The device performs feature extraction on the image, to obtain facial expression information corresponding to the face and facial feature information corresponding to the facial expression, wherein the facial feature information indicates an extent of the facial expression. The device determines facial emotion information according to the facial expression information. The device also determines facial feature expression information according to a target feature value corresponding to the facial emotion and the facial feature information. This expression recognition techniques disclosed herein can implement multi-task learning and reduce an amount of data required for model training, and can obtain both an emotion recognition result and a local expression recognition result, thereby improving efficiency and real-time performance of expression recognition and improving user experience.