Headset Facial Expression Recognition via Infrared and External Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional facial expression recognition technologies face challenges in accurately recognizing expressions, especially when users wear opaque devices or have curved facial features, and struggle with detecting skin deformation in the vertical direction, leading to reduced accuracy and user satisfaction.
Innovation Solution
A head-mounted display apparatus equipped with a sensing unit that measures infrared diffusion in both horizontal and vertical directions using a light emitter and receiver, combined with an image acquiring unit outside the apparatus to collect expression information around the eyes and mouth, allowing for accurate expression recognition through feature points and three-dimensional image acquisition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If an RGB-D camera is installed in a pipeline connected to a head mounted display apparatus to measure expression around the mouth, then expression information around the mouth can be collected, but the weight of the head mounted display increases and the center of gravity shifts forward, reducing user satisfaction
Solution Approach 1:
The system divides expression recognition into two separate components: (1) a head-mounted sensing unit that collects expression information around the eyes using infrared sensors, and (2) an external image acquiring unit (separate from the head-mounted display) that collects expression information around the mouth. This segmentation allows the head-mounted device to remain lightweight while still gathering comprehensive facial expression data through collaboration with the external unit.
Solution Approach 2:
The patent introduces an external image acquiring unit as an intermediary component that handles mouth region imaging, allowing the head-mounted display to focus on eye region sensing without the burden of carrying heavy camera equipment. The external unit acts as a mediator that complements the head-mounted sensing unit to achieve complete facial expression recognition.
2Measurement precision
If a deformation sensing unit is installed in the head mounted display apparatus to contact the skin of the user, then expression recognition can be achieved, but it cannot fully recognize the expression of curved parts of the face due to difficulty in fully contacting the skin
Solution Approach 1:
The patent replaces mechanical contact-based deformation sensing with optical-based infrared sensing. The sensing unit uses infrared light emitters and receivers to detect skin surface changes and expression information around the eyes without requiring physical contact with the user's face, thereby overcoming the limitation of unable to contact curved facial surfaces.
Solution Approach 2:
The system changes the detection parameter from mechanical deformation (requiring physical contact) to optical property changes (infrared light transmission and reflection). By measuring changes in infrared light properties as the skin surface deforms during expression, the system achieves accurate expression recognition without mechanical contact.
3Measurement precision
If conventional facial expression recognition technology uses only expression information of the entire face or partial information around the mouth, then the system is simple, but the recognition accuracy is low
Solution Approach 1:
The patent combines multiple sensing approaches: (1) infrared-based expression detection around the eyes using the head-mounted sensing unit, and (2) image-based expression detection around the mouth using the external image acquiring unit. By merging these complementary detection methods and combining the information from both eye and mouth regions, the system achieves high-accuracy expression recognition without excessive complexity.
Solution Approach 2:
The system applies different sensing technologies to different facial regions based on their specific characteristics: infrared sensing is applied to the eye region where subtle muscle movements occur, while image acquisition is applied to the mouth region for capturing larger expression movements. This localized approach optimizes recognition accuracy for each facial region.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enhances expression recognition speed and accuracy by simultaneously measuring and combining expression information from multiple angles, improving recognition even when partial face coverage occurs, such as with masks or headsets, and effectively addressing limitations of previous technologies.
Implementation Method 1
a sensing unit installed inside of said apparatus for sensing expression information around the eyes in at least one of a contact and a non-contact manner; wherein the sensing unit measures a difference in infrared diffusion in a horizontal direction and a vertical direction depending on tensile strength of the skin
Data Source
AI summary
The present invention discloses an expression recognition apparatus and methods using a head mounted display thereof. A head mounted display apparatus for performing expression recognition according to an embodiment of the present invention comprises a sensing unit including at least one of expression detection sensing units installed inside of said apparatus for sensing expression information around eyes by at least one of a contact or a non-contact manner; an image acquiring unit installed outside of said apparatus for collecting expression information around a mouth; and an acquisition unit for information of expressions for collecting expression information around the eyes and expression information around the mouth.