Drowsy Facial Expression Detection Under Changing Illumination

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing systems for detecting drowsy facial expressions in vehicle drivers face challenges under changing illumination conditions, as they require precise positioning of the eyes and are unreliable due to variations in light intensity and facial movements.

Innovation Solution

A method using an image sensor to capture a face image, detect the face region with a pattern classification algorithm, generate a 3D face model with facial muscles, and determine photometric and geometric flows to identify landmark points and remove noise caused by illumination variations, allowing for accurate detection of facial expressions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If digital imaging systems require precise positioning of the eye to obtain 10 points of interest within the palpebral fissure, then measurement precision is improved, but device complexity and ease of operation worsen under normal driving conditions

Engineering Contradiction:

Improveprecision of eye positioningVSAvoidfeasibility under normal driving conditions

Core Design Contradiction:

Measurement precisionVSEase of operation

Solution Approach 1:

The system changes the parameter of interest from precise eye positioning to facial landmark points that are easier to detect. Instead of requiring 10 points within the palpebral fissure, the system identifies 68 landmark points across the entire face using an active appearance model, making the system more feasible for normal driving conditions while maintaining measurement capability

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The system segments the face into multiple regions and identifies landmark points across different facial features (eyes, nose, mouth, cheeks) rather than concentrating all measurement points in one difficult-to-access region. This segmentation approach distributes the measurement burden across easier-to-detect facial areas

Inventive Principle:

Principle #1Segmentation

2Device complexity

If digital imaging systems monitor vertical eye width to determine drowsiness, then measurement simplicity is improved, but reliability worsens due to variations in light intensity, eyeglass reflections, and facial movements

Engineering Contradiction:

Improvesimplicity of measurementVSAvoiddetection reliability under varying conditions

Core Design Contradiction:

Device complexityVSReliability

Solution Approach 1:

The system introduces an active appearance model as an intermediary that captures facial geometry and texture information. This model serves as a mediator between the raw image data and the drowsiness detection, allowing the system to account for variations in lighting, reflections, and facial movements by comparing actual facial appearance against the learned model

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system changes from monitoring a single parameter (vertical eye width) to monitoring multiple parameters including positions of 68 landmark points, facial muscle movements, and photometric flows. This multi-parameter approach increases reliability by providing redundant information that can compensate for failures in individual measurements

Inventive Principle:

Principle #35Parameter changes

3Adaptability or versatility

If systems attempt to detect facial expressions under changing illumination conditions, then adaptability is improved, but measurement precision worsens due to noise from illumination variations

Engineering Contradiction:

Improveperformance under varying illuminationVSAvoidaccuracy of facial expression detection

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The system converts the harmful effect of illumination variations into a useful signal by computing photometric flows that capture temporal changes in facial appearance. By analyzing how facial regions change over time rather than their absolute appearance, the system transforms illumination noise into meaningful information about facial muscle movements and expressions

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Solution Approach 2:

The system uses feedback from the active appearance model to continuously refine landmark point detection and facial expression analysis. The model provides expected facial geometry and texture patterns that serve as a reference, allowing the system to correct for illumination variations by comparing actual observations against model predictions

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS8369608B2System and method for detecting drowsy facial expressions of vehicle drivers under changing illumination conditions

Publication Date: 2013.02.05 TOYOTA MOTOR CO LTD

AI summary

The present invention includes a method of detecting drowsy facial expressions of vehicle drivers under changing illumination conditions. The method includes capturing an image of a person's face using an image sensor, detecting a face region of the image using a pattern classification algorithm, and performing, using an active appearance model algorithm, local pattern matching to identify a plurality of landmark points on the face region of the image. The method also includes generating a 3D face model with facial muscles of the face region, determining photometric flows from the 3D face model using an extract photometric flow module, determining geometric flows from the 3D face model using a compute geometric flow module, determining a noise component generated by varying illuminations by comparing the geometric flows to the photometric flows, and removing the noise component by subtracting two photometric flows.