Driver Emotion Recognition Using Temporal Facial Image Sequences
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing emotion recognition methods in driving scenarios suffer from low accuracy due to limited information usage, particularly when the driver is alone or without passenger interaction, leading to ineffective audio-based solutions.
Innovation Solution
A method involving a sequence of temporally adjacent images of the driver's face is input into a pre-trained emotion recognition model, utilizing Hidden Markov Model, Conditional Random Field, or Recurrent Neural Network to predict subsequent emotional statuses based on prior and subsequent emotions, enhancing accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If single images are used for emotion recognition, then the system complexity is low, but the recognition accuracy is insufficient
Solution Approach 1:
The system performs preliminary actions by capturing a sequence of images before the actual emotion recognition occurs. These pre-captured images serve as input data that contains temporal information about emotional changes, enabling more accurate recognition without requiring complex real-time processing during the critical moment of emotion detection.
Solution Approach 2:
The patent transitions from analyzing single static images to analyzing sequences of images across the time dimension. By incorporating temporal information from multiple time points, the system adds a new dimension to the analysis, enabling detection of emotional transitions and changes that cannot be captured by single-frame analysis alone.
2Measurement precision
If single images are used for emotion recognition, then the processing time is short, but the recognition accuracy is low
Solution Approach 1:
The system performs preliminary image capture and preprocessing before the actual recognition task. By preparing the sequence of images in advance and organizing them properly, the system reduces the computational burden during the critical recognition phase, maintaining efficiency while improving accuracy through temporal analysis.
Solution Approach 2:
The system maintains continuous capture of facial images over time, creating an uninterrupted sequence that records emotional evolution. This continuous data collection ensures that no critical emotional transitions are missed, while the structured approach to processing this continuous stream maintains computational efficiency.
3Measurement precision
If temporal continuity of emotional states is considered, then the accuracy improves, but the computational complexity increases
Solution Approach 1:
The system segments the temporal sequence of images into discrete time points with specific emotional states. By dividing the continuous temporal data into manageable segments that can be individually analyzed and then integrated, the system reduces computational complexity while preserving the temporal continuity information needed for accurate recognition.
Data Source
Figure 1~2
Figure 3~5
Figure 6~7
AI summary
A method, an apparatus, a device, a medium, and a vehicle for recognizing an emotion of a driver. The method includes: obtaining a sequence of images to be recognized, wherein the sequence of images to be recognized includes at least two images to be recognized that are adjacent in time, and each image to be recognized includes a face of a driver; and inputting the sequence of images to be recognized into an emotion recognition model to obtain an sequence of emotion statuses, output by the emotion recognition model, that corresponds to the sequence of images to be recognized, wherein the emotion recognition model is pre-trained so that an nth emotion status in the sequence of emotion statuses is predicted based on at least one prior emotion status in the sequence of emotion statuses, wherein the at least one prior emotion status precedes the nth emotion status in time and includes an (n - 1)th emotion status, wherein n is an integer and 1 < n ≤ N, and N is a total number of the emotion statuses in the sequence of emotion statuses.