Dual-mode emotion identification method and system based on facial expression and eyeball movement

Abstract

The present invention relates to a dual-mode emotion identification method and system based on a facial expression and an eyeball movement. The method comprises a step of performing acquisition, a step of extracting a facial expression feature vector, a step of extracting an eyeball movement feature vector, a step of performing qualitative analysis on an emotional state, a step of performing matching-by-time and storage, a step of performing fusion and classification and a step of comparing emotional information. According to the method and system provided by the present invention, facial expression information of a to-be-tested object can be dynamically and accurately extracted and analyzed, and a correlation between the facial expression and the emotion is established; rich eye movement information can be accurately and efficiently acquired by means of tracking of an eye tracker, and the emotional state of the to-be-tested object is analyzed from the angle of the eyeball movement; and the facial expression feature vector and the eyeball movement feature vector are processed by using an SVR, so that the emotional state of the to-be-tested object can be obtained more accurately, and thus accuracy and reliability of emotion identification are improved.

Description

technical field [0001] The invention belongs to the field of emotion recognition, and more specifically relates to a dual-mode emotion recognition method based on facial expressions and eye movements. Background technique [0002] With the rapid development of information technology and the increasing dependence of human beings on robots, the ability of human-computer interaction has been widely valued. The current research on emotion recognition is divided into two categories at home and abroad, one is based on single-modal emotion recognition, and the other is based on multi-modal emotion recognition. Single-modal emotion recognition is to identify the emotional state of the subject by collecting information from a certain channel. Multimodal emotion recognition is to analyze the information collected by multiple channels, and finally obtain the emotional state of the testee more accurately through a series of technical means. But there are some obvious deficiencies in t...

AI Technical Summary

Problems solved by technology

Wang Jing uses the human eye as a feature point to solve the positioning problem of automatic face recognition, but it only uses the human eye as a feature point to assist the positioning of the face image, and does not use it as an independent modality for emotion recognition

Lu Yanpeng uses pupils to locate facial images, and uses the changes in the center of the eyes in consecutive frames as an aid for emotion recognition to judge whether the subject is nodding or shaking his head to judge whether the subject is positive or negative, but it does not No in-depth research on the trajectory of eyeballs, etc.

Images