Multi-modal driver emotion feature recognition method and system

Abstract

The invention relates to the field of multi-modal recognition, in particular to a multi-modal emotion feature recognition method and system for driver emotion judgment, and the core method is that the information is recognized through a recognition module, including the steps of performing data preprocessing on the visual information and voice information; respectively forming visual identification information and voice identification information; inputting the visual recognition information and the voice recognition information into a facial expression feature recognition model and a voice emotion feature recognition model under vision respectively to obtain a visual feature vector and a voice feature vector respectively, and inputting the visual feature vector and the voice feature vector into a bimodal emotion feature recognition model to obtain a bimodal emotion feature recognition model; and obtaining an emotion recognition result of decision-level fusion. According to the method, the technical problems of insufficient representativeness of multi-modal features and lengthy fusion algorithm in a traditional algorithm are effectively solved.

Description

technical field [0001] The invention relates to the field of multimodal recognition, in particular to a multimodal emotional feature recognition method and system for driver emotional judgment. Background technique [0002] With the continuous development of autonomous driving technology, vehicles are becoming more and more intelligent, but the driver's attention while driving is unconsciously decreased due to the vehicle's ability, especially for those operations that require a high degree of concentration Inattention or emotional instability is one of the important reasons for all kinds of accidents. Negative emotions such as anger, anxiety, sadness, etc. can seriously affect their concentration and lead to a decrease in the level of operation. Therefore, detecting the emotional state of such drivers in time is an effective defense method to avoid accidents. Therefore, some researchers propose to use machine learning, neural network, deep learning and other methods to st...

AI Technical Summary

Problems solved by technology

[0004] The present invention aims at improving the lack of representativeness of the multimodal features in the prior art and the redundancy of the fusion algorithm, and provides a fusion algorithm that can accurately identify the multimodal features of the driver's emotions and can be implemented with less computing power. Realized emotional feature recognition method and system

Images