Multimodal continuous emotion recognition method, service reasoning method and system

Abstract

The invention discloses that the invention provides a multi-modal continuous emotion recognition method, a service reasoning method and a system. The method includes: acquiring video data including user facial expressions and voice; for video image sequences, extracting human face images, performing feature extraction on the human face images to obtain expression and emotion features; performing continuous emotion recognition according to the expression and emotion features; Data, to obtain voice emotion features; according to voice emotion features, continuous emotion recognition is performed, and the results of expression emotion recognition and voice emotion recognition are fused together, which overcomes the shortcomings of single-modal continuous emotion recognition and improves the accuracy of emotion recognition; On this basis, service reasoning is performed based on the multi-entity Bayesian network model, so that the service robot can make dynamic adjustments to the service according to user emotions.

Description

technical field [0001] The invention belongs to the technical field of service robots, and in particular relates to a multimodal continuous emotion recognition method, a service reasoning method and a system. Background technique [0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art. [0003] As service robots play an important role in home scenarios, natural human-robot interaction has become one of the key factors affecting user satisfaction and human-robot coexistence comfort. The goal of home service robots is to have cognitive ability to users' emotions and provide high-quality services according to the user's emotional state. [0004] As far as the inventors know, the current recognition of human emotions is mainly discrete emotion models. However, since the expression of human emotions is a complex and continuous process, it is difficult for discrete emotion models t...

AI Technical Summary

Problems solved by technology

[0004] According to the inventor's understanding, the current recognition of human emotions is mainly based on discrete emotion models, but since the expression of human emotions is a complex and continuous process, it is difficult for discrete emotion models to fully express the user's emotional state. Necessary, at the same time, because the continuous emotional state calibration is complex, the data set is scarce, and the single-modal continuous emotion recognition has the disadvantages of low recognition accuracy and poor robustness, so in order to further reduce the impact of the scarcity of data sets and improve the accuracy of emotion recognition In order to enhance the robustness of the recognition system, it is necessary to explore the complementarity between the various modalities and realize the emotion recognition of multi-modal fusion, so as to improve the quality of the final emotion recognition.

[0005] The service target of home service robots is people. At present, the services provided by service robots rarely take into account the current emotional state of users, and the reasoning rules are hard reasoning, which does not take into account that the home environment is dynamically changing and full of various uncertainties. Factors, the inferred service results cannot serve users well, and cannot reflect the intelligence of service robots

Images