Multi-modal continuous emotion recognition method, service reasoning method and system

Abstract

The invention discloses a multi-modal continuous emotion recognition method and a service reasoning method and system. The method comprises the following steps: acquiring video data containing facial expressions and voices of a user; for the video image sequence, extracting face images, carrying out feature extraction on the face images, and obtaining expression emotion features; performing continuous emotion recognition according to the expression emotion features; for the voice data, acquiring voice emotion features; According to the method, continuous emotion recognition is carried out according to voice emotion features, and an expression emotion recognition result and a voice emotion recognition result are fused, so that the defects of a single mode in continuous emotion recognition are overcome, and the emotion recognition precision is improved; on the basis, service reasoning is carried out based on a multi-entity Bayesian network model, so that the service robot can dynamically adjust the service according to the emotion of the user.

Description

technical field [0001] The invention belongs to the technical field of service robots, and in particular relates to a multimodal continuous emotion recognition method, a service reasoning method and a system. Background technique [0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art. [0003] As service robots play an important role in household scenarios, natural human-computer interaction becomes one of the key factors affecting user satisfaction and the comfort of human-machine coexistence. The goal of the home service robot is to have the cognitive ability of the user's emotion and provide high-quality services according to the user's emotional state. [0004] According to the inventor's understanding, the current recognition of human emotions is mainly based on discrete emotion models, but since the expression of human emotions is a complex and continuous process, it i...

AI Technical Summary

Problems solved by technology

[0004] According to the inventor's understanding, the current recognition of human emotions is mainly based on discrete emotion models, but since the expression of human emotions is a complex and continuous process, it is difficult for discrete emotion models to fully express the user's emotional state. Necessary, at the same time, because the continuous emotional state calibration is complex, the data set is scarce, and the single-modal continuous emotion recognition has the disadvantages of low recognition accuracy and poor robustness, so in order to further reduce the impact of the scarcity of data sets and improve the accuracy of emotion recognition In order to enhance the robustness of the recognition system, it is necessary to explore the complementarity between the various modalities and realize the emotion recognition of multi-modal fusion, so as to improve the quality of the final emotion recognition.

[0005] The service target of home service robots is people. At present, the services provided by service robots rarely take into account the current emotional state of users, and the reasoning rules are hard reasoning, which does not take into account that the home environment is dynamically changing and full of various uncertainties. Factors, the inferred service results cannot serve users well, and cannot reflect the intelligence of service robots

Images