A cloud-based multimodal dialogue method, device and system for a home service robot

Abstract

The invention discloses a cloud multi-modal dialogue method, device and system for a home service robot. The method includes: receiving user voice information and scene image information in real time; converting the user voice information into text information, and performing word segmentation and named entity recognition processing , determine the dialogue type; preprocess the scene image information; filter the preprocessed image information and the processed text information, add the incremental knowledge base, and perform model incremental training when the dialogue service load rate is less than the threshold value, and update the dialogue service model; Perform scene perception, user perception and emotion perception on the preprocessed image information and / or processed text information respectively, and obtain scene perception information, user perception information and emotion perception information; combine the processed text information according to different dialogue types Scene perception information, user perception information and / or emotion perception information, through the dialogue service model, generate dialogue information for the interaction between the user and the robot.

Description

technical field [0001] The disclosure belongs to the technical field of home service robot voice dialogue, and relates to a home service robot cloud multi-modal dialogue method, device and system. Background technique [0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art. [0003] In recent years, problems such as population aging have become increasingly prominent, the urgent needs of society and the rapid development of artificial intelligence and robotics technology have greatly promoted the development of service robots. Voice dialogue technology provides the most humanized interaction method for home service robots. However, in the current human-computer voice interaction, the robot has no perception of scene information and interactive object information, often answers questions that are not relevant, and cannot generate satisfactory answers to users, which is very in...

AI Technical Summary

Problems solved by technology

[0004] (1) The traditional home service robot dialogue system only uses the user's voice interaction information, has no perception of the robot scene information, and has the problem of dialogue strategy deviation caused by the lack of scene information, and the generated dialogue cannot correspond to the user interaction content. poor experience

[0005] (2) The traditional home service robot dialogue system generates similar answers for the same voice input information of different users, and cannot generate personalized answers based on different user emotions, interests, etc.

[0006] (3) The traditional cloud-based family service person dialogue system has caused a lot of waste of time and computing power for massive user data and model update training methods, and the resource utilization rate is low

[0007] The above deficiencies lead to problems such as poor user experience in human-computer interaction and utilization of cloud resources in the home service robot dialogue system, hindering the application and promotion of home service robots

Images