Multimodal dialogue state processing method and device, medium and computing equipment

Abstract

The embodiment of the invention provides a multi-mode conversation state processing method and device for man-machine conversation, a medium and computing equipment. The method comprises the steps ofobtaining user text features of a current round according to user input text content of the current round; Aiming at each of at least one attribute category to be processed in the current round: for each preset attribute value corresponding to the attribute category, obtaining a user visual feature and a system visual feature in the current round; And obtaining combined characteristics of the usertext characteristics, the user visual characteristics and the system visual characteristics of the current round, so as to obtain probability distribution of the attribute category on a plurality ofcorresponding preset attribute values based on the combined characteristics. According to the multi-modal dialogue state processing method and device for man-machine dialogue, the medium and the computing equipment, multi-modal information input can be accurately understood, and the modals are adaptively integrated to track the dialogue state.

Description

technical field [0001] Embodiments of the present invention relate to the field of electronic information, and more specifically, embodiments of the present invention relate to a method, device, medium, and computing device for processing multimodal dialog states for man-machine dialog. Background technique [0002] With the development of computer and Internet technology, more and more people pay more and more attention to multi-modal dialogue system, which can carry out human-computer interaction in a more natural and informational way. Among them, the multimodality in the multimodal dialogue system includes two modalities of text and visual information (such as image). [0003] As one of its core components, the dialog state tracker matches the user's goals at every step of the dialog and provides a straightforward way to verify the ability to understand the dialog. [0004] However, existing dialog tracker techniques are mainly limited to textual modalities, which are n...

AI Technical Summary

Problems solved by technology

[0005] In this context, the embodiments of the present invention expect to provide a multimodal dialogue state processing method, device, medium and computing device for human-computer dialogue, so as to at least solve the problem that the existing dialogue tracking technology cannot obtain a multimodal system The problem of medium visual information, which leads to poor tracking effect

Images