Multimodal dialog state processing method, device, medium and computing device

Abstract

Embodiments of the present invention provide a multi-modal dialogue state processing method, device, medium and computing device for man-machine dialogue. The method includes: obtaining the user text features of the current round according to the user input text content of the current round; for each of at least one attribute category to be processed in the current round: for each attribute category corresponding to Preset attribute values ​​to obtain the user visual features and system visual features of the current round; obtain the combined features of the user text features, user visual features and system visual features of the current round, so as to obtain the attribute category in its The corresponding probability distributions on various preset attribute values. The above-mentioned multi-modal dialog state processing method, device, medium and computing device for man-machine dialog of the present invention can accurately understand multi-modal information input, and adaptively integrate these modes to track the dialog state.

Description

Technical field [0001] Embodiments of the present invention relate to the field of electronic information, more specifically, embodiments of the invention relate to a multimode dialog state processing method, apparatus, media, and computing devices for human-computer dialogue. Background technique [0002] With the development of computer and Internet technology, multi-mode dialogue systems are increasingly attached to people, and human-operated manner in a more natural and information. Among them, the multimode in the multi-mode dialogue includes two modalities for text and visual information (such as images). [0003] As one of its core components, the dialogatics tracker matches each step of the conversation, and provides a direct way to verify the ability of the dialogue. [0004] However, existing conversation tracker technology is primarily limited to text modality, which is not easy to scale to capture rich visual information in multi-modal systems. Inventive content [0...

AI Technical Summary

Problems solved by technology

[0005] In this context, the embodiments of the present invention expect to provide a multimodal dialogue state processing method, device, medium and computing device for human-computer dialogue, so as to at least solve the problem that the existing dialogue tracking technology cannot obtain a multimodal system The problem of medium visual information, which leads to poor tracking effect

Images