Visual Dialog Context Encoding for More Accurate Answers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing visual dialog systems face low accuracy in output answers when input questions carry extensive information due to insufficient integration of historical question answering dialog context.
Innovation Solution
A visual dialog method that combines image features with state vectors from multiple rounds of historical question answering dialog, using multimodal encoding and decoding to improve answer accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If historical question answering dialog context is not integrated, then the system complexity is low, but the answer accuracy deteriorates when input questions carry extensive information
Solution Approach 1:
The system segments the dialog context into discrete state vectors representing different rounds of historical question answering. Each state vector captures specific contextual information, allowing the system to manage complexity through structured division while maintaining comprehensive context integration for improved answer accuracy.
Solution Approach 2:
The system transitions from processing raw dialog text to operating in a vector space dimension, where historical dialogs are transformed into state vectors. This dimensional transformation enables efficient integration of extensive contextual information without proportionally increasing system complexity, as vector operations provide a compact representation mechanism.
2Measurement precision
If multiple rounds of historical dialog are integrated, then the answer accuracy improves, but the information processing complexity increases
Solution Approach 1:
The system creates compact vector representations (copies) of historical dialog information that preserve essential contextual meaning while reducing processing complexity. Instead of analyzing full text of multiple dialog rounds, the system operates on condensed state vector copies that capture the necessary information for accurate answer generation.
3Measurement precision
If contextual information from historical dialogs is combined with image features, then the understanding of implicit image information improves, but the computational requirements increase
Solution Approach 1:
The system performs preliminary encoding of image features into compact representations before integrating them with dialog context. This pre-processing step reduces the dimensional complexity of image data, enabling efficient subsequent integration with historical dialog state vectors while maintaining the ability to accurately understand implicit image information.
Data Source
AI summary
Disclosed in this application are a visual content dialog method performed by an electronic device. The method includes: acquiring an image feature of an input image and state vectors corresponding to first n rounds of historical question answering dialog, n being a positive integer; acquiring a question feature of a current round of questioning related to the input image; performing multimodal encoding on the image feature of the input image, the state vectors corresponding to the first n rounds of historical question answering dialog, and the question feature of the current round of questioning, to obtain a state vector corresponding to the current round of questioning; and performing multimodal decoding on the state vector corresponding to the current round of questioning and the image feature of the input image, to obtain an actual output answer corresponding to the current round of questioning.