Large-Model Speech Interaction Using Attention Fusion for Response Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AIGC models struggle to accurately understand user desires from multimodal information such as speech and video, leading to low matching between generated responses and user intentions, which reduces the efficiency and quality of human-computer interaction.
Innovation Solution
An interactive method using a large model that processes speech recognition features through an attention fusion layer to determine response words, aligning deep semantic understanding and avoiding hallucinations, enhancing the matching degree between responses and requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If AIGC models process multimodal information to generate responses, then user interaction capability is improved, but response accuracy and matching with user intentions deteriorates
Solution Approach 1:
The patent segments the response generation process into multiple stages: first generating candidate response words based on speech recognition features, then selecting the target response word through attention fusion with associated response word features. This segmentation allows the model to handle multimodal information systematically while improving response accuracy through staged processing.
Solution Approach 2:
The patent introduces an attention fusion layer as an intermediary mechanism between the speech recognition features and the response word selection. This intermediary component processes both the current response word feature and the associated response word feature (from previous context) to determine the target response word, thereby improving the matching accuracy with user intentions.
2Stability of the object's composition
If the model generates response words sequentially, then response coherence is improved, but the ability to avoid hallucinations deteriorates
Solution Approach 1:
The patent implements a feedback mechanism where the associated response word feature from the previously generated response word is fed back into the attention fusion layer for processing the current response word. This feedback loop allows the model to continuously adjust its response generation based on previous context, improving both coherence and reliability by avoiding hallucinations through contextual constraint.
Solution Approach 2:
The patent performs preliminary action by pre-processing the associated response word feature from the previous step before using it in the attention fusion layer. This preliminary preparation of contextual information enables the model to make more informed decisions about the current response word, reducing hallucinations while maintaining coherence.
3Measurement precision
If the attention fusion layer processes both current and associated response word features, then semantic understanding depth is improved, but computational complexity increases
Solution Approach 1:
The patent applies partial action by processing only the necessary features (current response word feature and associated response word feature) through the attention fusion layer, rather than processing all possible features simultaneously. This selective processing achieves deep semantic understanding while controlling computational complexity to manageable levels.
Data Source
AI summary
An interactive method based on a large model, a training method, and an intelligent agent, which relate to fields of artificial intelligence, speech recognition, speech interaction, deep learning, large models, and application scenarios of knowledge search, autonomous driving, intelligent customer service, intelligent speech control, smart e-commerce, AI healthcare. The interactive method includes: acquiring a request speech; performing a speech recognition on the request speech to obtain a speech recognition feature representing a request semantics; and processing the speech recognition feature using the large model to obtain a response text, where the response text includes response words arranged in sequence, a target response word among the response words is determined by processing the speech recognition feature and an associated response word feature using an attention fusion layer of the large model, and the associated response word feature is related to an associated response word arranged before the target response word.