Large-Model Speech Interaction With Attention Fusion for Intent Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AIGC models struggle to accurately understand user desires expressed through multimodal information such as speech and video, leading to low matching between generated responses and user intentions, which reduces interaction efficiency and user experience.
Innovation Solution
An interactive method and apparatus using a large model with an attention fusion layer to process speech recognition features, determining target response words by aligning them with associated response word features, enhancing deep semantic understanding and reducing hallucinations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If existing AIGC models process multimodal information to generate responses, then the system can provide automated interactions, but the accuracy of understanding user desires and matching responses to user intentions deteriorates
Solution Approach 1:
The model segments the response generation process into distinct stages: understanding user desires from multimodal input, generating candidate response words, and selecting target response words through attention fusion. This segmentation allows each stage to be optimized independently, improving overall accuracy while maintaining automation.
Solution Approach 2:
The patent introduces an attention fusion layer as an intermediary mechanism between the speech recognition features and the response word selection. This intermediary processes and aligns deep semantic features, enabling more accurate matching between user intentions and generated responses while preserving automated interaction capabilities.
2Speed
If existing AIGC models generate responses based on multimodal input, then interaction speed is maintained, but the matching degree between responses and user intentions deteriorates
Solution Approach 1:
The model performs preliminary processing of speech recognition features to extract deep semantic representations before response word generation. By preparing the semantic understanding in advance through the attention fusion layer, the system can quickly generate accurate responses without sacrificing interaction speed.
Solution Approach 2:
The patent transforms the speech recognition features into aligned deep semantic features through the attention fusion mechanism. This parameter transformation enables the model to maintain fast interaction while significantly improving the matching degree between responses and user intentions by changing how semantic information is represented and processed.
3Measurement precision
If the large model uses attention fusion layer to align deep semantic features, then the accuracy of response text generation is improved, but the computational complexity increases
Solution Approach 1:
The attention fusion layer applies local quality enhancement by focusing computational resources on aligning specific deep semantic features that are most critical for response accuracy. Rather than uniformly processing all features, the mechanism identifies and prioritizes key semantic alignments, improving accuracy while managing computational complexity.
4Reliability
If the model processes speech recognition features through attention fusion layer, then hallucinations are reduced, but the processing time increases
Solution Approach 1:
The attention fusion mechanism incorporates feedback loops that continuously align generated response words with the original speech recognition features. This feedback ensures that responses remain faithful to user intentions, reducing hallucinations. The iterative alignment process is optimized to minimize additional processing time while maintaining high reliability.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
An interactive method and apparatus based on a large model, a training method and apparatus, an intelligent agent, an electronic device, a storage medium, and a program product, which relate to the field of artificial intelligence, particularly in the technical fields such as speech recognition, speech interaction, deep learning, and large models, and may be applied to application scenarios such as knowledge search, autonomous driving, intelligent customer service, intelligent speech control, smart e-commerce, and AI healthcare, are provided. The interactive method based on the large model includes: acquiring a request speech; performing a speech recognition on the request speech to obtain a speech recognition feature representing a request semantics; and processing the speech recognition feature using the large model to obtain a response text, where the response text includes a plurality of response words arranged in sequence, a target response word among the plurality of response words is determined by processing the speech recognition feature and an associated response word feature using an attention fusion layer of the large model, and the associated response word feature is related to an associated response word arranged before the target response word.