Attention-Guided Text Generation for User-Defined Content Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text summarization methods, particularly abstractive summarization using seq2seq models, struggle with accurately generating text that aligns with user-defined content expectations due to insufficient attention to relevance and context, leading to suboptimal accuracy and controllability.
Innovation Solution
A method and apparatus that utilize pre-trained encoders and decoders to generate text by determining word and sentence attention weights based on hidden state vectors, element tags, and context features, ensuring relevance to user-defined content expectations through clustering and feature extraction techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If abstractive summarization using seq2seq models is used, then the ability to generate refined text is improved, but the accuracy and controllability regarding user-defined content expectations deteriorates
Solution Approach 1:
The patent introduces element attention weights that provide feedback mechanisms to guide the decoder in generating text aligned with user-defined content expectations. The element attention module calculates attention weights for different elements (words, phrases, sentences) based on their relevance to the element tag, creating a feedback loop that improves alignment between generated text and user requirements.
Solution Approach 2:
The patent changes the parameter space by introducing element tags and element attention weights as additional control parameters. Instead of only using standard word-level attention, the system incorporates element-level attention parameters that can be adjusted to control the relevance and coherence of generated text with respect to user-defined content expectations.
2Adaptability or versatility
If copying mechanism is introduced to generate new words, then the vocabulary flexibility is improved, but the coherence and relevance to original text deteriorates
Solution Approach 1:
The patent applies local quality by differentiating between different levels of text elements (words, phrases, sentences) and applying different attention mechanisms to each. The element attention module selectively focuses on specific local elements that are most relevant to the element tag, allowing the system to maintain coherence with the original text while still generating flexible and varied output.
Data Source
AI summary
A method and apparatus for generating text. The method comprises: receiving original text and an element tag (201); generating an encoded text feature vector, and an encoded sentence feature vector of each sentence in the original text; executing decoding steps comprising: determining a word attention weight of each word in the original text at the current moment on the basis of a hidden state vector of the decoder at the current moment and the encoded text feature vector (2031); determining a sentence attention weight of each sentence in the original text; determining a normalized word attention weight of each word in the original text at the current moment; and estimating a target word at the current moment; and generating target text on the basis of the target word output by the decoder at each moment (204).