Neural Text Generation Using Relevant Reference Sentences
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text generation techniques based on neural networks do not effectively incorporate information as text when generating summaries, relying instead on importance scores and length embeddings.
Innovation Solution
A text generation apparatus that acquires a predetermined number of relevant sentences from a set of sentences different from the input and generates text based on both the input and reference sentences using neural networks, specifically utilizing Bidirectional Encoder Representations from Transformers (BERT) and a Transformer-based pointer generator model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If information is provided as importance scores or length embeddings, then the neural network can process it numerically, but the information cannot be added as text
Solution Approach 1:
The patent introduces an intermediary text generation model that converts structured information (importance scores, length embeddings) into natural language text. This intermediary component acts as a bridge between the numerical representations and the text-based generation process, allowing the model to incorporate contextual information in human-readable form while maintaining the numerical processing capabilities of the neural network.
Solution Approach 2:
The patent transforms the representation parameters of input information by converting fixed numerical formats (importance scores, length embeddings) into variable text-based representations. This parameter change allows the system to flexibly incorporate different types of information as text while maintaining compatibility with the text generation architecture through learned parameter transformations.
2Loss of information
If only input sentence is used for generation, then the process is simple, but important contextual information from other sentences is lost
Solution Approach 1:
The patent applies preliminary action by pre-selecting and ranking sentences based on their relevance to the input sentence before the main generation process. The attention mechanism pre-processes the corpus to identify and prioritize important contextual sentences, making this information readily available during text generation without adding significant complexity to the overall system architecture.
Solution Approach 2:
The patent implements feedback through the attention mechanism, which continuously evaluates the relevance of different sentences in the corpus based on the input sentence. This feedback loop allows the model to dynamically adjust which contextual information to incorporate, ensuring that only the most relevant information is used while maintaining a relatively simple selection process through learned attention weights.
Data Source
AI summary
A text generation apparatus includes a memory and a processor configured to, based on learned parameters of neural networks, acquire, as a reference text, a predetermined number of two or more sentences having a relatively high relevance to an input sentence from a set of sentences different from the input sentence and generate text based on the input sentence and the reference text, such that information to be considered when generating text can be added as text.