Document-Level MT Model for Coherent Text Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural natural language generation (NLG) and machine translation (MT) models are insufficient for performing certain NLP tasks, particularly in data-to-text generation, where they lack accuracy, coherence, and adequacy, and fail to effectively utilize larger context beyond sentence-level text.
Innovation Solution
The development of a document-level machine translation model that leverages transfer learning between MT and NLG models, using structured data to generate coherent text by training an augmented document-level MT model to receive language-independent structured data and generate output in a target language, allowing for the use of more extensive training data and context.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If existing neural NLG models are used for data-to-text generation, then fluent text can be generated, but content selection accuracy is weak
Solution Approach 1:
The patent divides the NLG task into two separate modules: a content selection module that identifies important information from input data, and a verbalization module that generates coherent text. This segmentation allows each module to specialize, improving content selection accuracy while maintaining text fluency through the end-to-end training approach.
2Reliability
If sentence-level MT models are used, then translation performance is impressive, but larger context from other sentences or structured information cannot be utilized
Solution Approach 1:
The patent extends the MT model from sentence-level to document-level processing by incorporating additional dimensions: other sentences from the same document and language-independent structured information. This allows the model to maintain impressive translation performance while utilizing larger context through document-level encoding and structured data integration.
3Measurement precision
If document-level MT model with structured data is trained, then NLG accuracy and coherence are enhanced, but model complexity increases
Solution Approach 1:
The patent creates a universal document-level MT model that can handle multiple tasks: machine translation and natural language generation. By training the same model architecture on both MT and NLG tasks with structured data, the system achieves enhanced NLG accuracy and coherence without requiring separate specialized models, thus managing complexity through multi-functionality.
Data Source
AI summary
In methods for training a natural language generation (NLG) model using a processor a document-level machine translation (MT) model is provided by training an MT model to receive as input, token sequences in a first language, and to generate as output, token sequences in a second language. An augmented document-level MT model is provided by training the document-level MT model to receive as input, paired language-independent structured data and token sequences in the first language, and to generate as output, token sequences in the second language. The augmented document-level MT model is trained to receive as input, language-independent structured data, and to generate as output, token sequences in the second language.