Graph-to-Text Generation via Two-Step Fine-Tuning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional end-to-end data-to-text generation systems often generate fabricated facts or incorrectly divide triples when using pre-trained language models, and may not fully capture the structure of input data, leading to poor generation quality.
Innovation Solution
The system employs position aware embeddings to capture the structure and role of data, pre-training a generative language model with token, position, triple role, and tree-level embeddings to generate accurate and coherent textual descriptions from structured input data like RDF graphs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If end-to-end data-to-text generation using pre-trained language models is used, then generation speed is improved, but generation quality deteriorates due to fabricated facts and incorrect triple division
Solution Approach 1:
The generation process is divided into two distinct steps: first generating a structured representation (graph or tree) from the knowledge graph, then generating text from this structured representation. This segmentation allows explicit control over the reasoning process while maintaining generation efficiency, resolving the contradiction between speed and quality.
Solution Approach 2:
A structured representation (graph or tree) is introduced as an intermediary between the knowledge graph and the final text output. This intermediary explicitly captures the reasoning process and structural relationships, preventing fabricated facts and incorrect triple division while enabling efficient text generation from the structured form.
2Device complexity
If conventional models simply concatenate triples together, then device complexity is reduced, but information structure is lost leading to poor generation quality
Solution Approach 1:
The system performs preliminary structuring of the knowledge graph into a graph or tree representation before text generation. This preliminary action preserves the structural information and relationships in the data, ensuring that this information is not lost during the generation process while keeping the model relatively simple.
3Reliability
If complicated graph neural networks are used to encode triple structures, then generation quality is improved, but device complexity increases
Solution Approach 1:
The complex task of structure encoding and text generation is segmented into separate stages: first creating a structured representation that preserves relationships, then generating text from this structure using a language model. This avoids the need for complicated graph neural networks while maintaining generation quality.
Solution Approach 2:
Instead of using complex graph neural networks to encode and transform the knowledge graph structure, the system creates a simplified graph or tree representation that copies the essential structural information in a more manageable form, which can then be processed by standard language models.
4Ease of operation
If pre-trained language models are used directly, then ease of operation is improved, but reliability deteriorates due to fabricated facts
Solution Approach 1:
A structured representation (graph or tree) serves as an intermediary that grounds the language model in the actual knowledge graph structure. This intermediary prevents the model from fabricating facts by ensuring that all generated content is based on the explicit structure and relationships in the input data, while still allowing the use of pre-trained language models for text generation.
Data Source
AI summary
Embodiments described herein provide systems and methods for data-to-text generation. The embodiments receive input data that includes a resource description framework (RDF) triples in an RDF graph. A data-to-text generation system generates position aware embeddings, including position embeddings, triple role embeddings, and tree-level embeddings. Using the position aware embeddings and the RDF graph, the data-to-text generation system generates a textual description for the RDF graph.