Adapting Large Language Models with Knowledge Graphs for Industrial Factual Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-scale language models lack factual correctness in their generated text, leading to inaccuracies such as incorrect historical events.
Innovation Solution
A method and system for adapting a large-scale language model to an industrial domain by using a graph database to store a domain-specific knowledge graph, converting it into a natural language corpus, and pre-training the model with this corpus to enhance factual correctness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a large-scale language model is pre-trained on general internet text, then it can generate human-like language consistently, but it lacks factual correctness in generated statements
Solution Approach 1:
The patent introduces a knowledge graph as an intermediary component between the language model and factual information. The knowledge graph stores structured domain knowledge (e.g., industrial knowledge graphs with entities, attributes, and relationships) and serves as a mediator to guide the language model in generating factually correct statements. The system retrieves relevant knowledge from the graph and uses it to constrain or correct the model's generations, resolving the contradiction between fluent generation and factual accuracy.
Solution Approach 2:
The patent performs preliminary action by pre-training or fine-tuning the language model on domain-specific corpora derived from knowledge graphs before deployment. This preliminary training on structured, fact-based data establishes a foundation of domain knowledge in the model, enabling it to generate more accurate statements in the target domain while maintaining its natural language generation capabilities.
2Reliability
If domain-specific knowledge is integrated into the language model, then factual correctness improves, but system complexity increases
Solution Approach 1:
The patent segments the system into distinct functional modules: a language model for generation, a knowledge graph for knowledge storage, and a retrieval mechanism for connecting them. This segmentation allows each component to specialize in its function while working together through well-defined interfaces, managing complexity through modular architecture rather than monolithic integration.
Solution Approach 2:
The patent transitions from integrating knowledge directly within the model parameters (one dimension) to organizing knowledge in a separate structured space (another dimension - the knowledge graph). This dimensional separation allows the system to access domain knowledge without permanently altering the model's internal structure, reducing complexity while maintaining factual correctness.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
For adapting a large-scale language model (LLM) to an industrial domain, a graph database stores a domain-specific knowledge graph (KGD) containing instance-level semantic information about the industrial domain, including true triple statements about physical entities from the industrial domain and their interrelation. A knowledge graph to corpus translator (KG2CT) converts the domain-specific knowledge graph (KGD) into a natural language corpus (NLC). A domain adaptation component (DAC) pre-trains the large-scale language model (LLM) with the natural language corpus (NLC) to a provide a domain-adapted large-scale language model (LLMD). This approach does not require any assumptions on the structure of the domain-specific knowledge graph, for example, it does not require the domain-specific knowledge graph to contain only instance data. Neither does it need linguistic templates since the verbalization of triples can be built on their existing labels or relation names. The triple-based information encoding of RDF facilitates a natural translation into language during the converting operation. Advantageous embodiments create coherent, contextualised documents which describe local areas of the domain-specific knowledge graph. This increases the information content of individual inputs compared to systems which convert e.g. single triples or random samples of individual triples.