External Model Knowledge Injection for Current LLM Answers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) struggle to provide accurate answers to questions about real-time knowledge or previously unknown information due to their 'frozen' memory at the time of training, and retraining or fine-tuning these models is costly and can lead to the 'catastrophic forgetting' problem.
Innovation Solution
Augmenting LLMs with an external model, known as an external knowledge injector, which introduces real-time knowledge without modifying the original parameters, allowing the external model to overwrite incorrect or outdated responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LLMs are retrained or fine-tuned with new knowledge, then the knowledge accuracy is improved, but the device complexity and computational cost increase significantly
Solution Approach 1:
The patent segments the knowledge update process into two independent components: the frozen LLM that retains general language capabilities and the external knowledge injector that handles specific knowledge updates. This segmentation allows knowledge to be updated without retraining the entire model, reducing computational complexity while maintaining accuracy.
Solution Approach 2:
The external knowledge injector acts as an intermediary component between the frozen LLM and the knowledge base. It receives user queries, retrieves relevant knowledge from external sources, and injects this knowledge into the LLM's reasoning process without modifying the LLM's parameters, thus improving knowledge accuracy without increasing device complexity.
2Measurement precision
If LLMs are retrained with new training material, then the knowledge currency is improved, but the loss of existing knowledge occurs due to catastrophic forgetting
Solution Approach 1:
The LLM is pre-trained on comprehensive training material to establish a robust foundation of existing knowledge that remains frozen during operation. The external knowledge injector is configured to add new knowledge without overwriting or modifying the pre-trained parameters, thereby preventing catastrophic forgetting while ensuring knowledge currency.
Solution Approach 2:
Instead of modifying the original LLM parameters, the system creates a separate external knowledge injector that copies and processes knowledge from external sources. This copy mechanism allows new knowledge to be integrated without affecting the original trained model, preserving existing knowledge while updating currency.
3Measurement precision
If the external model overwrites the LLM output, then the response accuracy is improved, but the device complexity increases
Solution Approach 1:
The external knowledge injector serves as an intermediary layer between the LLM and the final output. It receives the LLM's initial response, verifies knowledge accuracy against external sources, and only overwrites the output when necessary. This intermediary approach improves response accuracy while maintaining relatively simple architecture by adding only one additional component.
Data Source
AI summary
A question is received at a first language machine learning model. In response to an external machine learning model providing a response to the question and a confidence determination for the response that exceeds a predetermined threshold, the response is injected into the first language machine learning model so that the response overwrites a vector state layer output of the first language machine learning model that provides another response to the question and without modifying original parameters of the first language machine learning model, where the external machine learning model was trained with training material with which the first language machine learning model was not trained.