LLM Parameter Updating Using Fact-Based Reward Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models suffer from factual emptiness or fabrication during long-form generation, leading to inaccuracies in text generation.
Innovation Solution
A method for updating a large language model parameters by generating target text, determining a fact set based on the text generation request, and using reward data to refine the model's performance, ensuring factual accuracy and comprehensiveness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing large language models are used for long-form generation, then text generation capability is provided, but factual accuracy deteriorates due to factual emptiness or fabrication
Solution Approach 1:
The patent implements a feedback mechanism where the generated text is evaluated against a knowledge base to identify factual errors, and this feedback is used to update the model parameters through reinforcement learning, thereby reducing factual fabrication in future generations
Solution Approach 2:
The patent introduces a knowledge base as an intermediary between the language model and the generation process. This knowledge base serves as a reference to verify factual accuracy and provides corrections when fabrications are detected, mediating the output quality
2Reliability
If model parameters are updated continuously to improve factual accuracy, then reliability improves, but computational complexity and training time increase
Solution Approach 1:
The patent extracts only the factual accuracy-related parameters for updating rather than retraining the entire model. By isolating and updating only the necessary components based on detected fabrications, the system reduces computational complexity while maintaining reliability improvements
Solution Approach 2:
The patent changes specific model parameters based on feedback from factual error detection rather than performing full model retraining. This selective parameter adjustment reduces training time and computational resources while still improving factual accuracy
Data Source
AI summary
A method for updating a parameter of a large language model is provided. The method may include: generating target text using the large language model based on a text generation request; determining facts to be relied on in the generation process of the target text according to the text generation request to obtain a target fact set; determining reward data according to the target fact set and an information set including unverified information in the target text; and updating the parameter of the large language model based on the reward data.