LLM Training With Negative Sampling to Prevent Data Leakage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) are prone to information leakage and hallucinations, particularly when trained on confidential data, leading to unreliable results and potential exposure of sensitive information.
Innovation Solution
Implement negative sampling and modified loss functions to train LLMs, using positive and negative examples to prevent data leakage and hallucinations, and employ neural networks with specific training prompts to ensure accurate data extraction without memorizing input data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LLMs are trained on confidential data to improve data extraction capability, then the model's ability to extract information is improved, but information leakage occurs and confidential data is exposed
Solution Approach 1:
The patent applies preliminary anti-action by training the LLM with negative sampling techniques that pre-teach the model what confidential information looks like and how to avoid reproducing it. The system prepares anti-examples during training that show the model the boundaries of acceptable data extraction, preventing information leakage before it can occur in production.
Solution Approach 2:
The patent converts the harmful effect of information leakage into a beneficial training signal by using adversarial examples and negative sampling. The model is deliberately exposed to potential leakage scenarios during training, and the loss function is modified to penalize reproducing confidential data, thereby turning the risk of leakage into a learning opportunity that strengthens the model's security.
2Ease of operation
If LLMs are trained to provide plausible responses to improve response quality, then the form and coherence of responses is improved, but hallucinations occur and accuracy deteriorates
Solution Approach 1:
The patent implements feedback mechanisms through modified loss functions that provide continuous signals during training about the accuracy and plausibility of responses. The system uses verification steps and reinforcement learning from human feedback to guide the model toward producing responses that are both high-quality and accurate, correcting hallucinations through iterative feedback loops.
Solution Approach 2:
The patent applies preliminary action by implementing verification and validation steps before finalizing model responses. The system pre-processes training data to ensure accuracy, uses fact-checking mechanisms during training, and employs multiple layers of verification to prevent hallucinations before they reach the user.
3Measurement precision
If LLMs memorize training data to improve data extraction accuracy, then the precision of extracting specific information is improved, but information leakage increases and confidential data is exposed
Solution Approach 1:
The patent applies local quality by differentiating between types of data in the training set and applying different treatment to each. Confidential data is marked and handled differently from public data, with the model learning to extract only non-confidential information while avoiding memorization of sensitive content through localized modifications to the training process.
Solution Approach 2:
The patent changes key training parameters including the loss function formulation, learning rates, and data weighting to prevent memorization while maintaining extraction precision. The system dynamically adjusts parameters during training to balance between learning useful patterns and avoiding rote memorization of confidential data.
Data Source
AI summary
LLMs can process input from various sources and handle natural language but they are prone to hallucinate non-existing information and may reproduce (or “leak”) input provided during training. Both hallucinations and leaks are undesirable. To reduce or avoid hallucination of non-existing data, systems and methods discussed herein use negative sampling. A training set is supplemented with examples in which non-existent data is requested and the LLM is trained not to create data to satisfy the request. To reduce or avoid memorization by the LLM of input data, systems and methods discussed herein use a modified loss function. Using the modified loss functions discussed herein, the LLM learns the relationship between the response and the input without learning the input itself. As a result, the trained LLM does not leak the input data.