Latent Variable Inference for Generative Model Tuning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training generative language models to generate accurate and interpretable chain-of-thought responses is challenging due to limited access to question, rationale, and answer training datasets, leading to biases from small sample sizes and inaccurate responses.
Innovation Solution
A computing system that processes input and output tokens to generate intermediary and output tokens, adjusting model parameters based on a combined gradient of a loss function, using a control variate to manage parameter adjustments and improve model performance, allowing for training without relying on labeled question, rationale, and answer triplets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If training data is limited to small sample sizes of question-rationale-answer triplets, then training becomes simpler and faster, but model accuracy and reasoning capability deteriorate due to training biases
Solution Approach 1:
The patent introduces intermediary tokens as a mediator between input tokens and output tokens. These intermediary tokens represent hidden reasoning states that enable the model to learn complex reasoning patterns without requiring explicit rationale annotations in the training data. The intermediary tokens serve as a bridge, allowing the model to infer reasoning steps implicitly from input-output pairs alone.
Solution Approach 2:
The patent extracts the reasoning capability from the traditional question-rationale-answer triplet structure. By removing the explicit rationale component and replacing it with learnable intermediary tokens, the patent enables the model to discover reasoning patterns autonomously from input-output pairs, eliminating the need for biased human-annotated rationales.
2Ease of operation
If chain-of-thought prompting is used to generate step-by-step responses, then response interpretability improves, but training difficulty increases due to limited access to labeled reasoning data
Solution Approach 1:
The model performs self-service by automatically generating and learning intermediary tokens during training without requiring external rationale annotations. The training process self-adapts to learn optimal intermediary representations that enable chain-of-thought reasoning, eliminating the need for complex manual labeling of reasoning steps.
Solution Approach 2:
Intermediary tokens serve as a mediator that enables chain-of-thought reasoning to emerge from simple input-output pair training. These tokens capture the hidden reasoning states that would otherwise require explicit annotation, allowing the model to learn interpretable step-by-step reasoning automatically.
3Use of energy by stationary object
If traditional supervised learning is used with limited training examples, then training resource consumption is reduced, but generalization capability deteriorates due to overfitting on small datasets
Solution Approach 1:
The patent changes the parameter space by introducing intermediary tokens with learnable parameters that capture abstract reasoning patterns. This parameter transformation enables the model to generalize from few examples by learning compact representations of reasoning logic, rather than memorizing specific training examples.
Solution Approach 2:
The intermediary tokens serve a universal function across different reasoning tasks. By learning general-purpose intermediary representations that capture underlying reasoning patterns, the model achieves multi-functionality and broad generalization without requiring task-specific training data for each application domain.
Data Source
AI summary
Systems and methods for generative language model tuning can include training the generative language model to generate sets of output text tokens with set of intermediary text tokens with training examples that include input and output pairs. The training can include processing the input with the language model to determine a predicted output and a predicted set of intermediary text tokens. The predicted set of intermediary text tokens can then be evaluated based at least in part on the output associated with the input and the predicted output.