All-Shot Training for Large Language Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for building large language models, such as in-context learning and instruction tuning, face inefficiencies and limitations in terms of computational cost, stability, and generalizability.
Innovation Solution
The all-shot training technique employs a form of in-context learning that teaches the model to learn 'how to learn in context' by providing it with various numbers of examples within a given context window, allowing the model to perform well regardless of the number of examples provided.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If instruction tuning is used to teach the model to follow instructions, then the model's ability to follow human instructions is improved, but the model loses a degree of generalizability
Solution Approach 1:
The patent combines in-context learning and instruction tuning into a unified training framework. The training data includes both instruction-following examples and in-context learning examples with varying shot counts, allowing the model to simultaneously learn instruction following and maintain adaptability to different context configurations without sacrificing either capability
2Loss of energy
If in-context learning is used to improve model performance without training, then computational cost is reduced, but the model becomes less stable and reliable
Solution Approach 1:
The patent performs preliminary training that incorporates in-context learning examples with varying shot counts (0-shot, 1-shot, 2-shot, etc.) before deployment. This preliminary action embeds the ability to handle different context configurations into the model weights, providing stability and reliability while maintaining the computational efficiency of in-context learning during inference
3Reliability
If prompt engineering is used to ensure in-context learning performance, then model performance is improved, but significant human efforts and heavy experimentations are required
Solution Approach 1:
The patent trains the model to automatically adapt to different shot counts and context configurations through exposure to varied training examples. The model learns to self-adjust its behavior based on the number of examples provided in context, eliminating the need for manual prompt engineering and heavy experimentation while maintaining high performance across different configurations
4Reliability
If more examples are provided in the context window for in-context learning, then model performance is improved, but the context window size limits the number of examples that can be provided
Solution Approach 1:
The patent trains the model with a variety of shot counts including cases that exceed typical context window limitations. By exposing the model to training examples with different numbers of shots during training, the model learns to efficiently utilize available context window space and generalizes to handle varying example counts without being constrained by fixed context window size during deployment
Data Source
AI summary
Embodiments described herein provide systems and techniques for training large language models. In one aspect, a process for performing in-context few-shot training for a transformer-based language model is disclosed. This process may begin by receiving the transformer-based language model having a context window of a predetermined size, as well as a training dataset comprising a set of prompt/completion examples. The process then constructs a training sequence based on the training dataset. Next, the process performs a single forward pass using the training sequence as input. The process subsequently performs a set of backward passes from a subset of examples in the training sequence, wherein each backward pass is conditioned on a selected subset of prompt/completion examples in the training sequence. Performing multiple backward passes conditioned on selected prompt/completion examples improves the language model's ability to generate a higher quality conditional probability distribution for next token without incurring additional training time.