Output Token Prediction with Ensemble Probability Adjustment for Rare Tokens
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sequence-to-sequence models, such as encoder-decoder neural networks, struggle to accurately predict rare output tokens that do not frequently appear in training data, leading to incorrect predictions when relying solely on a single model's probability scores.
Innovation Solution
An ensemble method is employed, combining the outputs of multiple neural networks (main and auxiliary models) where the probability of the auxiliary model is adjusted based on the rankings and probabilities of the main model, using scale and threshold coefficients, and normalized to ensure accurate prediction of rare tokens.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single neural network model is used to predict output tokens, then the model structure remains simple, but the prediction accuracy for rare tokens deteriorates
Solution Approach 1:
The patent combines multiple neural network models (first model and second model) into an ensemble system. The first model provides initial probability predictions, while the second model provides adjusted probability predictions. These are merged through a weighted sum calculation where the final probability is computed as: final probability = (weight1 × first probability) + (weight2 × adjusted probability). This merging approach improves prediction accuracy for rare tokens by leveraging complementary strengths of different models without requiring a single overly complex model.
Solution Approach 2:
The patent creates a composite prediction system where the final probability distribution is formed by combining predictions from multiple models with different characteristics. The composite approach allows the system to leverage the strengths of each model while compensating for their individual weaknesses, particularly improving performance on rare tokens that single models struggle to predict accurately.
2Measurement precision
If the auxiliary model's probability is adjusted based on rankings, then the prediction accuracy improves, but the computational complexity increases
Solution Approach 1:
The patent applies parameter changes by adjusting the probability values from the auxiliary model based on the rankings provided by the first model. Specifically, the adjusted probability is computed by multiplying the auxiliary model's probability by a ranking-based factor. This parameter adjustment allows the system to refine probability predictions without requiring complete re-computation, thereby improving precision while controlling computational complexity.
Solution Approach 2:
The patent applies partial action by adjusting probabilities only for tokens where the ranking information provides meaningful improvement. The adjustment is applied selectively based on the ranking position and the confidence levels of the models, rather than uniformly adjusting all predictions. This partial adjustment approach improves precision for critical predictions while reducing unnecessary computational overhead.
3Reliability
If multiple models are combined to improve prediction accuracy, then the reliability of rare token prediction improves, but the device complexity increases
Solution Approach 1:
The patent segments the prediction task into two separate models with distinct functions: the first model provides baseline probability predictions, while the second model provides adjusted probability predictions based on ranking information. This segmentation allows each model to be optimized for specific aspects of prediction, improving overall reliability while keeping individual model complexities manageable. The segmented approach also enables independent training and optimization of each model component.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for determining an output token includes predicting a first probability of each of candidate output tokens of a first model, predicting a second probability of each of the candidate output tokens of a second model interworking with the first model, adjusting the second probability of each of the candidate output tokens based on the first probability, and determining the output token among the candidate output tokens based on the first probability and the adjusted second probability.