Component Factorized Models for Online Learning Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine-learning models face challenges in achieving accurate and efficient predictions due to the limitations of batch learning and the high resource costs associated with training large models, especially in online continuous training settings where data distribution changes over time.
Innovation Solution
The system employs a large donor model to generate component data that is shared with multiple small recipient models, allowing for continuous online transfer and updates, which are stored persistently for efficient use by recipient models, thereby reducing resource costs and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large machine-learning model is trained to achieve high accuracy, then prediction accuracy is improved, but computing resource costs and training time increase significantly
Solution Approach 1:
The patent divides the large donor model into multiple smaller recipient models, each specialized for a specific task. The donor model generates component data (embeddings) that are shared with multiple recipient models, allowing each recipient to operate independently with fewer resources while maintaining high accuracy through the shared knowledge base.
Solution Approach 2:
The donor model serves multiple functions by generating component data that can be used by multiple different recipient models for different tasks. This universal component data generation allows a single large model to support multiple specialized models, reducing overall computing resource requirements while maintaining high prediction accuracy across diverse tasks.
2Measurement precision
If batch learning is used to train models on entire training data sets, then model parameters are generated comprehensively, but the system cannot adapt to changing data distributions in real-time
Solution Approach 1:
The patent implements dynamic online learning where the donor model continuously updates its component data representations as new data arrives. This dynamic update mechanism allows the system to adapt to changing data distributions in real-time while maintaining the comprehensive knowledge base built from historical data through continuous learning processes.
Solution Approach 2:
The system employs continuous online learning where the donor model continuously processes new data and updates its component data representations without interruption. This continuous action ensures the model remains adaptable to evolving data distributions while maintaining comprehensive parameter knowledge through ongoing learning processes.
3Measurement precision
If multiple separate large models are trained for different tasks, then each model achieves high accuracy for its specific task, but the overall system complexity and resource consumption increase
Solution Approach 1:
The patent merges multiple task-specific models into a unified system where a single donor model generates shared component data that is distributed to multiple recipient models. This merging approach reduces system complexity by eliminating redundant model parameters while maintaining high task-specific accuracy through specialized recipient models that process the shared component data.
4Use of energy by moving object
If component data is generated and stored persistently for use by multiple recipient models, then resource costs are reduced through sharing, but data freshness and timeliness may be compromised
Solution Approach 1:
The system implements feedback mechanisms where recipient models can request updated component data from the donor model when needed. This feedback loop allows the system to balance resource efficiency through persistent storage with data freshness requirements by updating components on-demand rather than requiring continuous real-time data generation for all models.
Data Source
AI summary
Systems and methods for improving accuracy of machine-learning models are described herein. The system can obtain, by one or more computing devices, component data from a donor model. The donor model can be an online model. The component data can have a first embedding. The system can store the component data in a persistent database. The component data can be input data for a plurality of recipient models. The system can receive from the first recipient model of the plurality of recipient models, a first request for the first embedding. The system can transmit, from the persistent database to the first recipient model, the first embedding. The system can process, using the first recipient model, the first embedding to generate a first recipient output.