Dynamic Submodel Injection for ML Resource Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning approaches require training and deploying separate models for each user, domain, context, and task, leading to inefficiencies in resource usage and inflexibility, as they rely on statically serving a single model for all scenarios, resulting in redundant computing resource consumption.
Innovation Solution
A system that dynamically combines a base model with personalized, domain-specific, or task-specific submodels during runtime, allowing for on-the-fly generation and updating of models by inserting or adding submodel parameters into the base model, enabling efficient deployment across different users, domains, and contexts without altering the base model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a single static model is trained to serve many users and domains, then the model can be deployed efficiently with low resource consumption, but the model lacks flexibility and adaptability to different users, domains, contexts, and tasks
Solution Approach 1:
The model is segmented into a base model that handles general tasks and domain-specific submodels that handle specialized tasks. This segmentation allows the system to maintain a simple base model structure while adding complexity only when needed through modular submodels, resolving the contradiction between adaptability and system complexity.
Solution Approach 2:
The base model is designed as a universal component that can serve multiple functions across different domains and tasks. By making the base model multi-functional and reusable, the system achieves high adaptability without proportionally increasing overall system complexity, as the same base model serves all users and domains.
2Adaptability or versatility
If separate models are trained and deployed for each user, domain, context, and task, then the system achieves high adaptability and performance for each specific scenario, but it causes significant redundancy in computing resources including processor usage, memory usage, and network bandwidth
Solution Approach 1:
Multiple domain-specific submodels are merged with a single base model to form combined models. This merging approach allows the system to achieve user-specific performance through submodels while avoiding the redundancy of training and deploying completely separate models for each user, domain, context, and task, thereby reducing computing resource consumption.
Solution Approach 2:
Instead of training separate models for each user and domain, the system uses parameter changes by loading different submodels with specialized parameters into the base model as needed. This allows the system to adapt to specific users and domains by changing model parameters rather than maintaining separate complete models, significantly reducing memory usage and computing resource consumption.
3Adaptability or versatility
If multiple complete models are loaded and unloaded into memory to satisfy different users and domains, then the system can provide specialized performance for each user, but it introduces significant redundancies in training and deployment operations
Solution Approach 1:
The base model is pre-loaded into memory once and remains there during operation. Domain-specific submodels are then loaded into the base model as needed rather than loading complete models from scratch. This preliminary action of having the base model ready reduces the time required to switch between different users and domains, as only the submodels need to be loaded rather than entire models.
Data Source
AI summary
The present disclosure relates generally to machine learning. More particularly, the present disclosure relates to on-the-fly feeding of personalized, domain-specific, context-specific, and/or task-specific submodels as input to an existing base model which has already been loaded into a memory (e.g., loaded into an existing session associated with execution of a machine learning library).