Fixed Model Core and Programmable Fine-Tuning Portion for ML Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The rapid growth in computational demands of applications like large machine learning models exceeds the processing capabilities of existing computing architectures, leading to bottlenecks and increased costs, necessitating more efficient and cost-effective methods to adapt these architectures for specific tasks.
Innovation Solution
A computing architecture is designed with a hard-wired model core and a programmable fine-tuning portion, where the model core stores pre-trained parameters and the fine-tuning portion adjusts parameters using a PEFT routine to adapt the model for specific applications, minimizing the need for full retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If a large machine learning model with hundreds of billions of parameters is used to meet computational requirements, then processing capability is improved, but hardware cost and complexity increase significantly
Solution Approach 1:
The patent segments the machine learning model into two distinct parts: a large-scale model core with hundreds of billions of parameters that remains fixed, and a small fine-tuning portion with minimal parameters that is programmably configured. This segmentation allows the system to leverage the processing power of large models while avoiding the hardware complexity of fully deploying and training entire large models, as only the small fine-tuning portion requires programmable resources.
Solution Approach 2:
The patent extracts the trainable parameters from the large model core and isolates them into a separate fine-tuning portion. This extraction allows the majority of the model (the model core) to be implemented in fixed, efficient hardware, while only the extracted fine-tuning parameters require programmable memory and configuration, significantly reducing the hardware complexity while maintaining the ability to adapt the model to different tasks.
2Adaptability or versatility
If a large machine learning model is retrained for different applications, then adaptability is improved, but computational cost and time increase
Solution Approach 1:
By segmenting the model into a fixed model core and a programmable fine-tuning portion, the patent enables rapid adaptation to different applications by simply reconfiguring the small fine-tuning parameters through programming, rather than retraining the entire large model. This segmentation transforms the adaptation process from a computationally intensive retraining operation to a efficient programming operation.
Solution Approach 2:
The patent performs preliminary action by pre-training the large model core once to learn general patterns and knowledge. This pre-trained model core serves as a foundation that can be rapidly adapted to different applications by programming the fine-tuning portion, eliminating the need to retrain the entire model for each new application and significantly reducing adaptation time.
3Productivity
If a hard-wired model core is used to store pre-trained parameters, then processing efficiency is improved, but flexibility to adapt to new tasks decreases
Solution Approach 1:
The patent segments the model into a hard-wired model core for efficient processing of general patterns and a programmable fine-tuning portion for task-specific adaptation. This segmentation allows the system to maintain high processing efficiency through the fixed model core while achieving flexibility through the programmable fine-tuning parameters that can be configured for different applications.
Solution Approach 2:
The patent merges two different approaches: the efficiency of hard-wired fixed models and the flexibility of programmable models. By combining a hard-wired model core with a programmable fine-tuning portion, the system achieves both high processing efficiency from the fixed architecture and task flexibility from the programmable parameters, resolving the contradiction between efficiency and adaptability.
Data Source
AI summary
Methods and systems which involve customized computing architectures are disclosed herein. A disclosed computing architecture comprises a model core and a fine-tuning portion. The model core stores a set of parameters of an ML model. The fine-tuning portion stores a set of fine-tuning values for a fine-tuned ML model. The fine-tuned ML model is a fine-tuned version of the ML model. The model core may be fixed during the fabrication of the customized computing architecture. The fine-tuning portion may be fixed after the model core is fixed. The model core may be less configurable than the fine-tuning portion. The set of parameters of the ML model may be defined during the fabrication of the computing architecture. The set of fine-tuning parameters for the fine-tuned ML model may be defined after the set of parameters of the ML model is defined.