Transfer Learning Model Selection for Edge Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems face inefficiencies and sub-optimality when training models for different scenarios due to the lack of readily available training data and varying computing requirements, as well as the inability to effectively share knowledge between devices performing different machine learning tasks across diverse contexts.
Innovation Solution
A transfer learning method that utilizes a database of existing machine learning models and metadata to select and fine-tune models for new scenarios, leveraging transfer learning data to optimize model performance across a network of devices, including edge and cloud computing components.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a trained ML model from one device is shared with another device, then training efficiency is improved, but model optimality for the new device's specific scenario deteriorates
Solution Approach 1:
The patent applies local quality by selectively transferring only certain components (weights, biases, architecture) of the ML model from the source device, rather than transferring the entire model. This allows the target device to adapt the transferred components to its specific local scenario while retaining efficiency benefits.
Solution Approach 2:
The patent uses preliminary action by pre-training the ML model on the source device with available data before transferring it to the target device. This preliminary training establishes a foundation that accelerates subsequent fine-tuning on the target device, resolving the contradiction between efficiency and accuracy.
2Manufacturing precision
If training data is collected locally for each device, then model accuracy for specific scenarios is improved, but training time increases
Solution Approach 1:
The patent applies preliminary action by collecting and utilizing training data from the source device before the target device begins its training process. This pre-collected data serves as a head start, reducing the time the target device would otherwise need to collect and process data independently.
Solution Approach 2:
The patent uses copying by replicating the trained model components (weights, biases, architecture) from the source device to the target device. This copying mechanism allows the target device to inherit learned patterns without retraining from scratch, significantly reducing training time while maintaining accuracy through subsequent fine-tuning.
3Manufacturing precision
If computing resources are increased for training, then model accuracy is improved, but device complexity increases
Solution Approach 1:
The patent applies taking out by extracting only the essential components (weights, biases, architecture) from the fully trained model on the source device, rather than requiring the target device to perform complete training. This extraction reduces the computational burden on the target device while preserving model accuracy through transfer learning.
Solution Approach 2:
The patent uses partial action by performing only the necessary fine-tuning steps on the target device rather than complete training. The majority of the training work is performed partially on the source device and transferred, reducing the excessive computing requirements that would otherwise be needed on the target device.
Data Source
AI summary
A transfer learning method for a system including a plurality of existing agents each including a trained machine learning model for modelling a respective existing machine learning scenario, and a new agent. The system includes a database comprising available models for modelling scenarios, including the trained models, existing scenario metadata indicative of existing scenarios, and transfer learning data indicative of parts of the trained models. The method comprises receiving new scenario metadata indicative of a new scenario to be modelled by the new agent, and receiving new scenario training data for training a model of the new scenario. The method also includes querying the database to: select an available model, based on the received data, to model the new scenario; and, select at least some of the transfer learning data, based on the received data, to train the selected model.