Meta Input for Pretrained Deep Learning Model Recycling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models experience significant performance degradation when user input data differs from the training data, requiring lengthy re-training and making it difficult to apply these models in user-centered environments such as IoT devices and edge-devices, due to catastrophic forgetting and environmental changes.
Innovation Solution
A meta input method and system that optimizes a meta input by aligning the distribution of user environment testing data with the training data distribution using gradient-based training algorithms, allowing the pretrained deep learning model to maintain performance without re-training, by adding the optimized meta input to the testing data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the deep learning model is re-trained with user testing data to adapt to user environment, then the inference performance on user data is improved, but the training time and computational resources are significantly increased
Solution Approach 1:
The patent applies preliminary action by pre-training the deep learning model on source domain data before deployment. The model is prepared in advance with general knowledge that can be adapted to user environments through meta-input adjustments rather than full re-training, saving significant training time while maintaining adaptability.
Solution Approach 2:
The patent utilizes parameter changes by modifying the input data distribution through meta-input transformation rather than changing model parameters through re-training. The meta-input adjusts the testing data distribution to match the training data distribution, enabling performance maintenance without time-consuming re-training processes.
2Measurement precision
If the deep learning model is trained with vast amounts of data to improve performance, then the inference accuracy is improved, but the model complexity and training resources are increased
Solution Approach 1:
The patent applies segmentation by separating the model training phase from the deployment phase. The model is trained once on source domain data with high accuracy, and then adapted to user environments through lightweight meta-input adjustments rather than requiring the model to be complex enough to handle all possible domain variations internally.
Solution Approach 2:
The patent introduces meta-input as an intermediary between the pre-trained model and user testing data. This intermediary transforms the testing data distribution to match the training data distribution, allowing the model to maintain high inference accuracy without needing to be re-trained or made more complex to handle domain shifts.
3Adaptability or versatility
If the deep learning model is re-trained frequently to adapt to environmental changes, then the adaptability to user environment is improved, but the catastrophic forgetting of original training data occurs
Solution Approach 1:
The patent applies preliminary action by pre-training the model on source domain data to acquire fundamental knowledge and patterns. This preliminary training establishes a robust foundation that remains preserved during deployment, while meta-input adjustments enable adaptation to user environments without requiring frequent re-training that would cause catastrophic forgetting.
Solution Approach 2:
The patent utilizes parameter changes at the input level rather than model parameter level. By transforming the testing data distribution through meta-input to match the training data distribution, the model can adapt to environmental changes while maintaining its original learned knowledge, avoiding catastrophic forgetting associated with re-training.
Data Source
AI summary
A meta input method and system and a user-centered inference method and system via a meta input for recycling of a pretrained deep learning model are provided. The meta input method for the recycling of the pretrained deep learning model performed by a computer device includes optimizing a meta input by considering a relation between input data and output prediction of the pretrained deep learning model and adding the optimized meta input to testing data in a user environment to transform distribution of the testing data into distribution of training data used to build the deep learning model.