Deep Learning Model Transfer via Source Data Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing transfer learning methods in deep learning face challenges when dealing with small data sets or data that is distant from the desired model, as they often rely on prior-trained models that may not adequately cover the required data space, leading to gaps in model coverage.
Innovation Solution
A method that involves creating a map of the model space covered by prior-trained models, clustering similar data, and using these clusters to train a new neural network model by combining data from clusters that are closest to the target data set, thereby filling gaps and optimizing data size and transfer value.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If prior-trained models are used for transfer learning, then training can be performed with relatively small data, but the models may not adequately cover the required data space leading to gaps in model coverage
Solution Approach 1:
The patent combines multiple prior-trained models into an ensemble structure, where each model is trained on different subsets of the source data. By merging their predictions through voting or averaging, the system achieves both small data efficiency (each model uses limited data) and comprehensive coverage (collectively the ensemble covers the entire data space).
Solution Approach 2:
The source data is segmented into multiple subsets, each used to train a separate prior-trained model. This segmentation allows each model to specialize in specific data regions while the collection of models collectively covers the full data space, resolving the contradiction between using small data and achieving comprehensive coverage.
2Measurement precision
If multiple prior-trained models are evaluated for similarity, then the best transfer candidate can be identified, but the computational complexity increases
Solution Approach 1:
The patent extracts only the necessary features or representations from each prior-trained model for similarity comparison, rather than evaluating entire models. This extraction reduces the computational burden while maintaining the precision needed to identify the best transfer candidate.
Solution Approach 2:
The system performs preliminary filtering or pre-computation of similarity metrics before full model evaluation. By pre-processing and identifying obvious candidates or eliminating poor matches early, the computational complexity of the subsequent detailed evaluation is reduced.
Data Source
AI summary
Whether to train a new neural network model can be determined based on similarity estimates between a sample data set and a plurality of source data sets associated with a plurality of prior-trained neural network models. A cluster among the plurality of prior-trained neural network models can be determined. A set of training data based on the cluster can be determined. The new neural network model can be trained based on the set of training data.