Metalearner for Pre-trained Model Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning algorithms face challenges in selecting the most suitable pre-trained models for new data sets, especially when access to the training data is restricted, as they often perform best on similar data distributions, and measuring differences between data sets is complex, especially in medical imaging where anatomical content, noise, and alignment properties are involved.
Innovation Solution
A medical system and method that uses a metalearner to determine the affinity between a new data set and pre-trained models by training a second model to output affinity-relating values, which represents the similarity between the data set and the pre-trained models, even without access to the original training data, by applying the pre-trained models to the new data set and using activation values to train the metalearner.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If pre-trained models are selected based on data distribution similarity, then model performance on new data is improved, but access to training data is restricted due to data sharing policies
Solution Approach 1:
The patent introduces a metalearner as an intermediary that compares data distributions without requiring direct access to the original training data. The metalearner is trained on metadata and distribution characteristics from source data, then uses this learned knowledge to assess similarity between new data and source data distributions, enabling model selection while respecting data sharing restrictions.
Solution Approach 2:
The patent creates a copy or representation of the source data distribution through metadata and distribution statistics that can be shared without sharing the actual sensitive data. This copied distribution information is used by the metalearner to perform similarity comparisons, achieving the same purpose as direct data access while maintaining data privacy and compliance with sharing policies.
2Measurement precision
If multiple pre-trained models are evaluated for new data sets, then model selection accuracy is improved, but computational complexity and time increase
Solution Approach 1:
The patent performs preliminary actions by pre-training the metalearner on distribution characteristics and metadata from source data before actual model selection is needed. This pre-computed knowledge is then rapidly applied during inference to evaluate multiple pre-trained models against new data, significantly reducing the time required for model selection while maintaining accurate comparison across multiple candidates.
3Adaptability or versatility
If data scientists manually select models for new tasks, then model applicability is improved, but the process requires expert input and is not scalable
Solution Approach 1:
The patent implements self-service by enabling the system to automatically perform model selection without requiring data scientist intervention. The metalearner autonomously compares data distributions, evaluates pre-trained models, and identifies the most suitable model for new data sets, allowing the system to serve itself in the model selection task while maintaining high adaptability to different data types and domains.
Data Source
AI summary
A medical system comprises processing circuitry configured to: receive a first trained model, wherein the trained model has been trained using a first data set acquired in a first cohort; receive a second data set acquired in a second cohort; input data included in the second data set and data representative of the first trained model into a second trained model; and receive from the second trained model an affinity-relating value which represents an affinity between the data included in the second data set and the first trained model.