Feature Representation Transfer Between ML Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models, particularly those running on lower-cost client devices, face challenges in achieving high accuracy without increasing complexity, which limits their performance in tasks like image recognition and feature extraction.
Innovation Solution
The system extracts feature representations from a source model using influence methods like spline fitting and transfers these representations to a destination model, enhancing its performance by improving evaluative metrics such as accuracy and precision with minimal impact on compute resources and interpretability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If more complex machine learning models are used to improve accuracy, then model performance improves, but computational complexity and infrastructure requirements increase
Solution Approach 1:
The patent extracts feature representations from a complex source model and transfers them to a simpler destination model. This allows the simpler model to achieve improved performance by leveraging the learned features from the complex model without requiring the full computational infrastructure of the source model.
Solution Approach 2:
The patent creates a copy of the feature representations learned by the complex source model and applies them to the simpler destination model. This copying approach allows the destination model to benefit from the source model's learned patterns while maintaining its own architectural simplicity and lower computational requirements.
2Reliability
If model complexity is increased to improve accuracy, then evaluative metrics improve, but compute resources required increase
Solution Approach 1:
The patent extracts the essential feature representations from the complex source model that contribute to its high evaluative metrics. By transferring only these critical features to the simpler destination model, the patent achieves improved reliability without proportionally increasing computational resource requirements.
3Measurement precision
If feature engineering is enhanced to improve model performance, then accuracy improves, but processing complexity increases
Solution Approach 1:
The patent performs feature extraction and representation learning in advance using the source model. These pre-computed feature representations are then transferred to the destination model, eliminating the need for the destination model to perform complex feature engineering from scratch and thereby reducing its processing complexity while maintaining accuracy.
Data Source
AI summary
Computer accesses source model and destination model. Each of source model and destination model is configured to compute output value based on values for each of set of features. Computer develops influence model for source model. Influence model computes a relative influence value for one or more feature of source model. Sum of all relative influence values corresponds to output value of source model. Computer determines a curve function mapping the one or more features of source model to the relative influence value of the one or more features. Computer creates an augmented input feature set by applying the curve function to augment the one or more features of source model. Computer modifies destination model by adding, to destination model, a preprocessing function to generate feature values for features of the augmented input feature set. Computer retrains modified destination model to leverage the curve function and the augmented input feature set.