Pretrained Model Branching for Class-Incremental Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for extending pretrained machine learning models to recognize additional object classes are time-consuming, expensive, and result in sub-optimal classification accuracies or significant performance reduction for existing classes due to catastrophic forgetting.
Innovation Solution
A two-stage incremental training pipeline is employed, where a pretrained model is extended by adding a new branch near the top to compute scores for new object classes, with a small fusion component to merge outputs and maintain accuracy for both old and new classes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a pretrained model is extended to recognize additional object classes through traditional retraining methods, then the model can identify new classes, but the classification accuracy for existing classes significantly degrades due to catastrophic forgetting
Solution Approach 1:
The model is segmented into multiple independent branches: a base branch that maintains knowledge of original classes and a novel branch that learns new classes. Each branch processes inputs independently and their outputs are combined through a gating mechanism, allowing the model to recognize both old and new classes simultaneously without catastrophic forgetting
Solution Approach 2:
A gating mechanism acts as an intermediary between the base branch and novel branch. The gate dynamically selects and combines outputs from both branches based on the input characteristics, enabling seamless integration of new class knowledge while preserving existing class recognition accuracy
2Productivity
If a new model is built from scratch to recognize new object classes, then the model can be trained efficiently on new data, but it requires separate deployment and combining outputs from multiple models which is complex and sub-optimal
Solution Approach 1:
The base model and novel model are merged into a single unified architecture with shared input processing. The base branch and novel branch are integrated within one model structure, allowing simultaneous recognition of all classes through a single API endpoint, eliminating the need to deploy and coordinate multiple separate models
3Adaptability or versatility
If traditional retraining methods are used to extend model capabilities, then the model can learn new classes, but the process is time-consuming and expensive
Solution Approach 1:
The base model is pre-trained on original classes and frozen to preserve its knowledge. When new classes need to be added, only the novel branch requires training while the base branch remains intact, significantly reducing training time and computational resources compared to retraining the entire model from scratch
Solution Approach 2:
The base branch is copied and used as the foundation for the novel branch. This allows the model to leverage pre-learned features and representations for new classes without retraining the entire model, accelerating the adaptation process while maintaining performance on original classes
Data Source
AI summary
Techniques for class-incremental learning with pretrained machine learning models are described. According to some examples, a pretrained machine learning model that is capable of recognizing objects belonging to a set of object classes is extended to add capability for recognizing additional object classes, without significantly reducing the accuracy of the model for recognizing objects belonging to any of the object classes in the original set of object classes (aka catastrophic forgetting).