Class-Incremental Sample Classification With Class-Wise Decorrelation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Catastrophic forgetting occurs in Class Incremental Learning (CIL) where model performance on previous training data degrades after being trained with new data, and existing solutions do not adequately address the importance of the initial phase, leading to unsatisfactory classification model performance.
Innovation Solution
The proposed Class-wise Decorrelation (CwD) solution regularizes feature distributions in the initial phase to mimic the oracle model by using a secondary loss function that decorrelates dimensions in the feature space, ensuring features are evenly scattered, thereby improving classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the classification model is trained with newly received training data in subsequent phases, then the model learns new classifications, but the model performance on previous training data degrades seriously (catastrophic forgetting)
Solution Approach 1:
The patent applies preliminary action by regularizing feature distributions in the initial phase before catastrophic forgetting occurs. The secondary loss function is introduced upfront to decorrelate feature dimensions and make features evenly scattered, creating a robust foundation that prevents future performance degradation when new data is introduced
Solution Approach 2:
The patent changes parameters by introducing a secondary loss function that modifies the feature distribution parameters in the initial phase. This loss function specifically targets the correlation between feature dimensions and adjusts them to be more evenly distributed, which fundamentally alters the feature space structure to prevent catastrophic forgetting
2Reliability
If existing solutions focus on addressing catastrophic forgetting without emphasizing the initial phase, then some forgetting mitigation is achieved, but classification model performance remains unsatisfactory
Solution Approach 1:
The patent emphasizes preliminary action by concentrating regularization efforts in the initial phase rather than only addressing forgetting in subsequent phases. The secondary loss function is applied during initial training to establish optimal feature distributions before the model encounters new data, which leads to significantly improved classification accuracy across all phases
Data Source
AI summary
A method is proposed for sample processing. A first group of data are received, here data in the first group of data comprises a sample and a classification of the sample, and the classification belonging to a first group of classifications in a plurality of classifications associated with the data. A plurality of data with the classification are selected from the first group of data. A first and a second loss function are determined for training a classification model that represents an association relationship between samples and classifications of the samples based on a plurality of samples comprised in the plurality of data and the classification, the first and second loss functions represent classification accuracy and a feature distribution for the classification model. The classification model is trained based on the first and second loss functions. Therefore, the accuracy of the classification model may be increased.