Incremental Learning via Segmented Feature Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models, particularly those based on artificial neural networks, face challenges with catastrophic forgetting, making incremental learning difficult and reducing model performance, and existing methods often compromise performance to address this issue.
Innovation Solution
A machine learning method that encodes training data with multiple class labels, constructs feature networks, selects significant feature networks based on performance, and updates their weights to incrementally update the model, allowing for easy incremental learning without performance reduction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If an ANN-based machine learning model is used to improve adaptability to continuously changed environments, then the model can perform supervised learning tasks, but catastrophic forgetting occurs where previously learned content is forgotten when new learning data is input
Solution Approach 1:
The patent segments the model into multiple independent feature networks (first feature network, second feature network, etc.), each responsible for specific class labels. When new data arrives, only the relevant feature network is updated, preventing interference with other networks and thus avoiding catastrophic forgetting while maintaining adaptability to new environments.
Solution Approach 2:
The patent extracts and isolates the weight parameters of each feature network separately. During incremental learning, only the weights of the specific feature network corresponding to new class labels are updated, while weights of other feature networks remain unchanged. This extraction and selective updating mechanism prevents catastrophic forgetting of previously learned content.
2Reliability
If gradient boosting is used to optimize all learning data in building a model, then better performance is achieved compared to ANN-based algorithms, but incremental learning becomes difficult
Solution Approach 1:
The patent divides the gradient boosting process into separate feature networks, each handling specific class labels independently. This segmentation allows the model to maintain high performance through gradient boosting optimization while enabling incremental learning by updating only the relevant feature network when new data arrives, rather than retraining all data.
Solution Approach 2:
The patent creates a dynamic model structure where feature networks can be added, removed, or updated based on the learning tasks. This dynamic architecture allows the model to adapt to changing requirements by incorporating new feature networks for new class labels while preserving existing networks, thus achieving both high performance and incremental learning capability.
3Adaptability or versatility
If the internal structure of an ANN-based machine learning model is made very complicated to improve learning capability, then the model can handle complex patterns, but it becomes difficult to describe the model or results
Solution Approach 1:
The patent segments the complex model into multiple simple, independent feature networks, each handling specific class labels. This segmentation maintains learning capability for complex patterns by distributing complexity across multiple simple units, while improving interpretability since each feature network can be understood and described independently rather than as one monolithic complex structure.
Data Source
AI summary
A machine learning method for incremental learning builds a model by using training data and incrementally updates the built model by using only a new weight generated based on new training data.