Neural Network Feature Value Segmentation for Incremental Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning models face challenges in maintaining learning accuracy when intermediate feature values calculated from pre-learned neural networks are used for incremental learning, as they often become redundant and lead to deteriorated performance over time.
Innovation Solution
The method involves generating a first feature value and a second feature value from original training data, using a first neural network, and learning parameters to reconstruct data such that the output from a decoder closely resembles the original training data, while ensuring the second feature value is not redundant, thereby improving incremental learning accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If intermediate feature values are stored and reused for incremental learning, then the amount of training data is increased, but the learning accuracy deteriorates due to redundancy
Solution Approach 1:
The patent divides the intermediate feature values into two separate features: a first feature value and a second feature value. The first feature value is used for reconstruction, while the second feature value is used for incremental learning. This segmentation prevents redundancy by ensuring that the second feature value contains only the necessary information for learning, without the redundancy introduced by direct reuse of intermediate feature values from pre-learned models.
Solution Approach 2:
The patent extracts the essential information from the intermediate feature values and separates it into two components. By taking out the reconstruction information (first feature value) from the learning information (second feature value), the system eliminates redundancy while preserving the necessary data for incremental learning, thus maintaining learning accuracy.
2Measurement precision
If the first neural network is used to calculate intermediate feature values for reconstruction, then data reconstruction accuracy is improved, but the complexity of the learning model increases
Solution Approach 1:
The first neural network serves multiple functions: it calculates the first feature value for reconstruction and also provides the second feature value for incremental learning. By making the first neural network multi-functional, the patent avoids adding separate components for each function, thus improving reconstruction accuracy without proportionally increasing model complexity.
Solution Approach 2:
The patent embeds the calculation of the second feature value within the first neural network's output structure. The first neural network outputs both the first feature value (for reconstruction) and the second feature value (for learning) in a nested manner, allowing the system to benefit from high reconstruction accuracy while avoiding the need for separate complex learning models.
Data Source
AI summary
A learning device generates a first feature value and a second feature value by inputting original training data to a first neural network included in a learning model. The learning device learns at least one parameter of the learning model and a parameter of a decoder, reconstructing data inputted to the first neural network, such that reconstruction data outputted from the decoder by inputting the first feature value and the second feature value to the decoder becomes close to the original training data, and that outputted data that is outputted from a second neural network, included in the learning model by inputting the second feature value to the second neural network becomes close to correct data of the original training data.