Convolutional Neural Network Incremental Learning via Kernel Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face limitations in prediction performance when encountering data with statistical characteristics different from their training data, as they cannot infinitely increase training data, leading to suboptimal performance in new data scenarios.
Innovation Solution
A method for incremental learning in CNNs is introduced, where the system determines whether to initiate incremental learning based on statistical differences between target and training data, identifies kernels and weight vectors with high mutual similarity, and updates these using random matrices or vectors to improve prediction performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the amount of training data is increased to improve prediction performance, then prediction performance is improved, but it cannot be increased infinitely leading to data availability limitations
Solution Approach 1:
The patent changes the parameters of existing kernels by generating new kernels through linear combination of similar kernels. Instead of adding more training data, the method modifies kernel parameters (weights and configurations) to adapt to new statistical characteristics of target data, thereby improving prediction performance without increasing data quantity
Solution Approach 2:
The patent introduces dynamic adaptability by enabling the CNN to automatically detect statistical differences between training data and target data, and dynamically generate new kernels based on detected similarity groups. This dynamic adjustment allows the system to adapt to changing data distributions without requiring additional training data
2Reliability
If incremental learning is initiated to adapt to new data characteristics, then prediction performance on target data is improved, but computational complexity increases due to kernel similarity measurement and selection
Solution Approach 1:
The patent segments the set of kernels into groups based on similarity measurement. By dividing kernels into similarity groups and only operating on representative kernels from each group, the computational complexity is reduced while maintaining the ability to adapt to new data characteristics
Solution Approach 2:
The patent performs partial action by selectively updating only those kernels that belong to similarity groups with representative kernels showing statistical differences. Instead of recalculating all kernels, the method applies changes only to necessary portions, reducing computational overhead
3Reliability
If kernels with high mutual similarity are identified and updated, then the CNN adapts to new data statistics improving prediction performance, but the number of parameters to be updated increases
Solution Approach 1:
The patent merges similar kernels by identifying kernels with high mutual similarity and combining them into groups. By updating a single representative kernel for each similarity group rather than individual kernels, the number of parameters to be updated is significantly reduced while maintaining adaptability to new data characteristics
Data Source
AI summary
Provided is a method of machine learning for a convolutional neural network (CNN). The method includes: receiving input target data; determining whether to initiate incremental learning on the basis of a difference between a statistical characteristic of the target data with respect to the CNN and a statistical characteristic of previously used training data with respect to the CNN; determining a set of kernels with a high degree of mutual similarity in each convolution layer included in the CNN when the incremental learning is determined to be initiated; and updating a weight between nodes to which kernels included in the set of kernels with a high degree of mutual similarity are applied.