Transfer Learning Convolutional Neural Network Subcategory Relabeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutive neural networks require a large number of annotated data for effective learning, which is not always available for target problems, and existing solutions for universal representation are either inefficient or require expensive manual annotation.
Innovation Solution
A process involving two networks of convolutive neurons is used, where the first network is learned from a database of images labeled with specific categories, and then subcategories are constructed and used to relabel the database, allowing the second network to learn from this relaxed database, thereby generating a more universal image descriptor.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a convolutional neural network is pre-trained on a source problem with sufficient annotated data, then the network can be used as a visual feature extractor for target problems, but the representation is not adapted to the target problem and performance is limited
Solution Approach 1:
The patent segments the training process into two distinct phases: first training a CNN on source problem data with sufficient annotations, then transferring and fine-tuning this pre-trained network on target problem data. This segmentation allows the system to leverage abundant source data while adapting to target problem requirements, resolving the contradiction between data availability and representation adaptability.
Solution Approach 2:
The patent applies preliminary action by pre-training the convolutional neural network on the source problem before deploying it to the target problem. This preliminary training establishes a robust feature extraction capability that can then be adapted to the target domain, allowing the system to overcome the lack of target annotated data while maintaining adaptability through subsequent fine-tuning.
2Adaptability or versatility
If manual annotation is performed to create subcategories for specific concepts, then more universal representations can be learned, but the cost and complexity of data production increases significantly
Solution Approach 1:
The patent applies self-service by enabling the system to automatically generate subcategories and learn universal representations without requiring manual human annotation. The convolutional neural network autonomously processes images, learns features, and adapts to create universal descriptors, eliminating the need for expensive and complex manual data production while achieving the goal of universality.
3Reliability
If a convolutional neural network is trained on a large database of annotated images, then the network achieves good performance on the training categories, but the network cannot be effectively applied to different target problems without retraining
Solution Approach 1:
The patent implements universality by designing a convolutional neural network that serves multiple functions: it performs well on source problem categories during training and can be transferred to various target problems. The network learns universal visual features from the source domain that are applicable across different domains, achieving both reliability on training data and versatility on target problems through the transfer learning framework.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
The invention relates to a method, implemented by a computer (ORD), for transferring learning from a first convolutional neural network to a second convolutional neural network, comprising the following steps: a) learning by the first convolutional neural network of a database of images labelled according to a plurality of categories of the most specific level of the database, each image being able to be represented by a characteristic vector; b) for each of the categories of the most specific level, constructing at least one subcategory; c) relabelling the database of images with the set of subcategories, in order to form a database of relabelled images; d) learning by the second convolutional neural network of the database of re-labelled images.