CNN New Class Detection via Eigenspace Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face challenges in accurately classifying new objects when they belong to classes for which the network has not been trained, leading to uncertainty and potential misclassification due to insufficient training data or feature drift.
Innovation Solution
The method involves determining eigen-distances in a kernel space using singular value decomposition (SVD) to map new samples to an eigenspace, identifying whether they belong to a learned class or a new class, and retraining the CNN with updated data for new classes, while also generating a taxonomy to contextualize relationships between classes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a CNN is trained for existing classes, then classification accuracy for learned classes is improved, but the system cannot correctly classify new objects from novel classes
Solution Approach 1:
The system performs preliminary analysis by mapping new samples to the eigenspace and calculating eigen-distances to existing class centroids before final classification. This preliminary action allows the system to identify potential new classes by detecting samples that fall outside the established class boundaries, enabling proactive adaptation rather than reactive retraining
Solution Approach 2:
The patent introduces an intermediary mechanism - the eigenspace with eigen-distance calculation - that acts as a mediator between the CNN's learned classes and new unseen classes. This intermediary layer allows the system to measure the distance of new samples from existing class centroids in a transformed feature space, providing a buffer zone to detect novel classes without directly compromising the CNN's existing classification performance
2Measurement precision
If training data is collected for new classes, then classification accuracy for new classes is improved, but time and resources are consumed
Solution Approach 1:
The system performs self-service by automatically detecting new classes through eigen-distance analysis and autonomously initiating the retraining process without human intervention. The classification system monitors its own performance, identifies when new classes appear, and triggers retraining automatically, eliminating the need for manual data collection and retraining initiation
Solution Approach 2:
The patent implements a feedback mechanism where the system continuously monitors classification confidence and eigen-distances of incoming samples. When samples consistently fall outside existing class boundaries (high eigen-distance), the system receives feedback that a new class may be present, triggering automatic retraining. This closed-loop feedback system reduces time loss by immediately responding to new class detection rather than relying on periodic manual reviews
3Adaptability or versatility
If the CNN is retrained frequently to include new classes, then adaptability to new classes is improved, but computational resources and training time are consumed
Solution Approach 1:
The system applies partial retraining by using only the new samples identified as belonging to novel classes, rather than retraining on the entire dataset. This selective approach reduces computational resources and energy consumption while still achieving adaptability to new classes. The eigen-distance threshold mechanism enables this partial action by identifying only the necessary samples for retraining
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system and computer-implemented method for automatically recognizing a new class in a classification system. The method includes accessing components of a trained convolutional neural network (CNN) that has been trained with available classes. The components are provided in a kernel space and include at least one of a plurality of kernels and a plurality of neurons of one or more layers of the CNN. Furthermore, the components are assigned to a class in accordance with the training. The method further includes applying a covariance matrix to map the components in the kernel space to eigenspace; determining, for each of the available classes, an eigen-distance between a sample and the components mapped to eigenspace; based on the eigen-distance, determining whether the sample is an outlier that does not belong to one of the classes; and creating a new class that includes the sample if determined that the sample is an outlier.