Image Classification Using Averaged Feature Matrices for Incremental Learning

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Image classification accuracy is not sufficiently high in incremental learning or learning of a small number of images, particularly due to issues of catastrophic forgetting and overfitting in deep neural networks.

Innovation Solution

An image classification technology that utilizes an image classification apparatus and method, incorporating a feature extraction unit, averaged feature calculation units, and similarity calculation units to replace weight matrices with averaged feature matrices, leveraging deep and shallow layer feature vectors to improve classification accuracy in incremental learning scenarios.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If deep neural network parameters are re-learned in response to the entirety of the dataset to adapt to changes in data distribution, then the model can learn new tasks, but the precision of estimation for old tasks decreases due to catastrophic forgetting

Engineering Contradiction:
Improveability to learn new tasksVSAvoidprecision of estimation for old tasks
Core Design Contradiction:
Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent segments the feature space by identifying representative feature vectors for each class and dividing the dataset into subsets associated with different classes. This segmentation allows the model to process and learn from different classes separately, reducing interference between old and new tasks while maintaining the ability to adapt to new data distributions.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary actions by pre-calculating representative feature vectors for each class before actual learning occurs. These representative vectors serve as anchors that guide the learning process, allowing the model to maintain knowledge of old classes while learning new tasks. The representative feature vectors are computed in advance and used to initialize or constrain the learning process.

Inventive Principle:
Principle #10Preliminary action

2Quantity of substance

If artificial intelligence using deep learning is trained on a small number of images, then the model can learn from limited data, but it falls into overfitting characterized by good local performance but poor generalization performance

Engineering Contradiction:
Improvenumber of images used for learningVSAvoidgeneralization performance
Core Design Contradiction:
Quantity of substanceVSReliability

Solution Approach 1:

The patent introduces representative feature vectors as intermediaries between the input images and the learning process. These representative vectors act as mediators that capture the essential characteristics of each class, allowing the model to learn from small datasets without overfitting. The intermediaries provide a stable reference that improves generalization performance even when training data is limited.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent changes the parameter representation by using representative feature vectors instead of raw pixel data or standard feature embeddings. This parameter transformation allows the model to work effectively with small datasets by focusing on the most discriminative features. The parameter changes enable the model to achieve better generalization with fewer training examples.

Inventive Principle:
Principle #35Parameter changes

3Adaptability or versatility

If the model learns new knowledge continuously, then the model can adapt to new tasks, but the result of learning old tasks is forgotten

Engineering Contradiction:
Improvecontinuous learning capabilityVSAvoidforgetting of old task knowledge
Core Design Contradiction:
Adaptability or versatilityVSLoss of information

Solution Approach 1:

The patent performs preliminary computation of representative feature vectors for each class before continuous learning begins. These pre-computed vectors serve as persistent memory anchors that preserve knowledge of old tasks. During continuous learning, the model can incorporate new tasks while referring back to these stable representative vectors, preventing catastrophic forgetting of previously learned tasks.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent creates copies of essential task knowledge in the form of representative feature vectors that are stored and reused across different learning tasks. Instead of relying solely on the neural network weights to retain knowledge, the system copies critical class characteristics into compact representative vectors that can be repeatedly used to guide learning of both old and new tasks, preventing information loss.

Inventive Principle:
Principle #26Copying

Data Source

PatentEP4668208A1Image classification device, image classification method, and image classification program
Publication Date: 2025.12.24 JVC KENWOOD CORP
  • EP4668208A1 patent drawingFigure 1
  • EP4668208A1 patent drawingFigure 2
  • EP4668208A1 patent drawingFigure 3

AI summary

A feature extraction unit (510) outputs first and second feature vectors of an input image. An averaged first/second feature calculation unit (520a, 520b) calculates an averaged first/second feature vector by averaging first/second feature vectors of a given class and obtains an averaged first/second feature matrix by aggregating averaged first/second feature vectors of all classes. A first/second feature similarity calculation unit (532a, 532b) calculates a first/second similarity from the first/second feature vector of the input image and a first/second weight matrix. The averaged first/second feature calculation unit (520a, 520b) replaces the first/second weight matrix of the first/second feature similarity calculation unit (532a, 532b) with the averaged first/second feature matrix.