A small sample image classification contrast learning method based on clustering

Abstract

The application discloses a two-stage meta-learning method for small sample image classification based on clustering, applies clustering and contrast learning to small sample image classification, and comprises the following steps: step 1, determining a small sample data set, and clustering small sample image data; step 2, designing a two-stage meta-learning network structure; step 3, designing a loss function; step 4, inputting the clustered pictures into the two-stage meta-learning network for contrast training; step 5, using the trained two-stage model to extract features of new class images; and step 6, calculating image similarity based on multiple features to output a classification result. Through the introduction of clustering and two-stage meta-learning network contrast training into small sample learning, the network can better extract the distinguishable features of images. On the network structure, the common feature extraction network and the individual feature extraction network are added, a triple loss function is used for training, similar samples are distinguished in a more fine-grained manner, and the accuracy of current small sample image recognition is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, and in particular to a small sample image classification contrast learning method based on clustering. BACKGROUND

[0002] In recent years, the deep learning theory has achieved great success in computer vision tasks. With the huge network structure and massive training parameters, the deep learning algorithm optimizes the training network with a large amount of labeled data, and shows excellent performance in many fields. On the other hand, considering the privacy of data in some fields (such as medical, military field, etc.), it is difficult to obtain data, and personnel with professional knowledge are needed to mark, which leads to the high cost of obtaining a large amount of labeled data. Therefore, how to realize the efficient learning of deep network under the condition of limited sample is urgent to be solved.

[0003] To solve this problem, in recent years, researchers have gradually paid attention to the small sample learning task. Small sample learning refers to the task of learning under the condition of small sample of training class. It is expected that the machine can learn the characteristics of new class quickly and make prediction by learning a large amount of base class data with only a small amount of new class sample. Usually, small sample learning can complete the image classification task with only 1 to 5 samples. Under this condition, the small sample image classification is applied to the data scarce field, which will save the huge cost of data acquisition, and has very high research value.

[0004] Currently, existing few-shot methods are mainly divided into three categories: model-based few-shot learning, parameter optimization-based few-shot learning, and data augmentation-based few-shot learning. Model-based few-shot learning achieves rapid parameter updates on a small number of samples through model structure design, establishing a mapping function between samples and predicted values. Parameter optimization-based few-shot learning updates model parameters by improving rapid parameter update strategies. Data augmentation-based few-shot learning addresses the fundamental problem of few-shot learning by attributing it to insufficient sample size and low sample diversity, improving performance by synthesizing new labeled data for the few-shot categories to expand the training data. Regarding training methods, meta-learning-based training is widely accepted by researchers. Currently, meta-learning training mainly consists of a meta-training phase and a meta-testing phase. The meta-training phase trains the model and tests its performance based on a large amount of base class data, while the meta-testing phase performs feature learning and prediction for new classes. In each phase, the algorithm simulates few-shot learning by randomly selecting N categories, with K samples from each category for parameter learning (called N-way, K-shot), iterating continuously to improve model performance. Although this method has achieved some performance improvement, its accuracy is still unsatisfactory, especially in distinguishing similar categories. This is because the random sampling mode of N-way and K-shot does not obtain a lot of sample information. Most samples are quite different and easy to identify, thus failing to drive the model to perform more fine-grained deep learning.

[0005] To address this problem, this invention proposes a clustering-based few-sample image classification contrastive learning method. It improves upon the original meta-learning model by introducing clustering technology and contrastive learning, optimizing the few-sample learning network structure and loss function, and significantly enhancing the recognition accuracy of similar categories. This invention has significant application value, enabling accurate image classification even under conditions of scarce image samples and extremely high image category similarity. Summary of the Invention

[0006] To address the aforementioned problems, this invention proposes a clustering-based few-shot image classification contrastive learning method, applying clustering and contrastive learning to few-shot image classification, including the following steps:

[0007] Step 1: Determine a small sample dataset and cluster the small sample image data;

[0008] Step 2: Design of a two-stage meta-learning network structure;

[0009] Step 3: Loss function design;

[0010] Step 4: Input the clustered images into the two-stage meta-learning network for comparative training;

[0011] Step 5: Feature extraction of new class images using the trained two-stage model;

[0012] Step 6: Calculate image similarity based on multiple features and output classification results.

[0013] Further, the data set contained in step 1 includes: mini-Imagenet, tiered-Imagenet and CUB200; the data set is divided into "base class data set" and "new class data set", and the two data sets do not overlap in class; in the sample space, the features of similar classes are relatively concentrated, and the features of dissimilar classes are far apart; K-Means clustering algorithm is used to cluster all images and distinguish the similarity between classes.

[0014] Further, in step 2, the convolutional neural network uses Resnet-12 as the main structure to extract image features; on this basis, a common feature network and a personal feature network are designed, the common feature network aims to learn the common features of different classes within the same cluster, and the personal feature network aims to learn the distinguishable personal features between different classes, so as to obtain two groups of different type features and provide strong feature support for fine-grained image classification.

[0015] Further, in step 3, the loss function includes cross-entropy loss function and triple loss function;

[0016] The cross-entropy loss function expression is:

[0017] ,

[0018] Where y is the expected value label output, x is the neural network actual label output, and n is the sample number; the cross-entropy loss is used at the end of the base feature extraction network, the common feature extraction network and the personal feature extraction network to ensure the correctness of the class label and the cluster label output by the network;

[0019] The triple loss function is expressed as:

[0020] ,

[0021] Where a and b are images of the same class, and a and c are images of different classes in the same cluster, is the distance parameter used to expand the feature distance between two similar classes.

[0022] Further, in step 4, compared with the traditional single-stage meta-learning training mode, a two-stage contrast training mode is adopted, wherein:

[0023] The first stage comprises: inputting the small sample image of the clustered base class data set into a Resnet-12 basic feature extraction network, then obtaining corresponding common features and individual features through a common feature network and an individual feature network, and respectively mapping the image features into classification labels through a classifier to calculate a loss function; wherein the Resnet-12 and the common feature network part use a cross-entropy loss function, and the individual feature network uses a cross-entropy loss function and a triple loss function;

[0024] The second stage is different from the first stage in that the small sample image of the clustered base class data set is divided into a support set and a query set, and the two sets are compared and learned; the fused features are used to calculate a cross-entropy loss according to the cosine similarity of the support set and the query set through a classifier; compared with single-stage training, the two-stage comparison training method can better identify the features of the image difference;

[0025] The cosine similarity expression is:

[0026]

[0027] wherein 、 are the feature vectors of the support set and the query set samples respectively.

[0028] Further, in step 5, the support set and the query set pictures in the new class data set are input into the meta-learning network to extract features, and two kinds of features, common features and individual features, are extracted from each picture.

[0029] Further, in step 6, according to the common features and individual features of the support set and the query set pictures in the new class obtained in step 5, the common similarity and the individual similarity are measured by using the cosine similarity in step 5, and the class label with the highest similarity to the query set picture to be tested is finally output by comprehensively considering both, to obtain the image classification result.

[0030] The present application has the following advantages: the present application innovatively introduces clustering and two-stage meta-learning network comparison training into small sample learning, compares and trains similar images of different classes in the same cluster, so that the network can better extract distinguishable features of images. On the network structure, the common feature extraction network and the individual feature extraction network are added, and the triple loss function is used for training, which can distinguish similar samples in finer granularity and improve the accuracy of current small sample image recognition. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a method flowchart of the present application;

[0032] Figure 2 is a composition process diagram of the data set clustering and the support set and the query set of the present application;

[0033] Figure 3 is a two-stage meta-learning network comparison training process diagram of the present application;

[0034] Figure 4 is a new class image prediction class process diagram of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical scheme and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the drawings and preferred embodiments. However, it should be noted that many details in the description are only to make the reader have a thorough understanding of one or more aspects of the present application, and the aspects of the present application can be realized even without these specific details.

[0036] The present application will be further described below in conjunction with the drawings of the specification:

[0037] As shown in the figure, a small sample image classification comparison learning method based on clustering includes the following steps: Figure 1

[0038] Step one: determine the small sample dataset, and cluster the small sample image data;

[0039] Specifically, the specific steps of step one are:

[0040] Figure 2 The process diagram for data set clustering and the composition of support set and query set. The data sets contained in step 1 are: mini-Imagenet, tiered-Imagenet and CUB200. First, the data set needs to be divided into "base class data set" and "new class data set", and the two data sets do not overlap in class. In the sample space, the features of similar classes are relatively clustered, and the features of dissimilar classes are far apart, so the K-Means clustering algorithm is used to cluster all images and measure the similarity between classes. Specifically, suppose the cluster is divided into When using Euclidean distance clustering, the distance between data x and the center point of the i-th cluster is calculated as follows:

[0041]

[0042] Where k is the number of clusters, which is set manually. The goal of the clustering algorithm is to minimize the squared error dist of the data object to the cluster center.

[0043] Step two: two-stage meta-learning network structure design;

[0044] Specifically, the specific steps of step two are:

[0045] ​The convolutional neural network extracts picture features with Resnet-12 as the main structure; on this basis, a common feature network and a personal feature network are designed, the common feature network aims to learn the common features of different categories of images in the same cluster, and the personal feature network aims to learn the distinguishable personal features between different categories of images, so as to obtain two groups of different types of features, which provide strong feature support for fine-grained image classification.

[0046] Step three: loss function design;

[0047] Specifically, the specific steps of step three are:

[0048] The loss function includes a cross-entropy loss function and a triple loss function.

[0049] The cross-entropy loss function expression is:

[0050] ,

[0051] Where y is the expected value label output, x is the neural network actual label output, and n is the sample number; the cross-entropy loss is used at the end of the basic feature extraction network, the common feature extraction network and the personal feature extraction network, to ensure the correctness of the class label and the cluster label output by the network.

[0052] The triple loss function is represented as:

[0053] ,

[0054] Where a and b are images of the same class, and a and c are images of different categories in the same cluster, is a distance parameter used to expand the feature distance between two similar categories.

[0055] Step four: input the clustered pictures into the two-stage meta-learning network for contrast training;

[0056] Specifically, the specific steps of step four are:

[0057] Figure 3 is a two-stage meta-learning network contrast training process diagram, compared with the traditional single-stage meta-learning training mode, the method adopts a two-stage contrast training mode, wherein:

[0058] The first stage includes: inputting the small sample images of the clustered base class data set into the Resnet-12 basic feature extraction network, and then obtaining the corresponding common features and personal features through the common feature network and the personal feature network, and respectively using the classifier to map the image features to the classification label to calculate the loss function. Among them, the Resnet-12 and the common feature network part use the cross-entropy loss function, and the personal feature network uses the cross-entropy loss function and the triple loss function;

[0059] The second stage is different from the first stage in that: the clustered base class data set small sample image is divided into a support set and a query set, and the two are compared and learned; the fused features use a classifier to calculate the cross-entropy loss according to the cosine similarity of the support set and the query set; compared with single-stage training, the two-stage contrast training method can better identify the features of image differences.

[0060] The cosine similarity expression is:

[0061]

[0062] Wherein , The feature vectors of the support set and the query set samples are respectively.

[0063] Step five: using the trained two-stage model to extract features of new class images;

[0064] Specifically, the specific steps of step five are:

[0065] The support set and the query set pictures in the new class data set are input into the meta-learning network to extract features, and two kinds of features are extracted from each picture: common features and individual features.

[0066] Step six: based on multi-feature calculation, the image similarity is output.

[0067] Specifically, the specific steps of step six are:

[0068] Figure 4 It is a new class image prediction class process diagram, according to step 5, the common features and individual features of the support set and the query set pictures in the new class are obtained, the cosine similarity of step 5 is used to measure the common similarity and the individual similarity, and finally the class label with the highest similarity to the query set picture to be measured is output, and the image classification result is obtained.

[0069] The basic principles, main features and advantages of the present application are shown and described above. It should be understood by those skilled in the art that the present application is not limited by the above examples, and the above examples and descriptions in the specification are only to illustrate the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application.

Claims

1. A clustering-based few-sample image classification and contrastive learning method, characterized in that, Applying clustering and contrastive learning to few-sample image classification includes the following steps: Step 1: Determine a small sample dataset and cluster the small sample image data; Step 2: Design of a two-stage meta-learning network structure; Step 3: Loss function design; Step 4: Input the clustered images into the two-stage meta-learning network for comparative training; Step 5: Use the trained two-stage model to extract features from the new class of images; Step 6: Calculate image similarity based on multiple features and output classification results; In step 2, the convolutional neural network uses ResNet-12 as the main structure to extract image features. Based on this, a common feature network and a unique feature network are designed. The common feature network aims to learn the common features of images of different categories within the same cluster, while the unique feature network aims to learn the distinguishable unique features between images of different categories, thus obtaining two sets of different types of features, providing strong feature support for fine-grained image classification. In step 4, compared to the traditional single-stage meta-learning training model, a two-stage comparative training method is adopted, in which: The first stage includes: inputting small sample images from the clustered base class dataset into the ResNet-12 basic feature extraction network, then passing them through the common feature network and the individual feature network to obtain the corresponding common and individual features, and using a classifier to map the image features to classification labels and calculate the loss function; among them, the ResNet-12 and common feature network parts use the cross-entropy loss function, while the individual feature network uses the cross-entropy loss function and the triple loss function; The difference between the second and first stages is that the clustered base class dataset is divided into a support set and a query set for comparative learning; the fused features are used by a classifier to calculate the cross-entropy loss based on the cosine similarity between the support set and the query set; the two-stage comparative training method can better identify the features that make the differences between images compared to the single-stage training. The cosine similarity expression is: ； in , These are the feature vectors of the support set and query set samples, respectively.

2. The clustering-based few-sample image classification and contrast learning method according to claim 1, characterized in that: The datasets included in step 1 are: mini-Imagenet, tiered-Imagenet, and CUB200. The datasets are divided into "base class datasets" and "new class datasets", and the categories covered by the two datasets do not overlap. In the sample space, the image features of similar categories are relatively clustered, while the image features of dissimilar categories are far apart. The K-Means clustering algorithm is used to cluster all images to distinguish the similarity between categories.

3. The clustering-based few-sample image classification and contrast learning method according to claim 1, characterized in that: In step 3, the loss function includes the cross-entropy loss function and the triple loss function; The expression for the cross-entropy loss function is as follows: ， Where y is the expected label output, x is the actual label output of the neural network, and n is the number of samples; cross-entropy loss is used at the ends of the basic feature extraction network, common feature extraction network, and individual feature extraction network to ensure the correctness of the category label and cluster label output by the network. The triple loss function is expressed as follows: ， Where a and b represent images of the same class, while a and c represent images of different classes within the same cluster. The distance parameter is used to expand the feature distance between two similar categories.

4. The clustering-based few-sample image classification and contrast learning method according to claim 1, characterized in that: In step 5, features are extracted from the support set and query set images in the new class dataset into the meta-learning network. Two types of features are extracted from each image: Common characteristics and individual characteristics.

5. The clustering-based few-sample image classification and contrast learning method according to claim 4, characterized in that: In step 6, based on the common and individual features of the images in the support set and query set obtained in step 5, the cosine similarity from step 5 is used to measure the common similarity and individual similarity. Taking both into account, the final output is the category label with the highest similarity to the category of the image in the query set to be tested, and the image classification result is obtained.

Images