Small-sample image classification method based on metric prototype calibration
By using a prototype-calibrated few-sample image classification method, and optimizing feature extraction through pseudo-class labels and prototype-query comparative learning, the inconsistency of feature measurement in few-sample image classification is solved, and the classification accuracy of the model in low-sample scenarios is improved.
Patent Information
- Application Number
- CN202410312685.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-19
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-03-19
AI Technical Summary
Existing few-sample image classification algorithms struggle to improve classification accuracy due to the high difficulty and scarcity of samples, as well as the high cost of annotation. In particular, there are inconsistencies in feature measurement methods.
A few-sample image classification method based on metric prototype calibration is adopted. Pseudo-class labels are generated by a pre-trained feature extractor to optimize the category decision boundary. The prototype-query contrastive learning method is used to improve feature discrimination. The few-sample image classification is performed by combining the metric prototype calibration model.
This method improves model performance in low-sample-volume scenarios, acquires more discriminative features, and is suitable for small-sample image classification tasks, thereby improving classification accuracy.
Smart Images

Figure CN117994591B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology and relates to a small sample image classification method based on metric prototype calibration. Background Technology
[0002] Deep learning artificial intelligence algorithms have achieved remarkable success in many fields. For example, in image recognition, deep learning network models have even surpassed the image recognition capabilities of the human eye. The success of these methods relies heavily on fitting deep network models with a large amount of trainable sample data. However, in real-world applications, obtaining usable data samples typically faces two major challenges: high difficulty in sample acquisition, high scarcity, and excessive labeling costs. Unlike the underlying principles of deep learning, humans possess the ability to learn and generalize from a single data sample or a small amount of information. For instance, a child can learn and generalize the characteristics of an animal (such as a penguin) from a single, unseen picture. Inspired by this intuitive human behavior, the concept of single-shot learning has gradually emerged in the field of machine learning. With increasing attention to this area, related research has deepened, and in recent years, few-shot learning has become a popular research direction in machine learning.
[0003] Current few-shot image classification algorithms can be broadly categorized into four main directions: transfer learning-based methods, meta-learning-based methods, data augmentation-based methods, and metric learning-based methods. Transfer learning focuses on effectively learning prior knowledge on large-scale pre-training datasets and efficiently transferring and generalizing it to few-shot datasets. Its core goal is to ensure good performance of pre-trained models on few-shot datasets. Meta-learning-based methods aim to enable models to "learn to learn," typically employing a "training-test consistency" paradigm. This allows the model to simulate few-shot task scenarios during pre-training, enabling it to quickly adapt and make accurate judgments when faced with new few-shot tasks. Data augmentation methods mainly include training data augmentation and feature space-based data augmentation. These methods aim to alleviate the challenge of scarce few-shot data by increasing the number or quality of samples. Metric learning-based methods focus on feature measurement methods, aiming to design an effective metric function to distinguish between similar and dissimilar samples. This method primarily improves the accuracy of few-shot image classification by optimizing feature representations and measurement methods. Summary of the Invention
[0004] In view of this, the purpose of this invention is to provide a few-sample image classification method based on metric prototype calibration. Based on transduction learning, pseudo-class labels are generated using unlabeled samples of the query set, and the metric prototype is calibrated. At the same time, considering the inconsistency between categories, the prototype-query comparison method is used to optimize the decision boundary between categories, so that the model's representation of the samples is more discriminative.
[0005] To achieve the above objectives, the present invention provides the following technical solution:
[0006] A few-sample image classification method based on metric prototype calibration, which includes the following steps:
[0007] S1: Use the base class samples in the dataset to pre-train the feature extractor and obtain a feature mapping space with prior information about the base class categories;
[0008] S2: Map the support set and query set samples in the small sample image classification task to high-dimensional vector representations, use the support set sample representations to calculate the category initialization prototype, use the query set sample representations to calculate pseudo-class label samples, expand the pseudo-class label samples into the support set, and use the augmented support set to calibrate the category metric prototype for bias.
[0009] S3: Construct positive and negative sample pairs using the initial class prototype and query set samples, learn the similarity and differences between samples, and obtain the optimized class decision boundary;
[0010] S4: Using samples from the small sample image classification task used for testing as input, embed the optimized mapping space, calculate the category prototype for prediction classification using the metric prototype calibration method, and finally obtain the prediction result by measuring the similarity between the query sample and the category prototype in the task.
[0011] Furthermore, in step S1, the dataset is preprocessed and partitioned, specifically including the following steps:
[0012] S11: Load the dataset and divide it into training, validation, and test sets. The training set contains 64 categories as base classes, the validation set contains 16 categories to record model performance, and the test set contains 20 categories as target categories for the few-sample image classification task.
[0013] S12: Preprocess the image samples, adjust each sample, unify the width and height of the image to 84×84 pixels, and perform normalization processing, divide each channel by 255, and convert each pixel value to a value in the range of 0-1.
[0014] S13: Classify the 64 base classes in the training set into 64 categories of images and use the cross-entropy loss function for model pre-training.
[0015] S14: The pre-training phase ends, and the fully connected layer at the end of the network structure is removed to obtain a feature extractor with prior information of the base class features.
[0016] Furthermore, in S2, the bias calibration of the category metric prototype is performed using the augmented support set, which is completed based on the metric prototype calibration model structure. The metric prototype calibration model structure consists of a pre-trained feature extractor, a prototype generation unit, a pseudo-class label generation unit, a prototype calibration unit, and a metric unit. The pre-trained feature extractor is used to perform vector representation of the support set and query set samples. In the prototype generation unit, the category initialization prototype p is generated by using a class-K-means algorithm on the samples of each category in the support set. i Pseudo-class label samples are generated using the similarity between the query set samples and the category initialization prototypes. The pseudo-class labels with high confidence are then expanded into the support set to obtain the augmented support set. For the samples in the augmented support set, similarity distance weights are calculated, and a calibrated category prototype p' is generated using a weighted sum. i This category of prototypes has more sample information and is more robust as a metric.
[0017] Furthermore, in S3, a query set sample of the same category as the initialization prototype is selected as the positive sample of the initialization prototype, and a sample of another category is randomly sampled as the negative sample. The initialization prototype is used as the anchor point to bring the distribution of samples of the same type in the feature space closer and to widen the interval between samples of different types, thereby obtaining a decision boundary with discriminative power.
[0018] Furthermore, in step S4, the category prototype used for predicting classification is calculated using the metric prototype calibration method, which is obtained by calculating the metric similarity through the category calibration prototype.
[0019]
[0020] Where d represents the metric function, p' j This represents the prototype for metric calibration, where N is the number of task categories. It is a prototype-query optimization feature extractor.
[0021] The beneficial effects of this invention are as follows: it is applicable to small sample scenarios with limited available sample size, thus improving the lower limit of model performance. Furthermore, the prototype-query contrastive learning method can serve as an optimization method for feature extractors in general classification tasks, helping to obtain more discriminative features for samples of different categories.
[0022] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description
[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:
[0024] Figure 1 This is a flowchart of the measurement prototype calibration unit proposed in this invention;
[0025] Figure 2 This is a complete flowchart of the few-sample image classification method proposed in this invention. Detailed Implementation
[0026] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0027] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0028] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.
[0029] Please see Figures 1-2 , Figure 1 This is a schematic diagram of the prototype calibration method in this invention. Figure 2 This is a complete flowchart of the few-shot image classification method in this invention. The training method for few-shot image classification using this model includes the following steps:
[0030] Step 1: Pre-train the features in the training set to obtain the feature extractor f. θ The specific steps are as follows:
[0031] Step 101: Preprocess the image samples, adjust each sample, and unify the width and height of the image to 84×84 pixels;
[0032] Step 102: Normalize the image by dividing each channel by 255 to convert each pixel value into a value in the range of 0-1;
[0033] Step 103: Divide 100 samples from each category in the training set data as pre-training validation samples to verify the pre-training effect and record the optimal parameter values to obtain the pre-trained feature extractor f. θ .
[0034] Step 2: Calculate the initial class prototypes using the support set from the few-shot task and the pre-trained feature extractor obtained in Step 1. The specific steps are as follows:
[0035] Step 201: Sample N-way, K-shot small sample tasks from C base class categories in the training set to form a support set S and a query set Q, and then use the pre-trained feature extractor f θ , which is transformed into a feature space vector;
[0036] Step 202: Utilize the prototype network algorithm From support set sample x s The initial prototype representation P for each category is calculated.
[0037] Step 203: Calculate the similarity of the query set sample x using cosine similarity as the metric function. Q The similarity to the initial prototype is calculated, and a similarity score matrix A is obtained, with the following formula. The Z samples with the highest scores in each category are then selected as pseudo-label samples x. pseudo This leads to the extended support set S';
[0038]
[0039] Step 204: Calculate the cosine similarity of each sample in the expanded support set S' to the prototype of the initialized class, and obtain the similarity metric weight w for each sample. Use the metric weight to perform a weighted summation of samples of the same class. The calibration metric category prototype P' is obtained.
[0040] Step 3: Predict the label value of the query sample using the calibration category prototype P', and compare it with the true label value. By comparison, the loss function L is obtained. proto :
[0041] Step 301: Measure the similarity between each treatment sample in Q and the calibration category prototype P', calculate each cosine similarity, and normalize it using softmax to obtain the probability distribution p of the prediction result, the formula of which is as follows;
[0042]
[0043] Step 302: Take the item with the highest category probability in the predicted probability distribution as the predicted label, and compare it with the true label value. The loss is measured by the prototype calibration classification loss L. proto .
[0044] Step 4: Compare the initialized prototype with the query set samples to obtain the prototype-query contrastive loss. The specific steps are as follows:
[0045] Step 401: Select any initial prototype P i Select samples x of the same category as the prototype from the query set Q. i and different class samples x j , with P i Positive and negative sample pairs (P) are formed for the anchor points. i ,x i ) + , (P i ,x j ) - ;
[0046] Step 402: Design Prototype - Query and Compare Loss L PQ The model's fine-tuning parameters are guided by the following formula, resulting in a model feature extractor that enhances feature extraction performance.
[0047] L PQ =log(1+exp(d(P)) i ,f θ (x i )) + -d(P i ,f θ (x j )) - ))
[0048] Step 403: Integrate the prototype-query loss and the metric prototype calibration loss through the hyperparameter α to obtain the total model training loss function L, the formula of which is as follows:
[0049] L=αL PQ +(1-α)L Proto
[0050] Step 5: After training the model, perform a few-sample task on the test set of new class samples. Then, optimize the feature extractor using the support set and query set samples as described in Step 2. The calibration metric prototype is obtained, and finally, the sample category label value is predicted according to step 4.
[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A small sample image classification method based on metric prototype calibration, characterized in that: The method includes the following steps: S1: Use the base class samples in the dataset to pre-train the feature extractor and obtain a feature mapping space with prior information about the base class. S2: In the few-sample image classification task, the support set and query set samples are mapped to high-dimensional vector representations. The class initialization prototype is calculated using the support set sample representations, and pseudo-class label samples are calculated using the query set sample representations. These pseudo-class label samples are then expanded into the support set. The augmented support set is used to calibrate the bias of the class metric prototype. This bias calibration is performed based on a metric prototype calibration model structure, which consists of a pre-trained feature extractor, a prototype generation unit, pseudo-class label generation, a prototype calibration unit, and a metric unit. The pre-trained feature extractor is used to represent the support set and query set samples as vectors. In the prototype generation unit, the class initialization prototype is generated from the support set samples using a K-means-like algorithm. Pseudo-class label samples are generated using the similarity between the query set samples and the category initialization prototypes. The pseudo-class labels with high confidence are then expanded into the support set to obtain the augmented support set. For the samples in the augmented support set, similarity distance weights are calculated, and a calibrated category prototype is generated using a weighted sum. This category of prototypes has more sample information and is more robust as a metric; S3: Construct positive and negative sample pairs using the category initialization prototype and query set samples, learn the similarity and difference between samples, and obtain the optimized category decision boundary; select a query set sample under the same category as the category initialization prototype as the positive sample of the category initialization prototype, and then randomly sample a sample from another category as the negative sample. Use the category initialization prototype as the anchor point to bring the distribution of samples of the same type in the feature space closer and widen the gap between samples of different classes, thereby obtaining a decision boundary with discriminative power. S4: Using samples from the small sample image classification task used for testing as input, embed the optimized mapping space, calculate the category prototype for prediction classification using the metric prototype calibration method, and finally obtain the prediction result by measuring the similarity between the query sample and the category prototype in the task.
2. The few-sample image classification method based on metric prototype calibration according to claim 1, characterized in that: In step S1, the dataset is preprocessed and partitioned, specifically including the following steps: S11: Load the dataset and divide it into training, validation, and test sets. The training set contains 64 categories as base classes, the validation set contains 16 categories to record model performance, and the test set contains 20 categories as target categories for the few-sample image classification task. S12: Preprocess image samples, adjusting each sample to unify the width and height of the image to 84. 84 pixels, and at the same time, normalization is performed by dividing each channel by 255 to convert each pixel value into a value in the range of 0-1; S13: Classify the 64 base classes in the training set into 64 categories of images and use the cross-entropy loss function for model pre-training. S14: The pre-training phase ends, and the fully connected layer at the end of the network structure is removed to obtain a feature extractor with prior information of the base class features.
3. The few-sample image classification method based on metric prototype calibration according to claim 1, characterized in that: In step S4, the category prototype used for classification prediction is calculated using the metric prototype calibration method, and the metric similarity is obtained by calculating the category calibration prototype. in, d Represents a metric function. This represents the prototype for metric calibration, where N is the number of task categories. It is a prototype-query optimization feature extractor.
