A few-shot image classification method and system

By obtaining local features and dynamically adjusting weights in few-sample image classification, the overfitting problem of traditional methods in data-scarce scenarios is solved, the classification accuracy and generalization ability are improved, and it is suitable for data-scarce scenarios.

CN119741534BActive Publication Date: 2025-10-21NANJING UNIV OF POSTS & TELECOMM

Patent Information

Application Number
CN202411778425.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-05
Publication Date
2025-10-21
Estimated Expiration
2044-12-05

AI Technical Summary

Technical Problem

Traditional deep learning methods are prone to overfitting in data-scarce scenarios, and existing few-shot learning methods fail to effectively utilize the importance differences of local features, resulting in decreased classification performance.

Method used

By obtaining a small number of sample image datasets, dividing the support set and query set, using the feature extraction network to obtain local features, generating prototypes and calculating weights, correcting the local features of the support set and query set, combining convolution operations and similarity tensors, dynamically adjusting the feature importance, and generating image classification results.

Benefits of technology

It improves the accuracy and generalization ability of few-sample image classification, reduces computational costs, adapts to the changeable and complex distribution of real data, and is suitable for data-scarce scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119741534B_ABST
    Figure CN119741534B_ABST
Patent Text Reader

Abstract

The application belongs to the technical field of image classification, and specifically discloses a few-shot image classification method and system, which comprises the following steps: obtaining a few-shot image dataset with category labels, dividing the dataset into multiple support sets and query sets, and obtaining local features of the support set and query set images through a feature extraction network; generating a prototype of each support class based on the local features of the support set images, and calculating the weight of each image in the support set; correcting the local features of the support set images; correcting the local features of the query set images; calculating the similarity between the query set images and each support class based on the two corrected local features and the weight of each image in the support set, and outputting the similarity as a probability to generate an image classification result. The application reduces the negative impact of outlying samples on the model, focuses the model on discriminative features related to the task, and improves the classification performance of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and in particular to a few-sample image classification method and system. Background Art

[0002] Traditional deep learning methods rely heavily on large amounts of labeled data, the collection and labeling of which typically requires significant time, money, and manpower. However, in some data-scarce application scenarios, such as rare species identification, military target detection, and rare disease diagnosis, collecting sufficient training samples is extremely difficult, or even impossible. In these situations, using traditional deep learning methods can lead to model overfitting and a significant decline in classification performance. Conversely, humans are able to quickly identify new categories with very few examples, inspiring the research direction of few-shot learning. Currently, mainstream approaches in few-shot learning research can be broadly divided into three categories: data augmentation-based methods, meta-learning-based methods, and metric learning-based methods.

[0003] However, 1. Data augmentation methods expand the amount of training data by performing various transformations on existing samples or generating synthetic samples to transform few-shot learning into a regular classification task, thereby improving the generalization performance of the model. However, while the above methods can alleviate the problem of sample shortage to a certain extent, overfitting is not completely solved. 2. The generalization performance of meta-learning usually depends on the prior task. Although meta-learning-based methods have achieved good results, their complex memory addressing structure and high computational overhead also pose a great challenge to training. 3. Methods based on metric learning include methods based on global features and methods based on local features. Among them, methods based on global features: The use of global features is prone to losing detailed information in the image and cannot accurately reflect the distribution of categories, making it difficult to obtain good classification results in few-shot scenarios. Methods based on local features: Although local feature methods capture image detail information, most of them do not consider the differences in the importance of local features. Information in irrelevant areas of the image may mislead the classification results. In addition, existing methods usually treat all samples in the task equally and do not consider the potential interference of outlier samples on the model. Summary of the Invention

[0004] The present invention aims to provide a few-sample image classification method and system to at least solve or improve one of the above technical problems.

[0005] In view of this, a first aspect of the present invention is to provide a few-sample image classification method, comprising:

[0006] S1. Obtain a dataset of few-shot images with class labels, divide the dataset into multiple support sets and query sets, and obtain local features of the support and query set images through a feature extraction network. Few-shot images refer to images in the training set that contain only a small number of labeled images for each class.

[0007] S2. Generate a prototype of each support class based on the local features of the support set image and calculate the weight of each image in the support set;

[0008] S3. Calculating the weight of the local feature of the support set based on the similarity vector between each local feature of the support set and each prototype, and correcting the local features of the support set image;

[0009] S4. Modifying the local features of the query set image based on the similarity tensor between the local features of the query set image and the support set image after the convolution operation;

[0010] S5. Based on the two corrected local features and the weight of each image in the support set, calculate the similarity between the query set image and each support class, and output the similarity in a probabilistic manner to generate an image classification result.

[0011] A second aspect of the present invention provides a few-sample image classification system, comprising:

[0012] A feature extraction module 201 is configured to obtain a dataset of few-shot images with class labels, divide the dataset into multiple support sets and query sets, and obtain local features of the support set and query set images through a feature extraction network. Few-shot images refer to images in the training set that contain only a small number of labeled images for each class.

[0013] a sample re-estimation module 202 for generating a prototype of each support class based on local features of the support set images and calculating a weight of each image in the support set;

[0014] A support correction module 203 is configured to calculate the weight of the local feature of the support set based on the similarity vector between each local feature of the support set and each prototype, and correct the local feature of the support set image;

[0015] A query correction module 204 is configured to correct the local features of the query set image based on the similarity tensor between the local features of the query set image and the support set image corrected by the convolution operation;

[0016] The output classification module 205 is used to calculate the similarity between the query set image and each support class based on the two corrected local features and the weight of each image in the support set, and output the similarity in a probabilistic manner to generate an image classification result.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1. This invention significantly improves the accuracy of few-sample image classification by introducing sample adaptation and local feature correction mechanisms. It calculates the distance between each support set image and the category prototype, dynamically assigns weights to each sample, and focuses on representative samples. This not only reduces the negative impact of outlier samples on the model, but also reduces computational costs.

[0019] 2. This invention uses a neural network to model the importance of each local feature for the current task, and makes corrections based on this, fully considering the differences in the importance of each local feature, focusing on the discriminative features in the image that are closely related to the category, and improving the classification performance of the model;

[0020] 3. The classification method of the present invention combines sample adaptation and local feature correction mechanism, so that the model can adapt to the changeable and complex real data distribution when processing small sample data, has higher generalization ability, and is suitable for application in various data-scarce scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0022] Figure 1 This is a flowchart of a few-sample image classification method of the present invention;

[0023] Figure 2 is a block diagram of a few-sample image classification system of the present invention;

[0024] Figure 3 The figure is a schematic structural diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0025] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0026] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0027] The following describes a few-sample image classification method and system according to some embodiments of the present invention.

[0028] Figure 1The figure is a flowchart of a few-sample image classification method of the present invention.

[0029] An embodiment of the present invention provides a method for classifying a small number of images. In some embodiments of the present invention, the method includes:

[0030] S1. Obtain a dataset of few-shot images with class labels, divide the dataset into multiple support sets and query sets, and obtain local features of the support and query set images through a feature extraction network. Few-shot images refer to images in the training set that contain only a small number of labeled images for each class.

[0031] The term "few-shot" needs to be defined from a task perspective: Few-shot image classification tasks involve classifying unknown samples when there are only a few labeled examples per class. This approach aims to address the problem of traditional machine learning methods requiring large amounts of training data and improve the model's generalization capabilities.

[0032] In one embodiment, to address image classification in data-scarce environments, a dataset of a few class-labeled images is obtained and partitioned into multiple support and query sets. A feature extraction network is then used to extract local features from the images in the support and query sets, yielding a high-dimensional representation of each image in feature space. This process ensures that the model can capture key discriminative features from a small number of samples, providing a foundation for subsequent classification.

[0033] Obtain a few-sample image dataset with category labels to construct a training set, and construct a support set and a query set based on the training set;

[0034] The local features of the support set and the query set are obtained through a feature extraction network consisting of a convolutional layer, a batch normalization layer, and a ReLU activation layer, where ReLU is the abbreviation of rectified linear unit.

[0035] In one embodiment, the local features of each image in the support set and query set are obtained through a feature extraction network composed of a convolutional layer, a batch normalization layer, and a ReLU activation layer. The feature extraction network outputs a feature representation whose size is H×W×C

[0036] Where H represents length, W represents width, and C represents the number of channels. Let m = H × W to obtain m local features of each image; the total number of local features of each image in the feature space is m = H × W.

[0037] The local feature set of each support class image set is expressed as: in, is the i-th local feature of the n-th image in the support set, Km is the number of local features of the n-th image in the support set, K is the number of n-th images in the support set, n = 1, 2, ..., N;

[0038] The local feature set of the image in the query set is expressed as: For example, is the jth local feature among the local features 1-m of the qth query set image;

[0039] The above steps optimize the dataset partitioning and feature extraction process. We collect image data with category labels based on the characteristics of the image recognition task and partition the dataset into multiple support and query sets. The feature extraction network extracts local features using convolutional layers, batch normalization layers, and ReLU activation layers. Convolutional layers capture local patterns in the image, batch normalization layers stabilize the training process, and ReLU activation layers add nonlinearity, ensuring the model can accurately extract local feature information even with a small number of samples.

[0040] Among them, ReLU (Rectified Linear Unit) is a rectified linear unit.

[0041] S2. Generate a prototype of each support class based on the local features of the support set image and calculate the weight of each image in the support set;

[0042] Generate prototypes for each supported class through a global average pooling operation;

[0043] In one embodiment, a set of N prototypes is generated by a global average pooling operation: {p1, p2, ..., p N},in The prototype of the nth class of the support set;

[0044] The distance between each image in the support set and each prototype is calculated, and the weight of each image is calculated based on the distance.

[0045] In one embodiment, for the i-th image of the n-th class in the support set Calculate With the c-type prototype p c Euclidean distance Among them, x j is an image The jth local descriptor of image Weight

[0046] The weight distribution mechanism used in the present invention enables the model to focus on representative samples of a category, improves classification accuracy, highlights representative samples of a category and reduces the influence of outlier samples.

[0047] S3. Calculating the weight of the local feature of the support set based on the similarity vector between each local feature of the support set and each prototype, and correcting the local features of the support set image;

[0048] Calculating the similarity vector between each local feature of the support set and each prototype using cosine similarity;

[0049] Inputting the similarity vector into a support weight network to obtain the weight of each local feature of the support set;

[0050] In one embodiment, for the jth local descriptor of the nth class Calculate the cosine similarity between it and all prototypes to obtain a similarity vector Where cos(·,·) represents the cosine similarity function, and the similarity vector Input into the support weight network and get The weight is Among them, the supporting weight network is a fully connected neural network, the number of neurons in the input layer is N, and the number of neurons in the output layer is 1.

[0051] The local features of the support set image are modified based on the weights.

[0052] In one embodiment, the modified support set local feature set is in, is the i-th local feature of the n-th image in the support set, Km is the number of local features of the n-th image in the support set, and K is the number of n-th images in the support set.

[0053] In one embodiment, local feature correction in the support set image is performed by calculating the similarity vector between each local feature and each prototype. This similarity vector is then input into a support weighting network to determine the weight of each local feature. By weighting each feature, the local features of the support set image are adjusted, making each feature point more discriminative. This correction process enhances information in category-relevant areas while reducing interference in irrelevant areas.

[0054] S4. Modifying the local features of the query set image based on the similarity tensor between the local features of the query set image and the support set image after the convolution operation;

[0055] Calculating the similarity tensor between the query set image and each category of the support set and reducing the number of channels by cross-channel pooling operation;

[0056] This invention reduces the number of channels through cross-channel pooling operations, thereby reducing the spatial size of the data, that is, reducing the dimensionality of the image. This helps reduce the computational effort and number of parameters in subsequent layers, while also preventing overfitting.

[0057] In one embodiment, for the K images of the nth class in the support set, a similarity tensor is constructed: The element in the kth channel, ith row, and jth column is represented as The calculation is as follows:

[0058]

[0059] in, represents the similarity between the i-th local feature of the query image and the j-th local feature of the k-th support image of the n-th class, where i∈{1,…,m}, j∈{1,…,m}, represents the jth local feature of the kth image in the nth category of the support set;

[0060] The N similarity tensors are pooled across channels, the number of channels is reduced to 1, and then a 3×3 convolution kernel Θ is used to correct them. Finally, the corrected similarity tensor is obtained through the ReLU layer. The similarity tensor between the corrected query set image and the nth class is exist Find the first k maximum values ​​in , and sum these k values ​​to form an m-dimensional similarity vector The i-th value Represents the similarity between the i-th local feature of the query set image and the n-th category. Repeat the above operation on all similarity tensors to obtain N similarity vectors {α 1 ,α 2 ,…,α N};

[0061] Concatenate the i-th value of N similarity vectors into one vector The vector a i Input into the query weight network to obtain the i-th local feature of the query set image The weight is Finally, each local feature of the query set image is multiplied by its corresponding weight to obtain a set of corrected local features.

[0062]

[0063] Among them, the query weight network is a fully connected neural network, the number of neurons in the input layer is N, and the number of neurons in the output layer is 1.

[0064] The query weight network determines the weights of each local feature of the query set image based on the modified similarity tensor to modify the local features of the query set image.

[0065] After performing a convolution operation on the similarity tensor, the similarity tensor between the query set image and each category of the support set is corrected through a ReLU layer;

[0066] The local features of the query set image are modified based on the similarity tensor between the modified local features of the query set image and the support set image.

[0067] In one embodiment, when correcting the local features of the query set image, first, the similarity tensor between the query set image and each category in the support set is calculated, and the number of channels is reduced through cross-channel pooling to improve computational efficiency. Then, a convolution operation is performed on the similarity tensor, and the similarity tensor between the query set image and each category in the support set is corrected through a ReLU layer. Based on the corrected similarity tensor, the query weight network determines the weight of each local feature and then corrects the local features of the query set image. This process helps to highlight the detailed information related to the category in the query set image.

[0068] S5. Based on the two corrected local features and the weight of each image in the support set, calculate the similarity between the query set image and each support class, and output the similarity in a probabilistic manner to generate an image classification result.

[0069] Based on the two modified local features and the weight of each image in the support set, a relationship matrix is ​​constructed, where each element corresponds to the similarity between the query set image and each support class local feature;

[0070] In one embodiment, based on the two modified local features and the weight of each image in the support set, for the nth class, a relationship matrix is ​​constructed: The element in row i and column j is, in is the local feature of the k-th sample of the n-th class;

[0071] In one embodiment, Find the top k maximum values ​​in each row of , forming a matrix The matrix B n The sum of all elements of is the similarity score s between the query set image and the nth category n ;

[0072] The maximum value of the set number is selected from each row of the relationship matrix to form the final similarity score as the probability output of the classification result.

[0073] In one embodiment, based on the modified local features, the similarity between the query image and each support class is calculated and converted into a probability output to generate a classification result. This method is implemented by constructing a relationship matrix. The maximum value set before each row in the relationship matrix is ​​selected to generate the final similarity score, which is output as the classification probability. This design ensures the reliability of the classification results and accurately reflects the degree of match between the query image and the support class.

[0074] The method further includes: performing model training using a cross entropy loss function, wherein the cross entropy loss function is constructed as follows:

[0075]

[0076] in, Represents the query set image X q The probability of belonging to category c, X q is the query set image, y q ∈{1,…,N} represents the query set image X q Corresponding label;

[0077] In one embodiment, during model training, a cross-entropy loss function is used to optimize classification results. The cross-entropy loss function measures the gap between the model's predicted classification probability and the true class label, ensuring that the model effectively distinguishes between different classes and improving the model's accuracy in classifying small samples.

[0078] The model training adopts the Adam optimization method to optimize the cross entropy loss function to obtain the final few-sample image classification model.

[0079] In one embodiment, the Adam optimization method is used during model training to optimize the cross-entropy loss function. The Adam optimization method combines the advantages of momentum and adaptive learning rate, making the model training process more stable and converging faster. Through this optimization method, the model can adjust parameters more quickly to cope with the complex feature distribution in few-shot classification tasks, thereby achieving efficient learning results on few-shot data.

[0080] Among them, Adam (Adaptive Moment Estimation) is an optimization algorithm that combines the momentum method and the adaptive gradient method, and is mainly used for parameter optimization in deep learning.

[0081] After the training of the few-sample image classification model is completed, the model is used in a new image classification task, and a new image classification result is predicted by the model.

[0082] In one embodiment, after training a few-shot image classification model, the model can be applied to new image classification tasks, using the pre-trained model to perform classification predictions on new images. This approach allows the model to quickly adapt to new scenarios and generate classification results that meet the requirements of few-shot classification, paving the way for the model to be widely used in practical applications.

[0083] Figure 2 A block diagram of a few-sample image classification system of the present invention.

[0084] The present invention also provides a few-sample image classification system, comprising:

[0085] A feature extraction module 201 is configured to obtain a dataset of few-shot images with class labels, divide the dataset into multiple support sets and query sets, and obtain local features of the support set and query set images through a feature extraction network. Few-shot images refer to images in the training set that contain only a small number of labeled images for each class.

[0086] a sample re-estimation module 202 for generating a prototype of each support class based on local features of the support set images and calculating a weight of each image in the support set;

[0087] A support correction module 203 is configured to calculate the weight of the local feature of the support set based on the similarity vector between each local feature of the support set and each prototype, and correct the local feature of the support set image;

[0088] A query correction module 204 is configured to correct the local features of the query set image based on the similarity tensor between the local features of the query set image and the support set image corrected by the convolution operation;

[0089] The output classification module 205 is used to calculate the similarity between the query set image and each support class based on the two corrected local features and the weight of each image in the support set, and output the similarity in a probabilistic manner to generate an image classification result.

[0090] Figure 3 The figure is a schematic structural diagram of an electronic device of the present invention.

[0091] In some embodiments of the present invention, Figure 3 As shown, an electronic device is provided, which includes: electronic devices such as desktop computers, notebooks, handheld computers and cloud servers. The electronic device 3 may include but is not limited to a processor 301 and a memory 302. Those skilled in the art will understand that Figure 3 This is merely an example of the electronic device 3 and does not limit the electronic device 3 . The electronic device 3 may include more or fewer components than shown in the figure, or different components.

[0092] The processor 301 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0093] The memory 302 can be an internal storage unit of the electronic device 3, such as a hard disk or memory of the electronic device 3. The memory 302 can also be an external storage device of the electronic device 3, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. The memory 302 can also include both an internal storage unit of the electronic device 3 and an external storage device. The memory 302 is used to store computer programs and other programs and data required by the electronic device.

[0094] The present invention also provides a computer-readable storage medium. In some embodiments of the present invention, a computer-readable storage medium is provided that, when executed by processor 301, implements the steps of the above method. Therefore, the computer-readable storage medium provided in the third aspect of the present invention has all the technical effects of the above steps and will not be further described here.

[0095] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0096] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0097] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0098] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0099] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present disclosure, and should all be included in the scope of protection of the present disclosure.

Claims

1. A few-sample image classification method, characterized in that: include: S1. Obtain a dataset of few-shot images with class labels, divide the dataset into multiple support sets and query sets, and obtain local features of the support and query set images through a feature extraction network. Few-shot images refer to images in the training set that contain only a small number of labeled images for each class. S2. Generate a prototype of each support class based on the local features of the support set image and calculate the weight of each image in the support set; S3. Calculating the weight of the local feature of the support set based on the similarity vector between each local feature of the support set and each prototype, and correcting the local features of the support set image; S4. Modifying the local features of the query set image based on the similarity tensor between the local features of the query set image and the support set image after the convolution operation; S5. Based on the two modified local features and the weight of each image in the support set, the similarity between the query set image and each support class is calculated, and the similarity is output probabilistically to generate an image classification result; Generating a prototype of each support class based on the local features of the support set image and calculating the weight of each image in the support set includes: Generate prototypes for each supported class through a global average pooling operation; Calculate the distance between each image in the support set and each prototype, and calculate the weight of each image based on the distance, The calculation formula for the weight of each image is as follows: ; in, Representing an image The weight of Indicates the Class images, Indicates the Class prototype; The weight of the local feature of the support set is calculated based on the similarity vector between each local feature of the support set and each prototype, and the local feature of the support set image is modified, including: Calculating the similarity vector between each local feature of the support set and each prototype using cosine similarity; Inputting the similarity vector into a support weight network to obtain the weights of each local feature of the support set, wherein the support weight network is a fully connected neural network; Modifying the local features of the support set image based on the weight; The method of modifying the local features of the query set image based on the similarity tensor between the local features of the query set image and the support set image modified by the convolution operation comprises: Calculate the similarity tensor between the query set image and each category of the support set and reduce the number of channels by cross-channel pooling operation, where the similarity tensor is calculated as follows: ; in, To support the The number of class images, Among them, Channel, Row, No. The elements of the column are calculated as follows: ; in, Represents the query set image local features and Class The first image in the support set The similarity between local features, where , , Express support for the set Class Image No. local features; After performing a convolution operation on the similarity tensor, the similarity tensor between the query set image and each category of the support set is corrected through a ReLU layer; The query weight network determines the weights of each local feature of the query set image based on the modified similarity tensor to modify the local features of the query set image; The similarity between the query set image and each support class is calculated based on the two corrected local features and the weight of each image in the support set, and the similarity is outputted probabilistically to generate an image classification result, including: Based on the two modified local features and the weight of each image in the support set, a relationship matrix is ​​constructed, where each element corresponds to the similarity between the query set image and each support class local feature. The expression for constructing the relationship matrix is ​​as follows: ; in, is the relationship matrix, To support the The number of local features of the class image; Among them, Row, No. The elements of the column are: ; in, For the Class The local features of the samples; Find the preceding Maximum values ​​are formed to form a matrix, and all elements of the matrix are summed to obtain the query set image and the The similarity score of the class is output as the probability of the classification result.

2. The few-sample image classification method according to claim 1, characterized in that: The method of obtaining a few-sample image dataset with category labels, dividing the dataset into multiple support sets and query sets, and obtaining local features of the support set and query set images through a feature extraction network includes: Obtain a few-sample image dataset with category labels to construct a training set, and construct a support set and a query set based on the training set; The local features of the support set and query set are obtained through a feature extraction network consisting of a convolutional layer, a batch normalization layer, and a ReLU activation layer. Among them, ReLU is the abbreviation of rectified linear unit.

3. The method for classifying a small number of images according to claim 1, wherein: The method also includes: using a cross entropy loss function to perform model training.

4. The method for classifying a small number of samples according to claim 3, wherein: The model training adopts the Adam optimization method to optimize the cross entropy loss function to obtain the final few-sample image classification model, wherein Adam is the abbreviation of an optimization algorithm that combines the momentum method and the adaptive gradient method.

5. The method for classifying a small number of samples according to claim 3, wherein: After the training of the few-sample image classification model is completed, the model is used in a new image classification task and the new image classification results are predicted by the model.

6. A system for implementing the few-sample image classification method according to any one of claims 1 to 5, characterized in that: include: A feature extraction module (201) is used to obtain a few-sample image dataset with category labels, divide the dataset into multiple support sets and query sets, and obtain local features of the support set and query set images through a feature extraction network, wherein the few-sample images refer to images in a training set that contain only a small number of labeled images for each category; A sample re-estimation module (202) is used to generate a prototype of each support class based on the local features of the support set images and calculate the weight of each image in the support set; A support correction module (203) is used to calculate the weight of the local feature of the support set based on the similarity vector between each local feature of the support set and each prototype, and to correct the local feature of the support set image; A query correction module (204) is used to correct the local features of the query set image based on the similarity tensor between the local features of the query set image and the support set image corrected by the convolution operation; The output classification module (205) is used to calculate the similarity between the query set image and each support class based on the two corrected local features and the weight of each image in the support set, and output the similarity in a probabilistic manner to generate an image classification result.

Citation Information

Patent Citations

  • Few-sample image classification method and system based on mutual neighbors

    CN112633382A

  • Relational network-based few-sample image classification method

    CN113326892A

Cited By

  • Few-sample image classification method based on hyperbolic space image-text local feature alignment

    CN121280799A