Few-sample image classification method and system
By introducing sample adaptive and local feature correction mechanisms in the few-sample image classification method, the problem of high overfitting and calculation costs in data scarce scenarios is solved, and high-precision and low-cost small-sample image classification is achieved, which improves the generalization ability of the model.
Patent Information
- Application Number
- CN202411778425.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-05
- Publication Date
- 2025-04-01
- Estimated Expiration
- 2044-12-05
AI Technical Summary
In application scenarios where data is scarce, traditional deep learning methods are prone to overfitting the model and degrading classification performance. The existing few-sample learning methods have problems such as overfitting not being completely solved, high computational cost, and no consideration of the differences in local feature importance.
A method of classification of few samples is proposed. By obtaining the data set of few samples with category labels, dividing the support set and query set, using the feature extraction network to obtain local features, and generating the prototype of each support class, calculating the weight of the support set image, correcting the local features of the support set and query set, and finally computing the similarity between the query set image and each support class, performing probability output to generate image classification results.
It significantly improves the accuracy of image classification in small samples, reduces the calculation cost, reduces the negative impact of outliers on the model, improves the generalization ability of the model, and is suitable for various data scarce scenarios.
Smart Images

Figure CN119741534A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image classification, and in particular to a few-sample image classification method and system. Background Art
[0002] Traditional deep learning methods rely heavily on a large amount of labeled data, and the collection and labeling of this data usually requires a lot of time, money, and manpower. However, in some data-scarce application scenarios, such as rare species identification, military target detection, rare case diagnosis, etc., it is extremely difficult or even impossible to collect enough training samples. In the above cases, if traditional deep learning methods are still used, the model will be overfitted and the classification performance will be greatly reduced. On the contrary, humans can quickly identify new categories with very few examples, which inspired the research direction of few-shot learning. At present, the mainstream methods of few-shot learning research can be roughly divided into three categories: methods based on data enhancement, methods based on meta-learning, and methods based on metric learning.
[0003] However, 1. Data augmentation methods expand the amount of training data by performing various transformations on existing samples or generating synthetic samples to transform few-sample learning into regular classification tasks, thereby improving the generalization performance of the model. However, although the above methods can alleviate the problem of lack of samples to a certain extent, overfitting has not been completely solved. 2. The generalization performance of meta-learning usually depends on the prior task. Although the meta-learning-based method has achieved good results, it is also a great challenge for training due to its complex memory addressing structure and high computational overhead. 3. Methods based on metric learning, including methods based on global features and methods based on local features. Among them, methods based on global features: the use of global features is prone to lose detailed information in the image, and cannot accurately reflect the distribution of categories, resulting in difficulty in obtaining good classification results in few-sample scenarios. Methods based on local features: Although local feature methods capture image detail information, most of them do not consider the differences in the importance of each local feature, and the information of irrelevant areas in the image may mislead the classification results; in addition, existing methods usually treat all samples in the task equally, and do not consider the potential interference of outlier samples on the model. Summary of the invention
[0004] The present invention aims to provide a few-sample image classification method and system to at least solve or improve one of the above-mentioned technical problems.
[0005] In view of this, a first aspect of the present invention is to provide a few-sample image classification method, comprising:
[0006] S1. Obtain a few-sample image dataset with category labels, divide the dataset into multiple support sets and query sets, and obtain local features of the support set and query set images through a feature extraction network, wherein the few-sample image refers to a training set that contains only a small number of labeled images for each category;
[0007] S2. Generate a prototype of each support class based on the local features of the support set image, and calculate the weight of each image in the support set;
[0008] S3. Calculate the weight of the local feature of the support set based on the similarity vector between each local feature of the support set and each prototype, and modify the local features of the support set image;
[0009] S4. Based on the similarity tensor between the local features of the query set image and the support set image corrected by the convolution operation, correct the local features of the query set image;
[0010] S5. Based on the two corrected local features and the weight of each image in the support set, the similarity between the query set image and each support class is calculated, and the similarity is output probabilistically to generate an image classification result.
[0011] A second aspect of the present invention is a few-sample image classification system, comprising:
[0012] The feature extraction module 201 is used to obtain a few-sample image dataset with category labels, divide the dataset into multiple support sets and query sets, and obtain local features of the support set and query set images through a feature extraction network, wherein the few-sample images refer to the training set that contains only a small number of labeled images for each category;
[0013] A sample re-estimation module 202, for generating a prototype of each support class based on the local features of the support set images, and calculating the weight of each image of the support set;
[0014] A support correction module 203, used to calculate the weight of the local feature of the support set based on the similarity vector between each local feature of the support set and each prototype, and correct the local feature of the support set image;
[0015] A query correction module 204, configured to correct the local features of the query set image based on the similarity tensor between the local features of the query set image and the support set image corrected by the convolution operation;
[0016] The output classification module 205 is used to calculate the similarity between the query set image and each support class based on the two corrected local features and the weight of each image in the support set, and output the similarity in a probabilistic manner to generate an image classification result.
[0017] Compared with the prior art, the present invention has the following beneficial effects:
[0018] 1. The present invention significantly improves the accuracy of few-sample image classification by introducing sample adaptation and local feature correction mechanisms, calculates the distance from each support set image to the category prototype, dynamically assigns weights to each sample, and focuses on representative samples, which not only reduces the negative impact of outlier samples on the model, but also reduces the computational cost;
[0019] 2. The present invention uses a neural network to model the importance of each local feature for the current task, and makes corrections on this basis, fully considering the importance differences of each local feature, focusing on the discriminative features in the image that are closely related to the category, and improving the classification performance of the model;
[0020] 3. The classification method of the present invention combines sample adaptation and local feature correction mechanism, so that the model can adapt to the changeable and complex real data distribution when processing small sample data, has higher generalization ability, and is suitable for application in various data-scarce scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0022] Figure 1 A flowchart of a few-sample image classification method of the present invention;
[0023] Figure 2 is a block diagram of a few-sample image classification system of the present invention;
[0024] Figure 3 The figure is a schematic diagram of the structure of an electronic device of the present invention. DETAILED DESCRIPTION
[0025] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0026] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.
[0027] The following describes a few-sample image classification method and system according to some embodiments of the present invention.
[0028] Figure 1The present invention is a flowchart of a few-sample image classification method.
[0029] An embodiment of the present invention proposes a method for classifying a few-sample image. In some embodiments of the present invention, the method includes:
[0030] S1. Obtain a few-sample image dataset with category labels, divide the dataset into multiple support sets and query sets, and obtain local features of the support set and query set images through a feature extraction network, wherein the few-sample image refers to a training set that contains only a small number of labeled images for each category;
[0031] Among them, few samples need to be defined from the perspective of tasks: the few-sample image classification task refers to classifying unknown samples when there are only a few labeled samples for each class. This method aims to solve the problem that traditional machine learning methods require a large amount of training data and improve the generalization ability of the model.
[0032] In one embodiment, to solve the image classification problem in the case of data scarcity, a small number of sample image datasets with category labels are obtained, and the datasets are divided into multiple support sets and query sets. A feature extraction network is used to extract local features from the images of the support set and the query set to obtain a high-dimensional representation of each image in the feature space. This process ensures that the model can capture key discriminative features from a small number of samples, providing a basis for subsequent classification.
[0033] Obtain a few sample image dataset with category labels to build a training set, and build a support set and a query set based on the training set;
[0034] The local features of the support set and the query set are obtained through a feature extraction network consisting of a convolutional layer, a batch normalization layer, and a ReLU activation layer, wherein ReLU is the abbreviation of rectified linear unit.
[0035] In one embodiment, the local features of each image in the support set and the query set are obtained through a feature extraction network composed of a convolutional layer, a batch normalization layer, and a ReLU activation layer. The feature extraction network outputs a feature representation whose size is H×W×C
[0036] Among them, H represents length, W represents width, C represents the number of channels, and m=H×W is set to obtain m local features of each image; the total number of local features of each image in the feature space is m=H×W.
[0037] The local feature set of each support class image set is expressed as: in, is the ith local feature of the nth image in the support set, Km is the number of local features of the nth image in the support set, K is the number of nth images in the support set, n = 1, 2, ..., N;
[0038] The local feature set of the image in the query set is expressed as: For example, is the jth local feature among the local features 1-m of the qth query set image;
[0039] The above steps optimize the partitioning of the dataset and the feature extraction process. By collecting image data with category labels according to the characteristics of the image recognition task, the dataset is divided into multiple support sets and query sets. The feature extraction network extracts local features through convolutional layers, batch normalization layers, and ReLU activation layers. The convolutional layer is used to capture local patterns in the image, the batch normalization layer stabilizes the training process, and the ReLU activation layer increases nonlinearity to ensure that the model can accurately extract local feature information under few-sample conditions.
[0040] Among them, ReLU (Rectified Linear Unit) is a rectified linear unit.
[0041] S2. Generate a prototype of each support class based on the local features of the support set image, and calculate the weight of each image in the support set;
[0042] Generate prototypes for each supported class through a global average pooling operation;
[0043] In one embodiment, a set of N prototypes is generated by a global average pooling operation: {p1, p2, ..., p N},in The prototype of the nth class of the support set;
[0044] The distance between each image in the support set and each prototype is calculated, and the weight of each image is calculated based on the distance.
[0045] In one embodiment, for the i-th image of the n-th class in the support set Calculate With the c-type prototype p c The Euclidean distance Among them, x j is an image The jth local descriptor of image Weight
[0046] The weight allocation mechanism used in the present invention enables the model to focus on representative samples of a category, improves classification accuracy, highlights representative samples of a category and reduces the impact of outlier samples.
[0047] S3. Calculate the weight of the local feature of the support set based on the similarity vector between each local feature of the support set and each prototype, and modify the local features of the support set image;
[0048] Calculate the similarity vector between each local feature of the support set and each prototype using cosine similarity;
[0049] Inputting the similarity vector into a support weight network to obtain the weight of each local feature of the support set;
[0050] In one embodiment, for the jth local descriptor of the nth class Calculate the cosine similarity between it and all prototypes to get a similarity vector Where cos(·,·) represents the cosine similarity function, and the similarity vector Input into the support weight network and get The weight is Among them, the supporting weight network is a fully connected neural network, the number of neurons in the input layer is N, and the number of neurons in the output layer is 1.
[0051] The local features of the support set image are modified based on the weights.
[0052] In one embodiment, the modified support set local feature set is in, is the ith local feature of the nth category image in the support set, Km is the number of local features of the nth category image in the support set, and K is the number of the nth category images in the support set.
[0053] In one embodiment, for the local feature correction of the support set image, the similarity vector between each local feature and each type of prototype is calculated, and the similarity vector is input into the support weight network to determine the weight of each local feature. By weighted correction of each feature, the local features of the support set image are adjusted so that each feature point is more discriminative. This correction process can enhance the information of the category-related area while reducing the interference of the irrelevant area.
[0054] S4. Based on the similarity tensor between the local features of the query set image and the support set image corrected by the convolution operation, correct the local features of the query set image;
[0055] Calculating the similarity tensor between the query set image and each category of the support set and reducing the number of channels by cross-channel pooling operation;
[0056] The present invention reduces the number of channels through cross-channel pooling operations, reduces the spatial size of the data, that is, reduces the dimension of the image, helps to reduce the amount of calculation and the number of parameters of subsequent layers, and prevents overfitting.
[0057] In one embodiment, for the K images of the nth class in the support set, a similarity tensor is constructed: The element in the kth channel, ith row, and jth column is represented as The calculation is as follows:
[0058]
[0059] in, represents the similarity between the i-th local feature of the query set image and the j-th local feature of the k-th support set image of the n-th category, where i∈{1,…,m}, j∈{1,…,m}, represents the jth local feature of the kth image of the nth category in the support set;
[0060] The N similarity tensors are pooled across channels, the number of channels is reduced to 1, and then a 3×3 convolution kernel θ is used to correct them. Finally, the corrected similarity tensor is obtained through the ReLU layer. The similarity tensor between the corrected query set image and the nth class is exist Find the first k maximum values in , and sum these k values to form an m-dimensional similarity vector The i-th value Represents the similarity between the i-th local feature of the query set image and the n-th category. Repeat the above operation on all similarity tensors to obtain N similarity vectors {α 1 ,α 2 ,…,α N};
[0061] Concatenate the i-th value of N similarity vectors into one vector The vector a i Input into the query weight network to obtain the i-th local feature of the query set image The weight is Finally, each local feature of the query set image is multiplied by its corresponding weight to obtain a set of corrected local features.
[0062]
[0063] Among them, the query weight network is a fully connected neural network, the number of neurons in the input layer is N, and the number of neurons in the output layer is 1.
[0064] Among them, the query weight network determines the weights of each local feature of the query set image based on the corrected similarity tensor to correct the local features of the query set image.
[0065] After performing a convolution operation on the similarity tensor, the similarity tensor between the query set image and each category of the support set is corrected through a ReLU layer;
[0066] The local features of the query set image are modified based on the similarity tensor between the modified local features of the query set image and the support set image.
[0067] In one embodiment, when correcting the local features of the query set image, first, the similarity tensor between the query set image and each category of the support set is calculated, and the number of channels is reduced by cross-channel pooling operation to improve computational efficiency. Then, a convolution operation is performed on the similarity tensor, and the similarity tensor between the query set image and each category of the support set is corrected by a ReLU layer. The query weight network determines the weight of each local feature based on the corrected similarity tensor, and then corrects the local features of the query set image. This process helps to highlight the detailed information related to the category in the query set image.
[0068] S5. Based on the two corrected local features and the weight of each image in the support set, the similarity between the query set image and each support class is calculated, and the similarity is output probabilistically to generate an image classification result.
[0069] Based on the two modified local features and the weight of each image in the support set, a relationship matrix is constructed, each element of which corresponds to the similarity between the query set image and each support class local feature;
[0070] In one embodiment, based on the two modified local features and the weight of each image in the support set, for the nth class, a relationship matrix is constructed: The element in row i and column j is, in is the local feature of the kth sample of the nth class;
[0071] In one embodiment, Find the first k maximum values in each row of , forming a matrix The matrix B n The sum of all elements of is the similarity score s between the query set image and the nth category. n ;
[0072] The maximum value of the pre-set number is selected from each row of the relationship matrix to form the final similarity score as the probability output of the classification result.
[0073] In one embodiment, based on the corrected local features, the similarity between the query set image and each support class is calculated, and the similarity is converted into a probability output to generate a classification result. The method is implemented by constructing a relationship matrix. The maximum value of the set number before each row in the relationship matrix is selected to generate the final similarity score as the classification probability output. This design ensures the reliability of the classification result and can accurately reflect the matching degree between the query set image and the support class.
[0074] The method further includes: using a cross entropy loss function to perform model training, wherein the cross entropy loss function is constructed as follows:
[0075]
[0076] in, Represents the query set image X q The probability of belonging to category c, X q is the query set image, y q ∈{1,…,N} represents the query set image X q Corresponding label;
[0077] In one embodiment, during the model training process, a cross entropy loss function is used to optimize the classification effect. The cross entropy loss function is used to measure the gap between the classification probability predicted by the model and the true category label, ensuring that the model effectively distinguishes different categories and improving the accuracy of the model for classifying a small number of samples.
[0078] The model training adopts the Adam optimization method to optimize the cross entropy loss function to obtain the final few-sample image classification model.
[0079] In one embodiment, the cross entropy loss function is optimized using the Adam optimization method in model training. The Adam optimization method combines the advantages of momentum and adaptive learning rate to make the model training process more stable and converge faster. Through this optimization method, the model can adjust parameters faster to cope with complex feature distributions in few-sample classification tasks, thereby achieving efficient learning effects on few-sample data.
[0080] Among them, Adam (Adaptive Moment Estimation) is an optimization algorithm that combines the momentum method and the adaptive gradient method, and is mainly used for parameter optimization in deep learning.
[0081] After the training of the few-sample image classification model is completed, the model is used in a new image classification task, and a new image classification result is predicted by the model.
[0082] In one embodiment, after the training of the few-sample image classification model is completed, the model can be applied to new image classification tasks, and the new images can be classified and predicted by the pre-trained model. This method allows the model to quickly adapt to new scenarios and generate classification results to meet the classification requirements under few-sample conditions, providing feasibility for the promotion of the model in practical applications.
[0083] Figure 2 A block diagram of a few-sample image classification system of the present invention.
[0084] The present invention also provides a few-sample image classification system, comprising:
[0085] The feature extraction module 201 is used to obtain a few-sample image dataset with category labels, divide the dataset into multiple support sets and query sets, and obtain local features of the support set and query set images through a feature extraction network, wherein the few-sample images refer to the training set that contains only a small number of labeled images for each category;
[0086] A sample re-estimation module 202, for generating a prototype of each support class based on the local features of the support set images, and calculating the weight of each image of the support set;
[0087] A support correction module 203, used to calculate the weight of the local feature of the support set based on the similarity vector between each local feature of the support set and each prototype, and correct the local feature of the support set image;
[0088] A query correction module 204, configured to correct the local features of the query set image based on the similarity tensor between the local features of the query set image and the support set image corrected by the convolution operation;
[0089] The output classification module 205 is used to calculate the similarity between the query set image and each support class based on the two corrected local features and the weight of each image in the support set, and output the similarity in a probabilistic manner to generate an image classification result.
[0090] Figure 3 The figure is a schematic diagram of the structure of an electronic device of the present invention.
[0091] In some embodiments of the present invention, Figure 3 As shown, an electronic device is provided, which includes: electronic devices such as desktop computers, notebooks, handheld computers, and cloud servers. The electronic device 3 may include but is not limited to a processor 301 and a memory 302. Those skilled in the art will understand that Figure 3 The electronic device 3 is merely an example and does not limit the electronic device 3 , and may include more or less components than those shown in the figure, or different components.
[0092] The processor 301 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0093] The memory 302 may be an internal storage unit of the electronic device 3, for example, a hard disk or memory of the electronic device 3. The memory 302 may also be an external storage device of the electronic device 3, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 3. The memory 302 may also include both an internal storage unit of the electronic device 3 and an external storage device. The memory 302 is used to store computer programs and other programs and data required by the electronic device.
[0094] The present invention also provides a computer-readable storage medium. In some embodiments of the present invention, a computer-readable storage medium is provided, and when the computer-readable storage medium is executed by the processor 301, the steps of the above method are implemented. Therefore, the computer-readable storage medium provided in the third aspect of the present invention has all the technical effects of the above steps, which will not be repeated here.
[0095] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0096] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.
[0097] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.
[0098] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.
[0099] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be included in the protection scope of the present disclosure.
Claims
1. A few-sample image classification method, characterized in that: include: S1. Obtain a few-sample image dataset with category labels, divide the dataset into multiple support sets and query sets, and obtain local features of the support set and query set images through a feature extraction network, wherein the few-sample image refers to a training set that contains only a small number of labeled images for each category; S2. Generate a prototype of each support class based on the local features of the support set image, and calculate the weight of each image in the support set; S3. Calculate the weight of the local feature of the support set based on the similarity vector between each local feature of the support set and each prototype, and modify the local features of the support set image; S4. Based on the similarity tensor between the local features of the query set image and the support set image corrected by the convolution operation, correct the local features of the query set image; S5. Based on the two corrected local features and the weight of each image in the support set, the similarity between the query set image and each support class is calculated, and the similarity is output probabilistically to generate an image classification result.
2. The method for classifying a few-sample image according to claim 1, characterized in that: The method of obtaining a few-sample image dataset with category labels, dividing the dataset into a plurality of support sets and query sets, and obtaining local features of the support set and query set images through a feature extraction network includes: Obtain a few sample image dataset with category labels to build a training set, and build a support set and a query set based on the training set; The local features of the support set and the query set are obtained through a feature extraction network consisting of a convolutional layer, a batch normalization layer, and a ReLU activation layer. Among them, ReLU is the abbreviation of rectified linear unit.
3. The few-sample image classification method according to claim 1, characterized in that: The step of generating a prototype of each support class based on the local features of the support set image and calculating the weight of each image of the support set includes: Generate prototypes for each supported class through a global average pooling operation; The distance between each image in the support set and each prototype is calculated, and the weight of each image is calculated based on the distance.
4. The few-sample image classification method according to claim 1, characterized in that: The weight of the local feature of the support set is calculated based on the similarity vector between each local feature of the support set and each prototype, and the local feature of the support set image is corrected, including: Calculate the similarity vector between each local feature of the support set and each prototype using cosine similarity; Inputting the similarity vector into a support weight network to obtain the weight of each local feature of the support set; The local features of the support set image are modified based on the weights.
5. The method for classifying a few-sample image according to claim 1, characterized in that: The method of modifying the local features of the query set image based on the similarity tensor between the local features of the query set image and the support set image modified by the convolution operation comprises: Calculating the similarity tensor between the query set image and each category of the support set and reducing the number of channels by cross-channel pooling operation; After performing a convolution operation on the similarity tensor, the similarity tensor between the query set image and each category of the support set is corrected through a ReLU layer; The local features of the query set image are modified based on the similarity tensor between the modified local features of the query set image and the support set image.
6. The method for classifying a few-sample image according to claim 1, characterized in that: The similarity between the query set image and each support class is calculated based on the two corrected local features and the weight of each image in the support set, and the similarity is outputted probabilistically to generate an image classification result, including: Based on the two modified local features and the weight of each image in the support set, a relationship matrix is constructed, each element of which corresponds to the similarity between the query set image and each support class local feature; A preset maximum value is selected from each row of the relationship matrix to form a final similarity score as a probability output of the classification result.
7. The method for classifying a few-sample image according to claim 1, characterized in that: The method also includes: using a cross entropy loss function to perform model training.
8. The method for classifying a few-sample image according to claim 7, characterized in that: The model training adopts the Adam optimization method to optimize the cross entropy loss function to obtain the final few-sample image classification model, wherein Adam is the abbreviation of an optimization algorithm that combines the momentum method and the adaptive gradient method.
9. The method for classifying a few-sample image according to claim 7, characterized in that: After the training of the few-sample image classification model is completed, the model is used in a new image classification task, and a new image classification result is predicted by the model.
10. A system for implementing the few-sample image classification method according to any one of claims 1 to 9, characterized in that: include: A feature extraction module (201) is used to obtain a few-sample image dataset with category labels, divide the dataset into a plurality of support sets and query sets, and obtain local features of the support set and query set images through a feature extraction network, wherein the few-sample images refer to images in a training set that contain only a small number of labeled images for each category; A sample re-estimation module (202), used to generate a prototype of each support class based on the local features of the support set image, and calculate the weight of each image in the support set; A support correction module (203) is used to calculate the weight of the local feature of the support set based on the similarity vector between each local feature of the support set and each prototype, and to correct the local feature of the support set image; A query correction module (204) is used to correct the local features of the query set image based on the similarity tensor between the local features of the query set image and the support set image corrected by the convolution operation; The output classification module (205) is used to calculate the similarity between the query set image and each support class based on the two corrected local features and the weight of each image in the support set, and output the similarity in a probabilistic manner to generate an image classification result.
Citation Information
Patent Citations
Few-sample image classification method and system based on mutual neighbors
CN112633382A
Relational network-based few-sample image classification method
CN113326892A
Small sample image classification method and device based on sample generation and storage medium
CN116168257A
Less-sample image classification method and device introducing local feature alignment and prototype correction mechanism
CN117576471A
Device and method for performing few-shot learning using multi similarity and attention network
KR102569598B1
Cited By
Sample generation method and device, image detection method and model training method
CN120673189A
Sample generation methods, devices, image detection methods, and model training methods
CN120673189B