A few-shot image classification method and device introducing local feature alignment and prototype revision mechanism

Abstract

The application discloses a few-shot image classification method and device introducing local feature alignment and prototype correction mechanism, and the method comprises the following steps: 1) dividing a few-shot training data set, and extracting local features and global features of images by using a feature extraction network; 2) realizing local feature alignment by intra-class difference correction and inter-class difference correction, and obtaining similarity of local features of a query image and support classes; 3) correcting class prototypes by using maximum mean difference to calculate weights of global features, and obtaining global similarity of global features of the query image and the support class prototypes by a distance function; 4) combining the global features and the local features to output probability, constructing a loss function to optimize a model, and obtaining a final few-shot image classification model applied to an image classification task. The application discloses a few-shot image classification method, and better image classification performance is realized by combining local features and global features.

Description

Technical Field

[0001] This invention proposes a few-shot image classification method and apparatus that incorporates local feature alignment and prototype correction mechanisms, belonging to the field of image classification technology. Background Technology

[0002] The success of deep learning is attributed to the driving force of large-scale data; collecting and labeling sufficient training samples requires significant time and human resources. In some low-data scenarios, it is impossible to collect large amounts of data, and models trained with limited data perform poorly. This problem hinders the development of deep learning. We call this task of learning from a small number of training samples a few-shot learning task. In recent years, few-shot learning has become a hot research topic, and many different methods have been proposed to address this problem.

[0003] Currently, few-shot learning solutions mainly fall into three categories: model fine-tuning-based methods, data augmentation-based methods, and transfer learning-based methods. Model fine-tuning-based methods pre-train the model on large-scale data and then fine-tune the neural network model's parameters on a target small-shot dataset. Data augmentation-based methods improve sample diversity through data augmentation. Transfer learning-based methods refer to the process of transferring knowledge between two different domains, using knowledge learned in the source domain to aid in learning tasks in the target domain. Transfer learning-based few-shot learning includes two branches: meta-learning-based methods and metric learning-based methods. Meta-learning-based methods learn meta-knowledge from a large number of prior tasks. Meta-knowledge refers to knowledge that can be learned outside the model training process, such as the model's hyperparameters and the initial parameters of the neural network. Meta-knowledge is used to guide the model to learn faster in new tasks. Metric learning-based methods learn an embedding function to map images to an embedding space, classifying images based on the distance between samples in the embedding space. Metric learning methods can be divided into two categories: those based on local features and those based on global features. Global feature-based methods capture the overall semantic information of the entire sample but neglect local details, which are invaluable for few-shot learning tasks. Furthermore, global feature-based methods typically assume the data follows a certain distribution; however, in practice, data distributions may be inconsistent. Local feature-based methods, on the other hand, capture the local structure and details of the sample. Weighting these features can reduce the impact of irrelevant noise on feature representations. However, these methods only consider information from local regions, ignoring contextual information and resulting in an insufficient understanding of the overall data distribution.

[0004] Therefore, using local features alone to represent images is not comprehensive or effective enough. How to reasonably combine global and local features to obtain more comprehensive and richer feature representations, thereby effectively improving the accuracy of few-shot recognition models, is a problem of significant research value. Summary of the Invention

[0005] The purpose of this invention is to provide a few-shot image classification method and apparatus that introduces local feature alignment and prototype correction mechanisms, combining local and global features to obtain a more comprehensive feature representation.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] This invention provides a few-shot image classification method that incorporates local feature alignment and prototype correction mechanisms, including:

[0008] A training set is constructed by acquiring a few-sample image dataset with category labels. A support set and a query set are constructed based on the training set. Local and global features of the support set images and query set images are obtained through a feature extraction network. The support set includes N support class image sets.

[0009] Based on intra-class and inter-class difference correction, the local features of the images in the query set and the image sets of each supporting class are aligned, and the similarity measure between the local features of the query image and the local features of each supporting class image set is calculated.

[0010] A prototype for each support class is generated based on the global features of the image sets of each support class, and a global feature similarity metric is calculated based on the global features of the query image and the prototypes of each support class.

[0011] The probability of the category to which the query image belongs is calculated based on the global feature similarity measure and the local feature similarity measure and is used as the output of the feature extraction network. The feature extraction network is trained and optimized based on the training set to obtain the final few-shot image classification model.

[0012] The few-sample image classification model is used to identify any few-sample image set to obtain the image classification result.

[0013] Furthermore, the feature extraction network includes convolutional layers, batch normalization layers, and ReLU activation layers;

[0014] The feature extraction network obtains the feature representation of each image in the support set and query set. Let m = h × w, and obtain m local features of each image, where h and w represent the length and width of the image, respectively.

[0015] The local feature set of each support class image set is represented as follows: in To support the i-th local feature of the c-th class image in the support set, Km is the number of local features of the c-th class image in the support set, K is the number of the c-th class image in the support set, and c = 1, 2, ..., N;

[0016] The set of local features of the images in the query set is represented as: in Let j be the j-th local feature of the Q-th query image;

[0017] Each and LR Q Global average pooling yields the global feature set of the c-th class of images in the support set: and the global features gr of the Qth query image Q ,in To support the global features of the k-th image of class c in the set.

[0018] Furthermore, the local features of the images in the query set and the image sets of each supporting class are aligned based on intra-class and inter-class difference correction, wherein the intra-class difference correction includes:

[0019] Calculate LR Q Local features and The similarity of local features is used to construct a relation matrix. in express The element in the j-th row and i-th column represents and Cosine similarity between them;

[0020] Find the relation matrix Each row in the vector contains v maximum values, and these v values ​​are summed to form an m-dimensional column vector α. c ;

[0021] Relationship matrix The query image is decomposed into K m×m matrices, each representing the relationship between the query image and each image in class c of the support set. The maximum value of each row in the K matrices is taken to generate K m-dimensional column vectors, which are then concatenated to form a matrix.

[0022] Calculate matrix The variance of each row's values ​​is used to obtain an m-dimensional column vector denoted as σ. c and the matrix The sum of the values ​​in each row yields an m-dimensional column vector η. c ;

[0023] Calculate the intra-class weight vector of local features of the query image for: Where t and h are hyperparameters.

[0024] Furthermore, the local features of the images in the query set and the image sets of each supporting class are aligned based on intra-class and inter-class difference correction, wherein the inter-class difference correction includes:

[0025] For relation matrix For each row, find the largest value and generate an m-dimensional column vector ω. c Since there are N classes in the support set, the generated N m-dimensional column vectors are denoted as follows:

[0026] Calculate R by row inter The variance of the values ​​yields the inter-class weight vector.

[0027] Furthermore, the calculation of the similarity measure between the local features of the query image and the local features of each supporting image set includes:

[0028]

[0029] in, This represents the similarity measure between local features of the query image and local features of class c in the support set.

[0030] Furthermore, the step of generating a prototype for each support class based on the global features of each support class image set includes:

[0031] The difference between supporting images is calculated based on the maximum mean difference, as follows:

[0032] MMD Φ (S c ,S c ′ )=||E(Φ(S c ))-E(Φ(S c ′ ))||;

[0033] Among them, MMD Φ (S c ,S c ′ ) represents S c and S c ′ The maximum mean difference between the samples is given by E(·), which represents the expectation. Φ(·) is the mapping function that maps the samples to the feature space, and ||·|| represents the norm of the vector. S represents the k-th supporting image of class c in the support set. c S represents all images in class c of the support set. c ′ This represents all images in the support set of class c, except for the k-th image.

[0034] The weights supporting the global features of the image are calculated as follows:

[0035]

[0036] in, for The weights;

[0037] Calculate the prototype p of the c-th class image in the support set. c for:

[0038]

[0039] in, This indicates that the i-th image of class c in the support set is supported. This represents the weights of the global features of the i-th supporting image in the c-th class of the support set. This represents the global features of the i-th image in class c of the support set.

[0040] Furthermore, the step of calculating the global feature similarity measure based on the global features of the query image and the prototypes of each supporting class includes:

[0041]

[0042] in, This represents the similarity measure between the global features of the Q-th query image and the global features of the c-th class image prototype in the support set.

[0043] Furthermore, during the training process of the feature extraction network,

[0044] The cross-entropy loss function for the classification task is as follows:

[0045] in, Indicates query image X Q The probability of belonging to category c.

[0046] X Q To query the image, y Q Indicates query image X Q Corresponding tags;

[0047] The Adam optimization method is used to optimize the cross-entropy loss function to obtain the final few-shot image classification model.

[0048] The present invention also provides a few-shot image classification apparatus for implementing the aforementioned few-shot image classification method that incorporates local feature alignment and prototype correction mechanisms, the apparatus comprising:

[0049] The data acquisition module is used to acquire a small sample image dataset with category labels to construct a training set, and to construct a support set and a query set based on the training set. The support set includes N support class image sets.

[0050] Feature extraction networks are used to obtain local and global features of the support set image and the query set image;

[0051] The local feature similarity measurement calculation module is used to align local features of the images in the query set and the image sets of each supporting class based on intra-class difference correction and inter-class difference correction, and to calculate the similarity measure between the local features of the query image and the local features of each supporting class image set.

[0052] The global feature similarity measurement calculation module is used to generate a prototype of each support class based on the global features of each support class image set, and to calculate the global feature similarity measurement based on the global features of the query image and the prototypes of each support class.

[0053] The model training module is used to calculate the probability of the category to which the query image belongs based on the global feature similarity measure and the local feature similarity measure as the output of the feature extraction network, and to train and optimize the feature extraction network based on the training set to obtain the final few-shot image classification model.

[0054] The classification module is used to identify any set of few-sample images using the few-sample image classification model to obtain image classification results.

[0055] The beneficial effects of this invention are as follows:

[0056] This invention provides a few-shot image classification method that incorporates local feature alignment and prototype correction mechanisms. It introduces intra-class and inter-class variability corrections to local features to select the most discriminative information. This process aims to minimize intra-class variability while maximizing inter-class variability, a key innovation that significantly improves model performance. A maximum mean difference (MMD) strategy is employed for global features, further enhancing classification performance by generating more representative prototypes for each support class. Finally, local and global features are combined. Local features help capture the microscopic details of the image, while global features help understand the overall semantics. By organically combining these two types of features, the model performs exceptionally well when processing images of different scales, angles, and lighting conditions. This makes the final classification model more reliable when dealing with complex and variable real-world data. Attached Figure Description

[0057] Figure 1 This is a flowchart of a few-shot image classification method that incorporates local feature alignment and prototype correction mechanisms, provided by the present invention.

[0058] Figure 2 This is a schematic diagram of the local feature difference correction process in this invention;

[0059] Figure 3 This is a schematic diagram of the process for supporting class prototype modification in this invention. Detailed Implementation

[0060] The present invention will be further described below. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0061] This invention provides a few-shot image classification method that incorporates local feature alignment and prototype correction mechanisms. See [link to relevant documentation]. Figure 1 Specifically, it includes:

[0062] S1: Obtain a small sample image dataset with class labels, and divide the image dataset into a training set D. train and test set D test Using D train Construct the support set and query set used in each iteration during training, and obtain the local and global features of the support and query images through a feature extraction network;

[0063] S2: Align local features based on intra-class and inter-class difference corrections, and calculate the similarity measure between local features of the query image and local features of the supporting image;

[0064] S3: Generate a prototype for each support class based on the maximum mean difference, and calculate the Euclidean distance between the global features of the query image and the prototype of each support class to obtain a global feature similarity measure;

[0065] S4: Combine global feature similarity measurement and local feature similarity measurement to obtain the probability of the image belonging to a category, optimize based on the training set, and obtain the final few-shot image classification model;

[0066] S5: Use the acquired few-sample image classification model to identify the image and obtain the image classification result.

[0067] In this invention, the specific implementation process of partitioning a small sample dataset is as follows:

[0068] S1.1: Collect image datasets similar to the image recognition task and labeled with class attributes, based on the characteristics of the task. Divide the collected image datasets into a training set D. train and test set D test .

[0069] It's important to note that the training set is the dataset used to train the model. The model learns feature information from samples in the training set and updates network parameters through backpropagation and optimization algorithms to minimize the loss. The test set is used to evaluate the model's performance on samples of unseen classes. The class information in the training and test sets does not overlap.

[0070] Both the training and test sets are divided into support sets and query sets. The support set contains N classes and K images, which is an N-way K-shot task. The model needs to use the knowledge learned from the support set to classify new samples in the query set.

[0071] In this invention, a few-shot dataset is used to construct an N-way K-shot task for each iteration during training, and each task contains a support set for training the model. and query sets used to evaluate model performance Where N represents the number of categories in the support set, with each category containing K labeled images, and M represents the number of images in the query set. Represents the j-th image in the query set. Represents the i-th image in the support set. and These represent the labels for the corresponding images;

[0072] S1.2: Construct a neural network ψ and randomly initialize the network parameters. The neural network includes convolutional layers, batch normalization layers, and ReLU activation layers.

[0073] The feature representations of images in the support set and query set are obtained through a neural network ψ. Each image is transformed into an h×w×d tensor by ψ, where h, w, and d represent the length, width, and number of channels, respectively.

[0074] Let m = h × w, then each image is transformed into m local features through ψ, and each local feature can represent local information at a certain location in the image.

[0075] The set of feature representations of the c-th class images in the support set after being mapped by the neural network is: in To support the i-th local feature of class c in the support set, Km represents the total number of local features of the c-th image in the support set;

[0076] The set of feature representations of the query image after being mapped by a neural network is: in To query the j-th local feature of the image set;

[0077] S1.3: respectively and LR QGlobal average pooling is used to obtain the global features of the c-th class of images in the support set: Represents the global features of the k-th image of class c in the support set, and the global features gr of each query image. Q .

[0078] In this invention, the specific implementation process of step S2 is described in [reference needed]. Figure 2 ,as follows:

[0079] S2.1: Calculate LR Q Local features and The similarity of local features is used to construct a relation matrix. in The element in the j-th row and i-th column represents and Cosine similarity between them;

[0080] S2.2: Find the relation matrix Each row in the array contains v maximum values, and these v values ​​are summed to form an m-dimensional column vector α. c ;

[0081] S2.3: Align local features based on intra-class and inter-class difference corrections, and calculate the similarity between the local features of the query image and the local features of each class of supporting images.

[0082] Furthermore, step S2.3 specifically includes the following steps:

[0083] S2.3.1: The specific calculation process for intra-class difference correction is as follows:

[0084] Will Decomposed into K m×m matrices by columns. Each matrix represents the relationship between the query image and each image in class c of the support set;

[0085] Generate K m-dimensional column vectors {r} by taking the maximum value of each row of K matrices. c,1 ,…,r c,K}, and splice them together to generate

[0086] calculate The variance of each row's values ​​is used to obtain an m-dimensional column vector denoted as σ. c ,Will The sum of the values ​​in each row yields the column vector η. c ;

[0087] Calculate the intra-class weight vector of local features of the query image: Where t and h are hyperparameters;

[0088] S2.3.2: The specific calculation process for inter-class difference correction is as follows:

[0089] right For each row, find the largest value and generate an m-dimensional column vector ω. c Since there are N classes in the support set, the generated N m-dimensional column vectors are denoted as follows:

[0090] Calculate R by row inter The variance of the values ​​yields the inter-class weight vector.

[0091] S2.3.3: Finally, calculate the similarity measure between the local features of the query image and the local features of the c-th class of supporting images in the support set. as follows:

[0092]

[0093] Where ⊙ represents element-wise multiplication of vectors.

[0094] In this invention, the specific implementation process of step S3 is described in [reference needed]. Figure 3 It includes the following steps:

[0095] S3.1: To account for the influence of different support images, the weight of each support image is calculated using the maximum mean difference (MMD) to correct the class prototype. MMD is a statistical method for measuring the difference between two probability distributions. The core idea of ​​MMD is to measure the difference between two distributions by comparing their means in the feature space. Given two distributions, MMD measures the degree of difference between them by calculating the distance between their sample means in the feature space.

[0096] S3.2: MMD maps the feature space by introducing a kernel function. Given two probability distributions P and Q, assuming the existence of a feature space, the kernel function maps the samples of each distribution into this feature space. Then, the difference between the distributions is measured by calculating the distance between the means of the samples in the feature space.

[0097] Specifically as follows:

[0098] MMD Φ (S c ,S c ′ )=||E(Φ(S c ))-E(Φ(S c ′ ))||;

[0099] Where E(·) represents the expectation, Φ(·) is the mapping function that maps the sample to the feature space, and ||·|| represents the norm of the vector. S represents the k-th supporting image of class c in the support set. c S represents all images in class c of the support set. c ′ This represents all images in the support set of class c, excluding the k-th image. A smaller MMD indicates that the sample is more consistent with the overall data distribution; a larger MMD indicates that the sample is less consistent with the data distribution.

[0100] S3.3: Supports global features of samples The weights are:

[0101]

[0102] S3.4: The final prototype for calculating the c-th class of images in the support set is:

[0103]

[0104] in, This indicates that the i-th image of class c in the support set is supported. This represents the weight of the global features of the i-th supporting image in class c. This represents the global features of the i-th image in class c of the support set;

[0105] S3.5: Calculate the Euclidean distance between the global features of the query image and the prototype of each supporting class to obtain the global feature similarity measure, as follows:

[0106]

[0107] In this invention, step S4 specifically includes the following steps:

[0108] S4.1: Combine global feature similarity measurement and local feature similarity measurement to calculate the query set image X. Q The probability of belonging to category c is:

[0109]

[0110] During training, the cross-entropy loss function for the classification task is constructed as follows:

[0111] The loss function is optimized using the Adam optimization method to obtain the final few-shot image classification model;

[0112] S4.2: Use the obtained few-shot image classification model to classify and identify samples in the test set.

[0113] Based on the above-described inventive concept, the present invention also provides a few-shot image classification apparatus for implementing the aforementioned few-shot image classification method that incorporates local feature alignment and prototype correction mechanisms. The apparatus includes:

[0114] The data acquisition module is used to acquire a small sample image dataset with category labels to construct a training set, and to construct a support set and a query set based on the training set. The support set includes N support class image sets.

[0115] Feature extraction networks are used to obtain local and global features of the support set image and the query set image;

[0116] The local feature similarity measurement calculation module is used to align local features of the images in the query set and the image sets of each supporting class based on intra-class difference correction and inter-class difference correction, and to calculate the similarity measure between the local features of the query image and the local features of each supporting class image set.

[0117] The global feature similarity measurement calculation module is used to generate a prototype of each support class based on the global features of each support class image set, and to calculate the global feature similarity measurement based on the global features of the query image and the prototypes of each support class.

[0118] The model training module is used to calculate the probability of the category to which the query image belongs based on the global feature similarity measure and the local feature similarity measure as the output of the feature extraction network, and to train and optimize the feature extraction network based on the training set to obtain the final few-shot image classification model.

[0119] The classification module is used to identify any set of few-sample images using the few-sample image classification model to obtain image classification results.

[0120] It is worth noting that this device embodiment corresponds to the above method embodiment. The implementation methods of the above method embodiments are all applicable to this device embodiment and can achieve the same or similar technical effects, so they will not be described in detail here.

[0121] In summary, this invention combines local and global features when performing few-shot image recognition. Local features incorporate intra-class and inter-class discriminative correction modules to select the most discriminative information, while global features refine the prototypes using Maximum Mean Difference (MMD). This invention aligns local features across different images by learning their weights, a process designed to minimize intra-class discriminative differences while maximizing inter-class discriminative differences—a key innovation in improving model performance. Secondly, the global features employ a Maximum Mean Difference (MMD) strategy, further enhancing classification performance by generating more representative prototypes for each support class. Finally, by combining local and global features—local features helping to capture microscopic details of the image, and global features aiding in understanding the overall semantics—the model performs exceptionally well when processing images of varying scales, angles, and lighting conditions. This makes the final classification model more reliable when dealing with complex and variable real-world data.

[0122] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A few-shot image classification method incorporating local feature alignment and prototype correction mechanisms, characterized in that, include: A training set is constructed by acquiring a few-sample image dataset with category labels. A support set and a query set are constructed based on the training set. Local and global features of the support set images and query set images are obtained through a feature extraction network. The support set includes N support class image sets. Based on intra-class and inter-class difference correction, the local features of the images in the query set and the image sets of each supporting class are aligned, and the similarity measure between the local features of the query image and the local features of each supporting class image set is calculated. A prototype for each support class is generated based on the global features of the image sets of each support class, and a global feature similarity metric is calculated based on the global features of the query image and the prototypes of each support class. The probability of the category to which the query image belongs is calculated based on the global feature similarity measure and the local feature similarity measure and is used as the output of the feature extraction network. The feature extraction network is trained and optimized based on the training set to obtain the final few-shot image classification model. The few-sample image classification model is used to identify any few-sample image set to obtain the image classification result.

2. The few-shot image classification method according to claim 1, which incorporates local feature alignment and prototype correction mechanisms, is characterized in that... The feature extraction network includes convolutional layers, batch normalization layers, and ReLU activation layers; The feature extraction network obtains the feature representation of each image in the support set and query set. Let m = h × w, and obtain m local features of each image, where h and w represent the length and width of the image, respectively. The local feature set of each support class image set is represented as follows: in To support the i-th local feature of the c-th class image in the support set, Km is the number of local features of the c-th class image in the support set, and K is the number of the c-th class image in the support set. c = 1, 2, ..., N; The set of local features of the images in the query set is represented as: in Let j be the j-th local feature of the Q-th query image; Each and LR Q Global average pooling yields the global feature set of the c-th class of images in the support set: and the global features gr of the Qth query image Q ,in To support the global features of the k-th image of class c in the set.

3. The few-shot image classification method according to claim 2, which incorporates local feature alignment and prototype correction mechanisms, is characterized in that... The method of aligning local features between images in the query set and image sets of each supporting class based on intra-class and inter-class difference correction includes: Calculate LR Q Local features and The similarity of local features is used to construct a relation matrix. in express The element in the j-th row and i-th column represents and Cosine similarity between them; Find the relation matrix Each row in the vector contains v maximum values, and these v values ​​are summed to form an m-dimensional column vector α. c ; Relationship matrix The data is decomposed into K m×m matrices, each representing the relationship between the query image and each image in the c-th class of the support set. Take the maximum value from each row of K matrices to generate K m-dimensional column vectors, and then concatenate them to form a matrix. Calculate matrix The variance of each row's values ​​is used to obtain an m-dimensional column vector denoted as σ. c and the matrix The sum of the values ​​in each row yields an m-dimensional column vector η. c ; Calculate the intra-class weight vector of local features of the query image for: Where t and h are hyperparameters.

4. The few-shot image classification method according to claim 3, which incorporates local feature alignment and prototype correction mechanisms, is characterized in that... The method of aligning local features between images in the query set and image sets of each supporting class based on intra-class and inter-class difference correction includes: For relation matrix For each row, find the largest value and generate an m-dimensional column vector ω. c Since there are N classes in the support set, the generated N m-dimensional column vectors are denoted as follows: Calculate R by row inter The variance of the values ​​yields the inter-class weight vector.

5. A few-shot image classification method incorporating local feature alignment and prototype correction mechanisms according to claim 4, characterized in that, The similarity metric for calculating the local features of the query image and the local features of each supporting image set includes: in, This represents the similarity measure between local features of the query image and local features of class c in the support set.

6. A few-shot image classification method incorporating local feature alignment and prototype correction mechanisms according to claim 5, characterized in that, The process of generating a prototype for each support class based on global features of each support class image set includes: The difference between supporting images is calculated based on the maximum mean difference, as follows: 5000 Φ (S c ,S c ′)=||E(Φ(S c ))-E(Φ(S c ′))||; Among them, MMD Φ (S c ,S c ′) represents S c and S c The maximum mean difference between the samples is E(·), which represents the expectation; Φ(·) is the mapping function that maps the samples to the feature space; and ||·|| represents the norm of the vector. S represents the k-th supporting image of class c in the support set. c S represents all images in class c of the support set. c ′ represents all images in the support set of class c except for the k-th image; The weights supporting the global features of the image are calculated as follows: in, for The weights; Calculate the prototype p of the c-th class image in the support set. c for: in, This indicates that the i-th image of class c in the support set is supported. This represents the weights of the global features of the i-th supporting image in the c-th class of the support set. This represents the global features of the i-th image in class c of the support set.

7. A few-shot image classification method incorporating local feature alignment and prototype correction mechanisms according to claim 6, characterized in that, The calculation of global feature similarity measure based on global features of the query image and prototypes of each supporting class includes: in, This represents the similarity measure between the global features of the Q-th query image and the global features of the c-th class image prototype in the support set.

8. A few-shot image classification method incorporating local feature alignment and prototype correction mechanisms according to claim 7, characterized in that, During the training of the feature extraction network The cross-entropy loss function for the classification task is as follows: in, Indicates query image X Q The probability of belonging to category c. X Q To query the image, y Q Indicates query image X Q Corresponding tags; The Adam optimization method is used to optimize the cross-entropy loss function to obtain the final few-shot image classification model.

9. A few-sample image classification device, characterized in that, The apparatus for implementing the few-shot image classification method according to any one of claims 1 to 8, which incorporates local feature alignment and prototype correction mechanisms, comprises: The data acquisition module is used to acquire a small sample image dataset with category labels to construct a training set, and to construct a support set and a query set based on the training set. The support set includes N support class image sets. Feature extraction networks are used to obtain local and global features of the support set image and the query set image; The local feature similarity measurement calculation module is used to align local features of the images in the query set and the image sets of each supporting class based on intra-class difference correction and inter-class difference correction, and to calculate the similarity measure between the local features of the query image and the local features of each supporting class image set. The global feature similarity measurement calculation module is used to generate a prototype of each support class based on the global features of each support class image set, and to calculate the global feature similarity measurement based on the global features of the query image and the prototypes of each support class. The model training module is used to calculate the probability of the category to which the query image belongs based on the global feature similarity measure and the local feature similarity measure as the output of the feature extraction network, and to train and optimize the feature extraction network based on the training set to obtain the final few-shot image classification model. The classification module is used to identify any set of few-sample images using the few-sample image classification model to obtain image classification results.

Images