A method for small sample image classification

Abstract

A method for classifying small sample images belongs to the field of image classification technology. The invention is to solve the problem of poor classification effect of small sample images. The invention performs feature extraction on the image to be classified and each of several categories of known category images respectively. In the process of feature extraction, the image is first sent to Swin Transformer for feature extraction, and then sent to spatial attention and channel attention CSAM processing; the vector corresponding to one image of the image to be classified after CSAM processing and the vector corresponding to one image of any category of known category images after CSAM processing are used to obtain the optimal matching flow using EMD distance, and the similarity score between the image representations is calculated based on the best match, and the category of the image to be classified is determined according to the score.

Description

Technical Field

[0001] The present invention belongs to the technical field of image classification, and in particular relates to a method for classifying small sample images. Background Art

[0002] With the advent of the big data era, deep learning models have achieved relatively good results in various fields of computer vision. However, the high accuracy of deep learning models often requires a large amount of training data and a lot of manual annotation. In order to save these costs, people think about how to accurately classify objects with a small amount of data. For example, a young child has never seen a rabbit. When he is given a card with a rabbit on it, when he sees a real rabbit, it will immediately appear in his mind and he will recognize the rabbit. Even if the size, hair color, and body posture of the rabbit are very different from the picture he has seen, he can still accurately identify it. Inspired by the human learning perspective, the concept of few-shot learning was proposed, making machine learning closer to human thinking.

[0003] In the standard few-shot learning task, the dataset is divided into two parts: one with C base The base set of classes and one with C novel A new set of classes, where C base is an auxiliary dataset, the purpose of which is to train the classifier through transfer learning. novel is a dataset for performing task classification. Training is usually done in C base The goal is to transfer the knowledge learned to C-based novel During testing, a support set S and a query set Q are created for each task. The support set S contains N classes, each with K images. The query set Q includes N×Q unlabeled images. In most literature, N is set to 5 and K is set to 1 or 5.

[0004] The core problem faced by small sample learning is that there are too few labeled data. This type of method based on data augmentation generates data in different ways to expand the data set, thereby alleviating the data sparsity problem faced in the small sample learning process. Data augmentation can be achieved in three main ways: the first is to transform the image or use a generative model to generate false data, and mix the generated data with the original data to achieve the purpose of expanding the data; the second is to use semi-supervised learning to learn from a large amount of unlabeled data using a small amount of labeled information to improve the model learning effect; the third is to let the model perform self-supervised learning. This training method is an effective way to avoid using labeled data. In deep learning, data augmentation is a common technique. This method expands the training data set through various transformation methods. Data augmentation methods are also applicable to small sample learning. In the process of small sample image recognition, the robustness of the feature encoding network can be effectively enhanced by rotating, translating, flipping, cropping or adding noise to the image, thereby improving the effect of small sample learning.

[0005] The research on small sample learning can be traced back to a generative target classification model proposed by Li et al. in 2006. This model learns target categories from a small number of labeled samples by applying the Bayesian theory framework to achieve the same learning effect as humans. At present, deep learning technology has been widely used in various fields, and small sample learning based on deep learning has also made good progress. However, since small sample learning was proposed a short time ago, the development of related technologies is not yet mature, and there are relatively few related articles. At present, a relatively systematic explanation of small sample learning technology has been made. An Shengbiao et al. introduced the commonly used algorithms for small sample learning, as well as the technical development of small sample image classification under supervision, semi-supervision and unsupervised; Liu Ying et al. divided small sample image classification algorithms into two categories: convolutional neural network models and graph neural network models. Zhao Kailin et al. introduced the methods of small sample learning from three directions: model fine-tuning, data enhancement, and transfer learning, and summarized them; Liu Chunlei et al. summarized the small sample learning methods into a paradigm based on transfer learning and a paradigm based on meta-learning, and then summarized the small sample target detection according to different improvement strategies; Zhang Zhenwei et al. summarized and analyzed the small sample target detection methods from six aspects: metric learning, data enhancement, model structure, and meta-learning. In general, the common point of the above literature is that they focus on the current technical development status of small sample learning, but the current small sample image classification effect is still poor. Summary of the invention

[0006] The present invention aims to solve the problem of poor classification effect of small sample images.

[0007] A method for classifying small sample images comprises the following steps:

[0008] For each of the images to be classified and several known categories of images, feature extraction is performed respectively. During the feature extraction process, the image is first sent to Swin Transformer for feature extraction, and then sent to spatial attention and channel attention CSAM processing;

[0009] After the image to be classified is processed by CSAM, one image is represented as a set of local feature vectors [u1, u2, .., u HW ], H and W are the length and width of the feature map after CSAM processing, and each vector u i Considered as a node in a set;

[0010] After any image in the known category is processed by CSAM, an image is represented as a set of local feature vectors [v1, v2, .., v HW ],v j It is also regarded as a node in the local feature vector set of another image;

[0011] By computing the embedding node u from two image features i and v j The pairwise distance between is used to obtain the unit cost:

[0012]

[0013] The relevance score is generated as the weight value by the dot product between the node feature and the average node feature in another feature map:

[0014]

[0015] Then to a i and b j Normalize to get and

[0016] Then based on the weight and c ij , using EMD distance to obtain the optimal matching flow The similarity score between the image representations is then calculated:

[0017]

[0018] in, The optimal matching flow Elements in

[0019] The category of the image to be classified is determined by using the similarity score between the image to be classified and each category of images of several categories of known categories.

[0020] Furthermore, after the image is sent to Swin Transformer for feature extraction, it needs to be processed by a layer of adaptive average pooling, and then the feature map after adaptive average pooling is sent to spatial attention and channel attention CSAM processing.

[0021] Furthermore, the Swin Transformer includes multiple block units, wherein the first block unit includes a linear embedding layer and several Swin Transformer blocks, and the remaining block units include a PatchMerging layer and several Swin Transformer blocks.

[0022] Furthermore, the Swin Transformer includes 4 block units, wherein the first block unit includes a linear embedding layer and 2 Swin Transformer blocks, and the second to fourth block units include 2, 6, and 2 Swin Transformer blocks, respectively.

[0023] Furthermore, the spatial attention and channel attention CSAM processing process includes the following steps:

[0024] The input feature map is denoted as F, which enters the channel attention path and the spatial attention path respectively, and is represented by M c (F) and M p (F) represents the channel attention map and spatial attention map;

[0025] Channel attention path M c (F) are as follows:

[0026] M C (F)=σ(FC(GlobalAvgPool(F))+FC(GlobalMaxPool(F)))

[0027] In the formula, GlobalAvgPool represents global average pooling; GlobalMaxPool represents global maximum pooling; FC represents the fully connected layer; σ represents the sigmoid activation function;

[0028] Spatial attention path M p (F) are as follows:

[0029] M p (F) = Conv 1×1 (Conv 3×3 ((Conv 3×3 (Conv 1×1 (F)))))

[0030] In the formula, Conv represents the convolution operation, 1×1 and 3×3 represent the convolution kernel size; σ represents the sigmoid activation function;

[0031] Then M c (F) and M p (F) Add the elements together, and after the sigmoid activation function, add the original feature map F element by element to get the final output F CSAM =F+σ(M C (F)+M P (F)).

[0032] Furthermore, for a i and b j Normalized to get and for:

[0033] Furthermore, based on the weight and c ij Using EMD distance to get the optimal matching flow In the process, the EMD formula is as follows:

[0034]

[0035] subject to x ij ≥0,i=1,...,HW,j=1,...,HW

[0036]

[0037] Among them, x ij Represents the matching distance between two nodes in EMD, corresponding to the best matching x ij ,Right now

[0038] Furthermore, in the process of extracting features from the image to be classified and each of the several classes of images of known categories, k-shot learning is used to extract features, including the following steps:

[0039] The process of extracting features from one image of a certain class of images of several known classes is recorded as 1-shot; the process of extracting features from k images of a certain class of images of several known classes is recorded as k-shot after the same processing; k images are processed by CSAM to obtain k feature maps, and the k feature maps are averaged into one feature map;

[0040] Then the averaged feature map is represented as a set of local feature vectors [v1, v2, .., v HW], and then with the vector u i Perform subsequent EMD processing.

[0041] Beneficial effects:

[0042] The present invention proposes an innovative small sample image classification method. First, the Swin Transformer network is used to extract image features, and the EMD algorithm is used to generate the best matching flow between structural units with the minimum matching cost, so as to more accurately represent the classification distance between images and effectively improve the classification effect of small sample images. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the present invention.

[0044] Figure 2 Schematic diagram of the feature extraction process combining Swin Transformer and EMD.

[0045] Figure 3 Schematic diagram of the Swin Transformer feature extraction process.

[0046] Figure 4 This is a schematic diagram of the Block processing process.

[0047] Figure 5 Schematic diagram of the channel attention processing process.

[0048] Figure 6 Schematic diagram of the spatial attention processing process.

[0049] Figure 7 Schematic diagram of channel and spatial attention processing.

[0050] Figure 8 Use k-shot classification method for EMD.

[0051] Fig. 9 Schematic diagram of N-way K-shot data.

[0052] Fig.10 Schematic diagram of weight normalization.

[0053] Fig.11 Schematic diagram of the best matching flow computed for a support set and a query set.

[0054] Fig.12 is the similarity score graph. DETAILED DESCRIPTION

[0055] Specific implementation method 1: Combination Figure 1 and Figure 2 To explain this embodiment,

[0056] This embodiment is a method for classifying small sample images, and the method includes:

[0057] S1. Image preprocessing:

[0058] S11. Set the size of all images in the Mini-imagenet dataset to 224×224, and the image size of the other four datasets tiered-imagenet, Cifar-fs, CUB and FC100 is set to 84×84.

[0059] S12. Use data augmentation strategies to enhance data. The data augmentation strategy uses random horizontal flipping, color jittering, and introduces some randomness in brightness, contrast, and saturation. Each training sample will make slight changes in these attributes with a certain probability, thereby increasing the diversity of the data, which helps the model better adapt to different changes in brightness, contrast, and color.

[0060] S13. Perform a normalization operation on the tensor after image conversion, and process the image pixel values ​​by mean and standard deviation.

[0061] S2. Perform Swin Transformer feature extraction on the preprocessed image:

[0062] S21, such as Figure 3 As shown in the figure, in the Swin Transformer network, the preprocessed image is first divided into non-overlapping 4x4 image blocks through the PatchPartition operation. Then, through the linear embedding layer, these image blocks are converted into a sequence and sent to the block unit for processing.

[0063] S22, such as Figure 4 As shown, Swin Transformer includes multiple block units, where the first block unit includes a linear embedding layer and several Swin Transformer blocks, and the remaining block units include a PatchMerging layer and several Swin Transformer blocks;

[0064] In this implementation, the Swin Transformer includes 4 block units, wherein the first block unit includes a linear embedding layer and 2 Swin Transformer blocks, and the second to fourth block units include 2, 6, and 2 Swin Transformer blocks, respectively.

[0065] In each Swin Transformer Block, the self-attention mechanism is used to extract the features of the image. Patch Merging downsamples the feature map, thereby reducing the width and height of the feature map but increasing the number of channels. This process extracts the deep features of the image through multiple Block and Patch Merging operations.

[0066] The structure of the Swin Transformer Block is as follows Figure 2 As shown, it contains four key modules: Layer Normalization (LN), Windowed Multi-head Self-Attention (W-MSA), Multi-LayerPerceptron (MLP) and Shifted Window Multi-head Self-Attention (SW-MSA).

[0067] The calculation process of Swin Transformer Block is as follows:

[0068]

[0069] Among them, z k-1 It is the feature map just input by the current Swin Transformer Block. is the feature map obtained after regularization and window attention mechanism, z l is the feature map after regularization and full connection layer, is the feature map after regularization and sliding window attention mechanism, z l+1 is the feature map after regularization and full connection layer.

[0070] The LN module is used to normalize the input features to ensure that the features of different channels have similar distributions. This helps improve the stability of the model and the speed of training convergence.

[0071] The W-MSA module is used to perform multi-head self-attention calculations within a window. It performs weighted aggregation of features by calculating attention weights, promoting interaction and information transfer between features at different positions. This helps the model better understand the relationship between different parts.

[0072] The MLP module is a fully connected feedforward network that performs complex nonlinear transformations on features through multiple fully connected layers and activation functions to capture richer feature representations. It helps the model learn higher-level feature information.

[0073] The SW-MSA module is used to perform window-shifted multi-head self-attention calculations. It translates the input features and reorganizes and integrates them in local areas to capture a wider range of contextual information. At the same time, it also preserves the relative position relationship between windows, which helps to effectively extract multi-scale features.

[0074] Through the sequential action of each module in the above Swin Transformer Block, the modeling and integration of the features in the window are realized. After Swin Transformer, the feature map of 224*224 size is output as a feature map of 7*7*768, and the feature map of 84*84 size is output as a feature map of 3*3*768. Both can be converted into a feature map of 5*5*768 through adaptive average pooling, and finally output as a feature map of 5*5*768.

[0075] S3. For the feature map output by Swin Transformer, spatial attention and channel attention CSAM are used for processing:

[0076] The spatial attention and channel attention CSAM processing process is as follows:

[0077] The feature maps enter the channel attention path and the spatial attention path respectively. c (F)∈R c and M p (F)∈R H′×W′ They represent channel attention map and spatial attention map respectively.

[0078] In order to effectively calculate channel attention, the spatial dimension of the input feature map needs to be compressed. The common method for aggregating spatial information is average pooling, while maximum pooling collects unique object features and can infer attention on finer channels. Therefore, the features obtained after average pooling and maximum pooling are used simultaneously. Channel Attention Path M c The calculation of (F) can be expressed as:

[0079] M C (F)=σ(FC(GlobalAvgPool(F))+FC(GlobalMaxPool(F))) (5)

[0080] In the formula, GlobalAvgPool represents global average pooling; GlobalMaxPool represents global maximum pooling; FC represents the fully connected layer; σ represents the sigmoid activation function.

[0081] Formula (5) means that the feature map F passes through the average pooling layer and the maximum pooling layer at the same time, and then passes through the fully connected layer, and then the two are added element by element, and then nonlinearly mapped through the sigmoid activation function, and finally the channel dimension output is obtained. The channel attention is as follows Figure 5 shown.

[0082] Graph spatial attention Figure 6 As shown, the spatial attention path M p The calculation of (F) can be expressed as:

[0083] M p (F) = Conv 1×1 (Conv 3×3 ((Conv 3×3 (Conv 1×1 (F))))) (6)

[0084] In the formula, Conv represents the convolution operation, 1×1 and 3×3 represent the convolution kernel size; σ represents the sigmoid activation function;

[0085] The feature map F is subjected to a series of convolution operations to extract local features and obtain M c (F), and then with M p (F) is added element by element; then, after the sigmoid activation function, in order to make the information propagation smoother, the low-level features can be directly transmitted to the high-level network, and then added element by element with the original feature map F to obtain the final module output F CSAM , the calculation process can be expressed as:

[0086] F CSAM =F+σ(M C (F)+M P (F)) (7)

[0087] Through the analysis of the channel-space fusion attention module structure, it can be seen that feature extraction of the channel dimension alone can better learn color information into the model, while learning the spatial dimension alone can allow the network to focus more on learning texture features. Finally, the two are combined to obtain the original feature Figure 1 Then the processed image output is subjected to EMD measurement.

[0088] S4. Calculate the EMD in small sample image classification:

[0089] EMD distance: EMD is actually the optimal solution to the transportation problem in linear programming. It uses two different ways to pile up a certain number of piles, and EMD calculates the sum of the minimum distances of moving small blocks required to turn one pile into another. Assume that a group of suppliers A = {a i|i=1,2,..m},need to transport the goods to a specified set of destinations B={b j |j=1, 2, ...k}, where: a i represents the i-th supplier; b J represents the jth destination. The unit cost from the supplier to the destination is c IJ , the number of units transported is x IJ , the goal of the transportation problem is to find the cheapest flow of goods From suppliers to demanders:

[0090]

[0091] subject to x ij ≥0, i=1,...,m,j=1,...,k

[0092]

[0093] For a known image, use Swin Transformer to generate image embedding U∈R H×W×C , where H and W represent the spatial size of the feature map, and C represents the feature dimension; then through the spatial attention and channel attention CSAM modules, each image representation contains a set of local feature vectors [u1, u2, .., u HW ], each vector u i It can be regarded as a node in the set; for the image to be classified, the same process is adopted, and each image representation contains a set of local feature vectors [v1, v2, .., v HW ],v j It is also regarded as a node in the set of local feature vectors of another image. The similarity between two images can be expressed as the best matching cost between the two sets of vectors. By calculating the embedding node u from the two image features i , v j The pairwise distance between is used to obtain the unit cost:

[0094]

[0095] Nodes with similar representations incur less matching cost between each other.

[0096] Weight generation: In small sample classification tasks, since it is difficult to determine the importance of local feature representation in a single image, in order to reduce the weight of high variance background areas in the two images and increase the weight of common areas, a cross-reference mechanism is used to generate a correlation score as a weight value through the dot product between the node feature and the average node feature in another structure:

[0097]

[0098] b j The same logic applies.

[0099]

[0100] Finally, all weights in the structure are normalized, and the result is as follows Fig.10 As shown, ensure that both parties have the same total weight in order to match

[0101]

[0102] According to the EMD formula in formula (8),

[0103]

[0104] subject to x tj ≥0,i=1,...,HW,j=1,...,HW

[0105]

[0106] Due to the weight and It can be calculated that c ij According to the above formula, it can also be calculated that 4 variables and three equations can obtain the best matching x ij ,Right now Once the optimal matching flow is obtained The results are as follows Fig.11 As shown, we can calculate the similarity score between image representations:

[0107]

[0108] Similarity score is Fig.12 shown.

[0109] During the training process, the loss is calculated through the cross entropy loss function based on the similarity score and the label of the known category image, so as to train the model parameters of the Swin Transformer+spatial attention and channel attention CSAM overall network.

[0110] When actually classifying small samples, formula (12) is directly used to obtain the similarity score between the image to be classified and each of several classes of known class images, and the class of the image to be classified is determined based on the similarity score.

[0111] The EMD in small sample image classification can realize the end-to-end training principle of the network model as follows:

[0112] In order to embed the optimal matching problem into the neural network for end-to-end training, The solution of is helpful with respect to the network model parameters θ. Applying the implicit function definition on the optimality (KKT) condition yields Jacobi's theorem. For completeness, we start with Equation (8), which compactly distributes in matrix form:

[0113] minimize c(θ) T x

[0114] subject to G(θ)x≤h(θ) (13)

[0115] E(θ)x=F(θ)

[0116] Where θ is a model-related parameter. E(θ)x=F(θ) represents the equation and The equality constraints of the upper and lower parts are combined together, G(θ)x≤h(θ) represents x in formula (8) ij ≥ 0. Therefore, the Lagrangian of the linear programming problem in formula (13) is given by:

[0117] L(θ, x, ν, λ) = c(θ) T x+λ T (G(θ)xh)+v T (E(θ)xF(θ)) (14)

[0118] where ν is the dual variable for equality constraints and λ>0 is the dual variable for inequality constraints.

[0119] Following the KKT conditions with sign-hedge, we can solve by using the primal-dual interior point method To obtain the best (x, v, λ) of the object function, denoted as in

[0120]

[0121] Theorem 1: Assumption When all derivatives exist, x is in the optimal solution The partial Jacobian comparable quantity relative to θ is It can be obtained by meeting the following conditions:

[0122]

[0123] Here, the Jacobian formula for the solution mapping is obtained by applying the implicit function theorem to the KKT condition. For example:

[0124] The Jacobian with respect to θ can be defined as

[0125]

[0126] Find the optimal solution You can get A closed-form expression for the gradient of the input parameter θ. This facilitates efficient backpropagation throughout the optimization process, thereby ensuring end-to-end training of the network model without interfering with the initialization and optimization trajectories.

[0127] Structured fully connected layer: The above process is regarded as the case of 1-shot, and k-shot needs to be established on this basis. Figure 8 As shown, k-shot learns a structured fully connected layer that uses EMD as a distance function to classify structured feature representations, and the learnable embedding of each class becomes a set of vectors. The trained 1-shot model (network model) is fixed as the feature extractor, and the parameters in the structured connection layer are learned by averaging features from sampled data in the support set. For example, k-shot is 5-shot, that is, 5 pictures of a category are randomly selected from the Mini-imagenet dataset, and the output is 5 5*5*768 feature maps, and these 5 feature maps are averaged into a 5*5*768 feature map.

[0128] Example

[0129] In this experiment, we used the Swin-T pre-trained model for initialization in the pre-training phase, with model parameters of 28288354 and FLOPs of 4.5G. The parameters of the Swin Transformer model were adjusted through training. Since the model finally trained to obtain a feature map of 7×7×768 or 3×3×768, we adaptively pooled it to 5×5×768, and used CSAM to facilitate downstream EMD to fine-tune the model, with a learning rate of 0.00005, momentum of 0.9, and weight decay of 0.05.

[0130] The beneficial effects of the present invention are:

[0131] In small sample image classification, the feature extraction module of traditional convolutional neural networks is usually limited by the local properties of the convolution kernel, and it is difficult to effectively process global information and long-distance dependencies. In order to solve this problem, the present invention proposes an innovative small sample image classification method based on the fusion of Swin Transformer and EMD (Earth Mover's Distance) technology. We use the Swin Transformer network to extract image features, and use the EMD algorithm to generate the best matching flow between structural units with the minimum matching cost, so as to more accurately represent the classification distance between images. We have used a large number of experiments to verify the effectiveness of our algorithm. We have achieved SOTA in the small sample field on two commonly used small sample data sets, mini-imagenet and tiered-imagenet, and also have good classification effects on data sets such as Cifar-fs, CUB, and FC100.

[0132] In terms of average accuracy: the STEMD method proposed in this invention surpasses the highest P>M>F method by 3.5% in 1 shot and 1.2% in 5 shots on the small sample dataset mini-Imagenet, and surpasses the highest TRIDENT method by 4.8% in 1 shot on the tiered-imagenet dataset, reaching SOTA in the small sample field on both datasets. The accuracy of the proposed STEMD method on the small sample dataset FC100 surpasses the highest BAVARDAGE method by 7% in 1 shot and 9% in 5 shots, reaching SOTA on the FC100 dataset. The performance on the small sample dataset Cifar-fs is particularly good, achieving a very good effect of 86.95% in 1 shot, which is 3% lower than the current best PT+MAP+SF+SOT method, but 1.2% higher in 5 shots, surpassing the PT+MAP+SF+SOT method and reaching SOTA. On the CUB dataset, the performance on 1 shot reaches 83.1%, and the performance on 5 shots reaches 92.88%.

[0133] The present invention improves the accuracy of small sample image classification, especially on the small sample public dataset, mini-Imagenet has reached SOTA, with a 1-shot accuracy of 98.8±0.1% and a 5-shot accuracy of 99.6±0.2%. It greatly improves the accuracy of the public dataset mini-imagenent in the field of small sample image classification.

[0134] The above calculation examples of the present invention are only used to explain the calculation model and calculation process of the present invention in detail, and are not intended to limit the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for classifying small sample images, characterized in that: The following steps are involved: For each of the images to be classified and several known categories of images, feature extraction is performed respectively. During the feature extraction process, the image is first sent to Swin Transformer for feature extraction, and then sent to spatial attention and channel attention CSAM processing; After the image to be classified is processed by CSAM, one image is represented as a set of local feature vectors [u1, u2, .., u HW ], H and W are the length and width of the feature map after CSAM processing, and each vector u i Considered as a node in a set; After any image in the known category is processed by CSAM, an image is represented as a set of local feature vectors [v1, v2, .., v HW ],v j It is also regarded as a node in the local feature vector set of another image; By computing the embedding node u from two image features i and v j The pairwise distance between is used to obtain the unit cost: The relevance score is generated as the weight value by the dot product between the node feature and the average node feature in another feature map: Then to a i and b j Normalize to get and Then based on the weight and c ij , using EMD distance to obtain the optimal matching flow The similarity score between the image representations is then calculated: in, The optimal matching flow Elements in The category of the image to be classified is determined by using the similarity score between the image to be classified and each category of images of several categories of known categories.

2. The method for classifying small sample images according to claim 1, characterized in that: During the feature extraction process, after the image is sent to Swin Transformer for feature extraction, it needs to go through a layer of adaptive average pooling layer, and then the feature map after adaptive average pooling is sent to spatial attention and channel attention CSAM processing.

3. The method for classifying small sample images according to claim 1, characterized in that: The SwinTransformer includes multiple block units, wherein the first block unit includes a linear embedding layer and several Swin Transformer blocks, and the remaining block units include a Patch Merging layer and several SwinTransformer blocks.

4. The method for classifying small sample images according to claim 3, characterized in that: The SwinTransformer includes 4 block units, wherein the first block unit includes a linear embedding layer and 2 SwinTransformer blocks, and the second to fourth block units include 2, 6, and 2 Swin Transformer blocks, respectively.

5. The method for classifying small sample images according to claim 1, characterized in that: The spatial attention and channel attention CSAM processing process includes the following steps: The input feature map is denoted as F, which enters the channel attention path and the spatial attention path respectively, and is represented by M c (F) and M p (F) represents the channel attention map and spatial attention map; Channel attention path M c (F) are as follows: M C (F)=σ(FC(GlobalAvgPool(F))+FC(GlobalMaxPool(F))) In the formula, GlobalAvgPool represents global average pooling; GlobalMaxPool represents global maximum pooling; FC represents the fully connected layer; σ represents the sigmoid activation function; Spatial attention path M p (F) are as follows: M p (F)=Conv 1×1 (Conv 3×3 ((Conv 3×3 (Conv 1×1 (F))))) In the formula, Conv represents the convolution operation, 1×1 and 3×3 represent the convolution kernel size; σ represents the sigmoid activation function; Then M c (F) and M p (F) Add the elements together, and after the sigmoid activation function, add the original feature map F element by element to get the final output F CSAM =F+σ(M C (F)+M P (F)).

6. A method for classifying small sample images according to any one of claims 1 to 5, characterized in that: to a i and b j Normalized to get and for:

7. The method for classifying small sample images according to claim 6, characterized in that: Based on weight and c ij Using EMD distance to get the optimal matching flow In the process, the EMD formula is as follows: subject to x ij ≥0,i=1,...,HW,j=1,...,HW Among them, x ij Represents the matching distance between two nodes in EMD, corresponding to the best matching x ij ,Right now 8. The method for classifying small sample images according to claim 1, characterized in that: In the process of extracting features from the image to be classified and each of several known category images, k-shot learning is used to extract features, including the following steps: The process of extracting features from one image of a certain class of images of several known classes is recorded as 1-shot; the process of extracting features from k images of a certain class of images of several known classes is recorded as k-shot after the same processing; k images are processed by CSAM to obtain k feature maps, and the k feature maps are averaged into one feature map; Then the averaged feature map is represented as a set of local feature vectors [v1, v2, .., v HW ], and then with the vector u i Perform subsequent EMD processing.

Images