A small sample image classification method based on frequency domain analysis

Abstract

The present invention discloses a small sample image classification method based on frequency domain analysis, including S1, constructing a base class data set and a new class data set; S2, extracting several N-way-K-shot meta-tasks on the base class data set, dividing each meta-task into a labeled support set and a labeled query set, using the labeled support set and the labeled query set as network input, outputting a cross entropy classification loss, and back-propagating the loss to adjust network parameters; S3, extracting several N-way-K-shot meta-tasks on the new class data set, dividing each meta-task into a labeled support set and an unlabeled query set, using the unlabeled query set as network input, and outputting the classification result of the network for the unlabeled query set. The present invention can use spatial domain data processing and frequency domain data processing in both training and testing stages. Both the spatial position information of the image and the resolution information of the image are retained.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification methods, and in particular to a small sample image classification method based on frequency domain analysis. Background Art

[0002] The success of deep learning is inseparable from the support of huge data. However, in actual scenarios, it is often difficult to obtain high-quality labeled samples of sufficient scale due to many factors. Many fields do not have the conditions to obtain large-scale data sets, and some fields involve privacy, ethics and other issues that make it difficult to obtain high-quality data. In addition, the huge labor cost of manually labeling large-scale data will also become one of the difficulties of deep learning image classification methods. Unlike deep learning models that require big data and computing resources, humans can quickly learn with a small number of data samples when facing new tasks. In order to enable deep learning models to have human-like capabilities, small sample learning has become an important and widely studied issue. Small sample learning can quickly learn new classes with only a small number of samples after learning a large number of base classes on a large data set, so that the model can adapt to unknown tasks. Small sample learning usually faces serious overfitting problems. Proposing a model that can extract rich information from a small number of samples and has generalization ability is still a hot topic in the field of small sample image classification research.

[0003] In recent years, research on small sample image classification can be roughly divided into three categories: metric-based methods, data enhancement-based methods, and meta-learning-based methods. Although their methods are different, they all extract features from images in the spatial domain. Previous methods believed that most convolutional neural networks only accept low-resolution RGB images, while the image resolution of most data sets (such as ImageNet) is usually much larger than the input of convolutional neural network models (such as ResNet). The scaling of images in the spatial domain causes information loss, while feature extraction from the frequency domain solves this problem by flexibly controlling the input image size. The benefits of frequency domain networks are more than that. It examines images from another perspective and can be combined with spatial domain networks to extract features more effectively and complementary. Recently, there have been many works that use attention mechanisms to enhance the feature extraction capabilities of backbone networks. The attention mechanism enables the network to pay more attention to important information and reduce the impact of information irrelevant to the target task. Channel attention represented by SENet learns the importance weights of different channels. Usually, in order to save overhead, the core step of the channel attention method is to compress each channel into a scalar through global average pooling for calculation. However, representing a channel with the mean is too simple and difficult to capture the complexity of the input. Some methods such as CANet capture long-range spatial dependencies by embedding position information in channel attention. However, its compression in the channel direction still uses one-dimensional average pooling, which does not solve the problem of information loss. Therefore, how to provide a small sample image classification method based on frequency domain analysis is a problem that technicians in this field urgently need to solve. Summary of the invention

[0004] One purpose of the present invention is to propose a small sample image classification method based on frequency domain analysis, which can simultaneously use spatial domain data processing and frequency domain data processing in the training and testing stages, thereby retaining both the spatial position information of the image and the resolution information of the image.

[0005] A small sample image classification method based on frequency domain analysis according to an embodiment of the present invention includes the following method steps:

[0006] S1, build base class dataset and new class dataset;

[0007] S2. Extract several N-way-K-shot meta-tasks on the base class dataset, divide each meta-task into a labeled support set and a labeled query set, use the labeled support set and the labeled query set as network input, output the cross entropy classification loss, and back-propagate the loss to adjust the network parameters;

[0008] S3. Extract several N-way-K-shot meta-tasks on the new class dataset, divide each meta-task into a labeled support set and an unlabeled query set, use the unlabeled query set as the network input, and output the network's classification result for the unlabeled query set.

[0009] Optionally, the multiple N-way-K-shot meta-tasks extracted from the base class data set are used for network training, and the multiple N-way-K-shot meta-tasks extracted from the new class data set are used for network testing.

[0010] Optionally, S2 and S3 include two branches, namely, spatial domain and frequency domain, and S2 and S3 include three stages, namely, data processing, feature extraction and prototype classification.

[0011] Optionally, the N-way-K-shot meta-task includes:

[0012] Given a labeled dataset:

[0013] D=(x i ,y i );

[0014] Among them, x i is the feature vector of the sample, y i is the category label of the sample;

[0015] The set of dataset categories C is divided into base class datasets C b And the new class dataset C n :

[0016]

[0017] C b ∪C n =C.

[0018] Optionally, the N-way-K-shot meta-task consists of two parts:

[0019] A support set containing labeled examples:

[0020]

[0021] Another query set with unlabeled samples:

[0022]

[0023] Among them, N represents the number of categories in S and Q, K represents the number of samples of each category in S, and M represents the number of samples of each category in Q.

[0024] Optionally, the data processing includes spatial domain data processing and frequency domain data processing;

[0025] The spatial domain data processing includes randomly cropping, shaking and flipping the image;

[0026] The frequency domain data processing includes converting the image into a frequency domain representation by discrete cosine transform:

[0027] Random cropping, jittering, and flipping;

[0028] Convert image to YC b C r Color space;

[0029] Divide each channel into blocks of size f×f, perform discrete cosine transform to obtain the transformed image feature map, and combine the discrete cosine transform coefficients at the same frequency into one channel, where f is the size of the discrete cosine transform filter;

[0030] Select the low-frequency channel with large influence as the input of the subsequent feature extractor, and select the low-frequency channel from Y and the low-frequency channel from C according to the sampling ratio. r and C b The low-frequency channel of the selected frequency channel number C is obtained fre ;

[0031] For C r and C b The frequency channels of are upsampled to the same size as the frequency channels of Y, and the selected channel feature maps are concatenated. The final input size after frequency domain preprocessing is:

[0032]

[0033] Where S img Indicates the image side length.

[0034] Optionally, the feature extraction includes:

[0035] The support set image and the query set image are input into the data processor together. The data processor is divided into spatial domain and frequency domain, which are used to convert the image into spatial domain form x1 and frequency domain form x2 respectively;

[0036] Input the spatial domain form x1 and the frequency domain form x2 into the spatial domain extraction network and the frequency domain extraction network respectively to obtain the spatial domain feature map F1 and the frequency domain feature map F2;

[0037] The spatial domain feature map F1 and the frequency domain feature map F2 are weightedly fused to obtain the fused feature F:

[0038] F = αF1 + (1-α)F2;

[0039] Among them, F represents the fusion feature map, F1 represents the spatial domain feature map, F2 represents the frequency domain feature map, and α represents the adaptive weighting coefficient;

[0040] Input the fused feature map F into the prototype classifier for classification;

[0041] Calculate the cross entropy classification loss based on the classification results or directly output the classification results.

[0042] Optionally, the spatial domain feature extractor uses ResNet as the backbone network and adds frequency domain channel spatial attention, and the frequency domain feature extractor uses ResNet with a slightly modified input layer as the backbone network.

[0043] Optionally, the frequency domain channel spatial attention is a coordinate attention, and the coordinate attention is a channel attention embedded with position information, and the coordinate attention includes:

[0044] Given an input X, discrete cosine transform pooling is performed along the height and width directions respectively to generate a height-aware feature map and a width-aware feature map;

[0045] Concatenate the height-aware feature map and the width-aware feature map and feed them into the convolution and activation layers;

[0046] The concatenated feature map is split into two tensors along the spatial dimension, and each tensor passes through a convolutional layer and an activation layer to obtain the output g. h and g w , the output of coordinate attention;

[0047]

[0048] Among them, x represents input, y represents output, c, w, h represent channel, width, height respectively, i, j represent coordinates in w and h directions respectively;

[0049] Insert the frequency-domain channel spatial attention layer into the residual module of ResNet.

[0050] Optionally, the prototype classification adds query set information to calculate an improved prototype based on the prototype network, and uses the improved prototype to classify the query set samples:

[0051] The initial prototype P is calculated by the mean of the fused features of the support set S;

[0052] The initial prototype is used to calculate the classification probability of each sample in the query set Q. The probability of each sample x∈Q becoming class k is estimated based on the similarity between its feature f(x) and the prototype P:

[0053]

[0054] Where d() represents the cosine similarity of two vectors, and τ represents a learnable parameter;

[0055] The probability of class k is used as the sample weight, the fusion features of all samples are weighted averaged, and the center of each class is re-estimated as the improved prototype P';

[0056] The improved prototype P' is used for prototype classification. The cross entropy loss is used as the loss function in the training phase, and the cross entropy loss is output. The classification result is directly output in the testing phase.

[0057] The beneficial effects of the present invention are:

[0058] (1) Unlike previous methods that are limited to preprocessing images in space, the present invention can simultaneously use spatial domain data processing and frequency domain data processing in the training and testing stages, thereby retaining both the spatial position information and the resolution information of the image.

[0059] (2) The model of the present invention uses two feature extractors to extract spatial and frequency domain features respectively, and the extracted features are fused and sent to the prototype classifier for classification tasks. A frequency domain channel spatial attention applied to the spatial domain extraction network is further proposed to enhance the feature extraction capability of the spatial domain backbone network. The channel attention is decomposed into a one-dimensional feature encoding process in two spatial directions, and discrete cosine transform is used instead of average pooling for channel compression to retain channel information and position information to the greatest extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0061] Figure 1 A schematic diagram of the flow of a small sample image classification method based on frequency domain analysis proposed by the present invention;

[0062] Figure 2 A schematic diagram of the process of frequency domain data processing in a small sample image classification method based on frequency domain analysis proposed by the present invention;

[0063] Figure 3 A schematic diagram of the spatial attention position and details of the frequency domain channel in a small sample image classification method based on frequency domain analysis proposed by the present invention;

[0064] Figure 4 This is a schematic diagram of discrete cosine transform pooling in a small sample image classification method based on frequency domain analysis proposed in the present invention. DETAILED DESCRIPTION

[0065] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0066] refer to Figure 1 , a small sample image classification method based on frequency domain analysis, comprising the following method steps:

[0067] S1, build base class dataset and new class dataset;

[0068] S2. Extract several N-way-K-shot meta-tasks on the base class dataset, divide each meta-task into a labeled support set and a labeled query set, use the labeled support set and the labeled query set as network input, output the cross entropy classification loss, and back-propagate the loss to adjust the network parameters;

[0069] S3. Extract several N-way-K-shot meta-tasks on the new class dataset, divide each meta-task into a labeled support set and an unlabeled query set, use the unlabeled query set as the network input, and output the network's classification result for the unlabeled query set.

[0070] In this implementation, several N-way-K-shot meta-tasks extracted from the base class data set are used for network training, and several N-way-K-shot meta-tasks extracted from the new class data set are used for network testing.

[0071] In this implementation, S2 and S3 include two branches, namely, spatial domain and frequency domain. S2 and S3 include three stages, namely, data processing, feature extraction and prototype classification.

[0072] In this implementation, the N-way-K-shot meta-task includes:

[0073] Given a labeled dataset:

[0074] D=(x i ,y i );

[0075] Among them, x i is the feature vector of the sample, y i is the category label of the sample;

[0076] The set of dataset categories C is divided into base class datasets C b And the new class dataset C n :

[0077]

[0078] C b ∪C n =C.

[0079] In this implementation, an N-way-K-shot meta-task consists of two parts:

[0080] A support set containing labeled examples:

[0081]

[0082] Another query set with unlabeled samples:

[0083]

[0084] Among them, N represents the number of categories in S and Q, K represents the number of samples of each category in S, and M represents the number of samples of each category in Q. Such a meta-task is called an N-way-K-shot problem.

[0085] Embodiment 1:

[0086] A base class dataset and a new class dataset are constructed, which have no intersection. 1200 epochs are constructed from the base class dataset, each epoch contains 100 small sample meta-tasks (5-way1-shot or 5-way5-shot) for training. 600 meta-tasks (5-way1-shot or 5-way5-shot) are randomly selected from the new class dataset for testing.

[0087] Each meta-task in the base set is divided into a support set and a query set. These images are used as network input together, and the output is the cross entropy loss. The loss is back-propagated to adjust the network parameters.

[0088] The meta-tasks in the new class data are used for network testing. Each meta-task is divided into a labeled support set and an unlabeled query set and input into the network, and the network's classification results for the query set are output.

[0089] In this embodiment, data processing includes spatial domain data processing and frequency domain data processing;

[0090] Spatial domain data processing includes random cropping, dithering, and flipping of images;

[0091] Frequency domain data processing involves converting the image into a frequency domain representation through discrete cosine transform:

[0092] Random cropping, jittering, and flipping;

[0093] Convert image to YC b C r Color space, this process uses the common sampling format 4:2:0;

[0094] Divide each channel into blocks of size f×f, perform discrete cosine transform to obtain the transformed image feature map, and combine the discrete cosine transform coefficients at the same frequency into one channel, where f is the size of the discrete cosine transform filter;

[0095] The low-frequency channels with greater influence are selected as the input of the subsequent feature extractor. Here, the 4×4 low-frequency channels from Y and the 4×4 low-frequency channels from C are selected according to the sampling ratio. r and C b Each 2×2 low-frequency channel, namely C fre =24, C fre is the selected frequency channel;

[0096] For C r and C b The frequency channels of are upsampled to the same size as the frequency channels of Y, and the selected channel feature maps are concatenated. The final input size after frequency domain preprocessing is:

[0097]

[0098] Where S img Indicates the image side length, such as 224. The input image size can be controlled by adjusting the DCT filter size. For example, using a filter with f=8 can process an input image of size 448×448. Compared with spatial domain image processing, larger images can be processed.

[0099] In this embodiment, in the feature extraction stage, the pre-processed spatial domain representation and frequency domain representation are respectively sent to the spatial domain feature extractor and the frequency domain feature extractor, and the feature maps extracted by the two feature extractors are adaptively weighted fused. The feature extraction includes:

[0100] The support set image and the query set image are input into the data processor together. The data processor is divided into spatial domain and frequency domain, which are used to convert the image into spatial domain form x1 and frequency domain form x2 respectively;

[0101] Input the spatial domain form x1 and the frequency domain form x2 into the spatial domain extraction network and the frequency domain extraction network respectively to obtain the spatial domain feature map F1 and the frequency domain feature map F2;

[0102] The spatial domain feature map F1 and the frequency domain feature map F2 are weightedly fused to obtain the fused feature F:

[0103] F = αF1 + (1-α)F2;

[0104] Among them, F represents the fusion feature map, F1 represents the spatial domain feature map, F2 represents the frequency domain feature map, and α represents the adaptive weighting coefficient, which is initially set to 0.5;

[0105] Input the fused feature map F into the prototype classifier for classification;

[0106] Calculate the cross entropy classification loss based on the classification results or directly output the classification results.

[0107] In this implementation, the spatial domain feature extractor uses ResNet as the backbone network and adds frequency domain channel spatial attention, and the frequency domain feature extractor uses ResNet with a slightly modified input layer as the backbone network.

[0108] The frequency domain feature extractor is a ResNet with a modified input layer. Since the image sizes of the frequency domain input and the spatial domain input are different, smaller in height and width dimensions but larger in channel dimension, the input layer of ResNet is skipped, that is, a convolution layer with a stride of 2 and a maximum pooling layer, and then the input channel size of the next layer is adjusted to the number of frequency domain input channels.

[0109] refer to Figure 3 In this implementation, the frequency domain channel spatial attention is a coordinate attention, and the coordinate attention is a channel attention that embeds position information. The coordinate attention includes:

[0110] Given an input X, discrete cosine transform pooling is performed along the height and width directions respectively to generate a height-aware feature map and a width-aware feature map;

[0111] Concatenate the height-aware feature map and the width-aware feature map and feed them into the 1×1 convolution and activation layer;

[0112] The concatenated feature map is split into two tensors along the spatial dimension, and each tensor passes through a 1×1 convolution layer and an activation layer to obtain the output g. h and g w , the output of coordinate attention;

[0113]

[0114] Among them, x represents input, y represents output, c, w, h represent channel, width, height respectively, i, j represent coordinates in w and h directions respectively, Figure 3 In , r is a coefficient for controlling the reduction ratio of the block size, which is set to r = 16;

[0115] Insert the frequency-domain channel spatial attention layer into the residual module of ResNet.

[0116] refer to Figure 4 , where DCT pooling considers the average pooling operation as a compression problem. The average pooling operation is a special case of the one-dimensional discrete cosine transform, and its result is equal to the result of the discrete cosine transform taking only the lowest frequency elements.

[0117] Therefore, the present invention proposes that the frequency domain channel spatial attention uses discrete cosine transform instead of average pooling operation to compress the channel to obtain more frequency information, reduce information loss, and only select limited frequency components for coordinate attention. Discrete cosine transform pooling selects low-frequency components to calculate the result, and selects the first n channels with the lowest frequency, such as n=4. The discrete cosine transform pooling process is as follows Figure 4 As shown, the features are divided into n parts along the channel dimension, and are multiplied element by element with the calculated discrete cosine transform coefficients of n frequencies respectively, and the calculated results are concatenated to obtain the pooled features.

[0118] In this implementation, prototype classification adds query set information to calculate improved prototypes based on the prototype network, and uses the improved prototypes to classify query set samples:

[0119] The initial prototype P is calculated by the mean of the fused features of the support set S;

[0120] The initial prototype is used to calculate the classification probability of each sample in the query set Q. The probability of each sample x∈Q becoming class k is estimated based on the similarity between its feature f(x) and the prototype P:

[0121]

[0122] Where d() represents the cosine similarity of two vectors, τ represents a learnable parameter, and its initial value is set to 10;

[0123] The probability of class k is used as the sample weight, the fusion features of all samples are weighted averaged, and the center of each class is re-estimated as the improved prototype P';

[0124] The improved prototype P' is used for prototype classification. The cross entropy loss is used as the loss function in the training phase, and the cross entropy loss is output. The classification result is directly output in the testing phase.

[0125] Embodiment 1:

[0126] Referring to Table 1 below, in order to evaluate the performance of our proposed method in image classification problems in small sample scenarios, this paper reports the performance comparison of the proposed method with the current state-of-the-art methods. We perform small sample classification on 600 tasks randomly selected from the test set on two datasets, miniImagenet and CUB-200-2011, and use the average accuracy with a 95% confidence interval as the evaluation indicator. In each 5-way1-shot / 5-way5-shot task, 15 query samples are randomly sampled from each class for evaluation. For miniImagenet, our method improves the best accuracy on 5-way 1-shot and 5-way 5-shot by 3.1% and 2.4%, respectively. For CUB, the accuracy of the two tasks is improved by 10.5% and 4.7%, respectively. Our method outperforms our baseline ProtoNet by 19.3% and 9.3% on the two tasks on miniImagenet, and by 10.5% and 6.4% on CUB, respectively. The results show that our method is more effective. In addition, the performance improvement of our method on the 1-shot task is significantly higher than that on the 5-shot task, because the problem of inaccurate feature extraction in the 1-shot task is more obvious than that in the 5-shot task.

[0127] Table 1 Performance comparison between the method of the present invention and advanced methods

[0128]

[0129] It can be seen from the data in Example 1 of Table 1 above that the present invention simultaneously extracts and fuses spatial and frequency domain features, making full use of spatial information and frequency information, which complement each other to improve the classification accuracy of the model; and by adding frequency domain channel spatial attention to the spatial branch, discrete cosine transform is introduced in the attention, reducing information loss on the channel. The present invention has conducted comprehensive experiments on two real data sets, miniImagenet and CUB, and improved the classification accuracy of images in small sample scenarios.

[0130] It can be seen from the above implementation modes that the present invention first proposes a dual-domain meta-learning network that combines the spatial domain and the frequency domain, which is divided into two branches, the spatial domain and the frequency domain. Different from the previous methods that are limited to preprocessing the image in space, this method uses spatial domain data processing and frequency domain data processing in both the training and testing stages, and extracts features complementarily in the spatial domain and the frequency domain to retain as much important information as possible in the image and obtain a more accurate category center.

[0131] Secondly, the present invention proposes a frequency domain channel spatial attention module, which is applied to the spatial domain feature extraction backbone network to improve the network's ability to extract important features by reducing the information loss in the average pooling process.

[0132] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A small sample image classification method based on frequency domain analysis, characterized in that: The method comprises the following steps: S1, build base class dataset and new class dataset; S2. Extract several N-way-K-shot meta-tasks on the base class dataset, divide each meta-task into a labeled support set and a labeled query set, use the labeled support set and the labeled query set as network input, output the cross entropy classification loss, and back-propagate the loss to adjust the network parameters; S3, extract several N-way-K-shot meta-tasks on the new class dataset, divide each meta-task into a labeled support set and an unlabeled query set, use the unlabeled query set as the network input, and output the network's classification result for the unlabeled query set; The multiple N-way-K-shot meta-tasks extracted from the base class data set are used for network training, and the multiple N-way-K-shot meta-tasks extracted from the new class data set are used for network testing; S2 and S3 include two branches, namely, space domain and frequency domain. S2 and S3 include three stages, namely, data processing, feature extraction and prototype classification. The N-way-K-shot meta-task consists of two parts: A support set containing labeled examples: Another query set with unlabeled samples: Where N represents the number of categories in S and Q, K represents the number of samples of each category in S, and M represents the number of samples of each category in Q.

2. The small sample image classification method based on frequency domain analysis according to claim 1, characterized in that: The N-way-K-shot meta-task includes: Given a labeled dataset: D=(x i ,y i ); Among them, x i is the feature vector of the sample, y i is the category label of the sample; The set of dataset categories C is divided into base class datasets C b And the new class dataset C n : C b ∪C n =C。 3. The small sample image classification method based on frequency domain analysis according to claim 2 is characterized in that: The data processing includes spatial domain data processing and frequency domain data processing; The spatial domain data processing includes randomly cropping, shaking and flipping the image; The frequency domain data processing includes converting the image into a frequency domain representation by discrete cosine transform: Random cropping, jittering, and flipping; Convert image to YC b C r Color space; Divide each channel into blocks of size f×f, perform discrete cosine transform to obtain the transformed image feature map, and combine the discrete cosine transform coefficients at the same frequency into one channel, where f is the size of the discrete cosine transform filter; Select the low-frequency channel with large influence as the input of the subsequent feature extractor, and select the low-frequency channel from Y and the low-frequency channel from C according to the sampling ratio. r and C b The low-frequency channel of the selected frequency channel number C is obtained fre ; For C r and C b The frequency channels of are upsampled to the same size as the frequency channels of Y, and the selected channel feature maps are concatenated. The final input size after frequency domain preprocessing is: Where S img Indicates the image side length.

4. The small sample image classification method based on frequency domain analysis according to claim 3 is characterized in that: The feature extraction comprises: The support set image and the query set image are input into the data processor together. The data processor is divided into spatial domain and frequency domain, which are used to convert the image into spatial domain form x1 and frequency domain form x2 respectively; Input the spatial domain form x1 and the frequency domain form x2 into the spatial domain extraction network and the frequency domain extraction network respectively to obtain the spatial domain feature map F1 and the frequency domain feature map F2; The spatial domain feature map F1 and the frequency domain feature map F2 are weightedly fused to obtain the fused feature F: F = αF1 + (1-α)F2; Among them, F represents the fusion feature map, F1 represents the spatial domain feature map, F2 represents the frequency domain feature map, and α represents the adaptive weighting coefficient; Input the fused feature map F into the prototype classifier for classification; Calculate the cross entropy classification loss based on the classification results or directly output the classification results.

5. The small sample image classification method based on frequency domain analysis according to claim 4 is characterized in that: The spatial domain feature extractor uses ResNet as the backbone network and adds frequency domain channel spatial attention, and the frequency domain feature extractor uses ResNet with a slightly modified input layer as the backbone network.

6. The small sample image classification method based on frequency domain analysis according to claim 5, characterized in that: The frequency domain channel spatial attention is a kind of coordinate attention, and the coordinate attention is a channel attention with embedded position information, and the coordinate attention includes: Given an input X, discrete cosine transform pooling is performed along the height and width directions respectively to generate a height-aware feature map and a width-aware feature map; Concatenate the height-aware feature map and the width-aware feature map and feed them into the convolution and activation layers; The concatenated feature map is split into two tensors along the spatial dimension, and each tensor passes through a convolutional layer and an activation layer to obtain the output g. h and g w , the output of coordinate attention; Among them, x represents input, y represents output, c, w, h represent channel, width, height respectively, i, j represent coordinates in w and h directions respectively; Insert the frequency-domain channel spatial attention layer into the residual module of ResNet.

7. The small sample image classification method based on frequency domain analysis according to claim 6, characterized in that: The prototype classification adds query set information to calculate the improved prototype based on the prototype network, and uses the improved prototype to classify the query set samples: The initial prototype P is calculated by the mean of the fused features of the support set S; The initial prototype is used to calculate the classification probability of each sample in the query set Q. The probability of each sample x∈Q becoming class k is estimated based on the similarity between its feature f(x) and the prototype P: Where d() represents the cosine similarity of two vectors, and τ represents a learnable parameter; The probability of class k is used as the sample weight, the fusion features of all samples are weighted averaged, and the center of each class is re-estimated as the improved prototype P'; The improved prototype P' is used for prototype classification. The cross entropy loss is used as the loss function in the training phase, and the cross entropy loss is output. The classification result is directly output in the testing phase.

Images