A small sample image classification method and system based on relevant areas

Abstract

The application discloses a small sample image classification method and system based on a relevant region. The application first considers the region autocorrelation relationship of the sample, and enhances the feature; then considers the cross-sample region correlation relationship between the support sets, extracts the commonness of the features of the current category, and further realizes feature enhancement; finally, the cross-sample region correlation relationship between the support sets and the query set is considered, the target region similar to the features of the support set and the query set is found, and the final classification result is obtained through similarity calculation. According to the qualitative analysis experimental result, after the method proposed in the application is used, the common region in the same category can be better paid attention to, and this part of the region is enhanced, so that the feature is more representative; in the quantitative analysis, through the ablation experiment and the comparison experiment, it can be found that the cross-sample attention module can improve the classification result between the support sets and between the support sets and the query set.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a method and system for classifying images in small sample scenarios. Background Technology

[0002] In the field of computer vision, image understanding is a crucial problem that needs to be solved, and image classification is a fundamental task within image understanding, playing a vital role. Image classification tasks take an image as input and output a prediction of the image's category. This typically requires training the model on a large-scale dataset to acquire the ability to analyze and process images. On one hand, depending on the number of training labels, the training process for image classification tasks can be divided into supervised training, semi-supervised training, and unsupervised training. Because labeling data is extremely time-consuming and labor-intensive, we aim to achieve good classification accuracy with as little labeled data as possible, which is a challenging task. On the other hand, in real-world scenarios, large-scale datasets are difficult to obtain. For example, in nature, the images of rare species are limited; in medical settings, identifying more related symptoms using only a small number of samples is of great significance to medical advancements; and for emerging dangerous goods or weapons, rapidly establishing a comprehensive security check process using a small number of samples can better protect public safety.

[0003] The goal of the few-sample image classification task is to address the above problems by reducing the model's dependence on large-scale labeled data, thereby enabling it to achieve higher classification accuracy for new classes with only a small number of samples.

[0004] The main methods for few-shot learning currently include the following four:

[0005] 1) Optimization-based methods. This approach treats few-shot learning as a series of new tasks, employing the idea of ​​meta-learning. The goal is for the model to learn how to learn, ultimately converging quickly when faced with a new set of learning tasks. Training based on this method typically consists of two loops. For example, MAML consists of a base learner and a meta-learner. During training, the base learner is trained internally for each independent task, while the meta-learner is optimized externally based on the validation results of the base learners. Finally, the optimal initialization parameters for the base learner, which can quickly adapt to new tasks, are obtained. However, this method has limitations. On one hand, this meta-learning method is only effective on shallow networks, and shallow networks have limited representation learning capabilities. On the other hand, training this type of model requires a large number of similar tasks, which are difficult to obtain. Furthermore, ANIL points out that in the MAML model structure, the inner loop is not necessary; enabling the network to learn better feature representations is more important than faster adaptation.

[0006] 2) Generative data augmentation methods. At the data level, generating new data can address the problem of insufficient original data. At the feature level, generative methods can not only learn the boundaries between specific categories but also obtain the complete boundaries of category distributions by introducing the concept of data distribution, thus handling the problem of category combinations. However, generative methods are usually based on large amounts of data and do not perform well in scenarios with small sample sizes.

[0007] 3) Metric-based learning methods. First, a feature extractor is trained. Images are processed through this feature extractor to obtain feature vectors in the feature space. Then, using a suitable metric function (such as Euclidean distance or cosine distance), the distances between different images are obtained, and the images are classified based on these distance relationships. In the prototype network, Euclidean distance is used as the metric function; in the matching network, cosine distance is used; and in the Negative-cosine method, an improved cosine classifier is used to add boundaries to the classification results, improving the performance of the classification model in few-shot learning.

[0008] 4) Feature embedding-based methods. Essentially, these methods require a sufficiently good feature extractor, whose extracted features are more discriminative across different categories in the feature space. The RFS paper also points out the importance of obtaining a better feature space in few-shot learning. Some methods improve the representational power of the feature extractor by constructing self-supervised learning tasks. Other methods obtain better classification boundaries through feature processing.

[0009] The methods described above, from different perspectives, have improved the classification accuracy of few-shot image classification tasks to a certain extent. This invention, from the perspective of metric learning, fully explores the regional similarity relationships between image samples to enhance features. In few-shot image classification tasks, the final classification target is provided by a small number of samples in the support set, and these samples are used as labels in the final classification prediction. However, under this premise, the quality of the support set will greatly affect the classification performance of few-shot tasks. A small number of samples is difficult to represent the category well, and it may also lead to the problem of focusing on the wrong target region. For example, if a support set image shows "a person riding a bicycle," when this photo is used as the classification target for the "bicycle" category, the network may incorrectly focus on the "person" part, causing other images containing "person" to be classified into this category in the final prediction, resulting in classification failure.

[0010] To address this issue, some current few-sample image classification methods start from image attention mechanisms, seeking regions of interest based on structural or semantic features. Some current research also focuses on cross-sample region attention, improving the accuracy of few-sample image classification by leveraging region similarity. Summary of the Invention

[0011] This invention proposes a cross-sample attention mechanism to find relevant regions between different samples of the same category, obtaining a correlation matrix, and further deriving weighted features with correlation information. Similarly, a similar correlation relationship can be constructed between the query set and the support set to obtain a correlation matrix of the query set relative to the support set, assisting in the classification of small sample images.

[0012] In this invention, each training task is divided into a support set and a query set. The support set provides the target categories and a small number of training samples for the current classification task, while the query set is used to calculate the classification accuracy. Model training consists of three steps: first, considering the autocorrelation regions of the samples to enhance features; then, considering the cross-sample region correlation between the support sets to extract commonalities in the features of the current category, further enhancing features; finally, considering the cross-sample region correlation between the support set and the query set to find target regions with similar features between the query set and the support set, and obtaining the final classification result through similarity calculation.

[0013] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0014] A few-sample image classification method based on relevant regions includes the following steps:

[0015] The support set is sequentially input into the feature extractor and the region autocorrelation module to obtain autocorrelation-enhanced support set features;

[0016] The query set is sequentially input into the feature extractor and the region autocorrelation module to obtain autocorrelation-enhanced query set features;

[0017] The autocorrelation-enhanced support set features are input into the cross-sample correlation module for feature enhancement, resulting in cross-sample enhanced support set features.

[0018] The cross-sample enhanced support set features and autocorrelation enhanced query set features are input into the cross-sample correlation module. The similarity between the support set features and query set features output by the cross-sample correlation module is calculated, thereby achieving the classification of the query set.

[0019] Furthermore, the feature extractor, the region autocorrelation module, and the cross-sample correlation module are trained using the following steps:

[0020] (1) Input all images of the support set and query set into the feature extractor based on the convolutional neural network, remove the pooling operation of the last layer, and obtain image features that retain the positional relationship.

[0021] (2) In order to help the model train better and improve the representation ability of the feature extractor, the classification accuracy in the base class is calculated during the training process. That is, after the extracted features are pooled, they are directly classified through the fully connected layer and the classification loss function is calculated.

[0022] (3) Input the image features obtained in step (1) into the region autocorrelation module to enhance the positional features of the image, find the spatially similar regions, and facilitate the extraction of better spatial features to assist in the subsequent cross-sample correlation calculation.

[0023] (4) Input samples of the same category from the support set into the cross-sample correlation module, and consider the spatial similarity relationship between different samples of the same category. Regions with high similarity can be regarded as the target region of the category, i.e., the classification target. This is used as the attention coefficient and applied to all samples of the category to achieve feature enhancement. The average of the enhanced features is taken as the category center of the category.

[0024] (5) Input the support set features obtained through feature enhancement in step (4) and the query set features obtained in step (3) into the cross-sample correlation module, which shares parameters with step (4), to find relatively matching regions between the query set samples and the support set samples. Regions with high similarity can be regarded as the regions of interest for the query set sample. Use these as attention coefficients for feature enhancement. Based on this, calculate the similarity between the query set features and the support set features to obtain the correlation loss function.

[0025] (6) Combine the classification loss function based on all samples in step (2) and the correlation loss function based on the task in step (5), iteratively optimize, and train to obtain the parameters of the feature extractor, region autocorrelation module and cross-sample correlation module required for testing.

[0026] A few-sample image classification system based on relevant regions using the above method includes a feature extractor, a region autocorrelation module, a cross-sample correlation module, and a classification module.

[0027] The feature extractor is used to extract features from the input query set and support set;

[0028] The regional autocorrelation module is used to enhance the regional correlation of a single sample, and obtain the support set features and query set features enhanced by autocorrelation.

[0029] The cross-sample correlation module is used to calculate the correlation between support sets, as well as the correlation between the support set and the query set;

[0030] The classification module is used to calculate the similarity between the support set features and query set features output by the cross-sample correlation module, thereby achieving the classification of the query set.

[0031] The beneficial effects of this invention are:

[0032] This invention is based on few-sample learning of regions of interest, aiming to improve the accuracy of few-sample image classification tasks by enhancing the features of these regions. First, it considers the regional autocorrelation of samples to enhance features. Then, it considers the cross-sample regional correlation between support sets, extracting commonalities in features of the current category to further enhance features. Finally, it considers the cross-sample regional correlation between the support set and the query set, finding target regions with similar features between the query and support sets, and calculating the final classification result through similarity. Experiments on the miniImageNet few-sample image classification dataset show improvements over baseline models and can be used as a post-processing method in other methods. Qualitative analysis of the experimental results shows that the proposed method can better focus on common regions within the same category and enhance these regions, resulting in more representative features. Quantitative analysis, through ablation and comparative experiments, reveals that the cross-sample attention module improves classification results both between support sets and between support and query sets. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the small sample correlation study of the cross-sample attention mechanism of the present invention; where A1 represents the attention matrix of the first support set sample, A2 represents the attention matrix of the second support set sample, and A represents the attention matrix of the query set sample.

[0034] Figure 2 This is a comparative diagram of the visualization results of the relevant regions of the present invention. From top to bottom, the diagram shows the original input image, the feature activation map obtained by directly using the feature extractor, and the feature activation map obtained by using the method of the present invention.

[0035] Figure 3 This is a comparative experiment between the present invention and ReNet under a 5-way-k-shot setting, comparing the differences in performance under different shots. Detailed Implementation

[0036] To make the above features and advantages of the present invention more apparent and understandable, specific embodiments are described below in conjunction with the accompanying drawings.

[0037] This embodiment establishes as follows: Figure 1 The image classification network structure shown here is mainly divided into three parts: region correlation enhancement for individual samples, correlation between support sets, and correlation between support sets and query sets. The cross-sample correlation module can be used as a post-processing operation in the feature extractor after pre-training or from scratch training to improve the final classification accuracy.

[0038] 1. Feature Extraction

[0039] The query set and support set are input into a convolutional neural network-based feature extractor. Figure 1 After the encoder in the code, the feature matrix F is obtained respectively. q and F s In this step, after inputting all images from the support set and query set into the feature extractor, the pooling operation of the last layer is removed to obtain image features that preserve positional relationships.

[0040] 2. Calculate the classification loss function

[0041] The obtained features are pooled to obtain a C-dimensional vector, which is then input into a fully connected network h to predict the category of the current image and calculate the classification loss function.

[0042] L cls =E x，y [L(h(F(x)), y)],

[0043] Where x is the input image, y is the class label of x, and F(x) represents the feature matrix F q and F s The pooled features do not distinguish between the support set and the query set here. L is the cross-entropy loss function, and E... x,y This represents the average loss over all possible x and y values.

[0044] 3. Enhanced regional correlation of individual samples

[0045] The characteristic matrix F q and F s The data are input into the autocorrelation region calculation model of the region autocorrelation module to calculate the correlation between each pixel and its surrounding neighbors. This correlation is then added to the original feature matrix to enhance the features.

[0046] F q =F q +SR(F q )

[0047] F s =F s +SR(F s )

[0048] SR represents the autocorrelation calculation function, which includes functions for calculating the correlation with surrounding neighbors and subsequent channel-wise convolution operations, preserving feature information.

[0049] 4. Correlation between support sets

[0050] The features obtained from the region autocorrelation module are used as input to the cross-sample correlation module. In this module, the support set features F are obtained by calculating similarity. s Cor: The pairwise correlation matrix of positions between each pair of elements.

[0051]

[0052] Where sim(·) represents the similarity calculation function, and Cor(s) i ,s j The feature has a size of H×W×H×W, where H represents the height of the feature and W represents the width of the feature. i ,s j ) represents s i ,s j The correlation matrix between the two inputs in spatial location. Further, the attention matrix A of the i-th input relative to the j-th input can be obtained. ij :

[0053]

[0054] Where m,n represent the spatial coordinates of the region block in input i, and τ is the temperature coefficient. This represents the spatial coordinates of the region block in input j. This represents the spatial coordinates of the region block in input i. The attention matrix A of the other images in the support set is averaged relative to the input to obtain the attention matrix A of the i-th support set image. i :

[0055]

[0056] Attention matrix A i As a weight, it is used in conjunction with the original features F of the support set. s Multiplying them yields a feature matrix weighted by the intra-class correlation of the support set.

[0057]

[0058] Further weighted averaging based on spatial location yields the C-dimensional feature vector f of the corresponding category for the support set samples. s That is, the support set feature vectors augmented across samples:

[0059]

[0060] 5. Support the relevance between the support set and the query set.

[0061] Bundle and the feature matrix F of the query set qInput the data into the cross-sample correlation module and repeat the above steps to obtain the correlation matrix between the query set and the support set:

[0062]

[0063] Furthermore, we can obtain the attention matrix A:

[0064]

[0065] Where m and n represent spatial coordinates, and τ is the temperature coefficient. This is used as the weights and the original features F of the query set. q Multiplying them yields a feature matrix weighted by the relevance between the query set and the support set.

[0066]

[0067] After averaging and pooling operations, the C-dimensional feature vector f of this class is obtained. q That is, the feature vector of the query set.

[0068]

[0069] Calculate the feature vector f of the query set q and support set feature vector f s Calculate the cosine similarity between them and the correlation loss function.

[0070]

[0071] Where cos(·) represents the cosine similarity calculation function, T represents the temperature coefficient, and s' and q' represent the features of the support set and query set of any two different categories within this input batch after calculating the relevance.

[0072] 6. Calculate the overall loss function

[0073] The overall loss function comprises the two parts mentioned above. The classification loss function in step 2 provides constraints for the training of the feature extractor, and the correlation loss function in step 5 provides constraints for the enhancement of the region of interest. The sum of these two parts enables end-to-end training. The overall loss function can be defined as follows:

[0074] L total =L cls +γL relation

[0075] Where γ is the coefficient of the correlation loss function.

[0076] 7. Obtain the classification results

[0077] During the testing phase, the cosine similarity between the query set and all support sets is compared, and the category corresponding to the support set with the highest similarity is the predicted category of the query set sample.

[0078] The image classification method for small sample scenarios proposed in this invention has the following test environment and experimental results:

[0079] (1) Test environment:

[0080] System environment: CentOS 7;

[0081] Hardware environment: Memory: 64GB, GPU: TITAN XP, Hard drive: 2TB;

[0082] (2) Experimental data:

[0083] This invention was conducted on the miniImageNet dataset, a small-sample image classification dataset. Considering that this part primarily addresses the region of interest in the support set for more than one sample, it is only meaningful to consider cross-sample correlations between support sets when the number of samples in the support set is greater than one. Following current mainstream experimental settings, this invention selected a 5-way, 5-shot configuration with five samples per class, and performed qualitative and quantitative analyses under this configuration. Furthermore, in the comparative experiments, to further investigate the model performance under different sample sizes, the number of samples for each class was increased from 1 to 10 sequentially.

[0084] The miniImageNet dataset is a subset of ImageNet, much smaller in size and scale than ImageNet. Training on this dataset requires fewer resources and is often used for few-shot classification tasks. It contains 100 classes: 64 for training, 16 for validation, and 20 for testing. Each class has 600 images, each 84x84 pixels. During training, images are padded with a 4-pixel border, randomly cropped to 84x84 pixels, and augmented with color perturbations and horizontal flipping, followed by regularization. During testing, only regularization is applied.

[0085] During training, the correlation calculation temperature coefficient was 5, the classification temperature coefficient T was 0.2, the batch size was 128, and the coefficient of the correlation loss function was 0.25. Training optimization used the SGD optimizer with an initial learning rate of 0.1, which decayed with a weight of 0.1 at the 50th and 70th training epochs, for a total of 80 training epochs. The momentum was 0.9. During testing, 2000 tasks were randomly sampled, each with 5 classes. The support set contained 10 samples per class, and the query set contained 15 samples per class. The model outputs the average classification accuracy as a quantitative result and the cross-sample regional correlation magnitude as a qualitative result.

[0086] (3) Visualize the experimental results:

[0087] To demonstrate the impact of cross-sample attention mechanisms on regions of interest, the research results on cross-sample related regions are presented through feature activation maps and attention coefficient visualization. For example... Figure 2 As shown, in each set of samples, the first row is the original input image, the second row is the visualization result of the feature activation map extracted by the feature extractor, and the third row is the visualization result of the attention map obtained after applying the cross-sample attention module. It can be observed that the features extracted by the feature extractor cannot effectively focus on the target region. After using the cross-sample correlation operation, the network can find the commonalities among these input images, thereby extracting the target region more accurately and achieving feature enhancement.

[0088] by Figure 2 Taking the upper part as an example, if the classification target is "curtain," directly selecting the feature activation map (second row) will easily treat the "curtain" as background in the image, while foreground objects such as "people," "musical instruments," and "seats" will receive more attention, leading to classification failure. After passing through a cross-sample attention module, the resulting attention map focuses on the commonality of these images, namely the "curtain" itself. The resulting attention map shows higher attention to the "curtain," thus obtaining a better feature representation and further improving the classification accuracy. Figure 2 In the lower half of the algorithm, the classification target is "lion." When directly selecting the feature activation map, the region of the "lion" that the network focuses on is not complete enough, or it mistakenly focuses on regions such as "tree branches" in the image. After passing through the cross-sample attention module, the network focuses on a more complete "lion," achieving feature enhancement and improving the classification accuracy.

[0089] (4) Quantitative experimental results:

[0090] To quantify the experimental results, this invention conducted a series of ablation experiments and comparative experiments.

[0091] The ablation experiment results are shown in Table 1. In the table, S2-CA represents cross-sample attention between support sets, and SQ-CA represents cross-sample attention between query set and support set. The study investigated whether these two modules were used. The experiments were conducted on the miniImageNet dataset, a few-shot learning dataset. Considering that correlation between support sets can only be performed when the number of samples is greater than 1, an experimental setting of five classes and five samples per class was chosen. As shown in Table 1, using cross-sample attention between support sets alone can bring a 0.51% improvement; using cross-sample attention between query set and support set alone can bring a 0.48% improvement; using both cross-sample attention mechanisms simultaneously is better than using them alone and further improves classification accuracy by 0.76%. The ablation experiment results show that both cross-sample attention between support sets and cross-sample attention between query set and support set bring feature enhancement effects, and using both simultaneously yields better results.

[0092] Table 1 Ablation Experiment Results

[0093] Serial Number S2-CA SQ-CA ACC (%) 1 × × 82.10±0.30 2 √ × 82.61±0.29 3 × √ 82.58±0.30 4 √ √ 82.86±0.30

[0094] To demonstrate the improvement in small-sample image classification performance through cross-sample attention between support sets, experiments were conducted with varying numbers of support set samples. Since ReNet does not consider cross-sample correlations between support sets, this experiment can serve as a comparative study of correlations between support sets. Experimental results are as follows: Figure 3 As shown, the horizontal axis represents different numbers of support sets (shots), and the results are represented by a line graph, where the solid line represents the average classification accuracy, the dashed line represents the average loss, red represents the performance of the proposed model, and blue represents the performance of the comparative experiment ReNet. From Figure 3 It can be observed that when the number of shots is greater than 1, the proposed method consistently outperforms the ReNet model that does not use cross-sample attention across the support set. Furthermore, the larger the number of support set samples (shots), the higher the average classification accuracy and the lower the loss. This is because as the number of samples increases, the description of a class becomes more comprehensive, which is more beneficial for the final classification. The cross-sample correlation calculation between support sets also stems from a better description of a class, enhancing common target regions by finding correlated regions within the same class.

[0095] Based on the same inventive concept, another embodiment of the present invention provides a small sample image classification system based on relevant regions using the above method, which includes a feature extractor, a region autocorrelation module, a cross-sample correlation module, and a classification module.

[0096] The feature extractor is used to extract features from the input query set and support set;

[0097] The regional autocorrelation module is used to enhance the regional correlation of a single sample, and obtain the support set features and query set features enhanced by autocorrelation.

[0098] The cross-sample correlation module is used to calculate the correlation between support sets, as well as the correlation between the support set and the query set;

[0099] The classification module is used to calculate the similarity between the support set features and query set features output by the cross-sample correlation module, thereby achieving the classification of the query set.

[0100] Based on the same inventive concept, another embodiment of the present invention provides an electronic device (computer, server, smartphone, etc.) including a memory and a processor, wherein the memory stores a computer program configured to be executed by the processor, and the computer program includes instructions for performing the steps of the method of the present invention.

[0101] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk), which stores a computer program that, when executed by a computer, implements the various steps of the method of the present invention.

[0102] Although the present invention has been disclosed above with reference to embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions made by those skilled in the art to the technical solutions of the present invention should be covered within the protection scope of the present invention, which is defined by the claims.

Claims

1. A small sample image classification method based on relevant regions, characterized in that, First, consider the regional autocorrelation relationship of the sample, enhance the features; Then consider the cross-sample regional correlation relationship between the support sets, realize feature enhancement; Finally, consider the cross-sample regional correlation relationship between the support set and the query set, find the target region similar to the query set and the support set features, and obtain the final classification result through similarity calculation; The method comprises the following steps: input two support sets into the feature extractor and the regional autocorrelation module in turn, and obtain the autocorrelation enhanced support set features of each support set; input the query set into the feature extractor and the regional autocorrelation module in turn, and obtain the autocorrelation enhanced query set features; input the autocorrelation enhanced support set features into the cross-sample correlation module for feature enhancement, and obtain the cross-sample enhanced support set features between the support sets; input the cross-sample enhanced support set features and the autocorrelation enhanced query set features into the cross-sample correlation module, calculate the similarity of the support set features and the query set features output by the cross-sample correlation module, and thus realize the classification of the query set; The feature extractor, the regional autocorrelation module and the cross-sample correlation module are trained by the following steps: (1) input all images of two support sets and a query set into the feature extractor based on the convolutional neural network, remove the pooling operation of the last layer, and obtain the image features retaining the position relationship; (2) after the features extracted by the feature extractor are subjected to the pooling operation, the classification is performed through the full connection layer, and the classification loss function is calculated; (3) input the image features obtained in step (1) into the regional autocorrelation module, enhance the position features of the image, and obtain the autocorrelation enhanced support set features and the autocorrelation enhanced query set features; (4) input the samples of the same category in the support set into the cross-sample correlation module, consider the spatial similarity relationship between different samples of the same category, regard the region with higher similarity as the target region of the category, that is, the classification target, and take the region as the attention coefficient acting on all samples of the category, so as to realize feature enhancement and obtain the cross-sample enhanced support set features; (5) input the cross-sample enhanced support set features obtained in step (4) and the autocorrelation enhanced query set features obtained in step (3) into the cross-sample correlation module, find the matching regions in the query set sample and the support set sample, calculate the similarity of the query set features and the support set features, and obtain the correlation loss function; (6) combine and calculate the classification loss function in step (2) and the correlation loss function in step (5), and iteratively optimize to obtain the parameters of the feature extractor, the regional autocorrelation module and the cross-sample correlation module.

2. The method of claim 1, wherein, The regional autocorrelation module calculates the correlation relationship of each pixel point relative to the surrounding neighbors, which is added to the feature matrix as a supplement to realize feature enhancement.

3. The method of claim 1, wherein, The cross-sample correlation module obtains the cross-sample enhanced support set features by the following steps: The features obtained by the region self-correlation module are taken as inputs of the cross-sample correlation module, and a support set feature F of the inputs is obtained by calculating the similarity in the cross-sample correlation module s a correlation matrix Cor between each pair of the features where sim(·) denotes a similarity computation function, Cor(s i ,s j ) is a correlation matrix of size HxWxHxW, and s i ,s j denotes the spatial position of the i-th input relative to the j-th input. ij ​ where m, n represent spatial position coordinates, and τ is a temperature coefficient, denotes the spatial position coordinates of the region block in input j, denotes the spatial position coordinates of the region block in input i; the attention matrix A of the i-th support set image is obtained by averaging the attention matrices of the support set other images relative to the input i : Attention matrix A i As weights, the original features F s of the support set are multiplied, resulting in a feature matrix weighted by the in-class relevance of the support set Then, the spatial position weighted average is obtained, and the cross-sample enhanced support set features are obtained:

4. The method of claim 3, wherein, The cross-sample correlation module calculates the correlation between the support set and the query set by the following steps: Will and the feature matrix F of the query set q to the cross-sample correlation module, to obtain the correlation matrix between the query set and the support set: Then the attention matrix A is obtained: This is the original feature F of the weight and query set q Multiplication, get the correlation between the query set and the support set Weighted feature matrix After the average and pooling operation, the query set feature vector is obtained: The cosine similarity between the set of computed query feature vectors f q and the set of support feature vectors f s is computed to yield a relevance loss function.

5. The method of claim 1, wherein, The total loss function is composed of the classification loss function and the correlation loss function, the training of the feature extractor is constrained by the classification loss function, the enhancement of the attention region is constrained by the correlation loss function, the two loss functions are added to realize end-to-end training.

6. A small sample image classification system based on relevant regions using the method of any one of claims 1 to 5, characterized in that, The method comprises a feature extractor, a region self-correlation module, a cross-sample correlation module and a classification module. The feature extractor is used to extract features from the input query set and support set. The region self-correlation module is used to perform region correlation enhancement of a single sample to obtain self-correlation enhanced support set features and self-correlation enhanced query set features. The cross-sample correlation module is used to calculate the correlation between the support sets and the correlation between the support set and the query set. The classification module is used to calculate the similarity of the support set features and the query set features output by the cross-sample correlation module, thereby realizing the classification of the query set.

7. An electronic device, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by a computer to realize the method of any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by a computer to realize the method of any one of claims 1-5.

Images