Remote sensing image small sample scene classification method based on multi-scale double-flow architecture

Abstract

The application discloses a kind of based on multiscale double-flow architecture remote sensing image small sample scene classification method, mainly solve the information loss that exists in prior art image discrimination, prone to remote sensing image complex background and object scale dramatic change influence problem.The scheme is: obtaining data set carries out data preprocessing;The data set after preprocessing is randomly sampled, generates support set and query set for training, verification and test;Global flow network, local flow network and key area positioning module are constructed into overall double-flow network;Loss function of global flow and local flow is defined, and overall double-flow network is trained and verified, and optimal network model is obtained;Optimal network model is used to classify test sample, and scene classification result is obtained.The application reduces the loss of remote sensing image discrimination information, avoids the influence of complex background and object scale dramatic change on scene classification, improves classification precision, and can be used for natural disaster detection, city planning, environmental monitoring and vegetation investigation.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image recognition technology, specifically relating to a method for classifying scenes in small samples of remote sensing images, which can be used for natural disaster detection, urban planning, environmental monitoring, vegetation mapping, and land cover analysis. Background Technology

[0002] Remote sensing is a detection technology that acquires target information over long distances. With the rapid development of remote sensing technology, remote sensing images are playing an increasingly important role in both military and civilian fields. Scene classification, which divides each remote sensing image into different scene categories based on its content, is an important means of understanding remote sensing images. It has broad application prospects in fields such as natural disaster detection, urban planning, environmental monitoring, vegetation mapping, and land cover analysis.

[0003] Before deep learning gained attention, scene classification models primarily relied on extracting low-level and mid-level features from images for classification, and then encoding these features. In recent years, thanks to the availability of large datasets, advancements in machine learning theory, and increased computing resources, deep learning models have demonstrated powerful learning capabilities. Convolutional neural networks (CNNs) are among the most mainstream deep learning models in image processing, and are also the most widely used and highest-performing network model in scene classification.

[0004] However, deep learning-based remote sensing image scene classification faces two fundamental problems. First, the model relies on a large number of labeled training samples, while obtaining manually labeled high-resolution remote sensing images is extremely difficult and time-consuming. If the available labeled data is insufficient, the deep learning model risks overfitting, leading to performance degradation. Second, while deep neural networks can achieve high accuracy in predicting test samples of scene categories that have already been trained on, they struggle to classify category samples that have never been seen during training.

[0005] Against this backdrop, research on scene classification in few-shot remote sensing images has received widespread attention. Few-shot learning, inspired by the rapid learning ability of humans, is a new research direction that enables machine vision systems to quickly learn new tasks from limited annotated data. Many existing few-shot learning models focus on designing different architectures. For example, metric-based few-shot methods seek an optimal metric space by designing different structures and metric methods, while meta-learning methods guide the learning algorithm by designing meta-learners, hoping that the model can quickly generalize to a new task.

[0006] In his paper "DLA-MatchNet for few-shot remote sensing image scene classification," Li proposed a deep few-shot learning method for remote sensing scene classification. This method improves feature representation capabilities by designing an adaptive discriminative learning matching network and employing an attention mechanism and feature fusion scheme, combining channel attention and spatial attention modules with the feature network. However, because this method extracts the image into a compact image-level representation, it loses most of the discriminative information. This loss is particularly difficult to recover when the number of training samples is small, affecting the final classification results.

[0007] In patent application number 202111495585.5, Gao Feng proposed a "Small Sample Scene Classification Method for Remote Sensing Images Based on Dual Prototype Networks". This method designs two operations: "prototype self-calibration" and "prototype mutual calibration". During the training process, the prototypes are made more representative and more conducive to subsequent classification prediction based on the prototypes. However, since this method does not deeply extract the hierarchical features of remote sensing images, it is easily affected by complex backgrounds and scale transformations, resulting in a decrease in classification accuracy. Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of the prior art by providing a remote sensing image small sample scene classification method based on a multi-scale dual-stream architecture, so as to reduce the loss of image discrimination information, avoid the impact of complex backgrounds and drastic changes in object scale on scene classification, and improve classification accuracy.

[0009] To achieve the above objectives, the technical solution of the present invention includes the following:

[0010] (1) Obtain three different remote sensing image datasets from public websites, and perform preprocessing on the images in the datasets in sequence, including cropping, random horizontal flipping, random brightness enhancement, random color enhancement, and random contrast enhancement;

[0011] (2) Randomly sample the preprocessed dataset to obtain the training support set S1 and the training query set T1, the validation support set S2 and the validation query set T2, and the test support set S3 and the test query set T3.

[0012] (3) Constructing an overall dual-stream network:

[0013] 3a) Construct a global flow network consisting of an attention deep embedding module A, a category-related attention module B, and a metric module C;

[0014] 3b) Select an existing prototype network as the local flow network;

[0015] 3c) Establish a key region localization module consisting of vector construction operations and greedy boundary search;

[0016] 3d) Connect the global flow network and the local flow network through the key region localization module to obtain the overall dual-flow network;

[0017] (4) Using the training support set S1 and the training query set T1, the entire dual-stream network is trained by the few-sample scenario training method to obtain the trained dual-stream network.

[0018] (5) Input the input validation support set S2 and the validation query set T2 into the trained dual-stream network to fine-tune the network parameters, and save the network with the highest index as the optimal dual-stream network model.

[0019] (6) Input the test support set S3 and the test query set T3 into the optimal two-stream network model to obtain the final classification result.

[0020] Compared with the prior art, the present invention has at least one or more of the following technical effects:

[0021] 1. This invention constructs a two-stream network to calculate the probability of a sample belonging to a category from the entire image and the most important region, thereby reducing the loss of image discrimination information and the impact of changes in object scale.

[0022] 2. This invention obtains scene category-related attention feature maps by designing a category-related attention module, which can increase the weight of descriptors during the measurement process and reduce the interference of background information on scene classification.

[0023] 3. This invention obtains key regions of an image by designing key region localization, which can quickly locate the region with the most information in the global image, and at the same time connect the global flow and the local flow to highlight important objects that are beneficial to scene classification.

[0024] Experimental results show that the present invention has better scene classification accuracy compared with other existing methods. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the overall implementation of the present invention;

[0026] Figure 2 This is a sub-flowchart for calculating and outputting the attention feature map M by the category-related attention module in this invention;

[0027] Figure 3 This is a sub-flowchart for establishing the key area positioning module in this invention. Detailed Implementation

[0028] The specific embodiments and effects of the present invention will be further described in detail below with reference to the accompanying drawings.

[0029] Reference Figure 1 The implementation steps of this invention are as follows:

[0030] Step 1: Obtain the remote sensing image dataset and perform data preprocessing:

[0031] Three different remote sensing image datasets were obtained from public websites. The images in the datasets were cropped to 224×224. The cropped images were then preprocessed by random horizontal flipping, random brightness enhancement, random color enhancement, and random contrast enhancement to obtain the preprocessed remote sensing images.

[0032] Step 2: Randomly sample the preprocessed dataset to obtain the support set and query set.

[0033] Random sampling is performed on the preprocessed dataset to obtain the training support set S1 and training query set T1, the validation support set S2 and validation query set T2, and the test support set S3 and test query set T3. The specific implementation is as follows:

[0034] 2.1) The three different remote sensing image datasets after preprocessing are divided into training, validation and test sets respectively. Specifically, the NWPU-RESISC45 dataset is divided into training, validation and test sets in a ratio of 25:10:10, the WHU-RS19 dataset is divided into training, validation and test sets in a ratio of 9:5:5, and the UC-Merced dataset is divided into training, validation and test sets in a ratio of 10:6:5.

[0035] 2.2) Randomly select C categories from the training set of each dataset, and randomly sample K images from each category. These C×K images constitute the training support set S1. At the same time, randomly select N images from each of the remaining images in these C categories to form the training query set T1.

[0036] 2.3) Randomly select C categories from the validation set of each dataset, and randomly sample K images from each category. These C×K images constitute the validation support set S2. Then, randomly select N images from each of the remaining images in these C categories to form the validation query set T2.

[0037] 2.4) Randomly select C categories from the test set of each dataset, and randomly sample K images from each category. These C×K images constitute the test support set S3. Randomly select N images from each of the remaining images in these C categories to form the test query set T3.

[0038] Step 3: Construct the overall dual-stream network.

[0039] 3.1) Establish a global flow network:

[0040] 3.1.1) Construct an attention-based deep embedding module A consisting of a convolutional layer, four convolutional blocks, an average pooling layer, and a cascaded 1×1 convolutional layer with 128 channels. The convolutional layer consists of a 7×7 convolutional filter with 64 channels and a 3×3 max pooling operation. Each convolutional block consists of four 3×3 convolutional filters. A skip connection operation is added after every two convolutional filters. The first convolutional block has 64 channels, the second convolutional block has 128 channels, the third convolutional block has 256 channels, and the fourth convolutional block has 512 channels.

[0041] 3.1.2) Establish a category-related attention module B to calculate the output attention feature map M:

[0042] Reference Figure 2 The specific calculations for attention module B related to this category are as follows:

[0043] The input features are processed by weights W respectively. g The fully connected layer obtains the compressed feature f g (x j ), after weight W k The fully connected layer and softmax function are used to obtain the global attention feature map f. k (x j );

[0044] For compression feature f g (x j ) and global attention feature map f k (x j Perform a weighted summation, and then pass the results sequentially through a weighted array. A fully connected layer, a ReLU activation function, and a weight of The fully connected layer and the Sigmoid function are used to obtain the weight vector d. i :

[0045]

[0046] Where δ represents the ReLU activation function and σ represents the Sigmoid activation function. and These are all weights of a fully connected network, used to proportionally reduce and expand the feature map dimension, respectively. N represents the total number of pixels in the feature map, and f... g (x j ) = W g ·x j It is a compression feature, f k (x j = softmax(W) k ·x j) is used to calculate the attention feature map along pixel j. Represents matrix multiplication;

[0047] The weight vector d i After summing the product with the input feature points, the module undergoes non-linear activation using the Sigmoid function to obtain its final output M(x):

[0048] M(x)=Sigmoid(∑d i f i )

[0049] Among them, f i The feature represents the i-th channel of the input.

[0050] 3.1.3) Establish a metric module C to calculate the similarity between the output query image q and class c. Its implementation is as follows:

[0051] For each descriptor x i First, find its k nearest neighbors in class c. Calculate x again i With each The similarity between them;

[0052] The similarity of descriptors is weighted and summed using an attention map to obtain the similarity between a given query image q and class c.

[0053]

[0054]

[0055] Where M(x) i ) represents the attention feature map at x i The response value at the location, x i Let represent the i-th descriptor of q, and m represent the total number of descriptors. x represents i In class c, the j-th nearest neighbor, cos(·) represents the cosine similarity between two vectors. Other distance functions can also be used here.

[0056] 3.1.4) Connect the input of the category-related attention module B to the output of the second convolutional block of the deep attention embedding module A, and then connect the output of the category-related attention module B and the output of the last 1×1 convolutional layer of the deep attention embedding module A to the metric module C. The output of the metric module C is the output of the global flow network.

[0057] 3.2) Using an existing prototype network as the local flow network, the key regions of the query image and support images are input, and the probability of the category to which the key regions of the query image belong is obtained, as follows:

[0058] 3.2.1) Input the key regions of the support set image and the key regions of the query set image into the ResNet18 network for feature extraction, and obtain the support sample features f respectively. φ (x i ) and query sample features f φ (x q ), using support sample features f φ (x i ) Calculate the prototype representation c of the features of category k in the support set. k :

[0059]

[0060] Where S k Let x represent the dataset with support set category k, and let S represent the dataset with support set category k. k In the sample, y is its corresponding category, c k That is, the prototype representation of category k is the average of all embedded features in the support set of that category;

[0061] 3.2.2) Calculate the embedding features f of the query image. φ (x q ) and category k prototype representation c k The distance between them is used to obtain the query image x. q The probability of belonging to category k is:

[0062]

[0063] Where d(·) represents the distance function, c k′ This represents the prototype representation of category k′.

[0064] 3.3) Establish a key region localization module to obtain the key region coordinates B = [x] of the attention feature map M. a x b y a y b ], refer to Figure 3 The key area positioning module is constructed as follows:

[0065] 3.3.1) Perform vector construction operations, that is, aggregate the attention feature maps into two one-dimensional structured energy vectors along the height and width directions of the space:

[0066]

[0067] in, It is the energy vector obtained by aggregating along the width direction. It is the energy vector obtained by aggregating along the height direction, M represents the obtained attention feature map, M(i, W) represents the value of feature map M at position (i, W), M(H, j) represents the value of feature map M at position (H, j), H represents the total height, and W represents the total width;

[0068] 3.3.2) Use a greedy boundary search to locate the most important one-dimensional region using the one-dimensional energy vector, and obtain the coordinates of the key region bounding box B = [x a x b y a y b ]:

[0069] To calculate the width boundary of the critical region [x] a x b Taking [example] as an example, the implementation steps of greedy boundary search are as follows:

[0070] First, initialize the width coordinates x1 and x2 of the feature map, and define the key region as the region that occupies the smallest area and contains energy no less than E of the total energy. Tr The region, namely E [x1，x2] / E [0：W] >E Tr E Tr Hyperparameters representing the proportion of energy. V represents the width vector w The sum of the energies of all the elements in it. This represents the sum of energy in the region from spatial width x1 to x2;

[0071] Next, the boundary [x1, x2] is iteratively adjusted to adjust its energy. converges to E Tr Nearby: When the ratio Higher than E Tr At that time, the [x1, x2] region needs to shrink along the direction of slowest energy decrease until the ratio is no higher than E. Tr Until; when the ratio Below E Tr At that time, the region needs to expand along the direction of the fastest energy increase until it is no less than E. Tr until;

[0072] Then, the boundary coordinates are mapped from the feature map to the input image to obtain the width boundary coordinates [x] of the key region in the mapped input image. a x b ]:

[0073] x a =I w x1 / W

[0074] x b =I w x2 / W

[0075] Among them, I w The input image width is given by W, which represents the width of the feature map.

[0076] Finally, the width boundary coordinates [x] are obtained using [x] a x b Using the same calculation method, the height boundary coordinates [y] of the key region in the input image are obtained. a y b ].

[0077] 3.4) Connect the output of the category-related attention module B of the global flow network to the local flow network through the key region localization module to obtain the overall dual-flow network.

[0078] Step 4: Using the training support set S1 and the training query set T1, train the overall two-stream network using the few-shot scenario training method to obtain the trained two-stream network.

[0079] 4.1) Set the maximum number of training iterations to 300,000, the initial learning rate to 0.0001, and the learning rate to decay every 100,000 generations;

[0080] 4.2) By adding additional margins to the space of the existing cosine loss function, an improved cosine loss function L is constructed. s :

[0081]

[0082] Where N represents the total number of samples in the query set. The query image q represents the class of global stream network computation. The similarity between them The query image q representing global stream network computation is transferred to class c. j The similarity between them, M is the added extra margin hyperparameter, k is the number of nearest neighbors, and m is the number of descriptors of the query image q;

[0083] 4.3) The improved cosine loss function L s Compared with the existing central loss function L c The summation serves as the loss function L for the global flow network. g :

[0084] L g =L s +L c

[0085] in, In the formula, Indicates supporting sample s i The class centers of the global features; m is the size of each scenario set, and in each set, the class centers are calculated by averaging the global features of the corresponding support classes;

[0086] 4.4) Calculate the negative logarithmic probability loss function L based on the image's true label and the predicted probability distribution. l Let it be set as the loss function of the local flow network:

[0087]

[0088] Where N represents the total number of samples in the query set, and C represents the total number of categories in the query set.

[0089] 4.5) Input the images in the training support set S1 and the images in the training query set T1 into the dual-stream network in batches, and calculate the value of the loss function based on the probability of the image prediction category output by the global stream network and the local stream network.

[0090] 4.6) The Adam algorithm is used to backpropagate the loss value and adjust the network parameters;

[0091] 4.7) Repeat steps 4.5)-4.6) until the preset maximum number of training iterations is reached to obtain the trained two-stream network.

[0092] Step 5: Verify the two-stream network:

[0093] Input the validation support set S2 and the validation query set T2 into the trained two-stream network to fine-tune the network parameters. Save the network with the highest index as the optimal two-stream network model and repeat 600 times.

[0094] Step 6: Test the dual-stream network:

[0095] The test support set S3 and the test query set T3 are input into the optimal two-stream network model, which outputs the probability that the test sample belongs to different categories. The category with the highest probability is taken as the final result of scene classification, thus completing the classification task.

[0096] The effects of this invention will be further illustrated below with simulation experiments:

[0097] I. Simulation Experiment Conditions:

[0098] 1. Operating platform configuration

[0099] The simulation platform for this experiment was a desktop computer with an Intel(R) Core(TM) i7-7800X CPU and 32GB of memory, running Ubuntu 18.04. The neural network was built using Python 3.6 and PyTorch 1.4, and accelerated using an NVIDIA RTX2080Ti and CUDA 10.0.

[0100] 2. Simulation Dataset

[0101] The NWPU-RESISC45 dataset contains 45 scene categories, with 700 256×256 RGB images for each category;

[0102] The WHU-RS19 dataset contains 19 scene categories and a total of 1005 RGB images of 600×600 pixels.

[0103] The UC-Merced dataset contains 21 scene categories, with 100 256×256 RGB images for each category.

[0104] 3. Simulation parameter settings

[0105] The simulation experiment used the Adam optimizer with an initial learning rate of 0.0001 and 300,000 training iterations. The learning rate was decayed every 100,000 generations. The number of nearest neighbors searched, k, in the metric module was set to 3. The hyperparameter M in the improved cosine loss function was set to 0.01. The hyperparameter E of the key region localization module was... Tr Set to 70%.

[0106] The small sample scenario is usually represented as a C-way K-shot problem based on the number of categories and samples in the selected support set and query set. In this embodiment, the most common 5-way 1-shot small sample scenario with N=15 and the 5-way 5-shot small sample scenario with N=10 are selected.

[0107] II. Simulation Content

[0108] Simulation 1: Scene classification was performed on the large-scale public remote sensing image datasets NWPU-RESISC45, WHU-RS19, and UC Merced 5-way 1-shot small sample scenes using the method of this invention and existing methods MatchingNet, DLA-MatchNet, and DN4. The classification results of each method were obtained, and their respective classification accuracies were calculated. The results are shown in Table 1.

[0109] Table 1 compares the classification accuracy of the present invention and existing methods in a 5-way 1-shot small sample scenario across three datasets.

[0110] method NWPU-RESISC45 UC Merced WHU-RS19 Existing MatchingNet 54.46％±0.77％ 46.16％±0.71％ 60.60％±0.68％ Existing DLA 68.80％±0.70％ 53.76％±0.62％ 68.27％±1.83％ Existing DN4 66.39％±0.86％ 57.25％±1.01％ 82.14％±0.80％ Method of the present invention 73.84％±0.80％ 68.12％±0.81％ 87.34％±0.62％

[0111] Simulation 2: Scene classification was performed on the large-scale public remote sensing image datasets NWPU-RESISC45, WHU-RS19, and UC Merced 5-way 5-shot small sample scenes using the method of this invention and existing methods MatchingNet, DLA-MatchNet, and DN4. The classification results of each method were obtained, and their respective classification accuracies were calculated. The results are shown in Table 2.

[0112] Table 2 compares the classification accuracy of the present invention and existing methods in a 5-way 5-shot small sample scenario across three datasets.

[0113] method NWPU-RESISC45 UC Merced WHU-RS19 Existing MatchingNet 67.87％±0.59％ 66.73％±0.56％ 82.99％±0.40％ Existing DLA 81.63％±0.46％ 63.01％±0.51％ 79.89％±0.33％ Existing DN4 83.24％±0.87％ 79.74％±0.78％ 96.02％±0.33％ Method of the present invention 87.86％±0.51％ 88.57％±0.52％ 98.25％±0.15％

[0114] As can be seen from the experimental results in Tables 1 and 2, the method of this invention has the highest accuracy on all three datasets, whether for the 5-way 1-shot or 5-way 5-shot task. This indicates that the classification performance of this invention is the best, which can reduce the loss of image discrimination information, avoid the impact of complex backgrounds and drastic changes in object scale on scene classification, and effectively improve the scene classification accuracy of remote sensing images in small sample scenarios. It has very important practical application value in real-world scenarios with few classification samples.

Claims

1. A method for classifying small-sample scenes in remote sensing images based on a multi-scale dual-stream architecture, characterized in that, include: (1) Obtain three different remote sensing image datasets from public websites, and perform preprocessing on the images in the datasets in sequence, including cropping, random horizontal flipping, random brightness enhancement, random color enhancement, and random contrast enhancement; (2) Randomly sample the preprocessed dataset to obtain the training support set S1 and the training query set T1, the validation support set S2 and the validation query set T2, and the test support set S3 and the test query set T3. (3) Constructing an overall dual-stream network: 3a) Construct a global flow network consisting of an attention deep embedding module A, a category-related attention module B, and a metric module C, with the following structure: The attention deep embedding module A consists of a convolutional layer, four convolutional blocks, an average pooling layer, and a cascaded 1×1 convolutional layer with 128 channels. The convolutional layer consists of a 7×7 convolutional filter with 64 channels and a 3×3 max pooling operation. Each convolutional block consists of four 3×3 convolutional filters, with a skip connection operation added after every two convolutional filters. The first convolutional block has 64 channels, the second convolutional block has 128 channels, the third convolutional block has 256 channels, and the fourth convolutional block has 512 channels. The category-related attention module B is used to output the attention feature map M. The measurement module C is used to output the similarity between a given query image q and class c. Connect the input of the category-related attention module B to the output of the second convolutional block of the deep attention embedding module A, and then connect the output of the category-related attention module B and the output of the last 1×1 convolutional layer of the deep attention embedding module A to the metric module C. The output of the metric module C is the output of the global flow network. 3b) Select an existing prototype network as the local flow network; 3c) Establish a key region localization module consisting of vector construction operations and greedy boundary search; 3d) Connect the global flow network and the local flow network through the key region localization module to obtain the overall dual-flow network; (4) Using the training support set S1 and the training query set T1, the entire dual-stream network is trained by the few-sample scenario training method to obtain the trained dual-stream network. (5) Input the validation support set S2 and the validation query set T2 into the trained two-stream network to fine-tune the network parameters, and save the network with the highest index as the optimal two-stream network model. (6) Input the test support set S3 and the test query set T3 into the optimal two-stream network model to obtain the final classification result.

2. The method according to claim 1, characterized in that, Step (2) is implemented as follows: 2a) The three different remote sensing image datasets after preprocessing are divided into training, validation and test sets respectively. The NWPU-RESISC45 dataset is divided into training, validation and test sets in a ratio of 25:10:

10. The WHU-RS19 dataset is divided into training, validation and test sets in a ratio of 9:5:

5. The UC-Merced dataset is divided into training, validation and test sets in a ratio of 10:6:

5. 2b) Randomly select a category from the training set of each dataset, and randomly sample K images from each category. These C×K images constitute the training support set S1. At the same time, randomly select N images from the remaining images of these C categories to form the training query set T1. 2c) Randomly select C categories from the validation set of each dataset, and randomly sample K images from each category. These C×K images constitute the validation support set S2. From the remaining images of these C categories, randomly select N images of equal quantity from each category to form the validation query set T2. 2d) Randomly select C categories from the test set of each dataset, and randomly sample K images from each category. These C×K images constitute the test support set S3. From the remaining images of these C categories, randomly select an equal number of N images from each category to form the test query set T. 3。 3. The method according to claim 1, characterized in that, Category-related attention module B calculates the output attention feature map M, and the calculation formula is as follows: M(x)=Sigmoid(∑d i f i ); Among them, f i d represents the feature of the i-th channel of the input. i The weight vector is calculated as follows: Where δ represents the ReLU activation function and σ represents the Sigmoid activation function. and These are all weights of a fully connected network, used to proportionally reduce and expand the feature map dimension, respectively. N represents the total number of pixels in the feature map, and f... g (x j ) = W g ·x j f k (x j = softmax(W) k ·x j ) is used to calculate the attention feature map along pixel j. This represents matrix multiplication.

4. The method according to claim 1, characterized in that, The metric module C calculates the similarity between a given query image q and its class. For each descriptor x i First, find its k nearest neighbors in class c. Calculate x again i With each The similarity between the descriptors is calculated, and the similarity of the descriptors is weighted and summed using an attention map to obtain the similarity between a given query image q and class c. Where M(x) i ) represents the attention feature map at x i The response value at the location, x i Let represent the i-th descriptor of q, and m represent the total number of descriptors. x represents i In class c, the j-th nearest neighbor, cos(·), represents the cosine similarity between two vectors.

5. The method according to claim 1, characterized in that, Step (3c) establishes a key region localization module consisting of vector construction operations and greedy boundary search, implemented as follows: 3c1) The vector construction operation aggregates the attention map into two one-dimensional structural energy vectors along the height and width directions of the space: in, It is an energy vector obtained by aggregating along the height direction. It is the energy vector obtained by aggregating along the width direction, M represents the obtained attention feature map, M(i, W) represents the value of feature map M at position (i, W), M(H, j) represents the value of feature map M at position (H, j), H represents the total height, and W represents the total width; 3c2) Greedy boundary search is a fast and accurate method to locate the most important one-dimensional region in a one-dimensional energy vector, obtaining the coordinates of the region's bounding box B = [x a ,x b ,y a ,y b ].

6. The method according to claim 1, characterized in that, In step (4), the training support set S1 and the training query set T1 are used to train the overall two-stream network using a few-shot scenario training method, as follows: 4a) Assume the maximum number of training iterations is 300,000, the initial learning rate is 0.0001, and the learning rate is decayed every 100,000 generations; 4b) Set the global flow network loss function to the sum of the central loss function and the improved cosine loss function, and set the local flow network loss function to the negative log probability loss function; 4c) Input the images in the training support set S1 and the images in the training query set T1 into the dual-stream network in batches, and calculate the value of the loss function based on the probability of the image prediction category output by the global stream network and the local stream network. 4d) The Adam algorithm is used to backpropagate the loss value and adjust the network parameters; 4e) Repeat steps (4c)-(4d) until the preset maximum number of training iterations is reached to obtain the trained two-stream network.

Images