Discriminative boosting based small sample remote sensing image scene classification method

Abstract

The application discloses a small sample remote sensing image scene classification method based on discriminative enhancement, and belongs to the field of image perception. The method first divides a remote sensing image scene classification dataset into three parts of a training set, a verification set and a test set; then constructs a scenario set based on the training set, the verification set and the test set; then constructs a deep neighbor neural network model based on discriminative enhancement; then trains and verifies the deep neighbor neural network model based on discriminative enhancement based on the scenario set; and finally tests the trained deep neighbor neural network model based on discriminative enhancement based on the scenario set. The application introduces an attention mechanism for distinguishing between relevant and irrelevant areas in a scene, proposes a local-global descriptor, comprehensively uses a center loss and an improved Softmax loss, and a deep embedding module can extract more discriminative features, so that the small sample remote sensing image scene classification problem is solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image perception, and particularly relates to a small sample remote sensing image scene classification method based on discriminative enhancement, which can be used for classifying small sample remote sensing images and providing support for environment perception. BACKGROUND

[0002] Remote sensing image scene classification has important roles in both military and civilian fields. However, in remote sensing images, there are large intra-class variances and large inter-class similarities, which make it difficult to correctly classify the scene images. The large intra-class variance means that the appearances of ground objects in the same semantic class are diverse. Some ground objects are different in style, shape, and distribution, for example, churches appear in different architectural styles, and airports and railway stations have different shapes. In addition, when airborne or space platforms capture remote sensing images, due to different imaging conditions, the color and radiation intensity in the same semantic class may have large differences due to the influence of factors such as weather, clouds, and fog. For example, beach scenes show great differences under different imaging conditions. The large inter-class similarity in remote sensing images is mainly due to the existence of the same objects in different scene classes or the high semantic overlap between scene classes, for example, some bridge and overpass scenes contain the same ground objects, and basketball courts and tennis courts have high semantic information. In addition, the fuzzy definition of scene classes also leads to a decrease in inter-class differences, so that some complex scenes may be visually similar. Therefore, for the cases of large intra-class diversity and large inter-class similarity, it may be extremely difficult to distinguish these scene classes.

[0003] Small sample learning is a topic that hopes that the model has the characteristics of fast learning, and studies how to quickly learn from a small number of labeled samples. Therefore, in the small sample task, the deep features learned by the model not only have good separability, but also have strong discriminativeness, so as to identify new classes in the case of a small number of labeled samples. Most small sample methods based on metric learning directly calculate the distance between the support samples and the query samples, and learn a classifier according to the distance, without fully exploiting the powerful feature extraction capability of the network, which also leads to weak discriminativeness of the features output by the model, and further limits the performance of the small sample model. In remote sensing images, the intra-class variance is large and the inter-class similarity is large, so the scene images are easily confused, for example, images not belonging to the same class are classified into one class, and images of the same class may be classified into different classes due to the diversity of samples. Therefore, learning a classifier that can further extract discriminative features of remote sensing images plays an important role in improving the performance of small sample remote sensing image scene classification. SUMMARY

[0004] In view of this, this invention proposes a discriminatively enhanced few-sample remote sensing image scene classification method, realizing an end-to-end few-sample remote sensing image scene classification framework. This invention introduces an attention mechanism to distinguish between classification-related and irrelevant regions in the scene; to obtain more robust features, this invention proposes a deep embedding model that combines local and global information, using local-global descriptors for classification; to increase intra-class feature compactness, this invention introduces a center loss to optimize global information, thereby increasing intra-class compactness and effectively pulling features of the same class to their center, thus overcoming significant intra-class diversity. Furthermore, the softmax loss function of the classification module is improved by incorporating cosine margins, which increases the distance between learned feature classes. Under the joint supervision of the center loss and the improved softmax loss, the deep embedding module can extract more discriminative features.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A few-sample remote sensing image scene classification method based on discriminative enhancement includes the following steps:

[0007] Step 1: Divide the remote sensing image scene classification dataset into a training set, a validation set, and a test set;

[0008] Step 2: Construct corresponding scenario sets based on the training set, validation set, and test set, respectively;

[0009] Step 3: Construct a deep nearest neighbor neural network model based on discriminative enhancement, including a deep embedding module and a metric module, and construct a model loss function based on center loss and class loss;

[0010] Step 4: Based on the scenario sets corresponding to the training and validation sets, train and validate the deep nearest neighbor neural network model based on discriminative enhancement.

[0011] Step 5: Test the trained deep nearest neighbor neural network model based on discriminative enhancement based on the scenario set corresponding to the test set;

[0012] Complete scene classification of small-sample remote sensing images based on discriminative enhancement.

[0013] Furthermore, the specific method for constructing the scenario set in step 2 is as follows:

[0014] 2a) For the training set, validation set, and test set, randomly select C classes from each set, with K samples from each class. Use the extracted sample set as the support set S = {(s...} i ,y i )|i=1,...,C×K}, where s i Let y represent the i-th sample in the support set.i This indicates the corresponding label, where C and K are set values;

[0015] 2b) From the remaining samples of each of the C categories, randomly select N samples to obtain the corresponding query set. Where q i This represents the j-th sample in the query set. This represents the corresponding label; where N is a set value.

[0016] 2c) Combine one support set and one query set to form a scenario set, and obtain the scenario sets corresponding to the training set, validation set and test set respectively.

[0017] Furthermore, the construction method of the deep nearest neighbor neural network model based on discriminative enhancement described in step 3 is as follows:

[0018] The discriminative enhancement-based deep nearest neighbor neural network model consists of two modules: a deep embedding module f based on an attention mechanism. ψ (·) and measurement module

[0019] 3a) Deep Embedding Module f Based on Attention Mechanism ψ The construction method of (·) is as follows:

[0020] A convolutional neural network is used as a deep embedding module for deep local-global descriptor extraction. The deep features f extracted by the deep embedding module are... ψ (X image Let be a tensor of size h×w×d, and represent it as a set of m d-dimensional depth descriptors:

[0021] f ψ (X image )=[x1,x2,...,x m ]∈R d×m

[0022] In the formula, w, h, and d represent the width, height, and number of channels of the extracted feature map, respectively, and m = h × w, x i This is the i-th depth local-global descriptor;

[0023] The deep embedding module includes a deep local embedding module and a deep global embedding module. The deep local embedding module is used to obtain the deep local descriptor, and the deep global embedding module is used to obtain the deep global descriptor. That is, the deep local-global descriptor is divided into two parts: the deep local descriptor and the deep global descriptor.

[0024] Here, the depth local descriptor is considered as a set of m d1-dimensional depth descriptors, represented as:

[0025] fψ1 (X image )=[x 11 ,x 21 ,...,x m1 ]∈R d1×m

[0026] The depth global descriptor is represented as:

[0027] f ψ2 (X image )=[x g1 ,x g2 ,...,x gd2 ]∈R d2×1

[0028] In the formula, d1+d2=d, for m d1-dimensional depth local descriptors, a d2-dimensional depth global descriptor is directly added after each depth local descriptor, thus obtaining the depth local-global descriptor;

[0029] 3b) An attention mechanism is introduced to the deep embedding module to divide the deep local descriptor into scene-related and scene-independent parts. This is accomplished by constructing a feature map for each pixel in the feature map obtained by the deep embedding module. Specifically, this is done by using a squeezed excitation network as the base module and employing a non-local attention mechanism. The calculation method is as follows:

[0030]

[0031] In the formula, σ and δ represent the ReLU activation function and the Sigmoid activation function, respectively, and W z1 and W z2 Both are fully connected weights, used to proportionally reduce and expand the feature map dimension, respectively. m represents the total number of pixels in the feature map, and f... g (x i1 ) = W g ·x i1 W g For the weight vector, f represents matrix multiplication. k (x i1 =Softmax(W) k ·x i1 ), where W k For feature weights;

[0032] Obtain the weight vector [am1,...,am] for each channel. d1 ]; where d1 is the number of channels in the feature map, am b This indicates whether the b-th channel is related to the scene category. If it is related, am b =1, otherwise, am b=0, and based on the learned feature channel weight vector, further obtain the category-related attention feature map, as shown below:

[0033] M(x)=Sigmoid(∑am b x i1 )

[0034] The scene-related channels are summed up and then passed through the Sigmoid function to obtain the attention feature map of each pixel position related to the scene category;

[0035] 3c) Measurement module The construction method is as follows:

[0036] Through the deep embedding module, each query image will obtain m deep local descriptors and deep local-global descriptors. For each deep local descriptor x i1 Find the k nearest neighbors in a certain class c using the nearest neighbor method. Calculate the corresponding x i and The similarity between them is calculated as follows;

[0037]

[0038]

[0039] In the formula, x represents the similarity between a given query image q and category c. i Let x represent the i-th local-global descriptor of the query image q. i1 This represents the i-th local descriptor of the query image q, where m represents the total number of local descriptors. x represents i1 The j-th nearest neighbor in category c x represents i The transpose of the vector, cos(·), represents the cosine similarity between the two vectors, M(x i1 ) represents the attention feature map at x i1 The responsiveness at the location, where k is a set value;

[0040] 3d) Construct the center loss function and the class loss function to obtain the overall loss function;

[0041] Central loss function L c for:

[0042]

[0043] In the formula, C represents the number of classes, K represents the number of samples in each class of the support set, and f ijf represents the global feature of the j-th sample in the i-th support set. ci The mean of the global features representing the samples in the support set of class i;

[0044] Category loss function L s for:

[0045]

[0046] In the formula, p ij The value represents the probability that the query image is correctly classified, C represents the number of classes in the support set, N represents the number of samples of each class in the query set, and M represents the similarity of each descriptor. Added spacing;

[0047] The final overall loss function L is expressed as:

[0048] L = L c +L s .

[0049] Compared with the prior art, the present invention has the following advantages:

[0050] 1. This invention addresses the problem of complex scene backgrounds in remote sensing image scene classification by introducing an attention mechanism to distinguish between classification-related and irrelevant regions in the scene, and by introducing a context training method to solve the problem of scene classification for small-sample remote sensing images.

[0051] 2. This invention proposes a deep embedding model that combines local and global information and uses local-global descriptors for classification, increasing the difference between descriptors of different categories.

[0052] 3. To increase the compactness of intra-class features, this invention introduces a center loss to optimize global information. By using the center loss, intra-class compactness is increased, effectively pulling features of the same class to their center, thereby overcoming significant intra-class diversity.

[0053] 4. This invention improves the Softmax loss function of the classification module by adding cosine margin, which increases the distance between the learned feature classes.

[0054] 5. This invention combines the use of center loss and improved Softmax loss, allowing the deep embedding module to extract more discriminative features. Attached Figure Description

[0055] Figure 1 This is a diagram of the overall architecture of the present invention.

[0056] Figure 2 This is a diagram of the local-global depth descriptor extraction architecture of the present invention.

[0057] Figure 3 This is a structural diagram of the attention module related to the category of this invention. Detailed Implementation

[0058] The technical solution and effects of the present invention will be further described in detail below with reference to the accompanying drawings.

[0059] Reference Figure 1 A few-sample remote sensing image scene classification method based on discriminative enhancement includes the following steps:

[0060] Step 1: Divide the remote sensing image scene classification dataset into three parts: training set, validation set, and test set. The specific division method is as follows:

[0061] The remote sensing scene classification dataset is divided into a training set, a validation set, and a test set. The training set, validation set, and test set have no overlap. The union of the training set, validation set, and test set is the remote sensing scene classification dataset.

[0062] Step 2: Construct scenario sets based on the training set, validation set, and test set respectively. The specific construction method is as follows:

[0063] 2a) For the training set, validation set, and test set, randomly select C classes from each set, with K samples from each class. Use the extracted sample set as the support set S = {(s...} i ,y i )|i=1,...,C×K}, where s i Let y represent the i-th sample in the support set. i This indicates the corresponding label, where C and K are set values;

[0064] 2b) From the remaining samples of each of the C categories, randomly select N samples to obtain the corresponding query set. Where q i This represents the j-th sample in the query set. This represents the corresponding label; where N is a set value.

[0065] 2c) Combine one support set and one query set to form a scenario set, and obtain the scenario sets corresponding to the training set, validation set and test set respectively.

[0066] Step 3: Construct a deep nearest neighbor neural network model based on discriminative enhancement. The specific construction method is as follows:

[0067] The discriminative enhancement-based deep nearest neighbor neural network model can be divided into two modules: a deep embedding module based on an attention mechanism, fi ψ (·) and measurement module

[0068] 3a) Deep Embedding Module f Based on Attention Mechanism ψ The construction method of (·) is as follows:

[0069] ① Any suitable convolutional neural network can be used as a deep embedding module, such as Conv-64F, VGG16, ResNet18, ResNet50, etc.;

[0070] ②For example Figure 2 As shown, the deep local-global descriptor extraction method is as follows: the deep features f extracted by the deep embedding module ψ (X image Essentially, it is a tensor of size h×w×d, and represented as a set of m d-dimensional depth descriptors:

[0071] f ψ (X image )=[x1,x2,...,x m ]∈R d×m

[0072] In the formula, w, h, and d represent the width, height, and number of channels of the extracted feature map, respectively, and m = h × w, x i This is the i-th depth local-global descriptor;

[0073] The deep embedding module can be divided into a deep local embedding module and a deep global embedding module. The deep local embedding module obtains a deep local descriptor, and the deep global embedding module obtains a deep global descriptor. That is, the deep local-global descriptor can be divided into two parts: a deep local descriptor and a deep global descriptor.

[0074] Here, the depth local descriptor is considered as a set of m d1-dimensional depth descriptors, represented as:

[0075] f ψ1 (X image )=[x 11 ,x 21 ,...,x m1 ]∈R d1×m

[0076] The depth global descriptor can be represented as:

[0077] f ψ2 (X image )=[x g1 ,x g2 ,...,x gd2 ]∈R d2×1

[0078] In the formula, d1+d2=d. For m d1-dimensional depth local descriptors, a d2-dimensional depth global descriptor is directly added after each depth local descriptor to obtain the depth local-global descriptor.

[0079] 3b) The category-related attention module introduces an attention mechanism to the deep embedding module, distinguishing the deep local descriptor into scene-related and scene-independent parts. This is accomplished by constructing a feature map for each pixel in the feature map obtained from the deep embedding module, such as... Figure 3 As shown, the specific approach involves using a squeeze-and-excitation (SE) network as the basic module and replacing the global average pooling (GAP) operation in SE with a non-local attention mechanism. The specific calculation method is as follows:

[0080]

[0081] In the formula, σ and δ represent the ReLU activation function and the Sigmoid activation function, respectively, and W z1 and W z2 Both are fully connected weights, used to proportionally reduce and expand the feature map dimension, respectively. m represents the total number of pixels in the feature map, and f... g (x i1 ) = W g ·x i1 W g For the weight vector, f represents matrix multiplication. k (x i1 =Softmax(W) k ·x i1 ), where W k For feature weights;

[0082] Obtain the weight vector [am1,...,am] for each channel. d1 ]; where d1 is the number of channels in the feature map, am b This indicates whether the b-th channel is related to the scene category. If it is related, am b =1, otherwise, am b =0, and based on the learned feature channel weight vector, further obtain the category-related attention feature map, as shown below:

[0083] M(x)=Sigmoid(∑am b x i1 )

[0084] The scene-related channels are summed up and then passed through the Sigmoid function to obtain the attention feature map of each pixel position related to the scene category;

[0085] 3c) Attention-based measurement module The construction method is as follows:

[0086] Through the deep embedding module, each query image will obtain m deep local descriptors and deep local-global descriptors. For each deep local descriptor x i1 Find the k nearest neighbors in a certain class c using the nearest neighbor method. Calculate the corresponding x i and The similarity between descriptors is considered, but not all local image regions are equally representative to describe the target object in the image. Therefore, this patent uses the attention map obtained in the deep embedding module based on the attention mechanism to perform a weighted summation of the similarity of the descriptors. Local descriptors that are strongly correlated with the scene category will have a greater impact on the final classification result due to their higher weight, while local descriptors of interference regions will have a reduced negative impact due to their lower weight. The calculation method of the measurement module is as follows:

[0087]

[0088]

[0089] In the formula, x represents the similarity between a given query image q and category c. i Let x represent the i-th local-global descriptor of the query image q. i1 This represents the i-th local descriptor of the query image q, where m represents the total number of local descriptors. x represents i1 The j-th nearest neighbor in category c x represents i The transpose of the vector, cos(·), represents the cosine similarity between the two vectors, M(x i1 ) represents the attention feature map at x i1 The responsiveness at the location, where k is a set value;

[0090] 3d) Construct the central loss function:

[0091]

[0092] In the formula, C represents the number of classes, K represents the number of samples in each class of the support set, and f ij f represents the global feature of the j-th sample in the i-th support set. ci The i-th class supports the mean of global features of the samples in the set.

[0093] A category loss function is constructed, using the similarity between the query image and each category as the probability of predicting the category. The network's loss function is based on Softmax loss, referencing the idea of ​​LMCL loss function, and calculates the similarity for each descriptor. Add a spacing M, each descriptor has k nearest neighbors, and an image has m descriptors. Then, querying image q... j To class c i The additional spacing added to the similarity between them is mkM, and the improved loss function formula is as follows:

[0094]

[0095] In the formula, p ij This represents the probability that the query image is correctly classified, where C represents the number of support set categories, and N represents the number of training samples for each category of the query set.

[0096] The final overall loss function L is expressed as:

[0097] L = L c +L s

[0098] Step 4: Based on the scenario sets corresponding to the training and validation sets, train and validate the deep nearest neighbor neural network model based on discriminative enhancement; the specific method is as follows:

[0099] The scenario set in the training set is used to train the network parameters, while the scenario set in the validation set is used to evaluate the scenario classification performance of the currently trained model.

[0100] Step 5: Test the trained deep nearest neighbor neural network model based on discriminative enhancement using the scenario set corresponding to the test set; the specific method is as follows:

[0101] Multiple scenario sets constructed based on the test set are used to classify samples in the query set based on the support set to obtain classification results. Finally, the average of the classification results of all scenario sets in the test set is taken as the test result.

[0102] The effectiveness of this method can be further illustrated by the following simulation experiments:

[0103] 1. Experimental conditions and methods

[0104] 1) Hardware environment:

[0105] ①CPU: Intel(R)Core(TM)i7-7800X CPU@3.50GHz 32GB;

[0106] ②GPU: NVIDIA Geforce RTX2080Ti 11GB.

[0107] 2) Software environment:

[0108] ① Programming language: Python 3.6;

[0109] ② Deep learning framework: PyTorch 1.4.0.

[0110] 3) Experimental methods:

[0111] ①MatchingNet;

[0112] ②RelationNet;

[0113] ③MAML;

[0114] ④ Meta-SGD;

[0115] ⑤DLA-MatchNet;

[0116] ⑥DN4;

[0117] ⑦ This patented method.

[0118] 4) Dataset:

[0119] ①NWPU-RESISC45 dataset;

[0120] ②UC Merced dataset;

[0121] ③WHU-RS19 dataset.

[0122] 5) Scene classification task:

[0123] 1) 5-way 1-shot task, which includes supporting sets containing 5 categories, with each category containing 1 image;

[0124] 2) 5-way 5-shot task, which includes five categories, with each category containing five images.

[0125] 6) Evaluation Indicators

[0126] The average accuracy of top-1, with a 95% confidence interval.

[0127] 2. Simulation Content and Results

[0128] The deep embedding module of this invention uses a ResNet18 network. The input image is randomly cropped to 224×224 and then enhanced with random horizontal flipping, brightness enhancement, color enhancement, and contrast enhancement. The number of nearest neighbors searched, k, is set to 3 in the classification module. The hyperparameter M in the loss function is set to 0:01.

[0129] A training set of 300,000 scenarios was constructed. The Adam algorithm was used to train the network, with an initial learning rate of 0.0001. The learning rate was decayed every 100,000 scenarios. 600 scenarios were randomly constructed on the validation set for validation. The average accuracy of the top-1 scenarios was taken as the training result of the current network, and the model with the highest accuracy was saved as the final model.

[0130] During testing, 600 small sample tasks were randomly sampled from the test set, and the top-1 average accuracy was calculated. This process was repeated 5 times, and the average of the 5 test results was taken as the final test result, with a 95% confidence interval provided. The model of this invention is trained from scratch in an end-to-end manner, requiring no fine-tuning during the testing phase.

[0131] The experimental results of different methods on the three datasets are shown in Tables 1-3, with bold numbers indicating the best results. The results show that the method of this invention achieves the highest accuracy on all three datasets, regardless of whether it's a 5-way 1-shot or 5-way 5-shot task. This indicates that the method of this invention has the best classification performance and can effectively improve the accuracy of remote sensing image scene classification in small-sample scenarios.

[0132] Table 1. Performance comparison of different methods on the NWPU-RESISC45 dataset.

[0133]

[0134]

[0135] Table 2 Performance comparison of different methods on the UC Merced dataset

[0136] Method 5-way 1-shot 5-way 5-shot MatchingNet 46.16％±0.71％ 66.73％±0.56％ RelationNet 48.89％±0.73％ 64.10％±0.54％ MAML 43.65％±0.68％ 58.43％±0.64％ Meta-SGD 50.52％±2.61％ 60.82％±2.00％ DLA-MatchNet 53.76％±0.62％ 63.01％±0.51％ DN4 57.25％±1.01 79.74％±0.78％ The method of the invention 67.27％±0.74％ 87.69％±0.44％

[0137] Table 3. Performance comparison of different methods on the WHU-RS19 dataset.

[0138] Method 5-way 1-shot 5-way 5-shot MatchingNet 60.60％±0.68％ 82.99％±0.40％ RelationNet 60.54％±0.71％ 76.24％±0.34％ MAML 46.72％±0.55％ 79.88％±0.41％ Meta-SGD 51.54％±2.31％ 61.74％±2.02％ DLA-MatchNet 68.27％±1.83％ 79.89％±0.33％ DN4 82.14％±0.80％ 96.02％±0.33％ The method of the invention 86.89％±0.57％ 97.63％±0.19％

[0139] To verify the effectiveness of the local-global descriptor and the Softmax loss function incorporating cosine margin, ablation experiments were conducted on the NWPU-RESISC45 dataset. Table 4 shows the performance of models with different modules on the NWPU-RESISC45 dataset, with bold numbers indicating the best results. The top-1 average accuracy was used as the evaluation metric in the ablation experiments. As can be seen from Table 4, both the local-global descriptor and cosine margin improve the model's classification results, especially for 5-way 1-shot classification, demonstrating the effectiveness of the proposed method.

[0140] Table 4 compares the performance of models with different modules on the NWPU-RESISC45 dataset.

[0141]

[0142] This invention also explores the value of the hyperparameter M. M represents the size of the added extraneous margin and plays a crucial role in the loss function. This invention performs a 5-way 5-shot task on the NWPU-RESISC45 dataset and varies the value of M. The results are shown in Table 5. It can be seen that the loss function without cosine margin (M=0) leads to poor model performance. As M increases, the model accuracy on the NWPU-RESISC45 dataset continuously improves, reaching saturation at M=0.01. This demonstrates the effectiveness of the cosine margin M, indicating that appropriately increasing the margin M can significantly improve the discriminative power of the learned features.

[0143] Table 5. Performance comparison of different hyperparameters M on the NWPU-RESISC45 dataset.

[0144] Method M＝0 M＝0.005 M＝0.01 M＝0.015 M＝0.02 The method of the patent 86.83％ 87.11％ 87.28 86.28 84.62％

Claims

1. A scene classification method for small-sample remote sensing images based on discriminative enhancement, characterized in that, Includes the following steps: Step 1: Divide the remote sensing image scene classification dataset into a training set, a validation set, and a test set; Step 2: Construct corresponding scenario sets based on the training set, validation set, and test set, respectively; Step 3: Construct a deep nearest neighbor neural network model based on discriminative enhancement, including a deep embedding module and a metric module, and construct a model loss function based on center loss and class loss; Step 4: Based on the scenario sets corresponding to the training and validation sets, train and validate the deep nearest neighbor neural network model based on discriminative enhancement. Step 5: Test the trained deep nearest neighbor neural network model based on discriminative enhancement based on the scenario set corresponding to the test set; Complete scene classification of small-sample remote sensing images based on discriminative enhancement; The construction method of the deep nearest neighbor neural network model based on discriminative enhancement described in step 3 is as follows: The discriminative enhancement-based deep nearest neighbor neural network model consists of two modules: a deep embedding module based on an attention mechanism. and measurement module ; 3a) Deep Embedding Module Based on Attention Mechanism The construction method is as follows: A convolutional neural network is used as a deep embedding module for deep local-global descriptor extraction. The deep features extracted by the deep embedding module are... For a size of The tensor, and expressed as indivual The set of depth descriptors of a dimension: ； In the formula, , , These represent the width, height, and number of channels of the extracted feature map, respectively. , For the first i A deep local-global descriptor; The deep embedding module includes a deep local embedding module and a deep global embedding module. The deep local embedding module is used to obtain the deep local descriptor, and the deep global embedding module is used to obtain the deep global descriptor. That is, the deep local-global descriptor is divided into two parts: the deep local descriptor and the deep global descriptor. Among them, the depth local descriptor is regarded as indivual The set of depth descriptors of dimension is represented as: ； The depth global descriptor is represented as: ； In the formula, ,against indivual 3D depth local descriptors, each of which is directly appended with... The depth-global descriptor is obtained, which is the depth-local-global descriptor; 3b) An attention mechanism is introduced to the deep embedding module to divide the deep local descriptor into scene-related and scene-independent parts. This is accomplished by constructing a feature map for each pixel in the feature map obtained by the deep embedding module. Specifically, this is done by using a squeezed excitation network as the base module and employing a non-local attention mechanism. The calculation method is as follows: ； In the formula, and These represent the ReLU activation function and the Sigmoid activation function, respectively. and All are fully connected weights, used to proportionally reduce and expand the feature map dimension, respectively. This represents the total number of pixels in the feature map. ,in For the weight vector, Represents matrix multiplication. ,in For feature weights; Obtain the weight vector for each channel. ;in, The number of channels in the feature map. Indicates the first Are the channels related to the scene category? If so, ,otherwise, Based on the learned feature channel weight vectors, the category-related attention feature maps are further obtained, as shown below: ； The scene-related channels are summed up and then passed through the Sigmoid function to obtain the attention feature map of each pixel position related to the scene category; 3c) Measurement module The construction method is as follows: Through the deep embedding module, each query image will obtain Each deep local descriptor and deep local-global descriptor The nearest neighbor method in a certain category Found nearest neighbors Calculate the corresponding and The similarity between them is calculated as follows; ； In the formula, Represents the given query image With category Similarity between them Indicates query image The A local-global descriptor Indicates query image The A local descriptor, This represents the total number of local descriptors. express In category The first in The nearest neighbor, express transpose, The cosine similarity between two vectors is represented by the following expression: Indicates attention feature map in Response at location k Set value; 3d) Construct the center loss function and the category loss function to obtain the overall loss function; Central loss function L c for: ； In the formula, The number of classes, This represents the number of samples in each class of the support set. Representing the Class support set Global features of a sample Representing the The class supports the mean of global features of a set of samples; Category loss function L s for: ； In the formula, This indicates the probability that the query image is correctly classified. Represents the number of support set categories. This represents the number of samples of each class in the query set. Similarity for each descriptor Added spacing; Final overall loss function Represented as: 。 2. The method for scene classification of remote sensing images based on discriminative enhancement in small samples according to claim 1, characterized in that, The specific method for constructing the scenario set in step 2 is as follows: 2a) For the training set, validation set, and test set, randomly select from them respectively. C Each category K For each sample, the extracted sample set is used as the support set. ,in Indicates support for the first in the centralization i One sample, Indicates the corresponding label, C and K Set value; 2b) In C From the remaining samples of each category, randomly select N Each sample yields the corresponding query set. ,in Indicates the first in the query set j One sample, Indicates the corresponding label; in N Set value; 2c) Combine one support set and one query set to form a scenario set, and obtain the scenario sets corresponding to the training set, validation set and test set respectively.

Images