Semi-supervised multi-spectral remote sensing image scene classification method, device, equipment and medium

Abstract

The present invention provides a semi-supervised multi-spectral remote sensing image scene classification method, device, equipment and medium, belonging to the technical field of remote sensing image scene classification. After performing weak and strong enhancement processing on unlabeled multi-spectral remote sensing images, they are input into a pre-constructed dual-branch network structure, and the strong and weak enhancement prediction results output by each branch network are fused to obtain pseudo-labels of the unlabeled multi-spectral remote sensing images. The dual-branch network structure is trained according to the pseudo-labels, and the trained dual-branch network structure is used as a scene classification model; based on the scene classification model, scene classification is performed on target multi-spectral remote sensing images to obtain target scene classification results. The dual-branch network structure makes full use of the spatial and spectral information of multi-spectral remote sensing images and the complementary information between different bands. The spectral feature branch network introduces spectral attention, which can further enhance the spectral information extraction ability, obtain more accurate pseudo-labels, and thus improve the classification performance of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image scene classification, and in particular to a semi-supervised multi-spectral remote sensing image scene classification method, device, equipment and medium. Background Technique

[0002] Existing semi-supervised learning techniques aim to improve the training efficiency and performance of models by combining a small amount of labeled data and a large amount of unlabeled data. In image classification tasks, semi-supervised methods are mainly divided into two categories: one is pseudo-label generation based on confidence, which guides model training by screening high-confidence pseudo-labels; the other is consistency regularization, which constrains the model to keep the predictions of the same data sample consistent by applying different perturbations (such as augmentation operations) to the input data.

[0003] In recent years, semi-supervised learning has also been widely applied in the field of remote sensing. Especially in the scene classification task of multi-spectral remote sensing images, such techniques are used to alleviate the problem of scarce labeled data. On the one hand, the data of multi-spectral remote sensing images is complex and inconsistent, and the consistency regularization method is not applicable to the scene classification task of multi-spectral remote sensing images. On the other hand, most existing scene classification models are mainly designed for RGB images, lacking the spectral information of multi-spectral remote sensing images, and the generated pseudo-labels have insufficient accuracy, thus limiting the performance of the model to a certain extent. Summary of the Invention

[0004] The present invention provides a semi-supervised multi-spectral remote sensing image scene classification method, device, equipment and medium, which can fully exploit the spectral information of multi-spectral remote sensing images, improve the accuracy of pseudo-labels, and enhance the classification performance of the model.

[0005] The present invention provides a semi-supervised multi-spectral remote sensing image scene classification method, including:

[0006] Obtain unlabeled multi-spectral remote sensing images, and perform weak augmentation processing and strong augmentation processing on the unlabeled multi-spectral remote sensing images respectively to obtain weakly augmented remote sensing images and strongly augmented remote sensing images;

[0007] Input the weakly augmented remote sensing image and the strongly augmented remote sensing image into a dual-branch network structure pre-constructed by a spatial feature branch network and a spectral feature branch network respectively. The spatial feature branch network is used to extract the respective spatial features of the weakly augmented remote sensing image and the strongly augmented remote sensing image, and obtain strongly augmented prediction results and weakly augmented prediction results corresponding to the spatial features according to the spatial features; the spectral feature branch network is used to extract the respective spectral features of the weakly augmented remote sensing image and the strongly augmented remote sensing image based on spectral attention, and obtain strongly augmented prediction results and weakly augmented prediction results corresponding to the spectral features according to the spectral features;

[0008] Fuse the weakly enhanced prediction results corresponding to the spatial features and the weakly enhanced prediction results corresponding to the spectral features to obtain the pseudo-labels of the unlabeled multispectral remote sensing image, and obtain an unsupervised loss function according to the pseudo-labels and the strongly enhanced prediction results corresponding to the spatial features and the spectral features respectively. The unsupervised loss function is used to determine the total loss function;

[0009] According to the total loss function, determine whether the dual-branch network structure is trained. If so, use the dual-branch network structure as the scene classification model. If not, adjust the parameters of the dual-branch network structure, and return to execute the step of obtaining the unlabeled multispectral remote sensing image until the training is completed;

[0010] Perform scene classification on the target multispectral remote sensing image based on the scene classification model to obtain the target scene classification result.

[0011] As an embodiment, the spatial feature branch network is obtained based on the deep residual network, and the spectral feature branch network is obtained by adding a spectral attention module to each network layer of the deep residual network;

[0012] The spectral attention module is used to determine the query parameter, key parameter, and value parameter of the input image feature map based on a linear layer, perform dimensionality transformation on the query parameter, the key parameter, and the value parameter, transmit the query parameter and the key parameter after dimensionality transformation to the normalization layer, determine the attention scores between different spectral bands of the image feature map based on the normalization layer, perform an operation on the attention scores and the value parameter after dimensionality transformation and then transmit it to the output linear layer, obtain the output image feature map based on the output linear layer, and perform dimensionality transformation on the output image feature map and then output it to the next network layer.

[0013] As an embodiment, the calculation formulas of the query parameter, the key parameter, and the value parameter are as follows:

[0014]

[0015] Among them, represents the query parameter, represents the key parameter, represents the value parameter, 、 、 respectively represent the parameter matrices of three linear layers, represents the input image feature map, , R represents the real number field, 、 、 respectively represent the number of channels, width, and height of the input image feature map;

[0016] Correspondingly, performing dimensionality transformation on the query parameter, the key parameter, and the value parameter includes:

[0017] respectively transforming the query parameter, the key parameter, and the value parameter from dimension to dimension.

[0018] As an embodiment, fusing the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature to obtain the pseudo-label of the unlabeled multi-spectral remote sensing image includes:

[0019] Fusing the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature based on the entropy weighting algorithm to obtain the pseudo-label of the unlabeled multi-spectral remote sensing image.

[0020] As an embodiment, fusing the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature based on the entropy weighting algorithm to obtain the pseudo-label of the unlabeled multi-spectral remote sensing image includes:

[0021] Determine the entropy of the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature respectively, and determine the confidence of the spatial feature branch network and the spectral feature branch network respectively according to the entropy;

[0022] According to the confidence of the spatial feature branch network and the spectral feature branch network respectively, determine the weights of the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature respectively;

[0023] According to the weights, perform weighted fusion on the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature to obtain the pseudo-label of the unlabeled multi-spectral remote sensing image.

[0024] As an embodiment, obtaining an unsupervised loss function according to the pseudo-label and the strongly enhanced prediction results corresponding to the spatial feature and the spectral feature respectively includes:

[0025] Substitute the pseudo-label and the strongly enhanced prediction results corresponding to the spatial feature and the spectral feature respectively into the cross-entropy loss function to obtain the unsupervised loss function.

[0026] As an embodiment, it further includes:

[0027] Obtain a labeled multi-spectral remote sensing image;

[0028] Input the labeled multi-spectral remote sensing image into the dual-branch network structure to obtain a prediction result corresponding to the labeled multi-spectral remote sensing image;

[0029] According to the label corresponding to the labeled multi-spectral remote sensing image and the prediction result, obtain a supervised loss function;

[0030] Take the sum of the unsupervised loss function and the supervised loss function as the total loss function.

[0031] The present invention also provides a semi-supervised multi-spectral remote sensing image scene classification device, including:

[0032] An acquisition module, configured to acquire unlabeled multi-spectral remote sensing images, and perform weak enhancement processing and strong enhancement processing on the unlabeled multi-spectral remote sensing images respectively to obtain weakly enhanced remote sensing images and strongly enhanced remote sensing images;

[0033] A prediction module, configured to input the weakly enhanced remote sensing image and the strongly enhanced remote sensing image into a dual-branch network structure pre-constructed by a spatial feature branch network and a spectral feature branch network respectively. The spatial feature branch network is configured to extract the respective spatial features of the weakly enhanced remote sensing image and the strongly enhanced remote sensing image, and obtain strongly enhanced prediction results and weakly enhanced prediction results corresponding to the spatial features according to the spatial features; the spectral feature branch network is configured to extract the respective spectral features of the weakly enhanced remote sensing image and the strongly enhanced remote sensing image based on spectral attention, and obtain strongly enhanced prediction results and weakly enhanced prediction results corresponding to the spectral features according to the spectral features;

[0034] A determination module, configured to fuse the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature to obtain a pseudo-label of the unlabeled multi-spectral remote sensing image, and obtain an unsupervised loss function according to the pseudo-label and the strongly enhanced prediction results corresponding to the spatial feature and the spectral feature respectively. The unsupervised loss function is used to determine the total loss function;

[0035] A training module, configured to determine whether the dual-branch network structure is trained according to the total loss function. If so, use the dual-branch network structure as a scene classification model. If not, adjust the parameters of the dual-branch network structure, and return to execute the step of acquiring unlabeled multi-spectral remote sensing images until the training is completed;

[0036] A classification module, configured to perform scene classification on a target multi-spectral remote sensing image based on the scene classification model to obtain a target scene classification result.

[0037] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the semi-supervised multi-spectral remote sensing image scene classification method as described in any one of the above is implemented.

[0038] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the semi-supervised multi-spectral remote sensing image scene classification method as described in any one of the above is implemented.

[0039] For the semi-supervised multi-spectral remote sensing image scene classification method, device, equipment, and medium provided by the present invention, the dual-branch network structure fully utilizes the spatial and spectral information of the multi-spectral remote sensing image and the complementary information between different bands. The spectral feature branch network introducing spectral attention can further enhance the spectral information extraction ability, obtain more accurate pseudo-labels, and thus improve the classification performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0041] Figure 1 FIG. 1 is one of the schematic flowcharts of the semi-supervised multi-spectral remote sensing image scene classification method provided by the present invention.

[0042] Figure 2 FIG. 2 is the schematic structural diagram of the spectral attention module provided by the present invention.

[0043] Figure 3 FIG. 3 is another schematic flowchart of the semi-supervised multi-spectral remote sensing image scene classification method provided by the present invention.

[0044] Figure 4 FIG. 4 is the schematic structural diagram of the semi-supervised multi-spectral remote sensing image scene classification device provided by the present invention.

[0045] Figure 5 FIG. 5 is the schematic structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0047] Figure 1 is one of the schematic flowcharts of the semi-supervised multi-spectral remote sensing image scene classification method provided by the present invention. As Figure 1 shown, the present invention provides a semi-supervised multi-spectral remote sensing image scene classification method, including steps S100 - S500. Among them, steps S100 - S400 are the model training stage, and step S500 is the model application stage.

[0048] Step S100: Obtain unlabeled multi-spectral remote sensing images, and perform weak enhancement processing and strong enhancement processing on the unlabeled multi-spectral remote sensing images respectively to obtain weakly enhanced remote sensing images and strongly enhanced remote sensing images.

[0049] Step S200: Input the weakly enhanced remote sensing image and the strongly enhanced remote sensing image into a dual-branch network structure pre-constructed by a spatial feature branch network and a spectral feature branch network respectively. The spatial feature branch network is used to extract the respective spatial features of the weakly enhanced remote sensing image and the strongly enhanced remote sensing image, and based on the spatial features, obtain strongly enhanced prediction results and weakly enhanced prediction results corresponding to the spatial features; the spectral feature branch network is used to extract the respective spectral features of the weakly enhanced remote sensing image and the strongly enhanced remote sensing image based on spectral attention, and based on the spectral features, obtain strongly enhanced prediction results and weakly enhanced prediction results corresponding to the spectral features.

[0050] Step S300: Fuse the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature to obtain a pseudo-label of the unlabeled multi-spectral remote sensing image, and based on the pseudo-label and the strongly enhanced prediction results corresponding to the spatial feature and the spectral feature respectively, obtain an unsupervised loss function, and the unsupervised loss function is used to determine the total loss function.

[0051] Step S400: According to the total loss function, determine whether the dual-branch network structure is completed training. If so, use the dual-branch network structure as the scene classification model. If not, adjust the parameters of the dual-branch network structure, and return to execute the step of obtaining unlabeled multi-spectral remote sensing images until the training is completed.

[0052] Step S500: Perform scene classification on the target multispectral remote sensing image based on the scene classification model to obtain the target scene classification result.

[0053] In the model training stage, labeled multispectral remote sensing images and unlabeled multispectral remote sensing images can be obtained as the dataset. The dataset is divided into a training set, a test set, and a validation set according to a certain ratio. The pre-constructed dual-branch network structure is trained, tested, and validated using the training set, the test set, and the validation set respectively to obtain the scene classification model.

[0054] The labeled multispectral remote sensing images can be directly input into the dual-branch network structure for feature extraction and result prediction. The unlabeled multispectral remote sensing images need to be subjected to weak enhancement processing and strong enhancement processing respectively. Weak enhancement processing usually refers to making small adjustments to the image to improve its visual quality without significantly changing the content or structure of the image. Common weak enhancement methods include: brightness and contrast adjustment, filtering, and color correction, etc. Strong enhancement processing is to perform significant transformations and alterations on the image, which is often used to highlight certain features or information and may change the overall structure or content of the image. Common methods include: geometric transformation, feature extraction, image synthesis, etc.

[0055] The present invention uses the two branches of the dual-branch network structure to perform feature extraction and prediction on the weakly enhanced remote sensing image and the strongly enhanced remote sensing image respectively, and then uses the weakly enhanced prediction results of the two branches to obtain pseudo-labels, realizing the complementarity of spatial features and spectral features. At the same time, the spectral feature branch network performs feature extraction based on spectral attention, enhancing the extraction of spectral information and fully mining the deep features of the multispectral remote sensing image, further improving the accuracy of the pseudo-labels of the unlabeled multispectral remote sensing images.

[0056] Correspondingly, the accuracy of the unsupervised loss function obtained from the higher-quality pseudo-labels with higher accuracy is also higher. Based on the dataset consisting of labeled multispectral remote sensing images and unlabeled multispectral remote sensing images, the loss function of the model includes a supervised loss function and an unsupervised loss function, which can more accurately evaluate the gap between the model prediction value and the true value, and determine whether the prediction error of the dual-branch network structure reaches the preset value. If so, it can be determined that the dual-branch network structure has completed training. Otherwise, it is judged whether the preset number of training iterations is reached. If so, it is determined that the dual-branch network structure has completed training. Otherwise, adjust the parameters of the dual-branch network structure, randomly select new samples from the training set, and repeat steps S100 - S400 to realize iterative training of the dual-branch network structure.

[0057] After completing the model training, in response to a user request, obtain the target multispectral remote sensing image to be classified input by the user, input the target multispectral remote sensing image into the scene classification model, and the two branches of the scene classification model respectively extract the spatial features and spectral features of the target multispectral remote sensing image and make predictions to obtain the target scene classification result.

[0058] It can be understood that the dual-branch network structure of the present invention makes full use of the complementary information between different bands and fully excavates the spectral information in the multispectral image. The spectral feature branch network introduces spectral attention to be able to excavate the spectral information of the multispectral remote sensing image, obtain more accurate pseudo-labels, and thus improve the classification performance of the model.

[0059] Based on the above embodiments, as an optional embodiment, the spatial feature branch network is obtained based on the deep residual network, and the spectral feature branch network is obtained by adding a spectral attention module to each network layer of the deep residual network.

[0060] The spectral attention module is used to determine the query parameter, key parameter, and value parameter of the input image feature map based on a linear layer, perform dimensional transformation on the query parameter, the key parameter, and the value parameter, transmit the query parameter and the key parameter after dimensional transformation to the normalization layer, determine the attention score between different spectral bands of the image feature map based on the normalization layer, perform an operation on the attention score and the value parameter after dimensional transformation and then transmit it to the output linear layer, obtain the output image feature map based on the output linear layer, and perform dimensional transformation on the output image feature map and then output it to the next network layer.

[0061] To ensure the complementarity of the spatial feature branch network and the spectral feature branch network, the spatial feature branch network and the spectral feature branch network are obtained based on the same feature extraction network. The difference is that to enhance the feature extraction ability of the spectral feature branch network to excavate deep spectral information, the spectral attention module is introduced into the feature extraction network. Specifically, in the embodiment of the present invention, the original deep residual network ResNet50 is used as the spatial feature branch network, and the spectral attention module is introduced into each network layer of the deep residual network ResNet50, and the modified ResNet50 is used as the spectral feature branch network, further enhancing its professionalism in capturing spectral features.

[0062] As Figure 2 shown, as an optional embodiment, the calculation formulas of the query parameter, the key parameter, and the value parameter are as follows:

[0063]

[0064] Among them, represents the query parameter, represents the key parameter, represents the value parameter, , , respectively represent the parameter matrices of three linear layers, represents the input image feature map, , R represents the real number field, , , respectively represent the number of channels, width, and height of the input image feature map. That is, for a given image feature map , R represents the real number field, first passes through three linear layers , and to obtain, , , represent the parameter matrices corresponding to obtaining Q, K, and V.

[0065] Correspondingly, dimensionality transformation is performed on the query parameter, the key parameter, and the value parameter, including: respectively transforming the query parameter, the key parameter, and the value parameter from dimension to dimension. By performing dimensionality transformation on and , the relationship between different bands can be modeled.

[0066] The calculation formula for the attention score is as follows:

[0067]

[0068] where, represents the attention score between different bands, represents the query parameter after dimensionality transformation, represents the key parameter after dimensionality transformation, and Softmax represents the Softmax function.

[0069] The calculation formula for the output image feature map is as follows:

[0070]

[0071] where, represents the output image feature map, represents the parameter matrix of the output linear layer, represents the value parameter after dimensionality transformation. The output linear layer is used to perform a linear transformation on the attention score.

[0072] To make the output image feature map It can be continuously input into the next network layer, and its dimension needs to be adjusted back to .

[0073] It can be understood that in order to effectively process multi-spectral images and make full use of unlabeled data, the present invention adopts a dual-branch network structure, which is respectively used to extract spatial and spectral features. The spectral attention module is used in the channel dimension, which can fully mine the spectral information in the unlabeled data and realize the full utilization of the unlabeled data.

[0074] Such as Figure 3 shown, on the basis of the above embodiment, as an optional embodiment, obtaining the pseudo-label of the unannotated multi-spectral remote sensing image according to the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature includes:

[0075] Fusing the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature based on the entropy weighting algorithm to obtain the pseudo-label of the unannotated multi-spectral remote sensing image.

[0076] In order to improve the quality of the pseudo-label, the present invention adopts an entropy weighting fusion method to fuse the respective prediction results of the two branches on the weakly enhanced image, and retains the prediction result corresponding to the largest category in the fusion result whose score is greater than the threshold as the final pseudo-label.

[0077] As an optional embodiment, fusing the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature based on the entropy weighting algorithm to obtain the pseudo-label of the unannotated multi-spectral remote sensing image includes steps S310 - step S330.

[0078] Step S310, determine the entropy of the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature respectively, and determine the confidence of the spatial feature branch network and the spectral feature branch network respectively according to the entropy.

[0079] The confidence of each of the two branches of the spatial feature branch network and the spectral feature branch network is determined by the entropy of its prediction. The common calculation formula of entropy is as follows:

[0080]

[0081] Wherein, represents the entropy of a certain branch for the prediction result z, represents the probability that the prediction result z belongs to the category i , and are the network's predictions for the categoryi, j The output logits.

[0082] The confidence is the reciprocal of the entropy, and the common calculation formula for the confidence is as follows:

[0083]

[0084] Where, represents the confidence of a certain branch for the prediction result z.

[0085] Substitute the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature into the above formula to obtain the confidence of the spatial feature branch network and the confidence of the spectral feature branch network .

[0086] Step S320, determine the weights of the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature respectively according to the confidences of the spatial feature branch network and the spectral feature branch network.

[0087] The calculation formula for the weight of the weakly enhanced prediction result corresponding to the spatial feature is as follows:

[0088]

[0089] Where, represents the weight of the weakly enhanced prediction result corresponding to the spatial feature.

[0090] The calculation formula for the weight of the weakly enhanced prediction result corresponding to the spectral feature is as follows:

[0091]

[0092] Where, represents the weight of the weakly enhanced prediction result corresponding to the spectral feature.

[0093] Step S330, perform weighted fusion on the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature according to the weights to obtain the pseudo-label of the unlabeled multi-spectral remote sensing image.

[0094] The calculation formula for the pseudo-label is as follows:

[0095]

[0096] Where, represents the pseudo-label, which is a weighted combination of the prediction results of the two branches, and are the weakly enhanced prediction results obtained by the two branches respectively.

[0097] It can be understood that the present invention fuses the prediction results of the spatial feature branch network and the spectral feature branch network in an entropy-weighted manner to obtain high-quality pseudo-labels. The obtained pseudo-labels fuse spatial information and spectral information at the same time, with higher quality of pseudo-labels, and also fully exploit the spectral information in the unlabeled data, realizing the full utilization of unlabeled data.

[0098] Based on the above embodiments, as an optional embodiment, obtaining an unsupervised loss function according to the pseudo-labels and the strongly augmented prediction results corresponding to the spatial features and the spectral features respectively includes:

[0099] Substitute the pseudo-labels and the strongly augmented prediction results corresponding to the spatial features and the spectral features respectively into the cross-entropy loss function to obtain an unsupervised loss function.

[0100] The unsupervised loss function is obtained based on the pseudo-labels and the prediction results of the strongly augmented remote sensing images by the dual-branch network structure. The calculation formula of the unsupervised loss function is as follows:

[0101]

[0102]

[0103]

[0104] Among them, represents the unsupervised loss function corresponding to the spatial feature branch network, represents the unsupervised loss function corresponding to the spectral feature branch network, represents the batch size, represents the prediction result of the strongly augmented remote sensing images by the spatial feature branch network, represents the prediction result of the strongly augmented remote sensing images by the spectral feature branch network, represents the unsupervised loss function, represents cross-entropy, represents the unlabeled multi-spectral remote sensing images for strong augmentation processing.

[0105] It can be understood that the present invention uses pseudo-labels to realize the training supervision of strongly augmented remote sensing images, realizes the complementarity between spatial features and spectral features, and is beneficial to improving the classification performance.

[0106] Based on the above embodiments, as an optional embodiment, the semi-supervised multi-spectral remote sensing image scene classification method provided by the present invention further includes the following steps.

[0107] Obtain the labeled multi-spectral remote sensing image. This step can be executed synchronously with step S100.

[0108] Input the labeled multi-spectral remote sensing image into the dual-branch network structure to obtain a prediction result corresponding to the labeled multi-spectral remote sensing image. This step can be executed synchronously with step S200.

[0109] According to the label corresponding to the labeled multi-spectral remote sensing image and the prediction result, obtain the supervised loss function. This step can be executed synchronously with step S300.

[0110] Take the sum of the unsupervised loss function and the supervised loss function as the total loss function.

[0111] The supervised loss function is calculated based on the predictions of the two branches for the weakly augmented image, and it can be expressed by the following formula:

[0112]

[0113]

[0114]

[0115] where represents the true label corresponding to the input , represents the supervised loss function corresponding to the spatial feature branch network, represents the supervised loss function corresponding to the spectral feature branch network, represents the batch size, represents the prediction result of the spatial feature branch network for the weakly augmented remote sensing image, represents the prediction result of the spectral feature branch network for the weakly augmented remote sensing image, represents the supervised loss function, represents the weak augmentation processing of the unlabeled multi-spectral remote sensing image .

[0116] The total loss function is calculated as follows:

[0117]

[0118] where is a hyperparameter used to balance the proportion of supervised and unsupervised losses.

[0119] Next, the hardware and software environments for the experiment, the datasets used in the experiment, the experimental settings, and the experimental evaluation metrics are introduced in detail, and the experimental results are compared with the results of previous methods.

[0120] (1) Experimental environment.

[0121] The detailed information of the environmental configuration is shown in Table 1.

[0122] Table 1 Experimental environmental configuration

[0123]

[0124] (2) Experimental datasets.

[0125] The method of the present invention is verified on the multi - spectral remote sensing image datasets EuroSAT and SEN12MS.

[0126] (3) Experimental settings.

[0127] The present invention conducts six experimental settings on each dataset, which are using only 5 labeled data, 10 labeled data, 50 labeled data, 100 labeled data, 200 labeled data, and 300 labeled data for each category. During the training process, the learning rate, optimizer, , and batch size are set to 1×10 -4 , Adam, 0.9, 0.999, and 4 respectively. The pseudo - label threshold is set to 0.8. The total number of training steps is .

[0128] (4) Experimental results.

[0129] Table 2 Experimental results of EuroSAT dataset

[0130]

[0131] PseudoLabel, MixMatch, FixMatch, FlexMatch, FreeMatch, and MSMatch are all existing scene classification methods. As can be seen from Table 2, the present invention has the highest classification accuracy in the case of multiple numbers of labels. Only MSMatch is slightly higher than the present invention when the number of labels is 2000, but MSMatch migrates the FixMatch method to multi - spectral data, but its backbone network only uses the Efficient network and does not conduct specialized design and optimization for the multi - channel characteristics of multi - spectral images, and does not fully exploit the spectral information.

[0132] Table 3 Experimental results of SEN12MS

[0133]

[0134] As can be seen from Table 3, regardless of the number of labels, the accuracy of the present invention is the highest.

[0135] In summary, the present invention designs a dual-branch network structure for the characteristics of multi-spectral images, which is used to extract spatial information and spectral information in multi-spectral images respectively. Among them, the spatial feature extraction branch uses the ResNet50 network, and the spectral feature extraction branch enhances the extraction of spectral information by introducing a spectral attention module into ResNet50. In addition, before the image is input, two enhancements of weak and strong are performed, and the weak enhancement prediction results are fused to obtain high-quality pseudo-labels, which supervise the prediction of the strongly enhanced image by the network, so as to fully exploit the information of unlabeled images. Existing methods usually use only one network branch to obtain pseudo-labels, while the present invention fuses the prediction results of the spatial and spectral branches based on entropy weighting to obtain high-quality pseudo-labels. The obtained pseudo-labels fuse both spatial information and spectral information, with higher quality than pseudo-labels of other methods, and also fully exploit the spectral information in unlabeled data, realizing the full utilization of unlabeled data. Therefore, the present invention provides a new semi-supervised multi-spectral remote sensing image scene classification method. In addition, the attention mechanism adopted by the present invention is also different from most methods. The present invention uses it in the channel dimension, which is different from the implementation methods of existing attention mechanisms.

[0136] The following describes the semi-supervised multi-spectral remote sensing image scene classification device provided by the present invention. The semi-supervised multi-spectral remote sensing image scene classification device described below can be mutually referred to the semi-supervised multi-spectral remote sensing image scene classification method described above.

[0137] Figure 4 is a schematic structural diagram of the semi-supervised multi-spectral remote sensing image scene classification device provided by the present invention. As Figure 4 shown, the present invention also provides a semi-supervised multi-spectral remote sensing image scene classification device, including the following modules.

[0138] An acquisition module 410, configured to acquire unlabeled multi-spectral remote sensing images, and perform weak enhancement processing and strong enhancement processing on the unlabeled multi-spectral remote sensing images respectively to obtain weakly enhanced remote sensing images and strongly enhanced remote sensing images;

[0139] A prediction module 420, configured to input the weakly enhanced remote sensing image and the strongly enhanced remote sensing image into a dual-branch network structure pre-constructed by a spatial feature branch network and a spectral feature branch network respectively. The spatial feature branch network is configured to extract the respective spatial features of the weakly enhanced remote sensing image and the strongly enhanced remote sensing image, and obtain strong enhancement prediction results and weak enhancement prediction results corresponding to the spatial features according to the spatial features; the spectral feature branch network is configured to extract the respective spectral features of the weakly enhanced remote sensing image and the strongly enhanced remote sensing image based on spectral attention, and obtain strong enhancement prediction results and weak enhancement prediction results corresponding to the spectral features according to the spectral features;

[0140] A determination module 430 is configured to fuse the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature to obtain a pseudo-label of the unannotated multispectral remote sensing image, and obtain an unsupervised loss function according to the pseudo-label and the strongly enhanced prediction results corresponding to the spatial feature and the spectral feature respectively, where the unsupervised loss function is used to determine a total loss function;

[0141] A training module 440 is configured to determine whether the double-branch network structure is trained according to the total loss function. If so, the double-branch network structure is used as a scene classification model. If not, the parameters of the double-branch network structure are adjusted, and the step of obtaining the unannotated multispectral remote sensing image is returned to be executed until the training is completed;

[0142] A classification module 450 is configured to perform scene classification on a target multispectral remote sensing image based on the scene classification model to obtain a target scene classification result.

[0143] As an embodiment, the spatial feature branch network is obtained based on a deep residual network, and the spectral feature branch network is obtained by adding a spectral attention module to each network layer of the deep residual network;

[0144] The spectral attention module is configured to determine query parameters, key parameters, and value parameters of an input image feature map based on a linear layer, perform dimensionality transformation on the query parameters, the key parameters, and the value parameters, transmit the query parameters and the key parameters after dimensionality transformation to a normalization layer, determine attention scores between different spectral bands of the image feature map based on the normalization layer, perform an operation on the attention scores and the value parameters after dimensionality transformation and then transmit them to an output linear layer, obtain an output image feature map based on the output linear layer, and perform dimensionality transformation on the output image feature map and then output it to the next network layer.

[0145] As an embodiment, the calculation formulas of the query parameters, the key parameters, and the value parameters are as follows:

[0146]

[0147] Where represents the query parameter, represents the key parameter, represents the value parameter, 、 、 respectively represent parameter matrices of three linear layers, represents the input image feature map, , R represents the real number field, 、 、 respectively represent the number of channels, width, and height of the input image feature map;

[0148] Correspondingly, performing dimensionality transformation on the query parameter, the key parameter, and the value parameter includes:

[0149] respectively transforming the query parameter, the key parameter, and the value parameter from dimension to dimension.

[0150] As an embodiment, the determining module 430 is further configured to:

[0151] Based on the entropy weighting algorithm, fusing the weakly enhanced prediction results corresponding to the spatial features and the weakly enhanced prediction results corresponding to the spectral features to obtain the pseudo-label of the unlabeled multi-spectral remote sensing image.

[0152] As an embodiment, the determining module 430 is further configured to:

[0153] Determine the entropy of the weakly enhanced prediction results corresponding to the spatial features and the weakly enhanced prediction results corresponding to the spectral features respectively, and determine the confidence of the spatial feature branch network and the spectral feature branch network respectively according to the entropy;

[0154] According to the confidence of the spatial feature branch network and the spectral feature branch network respectively, determine the weights of the weakly enhanced prediction results corresponding to the spatial features and the weakly enhanced prediction results corresponding to the spectral features respectively;

[0155] According to the weights, perform weighted fusion on the weakly enhanced prediction results corresponding to the spatial features and the weakly enhanced prediction results corresponding to the spectral features to obtain the pseudo-label of the unlabeled multi-spectral remote sensing image.

[0156] As an embodiment, the determining module 430 is further configured to:

[0157] Substitute the pseudo-label, the strongly enhanced prediction results corresponding to the spatial features and the spectral features respectively into the cross-entropy loss function to obtain an unsupervised loss function.

[0158] As an embodiment, it further includes:

[0159] Obtain a labeled multi-spectral remote sensing image;

[0160] Input the labeled multi-spectral remote sensing image into the dual-branch network structure to obtain a prediction result corresponding to the labeled multi-spectral remote sensing image;

[0161] According to the label corresponding to the labeled multi-spectral remote sensing image and the prediction result, obtain a supervised loss function;

[0162] Take the sum of the unsupervised loss function and the supervised loss function as the total loss function. The semi-supervised multi-spectral remote sensing image scene classification device provided by the present invention is used to execute the semi-supervised multi-spectral remote sensing image scene classification method described in any of the above embodiments, and has technical effects corresponding to the semi-supervised multi-spectral remote sensing image scene classification method, which will not be elaborated here.

[0163] Figure 5 An example of the physical structure diagram of an electronic device is as Figure 5 shown. The electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540. Among them, the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call the logical instructions in the memory 530 to execute the semi-supervised multi-spectral remote sensing image scene classification method, and the method includes: obtaining unlabeled multi-spectral remote sensing images, respectively performing weak enhancement processing and strong enhancement processing on the unlabeled multi-spectral remote sensing images to obtain weakly enhanced remote sensing images and strongly enhanced remote sensing images; respectively inputting the weakly enhanced remote sensing images and the strongly enhanced remote sensing images into a dual-branch network structure pre-constructed by a spatial feature branch network and a spectral feature branch network. The spatial feature branch network is used to respectively extract the spatial features of the weakly enhanced remote sensing images and the strongly enhanced remote sensing images, and based on the spatial features, obtain strongly enhanced prediction results and weakly enhanced prediction results corresponding to the spatial features; the spectral feature branch network is used to respectively extract the spectral features of the weakly enhanced remote sensing images and the strongly enhanced remote sensing images based on spectral attention, and based on the spectral features, obtain strongly enhanced prediction results and weakly enhanced prediction results corresponding to the spectral features; fuse the weakly enhanced prediction results corresponding to the spatial features and the weakly enhanced prediction results corresponding to the spectral features to obtain pseudo-labels of the unlabeled multi-spectral remote sensing images, and based on the pseudo-labels and the strongly enhanced prediction results corresponding to the spatial features and the spectral features respectively, obtain an unsupervised loss function, and the unsupervised loss function is used to determine the total loss function; according to the total loss function, determine whether the dual-branch network structure has completed training. If so, use the dual-branch network structure as the scene classification model. If not, adjust the parameters of the dual-branch network structure, and return to execute the step of obtaining unlabeled multi-spectral remote sensing images until the training is completed; perform scene classification on the target multi-spectral remote sensing image based on the scene classification model to obtain the target scene classification result.

[0164] In addition, when the logical instructions in the above-mentioned memory 530 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0165] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the semi-supervised multi-spectral remote sensing image scene classification method provided by the above-mentioned various methods. The method includes: acquiring unlabeled multi-spectral remote sensing images, respectively performing weak enhancement processing and strong enhancement processing on the unlabeled multi-spectral remote sensing images to obtain weakly enhanced remote sensing images and strongly enhanced remote sensing images; respectively inputting the weakly enhanced remote sensing images and the strongly enhanced remote sensing images into a dual-branch network structure pre-constructed by a spatial feature branch network and a spectral feature branch network. The spatial feature branch network is used to respectively extract the spatial features of the weakly enhanced remote sensing images and the strongly enhanced remote sensing images, and based on the spatial features, obtain strongly enhanced prediction results and weakly enhanced prediction results corresponding to the spatial features; the spectral feature branch network is used to respectively extract the spectral features of the weakly enhanced remote sensing images and the strongly enhanced remote sensing images based on spectral attention, and based on the spectral features, obtain strongly enhanced prediction results and weakly enhanced prediction results corresponding to the spectral features; fusing the weakly enhanced prediction results corresponding to the spatial features and the weakly enhanced prediction results corresponding to the spectral features to obtain pseudo-labels of the unlabeled multi-spectral remote sensing images, and based on the pseudo-labels and the strongly enhanced prediction results corresponding to the spatial features and the spectral features respectively, obtain an unsupervised loss function, and the unsupervised loss function is used to determine a total loss function; according to the total loss function, determine whether the dual-branch network structure has completed training. If so, use the dual-branch network structure as a scene classification model. If not, adjust the parameters of the dual-branch network structure, and return to execute the step of acquiring unlabeled multi-spectral remote sensing images until the training is completed; perform scene classification on the target multi-spectral remote sensing image based on the scene classification model to obtain a target scene classification result.

[0166] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the semi-supervised multi-spectral remote sensing image scene classification method provided by the above-mentioned various methods. The method includes: obtaining an unlabeled multi-spectral remote sensing image, respectively performing weak enhancement processing and strong enhancement processing on the unlabeled multi-spectral remote sensing image to obtain a weakly enhanced remote sensing image and a strongly enhanced remote sensing image; respectively inputting the weakly enhanced remote sensing image and the strongly enhanced remote sensing image into a dual-branch network structure pre-constructed by a spatial feature branch network and a spectral feature branch network. The spatial feature branch network is used to respectively extract the spatial features of the weakly enhanced remote sensing image and the strongly enhanced remote sensing image, and based on the spatial features, obtain strongly enhanced prediction results and weakly enhanced prediction results corresponding to the spatial features; the spectral feature branch network is used to respectively extract the spectral features of the weakly enhanced remote sensing image and the strongly enhanced remote sensing image based on spectral attention, and based on the spectral features, obtain strongly enhanced prediction results and weakly enhanced prediction results corresponding to the spectral features; fusing the weakly enhanced prediction result corresponding to the spatial features and the weakly enhanced prediction result corresponding to the spectral features to obtain a pseudo-label of the unlabeled multi-spectral remote sensing image, and based on the pseudo-label and the strongly enhanced prediction results corresponding to the spatial features and the spectral features respectively, obtain an unsupervised loss function, and the unsupervised loss function is used to determine a total loss function; according to the total loss function, determine whether the dual-branch network structure is completed training. If so, use the dual-branch network structure as a scene classification model. If not, adjust the parameters of the dual-branch network structure, and return to execute the step of obtaining an unlabeled multi-spectral remote sensing image until the training is completed; perform scene classification on a target multi-spectral remote sensing image based on the scene classification model to obtain a target scene classification result.

[0167] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0168] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semi-supervised multi-spectral remote sensing image scene classification method, characterized in that, Including: Obtain an unlabeled multispectral remote sensing image, perform weak enhancement processing and strong enhancement processing on the unlabeled multispectral remote sensing image respectively to obtain a weakly enhanced remote sensing image and a strongly enhanced remote sensing image; Input the weakly enhanced remote sensing image and the strongly enhanced remote sensing image into a dual-branch network structure pre-constructed by a spatial feature branch network and a spectral feature branch network respectively. The spatial feature branch network is used to extract the respective spatial features of the weakly enhanced remote sensing image and the strongly enhanced remote sensing image, and based on the spatial features, obtain strongly enhanced prediction results and weakly enhanced prediction results corresponding to the spatial features; the spectral feature branch network is used to extract the respective spectral features of the weakly enhanced remote sensing image and the strongly enhanced remote sensing image based on spectral attention, and based on the spectral features, obtain strongly enhanced prediction results and weakly enhanced prediction results corresponding to the spectral features; Fuse the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature to obtain a pseudo-label of the unlabeled multispectral remote sensing image, and based on the pseudo-label and the strongly enhanced prediction results corresponding to the spatial feature and the spectral feature respectively, obtain an unsupervised loss function, and the unsupervised loss function is used to determine a total loss function; According to the total loss function, determine whether the dual-branch network structure is trained. If so, use the dual-branch network structure as a scene classification model. If not, adjust the parameters of the dual-branch network structure, and return to execute the step of obtaining the unlabeled multispectral remote sensing image until the training is completed; Perform scene classification on a target multispectral remote sensing image based on the scene classification model to obtain a target scene classification result; The spatial feature branch network is obtained based on a deep residual network, and the spectral feature branch network is obtained by adding a spectral attention module to each network layer of the deep residual network; The spectral attention module is used to determine query parameters, key parameters, and value parameters of an input image feature map based on a linear layer, perform dimensionality transformation on the query parameters, the key parameters, and the value parameters, transmit the query parameters and the key parameters after dimensionality transformation to a normalization layer, determine attention scores between different spectral bands of the image feature map based on the normalization layer, perform an operation on the attention scores and the value parameters after dimensionality transformation and then transmit them to an output linear layer, obtain an output image feature map based on the output linear layer, perform dimensionality transformation on the output image feature map and then output it to the next network layer.

2. The semi-supervised multi-spectral remote sensing image scene classification method according to claim 1, wherein The calculation formulas of the query parameters, the key parameters, and the value parameters are as follows: ； Among them, represents the query parameter, represents the key parameter, represents the value parameter, 、 、 respectively represent the parameter matrices of three linear layers, represents the input image feature map, , R represents the real number field, 、 、 respectively represent the number of channels, width, and height of the input image feature map; Correspondingly, performing dimensionality transformation on the query parameters, the key parameters, and the value parameters includes: Convert the query parameter, the key parameter, and the value parameter respectively from dimension to dimension.

3. The semi-supervised multi-spectral remote sensing image scene classification method according to any one of claims 1-2, characterized in that, The fusing the weakly enhanced prediction result corresponding to the spatial feature and the weakly enhanced prediction result corresponding to the spectral feature to obtain the pseudo-label of the unlabeled multispectral remote sensing image includes: Fuse the weakly augmented prediction results corresponding to the spatial features and the weakly augmented prediction results corresponding to the spectral features based on the entropy weighting algorithm to obtain the pseudo-labels of the unlabeled multispectral remote sensing images.

4. The semi-supervised multi-spectral remote sensing image scene classification method according to claim 3, wherein The fusing of the weakly augmented prediction results corresponding to the spatial features and the weakly augmented prediction results corresponding to the spectral features based on the entropy weighting algorithm to obtain the pseudo-labels of the unlabeled multispectral remote sensing images includes: Determine the entropy of the weakly augmented prediction results corresponding to the spatial features and the weakly augmented prediction results corresponding to the spectral features respectively, and determine the confidence of the spatial feature branch network and the spectral feature branch network respectively according to the entropy; Determine the weights of the weakly augmented prediction results corresponding to the spatial features and the weakly augmented prediction results corresponding to the spectral features respectively according to the confidence of the spatial feature branch network and the spectral feature branch network respectively; Perform weighted fusion on the weakly augmented prediction results corresponding to the spatial features and the weakly augmented prediction results corresponding to the spectral features according to the weights to obtain the pseudo-labels of the unlabeled multispectral remote sensing images.

5. The semi-supervised multi-spectral remote sensing image scene classification method according to claim 1, characterized in that The obtaining of the unsupervised loss function according to the pseudo-labels and the strongly augmented prediction results corresponding to the spatial features and the spectral features respectively includes: Substitute the pseudo-labels and the strongly augmented prediction results corresponding to the spatial features and the spectral features respectively into the cross-entropy loss function to obtain the unsupervised loss function.

6. The semi-supervised multi-spectral remote sensing image scene classification method according to claim 1, wherein, It further includes: Obtain labeled multispectral remote sensing images; Input the labeled multispectral remote sensing images into the dual-branch network structure to obtain the prediction results corresponding to the labeled multispectral remote sensing images; Obtain the supervised loss function according to the labels and the prediction results corresponding to the labeled multispectral remote sensing images; Take the sum of the unsupervised loss function and the supervised loss function as the total loss function.

7. A semi-supervised multi-spectral remote sensing image scene classification device, characterized in that, It includes: An acquisition module, configured to acquire unlabeled multispectral remote sensing images, and perform weak augmentation processing and strong augmentation processing on the unlabeled multispectral remote sensing images respectively to obtain weakly augmented remote sensing images and strongly augmented remote sensing images; A prediction module, configured to input the weakly augmented remote sensing images and the strongly augmented remote sensing images respectively into a dual-branch network structure pre-constructed by a spatial feature branch network and a spectral feature branch network. The spatial feature branch network is configured to extract the spatial features of the weakly augmented remote sensing images and the strongly augmented remote sensing images respectively, and obtain the strongly augmented prediction results and the weakly augmented prediction results corresponding to the spatial features according to the spatial features; The spectral feature branch network is configured to extract the spectral features of the weakly augmented remote sensing images and the strongly augmented remote sensing images respectively based on spectral attention, and obtain the strongly augmented prediction results and the weakly augmented prediction results corresponding to the spectral features according to the spectral features; A determination module is configured to fuse the weakly augmented prediction results corresponding to the spatial features and the weakly augmented prediction results corresponding to the spectral features to obtain the pseudo-labels of the unlabeled multispectral remote sensing image, and obtain an unsupervised loss function according to the pseudo-labels and the strongly augmented prediction results corresponding to the spatial features and the spectral features respectively, where the unsupervised loss function is used to determine the total loss function; A training module is configured to determine, according to the total loss function, whether the dual-branch network structure is completed training. If so, use the dual-branch network structure as the scene classification model. If not, adjust the parameters of the dual-branch network structure, and return to execute the step of obtaining the unlabeled multispectral remote sensing image until the training is completed; A classification module is configured to perform scene classification on the target multispectral remote sensing image based on the scene classification model to obtain the target scene classification result; The spatial feature branch network is obtained based on the deep residual network, and the spectral feature branch network is obtained by adding spectral attention modules to each network layer of the deep residual network; The spectral attention module is configured to determine the query parameter, key parameter, and value parameter of the input image feature map based on the linear layer, perform dimensionality transformation on the query parameter, the key parameter, and the value parameter, transmit the query parameter and the key parameter after dimensionality transformation to the normalization layer, determine the attention scores between different spectral bands of the image feature map based on the normalization layer, perform an operation on the attention scores and the value parameter after dimensionality transformation and then transmit it to the output linear layer, obtain the output image feature map based on the output linear layer, and perform dimensionality transformation on the output image feature map and then output it to the next network layer.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the semi-supervised multispectral remote sensing image scene classification method according to any one of claims 1-6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the semi-supervised multispectral remote sensing image scene classification method according to any one of claims 1-6.

Images