Remote sensing image scene classification method and device based on multi-stream self-distillation architecture

Abstract

The invention provides a remote sensing image scene classification method and device based on a multi-stream self-distillation architecture. The multi-stream self-distillation network architecture composed of a main stream and two branch streams is constructed. The main stream is constructed based on a CNN model, and the two branch streams are constructed based on a MobileNetV2 model. Attention modules are introduced into the two branch streams to enhance feature interaction with the main stream. In addition, a feature fusion strategy and a classification result integration strategy are introduced into the architecture to construct a feature fusion classifier and an integrated classifier. During architecture training, a self-distillation strategy is adopted, and an integrated classifier is used as a teacher model to guide training of each branch model, a trunk model and a feature fusion classifier. Experiments on four remote sensing image data sets show that the multi-stream self-distillation architecture can improve the classification precision on the basis of not changing the CNN network structure, and output multiple classifiers with different precision and resource consumption levels to adapt to multiple practical application scenes such as edge calculation and high-precision recognition.

Description

Technical Field

[0001] This disclosure relates to the field of image classification in computer vision, and more particularly to a method and apparatus for classifying remote sensing image scenes based on a multi-stream self-distillation architecture. Background Technology

[0002] The rapid development of remote sensing imaging equipment and technology has significantly improved the ability to acquire remote sensing images, providing data support for remote sensing image interpretation tasks. Therefore, how to extract valuable information from massive amounts of remote sensing images and achieve efficient and accurate interpretation has become a key problem that urgently needs to be solved in the field of remote sensing. Remote sensing image scene classification, as a fundamental task in remote sensing image interpretation, has important application value in fields such as urban planning, disaster assessment, and land use. This task classifies images into predefined scene categories based on the features of ground objects in the remote sensing images. However, the diversity of ground object categories and the imbalance of key target areas in remote sensing images make scene classification quite difficult. Therefore, extracting highly discriminative features from remote sensing images and accurately capturing the corresponding key semantic information of different scene categories is crucial to achieving high-precision scene classification. Currently, features used for scene classification in remote sensing images are mainly divided into two categories: handcrafted features and deep features. Early scene classification methods relied primarily on handcrafted features, such as Scale-Invariant Feature Transform (SIFT), Histogram of Oriented Gradient (HOG), and Local Binary Pattern (LBP). While these handcrafted features can capture the structural and textural information of images to some extent, their representational capabilities are limited, making it difficult to represent high-level semantic information in images, and the feature design process requires manual intervention. In contrast to handcrafted features, deep features possess the ability to express high-level semantic information and can be automatically acquired through the end-to-end training mechanism of deep learning models. Currently, the mainstream models for extracting deep features include Convolutional Neural Networks (CNNs) and ViT (Vision Transformer). Among them, ViT relies on a self-attention mechanism, which has advantages in long-distance dependency modeling and global semantic information acquisition, but it has a large number of parameters, high computational complexity, and significantly higher requirements for storage resources and computing power than traditional CNN architectures. In contrast, CNN models are easier to deploy, but their feature extraction capabilities are weaker, and the extracted features are more localized. Achieving even a slight improvement in accuracy often relies on more complex network structures. Furthermore, a single CNN model is insufficient to meet the diverse performance and resource consumption requirements of various application scenarios. Summary of the Invention

[0003] This disclosure provides at least one method and apparatus for classifying remote sensing image scenes based on a multi-stream self-distillation architecture, in order to solve at least one of the above-mentioned technical problems.

[0004] According to one aspect of this disclosure, a remote sensing image scene classification method based on a multi-stream self-distillation architecture is provided, comprising the following steps: S1. Construct a multi-stream remote sensing image scene classification model; the multi-stream remote sensing image scene classification model includes a multi-stream self-distillation architecture; the multi-stream self-distillation architecture includes a main stream, two branch streams, a feature fusion classifier, and an ensemble classifier; S2. Self-distillation training of the multi-stream remote sensing image scene classification model based on the joint loss function; S3. Classify remote sensing images using a trained multi-stream remote sensing image scene classification model.

[0005] According to another aspect of this disclosure, a remote sensing image scene classification device based on a multi-stream self-distillation architecture is provided, comprising the following steps: The model building module is used to construct a multi-stream remote sensing image scene classification model; the multi-stream remote sensing image scene classification model includes a multi-stream self-distillation architecture; the multi-stream self-distillation architecture includes a main stream, two branch streams, a feature fusion classifier, and an ensemble classifier; The training module is used for self-distillation training of the multi-stream remote sensing image scene classification model based on the joint loss function; The classification module is used to classify remote sensing images using a trained multi-stream remote sensing image scene classification model.

[0006] This disclosure presents a remote sensing image scene classification method and apparatus based on a multi-stream self-distillation architecture. The architecture utilizes a CNN model as the backbone classifier and two lightweight MobileNetV2 models as branch classifiers. Feature fusion and classification result ensemble strategies are introduced to construct the feature fusion classifier and ensemble classifier, respectively. During training, joint optimization among the classifiers is achieved through self-distillation. After training, the architecture can output five classifiers with different accuracy levels and deployment costs to meet diverse deployment requirements in real-world applications. Furthermore, this disclosure ensures improved classification accuracy without altering the CNN network structure.

[0007] To address the issue of limited model structure and lack of diversity in remote sensing image scene classification applications, a multi-stream self-distillation architecture is proposed. This architecture, based on existing models (CNN and MobileNetV2), additionally constructs a feature fusion classifier and an ensemble classifier, providing diverse model choices for application scenarios under different computational resource constraints.

[0008] To address the performance bottleneck caused by the rigid structure of the backbone CNN model, a self-distillation training strategy is adopted. This strategy is introduced into a multi-stream self-distillation architecture, using the ensemble classifier as the teacher model and the other classifiers as student models for self-distillation training. The proposed strategy can overcome the performance limitations of CNN models without structural expansion.

[0009] To address the lack of information exchange between CNN and MobileNetV2 models within a multistream self-distillation architecture, an attention module was designed as a connection between the models. This module generates feature weights using CNN feature maps and provides them to the MobileNetV2 model, thereby enabling cross-model feature information transfer.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is a multi-stream remote sensing image scene classification model, i.e., a multi-stream self-distillation architecture diagram; Figure 2 This is a schematic diagram of the attention module structure; Figure 3 This is a schematic diagram of the feature alignment layer map. Detailed Implementation

[0012] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0013] The technical solution of this disclosure will be described below through specific embodiments.

[0014] The execution entity of the remote sensing image scene classification method with a multi-stream self-distillation architecture in this embodiment is a computing device or component with data processing capabilities. Specifically, the method in this embodiment may include the following steps: S1. Construct a multi-stream remote sensing image scene classification model; the multi-stream remote sensing image scene classification model includes a multi-stream self-distillation architecture; the multi-stream self-distillation architecture includes a main stream, two branch streams, a feature fusion classifier, and an ensemble classifier.

[0015] S2. Self-distillation training of the multi-stream remote sensing image scene classification model based on the joint loss function.

[0016] S3. Classify remote sensing images using a trained multi-stream remote sensing image scene classification model.

[0017] Based on the same inventive concept, this disclosure provides a remote sensing image scene classification device based on a multi-stream self-distillation architecture. The steps performed by the components of this device are the same as or similar to the methods described above, therefore, similar details will not be repeated. The remote sensing image scene classification device based on a multi-stream self-distillation architecture in this embodiment includes: The model building module is used to construct a multi-stream remote sensing image scene classification model. This model includes a multi-stream self-distillation architecture, comprising a main stream, two branch streams, a feature fusion classifier, and an ensemble classifier. The training module is used to self-distill the multi-stream remote sensing image scene classification model based on a joint loss function. The classification module is used to classify remote sensing images using the trained multi-stream remote sensing image scene classification model.

[0018] The solution disclosed herein will be described below through specific embodiments.

[0019] like Figures 1-3 As shown, the remote sensing image scene classification scheme based on a multi-stream self-distillation architecture disclosed in this paper includes the following: S1 multi-stream remote sensing image scene classification framework, i.e., multi-stream remote sensing image scene classification model: Overall Framework Construction. This embodiment proposes a multi-stream self-distillation architecture for remote sensing image scene classification. The architecture comprises a main stream and two branch streams (Branch Stream 1 and Branch Stream 2). The main stream contains only one CNN model as the backbone classifier; each branch stream contains only one branch classifier, which is built upon a MobileNetV2 model and incorporates an attention module to enhance information interaction between the branch stream and the main stream. Both the backbone classifier and the two branch classifiers use remote sensing images as input. Based on the difference in the dimension of the feature maps within the models, the internal convolutional structures of both the CNN and MobileNetV2 models are divided into four modules. Furthermore, the multi-stream self-distillation architecture includes a feature fusion classifier and an ensemble classifier. The feature fusion classifier fuses remote sensing image features extracted by the CNN and the two MobileNetV2 models for classification. The ensemble classifier obtains its classification result by averaging the classification results from the CNN, the two MobileNetV2 models, and the feature fusion classifier.

[0020] The backbone consists of only one CNN model, which includes a convolutional structure and a classification layer. The convolutional structure is divided into four modules, denoted as CNN module i, i = 1, 2, 3, 4, based on the dimensionality of the feature maps within the model. Each module includes a convolutional layer, a batch normalization layer, and the ReLU activation function to extract image features. The classification layer consists of an adaptive average pooling layer and a fully connected layer to generate the classification result. Assuming the input image has dimensions H×W×C, where H and W represent the height and width of the image, respectively, and C represents the number of channels, after feature extraction by CNN module i, the output feature map Mi has the following dimensions: , where C i This represents the number of channels in the output feature map of CNN module i; the specific value depends on the structural design of the CNN model. The input image undergoes feature extraction through four CNN modules, ultimately yielding feature map M4. This M4 then passes through an adaptive average pooling layer and a fully connected layer in the classification layer, outputting the main stream classification result y3. (1) In the above formula, z represents the input image; This represents the feature extraction operation of CNN module i (i = 1, 2, 3, 4); "Indicates the process of forward propagation; This represents the classification layer in the main stream.

[0021] The structure of branch flow 1. Branch flow 1 contains a branch classifier 1, which includes a MobileNetV2 module, an attention module A, and a CNN module 1. The MobileNetV2 module includes convolutional structures and a classification layer. Each convolutional structure is divided into four modules, denoted as Mv2_A module i, i = 1, 2, 3, 4, based on the dimensionality of the internal feature map. Each Mv2_A module consists of a depthwise separable convolution, a BN layer, and a ReLU6 activation function. The classification layer mainly consists of an adaptive average pooling layer and a fully connected layer, used to generate the classification result. The input image of this flow is the same as the main flow input image. After feature extraction by Mv2_A module i, the resulting output feature map Vi has the following dimensions: ,in, This represents the number of channels in the output feature map of module i in Mv2_A; where the output feature map Vi of module i has the same height and width as the output feature map Mi of CNN module i. Attention module A mainly consists of three convolutional branches and a sigmoid activation function. Convolutional branch 1 consists of a standard 1×1 convolution, a batch normalization layer, and a ReLU6 activation function; convolutional branch 2 consists of depthwise separable convolutions (depthwise convolution and pointwise convolution), a batch normalization layer, and a ReLU6 activation function; convolutional branch 3 consists of grouped convolutions, a batch normalization layer, and a ReLU6 activation function. The remote sensing image input branch stream 1 passes through Mv2... _ After feature extraction by module A, feature map V1 is obtained, which is then multiplied element-wise with attention mask Mask1 generated by attention module A: (2) In the above formula This is represented as element-wise multiplication; then, Following branch 1, Mv2 _ Module A and the classification layer process the data, ultimately outputting the classification result y1 of branch flow 1: (3) In the above formula This represents the feature extraction operation for module i (i = 1, 2, 3, 4) of Mv2_A; This represents the classification layer in branch flow 1; The output feature map is denoted as Vi; it should be noted that the input feature map of attention module A is the output feature map M1 of CNN module 1.

[0022] The structure of branch flow 2. Branch flow 2 contains a branch classifier 2, which includes a MobileNetV2 module, an attention module B, and CNN module 1 and CNN module 2. The internal convolutional structure of MobileNetV2 is also divided into four modules, denoted as Mv2_B module i, i = 1, 2, 3, 4, in the same way as MobileNetV2 in branch classifier 1. Each Mv2_B module includes a depthwise separable convolution, a BN layer, and a ReLU6 activation function. The classification layer of MobileNetV2 includes an adaptive average pooling layer and a fully connected layer to produce the classification result. The input image of this flow is the same as the input image of the main flow. After features are extracted by Mv2_B module i, the output feature map Qi has the following dimensions: ,in, This represents the number of channels in the output feature map of module i (Mv2_B). The output feature map Qi of module i has the same height and width as the output feature map Mi of CNN module i. The internal structure of attention module B is exactly the same as that of attention module A, except that the input feature map is different; the input feature map of attention module B is the output feature map M2 of CNN module 2. The remote sensing image input branch stream 2 passes through Mv2... _ Module B1 and Mv2 _ After feature extraction by module B2, feature map Q2 is obtained, which is then multiplied element-wise with the attention mask Mask2 generated by attention module B: (4) In the above formula This is represented as element-wise multiplication; then, Following branch 2, the subsequent Mv2 _ Module B and the classification layer process the data, ultimately outputting the classification result y2 of branch flow 2: (5) In the above formula This represents the feature extraction operation for module i (i = 1, 2, 3, 4) of Mv2_B; This represents the classification layer in branch flow 2; The output feature map is denoted as Qi.

[0023] The feature fusion classifier includes CNN module 1, CNN module 2, CNN module 3, CNN module 4, Mv2_A module 1, Mv2_A module 2, Mv2_A module 3, Mv2_A module 4, attention module A, Mv2_B module 1, Mv2_B module 2, Mv2_B module 3, Mv2_B module 4, attention module B, a classification layer, and a feature alignment layer. The classification layer consists of an adaptive average pooling layer and a fully connected layer to generate the classification result. The feature alignment layer consists of a standard 1×1 convolution, a BN layer, and a ReLU activation function. The feature alignment layer takes the output feature map M4 of CNN module 4 as its input feature map, resulting in a feature map with dimension M4. The output feature map F is provided to the classification layer; where the number of channels in the output feature map F is... The number of channels in the output feature map V4 of module 4, Mv2_A. The number of channels in the output feature map Q4 of module 4, Mv2_B. They are completely identical. The fused feature map is obtained by adding the three feature maps element-wise: (6) in This is represented as element-wise addition. This represents the fused feature map; subsequently, The classification layer of the feature fusion classifier outputs the classification result y4 of the feature fusion classifier: (7) In the above formula This represents the classification layer of the feature fusion classifier.

[0024] The ensemble classifier consists of CNN module 1, CNN module 2, CNN module 3, CNN module 4, the classification layer in the main stream, Mv2_A module 1, Mv2_A module 2, Mv2_A module 3, Mv2_A module 4, attention module A, the classification layer in branch stream 1, Mv2_B module 1, Mv2_B module 2, Mv2_B module 3, Mv2_B module 4, attention module B, the classification layer in branch stream 2, the feature alignment layer, and the classification layer in the feature fusion classifier. The ensemble classifier does not require a separate classification layer; it obtains the classification result y5 by integrating the classification results y1, y2, y3, and y4 from branch stream 1, branch stream 2, the main stream, and the feature fusion classifier. (8) This result is used as the classification result of the teacher model to supervise the training of branch flow 1, branch flow 2, main flow, and feature fusion classifiers.

[0025] S2 Training Loss Function The multi-stream self-distillation architecture contains five classification results: y1 for branch stream 1, y2 for branch stream 2, y3 for the main stream, y4 for the feature fusion classifier, and y5 for the ensemble classifier. Before loss calculation, each classification result is processed by a SoftMax function incorporating a temperature parameter to convert the classification result into a classification probability. (9) In the above formula, This represents the classification probability of category i. Let L represent the classification result of category i, and L be the total number of categories. T is a temperature parameter, which is used to adjust the smoothness of the probability distribution. When T=1, equation (9) is transformed into the standard SoftMax function.

[0026] During the training phase of the multi-stream self-distillation architecture, the loss function consists of two parts: cross-entropy loss and self-distillation loss. The cross-entropy loss measures the difference between the model output and the true distribution by comparing the classification probability obtained after processing each classification result using the SoftMax function with the true image label. (10) In the above formula, For cross-entropy loss, y trueFor the true label of the image, The cross-entropy loss function; The SoftMax function corresponding to T=1 in expression (9); The self-distillation loss treats the ensemble classifier as the teacher model and branch flow 1, branch flow 2, the main flow, and the feature fusion classifier as student models. By minimizing the difference between their classification probabilities and those of the teacher model, it guides each student model to learn the "knowledge" inherent in the ensemble model. This loss uses Kullback-Leibler divergence (KL divergence) as a metric. (11) In the above formula, This is due to losses from self-distillation. Let KL divergence be a metric. The SoftMax function corresponding to T>1 in expression (9) will be explained in the experimental setup; In summary, the weighted sum of the two losses is taken as the final training loss: (12) In the above formula, For the final training loss, is a hyperparameter, used as the weight for self-distillation loss.

[0027] S3 Experiment S31 Experimental Dataset The experiment used four public remote sensing image datasets for evaluation: AID (Aerial Image Dataset), UCMerced_LandUse, NWPU-RESISC45, and RSSCN7. The AID dataset contains 10,000 remote sensing images covering 30 typical land use scenarios with resolutions ranging from 0.5 to 8 meters. The UCMerced_LandUse dataset contains 2,100 remote sensing images covering 21 urban land use scenarios with a resolution of 0.3 meters. The NWPU-RESISC45 dataset contains 31,500 remote sensing images covering 45 land cover or land use types. The RSSCN7 dataset contains 2,800 remote sensing images covering 7 natural landform scenarios.

[0028] UCMerced_LandUse uses two training ratios of 50% and 80% for random partitioning; RSCN7 and AID use two training ratios of 20% and 50% for random partitioning; NWPU-RESISC45 uses two training ratios of 10% and 20% for random partitioning. The data preprocessing process includes the following steps: First, while maintaining the aspect ratio, the short side of the image is scaled to 224 pixels, and then a 224×224 area is cropped from the center of the image; then, the image is randomly horizontally flipped with a 50% probability, but this flipping operation is not performed during the testing phase; then, the image is converted to a channel-first tensor format; finally, the image is standardized using a mean of ([0.485, 0.456, 0.406]) and a standard deviation of ([0.229, 0.224, 0.225]).

[0029] S32 Experiment Setup Two CNN models (ResNet18 and VGG11) were used to build a multi-stream self-distillation architecture for the main stream, and experiments were conducted on the four datasets mentioned above. Before model training, both ResNet18 and VGG11 were loaded with weights pre-trained on ImageNet. The Adam optimizer was used for optimization in the experiment, with an initial learning rate of 1e-4, and the learning rate was adjusted to 0.1 times the original rate after every 20 epochs. The temperature parameter T was set to 3. The hyperparameters in the loss function (12) Set to 1. Train all models for 50 epochs, with a batch size of 64.

[0030] S33 Experimental Results The experimental results are evaluated using Overall Accuracy (OA). To ensure the reliability of the results, five random partitions and experiments were performed on each of the four datasets. The mean and standard deviation of the five experimental results were calculated to obtain the final OA, as shown in Table 1. In Table 1, "Baseline" represents the OA obtained by training only the backbone CNN model, serving as a performance reference. All other results are based on a multi-stream self-distillation architecture. Specifically, "Branch 1" represents the OA obtained by classifier 1 in branch flow 1; "Branch 2" represents the OA obtained by classifier 2 in branch flow 2; "Backbone" represents the OA obtained by the CNN model in the backbone flow; "Feature Fusion" represents the OA obtained by the feature fusion classifier; and "Ensemble" represents the OA obtained by the ensemble classifier.

[0031] As shown in Table 1, regardless of whether ResNet18 or VGG11 is used as the backbone CNN model, the feature fusion classifier or ensemble classifier achieves the highest OA on all four datasets under different training ratios. This indicates that fusing multi-stream features or ensemble multi-model predictions can effectively enhance feature representation and classification performance. Furthermore, comparing the baseline and backbone OA values ​​reveals that the OA results of all backbone classifiers are superior to the corresponding baseline results, indicating that the multi-stream self-distillation architecture can effectively improve model performance without changing the backbone model structure. Finally, analyzing the OA results of branch 1 and branch 2 shows that the branch streams built based on lightweight models can achieve discrimination performance comparable to or even better than the backbone, demonstrating good application potential. For example, on the AID dataset with a training ratio of 20%, when using VGG11 as the backbone CNN model, the OA of branch classifier 1 and branch classifier 2 are 94.20% and 94.13%, respectively, both superior to the 93.22% achieved by the backbone CNN model.

[0032] In summary, the multi-stream self-distillation architecture proposed in this invention can provide users with a variety of classifier options with different deployment costs based on the resource constraints and performance requirements of specific application scenarios: Branch 1 and Branch 2 are built based on the lightweight MobileNetV2 model, suitable for edge devices or real-time scenarios with limited computing resources; the backbone represents the classification capability of the standard CNN model; the feature fusion classifier and ensemble classifier can achieve higher classification accuracy under relatively high computational overhead, suitable for tasks with high accuracy requirements. The overall scheme provides an effective design approach for balancing model performance and deployment cost, and can provide a reference for subsequent related research.

[0033] Table 1. Experimental Results The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A remote sensing image scene classification method based on a multi-stream self-distillation architecture, characterized in that, Includes the following steps: S1. Construct a multi-stream remote sensing image scene classification model; the multi-stream remote sensing image scene classification model includes a multi-stream self-distillation architecture; the multi-stream self-distillation architecture includes a main stream, two branch streams, a feature fusion classifier, and an ensemble classifier; S2. Self-distillation training of the multi-stream remote sensing image scene classification model based on the joint loss function; S3. Classify remote sensing images using a trained multi-stream remote sensing image scene classification model.

2. The method according to claim 1, characterized in that, The construction of the multi-stream self-distillation architecture includes: The multi-stream self-distillation architecture comprises a main stream, two branch streams, a feature fusion classifier, and an ensemble classifier. The main stream uses a CNN model as the backbone classifier. Each branch stream contains only one branch classifier, built upon a MobileNetV2 model, and incorporates an attention module to enhance information interaction between the branch stream and the main stream. Both the backbone classifier and the two branch classifiers use remote sensing images as input. Based on the difference in the dimension of the feature maps within the models, the internal convolutional structures of both the CNN and MobileNetV2 models are divided into four modules. The feature fusion classifier fuses remote sensing image features extracted by the CNN and the two MobileNetV2 models for classification. The ensemble classifier obtains the classification result by averaging the classification results from the CNN, the two MobileNetV2 models, and the feature fusion classifier. The main pipeline contains only one CNN model, which includes a convolutional structure and a classification layer. The convolutional structure is divided into four modules, denoted as CNN module i, i = 1, 2, 3, 4, based on the dimensionality of the internal feature map. Each module includes a convolutional layer, a batch normalization layer, and the ReLU activation function to extract image features. The classification layer consists of an adaptive average pooling layer and a fully connected layer to generate the classification result. Assuming the input image has dimensions H×W×C, where H and W represent the height and width of the image, respectively, and C represents the number of channels, after feature extraction by CNN module i, the output feature map Mi has the following dimensions: , where C i This represents the number of channels in the output feature map of CNN module i. The specific value depends on the structural design of the CNN model. The input image is processed stepwise by four CNN modules to extract features, finally obtaining feature map M4. After passing through the adaptive average pooling layer and fully connected layer in the classification layer, the main stream classification result y3 is output. (1) In the above formula, z represents the input image; This represents the feature extraction operation of CNN module i (i = 1, 2, 3, 4); "Indicates the process of forward propagation; This represents the classification layer in the main stream; One of the two branch flows, namely branch flow 1, contains a branch classifier 1, which includes a MobileNetV2 module, an attention module A, and a CNN module 1. The MobileNetV2 module includes convolutional structures and a classification layer. Each convolutional structure is divided into four modules, denoted as Mv2_A module i, i = 1, 2, 3, 4, based on the dimensionality of the internal feature map. Each Mv2_A module consists of a depthwise separable convolution, a BN layer, and a ReLU6 activation function. The classification layer mainly consists of an adaptive average pooling layer and a fully connected layer, used to generate the classification result. The input image of this flow is the same as the input image of the main flow. After feature extraction by Mv2_A module i, the resulting output feature map Vi has the following dimensions: ,in, This represents the number of channels in the output feature map of module i in Mv2_A; where the output feature map Vi of module i has the same height and width as the output feature map Mi of module i in CNN; the attention module A mainly consists of three convolutional branches and a sigmoid activation function, where convolutional branch 1 consists of a standard 1×1 convolution, a batch normalization layer, and a ReLU6 activation function; convolutional branch 2 consists of depthwise separable convolution (depthwise convolution and pointwise convolution), a batch normalization layer, and a ReLU6 activation function; convolutional branch 3 consists of grouped convolution, a batch normalization layer, and a ReLU6 activation function; the remote sensing image input branch stream 1 passes through Mv2 _ After feature extraction by module A, feature map V1 is obtained, which is then multiplied element-wise with attention mask Mask1 generated by attention module A: (2) In the above formula This is represented as element-wise multiplication; then, Following branch 1, Mv2 _ Module A and the classification layer process the data, ultimately outputting the classification result y1 of branch flow 1: (3) In the above formula This represents the feature extraction operation for module i (i = 1, 2, 3, 4) of Mv2_A; This represents the classification layer in branch flow 1; The output feature map is denoted as Vi; it should be noted that the input feature map of attention module A is the output feature map M1 of CNN module 1; The other branch of the two branch flows, namely branch flow 2, contains a branch classifier 2, which includes a MobileNetV2 module, an attention module B, and CNN modules 1 and 2. The internal convolutional structure of MobileNetV2 is also divided into four modules, denoted as Mv2_B module i, i = 1, 2, 3, 4, in the same way as MobileNetV2 in branch classifier 1. Each Mv2_B module includes a depthwise separable convolution, a BN layer, and a ReLU6 activation function. The classification layer of MobileNetV2 includes an adaptive average pooling layer and a fully connected layer to produce the classification result. The input image of this flow is the same as the input image of the main flow. After features are extracted by Mv2_B module i, the resulting output feature map Qi has the following dimensions: ,in, This represents the number of channels in the output feature map of module i (Mv2_B); where the height and width of the output feature map Qi of module i (Mv2_B) are the same as those of the output feature map Mi of CNN module i; the internal structure of attention module B is exactly the same as that of attention module A, the difference being the input feature map, which is the output feature map M2 of CNN module 2; the remote sensing image input branch stream 2 passes through Mv2... _ Module B1 and Mv2 _ After feature extraction by module B2, feature map Q2 is obtained, which is then multiplied element-wise with the attention mask Mask2 generated by attention module B: (4) In the above formula This is represented as element-wise multiplication; then, Following branch 2, the subsequent Mv2 _ Module B and the classification layer process the data, ultimately outputting the classification result y2 of branch flow 2: (5) In the above formula This represents the feature extraction operation for module i (i = 1, 2, 3, 4) of Mv2_B; This represents the classification layer in branch flow 2; The output feature map is denoted as Qi; The feature fusion classifier includes CNN module 1, CNN module 2, CNN module 3, CNN module 4, Mv2_A module 1, Mv2_A module 2, Mv2_A module 3, Mv2_A module 4, attention module A, Mv2_B module 1, Mv2_B module 2, Mv2_B module 3, Mv2_B module 4, attention module B, a classification layer, and a feature alignment layer. The classification layer consists of an adaptive average pooling layer and a fully connected layer to generate the classification result. The feature alignment layer consists of a standard 1×1 convolution, a BN layer, and a ReLU activation function. The feature alignment layer takes the output feature map M4 of CNN module 4 as its input feature map, resulting in a feature map with dimension M4. The output feature map F is provided to the classification layer; where the number of channels in the output feature map F is... The number of channels in the output feature map V4 of module 4, Mv2_A. The number of channels in the output feature map Q4 of module 4, Mv2_B. Completely identical; the three feature maps are added element-wise to obtain the fused feature map: (6) in This is represented as element-wise addition. This represents the fused feature map; subsequently, The classification layer of the feature fusion classifier outputs the classification result y4 of the feature fusion classifier: (7) In the above formula This represents the classification layer of the feature fusion classifier; The ensemble classifier consists of CNN module 1, CNN module 2, CNN module 3, CNN module 4, the classification layer in the main stream, Mv2_A module 1, Mv2_A module 2, Mv2_A module 3, Mv2_A module 4, attention module A, the classification layer in branch stream 1, Mv2_B module 1, Mv2_B module 2, Mv2_B module 3, Mv2_B module 4, attention module B, the classification layer in branch stream 2, the feature alignment layer, and the classification layer in the feature fusion classifier. The ensemble classifier does not require a separate classification layer; it obtains the classification result y5 by integrating the classification results y1, y2, y3, and y4 from branch stream 1, branch stream 2, the main stream, and the feature fusion classifier. (8) This result is used as the classification result of the teacher model to supervise the training of branch flow 1, branch flow 2, main flow, and feature fusion classifiers.

3. The method according to claim 2, characterized in that, The self-distillation training of the multi-stream remote sensing image scene classification model based on the joint loss function includes: The multi-stream self-distillation architecture contains five classification results: y1 for branch stream 1, y2 for branch stream 2, y3 for the main stream, y4 for the feature fusion classifier, and y5 for the ensemble classifier. Before loss calculation, each classification result is processed by a SoftMax function incorporating a temperature parameter to convert the classification result into a classification probability. (9) In the above formula, This represents the classification probability of category i. The classification result of category i is represented by L, where L is the total number of categories; T is a temperature parameter, which is used to adjust the smoothness of the probability distribution; when T=1, equation (9) is transformed into the standard SoftMax function. During the training phase of the multi-stream self-distillation architecture, the loss function consists of two parts: cross-entropy loss and self-distillation loss. The cross-entropy loss measures the difference between the model output and the true distribution by comparing the classification probability obtained after processing each classification result using the SoftMax function with the true image label. (10) In the above formula, For cross-entropy loss, y true For the true label of the image, The cross-entropy loss function; The SoftMax function corresponding to T=1 in expression (9); The self-distillation loss treats the ensemble classifier as the teacher model and branch flow 1, branch flow 2, the main flow, and the feature fusion classifier as student models. By minimizing the difference between their classification probabilities and those of the teacher model, it guides each student model to learn the "knowledge" inherent in the ensemble model. This loss uses Kullback-Leibler divergence (KL divergence) as a metric. (11) In the above formula, This is due to losses from self-distillation. Let KL divergence be a metric. The SoftMax function corresponding to T>1 in expression (9) will be explained in the experimental setup; In summary, the weighted sum of the two losses is taken as the final training loss: (12) In the above formula, For the final training loss, is a hyperparameter, used as the weight for self-distillation loss.

4. A remote sensing image scene classification device based on a multi-stream self-distillation architecture, characterized in that, include: The model building module is used to construct a multi-stream remote sensing image scene classification model; the multi-stream remote sensing image scene classification model includes a multi-stream self-distillation architecture; the multi-stream self-distillation architecture includes a main stream, two branch streams, a feature fusion classifier, and an ensemble classifier; The training module performs self-distillation training on the multi-stream remote sensing image scene classification model based on the joint loss function; The classification module is used to classify remote sensing images using a trained multi-stream remote sensing image scene classification model.

Images