A remote sensing image classification method based on data enhancement and knowledge distillation

Abstract

The present invention discloses an innovative remote sensing image classification method, which aims to improve the classification performance by combining multi-source data enhancement and knowledge distillation strategies under a few-sample mechanism. Considering the diversity and complexity of remote sensing image data, a multi-source data enhancement technology is firstly adopted, including multi-scale scaling and multi-angle rotation enhancement, to increase the generalization ability of the model and the recognition ability of complex scene features. Secondly, a lightweight deep learning model is designed, combining a feature pyramid network and an attention mechanism to efficiently process multi-scale information and emphasize key features. Finally, a knowledge distillation method is used to transfer the knowledge of the complex model to a lightweight model, and the learning process is optimized through scale-aware distillation and mixed sample feature distillation. Experimental verification is carried out on four public remote sensing image datasets. Compared with the existing technology, this method significantly improves the accuracy of remote sensing image classification, proving its effectiveness and practicality in resource-constrained environments.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image classification and artificial intelligence technology, relates to a method for data enhancement and knowledge enhancement of remote sensing images, and is specifically applied to provide an effective solution for remote sensing scene classification under a few-sample mechanism. Background Art

[0002] With the rapid development of remote sensing technology, it has become easier to obtain a large number of high-resolution remote sensing images, which provides valuable data resources for the fields of land cover classification, resource monitoring, and environmental change assessment. Remote sensing scene classification, as one of the key technologies for understanding and analyzing these images, aims to automatically identify and classify objects and scenes in remote sensing images, and is an important research direction in the field of remote sensing image processing [2]. The research on remote sensing scene classification has important theoretical and practical significance. It not only promotes the development of remote sensing science and geographic information science, but also has a profound impact on the practical application in many fields such as environmental monitoring, urban planning, disaster management, and agricultural development. The research on remote sensing image classification continues to promote the deep integration of remote sensing technology with artificial intelligence, machine learning, and other fields, and promotes the emergence of new algorithms, new models, and new applications. This interdisciplinary intersection has promoted theoretical innovation and technological progress in related fields.

[0003] Early research on remote sensing image classification mainly relied on traditional machine learning methods, such as support vector machines (SVM), decision trees, random forests, and K-nearest neighbors (KNN). These methods usually rely on hand-crafted feature extraction, such as texture, color, shape, etc., to distinguish different objects or scenes. Research on remote sensing image classification has made significant progress in recent years, especially the application of deep learning technology, which provides new solutions to the challenges of complex and changeable remote sensing data. These studies cover a variety of innovative methods to improve the accuracy and efficiency of classification. In recent years, deep learning methods, especially convolutional neural networks (CNN), have become the mainstream technology for remote sensing image classification. Deep learning methods can automatically learn complex feature representations from raw images, thereby significantly improving classification performance. Researchers have developed a variety of models using deep learning frameworks, such as ResNet, VGG, Inception, and GoogleNet, to handle high-dimensional data and complex backgrounds of remote sensing images. Traditional remote sensing image classification methods usually rely on a large amount of labeled data and complex deep learning models, which are difficult to implement in resource-constrained environments. In addition, the high-dimensional characteristics of remote sensing images and the large-scale variations between different geographical regions further increase the difficulty of the classification task. Therefore, how to improve the accuracy and efficiency of remote sensing image classification under the mechanism of limited labeled data and limited computing resources has become an urgent problem to be solved.

[0004] Although deep learning technology has made significant progress in remote sensing image classification in recent years, it still faces significant challenges in the few-shot environment, such as limited labeled data and computing resources. (1) Data scarcity and lack of diversity: In the few-shot environment, obtaining a large amount of high-quality and diverse labeled remote sensing data is a significant challenge. Remote sensing image data usually requires professional knowledge to accurately label, which is time-consuming and expensive. In addition, images in specific geographical locations or special scenes may be difficult to obtain, resulting in insufficient geographical coverage and image diversity in the training dataset. This scarcity and limited data limit the ability of deep learning models to learn complex image features, which in turn affects classification accuracy and model generalization. (2) Computational resource limitations: Deep learning models, especially the current state-of-the-art models, usually require a lot of computing resources for training and inference. In the few-shot environment, such as the lack of high-performance GPUs or limited storage space, training these large models becomes impractical. This not only prolongs the training time of the model, but also limits the model complexity and size, which may result in the model being unable to fully learn the features in the data. Therefore, how to reduce computing and storage requirements while ensuring model performance has become an important research challenge.

[0005] Existing datasets may not be sufficient to fully cover the needs of practical applications in terms of geographical distribution, scene diversity, and seasonal changes, which limits the generalization ability of the model. Therefore, how to improve the learning efficiency and generalization ability of the model when labeled data is scarce has become a major research motivation. Under the condition of limited computing resources, how to design a lightweight model with high computational efficiency, small model size, and not sacrificing too much performance is an urgent problem to be solved. This is of great significance for promoting the practical application of remote sensing scene classification, especially in resource-constrained areas or on mobile devices. The complexity of remote sensing images and the variability of scenes require classification models to have high accuracy and robustness. How to accurately classify complex and changing remote sensing scenes while ensuring the simplicity of the model and computational efficiency is another important motivation for this study. This includes developing new data augmentation methods, designing more appropriate model structures, and using advanced learning strategies such as knowledge distillation. Summary of the invention

[0006] This study proposes an innovative remote sensing image classification method, which aims to improve the classification performance by combining multi-source data enhancement and knowledge distillation under the few-sample mechanism. First, we introduced a multi-source data enhancement technique, including multi-scale and multi-angle rotation enhancement and mixed sample data enhancement, to make full use of limited data resources and increase the generalization ability of the model and the recognition ability of complex images. Secondly, considering the limitation of computing resources, we designed and implemented a lightweight deep learning model, combined with feature pyramid network and attention mechanism, to effectively process multi-scale information and highlight key features. Finally, through the knowledge distillation method, the knowledge of the complex model is effectively transferred to the lightweight model, which further improves the performance of the model under the limited resource mechanism.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A remote sensing image classification method based on data enhancement and knowledge distillation includes the following steps:

[0009] S1. By applying multi-scale enhancement technology and multi-angle rotation enhancement technology Expand the original remote sensing image dataset D to obtain an enhanced dataset

[0010] S2. Design ShuffleNet network architecture lightweight deep learning model Integrate the attention mechanism SE (Squeeze-and-Excitation) module to explicitly model the dependencies between channels and improve the model's sensitivity to important features;

[0011] S3. Using pre-trained complex teacher models through the knowledge distillation process To guide students model training, including Pre-trained on the complete enhanced dataset D′ to capture rich scene features; knowledge distillation is achieved through the loss function Implementation, this function combines the traditional classification loss and the knowledge transfer loss between the teacher-student model;

[0012] S4, optimized lightweight model Effectively analyze new remote sensing images new Classification is performed to improve the accuracy and efficiency of remote sensing image classification under the few-sample mechanism.

[0013] The multi-scale enhancement technology in S1 and multi-angle rotation enhancement technology for:

[0014] Multi-scale enhancement By adjusting the size of the image to simulate remote sensing images captured from different heights, the model's adaptability to scale changes can be enhanced. Given an original image x i , multi-scale enhancement is expressed as:

[0015]

[0016] in, is the image scaling function, s is the scaling factor, which can take a series of predefined values ​​to generate images of different scales;

[0017] Multi-angle rotation enhancement By rotating the image at different angles, the diversity of viewing angles can be increased. Given an original image x i , multi-angle rotation enhancement can be expressed as:

[0018]

[0019] in, is the image rotation function, θ is the rotation angle, and a random angle is usually selected to increase data diversity.

[0020] The ShuffleNet network architecture in S2 is a lightweight deep learning model for:

[0021] Lightweight model The efficient network architecture ShuffleNet is adopted. For remote sensing image data, the input of the model is x∈R H×W×C , where H, W and C represent the height, width and number of channels of the image respectively; the model output is a category probability distribution Where θ represents the model parameters, p∈R K The predicted probabilities corresponding to K categories;

[0022] Depthwise separable convolution decomposes the traditional convolution operation into two steps: depthwise convolution and pointwise convolution. Given an input feature map F∈R H′×W′×C′ , where R indicates that F belongs to the real number domain, H′ indicates the height of the feature map, W′ indicates the width of the feature map, and C′ indicates the number of channels of the feature map;

[0023] Depthwise separable convolution first applies a depthwise convolution F. depthwise =F depth (F,K depth ), where F depth is the feature map after deep convolution operation, K depth is the depthwise convolution kernel; then the pointwise convolution operation F is applied pointwise =F point (F depthwise,K point ), where F point is a function representing the point-by-point convolution operation, which is outputted at the depth convolution F depthwise Based on this, we use the point-by-point convolution kernel K point Perform convolution.

[0024] The integrated attention mechanism SE (Squeeze-and-Excitation) module in S2 is:

[0025] The integrated attention mechanism SE module first performs pointwise Perform global average pooling to obtain a channel descriptor, and recalibrate the weight of each channel through the fully connected layer:

[0026] z=F avg (F pointwise )

[0027] e=σ(W2δ(W1z))

[0028]

[0029] Among them, z is the compressed feature, F avg is the global average pooling operation, e is the intermediate vector, W1 and W2 are the weights of the fully connected layer, δ is the ReLU activation function, σ is the sigmoid activation function, It is a weighted feature map for further classification.

[0030] The knowledge distillation process and loss function in S3 for:

[0031] Large Teacher Model The GoogleNet model is implemented using the Inception architecture and pre-trained using the remote sensing image AID dataset; lightweight student model Implemented using ShuffleNet; the outputs of the teacher model and the student model are and Represents the category probability prediction for the same input x; the goal of knowledge distillation is to minimize the difference between the output of the student model and the output of the teacher model, usually through the loss function accomplish;

[0032] The loss function of knowledge distillation consists of two parts: hard target loss and soft target loss; hard target loss Directly compare predictions of student models and the true label y i :

[0033]

[0034] Soft target loss Compare the soft labels output by the teacher model and the student model, usually calculated using the temperature-scaled Softmax function:

[0035]

[0036] in, and are the predicted probability distributions of the teacher model and the student model for sample i, represents the unnormalized output of the teacher model for sample i, represents the unnormalized output of the student model for sample i, T is the temperature parameter used to adjust the smoothness of the Softmax output, and KL is the Kullback-Leibler divergence, which is used to measure the difference between two probability distributions p T and p S The differences between

[0037]

[0038] The total loss function is the weighted sum of hard target loss and soft target loss:

[0039]

[0040] Among them, α is the weight parameter used to balance the two losses.

[0041] The S4 is an optimized lightweight model Effectively analyze new remote sensing images new Classify as:

[0042] In the testing phase, when a new remote sensing image x new When entering the model, after forward propagation, the model Will output its corresponding category y new :

[0043]

[0044] Among them, argmax represents the index corresponding to the maximum value in the probability distribution, and Softmax is a normalization function used to convert the output score of the model into a probability distribution.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] (1) This paper proposes a technology that combines multi-scale and multi-angle rotation enhancement and mixed sample data enhancement, which significantly improves the diversity and coverage of training data. Especially in the case of a small number of samples with limited labeled data, this data enhancement technology effectively expands the available training samples and helps the model learn richer and more complex scene features, thereby improving the accuracy of remote sensing image classification.

[0047] (2) Considering the need for computational efficiency under the few-sample mechanism, the present invention designs a lightweight deep learning model, which particularly optimizes the number of model parameters and computational complexity. In order to overcome the information loss problem caused by insufficient labeled data, the present invention adopts a knowledge distillation strategy to effectively transfer the knowledge of the large pre-trained model to the lightweight model.

[0048] (3) Extensive experiments were conducted on four public remote sensing image datasets: UCM, WHU-RS19, NWPU-RESISC45, and AID, verifying the superiority of the proposed method over the existing technology. The knowledge distillation strategy of the present invention not only improves the performance of the model, but also provides an effective solution for remote sensing image classification under the few-sample mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0050] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0051] Figure 1 is a flow chart of an embodiment of the present invention.

[0052] Figure 2 An innovative policy model framework combining multi-source data augmentation and knowledge distillation. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. These descriptions are only to further illustrate the features and advantages of the present invention, rather than to limit the claims of the present invention. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0054] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0055] Given a remote sensing image dataset where x i represents the i-th remote sensing image, y i is the corresponding scene label, and N is the total number of images in the dataset. The goal is to design a classification model f(·) that can accurately classify remote sensing images x into the correct scene category y. train The data subset used for training. test Data subset for testing. x′ is a remote sensing image after data augmentation. is the teacher model, a pre-trained large deep learning model. It is a student model, a lightweight deep learning model. is the loss function used to evaluate the performance of the model during training. α is the adjustment parameter in the data enhancement and knowledge distillation strategies.

[0056] Generate new training samples through multi-scale and multi-angle rotation enhancement and mixed sample data enhancement technology Where M is the total number of images in the augmented dataset. Designing a lightweight model To effectively train and reason with limited computing resources. and student models The knowledge distillation process between knowledge transfer By defining a suitable loss function to achieve this process.

[0057] like Figure 1 As shown, the framework of the present invention is mainly divided into the following four steps, which are connected layer by layer and finally integrated. The process mainly includes the following steps:

[0058] S1. By applying multi-scale enhancement technology and multi-angle rotation enhancement technology Expand the original remote sensing image dataset D to obtain an enhanced dataset

[0059] The multi-scale enhancement technology in S1 and multi-angle rotation enhancement technology for:

[0060] Multi-scale enhancement By adjusting the size of the image to simulate remote sensing images captured from different heights, the model's adaptability to scale changes can be enhanced. Given an original image x i , multi-scale enhancement is expressed as:

[0061]

[0062] in, is the image scaling function, s is the scaling factor, which can take a series of predefined values ​​to generate images of different scales;

[0063] Multi-angle rotation enhancement By rotating the image at different angles, the diversity of viewing angles can be increased. Given an original image x i , multi-angle rotation enhancement can be expressed as:

[0064]

[0065] in, is the image rotation function, θ is the rotation angle, and a random angle is usually selected to increase data diversity.

[0066] Through these multi-source data enhancement strategies, the diversity and quantity of training data can be significantly increased, thereby improving the generalization ability and performance of remote sensing scene classification models under the few-sample mechanism. These strategies not only take into account the particularities of remote sensing images, such as the diversity of scales and perspectives, but also introduce innovative mixed sample techniques to simulate complex scene interactions, providing a rich data foundation for training more powerful models.

[0067] In order to adapt to the remote sensing scene classification task under the few-sample mechanism, the design of lightweight models focuses on reducing model parameters and computational complexity while maintaining or improving classification accuracy. The following are the key components of lightweight model design, including the choice of network architecture, the application of depthwise separable convolution, and the integration of attention mechanism.

[0068] S2. Design ShuffleNet network architecture lightweight deep learning model Integrate the attention mechanism SE (Squeeze-and-Excitation) module to explicitly model the dependencies between channels and improve the model's sensitivity to important features;

[0069] The ShuffleNet network architecture in S2 is a lightweight deep learning model for:

[0070] Lightweight model The efficient network architecture ShuffleNet is adopted. For remote sensing image data, the input of the model is x∈R H×W×C , where H, W and C represent the height, width and number of channels of the image respectively; the model output is a category probability distribution Where θ represents the model parameters, p∈R K The predicted probabilities corresponding to K categories;

[0071] Depthwise separable convolution decomposes the traditional convolution operation into two steps: depthwise convolution and pointwise convolution. Given an input feature map F∈R H′×w′×C′ , where R indicates that F belongs to the real number domain, H′ indicates the height of the feature map, W′ indicates the width of the feature map, and C′ indicates the number of channels of the feature map;

[0072] Depthwise separable convolution first applies a depthwise convolution F. depthwise =F depth (F,F depth ), where F depth is the feature map after deep convolution operation, K depth is the depthwise convolution kernel; then the pointwise convolution operation F is applied pointwise =F point (F depthwise ,K point ), where F point is a function representing the point-by-point convolution operation, which is outputted at the depth convolution F depthwise Based on this, we use the point-by-point convolution kernel K point Perform convolution.

[0073] The integrated attention mechanism SE (Squeeze-and-Excitation) module in S2 is:

[0074] The integrated attention mechanism SE module first performs pointwise Perform global average pooling to obtain a channel descriptor, and recalibrate the weight of each channel through the fully connected layer:

[0075] z=F avg (F pointwise )

[0076] e=σ(W2δ(W1z))

[0077]

[0078] Among them, z is the compressed feature, Favg is the global average pooling operation, e is the intermediate vector, W1 and W2 are the weights of the fully connected layer, δ is the ReLU activation function, σ is the sigmoid activation function, It is a weighted feature map for further classification.

[0079] Through these design strategies, the lightweight model effectively improves the performance of remote sensing scene classification tasks while maintaining low computational complexity and parameter count. The application of these technologies makes the model not only suitable for remote sensing data processing under the few-sample mechanism, but also has good scalability and adaptability.

[0080] Knowledge distillation is a technique for compressing knowledge by allowing a small model (the student model) to learn from a large model (the teacher model). This strategy is particularly valuable in the few-shot regime as it can improve the performance of lightweight models without requiring additional computational resources.

[0081] S3. Leverage pre-trained complex teacher models through the knowledge distillation process To guide students model training, including Pre-trained on the complete enhanced dataset D′ to capture rich scene features; knowledge distillation is achieved through the loss function Implementation, this function combines the traditional classification loss and the knowledge transfer loss between the teacher-student model;

[0082] Figure 2 The architecture diagram of the teacher model and the student model is given.

[0083] The knowledge distillation process and loss function in S3 for:

[0084] Large Teacher Model The GoogleNet model is implemented using the Inception architecture and pre-trained using the remote sensing image AID dataset; lightweight student model Implemented using ShuffleNet; the outputs of the teacher model and the student model are and Represents the category probability prediction for the same input x; the goal of knowledge distillation is to minimize the difference between the output of the student model and the output of the teacher model, usually through the loss function accomplish;

[0085] The loss function of knowledge distillation consists of two parts: hard target loss and soft target loss; hard target loss Directly compare predictions of student models and the true label y i :

[0086]

[0087] Soft target loss Compare the soft labels output by the teacher model and the student model, usually calculated using the temperature-scaled Softmax function:

[0088]

[0089] in, and are the predicted probability distributions of the teacher model and the student model for sample i, represents the unnormalized output of the teacher model for sample i, represents the unnormalized output of the student model for sample i, T is the temperature parameter used to adjust the smoothness of the Softmax output, and KL is the Kullback-Leibler divergence, which is used to measure the two probability distributions p T and p S The differences between

[0090]

[0091] The total loss function is the weighted sum of hard target loss and soft target loss:

[0092]

[0093] Among them, α is the weight parameter used to balance the two losses.

[0094] By minimizing the above loss function, the student model not only learns the correct category labels, but also learns the output distribution of the teacher model. This helps improve the student model's ability to recognize uncertainty and subtle differences, thereby improving the performance of remote sensing scene classification under the few-sample mechanism.

[0095] S4, optimized lightweight model Effectively analyze new remote sensing images new Classification is performed to improve the accuracy and efficiency of remote sensing image classification under the few-sample mechanism.

[0096] In the testing phase, when a new remote sensing image x new When entering the model, after forward propagation, the model Will output its corresponding category y new :

[0097]

[0098] Among them, argmax represents the index corresponding to the maximum value in the probability distribution, and Softmax is a normalization function used to convert the output score of the model into a probability distribution.

[0099] The present invention uses three datasets that are often used in remote sensing scene classification, namely the UCM dataset, the WHU-RS19 dataset and the NWPU-RESISC45 dataset. Detailed information is shown in Table 1. The UCM (University of California, Merced Land Use Dataset) dataset is a public remote sensing image dataset that contains 21 different ground scene categories (such as farmland, forest, highway, etc.), which are extracted from high-resolution aerial photographs of the U.S. National Map Service. WHU-RS19 is another commonly used remote sensing dataset that contains 19 scene categories with diverse image sources, including various natural and artificial environments such as cities, rural areas, forests, and waters. The NWPU-RESISC45 dataset contains 45 categories, each with approximately 700 images, for a total of 31,500 images. The image resolution is 256×256 pixels, covering a wide range of scene types from farmland to industrial areas.

[0100] In the data set used in the present invention, a small number of samples of M-Way-N-Shot under the few-sample learning mechanism are adopted as the training set, where M is the number of minority classes and N is the number of samples in each minority class; the remaining data in the data set is used as the test set.

[0101] Table 1 Details of the UCM dataset, WHU-RS19 dataset, and NWPU-RESISC45 dataset

[0102] Dataset name Number of categories Total number of samples Image pixel size UCM 21 2100 256×256 WHU-RS19 19 1005 600x600 NWPU-RESISC45 45 31500 256×256

[0103] The experiments in this paper were run using PyTorch 2.1.2 on a Windows system equipped with Nvidia RTX A4000 GPU 16GB*2 and Intel(R) Xeon(R) W-2175CPU@2.50GHz.

[0104] Data augmentation uses multi-scale scaling factors of 0.5, 0.8, and 2.0 for scaling at different scales, and random rotations from -60° to 60°. The learning rate for model training is set to 0.001, the batch size for training and validation is 32, and the Adam optimizer is used to optimize model parameters. In the knowledge distillation loss function of the model, the temperature parameter T is set to 3.0, and the weight parameter α of the hard target loss and soft target loss is set to 0.5. The total number of training cycles is 50, and training is stopped when the performance on the validation set does not improve for 10 consecutive cycles.

[0105] In the knowledge distillation, the teacher model uses the GoogleNet model pre-trained on the AID remote sensing image dataset, and the student model uses the architecture of the lightweight model ShuffleNet-v2-x0.5. AID is a new large-scale aerial image dataset that collects sample images from Google Earth images. The AID dataset has 10,000 images in 30 categories.

[0106] In the study of remote sensing scene classification, the present invention uses accuracy as a performance indicator, which indicates the proportion of samples correctly classified by the model to the total number of samples.

[0107] (1) MLLD

[16] : Yuan et al. proposed a framework based on metric learning and local descriptors to improve the classification of remote sensing scenes based on new lenses. MLLD uses task-level training to improve the model's ability to recognize different categories by learning meta-knowledge.

[0108] (2) Model-agnostic

[30] : Finn et al. explicitly trained the model parameters using a small amount of gradient training using a small amount of training data from the new task.

[0109] (3) Matching networks

[31] : Vinyals et al. adopted the idea of ​​metric learning based on deep neural features and the recent progress of neural networks enhanced with external memory. The network maps a small labeled support set and an unlabeled example to its label, eliminating the need for fine-tuning to adapt to new category types.

[0110] (4) Relation network

[32] : Sung proposed a conceptually simple, flexible, and general “few-shot learning” framework, where the classifier must learn to recognize new categories given only a few examples per category. This approach is called relation network (RN) and is trained end-to-end from scratch.

[0111] (5) DLA-MatchNet

[33] : An end-to-end network proposed by Li to improve scene classification of several remote sensing images, called discriminative learning with adaptive matching networks.

[0112] (6) TPN

[34] : Liu et al. proposed a new meta-learning framework for inductive reasoning, the Inductive Propagation Network, which can classify the entire test set at one time, thereby alleviating the problem of low data volume.

[0113] (7)DAKD: A method based on data enhancement and knowledge distillation technology under the few-sample mechanism proposed in this paper.

[0114] We compared the accuracy of the DAKD method proposed in this invention with the six comparison methods, and the results are shown in Table 2. The following conclusions can be drawn from Table 2:

[0115] Table 2 5-Way 1-Shot and 5-Way 5-Shot accuracy (%) results of the proposed method and the baseline method on three datasets UCM, WHU-RS19, and NWPU-RESISC45

[0116]

[0117]

[0118] (1) DAKD shows the best performance under all test conditions, especially on the WHU-RS19 dataset, where it achieves 94.59% accuracy in the 5-Way 5-Shot setting, demonstrating the powerful capabilities of data augmentation and knowledge distillation techniques. The DAKD method has achieved remarkable success in improving the model's generalization ability for new scene classification by combining data augmentation and knowledge distillation techniques. This method is particularly suitable for few-sample learning scenarios and can effectively utilize limited samples for effective learning.

[0119] (2) MLLD performs well on the UCM and WHU-RS19 datasets, especially in the 5-Way5-Shot setting, indicating that the framework based on metric learning and local descriptors is effective in handling remote sensing scene classification tasks. The MLLD method shows that the combination of metric learning and local descriptors can significantly improve the effect of remote sensing scene classification, especially in the scenario of new shots.

[0120] (3) DLA-MatchNet and TPN methods are competitive with DLA-MatchNet in a few cases, but perform poorly in most cases and generally lag behind DAKD methods in other settings. Model-agnostic and TPN methods emphasize explicit training of model parameters and rapid adaptability to new tasks, which is an important consideration in few-shot learning scenarios.

[0121] (4) The performance of model-agnostic, matching networks, and relation network methods fluctuates greatly under different data sets and settings, and is usually lower than that of MLLD and DAKD methods. Matching networks and relation networks improve classification performance by utilizing external memory and learning the relationship between samples, but under the experimental settings of the present invention, their performance is usually inferior to data augmentation and knowledge distillation methods. The performance improvement of model-agnostic and matching networks methods in the 5-Way 5-Shot scenario shows that using more samples for training is crucial to improving the model's ability to adapt to new tasks.

[0122] The present invention first compares the performance differences between not using data enhancement, using traditional data enhancement methods (such as rotation, scaling, flipping) and using the proposed multi-source data enhancement method, and the experimental results are shown in Table 3. The multi-source data enhancement strategy should provide the best accuracy in the 5-Way 5-Shot classification task, proving its effectiveness in processing remote sensing data with complex backgrounds and diverse scenes. The effectiveness of data enhancement is not only reflected in improving the accuracy, but also should include enhancing the robustness of the model so that it can maintain stable performance under a wider range of real-world conditions. No data enhancement usually provides a baseline result, showing the original performance of the model without any external help. Because the model can only use limited data, there is no additional information to help generalization. Traditional data enhancement methods can increase the data diversity in the model training process to a certain extent and improve the model's adaptability to new scenes.

[0123] Table 3 5-Way 5-Shot accuracy (%) results of DAKD method under different data enhancement strategies (no data enhancement, traditional data enhancement and multi-source data enhancement)

[0124] method UCM WHU-RS19 NWPU-RESISC45 No data augmentation 43.27 46.54 49.38 Traditional Data Augmentation 53.20 71.05 62.21 Multi-source data enhancement 86.60 94.59 83.57

[0125] The performance of the models using only lightweight models, using pre-trained models without knowledge distillation, and using knowledge distillation strategies are compared, and the experimental results are shown in Table 4. The knowledge distillation model transfers the knowledge of the complex model to the lightweight model through knowledge distillation based on the pre-trained model. This method aims to combine the high accuracy of the pre-trained model with the computational efficiency of the lightweight model, and achieves higher accuracy compared to other methods. Fine-tuning the pre-trained model is expected to significantly improve the accuracy because these models have learned rich feature representations. The lightweight model provides a baseline result to show the performance of the model when computing resources are limited.

[0126] Table 4 Comparison of the results of lightweight model, pre-trained model, and model after using knowledge distillation strategy on three datasets UCM, WHU-RS19, and NWPU-RESISC45

[0127] method UCM WHU-RS19 NWPU-RESISC45 Lightweight model 20.48 21.24 20.00 Pre-trained models 41.26 52.34 46.57 Model after using knowledge distillation strategy 86.60 94.59 83.57

[0128] The accuracy changes of our proposed DAKD method under 5-way N-shot with N=1, 5, 10, and 15 are compared, as shown in Table 5. It can be found from Table 5 that analyzing the accuracy changes under different shot conditions can evaluate the sensitivity of the model to changes in the number of samples and the elasticity of its learning ability. On the three datasets, as the number of training samples increases, the performance of the model should gradually improve, but the growth rate may be different. N=1Shot has a low accuracy when only one sample is used for training each class, which tests the generalization ability of the model under extreme few-shot learning conditions. N=5Shots As the number of samples increases, the learning ability of the model should improve, and the accuracy is expected to be significantly improved compared to the 1-shot condition, because the model can learn more about each category from more data. Under the conditions of N=10 and 15Shots, the accuracy of the model generally continues to improve. This shows that the performance of the model tends to stabilize and improve with the increase of training samples. The model has a more comprehensive understanding of the characteristics of each category, thereby classifying more accurately. In order to verify the effectiveness of the method proposed by the present invention under the few-sample mechanism, the present invention compares the accuracy with the baseline method under N=5Shots.

[0129] Table 5 Accuracy (%) changes of DAKD method under 5-way N-shot with N=1, 5, 10, 15

[0130] N 1 5 10 15 UCM 63.40 86.60 92.2 96.4 WHU-RS19 89.96 94.59 91.89 93.82 NWPU-RESISC45 73.86 83.57 85.71 89.26

[0131] Only the preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention, and various changes should be included in the protection scope of the present invention.

Claims

1. A remote sensing image classification method based on data enhancement and knowledge distillation, characterized in that: The following steps are involved: S1. By applying multi-scale enhancement technology and multi-angle rotation enhancement technology Expand the original remote sensing image dataset D to obtain an enhanced dataset S2. Design ShuffleNet network architecture lightweight deep learning model Integrate the attention mechanism SE (Squeeze-and-Excitation) module to explicitly model the dependencies between channels and improve the model's sensitivity to important features; S3. Using pre-trained complex teacher models through the knowledge distillation process To guide students model training, including Pre-trained on the complete augmented dataset D′ to capture rich scene features; Knowledge distillation through loss function Implementation, this function combines the traditional classification loss and the knowledge transfer loss between the teacher-student model; S4, optimized lightweight model Effectively analyze new remote sensing images new Classification is performed to improve the accuracy and efficiency of remote sensing image classification under the few-sample mechanism.

2. A remote sensing image classification method based on data enhancement and knowledge distillation according to claim 1, characterized in that: The multi-scale enhancement technology in S1 and multi-angle rotation enhancement technology for: Multi-scale enhancement By adjusting the size of the image to simulate remote sensing images captured from different heights, the model's adaptability to scale changes can be enhanced. Given an original image x i , multi-scale enhancement is expressed as: in, is the image scaling function, s is the scaling factor, which can take a series of predefined values ​​to generate images of different scales; Multi-angle rotation enhancement By rotating the image at different angles, the diversity of viewing angles can be increased. Given an original image x i , multi-angle rotation enhancement can be expressed as: in, is the image rotation function, θ is the rotation angle, and a random angle is usually selected to increase data diversity.

3. The remote sensing image classification method based on data enhancement and knowledge distillation according to claim 1, characterized in that: The ShuffleNet network architecture in S2 is a lightweight deep learning model for: Lightweight model The efficient network architecture ShuffleNet is adopted. For remote sensing image data, the input of the model is x∈R H×W×C , where H, W and C represent the height, width and number of channels of the image respectively; the model output is a category probability distribution Where θ represents the model parameters, p∈R K The predicted probabilities corresponding to K categories; Depthwise separable convolution decomposes the traditional convolution operation into two steps: depthwise convolution and pointwise convolution. Given an input feature map F∈R H′×W′×C′ , where R indicates that F belongs to the real number domain, H′ indicates the height of the feature map, W′ indicates the width of the feature map, and C′ indicates the number of channels of the feature map; Depthwise separable convolution first applies a depthwise convolution F. depthwise =F depth (F,K depth ), where F depth is the feature map after deep convolution operation, K depth is the depthwise convolution kernel; then the pointwise convolution operation F is applied pointwise =F point (F depthwise , K point ), where F point is a function representing the point-by-point convolution operation, which is outputted at the depth convolution F depthwise Based on this, we use the point-by-point convolution kernel K point Perform convolution.

4. The remote sensing image classification method based on data enhancement and knowledge distillation according to claim 1, characterized in that: The integrated attention mechanism SE (Squeeze-and-Excitation) module in S2 is: The integrated attention mechanism SE module first performs pointwise Perform global average pooling to obtain a channel descriptor, and recalibrate the weight of each channel through the fully connected layer: z=F avg (F pointwise ) e=σ(W2δ(W1z)) Among them, z is the compressed feature, F avg is the global average pooling operation, e is the intermediate vector, W1 and W2 are the weights of the fully connected layer, δ is the ReLU activation function, σ is the sigmoid activation function, It is a weighted feature map for further classification.

5. The remote sensing image classification method based on data enhancement and knowledge distillation according to claim 1, characterized in that: The knowledge distillation process and loss function in S3 for: Large Teacher Model The GoogleNet model of the Inception architecture is used for pre-training using the remote sensing image AID dataset; the lightweight student model S is implemented using ShuffleNet; the outputs of the teacher model and the student model are and Represents the category probability prediction for the same input x; the goal of knowledge distillation is to minimize the difference between the output of the student model and the output of the teacher model, usually through the loss function accomplish; The loss function of knowledge distillation consists of two parts: Composition: hard target loss and soft target loss; hard target loss Directly compare predictions of student models and the true label y i : Soft target loss Compare the soft labels output by the teacher model and the student model, usually calculated using the temperature-scaled Softmax function: in, and are the predicted probability distributions of the teacher model and the student model for sample i, represents the unnormalized output of the teacher model for sample i, represents the unnormalized output of the student model for sample i, T is the temperature parameter used to adjust the smoothness of the Softmax output, and KL is the Kullback-Leibler divergence, which is used to measure the two probability distributions p T and p s The differences between The total loss function is the weighted sum of hard target loss and soft target loss: Among them, α is the weight parameter used to balance the two losses.

6. The remote sensing image classification method based on data enhancement and knowledge distillation according to claim 1, characterized in that: The S4 is an optimized lightweight model Effectively analyze new remote sensing images new Classify as: In the testing phase, when a new remote sensing image x new When entering the model, after forward propagation, the model Will output its corresponding category y new : Among them, argmax represents the index corresponding to the maximum value in the probability distribution, and Softmax is a normalization function used to convert the output score of the model into a probability distribution.

Images