A model training and method for remote sensing scene classification based on collaborative learning

Abstract

The embodiment of the application provides a model training and method for remote sensing scene classification based on cooperative learning, local features and long-distance features are extracted through a convolutional neural network and a Vision Transformer respectively; a cross-feature calibration module (CFC) is used for fusing the local features and the long-distance features; a joint loss is used to further improve the performance of the fused features and a double-branch feature extractor; a two-stage semi-supervised training strategy is used to excavate the performance of unlabeled data; and a classifier is used for classifying the fused features. The classification effect is improved by constructing more discriminative representation features of remote sensing scenes.

Description

Technical Field

[0001] This invention relates to the field of remote sensing detection, and more particularly to a model training and method for remote sensing scene classification based on collaborative learning. Background Technology

[0002] Land use / land cover information interpretation is an important research area in remote sensing. However, most previous studies focused on land cover, with only a few addressing the representativeness of land use. In recent years, with the development of high-resolution satellite sensors, remote sensing interpretation has gradually evolved from pixel-level and object-level to scene-level, aiming to acquire higher-level semantic information. Therefore, remote sensing scene classification has received more attention because it can be used for land use classification, urban functional zone identification, and other related fields. However, due to the complexity and large-scale variation of geographic objects in high-resolution remote sensing scenes, extracting more discriminative features from the scene remains an important and challenging task.

[0003] Commonly used features can be categorized into two main types: handcrafted features and deep features. Handcrafted features include low-level features such as spectral and texture data, as well as mid-level features based on visual word packets. Compared to handcrafted features, deep features are more abstract but contain richer semantic information. In particular, convolutional neural networks have achieved a dominant role and state-of-the-art performance in scene classification.

[0004] Because convolutional neural networks rely on local convolutional kernels, they possess excellent local feature representations but struggle with representing long-range information in images. Several methods, such as feature pyramids and multi-scale strategies, have been proposed to overcome this problem. However, they still have limitations in addressing this issue. Recently, the Transformer, a widely used architecture in natural language processing, has been increasingly applied to image processing. It can directly extract long-range information from images, providing a new perspective for scene classification in remote sensing images.

[0005] Vision Transformers can capture long-range features that are difficult to model using convolutional neural networks. However, due to their origins in natural language processing, using them in image processing still has some drawbacks. First, it directly expands the image into a one-dimensional vector, which is ineffective for modeling the local structural information of the image. Second, redundant attention modules introduce computational burden. In summary, convolutional neural networks are good at representing local structural features but struggle to capture long-range information, while Vision Transformers excel at extracting long-range information but tend to overlook local features. Therefore, it is extremely important to study how to effectively combine convolutional neural networks and Vision Transformers. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a model training and method for remote sensing scene classification based on collaborative learning. It combines convolutional neural networks and Vision Transformer to construct a remote sensing scene classification based on local and long-distance collaborative learning strategies.

[0007] In a first aspect, the present invention provides a model training method for remote sensing scene classification based on collaborative learning, the method comprising:

[0008] Obtain the unlabeled sample set and the labeled sample set;

[0009] The labeled sample set is input into the remote sensing scene classification model, and the first joint loss function is used for the first stage of training to obtain a local classifier, a long-range classifier, and a fusion feature classifier.

[0010] The local classifier, long-range classifier, and fusion feature classifier are used to label and filter the unlabeled sample set to obtain the augmented sample set.

[0011] The labeled sample set and the amplified sample set are input into the remote sensing scene classification model, and the second joint loss function is used for the second stage of training to obtain the trained remote sensing scene classification model.

[0012] Wherein, the first joint loss function is determined based on the deep supervision loss function and the prediction loss function; the deep supervision loss function is obtained based on the local features and long-range features of the remote sensing image, and the prediction loss function is obtained based on the fused features of the remote sensing image; the local features and long-range features are obtained by extracting from the remote sensing image; the fused features are obtained by fusing the local features and long-range features.

[0013] The second joint loss function is determined based on the deep supervision loss function, the deep mutual learning loss function, and the prediction loss function. The deep mutual learning loss function is obtained based on the deep mutual learning loss of local-long-distance features, local-fusion features, and fusion-long-distance feature branches.

[0014] On the other hand, embodiments of the present invention provide a method for remote sensing scene classification based on collaborative learning, the method comprising:

[0015] Acquire remote sensing image data;

[0016] The remote sensing image data is input into a dual-branch feature extractor built using a convolutional neural network and a Vision Transformer to obtain the local features and long-range features of the remote sensing image data, respectively.

[0017] The local features and long-range features are input into the cross-feature calibration module for fusion to obtain the fused features of the remote sensing image data;

[0018] The local features, long-range features, and fused features are respectively input into the local classifier, the long-range classifier, and the fused feature classifier. The fused features are then classified to obtain the remote sensing scene classification result of the remote sensing image data.

[0019] On the other hand, embodiments of the present invention provide a remote sensing scene classification system based on collaborative learning, the system comprising:

[0020] The input module is used to receive and acquire remote sensing image data, an unlabeled sample set, and a labeled sample set;

[0021] The network structure module includes a dual-branch feature extractor and a cross-feature calibration module. The dual-branch feature extractor is built based on a convolutional neural network and a Vision Transformer, and is used to extract local features and long-range features from the input remote sensing image data. The cross-feature calibration module is used to fuse the local features and long-range features into the cross-feature calibration module to obtain fused features.

[0022] A joint loss module is provided to determine a first joint loss function and a second joint loss function. The first joint loss function is determined by a deep supervision loss function and a prediction loss function. The deep supervision loss function is obtained based on the deep supervision loss of local features and long-range features of the remote sensing image, and the prediction loss function is obtained based on the prediction loss of the fused features of the remote sensing image. The local features and long-range features are extracted from the remote sensing image, and the fused features are obtained by fusing the local features and long-range features. The second joint loss function is determined by the deep supervision loss function, the deep mutual learning loss function, and the prediction loss function. The deep mutual learning loss function is obtained based on the deep mutual learning loss of the local-long-range feature, local-fused feature, and fused-long-range feature branches.

[0023] A two-stage semi-supervised training strategy module is used to train a remote sensing scene classification model. The training includes inputting a labeled sample set into the remote sensing scene classification model, performing a first-stage training using a first joint loss function to obtain a local classifier, a long-range classifier, and a fusion feature classifier; using the local classifier, long-range classifier, and fusion feature classifier to label and filter the unlabeled sample set to obtain an augmented sample set; inputting the labeled sample set and the augmented sample set into the remote sensing scene classification model, performing a second-stage training using a second joint loss function to obtain the trained remote sensing scene classification model.

[0024] The output module is used to output the remote sensing scene classification results of the remote sensing image data.

[0025] On the other hand, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, characterized in that the processor, when executing the computer program, implements the above-described evaluation method for algorithm implementation in chip design.

[0026] On the other hand, embodiments of the present invention provide a computer-readable medium having processor-executable non-volatile program code, characterized in that the program code causes the processor to execute the above-described evaluation method for algorithm implementation in chip design.

[0027] Compared to existing technologies, this invention provides a remote sensing scene classification method based on local and long-range collaborative learning. It extracts local and long-range features using a convolutional neural network and a Vision Transformer, respectively; fuses the local and long-range features using a cross-feature calibration module (CFC); further improves the performance of the fused features and the dual-branch feature extractor using joint loss; leverages a two-stage semi-supervised training strategy to mine the performance of unlabeled data; and classifies the fused features based on a classifier. The classification effect is improved by constructing more discriminative representation features of the remote sensing scene. Attached Figure Description

[0028] Figure 1 This is a diagram illustrating the overall architecture of a remote sensing scene classification model based on local and long-range collaborative learning.

[0029] Figure 2 A flowchart illustrating a model training method for remote sensing scene classification based on local and long-range collaborative learning;

[0030] Figure 3 This is a schematic diagram of the cross-feature calibration module of a remote sensing scene classification model based on local and long-range collaborative learning.

[0031] Figure 4 A schematic diagram illustrating the process of obtaining fused features based on local and long-range features using the cross-feature calibration module;

[0032] Figure 5 This is a schematic diagram of deep mutual learning in the training method of remote sensing scene classification model based on local and long-distance collaborative learning;

[0033] Figure 6 A classification confusion matrix diagram of the AID dataset in the training method of remote sensing scene classification model based on local and long-range collaborative learning;

[0034] Figure 7 A flowchart illustrating a remote sensing scene classification method based on local and long-range collaborative learning;

[0035] Figure 8 This is a system architecture diagram for remote sensing scene classification based on local and long-distance collaborative learning. Detailed Implementation

[0036] To address the problems existing in the prior art, this invention provides a model training and method for remote sensing scene classification based on collaborative learning. It combines convolutional neural networks and Vision Transformer to construct a remote sensing scene classification system based on local and long-range collaborative learning strategies. Figure 1 The figure shows the overall architecture of a remote sensing scene classification model based on local and long-range collaborative learning. The basic framework of the proposed remote sensing scene classification model consists of three parts: a network structure part, a multi-layer joint loss part, and a two-stage semi-supervised training part. The network structure part includes a bi-branch feature extractor and a cross-feature calibration module. Deep supervision and deep mutual learning are combined to form a joint loss, further improving the fused features and enhancing the bi-branch feature extractor. In the two-stage semi-supervised training, the first training stage is based on labeled samples, obtaining expanded labeled samples through self-labeling; the second training stage is based on the labeled samples and the expanded labeled samples; finally, the inference model is obtained by pruning the local classifier and the long-range classifier. Furthermore, in the embodiments disclosed in this invention, deep mutual learning is not applied in the first training stage.

[0037] On the one hand, the present invention provides a model training method for remote sensing scene classification based on collaborative learning. The remote sensing scene classification model includes at least a local classifier, a long-range classifier, and a fusion feature classifier.

[0038] Figure 2 This is a flowchart of a model training method for remote sensing scene classification based on local and long-range collaborative learning. The method includes:

[0039] S210: Obtain the unlabeled sample set and the labeled sample set

[0040] In one implementation, given an unlabeled sample x u ∈ U, where U represents the unlabeled sample set, and given a labeled sample set S 1b .

[0041] S220: First-stage training of the remote sensing scene classification model using the first joint loss function.

[0042] In one implementation, the labeled sample set is input into the remote sensing scene classification model, and a first-stage training is performed using a first joint loss function to obtain a local classifier, a long-range classifier, and a fusion feature classifier.

[0043] The first joint loss function is determined based on the deep supervision loss function and the prediction loss function. The deep supervision loss function is obtained based on the local and long-range features of the remote sensing image; the prediction loss function is obtained based on the fused features of the remote sensing image.

[0044] The specific implementation is as follows:

[0045] In the embodiments disclosed in this invention, local features and long-range features are obtained by extracting remote sensing image data. The specific extraction method is as follows:

[0046] Acquire remote sensing image data, and input the remote sensing image data into the dual-branch feature extractor to extract local features and long-range features respectively.

[0047] In one possible embodiment, the dual-branch feature extractor is built using a convolutional neural network and a VisionTransformer;

[0048] In another possible embodiment, the dual-branch feature extractor is built using ResNet and DeiT as the underlying networks.

[0049] In the embodiments disclosed in this invention, the fused features are obtained by fusing local features and long-range features into a cross-feature calibration module graph. Figure 3 This is a diagram of a cross-feature calibration module for remote sensing scene classification methods based on local and long-range collaborative learning. Figure 4 The diagram illustrates the process of obtaining fused features based on local and long-range features using a cross-feature calibration module. The fusion method is as follows:

[0050] S221: Stack local features and long-range features to obtain initial stacked features.

[0051] First, the local features and long-range features are stacked as follows:

[0052] z concat =Concat(z lc ,z lr )

[0053] Where z lc and z lr These represent local features and long-range features, respectively, with z_concat representing the initial stacked features.

[0054] S222: Input the initial stacked features into the nonlinear unit to obtain the calibration weights of the initial stacked features.

[0055] Then, the calibration weights W of the initial stacked features are obtained by inputting the initial features obtained in step S222 into the nonlinear unit. The nonlinear unit is constructed based on two fully connected layers including a dimensionality reduction layer fc1 and a dimensionality increase layer fc2, as well as ReLU and Sigmoid activation functions. The specific implementation is as follows:

[0056] w=σ(w2δ(w1z concat ))

[0057] The dimensionality reduction ratio r is used to control the value of d:

[0058] d = max(H / r, L) dn )

[0059] Where w1∈R d×H and w2∈R H×d Let fc1 and fc2 represent the weight parameters respectively, and H be the weight of z. concat The dimension of L, where δ and σ represent the ReLU and Sigmoid activation functions respectively, and L dn This represents the minimum value of d.

[0060] The nonlinear unit is constructed based on two fully connected layers, including a dimensionality reduction layer and a dimensionality increase layer, as well as ReLU and Sigmoid activation functions.

[0061] S223: Obtain fused features by correcting the initial stacked features using calibration weights.

[0062] Finally, the initial stacked features are corrected using the calibration weights obtained in step S223 to obtain the fused features, and the fused features z fs By multiplying by w and z concat By controlling the calibration weights, well-represented features are further enhanced, while some redundant features are reduced.

[0063] In one implementation, a joint loss framework including a first joint loss function and a second joint loss function is designed for training a remote sensing scene classification model.

[0064] The joint loss framework consists of three losses: deep supervision loss L DS Intra-network deep mutual learning loss L DML The prediction loss L based on the final features (i.e., fused features) PL The deep supervision loss and prediction loss are calculated using cross-entropy loss, while the deep mutual learning loss is calculated using L2 loss (Least Squares Error, LSE). Figure 5 This is a schematic diagram of deep mutual learning in the training method of remote sensing scene classification model based on local and long-range collaborative learning. The specific implementation is as follows:

[0065] First, based on the predictions made by local and long-range classifiers using local and long-range features, a deep supervision loss is obtained, calculated using local and long-range features. Then, using cross-entropy, the deep supervision loss function is derived. The specific implementation is as follows:

[0066] In one implementation, an auxiliary classifier with pre-defined local features, long-range features, and fused features is used. L is calculated based on the labels using the prediction results of the auxiliary classifier. DS Deeply supervised loss. The mathematical expression for the local branch is:

[0067]

[0068] The mathematical expression for long-distance branching is:

[0069]

[0070] Where χ represents the sample set, and K and M represent the number of samples and the number of classes, respectively. and Let w represent the depth supervision loss for local branches and long-range branches, respectively. lc and w l These represent the parameters of the local branch and long-distance branch networks, respectively. and The parameter that links the feature to the prediction (local features, parameters of the long-range auxiliary classifier), the function p represents the probability that sample k belongs to class m, and I{·} is an indicator function (returns 1 if the label of sample k is equal to m; otherwise, it returns 0).

[0071] Based on the depth supervision loss of the local branches and long-distance branches, and using cross-entropy to determine the depth supervision loss function, the mathematical expression of the depth supervision loss function is as follows:

[0072]

[0073] Then, using cross-entropy, the prediction loss function is obtained by using the fusion auxiliary classifier to predict the labeled samples based on the fusion features. The specific implementation is as follows:

[0074] In one implementation, the deep mutual learning loss function uses L2 loss, and the prediction loss function L... PL The mathematical expression is:

[0075]

[0076] in This represents the parameters of the auxiliary classifier based on fused features.

[0077] Finally, based on the L2 loss, deep mutual learning losses for local-fusion feature, long-distance-fusion feature, and local-long-distance feature branches are obtained based on local features and fused features, long-distance features and fused features, and local features and long-distance features, respectively.

[0078] Based on the deep mutual learning loss of local-fusion features, long-range-fusion features, and local-long-range feature branches, a deep mutual learning loss function is obtained.

[0079] In one implementation, the mathematical expression for the deep mutual learning loss with local-fusion feature and long-range-fusion feature branches is:

[0080]

[0081]

[0082] Where W represents the parameters of the bi-branch feature extractor and the cross-feature calibration module. The deep mutual learning loss with local-long-range feature branches can be expressed as:

[0083]

[0084] Through the above implementation, the three components can learn from each other to mitigate their shortcomings and improve the framework's representation. Deep mutual learning loss L... DML It can be represented as:

[0085]

[0086] The first joint loss function is determined based on the deep supervision loss function and the prediction loss function. As shown above, the mathematical expression of the first joint loss function is:

[0087]

[0088] The second joint loss function is determined based on the deep supervision loss function, the deep mutual learning loss function, and the prediction loss function. As shown above, the mathematical expression of the second joint loss function is:

[0089]

[0090] After determining the first joint loss function, the labeled sample set S obtained in step S210 is... 1b Input the remote sensing scene classification model and use the first joint loss function for the first stage of training. Through training, adjust the parameters of the preset local features, long-distance features and fused features auxiliary classifiers to obtain the local classifier, long-distance classifier and fused feature classifier.

[0091] S230: Use three classifiers to label and filter the unlabeled sample set to obtain the augmented sample set.

[0092] In one implementation, a simple self-labeling strategy using a radix-3 classifier is designed to label and filter the unlabeled sample set, resulting in an augmented sample set. The specific implementation is as follows:

[0093] The local classifier, long-range classifier, and fusion feature classifier obtained in step S220 are used to obtain the predicted labels and class scores of the step.

[0094] The predicted labels are then assigned to the unlabeled samples U obtained from S210, which can be added to the amplified sample set S with high confidence. el The labeled sample set includes predicted labels and class scores for local features, long-range features, and fused features.

[0095] The specific implementation is as follows:

[0096] y n ,p n =f n (x u )

[0097] Among them, y n and p n These are the predicted label and category scores, where n represents local, long-range, and fusion.

[0098] For sample selection, it's necessary to determine whether the predicted values ​​meet the high confidence criteria. This is done by checking different classifiers to confirm consistency, i.e., whether they provide the same predicted labels based on local features, long-range features, and fused features. The three categories need to meet the following criteria:

[0099] min(p1,p2,p3)≥λ

[0100] Where λ is a constant, and P1, P2, and P3 represent the class scores based on local features, long-range features, and fused features, respectively. Samples satisfying U are added to the amplified sample set S. el .

[0101] S240: The trained remote sensing scene classification model is obtained by training using the second joint loss function.

[0102] The labeled sample set and the augmented sample set are input into the remote sensing scene classification model, and the second joint loss function is used for the second stage of training to obtain the trained remote sensing scene classification model.

[0103] In one implementation, S 1b and S elTogether, they serve as input to the model, and the second joint loss determined in step S220 is used to train the remote sensing scene classification model in the second stage, finally obtaining the trained remote sensing scene classification model. Furthermore, to reduce the size of the inference model, the local classifier and long-range classifier of the remote sensing scene classification model are pruned.

[0104] Figure 6 The figure shows the classification confusion matrix of the AID dataset in the training method of remote sensing scene classification model based on local and long-range collaborative learning. The first and second joint loss functions optimize the performance of the remote sensing scene classification model by enhancing the dual-branch feature extractor and further improving the fusion features. Specifically, they include:

[0105] The first and second joint loss functions enhance the performance of the remote sensing scene classification model in three ways:

[0106] 1) Supervised loss helps the branch feature extractor obtain more discriminative features;

[0107] 2) Intra-network deep mutual learning loss helps local branches (convolutional neural network branches) and long-range branches (Vision Transformer branches) compensate for their shortcomings and enhance fusion features;

[0108] 3) Deep supervision loss and deep mutual learning loss within the network are complementary. On the one hand, deep mutual learning can supplement deep supervision information. On the other hand, deep supervision can help correct erroneous knowledge that may be introduced by deep mutual learning.

[0109] On the other hand, embodiments of the present invention provide a method for remote sensing scene classification based on collaborative learning, which classifies remote sensing scenes using a remote sensing scene classification model based on collaborative learning. Figure 7 This is a flowchart of a remote sensing scene classification method based on local and long-range collaborative learning. The method includes:

[0110] S710: Acquire remote sensing image data;

[0111] S720: Input the remote sensing image data into a dual-branch feature extractor built using a convolutional neural network and a Vision Transformer to obtain the local features and long-range features of the remote sensing image data, respectively.

[0112] S730: Input the local features and long-range features into the cross-feature calibration module for fusion to obtain the fused features of the remote sensing image data;

[0113] S740 inputs the local features, long-range features, and fused features into the local classifier, long-range classifier, and fused feature classifier, respectively, and classifies the fused features to obtain the remote sensing scene classification result of the remote sensing image data.

[0114] In one implementation, the remote sensing image data is input into a remote sensing scene classification model based on collaborative learning to obtain the remote sensing scene classification result of the remote sensing image data.

[0115] On the other hand, the present invention provides a remote sensing scene classification system based on collaborative learning. Figure 8 This is a system architecture diagram for remote sensing scene classification based on local and long-range collaborative learning. The system includes:

[0116] S810: Input Module

[0117] The input module is used to receive and acquire remote sensing image data, unlabeled sample sets, and labeled sample sets;

[0118] S820: Network Structure Module

[0119] The network structure module includes a dual-branch feature extractor and a cross-feature calibration module. The dual-branch feature extractor, based on a convolutional neural network and a Vision Transformer, is used to extract local features and long-range features from the input remote sensing image data. The cross-feature calibration module is used to fuse the local features and long-range features to obtain fused features.

[0120] S830: Joint Loss Module

[0121] The joint loss module is used to determine a first joint loss function and a second joint loss function. The first joint loss function is determined by a deep supervision loss function and a prediction loss function. The deep supervision loss function is obtained based on the deep supervision loss of local features and long-range features of the remote sensing image, and the prediction loss function is obtained based on the prediction loss of the fused features of the remote sensing image. The local features and long-range features are obtained by extracting from the remote sensing image, and the fused features are obtained by fusing the local features and long-range features. The second joint loss function is determined by the deep supervision loss function, the deep mutual learning loss function, and the prediction loss function. The deep mutual learning loss function is obtained based on the deep mutual learning loss of the local-long-range feature, local-fused feature, and fused-long-range feature branches.

[0122] S840: Two-stage semi-supervised training strategy module

[0123] A two-stage semi-supervised training strategy module is used to train a remote sensing scene classification model. The training includes inputting a labeled sample set into the remote sensing scene classification model, performing a first-stage training using a first joint loss function to obtain a local classifier, a long-range classifier, and a fusion feature classifier; using the local classifier, long-range classifier, and fusion feature classifier to label and filter the unlabeled sample set to obtain an augmented sample set; inputting the labeled sample set and the augmented sample set into the remote sensing scene classification model, performing a second-stage training using a second joint loss function to obtain the trained remote sensing scene classification model.

[0124] S850: Output Module

[0125] The output module is used to output the remote sensing scene classification results of the remote sensing image data.

[0126] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0127] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0128] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A model training method for remote sensing scene classification based on collaborative learning, wherein the model includes at least a local classifier, a long-range classifier, and a feature fusion classifier, characterized in that, The method includes: Obtain the unlabeled sample set and the labeled sample set; The labeled sample set is input into the remote sensing scene classification model, and the first joint loss function is used for the first stage of training to obtain a local classifier, a long-range classifier, and a fusion feature classifier. The local classifier, long-range classifier, and fusion feature classifier are used to label and filter the unlabeled sample set to obtain the augmented sample set. The labeled sample set and the amplified sample set are input into the remote sensing scene classification model, and the second joint loss function is used for the second stage of training to obtain the trained remote sensing scene classification model. Wherein, the first joint loss function is determined based on the deep supervision loss function and the prediction loss function; the deep supervision loss function is obtained based on the local features and long-range features of the remote sensing image, and the prediction loss function is obtained based on the fused features of the remote sensing image; the local features and long-range features are obtained by extracting from the remote sensing image; the fused features are obtained by fusing the local features and long-range features. The second joint loss function is determined based on the deep supervision loss function, the deep mutual learning loss function, and the prediction loss function. The deep mutual learning loss function is obtained based on the deep mutual learning loss of local-long-distance features, local-fusion features, and fusion-long-distance feature branches.

2. The method according to claim 1, characterized in that, The methods for obtaining the deep supervision loss function and the prediction loss function include: Based on the prediction of labeled samples by local and long-range classifiers using local and long-range features, the deep supervision loss of local and long-range features is obtained. The deep supervision loss function is obtained by using cross-entropy based on the deep supervision loss of local and long-range features. Using cross-entropy, a prediction loss function is obtained by using a fusion classifier to predict labeled samples based on fusion features.

3. The method according to claim 1, characterized in that, The local and long-range features are obtained by extracting from remote sensing images, and the extraction includes: Remote sensing image data is acquired, and a dual-branch feature extractor is built using a convolutional neural network and a Vision Transformer. The dual-branch feature extractor is used to extract the local features and long-range features from the remote sensing image data.

4. The method according to claim 3, characterized in that, The method of building a dual-branch feature extractor using convolutional neural networks and VisionTransformer also includes: A dual-branch feature extractor is built using ResNet and DeiT as the base networks.

5. The method according to claim 1, characterized in that, The fusion feature is obtained by fusing local features and long-range features, and the fusion includes: The local features and long-range features are stacked to obtain initial stacked features; The initial stacked features are input into a nonlinear unit to obtain the calibration weights of the initial stacked features; the nonlinear unit is constructed based on two fully connected layers including a dimensionality reduction layer and a dimensionality increase layer, as well as ReLU and Sigmoid activation functions; The initial stacked features are corrected according to the calibration weights to obtain the fused features.

6. The model training method according to claim 1, characterized in that, The methods for obtaining the deep mutual learning loss function include: Based on L2 loss, deep mutual learning losses for local-fusion feature, long-distance-fusion feature, and local-long-distance feature branches are obtained, respectively, based on local features and fused features, long-distance features and fused features, and local features and long-distance features. Based on the deep mutual learning loss of local-fusion features, long-range-fusion features, and local-long-range feature branches, a deep mutual learning loss function is obtained.

7. A method for remote sensing scene classification based on collaborative learning, using a model trained according to any one of claims 1-6, comprising: Acquire remote sensing image data; The remote sensing image data is input into a dual-branch feature extractor built using a convolutional neural network and a Vision Transformer to obtain the local features and long-range features of the remote sensing image data, respectively. The local features and long-range features are input into the cross-feature calibration module for fusion to obtain the fused features of the remote sensing image data; The local features, long-range features, and fused features are respectively input into the local classifier, the long-range classifier, and the fused feature classifier. The fused features are then classified to obtain the remote sensing scene classification result of the remote sensing image data.

8. A remote sensing scene classification system based on collaborative learning, applied to the method described in any one of claims 1-7, characterized in that, The system includes: The input module is used to receive and acquire remote sensing image data, an unlabeled sample set, and a labeled sample set; The network structure module includes a dual-branch feature extractor and a cross-feature calibration module. The dual-branch feature extractor is built based on a convolutional neural network and a Vision Transformer, and is used to extract local features and long-range features from the input remote sensing image data. The cross-feature calibration module is used to fuse the local features and long-range features into the cross-feature calibration module to obtain fused features. A joint loss module is provided to determine a first joint loss function and a second joint loss function. The first joint loss function is determined by a deep supervision loss function and a prediction loss function. The deep supervision loss function is obtained based on the deep supervision loss of local features and long-range features of the remote sensing image, and the prediction loss function is obtained based on the prediction loss of the fused features of the remote sensing image. The local features and long-range features are extracted from the remote sensing image, and the fused features are obtained by fusing the local features and long-range features. The second joint loss function is determined by the deep supervision loss function, the deep mutual learning loss function, and the prediction loss function. The deep mutual learning loss function is obtained based on the deep mutual learning loss of the local-long-range feature, local-fused feature, and fused-long-range feature branches. A two-stage semi-supervised training strategy module is used to train a remote sensing scene classification model. The training includes inputting a labeled sample set into the remote sensing scene classification model, performing a first-stage training using a first joint loss function to obtain a local classifier, a long-range classifier, and a fusion feature classifier; using the local classifier, long-range classifier, and fusion feature classifier to label and filter the unlabeled sample set to obtain an augmented sample set; inputting the labeled sample set and the augmented sample set into the remote sensing scene classification model, performing a second-stage training using a second joint loss function to obtain the trained remote sensing scene classification model. The output module is used to output the remote sensing scene classification results of the remote sensing image data.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the method described in any one of claims 1 to 7.

10. A computer-readable medium having processor-executable non-volatile program code, characterized in that, The program code causes the processor to execute the method according to any one of claims 1 to 7.

Images