Remote sensing scene image classification method based on re-parameterized feature aggregation convolutional neural network

Abstract

The invention discloses a remote sensing scene image classification method based on a re-parameterized feature aggregation convolutional neural network, and relates to a remote sensing scene image classification method. The objective of the invention is to solve the problems of secondary complexity, large calculation amount and low image classification accuracy of a self-attention module in an existing Transform structure. The method comprises the following steps: constructing a re-parameterized feature aggregation convolutional neural network model; the model classifies remote sensing scene images to be detected; the model comprises a first feature extraction module, a first RepFormer, a first average pooling layer, a second RepFormer, a second average pooling layer, a third RepFormer, a third average pooling layer, a fourth RepFormer, a first linear layer, a second feature extraction module, a third feature extraction module, a first multi-head hybrid convolution module, a second multi-head hybrid convolution module, a first cascade layer and a fourth feature extraction module. A third multi-head hybrid convolution module, a second linear layer, a second cascade layer and a feature fusion classifier; the method is applied to the field of remote sensing scene image classification.

Description

Technical Field

[0001] The invention relates to a remote sensing scene image classification method. Background Art

[0002] With the development of high-resolution satellite sensors, remote sensing scene image classification has received increasing attention. It is of irreplaceable importance in interpreting earth surface information, urban planning, agricultural monitoring [1]-[4], etc. Due to the complex background and multi-scale objects in high-resolution remote sensing scenes, how to effectively extract features remains an important and challenging task.

[0003] Traditional remote sensing image classification methods mainly rely on manually designed features (e.g., texture features [5][6], shape features [7][8]) and machine learning classifiers (support vector machines [9] and decision trees

[10] ). These methods usually rely on a large amount of domain expert knowledge, and especially when dealing with complex remote sensing scenes, it is often difficult to capture rich feature information. Although such methods have successfully solved the remote sensing image classification problem to a certain extent, their performance is gradually limited as the data size increases and the complexity of the task increases.

[0004] The rise of deep learning has enabled convolutional neural networks to achieve remarkable success in the field of image processing. The introduction of convolutional layers enables the network to effectively capture local features of images, while reducing the number of parameters through the weight sharing mechanism, thereby improving the computational efficiency of the model. This design greatly enhances the neural network's ability to abstract and express image features, enabling it to perform well in tasks such as image classification

[11] and object detection

[12] .

[0005] Convolutional neural networks (CNNs) are outstanding in extracting features and are widely used in remote sensing scene image classification. Shi et al.

[13] introduced a branch feature fusion convolutional neural network that fuses feature information extracted by two independent branches and uses depthwise separable convolution to reduce model complexity. Zhao et al.

[14] proposed a structure that integrates global texture features, local structural features, and spectral features. Li et al.

[15] combined transfer learning and convolutional neural networks to generate robust visual features, thereby improving classification accuracy in the case of limited data labels. Singh et al.

[16] introduced a heterogeneous convolution method that uses two types of convolution kernels, where k×k convolution kernels are used for some channels and 1×1 convolution kernels are used for the remaining channels, and the ratio between channels is adjusted by the hyperparameter p. It aims to improve image processing performance by optimizing the convolution structure. Chen et al.

[17] proposed a dynamic convolution attention mechanism that dynamically aggregates multiple small-sized convolution kernels to enhance feature representation and computational efficiency in a nonlinear manner without increasing the depth and width of the network. Liu et al.

[18] proposed self-calibrated convolution, which adaptively establishes the long-range spatial and channel dependencies of each spatial position, and generates more discriminative features through self-calibration operation. The octave convolution proposed by Chen et al.

[19] divides the input features into high-frequency and low-frequency features from the frequency perspective, and controls the ratio of the two through the hyperparameter α to reduce the number of parameters and calculations, and effectively improve the feature expression ability. Han et al.

[20] improved the traditional convolution and proposed ghost convolution. This method extracts feature information through traditional convolution and uses linear transformation to generate redundant information, thereby reducing the computational complexity of the model. Compared with traditional convolution, the convolution parameters of ghost convolution are no longer shared for all samples. Yang et al.

[21] proposed conditional parameterized convolution, which obtains customized convolution kernels for each input sample in each batch to improve the model capacity and maintain efficient operation speed. Cao et al.

[22] combined depthwise separable convolution with traditional convolution and proposed depthwise separable over-parameterized convolution, which provides new ideas for research and application in the field of image processing. Xie et al.

[23] improved ResNet by introducing the idea of ​​group convolution and proposed ResNext. Zhang et al.

[24] proposed ShuffleNet, which is a generalized form of group convolution. ShuffleNet introduced a channel shuffling operation based on group convolution, which effectively solved the problem of insufficient information exchange between different groups caused by group convolution. Wu et al.

[25] successfully improved the feature representation ability of the network by grouping the input along the channel dimension and using channel attention to readjust the features of each group. Liu et al.

[26] proposed a lightweight hybrid group convolutional neural network that combines traditional convolution and dilated convolution of different groups. The information exchange of convolution fusion features is achieved through channel rearrangement operation, thereby improving network performance. Shen et al.

[27] adopted a group attention fusion strategy to improve the classification performance of the network.

[0006] The present invention roughly divides the existing CNN-based methods into three groups. The first group of methods focuses on learning global visual features for remote sensing (RS) scene classification. For example, Li et al.

[28] adopted a method combining transfer learning and CNN, aiming to learn robust visual feature representations to improve the classification accuracy of limited labeled samples. At the same time, in order to overcome the limitation of pre-trained networks on fixed input size, a scale-free CNN was proposed in

[29] , which can handle RS scenes of arbitrary size by replacing fully connected (FC) layers with convolutional layers. Its positive classification results confirmed the effectiveness of the method. To further improve the classification performance, Zhang et al.

[30] proposed an attribute collaborative CNN, which fully utilized the attribute information of RS scenes. These obtained attributes are regarded as additional information to extract discriminative visual features for RS scene classification. The second group of models focuses on integrating multi-layer features for RS scene classification. Lu et al.

[31] extracted multi-layer convolutional features from RS scenes by using pre-trained CNN and introduced a supervised feature encoding module, aiming to aggregate these multi-layer features to improve the classification results. In

[32] , a deep feature aggregation framework driven by graph convolutional networks (DFAGCN) was proposed, which can fully capture and fuse global visual features and multi-layer contextual information to improve classification accuracy. Wang et al.

[33] developed an enhanced feature pyramid network (EFPN) to extract multi-level features from RS scenes. Then, a two-branch deep feature fusion module was designed to aggregate these multi-level features. Therefore, by leveraging the complementary advantages of multi-level features, this method achieved significant performance improvement in remote sensing scene classification tasks. The third group of methods aims to combine CNN with other techniques to further improve the performance of RS scene classification. In

[34] , a discriminative CNN was proposed to improve the performance of RS scene classification. This method introduced a metric learning mechanism that can capture more discriminative features. On the other hand, based on the neural architecture search (NAS) method,

[35] proposed a model that uses an automatic search strategy to find the most suitable network architecture for RS scene classification.

[0007] Shi et al.

[36] explored a multi-branch feature fusion strategy based on the attention mechanism, which comprehensively used channel and spatial attention, and fully extracted the deep features of the image through the synergy of multiple convolutional layers. Tang et al.

[37] proposed a method called Attention Consistency Network (ACNet) specifically for remote sensing scene classification. The network combines spatial and channel attention to capture basic features from remote sensing scenes. In addition, they also constructed an attention consistency model to unify salient regions to more accurately extract discriminative features. Chen et al.

[38] designed a multi-branch local attention network and introduced a convolutional local attention module to effectively obtain channel and spatial attention weights. This method can automatically highlight or suppress important or redundant information in remote sensing scenes. Lu et al.

[39] proposed a Vision Transformer based on spatial channel feature preservation. Considering the geometric detail information of high spatial resolution images and the contribution between different channels, a model was designed to introduce lightweight channel attention after self-attention. First, the local structure of the image is extracted by gradually aggregating adjacent overlapping patches. Second, a multi-head self-attention mechanism is used to build global information interaction. Then, a lightweight channel attention module is used to consider the importance of different channels in the classification token. Finally, a multi-layer perceptron is used to obtain the final classification result. Ma et al.

[40] proposed a homogeneous-heterogeneous Transformer learning framework. First, a patch generation module is designed to generate homogeneous and heterogeneous patches. Then, a dual-branch feature learning module is proposed to mine homogeneous and heterogeneous information in remote sensing scenes. Finally, a classification module is used for the final classification. Shi et al.

[41] proposed a multi-branch fusion attention network, adaptively fused two parallel sub-modules, and designed a multi-branch fusion attention module to help the model focus on clues in the image that are difficult to classify. Summary of the invention

[0008] The purpose of the present invention is to solve the problems of quadratic complexity, large amount of calculation and low image classification accuracy of the self-attention module in the existing Transformer structure, and to propose a remote sensing scene image classification method based on a re-parameterized feature aggregation convolutional neural network.

[0009] The specific process of the remote sensing scene image classification method based on the re-parameterized feature aggregation convolutional neural network is as follows:

[0010] Construct a re-parameterized feature aggregation convolutional neural network model;

[0011] Classify the remote sensing scene images based on the constructed re-parameterized feature aggregation convolutional neural network model;

[0012] The re-parameterized feature aggregation convolutional neural network model includes a trunk branch, an aggregation branch and a fusion branch;

[0013] The trunk branch includes a first feature extraction module, a first RepFormer module, a first average pooling layer, a second RepFormer module, a second average pooling layer, a third RepFormer module, a third average pooling layer, a fourth RepFormer module, and a first linear layer;

[0014] The aggregation branch includes a second feature extraction module, a third feature extraction module, a first multi-head mixed convolution module, a second multi-head mixed convolution module, a first cascade layer, a fourth feature extraction module, a third multi-head mixed convolution module, and a second linear layer;

[0015] The fusion branch includes a second cascade layer and a feature fusion classifier;

[0016] The working process of the re-parameterized feature aggregation convolutional neural network model is:

[0017] The remote sensing scene image is input into the first feature extraction module, and the first feature extraction module outputs the shallow feature A of the image;

[0018] The shallow feature A of the image is sequentially input into the first RepFormer module, the first average pooling layer, the second RepFormer module, and the second average pooling layer. The second average pooling layer outputs feature B.

[0019] Feature B is sequentially input into the third RepFormer module and the third average pooling layer, and the third average pooling layer outputs feature C;

[0020] Feature C is input into the fourth RepFormer module, and the fourth RepFormer module outputs feature D;

[0021] Feature D outputs the feature into the first linear layer, and the first linear layer outputs the predicted probability;

[0022] Feature B is input into the second feature extraction module, and the second feature extraction module outputs feature E;

[0023] Feature E is input into the first multi-head mixed convolution module, and the first multi-head mixed convolution module outputs feature F;

[0024] Feature C is input into the third feature extraction module, and the third feature extraction module outputs feature G;

[0025] Feature G is input into the second multi-head mixed convolution module, and the second multi-head mixed convolution module outputs feature H;

[0026] Input features F and H into the first cascade layer, and the first cascade layer outputs feature I;

[0027] Feature I is input into the fourth feature extraction module, and the fourth feature extraction module outputs feature J;

[0028] Feature J is input into the third multi-head mixed convolution module, and the third multi-head mixed convolution module outputs feature K;

[0029] Feature K is input into the second linear layer, and the second linear layer outputs the predicted probability;

[0030] Input feature D and feature K into the second cascade layer, and the second cascade layer outputs feature L;

[0031] Feature L is input into the feature fusion classifier, and the feature fusion classifier outputs feature O.

[0032] The beneficial effects of the present invention are:

[0033] With the development of deep learning technology, Transformer has been introduced into the remote sensing (RS) scene image classification task to take advantage of its ability to build long-distance dependencies. However, considering the quadratic complexity of the self-attention module in the Transformer structure, the present invention proposes to replace it with a convolutional structure to reduce the computational complexity. At the same time, in order to make up for the problem of insufficient receptive field of small convolution kernels, a large convolution kernel is introduced to expand the receptive field, so as to better capture the global information in the image. In this study, a re-parameterized feature aggregation convolutional neural network (Re-parameterized Feature Aggregation Convolutional Neural Network, RepFACNN) is proposed. First, a re-parameterized Transformer (RepFormer) is used to extract multi-level features. Then, a multi-head mixed convolution module is used to capture spatial features at multiple scales, and a large convolution kernel is used in the multi-head convolution to expand the receptive field. Finally, a feature fusion classification module is proposed to fuse the features of the two branches and use the fused features for classification. To demonstrate the effectiveness of the RepFACNN method, experiments were conducted on three widely used RS scene classification datasets: UC_Merced, AID, and NWPU. The experimental results show that the performance of RepFACNN is better than some of the most advanced scene classification methods.

[0034] In the present invention, a new RS scene classification method is proposed, which is called a reparameterized feature aggregation convolutional neural network. First, the RepFormer module is used to extract global visual features and multi-level convolution features. Secondly, a multi-head mixed convolution module is proposed to extract multi-scale information. Multiple convolutions with different kernel sizes are introduced to enable it to capture various spatial features across multiple scales. Finally, a feature fusion classification module is proposed. Channel attention is first used to weight the channel features of the cascaded features, and then two linear layers are used for classification. The present invention proposes a RepFormer module, which retains parallel 3×3 depth convolutions, 1×1 depth convolutions and residual branches during training, and uses a reparameterization method to convert the three parallel branches into a 3×3 depth convolution during reasoning. Compared with using only 3×3 depth convolutions, the performance is improved. Compared with using three parallel branches, the reasoning speed is improved. The multi-head mixed convolution module of the present invention is proposed, and multiple convolutions with different convolution kernel sizes are introduced, so that the model can capture spatial features at multiple scales, and large convolution kernels can be used to expand the receptive field and enhance the ability to model long-range dependencies. The feature fusion classification module of the present invention is proposed. Channel attention is first used to perform channel weighting on the cascaded features, thereby improving the fusion capability of the two branch features. Then two linear layers are used to classify the fused features. Finally, the joint cross entropy loss is used to supervise the learning of the model.

[0035] The present invention proposes a new remote sensing image scene classification method, named RepFACNN. The method mainly includes three modules: RepFormer, multi-head mixed convolution module and feature fusion classification module. First, RepFormer is used to extract multi-level features and global information. Secondly, the multi-head mixed convolution module is used to capture the multi-scale spatial features in the scene. Then, the feature fusion classification module is used to fuse the features of the two branches and classify the fused features. Finally, the learning of the model is supervised by the joint cross entropy loss. Experiments on three widely used datasets show that the method of the present invention achieves advanced performance. A large number of ablation experiments and visualization analysis prove the effectiveness of the method proposed in the present invention. The proposed RepFACNN achieves high classification accuracy with fewer parameters and complexity.

[0036] Although the proposed RepFACNN achieved encouraging classification results, there is still room for improvement. In order to avoid increasing the complexity of the model, the present invention directly aggregates the two middle-layer features of the backbone network, which will limit the performance of the model. The future work of the present invention is to develop more ways to aggregate middle-layer features to enhance the performance of the model in future work. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1The overall flow chart of the reparameterized feature aggregation convolutional neural network; Figure 2 It is the reparameterization flow chart of the RepFormer module, assuming that the number of input and output channels is 2. (A) represents the reparameterization from the structural perspective, (B) represents the reparameterization from the parameter perspective, Conv Parameter is the convolution parameter, BN Parameter is the BN parameter, AParameter represents a trainable parameter, AZero Value represents the zero value, Perspective ofstructure is the structural perspective, and Perspective ofparameter is the parameter perspective; Figure 3 This is a schematic diagram of a multi-head mixed convolution module. Channels refers to channels, Split Channels refers to channel division, Feature refers to features, Shuffled Feature refers to mixed features, DW Conv refers to depthwise convolution, Group Conv refers to group convolution, Concatenate refers to cascade, and Conv refers to convolution. Figure 4 Schematic diagram of feature fusion classification module; Figure 5 These are sample images randomly selected from the three datasets; Figure 6 This is the confusion matrix diagram obtained under 80% training ratio of UC-Merced dataset, True Label is the real label, and Predicted Label is the predicted label; Figure 7 This is the confusion matrix diagram obtained at 50% training ratio on AID; Figure 8 This is the confusion matrix diagram obtained with a training ratio of 20% for the NWPU dataset; Fig. 9 Ablation experiment diagram on three datasets; Fig.10 This is a heat map on the AID dataset; Fig.11 : are T-SNE visualization result diagrams, a is the T-SNE visualization result diagram of the present invention at a 20% training ratio on the NWPU dataset, and b is the T-SNE visualization result diagram of the EMTCAL method at a 20% training ratio on the NWPU dataset. DETAILED DESCRIPTION

[0038] Specific implementation method 1: The specific process of the remote sensing scene image classification method based on the re-parameterized feature aggregation convolutional neural network in this implementation method is as follows:

[0039] The overall framework diagram of the method proposed in the present invention is as follows Figure 1As shown in the figure, it includes RepFormer, feature aggregation branch and feature fusion classification module. RepFormer aims to extract multi-level features from remote sensing scenes, the goal of feature aggregation branch is to aggregate intermediate layer features, and the goal of feature fusion classification module is to fuse global visual features and aggregated features to construct discriminative feature representation for classification. First, 3×3 convolution with a stride of 2, batch normalization layer (BN), 3×3 convolution with a stride of 2 and BN are used to extract shallow features of the image. Then, the backbone network composed of RepFormer is used to extract multi-level features in RS scenes. Secondly, the feature aggregation branch composed of multi-head mixed convolution modules is used to aggregate the intermediate layer features of the backbone network. Finally, the feature fusion classification module is used to fuse the features of the backbone network and the branch network and perform the final classification.

[0040] Construct a re-parameterized feature aggregation convolutional neural network model;

[0041] Classify the remote sensing scene images based on the constructed re-parameterized feature aggregation convolutional neural network model;

[0042] The re-parameterized feature aggregation convolutional neural network model includes a trunk branch, an aggregation branch and a fusion branch;

[0043] The trunk branch includes a first feature extraction module, a first RepFormer module, a first average pooling layer, a second RepFormer module, a second average pooling layer, a third RepFormer module, a third average pooling layer, a fourth RepFormer module, and a first linear layer;

[0044] The aggregation branch includes a second feature extraction module, a third feature extraction module, a first multi-head hybrid convolution module (MHMCM), a second multi-head hybrid convolution module (MHMCM), a first cascade layer, a fourth feature extraction module, a third multi-head hybrid convolution module (MHMCM), and a second linear layer;

[0045] The fusion branch includes a second cascade layer and a feature fusion classifier (FFCM);

[0046] The working process of the re-parameterized feature aggregation convolutional neural network model is:

[0047] The remote sensing scene image is input into the first feature extraction module, and the first feature extraction module outputs the shallow feature A of the image;

[0048] The shallow feature A of the image is sequentially input into the first RepFormer module, the first average pooling layer, the second RepFormer module, and the second average pooling layer. The second average pooling layer outputs feature B.

[0049] Feature B is sequentially input into the third RepFormer module and the third average pooling layer, and the third average pooling layer outputs feature C;

[0050] Feature C is input into the fourth RepFormer module, and the fourth RepFormer module outputs feature D;

[0051] Feature D outputs the feature into the first linear layer, and the first linear layer outputs the predicted probability;

[0052] Feature B is input into the second feature extraction module, and the second feature extraction module outputs feature E;

[0053] Feature E is input into the first multi-head hybrid convolution module (MHMCM), and the first multi-head hybrid convolution module (MHMCM) outputs feature F;

[0054] Feature C is input into the third feature extraction module, and the third feature extraction module outputs feature G;

[0055] Feature G is input into the second multi-head hybrid convolution module (MHMCM), and the second multi-head hybrid convolution module (MHMCM) outputs feature H;

[0056] Input features F and H into the first cascade layer, and the first cascade layer outputs feature I;

[0057] Feature I is input into the fourth feature extraction module, and the fourth feature extraction module outputs feature J;

[0058] Feature J is input into the third multi-head hybrid convolution module (MHMCM), and the third multi-head hybrid convolution module (MHMCM) outputs feature K;

[0059] Feature K is input into the second linear layer, and the second linear layer outputs the predicted probability;

[0060] Input feature D and feature K into the second cascade layer, and the second cascade layer outputs feature L;

[0061] Feature L is input into the feature fusion classifier (FFCM), and the feature fusion classifier (FFCM) outputs feature O.

[0062] Specific implementation method 2: This implementation method is different from specific implementation method 1 in that the first feature extraction module sequentially includes a 3×3 convolution with a stride of 2, a first batch normalization layer (BN), a 3×3 convolution with a stride of 2, and a second batch normalization layer (BN);

[0063] The working process of the first feature extraction module is:

[0064] The remote sensing scene image is sequentially input into the 3×3 convolution with the first step size of 2, the first batch normalization layer, the second 3×3 convolution with the second step size of 2, and the second batch normalization layer. The second batch normalization layer outputs the shallow features A of the image.

[0065] The other steps and parameters are the same as those in the first embodiment.

[0066] Specific implementation method three: This implementation method is different from specific implementation method one or two in that the second feature extraction module includes a third 3×3 convolution, a first 1×1 convolution, a third batch normalization layer BN, and a first ReLU activation function layer in sequence;

[0067] The working process of the second feature extraction module is as follows:

[0068] Feature B is sequentially input into the third 3×3 convolution, the first 1×1 convolution, the third batch normalization layer BN, and the first ReLU activation function layer, and the first ReLU activation function layer outputs feature E.

[0069] The other steps and parameters are the same as those in the first or second embodiment.

[0070] Specific implementation method 4: This implementation method is different from any one of specific implementation methods 1 to 3 in that the third feature extraction module sequentially includes a fourth 3×3 convolution, a second 1×1 convolution, a fourth batch normalization layer BN, and a second ReLU activation function layer;

[0071] The working process of the third feature extraction module is:

[0072] Feature C is sequentially input into the fourth 3×3 convolution, the second 1×1 convolution, the fourth batch normalization layer BN, and the second ReLU activation function layer, and the second ReLU activation function layer outputs feature G.

[0073] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0074] Specific implementation mode 5: This implementation mode is different from any one of the specific implementation modes 1 to 4 in that each of the first RepFormer module, the second RepFormer module, the third RepFormer module, and the fourth RepFormer module comprises:

[0075] The fifth 3×3 convolution, the fifth batch normalization layer BN, the third 1×1 convolution, the sixth batch normalization layer BN, the seventh batch normalization layer BN, the third linear layer, the GELU activation function layer, and the fourth linear layer;

[0076] The working process of each RepFormer module is:

[0077] The feature β is sequentially input into the fifth 3×3 convolution and the fifth batch normalization layer BN, and the fifth batch normalization layer BN outputs the feature β′;

[0078] The feature β is sequentially input into the third 1×1 convolution and the sixth batch normalization layer BN, and the sixth batch normalization layer BN outputs the feature β″;

[0079] The feature β is input into the seventh batch normalization layer BN, and the seventh batch normalization layer BN outputs the feature β″′;

[0080] Add the features β′, β″, and β″′ element by element to get the feature

[0081] feature Input the third linear layer, GELU activation function layer, and fourth linear layer in sequence, and the fourth linear layer outputs features

[0082] The characteristics and Features Add each element to get the feature

[0083] feature As the output features of each RepFormer module.

[0084] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0085] Specific implementation method 6: This implementation method is different from any one of the specific implementation methods 1 to 5 in that each of the first multi-head hybrid convolution module (MHMCM), the second multi-head hybrid convolution module (MHMCM), and the third multi-head hybrid convolution module (MHMCM) includes:

[0086] 1×1 depth convolution layer, 3×3 depth convolution layer, 5×5 depth convolution layer, 7×7 depth convolution layer, 1×1 group convolution (1×1 convolution layer), eighth batch normalization layer BN, GELU activation function layer, fourth 1×1 convolution layer;

[0087] The working process of each multi-head mixed convolution module is:

[0088] The input channels are divided into 4 heads, and the input feature x is divided into 4 heads [x1, x2, x3, x4] (4 heads, a total of 512 channels, 128 channels per head), each head has M channels;

[0089] Input x1 into the 1×1 depth convolution layer, and the 1×1 depth convolution layer outputs the feature

[0090] Input x2 into the 3×3 depth convolution layer, and the 3×3 depth convolution layer outputs the feature

[0091] Input x3 into the 5×5 depth convolution layer, and the 5×5 depth convolution layer outputs the feature

[0092] Input x4 into the 7×7 depth convolution layer, and the 7×7 depth convolution layer outputs features

[0093] Take features separately The features of the first channel in , 4 features are concatenated as the first new mixed feature;

[0094] Take features separately The features of the second channel in , 4 features are concatenated as the second new mixed feature;

[0095] Until,

[0096] Take features separately The features of the Mth channel in , 4 features are concatenated as the Mth new mixed features;

[0097] Each new mixed feature is input into a 1×1 group of convolutions (128 channels), and each 1×1 group of convolution output features is input into the eighth batch normalization layer BN and the GELU activation function layer in turn, and the GELU activation function layer outputs the features respectively;

[0098] The M features output by the GELU activation function layer are cascaded, and the cascaded features are input into the fourth 1×1 convolutional layer. The output features of the fourth 1×1 convolutional layer are used as the output features of each multi-head mixed convolution module.

[0099] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0100] Specific implementation method 7: This implementation method is different from any one of the specific implementation methods 1 to 6 in that the feature fusion classifier (FFCM) includes:

[0101] One-dimensional convolution layer with a convolution kernel size of 5, Sigmoid activation function layer, fifth linear layer, GELU activation function layer, sixth linear layer;

[0102] The working process of the feature fusion classifier (FFCM) is:

[0103] The feature L is sequentially input into the one-dimensional convolution layer with a convolution kernel size of 5 and the Sigmoid activation function layer. The output feature of the Sigmoid activation function layer is multiplied element-wise with the feature L, and the obtained feature is sequentially input into the fifth linear layer, the GELU activation function layer, and the sixth linear layer. The sixth linear layer outputs feature O, which is used as the output feature of the feature fusion classifier (FFCM).

[0104] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0105] Specific implementation eight: This implementation differs from any one of specific implementations one to seven in that the training process of the re-parameterized feature aggregation convolutional neural network model is:

[0106] Suppose there are K samples and M categories in the sample set

[0107] The total prediction loss of the reparameterized feature aggregation convolutional neural network model is

[0108] The prediction loss of the trunk branch is

[0109] The prediction loss of the aggregation branch is

[0110] The prediction loss of the fusion branch is

[0111] Prediction loss of trunk branches and the prediction loss of the aggregation branch Respectively expressed as:

[0112]

[0113]

[0114] where w A and w B Represent the parameters of the aggregation branch and trunk branch respectively;

[0115] y km Indicates the label of sample k. It takes 1 if the label of sample k is m, otherwise it takes 0.

[0116] p(m|x k ) represents the sample x k The probability of belonging to category m;

[0117] p(m|x k ;w B ) represents the sample x under the parameters of the trunk branch k The probability of belonging to category m;

[0118] p(m|x k ;w A ) represents the sample x under the parameters of the aggregation branch k The probability of belonging to category m;

[0119] Prediction loss based on trunk branches and the prediction loss of the aggregation branch Get auxiliary prediction loss It is expressed as:

[0120]

[0121] in, is the prediction loss of the aggregated features, The prediction loss for the backbone features;

[0122] Prediction loss of fused branches It is expressed as:

[0123]

[0124] Among them, p(m|x k ; W) represents the parameter W of the fusion branch and the sample x k The probability of belonging to category m; W represents the parameter of the fusion branch;

[0125] Based on auxiliary prediction loss The prediction loss of the fusion branch is Obtain the total prediction loss of the reparameterized feature aggregation convolutional neural network model It is expressed as:

[0126]

[0127] The reparameterized feature aggregation convolutional neural network model is trained until the total prediction loss of the reparameterized feature aggregation convolutional neural network model is Convergence, obtain the trained re-parameterized feature aggregation convolutional neural network model.

[0128] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0129] Specific implementation method 9: This implementation method is different from any one of specific implementation methods 1 to 8 in that the remote sensing scene image to be measured is classified based on the constructed re-parameterized feature aggregation convolutional neural network model;

[0130] The specific process is:

[0131] The image of the remote sensing scene to be tested is input into the trained re-parameterized feature aggregation convolutional neural network model, and the trained re-parameterized feature aggregation convolutional neural network model outputs the category of the image of the remote sensing scene to be tested;

[0132] The specific process is:

[0133] The remote sensing scene image to be measured is input into the first feature extraction module, and the first feature extraction module outputs the shallow features of the image

[0134] Shallow features of images Input the first RepFormer module, the first average pooling layer, the second RepFormer module, the second average pooling layer, and the second average pooling layer outputs features

[0135] feature Input the third RepFormer module and the third average pooling layer in sequence, and the third average pooling layer outputs features

[0136] feature Input the fourth RepFormer module, the fourth RepFormer module outputs features

[0137] feature Input the second feature extraction module, the second feature extraction module outputs the feature

[0138] feature Input the first multi-head hybrid convolution module (MHMCM), the first multi-head hybrid convolution module (MHMCM) outputs features

[0139] feature Input the third feature extraction module, the third feature extraction module outputs the feature

[0140] feature Input the second multi-head mixed convolution module (MHMCM), the second multi-head mixed convolution module (MHMCM) outputs features

[0141] The characteristics and Features Input the first cascade layer, the first cascade layer outputs features

[0142] feature Input the fourth feature extraction module, the fourth feature extraction module outputs the feature

[0143] feature Input the third multi-head mixed convolution module (MHMCM), the third multi-head mixed convolution module (MHMCM) outputs features

[0144] The characteristics and Features Input the second cascade layer, the second cascade layer outputs features

[0145] feature Input feature fusion classifier (FFCM), and feature fusion classifier (FFCM) outputs classification results.

[0146] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0147] Specific implementation method 10: This implementation method is different from the specific implementation methods 1 to 9 in that the shallow features of the image Input the first RepFormer module, the first average pooling layer, the second RepFormer module, the second average pooling layer, and the second average pooling layer outputs features

[0148] feature Input the third RepFormer module and the third average pooling layer in sequence, and the third average pooling layer outputs features

[0149] feature Input the fourth RepFormer module, the fourth RepFormer module outputs features

[0150] The specific process is:

[0151] Each of the first RepFormer module, the second RepFormer module, the third RepFormer module, and the fourth RepFormer module includes:

[0152] 3×3 convolution layer, batch normalization layer BN, linear layer, GELU activation function layer, linear layer;

[0153] The working process of each RepFormer module in the first RepFormer module, the second RepFormer module, the third RepFormer module, and the fourth RepFormer module is as follows:

[0154] The features are sequentially input into the 3×3 convolutional layer and the batch normalization layer BN, and the batch normalization layer BN outputs the feature P;

[0155] Feature P is sequentially input into the linear layer, GELU activation function layer, and linear layer, and outputs feature Q;

[0156] Features P and Q are added element by element, and the added features are used as the output features of each RepFormer module.

[0157] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0158] The RepFormer module is as follows Figure 2 As shown, the module includes three parallel branches in the training phase, and the three parallel branches can be converted into a single branch in the reasoning phase. The present invention will describe how to convert the three parallel branches in training into a single branch for verifying reasoning. represents the convolution kernel of a 3×3 convolutional layer with C1 input channels and C2 output channels, Represents the convolution kernel of the 1×1 convolution layer. Use μ (3) ,σ (3) ,γ (3) ,β (3) Represents the mean, standard deviation, scaling factor, and bias of the BN layer after the 3×3 convolutional layer. Use μ (1) ,σ (1) ,γ (1) ,β (1) Represents the mean, standard deviation, scaling factor, and bias of the BN layer after the 1×1 convolutional layer. (0) ,σ (0) ,γ (0) ,β (0) represents the mean, standard deviation, scaling factor, and bias of the BN layer in the residual branch. and Represent input and output respectively, * represents convolution operation, when C1=C2, H1=H2, W1=W2, there is

[0159] O=BN(I*W (3) ,μ (3) ,σ (3) ,γ (3) ,β (3) )

[0160] +BN(I*W (1) ,μ (1) ,σ (1) ,γ (1) ,β (1) )

[0161] +BN(I,μ (0) ,σ (0) ,γ (0) ,β (0) )

[0162] Since each branch has a BN layer, convolution and BN need to be fused, which is also a common operation in operator folding. Convolution can be expressed as: Conv(x) = W(x) + b, where W represents the convolution kernel and b represents the bias;

[0163] BN can be expressed as:

[0164] Where μ, σ, γ, β represent the mean, standard deviation, scaling factor and bias in BN respectively, and ε represents a very small number. Substituting convolution into BN, we have:

[0165]

[0166] When using BN after convolution, bias is usually set to False, so the above formula can be simplified to:

[0167]

[0168] By integrating BN into convolution, the present invention obtains a 3×3 convolution kernel, two 1×1 convolution kernels and three bias vectors. The final bias is obtained by adding the three bias vectors. The 1×1 convolution kernel is added to the center point of the 3×3 convolution kernel to obtain the final 3×3 convolution kernel. This can be achieved by zero-padding the two 1×1 convolution kernels to convert the 1×1 convolution kernel into a 3×3 convolution kernel, and then adding the three convolution kernels.

[0169] After the re-parameterized convolution, a multi-layer perceptron structure is used. The multi-layer perceptron (MLP) consists of two linear layers and a GELU activation function. Since the linear mapping is performed on the channel dimension, the present invention also calls the multi-layer perceptron structure a channel mixer. In ViT, the channel mixer usually uses a 4-fold expansion ratio, which means that the hidden dimension in the MLP is 4 times wider than the input dimension, which consumes a large part of the computing resources. In the proposed RepFormer structure, the channel expansion ratio is set to 2 to reduce parameter redundancy and delay.

[0170] The channel mixer can be expressed by the formula: MLP(x)=x+W2σ(W1(x));

[0171] W1 represents the weight of the first linear layer, σ represents the GELU activation function, and W2 represents the weight of the second linear layer;

[0172] The multi-head mixed convolution module is proposed by the present invention, such as Figure 3As shown. The input channel is divided into M heads, and depth-wise separable convolution with different convolution kernel sizes is used for each head. The initial convolution kernel size is set to 1×1, and the convolution kernel size of each head is gradually increased by 2. By adjusting the number of heads, the range of the receptive field is adjusted and multi-granularity information is obtained. The multi-head convolution of this process can be expressed as:

[0173] Here x=[x1,x2,…,x n ] means splitting the input feature x into multiple heads in the channel dimension, k i ∈{1,3,5,…,K} means that the convolution kernel of each head increases by 2.

[0174] In order to enhance the information interaction between multiple heads, the present invention introduces 1×1 group convolution and 1×1 convolution as well as channel shuffling and reorganization operations after multi-head convolution. Specifically, the present invention selects a channel from each head to construct a group, and then performs channel feature fusion operations through 1×1 group convolution and 1×1 convolution. Let the input The number of groups is inversely proportional to the number of multiples. The process of channel fusion can be expressed as:

[0175] M=W([G1,G2,…,G M ]),

[0176]

[0177]

[0178] Here W and W g represents the weight matrix of 1×1 convolution and 1×1 group convolution. j∈{1,2,…,N} and i∈{1,2,…,M}, where N and Represent the number of longs and the number of groups respectively. represents the jth head, represents the i-th channel of the j-th head.

[0179] The multi-head hybrid convolution module can be expressed by the following formula: MHMCM(x) = M(G(MHC(x)));

[0180] Among them, MHC represents multi-head convolution, G represents 1×1 group convolution, and M represents 1×1 convolution.

[0181] In previous work, feature fusion methods usually directly adopt a cascade approach. Due to the lack of interaction between each other, this limits the representation of fused features. Therefore, the present invention designs FFCM to model the fusion between backbone features and aggregated features. Figure 4 shown.

[0182] First, the present invention connects the backbone features and the aggregated features, which are expressed as follows: concat =Concat(F B ,F A );

[0183] Here F B and F A They represent backbone features and aggregate features respectively.

[0184] Then, channel attention is used to obtain the correlation between feature channels. The information interaction between channels is achieved through one-dimensional convolution with a convolution kernel size of 5, which can be expressed as: ω = σ (Conv1D k (x));

[0185] Conv1D represents one-dimensional convolution, k represents the convolution kernel size, and σ represents the Sigmoid activation function. It only involves K parameter information, and this method of cross-channel information interaction guarantees performance results and model efficiency.

[0186] Finally, two linear layers are used for the final classification, which can be expressed as: Classifier(x) = W2σ(W1(x)), where W1 represents the weight of the first linear layer, W2 represents the weight of the second linear layer, and σ represents the GELU activation function.

[0187] This paper designs a joint loss to supervise the update of model parameters. It consists of three losses: prediction loss of backbone features Prediction loss of aggregated features And the prediction loss of the final fusion feature use and To help the model learn discriminative representations, such as Figure 1 Specifically, two auxiliary classifiers are added based on the backbone features and the aggregated features. and It is calculated based on the prediction results of the auxiliary classifier.

[0188] Assume there are K samples and M categories of sample sets And the corresponding label set y k ∈{1,2,...,M}, sample x k The probability of belonging to class m can be calculated as follows: Among them, m Represents the output of the model.

[0189] The cross entropy loss is used to represent the error between the prediction and the label. The formula of the cross entropy loss can be expressed as: where ykm The value is 1 when the label of sample k is m, otherwise it is 0.

[0190] Therefore, the prediction loss of the backbone feature and the prediction loss of the aggregated features They can be expressed as:

[0191]

[0192]

[0193] where w A and w B They represent the parameters of the feature aggregation branch and the backbone network respectively. The total auxiliary prediction loss can be expressed as:

[0194] Prediction loss of fused features It can be expressed as:

[0195] Where W represents the parameters of the feature fusion classifier. Joint loss is the average of the auxiliary prediction loss and the fusion feature prediction loss, expressed as follows:

[0196] Algorithm implementation details: First, preprocess the data to obtain the input tensor x. Then input the tensor into the backbone network composed of RepFormer modules to obtain feature maps at different stages. Then input the two intermediate layer feature maps into the multi-head mixed convolution module to generate multi-scale feature maps. Next, use 1×1 convolution and multi-head mixed convolution modules to aggregate the two intermediate layer features. Finally, input the global features of the backbone network and the aggregated intermediate layer features into the feature fusion classification module to fuse the features and perform the final classification. Model learning is supervised by calculating the joint cross entropy loss. The entire process of RepFACNN model learning is shown in Algorithm 1.

[0197]

[0198]

[0199] Example:

[0200] In order to evaluate the effectiveness of the proposed RepFACNN, the present invention conducted various experiments on three public and challenging datasets and compared them with some state-of-the-art classification methods. The three datasets are UC-Merced dataset

[42] , AID

[43] and NWPU dataset

[44] . The experimental results show that the proposed method exhibits good performance. The present invention provides an introduction to the three datasets used in the experiment. Some sample examples selected from these datasets are as follows: Figure 5 shown.

[0201] 1) UC-Merced: UC-Merced is a remote sensing image dataset widely used in object classification tasks, created and provided by the University of California, Merced. The dataset contains 21 unique object categories, each category contains 100 images, totaling 2100 images. The resolution of each image is 256×256 pixels and is presented in RGB format. The images cover a variety of object types, such as cities, farmlands, forests, rivers, and parks. The training ratio is divided into 50% and 80%, and the remaining 50% and 20% are used for testing.

[0202] 2) AID: AID (Aerial Image Dataset) is a dataset widely used for aerial image analysis, mainly used for remote sensing image classification and target detection tasks. The dataset was created by the Institute of Automation, Chinese Academy of Sciences, and contains 30 unique land object categories, covering multiple land object types such as cities, farmlands, forests, grasslands, roads, rivers, lakes, buildings, etc. There are about 200-400 images in each category, totaling about 10,000 images. The image resolution is 600×600 pixels, and the format is color image (RGB).

[0203] 3) NWPU: NWPU is a widely used dataset for remote sensing image classification tasks. It is provided by Northwestern Polytechnical University (NWPU) in China and is one of the datasets for the RESISC45 competition. The dataset contains 45 unique remote sensing image scene categories, each category contains 700 images, totaling 31,500 images. The resolution of each image is 256x256 pixels and the format is a color image (RGB). NWPU's images cover a variety of geographical environments and scenes, including cities, farmlands, rivers, forests, grasslands, airports, etc.

[0204] The ratio of training and testing samples for the three datasets (UCM, AID, and NWPU) is set to 50%-50% and 80%-20%, 20%-80% and 50%-50%, 10%-90%, and 20%-80%, respectively. Table 1 provides the detailed information of these datasets, including the number of images per category, the number of scene categories, the total number of images, spatial resolution, and image size.

[0205] Table 1: Data information of three datasets

[0206]

[0207] All experiments were implemented on a workbench with a GeForce RTX 3070Ti, using the Pytorch framework. Adaptive moment estimation was used to optimize the model, the initial learning rate of the network was set to 0.001, and 150 epochs were trained. Cosine decay was used to adjust the learning rate. The image size was adjusted to 224×224, and random horizontal flipping and random vertical flipping were used to enhance the image. In order to reduce the impact of randomness, the main experiment of the present invention was repeated five times and the average results were reported.

[0208] In the experiment, the present invention used overall accuracy (OA) and confusion matrix (CM) to evaluate the effectiveness of the method proposed in the present invention. Overall accuracy OA is defined as the number of correctly classified images divided by the total number of test images, which reflects the overall performance level of the classification model. The confusion matrix CM is used to analyze the classification errors and confusions between different scene categories. The matrix shows the classification of the model in each category, including the number of correct classifications and the number of incorrect classifications. By analyzing the confusion matrix, the present invention can gain an in-depth understanding of the performance differences of the model between different categories, identify which categories are easily confused, and thus more comprehensively evaluate the classification effect of the model. The combined use of these two evaluation indicators helps to fully understand the performance of the proposed method in different scene classification tasks.

[0209] Experimental results and analysis: In order to evaluate the method proposed in the present invention, the present invention conducted a series of experiments. The data of each data set experiment is listed in the table. Table 2 lists the methods used for comparison in the present invention and the year of publication.

[0210] Table 2: Different comparison methods and publication years in the experiment

[0211]

[0212] 1) Experimental results on the UC-Merced dataset: Methods with good classification performance on the UC-Merced dataset in recent years are selected for comparison with the proposed method, and the experimental results are shown in Table 3. When the training ratio is 80%, the classification accuracy of the proposed method reaches 99.88%, which exceeds all the compared methods. The proposed method is 0.31% higher than the OA of EMTCAL, 0.59% higher than the OA of ViT, 0.31% higher than the OA of SCViT, and 0.22% higher than the OA of MBFANet. When the training ratio is 50%, the classification accuracy of the proposed method reaches 99.33%, which exceeds all the compared methods. The proposed method is 3.14% higher than the OA of EFPN-DSE-TDFF, 0.66% higher than the OA of EMTCAL, 0.43% higher than the OA of SCViT, and 0.58% higher than the OA of ViT.

[0213] Table 3: Comparison of the proposed method with the methods proposed in recent years on the UC-Merced dataset.

[0214]

[0215] At a training ratio of 80%, the confusion matrix obtained by the present invention is as follows: Figure 6 As shown. Figure 6 It can be clearly observed that excellent classification results are achieved for each category of the UC-Merced dataset. This result shows that the model of the present invention performs well on the given dataset and successfully distinguishes different categories. On AID, the present invention compares the proposed method with some methods in recent years, and the experimental results are shown in Table 4. When the training ratio is 20%, the OA of the proposed method is 96.04%. It is 1.14% higher than the OA of ViT, 1.35% higher than the OA of EMTCAL, 0.48% higher than the OA of SCViT, 2.06% higher than the OA of MBFANet, and 0.87% higher than the OA of SAGN. When the training ratio of AID is 50%, the OA of the proposed method reaches 97.54%, exceeding all the comparison methods. It is 1.05% higher than the OA of ViT, 1.13% higher than the OA of EMTCAL, 0.61% higher than the OA of MBFANet, 0.77% higher than the OA of SAGN, and 0.56% higher than the OA of SCViT.

[0216] Table 4: Comparison of the method proposed in this invention with the methods proposed in recent years on AID

[0217]

[0218]

[0219] The confusion matrix at 50% training ratio is as follows Figure 7 As shown, among the 30 categories, the classification accuracy of 29 categories exceeds 90%, and only one category, "resort", has a classification accuracy that does not reach 90%. The "resort" scene is mainly misclassified into scenes such as "school", "park", "industrial area" and "square". This is because the "resort" scene has a high degree of inter-class similarity with these scenes. How to further reduce the impact of inter-class similarity is the future work of the present invention.

[0220] 3) Classification results on the NWPU dataset: The comparison of the classification performance of the proposed method with other methods on the NWPU dataset is summarized in Table 5. At a training ratio of 10%, the overall accuracy reached 93.18%, exceeding all compared methods. At a training ratio of 20%, the overall accuracy reached 94.85%, exceeding all compared methods. At a training ratio of 10%, the proposed RepFACNN is 2.09% higher than the OA of ACNet, 1.55% higher than the OA of EMTCAL, 1.45% higher than the OA of SAGN, and 1.05% higher than the OA of VSDNet-ResNet34. At a training ratio of 20%, the proposed RepFACNN is 2.43% higher than the OA of ACNet, 1.2% higher than the OA of EMTCAL, and 1.36% higher than SAGN.

[0221] Table 5: Comparison of the proposed method with the methods that have performed well in classification in recent years on the NWPU dataset

[0222]

[0223] The confusion matrix obtained at the 20% training ratio of the NWPU dataset is as follows Figure 8 As shown in the figure. The classification accuracy of 30 of the 45 categories reached more than 95%, and the classification accuracy of 3 categories did not reach 90%. The 3 scene categories with classification accuracy not reaching 90% are "church", "medium-sized residential area" and "palace". Among these three scenes with low classification accuracy, the "church" scene is mainly misclassified as the "palace" scene, and the "palace" scene is also mainly misclassified as the "church" scene. The reason is that the two scenes have similar architectural styles.

[0224] Model size evaluation: The present invention compares the number of parameters and FLOPs of various methods, where FLOPs reflects the complexity of the model, and the number of parameters measures the size of the model. The experimental results are detailed in Table 6. It can be clearly seen from Table 6 that the model of the present invention is not only far lower than TECN and KFBNet in terms of the number of parameters and complexity, but also has advantages in classification accuracy. Compared with methods such as ViT, Contourlet CNN and EMTCAL, the method proposed in the present invention not only has advantages in the number of parameters and FLOPs, but also has a significant lead in classification accuracy. Compared with GoogLeNet, SE-MDPMNet, DenseNet121 and LCNN-BFF, although the RepFACNN of the present invention is slightly higher than these methods in terms of parameter number, it far exceeds them in terms of accuracy.

[0225] Table 6: Comparison of OA, parameters, and FLOPs of the proposed method with some advanced methods on AID.

[0226]

[0227] In order to comprehensively evaluate the effectiveness of the proposed method, a series of ablation experiments, heat map analysis and T-SNE visualization analysis were performed. In order to verify the effectiveness of the proposed reparameterization method, the present invention tested the inference time of the model before and after reparameterization. The experimental results are detailed in Table 7. Before reparameterization, the time used to infer an image was 6.64 milliseconds. After using the reparameterization method, the time used to infer an image was 5.74 milliseconds. After using the reparameterization method, the time used by the model to infer an image was shortened by 0.90 milliseconds, saving inference time.

[0228] Table 7: Inference time of the model before and after reparameterization

[0229]

[0230] In order to verify the effectiveness of the three proposed modules, the present invention conducted ablation experiments on three datasets. The training and testing ratios adopted on the UC-Merced, AID and NWPU datasets were 80%-20%, 50%-50% and 20%-80%, respectively. The proposed RepFACNN consists of three modules, including RepFormer, MHMCM and FFCM. The RepFormer module is used to extract global features, MHMCM is used to capture multi-scale spatial features, and FFCM is used for feature fusion and classification.

[0231] The present invention divides the ablation experiment into four cases. The first case is not adding any modules, the second case is adding RepFormer on the basis of the original network, the third case is adding RepFormer and MHMCM, and the fourth case is adding RepFormer, MHMCM and FFCM. Fig. 9 It can be observed that the three proposed modules all contribute positively to the classification performance. Taking the ablation experiment on the NWPU dataset as an example, the classification performance of the model in the first case is the worst. The classification performance of the model in the second case is improved by 1.65%. The classification accuracy of the third case is 0.67% higher than that of the second case. The fourth case obtains the highest classification accuracy of 94.85, which is 0.41% higher than the performance of the third case.

[0232] Grad-CAM can be used to generate attention maps, also called heat maps, which can show the areas of interest to the model. The present invention randomly selects scenes such as "airport", "port", "church", "bridge" and "parking lot" from the AID dataset to generate heat maps. Fig.10 In FIG, the top row is a heat map obtained by using the method proposed in the present invention, and the second row represents a heat map generated by the basic network without using the module proposed in the present invention.

[0233] from Fig.10 As can be seen in the heat map of the second row, the network can only focus on the local part of the scene, the region of interest is not comprehensive, and it is unable to focus on the global information of the scene. However, the method proposed in the present invention can obtain a more comprehensive region of interest, which fully proves that the proposed model focuses on global information.

[0234] This paper uses the T-distributed stochastic neighbor embedding (T-SNE) method to visualize the classification results on the UCM dataset. T-SNE provides an intuitive way to evaluate the classification results of the model by mapping high-dimensional features to low-dimensional space. The visualization results are presented in Fig.11 In. From Fig.11 It can be observed that compared with the EMTCAL method, the model proposed in this paper enhances the separability and relative distance between various semantic clusters, which indicates that different scene categories can be distinguished more accurately, thereby improving the classification performance.

[0235] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

[0236] References

[0237] [1]Jaiswal,R.K.;Saxena,R.;Mukherjee,S.Application of remote sensingtechnology for land use / land cover change analysis.J.IndianSoc.RemoteSens.1999,27,123–128.[CrossRef]

[0238] [2]Chova,L.G.;Tuia,D.;Moser,G.;Valls,G.C.Multimodal classificationofremote sensing images:Areview and future directions.IEEEProc.2015,103,1560–1584.[CrossRef]

[0239] [3]Cheng,G.;Zhou,P.;Han,J.Learning Rotation-Invariant ConvolutionalNeural Networks for Object Detection in VHR OpticalRemote SensingImages.IEEETrans.Geosci.Remote Sens.2016,54,7405–7415.[CrossRef]

[0240] [4]Zhang,L.;Zhang,L.;Du,B.Deep learning for remote sensing data:Atechnical tutorial on the state-of-the-art.IEEE Geosci.Remote Sens.Mag.2016,4,22–40.[CrossRef]

[0241] [5]S.E.Grigorescu,N.Petkov,and P.Kruizinga,“Comparison oftexturefeatures based on Gabor filters,”IEEE Trans.ImageProcess.,vol.11,no.10,pp.1160–1167,Oct.2002.

[0242] [6]X.Tang,L.Jiao,andW.J.Emery,“SARimagecontentretrievalbasedonfuzzysimilarityandrelevancefeedback,”IEEE J.Sel.TopicsAppl.EarthObserv.RemoteSens.,vol.10,no.5,pp.1824–1842,May2017.

[0243] [7]H.Soltanian-Zadeh,F.Rafiee-Rad,and S.Pourabdollah-Nejad,“Comparison of multiwavelet,wavelet,haralick,and shape features formicrocalcification classification inmammograms,”PatternRecognit.,vol.37,no.10,pp.1973–1986,Oct.2004.

[0244] [8]X.Tang,L.Jiao,W.J.Emery,F.Liu,and D.Zhang,“Two-stage reranking forremote sensing image retrieval,”IEEE Trans.Geosci.Remote Sens.,vol.55,no.10,pp.5798–5817,Oct.2017.

[0245] [9]L.Zhang,W.Zhou,and L.Jiao,“Wavelet support vector machine,”IEEETrans.Syst.,Man,Cybern.B,Cybern.,vol.34,no.1,pp.34–39,Feb.2004.

[0246]

[10] M.A.Friedl and C.E.Brodley,“Decision tree classification of landcover from remotely sensed data,”Remote Sens.Environ.,vol.61,pp.399–409,Sep.1997.

[0247]

[11] Feng,J.;Li,D.;Gu,J.;Cao,X.;Shang,R.;Zhang,X.;Jiao,L.DeepReinforcement Learning for Semisupervised Hyperspectral Band Selection.IEEETrans.Geosci.Remote Sens.2021,1–19.[CrossRef]

[0248]

[12] Girshick,R.;Donahue,J.;Darrell,T.;Malik,J.Rich FeatureHierarchies for Accurate Object Detection and Semantic Segmentation.

[0249]

[13] Cuiping Shi,Tao Wang,Liguo Wang.Branch Feature Fusion ConvolutionNetwork for Remote Sensing Scene Classification[J],IEEE Journal of SelectedTopics in Applied Earth Observations and Remote Sensing,2020,13(9):5194-5210.DOI:10.1109 / JSTARS.2020.3018307.

[0250]

[14] Zhao B,Zhong Y,Xia G S,et al.Dirichlet-derived multiple topicscene classification model for high spatial resolution remote sensing imagery[J].IEEE Transactions on Geoscience and Remote Sensing,2015,54(4):2108-2123.

[0251]

[15] W.Li et al.,“Classification ofhigh-spatial-resolution remotesensing scenes method using transfer learning and deep convolutional neuralnetwork,”IEEE J.Sel.TopicsAppl.Earth Observ.Remote Sens.,vol.13,pp.1986–1995,2020.

[0252]

[16] Singh,P.;Verma,V.K.;Rai,P.;Namboodiri,V.P.HetConv:HeterogeneousKernel-Based Convolutions for Deep CNNs.arXiv 2019,arXiv:1903.04120.

[0253]

[17] Chen,Y.;Dai,X.;Liu,M.;Chen,D.;Yuan,L.;Liu,Z.Dynamic Convolution:Attention Over Convolution Kernels.In Proceedings of the IEEE / CVF Conferenceon Computer Vision and Pattern Recognition(CVPR),Seattle,WA,USA,14–19 June2020;pp.11030–11039.

[0254]

[18] Liu,J.J.;Hou,Q.;Cheng,M.M.;Wang,C.;Feng,J.Improving ConvolutionalNetworks with Self-Calibrated Convolutions.In Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition(CVPR),Seattle,WA,USA,14–19 June 2020;pp.10096–10105.

[0255]

[19] Chen,Y.;Fan,H.;Xu,B.;Yan,Z.;Kalantidis,Y.;Rohrbach,M.;Yan,S.;Feng,J.Drop an Octave:Reducing Spatial Redundancy in Convolutional NeuralNetworks with Octave Convolution.arXiv2019,arXiv:1904.05049.

[0256]

[20] Han,K.;Wang,Y.;Tian,Q.;Guo,J.;Xu,C.;Xu,C.GhostNet:More Featuresfrom Cheap Operations.arXiv 2020,arXiv:1911.11907.

[0257]

[21] Yang,B.;Bender,G.;Le,Q.V.;Ngiam,J.CondConv:ConditionallyParameterized Convolutions for Efficient Inference.arXiv2019,arXiv:1904.04971[cs.CV].

[0258]

[22] Cao,J.;Li,Y.;Sun,M.;Chen,Y.;Lischinski,D.;Cohen-Or,D.;Chen,B.;Tu,C.Depthwise Over-parameterized Convolution.arXiv2020,arXiv:2006.12030[cs.CV].

[0259]

[23] Xie,S.;Girshick,R.;Dollar,P.;Tu,Z.;He,K.Aggregated ResidualTransformations for Deep Neural Networks.arXiv2017,arXiv:1611.05431v2[cs.CV].

[0260]

[24] Zhang,X.;Zhou,X.;Lin,M.;Sun,J.ShuffleNet:An Extremely EfficientConvolutional Neural Network for Mobile.arXiv 2017,arXiv:1707.01083v2[cs.CV].

[0261]

[25] Wu,P.;Cui,Z.;Gan,Z.;Liu,F.Residual Group Channel and SpaceAttention Network for Hyperspectral Image Classification.Remote Sens.2020,12,2035.[CrossRef]

[0262]

[26] Liu,Y.;Gao,L.;Xiao,C.;Qu,Y.;Zheng,K.;Marinoni,A.HyperspectralImage ClassificationBased on a Shuffled Group Convolutional NeuralNetworkwith Transfer Learning.Remote Sens.2020,12,1780.[CrossRef]

[0263]

[27] Shen,J.;Zhang,T.;Wang,Y.;Wang,R.;Wang,Q.;Qi,M.A Dual-ModelArchitecture with Grouping-Attention-Fusion for Remote Sensing SceneClassification.Remote Sens.2021,13,433.[CrossRef]

[0264]

[28] W.Li et al.,“Classification of high-spatial-resolution remotesensing scenes method using transfer learning and deep convolutional neuralnetwork,”IEEE J.Sel.TopicsAppl.Earth Observ.Remote Sens.,vol.13,pp.1986–1995,2020.

[0265]

[29] J.Xie,N.He,L.Fang,and A.Plaza,“Scale-free convolutional neuralnetwork for remote sensing scene classification,”IEEE Trans.Geosci.RemoteSens.,vol.57,no.9,pp.6916–6928,Sep.2019.

[0266]

[30] Y.Zhang,X.Zheng,Y.Yuan,and X.Lu,“Attribute-cooperatedconvolutional neural network for remote sensing image classification,”IEEETrans.Geosci.Remote Sens.,vol.58,no.12,pp.8358–8371,Dec.2020.

[0267]

[31] X.Lu,H.Sun,and X.Zheng,“A feature aggregation convolutionalneural network for remote sensing scene classification,”IEEETrans.Geosci.Remote Sens.,vol.57,no.10,pp.7894–7906,Oct.2019.

[0268]

[32] K.Xu,H.Huang,P.Deng,and Y.Li,“Deep feature aggregation frameworkdriven by graph convolutional network for scene classification in remotesensing,”IEEE Trans.Neural Netw.Learn.Syst.,early access,Apr.15,2021,doi:10.1109 / TNNLS.2021.3071369.

[0269]

[33] X.Wang,S.Wang,C.Ning,and H.Zhou,“Enhanced feature pyramid networkwith deep semantic embedding for remote sensing scene classification,”IEEETrans.Geosci.Remote Sens.,vol.59,no.9,pp.7918–7932,Sep.2021.

[0270]

[34] G.Cheng,C.Yang,X.Yao,L.Guo,and J.Han,“When deep learning meetsmetric learning:Remote sensing image scene classification via learningdiscriminative CNNs,”IEEE Trans.Geosci.Remote Sens.,vol.56,no.5,pp.2811–2821,May2018.

[0271]

[35] J.Wang,Y.Zhong,Z.Zheng,A.Ma,and L.Zhang,“RSNet:The search forremote sensing deep neural networks in recognition tasks,”IEEETrans.Geosci.Remote Sens.,vol.59,no.3,pp.2520–2534,Mar.2021.

[0272]

[36] Cuiping Shi,Xin Zhao,Liguo Wang*.A Multi-branch Feature FusionStrategy Based on Attention Mechanism for Remote Sensing Image SceneClassification[J].Remote Sensing,2021,13(10),1950.

[0273]

[37] X.Tang,Q.Ma,X.Zhang,F.Liu,J.Ma,and L.Jiao,“Attention consistentnetwork for remote sensing scene classification,”IEEE J.Sel.TopicsAppl.EarthObserv.Remote Sens.,vol.14,pp.2030–2045,2021.

[0274]

[38] S.-B.Chen,Q.-S.Wei,W.-Z.Wang,J.Tang,B.Luo,and Z.-Y.Wang,“Remotesensing scene classification via multi-branch local attention network,”IEEETrans.Image Process.,vol.31,pp.99–109,2021.

[0275]

[39] P.Lv,W.Wu,Y.Zhong,F.Du and L.Zhang,"SCViT:A Spatial-ChannelFeature Preserving Vision Transformer for Remote Sensing Image SceneClassification,"in IEEE Transactions on Geoscience and Remote Sensing,vol.60,pp.1-12,2022,Art no.4409512,doi:10.1109 / TGRS.2022.3157671.

[0276]

[40] Ma J,Li M,Tang X,et al.Homo–heterogenous transformer learningframework for RS scene classification[J].IEEE Journal ofSelected TopicsinApplied Earth Observations and Remote Sensing,2022,15:2223-2239.

[0277]

[41] Shi J,Liu W,Shan H,et al.Remote Sensing Scene ClassificationBased on Multibranch Fusion Attention Network[J].IEEE Geoscience and RemoteSensing Letters,2023,20:1-5.

[0278]

[42] Y.Yang and S.Newsam,“Bag-of-visual-words and spatial extensionsfor land-use classification,”in Proc.18th SIGSPATIALInt.Conf.Adv.Geograph.Inf.Syst.,2010,pp.270–279.

[0279]

[43] G.S.Xia et al.,“AID:A benchmark data set for performanceevaluation of aerial scene classification,”IEEE Trans.Geosci.Remote Sens.,vol.55,no.7,pp.3965–3981,Jul.2017.

[0280]

[44] G.Cheng,J.Han,and X.Lu,“Remote sensing image sceneclassification:Benchmark and state ofthe art,”Proc.IEEE,vol.105,no.10,pp.1865–1883,Oct.2017.

[0281]

[45] W.Zhang,P.Tang,andL.Zhao,“Remote sensing image sceneclassification using CNN-CapsNet,”Remote Sens.,vol.11,no.5,p.494,2019.

[0282]

[46] N.He,L.Fang,S.Li,J.Plaza,andA.Plaza,“Skip-connected covariancenetwork for remote sensing scene classification,”IEEETrans.NeuralNetw.Learn.Syst.,vol.31,no.5,pp.1461–1474,May 2020.

[0283]

[47] H.Sun,S.Li,X.Zheng,and X.Lu,“Remote sensing scene classificationby gated bidirectional network,”IEEE Trans.Geosci.Remote Sens.,vol.58,no.1,pp.82–96,Jan.2020.

[0284]

[48] S.Wang,Y.Guan,and L.Shao,“Multi-granularity canonical appearancepooling for remote sensing scene classification,”IEEE Trans.Image Process.,vol.29,pp.5396–5407,2020.

[0285]

[49] Q.Bi,K.Qin,Z.Li,H.Zhang,K.Xu,and G.-S.Xia,“A multiple-instancedensely-connected ConvNet for aerial scene classification,”IEEE Trans.ImageProcess.,vol.29,pp.4911–4926,2020.

[0286]

[50] X.Wang,L.Duan,A.Shi,and H.Zhou,“Multilevel feature fusionnetworks with adaptive channel dimensionality reduction for remote sensingscene classification,”IEEE Geosci.Remote Sens.Lett.,vol.19,pp.1–5,2021.

[0287]

[51] G.Zhang et al.,“Amultiscale attention network for remote sensingscene images classification,”IEEE J.Sel.TopicsAppl.Earth Observ.Remote Sens.,vol.14,pp.9530–9545,2021.

[0288]

[52] X.Wang,L.Duan,C.Ning,and H.Zhou,“Relation-attention networks forremote sensing scene classification,”IEEE J.Sel.TopicsAppl.EarthObserv.Remote Sens.,vol.15,pp.422–439,2021.

[0289]

[53] Dosovitskiy A,Beyer L,Kolesnikov A,et al.An image is worth 16x16words:Transformers for image recognition at scale[J].arXivpreprint arXiv:2010.11929,2020.

[0290]

[54] X.Tang,M.Li,J.Ma,X.Zhang,F.Liu and L.Jiao,"EMTCAL:EfficientMultiscale Transformer and Cross-Level Attention Learning for Remote SensingScene Classification,"in IEEE Transactions on Geoscience and Remote Sensing,vol.60,pp.1-15,2022,Art no.5626915,doi:10.1109 / TGRS.2022.3194505.

[0291]

[55] Q.Meng,M.Zhao,L.Zhang,W.Shi,C.Su and L.Bruzzone,"MultilayerFeature FusionNetwork With SpatialAttention and Gated Mechanism for RemoteSensing Scene Classification,"in IEEE Geoscience and Remote Sensing Letters,vol.19,pp.1-5,2022,Artno.6510105,doi:10.1109 / LGRS.2022.3173473.

[0292]

[56] P.Deng,H.Huang and K.Xu,"A Deep Neural Network Combined WithContext Features for Remote Sensing Scene Classification,"in IEEE Geoscienceand Remote Sensing Letters,vol.19,pp.1-5,2022,Art no.8000405,doi:10.1109 / LGRS.2020.3016769.

[0293]

[57] S.Chen,W.Li,Y.Cao and X.Lu,"Combining the Convolution andTransformer for Classification ofSmoke-Like Scenes in Remote Sensing Images,"in IEEE Transactions on Geoscience and Remote Sensing,vol.60,pp.1-19,2022,Artno.4512519,doi:10.1109 / TGRS.2022.3208120.

[0294]

[58] Y.Hu,X.Huang,X.Luo,J.Han,X.Cao and J.Zhang,"Variational Self-Distillation for Remote Sensing Scene Classification,"in IEEE Transactions onGeoscience and Remote Sensing,vol.60,pp.1-13,2022,Art no.5627313,doi:10.1109 / TGRS.2022.3194549.

[0295]

[59] Shi C,Zhang X,Wang L.A Lightweight Convolutional Neural NetworkBased on Channel Multi-Group Fusion for Remote Sensing SceneClassification.Remote Sensing.2022;14(1):9.https: / / doi.org / 10.3390 / rs14010009

[0296]

[60] Y.Yang,X.Tang,Y.-M.Cheung,X.Zhang and L.Jiao,"SAGN:Semantic-AwareGraph Network for Remote Sensing Scene Classification,"in IEEE Transactionson Image Processing,vol.32,pp.1011-1025,2023,doi:10.1109 / TIP.2023.3238310.

[0297]

[61] Zhang,B.;Zhang,Y.;Wang,S.A lightweight and discriminative modelfor remote sensing scene classifification with multidilationpoolingmodule.IEEE J.Sel.TopicsAppl.Earth Observ.Remote Sens.2019,12,2636–2653.[CrossRef]

[0298]

[62] G.Huang,Z.Liu,L.Van.Der.Maaten,et al.Densely connectedconvolutional networks[C] / / Proceedings of the IEEE conference on computervision andpatternrecognition.2017:4700-4708.

[0299]

[63] Liu,M.;Jiao,L.;Liu,X.;Li,L.;Liu,F.;Yang,S.C-CNN:Contourletconvolutional neural networks.IEEE Trans.Neural Netw.Learn.Syst.2021,32,2636–2649.[CrossRef]

[0300]

[64] F.Li,R.Feng,W.Han,et al.High-resolution remote sensing image sceneclassification via key filter bank based on convolutional neural network[J].IEEETransactions on Geoscience andRemote Sensing,2020,58(11):8077-8092。

Claims

1. A remote sensing scene image classification method based on a re-parameterized feature aggregation convolutional neural network, characterized by: The specific process of the method is: Construct a re-parameterized feature aggregation convolutional neural network model; Classify the remote sensing scene images based on the constructed re-parameterized feature aggregation convolutional neural network model; The re-parameterized feature aggregation convolutional neural network model includes a trunk branch, an aggregation branch and a fusion branch; The trunk branch includes a first feature extraction module, a first RepFormer module, a first average pooling layer, a second RepFormer module, a second average pooling layer, a third RepFormer module, a third average pooling layer, a fourth RepFormer module, and a first linear layer; The aggregation branch includes a second feature extraction module, a third feature extraction module, a first multi-head mixed convolution module, a second multi-head mixed convolution module, a first cascade layer, a fourth feature extraction module, a third multi-head mixed convolution module, and a second linear layer; The fusion branch includes a second cascade layer and a feature fusion classifier; The working process of the re-parameterized feature aggregation convolutional neural network model is: The remote sensing scene image is input into the first feature extraction module, and the first feature extraction module outputs the shallow feature A of the image; The shallow feature A of the image is sequentially input into the first RepFormer module, the first average pooling layer, the second RepFormer module, and the second average pooling layer. The second average pooling layer outputs feature B. Feature B is sequentially input into the third RepFormer module and the third average pooling layer, and the third average pooling layer outputs feature C; Feature C is input into the fourth RepFormer module, and the fourth RepFormer module outputs feature D; Feature D outputs the feature into the first linear layer, and the first linear layer outputs the predicted probability; Feature B is input into the second feature extraction module, and the second feature extraction module outputs feature E; Feature E is input into the first multi-head mixed convolution module, and the first multi-head mixed convolution module outputs feature F; Feature C is input into the third feature extraction module, and the third feature extraction module outputs feature G; Feature G is input into the second multi-head mixed convolution module, and the second multi-head mixed convolution module outputs feature H; Input features F and H into the first cascade layer, and the first cascade layer outputs feature I; Feature I is input into the fourth feature extraction module, and the fourth feature extraction module outputs feature J; Feature J is input into the third multi-head mixed convolution module, and the third multi-head mixed convolution module outputs feature K; Feature K is input into the second linear layer, and the second linear layer outputs the predicted probability; Input feature D and feature K into the second cascade layer, and the second cascade layer outputs feature L; Feature L is input into the feature fusion classifier, and the feature fusion classifier outputs feature O.

2. The remote sensing scene image classification method based on a re-parameterized feature aggregation convolutional neural network according to claim 1 is characterized in that: The first feature extraction module includes, in sequence, a 3×3 convolution with a stride of 2, a first batch normalization layer, a 3×3 convolution with a stride of 2, and a second batch normalization layer; The working process of the first feature extraction module is: The remote sensing scene image is sequentially input into the 3×3 convolution with the first step size of 2, the first batch normalization layer, the second 3×3 convolution with the second step size of 2, and the second batch normalization layer. The second batch normalization layer outputs the shallow features A of the image.

3. The remote sensing scene image classification method based on a re-parameterized feature aggregation convolutional neural network according to claim 2 is characterized in that: The second feature extraction module includes a third 3×3 convolution, a first 1×1 convolution, a third batch normalization layer BN, and a first ReLU activation function layer in sequence; The working process of the second feature extraction module is as follows: Feature B is sequentially input into the third 3×3 convolution, the first 1×1 convolution, the third batch normalization layer BN, and the first ReLU activation function layer, and the first ReLU activation function layer outputs feature E.

4. The remote sensing scene image classification method based on a re-parameterized feature aggregation convolutional neural network according to claim 3 is characterized in that: The third feature extraction module includes a fourth 3×3 convolution, a second 1×1 convolution, a fourth batch normalization layer BN, and a second ReLU activation function layer in sequence; The working process of the third feature extraction module is: Feature C is sequentially input into the fourth 3×3 convolution, the second 1×1 convolution, the fourth batch normalization layer BN, and the second ReLU activation function layer, and the second ReLU activation function layer outputs feature G.

5. The remote sensing scene image classification method based on a re-parameterized feature aggregation convolutional neural network according to claim 4 is characterized in that: Each of the first RepFormer module, the second RepFormer module, the third RepFormer module, and the fourth RepFormer module comprises: The fifth 3×3 convolution, the fifth batch normalization layer BN, the third 1×1 convolution, the sixth batch normalization layer BN, the seventh batch normalization layer BN, the third linear layer, the GELU activation function layer, and the fourth linear layer; The working process of each RepFormer module is: The feature β is sequentially input into the fifth 3×3 convolution and the fifth batch normalization layer BN, and the fifth batch normalization layer BN outputs the feature β′; The feature β is sequentially input into the third 1×1 convolution and the sixth batch normalization layer BN, and the sixth batch normalization layer BN outputs the feature β″; The feature β is input into the seventh batch normalization layer BN, and the seventh batch normalization layer BN outputs the feature β″′; Add the features β′, β″, and β″′ element by element to get the feature feature Input the third linear layer, GELU activation function layer, and fourth linear layer in sequence, and the fourth linear layer outputs features The characteristics and Features Add each element to get the feature feature As the output features of each RepFormer module.

6. The remote sensing scene image classification method based on a re-parameterized feature aggregation convolutional neural network according to claim 5, characterized in that: Each of the first multi-head mixed convolution module, the second multi-head mixed convolution module, and the third multi-head mixed convolution module includes: 1×1 depth convolution layer, 3×3 depth convolution layer, 5×5 depth convolution layer, 7×7 depth convolution layer, 1×1 group convolution, eighth batch normalization layer BN, GELU activation function layer, fourth 1×1 convolution layer; The working process of each multi-head mixed convolution module is: The input channel is divided into 4 heads, and the input feature x is divided into 4 heads [x1,x2,x3,x4], each head has M channels; Input x1 into the 1×1 depth convolution layer, and the 1×1 depth convolution layer outputs the feature Input x2 into the 3×3 depth convolution layer, and the 3×3 depth convolution layer outputs the feature Input x3 into the 5×5 depth convolution layer, and the 5×5 depth convolution layer outputs the feature Input x4 into the 7×7 depth convolution layer, and the 7×7 depth convolution layer outputs features Take features separately The features of the first channel in , 4 features are concatenated as the first new mixed feature; Take features separately The features of the second channel in , 4 features are concatenated as the second new mixed feature; Until Take features separately The features of the Mth channel in , 4 features are concatenated as the Mth new mixed features; Each new mixed feature is input into a 1×1 group of convolutions, and each 1×1 group of convolution output features is input into the eighth batch normalization layer BN and the GELU activation function layer in sequence, and the GELU activation function layer outputs features respectively; The M features output by the GELU activation function layer are cascaded, and the cascaded features are input into the fourth 1×1 convolutional layer. The output features of the fourth 1×1 convolutional layer are used as the output features of each multi-head mixed convolution module.

7. The remote sensing scene image classification method based on a re-parameterized feature aggregation convolutional neural network according to claim 6, characterized in that: The feature fusion classifier comprises: One-dimensional convolution layer with a convolution kernel size of 5, Sigmoid activation function layer, fifth linear layer, GELU activation function layer, sixth linear layer; The working process of the feature fusion classifier is as follows: The feature L is sequentially input into the one-dimensional convolution layer with a convolution kernel size of 5 and the Sigmoid activation function layer. The output feature of the Sigmoid activation function layer is multiplied element-wise with the feature L, and the obtained feature is sequentially input into the fifth linear layer, the GELU activation function layer, and the sixth linear layer. The sixth linear layer outputs feature O, which is used as the output feature of the feature fusion classifier.

8. The remote sensing scene image classification method based on a re-parameterized feature aggregation convolutional neural network according to claim 7, characterized in that: The training process of the re-parameterized feature aggregation convolutional neural network model is: Suppose there are K samples and M categories in the sample set The total prediction loss of the reparameterized feature aggregation convolutional neural network model is The prediction loss of the trunk branch is The prediction loss of the aggregation branch is The prediction loss of the fusion branch is Prediction loss of trunk branches and the prediction loss of the aggregation branch Respectively expressed as: where w A and w B Represent the parameters of the aggregation branch and trunk branch respectively; y km Indicates the label of sample k. It takes 1 if the label of sample k is m, otherwise it takes 0. p(m|x k ) represents the sample x k The probability of belonging to category m; p(m|x k ;w B ) represents the sample x under the parameters of the trunk branch k The probability of belonging to category m; p(m|x k ;w A ) represents the sample x under the parameters of the aggregation branch k The probability of belonging to category m; Prediction loss based on trunk branches and the prediction loss of the aggregation branch Get auxiliary prediction loss It is expressed as: in, is the prediction loss of the aggregated features, The prediction loss for the backbone features; Prediction loss of fused branches It is expressed as: Among them, p(m|x k ; W) represents the parameter W of the fusion branch and the sample x k The probability of belonging to category m; W represents the parameter of the fusion branch; Based on auxiliary prediction loss The prediction loss of the fusion branch is Obtain the total prediction loss of the reparameterized feature aggregation convolutional neural network model It is expressed as: The reparameterized feature aggregation convolutional neural network model is trained until the total prediction loss of the reparameterized feature aggregation convolutional neural network model is Convergence, obtain the trained re-parameterized feature aggregation convolutional neural network model.

9. The remote sensing scene image classification method based on a re-parameterized feature aggregation convolutional neural network according to claim 8, characterized in that: The constructed re-parameterized feature aggregation convolutional neural network model is used to classify the remote sensing scene image to be measured; the specific process is: The image of the remote sensing scene to be tested is input into the trained re-parameterized feature aggregation convolutional neural network model, and the trained re-parameterized feature aggregation convolutional neural network model outputs the category of the image of the remote sensing scene to be tested; The specific process is: The remote sensing scene image to be measured is input into the first feature extraction module, and the first feature extraction module outputs the shallow features of the image Shallow features of images Input the first RepFormer module, the first average pooling layer, the second RepFormer module, the second average pooling layer, and the second average pooling layer outputs features feature Input the third RepFormer module and the third average pooling layer in sequence, and the third average pooling layer outputs features feature Input the fourth RepFormer module, the fourth RepFormer module outputs features feature Input the second feature extraction module, the second feature extraction module outputs the feature feature Input the first multi-head mixed convolution module, the first multi-head mixed convolution module outputs features feature Input the third feature extraction module, the third feature extraction module outputs the feature feature Input the second multi-head mixed convolution module, the second multi-head mixed convolution module outputs features The characteristics and Features Input the first cascade layer, the first cascade layer outputs features feature Input the fourth feature extraction module, the fourth feature extraction module outputs the feature feature Input the third multi-head mixed convolution module, the third multi-head mixed convolution module outputs features The characteristics and Features Input the second cascade layer, the second cascade layer outputs features feature Input the feature fusion classifier, and the feature fusion classifier outputs the classification result.

10. The remote sensing scene image classification method based on a re-parameterized feature aggregation convolutional neural network according to claim 9, characterized in that: Shallow features of the image Input the first RepFormer module, the first average pooling layer, the second RepFormer module, the second average pooling layer, and the second average pooling layer outputs features feature Input the third RepFormer module and the third average pooling layer in sequence, and the third average pooling layer outputs features feature Input the fourth RepFormer module, the fourth RepFormer module outputs features The specific process is: Each of the first RepFormer module, the second RepFormer module, the third RepFormer module, and the fourth RepFormer module includes: 3×3 convolution layer, batch normalization layer BN, linear layer, GELU activation function layer, linear layer; The working process of each RepFormer module in the first RepFormer module, the second RepFormer module, the third RepFormer module, and the fourth RepFormer module is as follows: The features are sequentially input into the 3×3 convolutional layer and the batch normalization layer BN, and the batch normalization layer BN outputs the feature P; Feature P is sequentially input into the linear layer, GELU activation function layer, and linear layer, and outputs feature Q; Features P and Q are added element by element, and the added features are used as the output features of each RepFormer module.

Images