A remote sensing image scene classification method based on a transformer
By employing preprocessing, image enhancement, and improved Transformer feature extraction, the problems of noise interference, cloud and fog occlusion, and data imbalance in remote sensing image scene classification were solved, achieving efficient and accurate remote sensing image scene classification and improving the model's robustness and computational efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI YINFAN INFORMATION TECH CO LTD
- Filing Date
- 2026-01-28
- Publication Date
- 2026-07-24
AI Technical Summary
Traditional remote sensing image scene classification methods struggle to balance the efficiency and accuracy of feature extraction. Transformer-based methods suffer from high computational cost, insufficient adaptation to multi-scale ground features, and issues such as noise, cloud and fog interference, and data imbalance when processing remote sensing images.
The preprocessing stage combines multi-scale denoising and adaptive threshold segmentation, image enhancement and geometric transformation and class balancing strategies, and the improved Transformer backbone network optimizes feature extraction through hierarchical sparse attention modules, constructs multi-scale parallel branches, and outputs multi-scale fused basic features.
It effectively eliminates noise and cloud/shadow interference, solves the problem of imbalanced class data in small samples, improves classification accuracy and computational efficiency, enhances the robustness of the model, and provides efficient and accurate technical support for land spatial planning and environmental monitoring.
Smart Images
Figure CN121982500B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image recognition technology, and in particular relates to a method for classifying remote sensing image scenes based on Transformer. Background Technology
[0002] Remote sensing image scene classification, as an important branch of image recognition technology, has key application value in fields such as land spatial planning, environmental monitoring, and disaster emergency response. With the development of remote sensing technology, the ground feature information in high-resolution remote sensing images is becoming increasingly complex, encompassing diverse scene types such as buildings, vegetation, and water bodies. It is also frequently affected by interference factors such as noise, clouds, fog, and shadows. Furthermore, some scene categories suffer from insufficient sample sizes and uneven data distribution, posing a significant challenge to accurate classification. Traditional classification methods struggle to balance the efficiency and accuracy of feature extraction. While Transformer-based methods have advantages in global feature capture, they still face pain points such as high computational cost and insufficient adaptation to multi-scale ground feature features when processing remote sensing images. Therefore, there is an urgent need to optimize technical solutions to meet practical application requirements. Summary of the Invention
[0003] To address the technical problems existing in the background art described above, this invention proposes a Transformer-based method for remote sensing image scene classification.
[0004] To achieve the above objectives, the technical solution adopted by the present invention includes the following steps:
[0005] S1. Preprocess the remote sensing image; the preprocessing includes image noise removal, removing cloud, fog and shadow areas through threshold segmentation and morphological operations, unifying the image resolution to a preset size, and performing standardization processing on pixel values based on the statistical mean and standard deviation of the remote sensing dataset to obtain the preprocessed image.
[0006] S2. Perform image enhancement on the preprocessed image; the image enhancement includes geometric enhancement and category balance enhancement.
[0007] S3. The preprocessed and enhanced remote sensing image is divided into non-overlapping image blocks, mapped to fixed-dimensional feature vectors, positional encoding is added, and the blocks are input into an improved Transformer backbone network. The improved Transformer backbone network optimizes feature extraction efficiency through a hierarchical sparse attention module, and at the same time constructs multi-scale parallel branches to extract features of different granularities and complete the fusion, outputting multi-scale fused basic features.
[0008] S4. After passing the multi-scale fused basic features through a fully connected layer and a classification activation function, the output remote sensing image is predicted to have a scene classification probability distribution. The final scene classification result is then determined based on the maximum probability criterion.
[0009] Preferably, step S1 involves preprocessing the remote sensing image. This preprocessing includes image noise removal, removing cloud, fog, and shadow areas through threshold segmentation and morphological operations, unifying the image resolution to a preset size, and standardizing pixel values based on the statistical mean and standard deviation of the remote sensing dataset to obtain the preprocessed image. The specific implementation is as follows:
[0010] The original remote sensing image I is decomposed into multiple scales using a Gaussian pyramid. For each scale... Calculate pixels Local noise variance at: ,in, For The 3×3 neighborhood centered on the center Given the pixel mean of the neighborhood, adaptive weights are constructed based on the noise variance at each scale. ;
[0011] Nonlocal mean variant denoising is applied to images at all scales: ,in, For 11x11 search windows, The denoised image is obtained by fusing the denoising results from each scale using adaptive smoothing parameters. Where L is the total number of scales, For the first The mean value of denoising across all points at the scale;
[0012] Based on prior statistics from remote sensing datasets, adaptive thresholds for clouds, fog, and shadows are determined. Morphological optimization operations are then used to remove regions, resulting in the removed image. ;
[0013] right Unify to preset size: ,in For point The coordinates of the four surrounding pixels;
[0014] Will The region is divided into multiple non-overlapping sub-regions, and the m-th sub-region is denoted as . The sub-region was calculated. The mean and standard deviation are used to perform a standardization operation for each pixel: ,in To avoid extremely small constants with a denominator of 0, The mean and standard deviation of the m-th region are used to obtain the preprocessed image.
[0015] Preferably, the image enhancement in step S2 includes geometric enhancement and class balance enhancement, specifically implemented as follows:
[0016] The preprocessed image is geometrically enhanced by rotation to obtain multiple images at different angles; class balance enhancement is then performed on all geometrically enhanced images to extract pixel features F from the images.
[0017] Weighted distance calculation based on pixel features: Where D is the total dimension of pixel features. For dimension weights, The weighted distance is the pixel feature vector sum of two remote sensing image samples whose distance is to be calculated.
[0018] For each sample in the minority class Calculate its comparison with other samples of the same type. Choose the one with the smallest distance. Each sample yields a nearest neighbor set;
[0019] Randomly select a sample from the nearest neighbor set. Intermediate features are generated by interpolation. ;
[0020] By statistically analyzing the pixel value distribution of all original samples across all small sample categories, a reasonable region for each channel is obtained. The interpolated region is then... The image is restored. For each pixel in the restored image, the pixel variance of the 3×3 neighborhood is calculated. If the pixel variance is greater than the average local neighborhood variance of the original samples of the small sample class, the interpolation is performed again until the condition is met.
[0021] For small sample categories, repeat the above steps to generate new samples until the set sample balance threshold is reached.
[0022] Preferably, the improved Transformer backbone network in step S3 optimizes feature extraction efficiency through hierarchical sparse attention modules as follows:
[0023] The non-overlapping image patches are segmented and their saliency scores are calculated for each patch using a lightweight CNN.
[0024] Patches are divided into high-significance and low-significance groups based on their saliency scores. The grouping is determined by an adaptive threshold obtained from the dataset statistics, which is then used to generate a dynamic sparse mask. The mask retains only the fully connected connections between high-significance patches and the sparse connections between low-significance patches.
[0025] By incorporating the mask into self-attention computation, hierarchical sparse attention output features are obtained.
[0026] Preferably, the rules for sparse connections between low-salience patches include:
[0027] All patches in the low saliency group are divided into non-overlapping grid cells according to image spatial coordinates, and each cell is denoted as g, where the grid size is adaptively determined by the total number of patches in the low saliency group.
[0028] For each grid cell g, calculate the pixel feature similarity score of its internal patches to characterize background homogeneity: ,in, Let g be the number of patches in the g-th grid. These are the pixel mean and standard deviation of the patch, respectively;
[0029] The corresponding pixel feature similarity scores are mapped to connection retention intervals according to preset rules.
[0030] For each grid cell, anchor patch is dynamically selected at the reserved interval, and only the connection of anchor patch is retained to complete the sparse connection between low saliency patches.
[0031] As a preferred embodiment, based on the improved Transformer backbone network in step S3, a multi-scale parallel branch is constructed to extract features of different granularities and perform fusion, resulting in the output of multi-scale fused basic features. The specific implementation of this method is as follows:
[0032] Based on the prior features of land cover scale in remote sensing images, remote sensing image patches are adaptively divided into non-overlapping image patches of different scales;
[0033] Features at each scale are extracted separately by optimizing feature extraction efficiency through a hierarchical sparse attention module.
[0034] In the high significance group of patches at the fine scale, the top 5% of patches with the highest significance scores are selected as the cross-scale anchor set to obtain the corresponding feature set.
[0035] For each patch at the fine scale, calculate its center coordinates in the original image, map them to the coarse scale coordinates according to the scale ratio, find the coarse scale anchor point containing the coarse scale coordinates, and map the set of fine scale patches to the anchor point.
[0036] Enhance the feature vectors of the coarse and fine scales of the patch, and determine the rectangular region corresponding to each patch in the original image for each scale. Assign the fixed-dimensional feature vector corresponding to the patch to all pixel positions within the rectangular region to obtain the corresponding global features.
[0037] The information content of each scale is calculated based on the global features. Then, the information content is transformed into normalized weights through Softmax, and the global features of the two scales are weighted and summed to obtain the fused features.
[0038] The fused features are then subjected to attention-gated noise filtering to finally output multi-scale fused basic features.
[0039] Compared with existing technologies, the advantages and positive effects of this invention are as follows: the preprocessing stage combines multi-scale denoising and adaptive threshold segmentation to effectively remove interference such as noise, fog, and shadows; image enhancement integrates geometric transformation and class balancing strategies to solve the problem of imbalanced class data in small samples. The improved Transformer simplifies computation through hierarchical sparse attention modules, while constructing multi-scale parallel branches to fuse features of different granularities, accurately adapting to complex terrain scenarios. The overall solution balances classification accuracy and computational efficiency, significantly enhancing model robustness and providing more accurate and efficient technical support for practical applications such as land spatial planning and environmental monitoring. Attached Figure Description
[0040] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0041] Figure 1 This is a flowchart of a Transformer-based remote sensing image scene classification method. Detailed Implementation
[0042] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.
[0043] Numerous specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways than those described herein, and therefore the invention is not limited to the specific embodiments disclosed in the following specification.
[0044] This embodiment addresses the technical pain points in existing remote sensing image scene classification, such as noise interference, cloud and fog occlusion, imbalanced sample distribution, high computational cost of Transformer models, and insufficient adaptation to multi-scale ground feature features. It verifies the technical advantages of this invention in improving classification accuracy, optimizing computational efficiency, and enhancing model robustness by implementing core technical solutions in stages, including preprocessing, image enhancement, improved Transformer feature extraction, and classification decision-making. This provides an efficient and accurate remote sensing image scene classification solution for practical applications such as land spatial planning, environmental monitoring, and disaster emergency response. The specific implementation process is as follows: Figure 1 As shown.
[0045] First, the remote sensing image is preprocessed. The preprocessing includes image noise removal, removing cloud, fog and shadow areas through threshold segmentation and morphological operations, unifying the image resolution to a preset size, and performing standardization processing on the pixel values based on the statistical mean and standard deviation of the remote sensing dataset to obtain the preprocessed image.
[0046] Specifically, the original remote sensing image I is decomposed into multiple scales using a Gaussian pyramid. For each scale... Calculate pixels Local noise variance at: ,in, For The 3×3 neighborhood centered on the center Given the pixel mean of the neighborhood, adaptive weights are constructed based on the noise variance at each scale. Adaptive weights are constructed based on the noise variance at each scale. ,in, Let be the global noise variance at the l-th layer scale, and be the mean of the variances of all pixels.
[0047] Nonlocal mean variant denoising is applied to images at all scales: ,in, For 11x11 search windows, The denoised image is obtained by fusing the denoising results from each scale using adaptive smoothing parameters. Where L is the total number of scales, For the first The mean of denoising at all points in the scale.
[0048] Based on prior statistics from remote sensing datasets, adaptive thresholds for clouds, fog, and shadows are determined. Morphological optimization operations are then used to remove regions, resulting in the removed image. Specifically, an initial segmentation mask is used based on an adaptive threshold to obtain the region to be removed and the effective region, and then morphological operations are performed to optimize the mask: ,in This is a preliminary segmentation mask, where 1 represents the region to be removed and 0 represents the valid region. For expansion operation, For corrosion operation, It is a 3×3 rectangular structural element. The structural element is a 5x5 disk. Then, interference areas are removed. ,in, This is a pixel-by-pixel multiplication operation.
[0049] right Unify to preset size: ,in For point The coordinates of the four surrounding pixels are (0,1), (0,0), (1,1), and (1,0).
[0050] Will The region is divided into multiple non-overlapping sub-regions, and the m-th sub-region is denoted as . The sub-region was calculated. The mean and standard deviation are used to perform a standardization operation for each pixel: ,in To avoid extremely small constants with a denominator of 0, The mean and standard deviation of the m-th region are used to obtain the preprocessed image.
[0051] The preprocessed image is then enhanced; the image enhancement includes geometric enhancement and category balance enhancement.
[0052] Furthermore, the pre-processed image undergoes geometric enhancement and rotation to obtain multiple images at different angles; multiple angles can be selected from 0 to 360 degrees to obtain images at different angles.
[0053] All geometrically enhanced images are subjected to class-balanced enhancement, and pixel features F are extracted from the images. The extracted pixel features include the mean, standard deviation, and covariance between channels. The captured images are RGB images, so the number of channels is 3.
[0054] Weighted distance calculation based on pixel features: Where D is the total dimension of pixel features. For dimension weights, This is the weighted distance between the pixel feature vectors of two remote sensing image samples whose distance is to be calculated.
[0055] For each sample in the minority class Calculate its comparison with other samples of the same type. Choose the one with the smallest distance. Each sample yields a nearest neighbor set.
[0056] Randomly select a sample from the nearest neighbor set. Intermediate features are generated by interpolation. ; ,in These are random interpolation coefficients ranging from 0 to 1.
[0057] By statistically analyzing the pixel value distribution of all original samples across all small sample categories, a reasonable region for each channel is obtained. The interpolated region is then... The image is restored, and the pixel variance of the 3×3 neighborhood of each pixel in the restored image is calculated. If the pixel variance is greater than the average local neighborhood variance of the original samples of the small sample class, the interpolation is re-performed until the condition is met.
[0058] For small sample categories, repeat the above steps to generate new samples until the set sample balance threshold is reached, and obtain the image enhancement result.
[0059] Next, the preprocessed and enhanced remote sensing images are segmented into non-overlapping image patches, mapped to fixed-dimensional feature vectors, positional encoding is added, and the patches are input into an improved Transformer backbone network. The Transformer is a deep learning model based on the self-attention mechanism. It was initially used in the field of natural language processing (NLP) and has since been widely applied to computer vision (CV) tasks (such as image classification and object detection). Its core advantage lies in its ability to capture global features. The input image is segmented into non-overlapping image patches of fixed size. Each patch is mapped to a fixed-dimensional feature vector (i.e., embedding vector) through linear projection. Positional Encoding: Since the self-attention mechanism lacks positional sensitivity, spatial positional information is injected into the feature vector by adding positional encoding vectors. Self-Attention Mechanism Calculation: The core module is Multi-Head Attention, which calculates the similarity between each feature vector (query Q) and all feature vectors (key K) to obtain attention weights, and then weighted sums with the value vector (V) to achieve global feature interaction and dependency capture. Feedforward Neural Network (FFN) and Normalization: Layer Normalization and the feedforward neural network are used to perform nonlinear transformations and enhancements on the features, and finally output globally fused features for subsequent classification or detection tasks.
[0060] The improved Transformer backbone network optimizes feature extraction efficiency through hierarchical sparse attention modules as follows: After segmenting non-overlapping image patches, a lightweight CNN is used to calculate the saliency score of each patch. Specifically, the segmented non-overlapping image patches are denoted as... First, a saliency score for each patch is calculated using a lightweight CNN (such as MobileNetV3): ,in, for Max pooling, This outputs a single-channel significance value for the convolution.
[0061] Patches are divided into high-salience and low-salience groups based on saliency scores. This grouping is determined by an adaptive threshold obtained from dataset statistics, thereby generating a dynamic sparse mask. The mask retains only the fully connected connections between high-salience patches and the sparse connections between low-salience patches. The mask is then integrated into self-attention computation to obtain hierarchical sparse attention output features. Specifically, patches are divided into high-salience and low-salience groups based on saliency scores, and this grouping is determined by an adaptive threshold obtained from dataset statistics, thereby generating a dynamic sparse mask. The mask values for any two patches are: The mask retains only the fully connected connections between highly saliency patches and the sparse connections between low-saliency patches, thus dynamically reducing the computational cost of attention. Finally, the mask is incorporated into the self-attention calculation to obtain hierarchical sparse attention weights: ,in, The output features of the sparse attention mechanism are the final feature vectors after dynamic masking and weight calculation. QKV are the three core matrices of the attention mechanism. Scaling factor The Hadamard product is used, M is a dynamic sparse mask, and finally, softmax normalization is performed to generate the attention weight matrix. Each element value is between [0,1], representing the degree of attention the current patch pays to other patches. The attention weight matrix is multiplied by V to obtain the weighted feature vector, which is the output of the attention mechanism.
[0062] Furthermore, the rules for sparse connections between low-saliency patches include: dividing all patches in the low-saliency group into non-overlapping grid cells according to image spatial coordinates, with each cell denoted as g, where the grid size is adaptively determined by the total number of patches in the low-saliency group; for each grid cell g, calculating the pixel feature similarity score of its internal patches to characterize background homogeneity. ,in, Let g be the number of patches in the g-th grid. These represent the pixel mean and standard deviation of the patch, respectively; the corresponding pixel feature similarity scores are mapped to connection retention intervals according to preset rules; for each grid cell, anchor patch is dynamically selected according to the retention interval, and only the connections of anchor patches are retained to complete the sparse connections between low-salience patches.
[0063] The improved Transformer backbone network simultaneously constructs multi-scale parallel branches to extract features of different granularities and perform fusion, outputting multi-scale fused basic features. The implementation involves: adaptively dividing remote sensing image patches into non-overlapping image patches of different scales based on prior features of land cover scale in the remote sensing image; extracting features at each scale by optimizing feature extraction efficiency through a hierarchical sparse attention module; selecting the top 5% of patches with the highest saliency scores in the fine-scale high-saliency group patches as the cross-scale anchor point set to obtain the corresponding feature set; for each patch at the fine scale, calculating its center coordinates in the original image and mapping them to the scale proportionally. For coarse-scale coordinates, find coarse-scale anchor points containing the coarse-scale coordinates, and map them to a set of fine-scale patches. Enhance the feature vectors of the coarse and fine-scale patches, and determine the corresponding rectangular region in the original image for each patch at each scale. Assign the fixed-dimensional feature vector corresponding to the patch to all pixel positions within the rectangular region to obtain the corresponding global features. Calculate the information content of each scale based on the global features, and then use Softmax to transform the information content into normalized weights. Sum the global features of the two scales using weighted summation to obtain the fused features. Apply attention gating to the fused features to filter noise, and finally output the multi-scale fused basic features.
[0064] Specifically, based on the prior features of land cover scale in remote sensing images, remote sensing image patches are adaptively divided into non-overlapping image patches of different scales; including scale 1: Blocks and Scale 2: Block. Features at each scale c are extracted separately using a method that optimizes feature extraction efficiency through hierarchical sparse attention modules. ;in, This represents the total number of patches at this scale; within the highly significant group of patches at fine scale c, the top 5% of patches by significance score are selected as the cross-scale anchor set. The corresponding feature set is ,in The number of anchor points; for each patch at scale c, calculate its center coordinates in the original image. Map the coordinates to the coarse scale c+1 according to the scale ratio: ,in, For each patch size at the corresponding scale, find the one containing... c+1 scale anchor point This maps to the set of patches at a fine scale c, which is then enhanced by improving the feature vectors of the fine-scale patches at c. ,in To achieve dot product attention, normalize the similarity. For fine-scale interaction weights. c+1 coarse-scale Patch feature vector enhancement: ,in For coarse-scale interaction weights, This is the set of fine-scale patches for anchor point a. For each patch at scale c, its corresponding rectangular region in the original image is determined. The corresponding global features are obtained by assigning values to all pixel positions within the rectangular region. Define the amount of information at each scale. , The feature entropy function is used, and then the information content is transformed into normalized weights through Softmax. The global features of the two scales are then weighted and summed to obtain the fused features. Attention gating is applied to the fused weights to filter noise, and finally, the multi-scale fused basic features are output: first, the gating coefficient is calculated, which is between 0 and 1. ,in, To obtain the fused features through weighted summation, a gating filter operation is finally performed. Thus, the basic features of multi-scale fusion were obtained.
[0065] Finally, the multi-scale fused basic features are processed through a fully connected layer and a classification activation function to output the scene classification prediction probability distribution of the remote sensing image. The final scene classification result is determined based on the maximum probability criterion. Specifically, firstly, the multi-scale fused basic features are globally aggregated using global average pooling adapted to the global semantic characteristics of remote sensing. The spatial dimensions of the features are averaged one-dimensionally to obtain a one-dimensional global feature vector. To eliminate amplitude differences, this vector is L2 normalized to retain the global semantic information of the multi-scale fusion. Then, the category scores are mapped through a fully connected layer: assuming that the remote sensing scene classification includes multiple categories such as city and farmland, the fully connected layer uses a weight matrix and a bias vector as parameters to map the one-dimensional global features into the original scores of multiple categories. Each score corresponds to the initial matching degree of a scene category. Next, a probability distribution is generated using a Softmax activation function: the original scores are transformed into probabilities between 0 and 1, and the sum of the probabilities of all categories is 1. Each probability value corresponds to the confidence that the remote sensing image belongs to that scene category. Finally, the decision is made based on the maximum probability criterion, traversing the probability distribution to find the maximum value, and the category corresponding to this maximum value is the classification result.
[0066] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments for application in other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.
Claims
1. A Transformer-based remote sensing image scene classification method, characterized in that, Includes the following steps: S1. Preprocess the remote sensing image; the preprocessing includes image noise removal, removing cloud, fog and shadow areas through threshold segmentation and morphological operations, unifying the image resolution to a preset size, and performing standardization processing on pixel values based on the statistical mean and standard deviation of the remote sensing dataset to obtain the preprocessed image. S2. Perform image enhancement on the preprocessed image; the image enhancement includes geometric enhancement and category balance enhancement. S3. The preprocessed and enhanced remote sensing image is divided into non-overlapping image blocks, mapped to fixed-dimensional feature vectors, positional encoding is added, and the blocks are input into an improved Transformer backbone network. The improved Transformer backbone network optimizes feature extraction efficiency through a hierarchical sparse attention module, and at the same time constructs multi-scale parallel branches to extract features of different granularities and complete the fusion, outputting multi-scale fused basic features. S4. After passing the multi-scale fused basic features through a fully connected layer and a classification activation function, output the scene classification prediction probability distribution of the remote sensing image, and determine the final scene classification result based on the maximum probability criterion. The specific implementation of optimizing feature extraction efficiency through hierarchical sparse attention modules in the improved Transformer backbone network in step S3 is as follows: The non-overlapping image patches are segmented and their saliency scores are calculated for each patch using a lightweight CNN. Based on the saliency score, the patches are divided into high saliency groups and low saliency groups, and then a dynamic sparse mask is generated; the mask only retains the full connections between high saliency patches and the sparse connections between low saliency patches. By incorporating the mask into self-attention computation, hierarchical sparse attention output features are obtained; In step S3, the improved Transformer backbone network constructs multi-scale parallel branches, extracts features of different granularities, and completes fusion to output multi-scale fused basic features. The specific implementation is as follows: Based on the prior features of land cover scale in remote sensing images, remote sensing image patches are adaptively divided into non-overlapping image patches of different scales; Features at each scale are extracted separately by optimizing feature extraction efficiency through a hierarchical sparse attention module. In the high significance group of patches at the fine scale, the top 5% of patches with the highest significance scores are selected as the cross-scale anchor set to obtain the corresponding feature set. For each patch at the fine scale, calculate its center coordinates in the original image, map them to the coarse scale coordinates according to the scale ratio, find the coarse scale anchor point containing the coarse scale coordinates, and map the set of fine scale patches to the anchor point. Enhance the feature vectors of the coarse and fine scales of the patch, and determine the corresponding rectangular region in the original image for each patch at each scale. Assign the fixed-dimensional feature vector corresponding to the patch to all pixel positions within the rectangular region to obtain the corresponding global features. The information content of each scale is calculated based on the global features. Then, the information content is transformed into normalized weights through Softmax, and the global features of the two scales are weighted and summed to obtain the fused features. The fused features are then subjected to attention-gated noise filtering to finally output multi-scale fused basic features.
2. The remote sensing image scene classification method based on Transformer according to claim 1, characterized in that, In step S1, the remote sensing image is preprocessed. This preprocessing includes image noise removal, removing cloud, fog, and shadow areas through threshold segmentation and morphological operations, unifying the image resolution to a preset size, and standardizing pixel values based on the statistical mean and standard deviation of the remote sensing dataset to obtain the preprocessed image. The specific implementation is as follows: The original remote sensing image is decomposed into multiple scales using a Gaussian pyramid. For each scale, the pixel count is calculated. Local noise variance at: ,in, For The 3×3 neighborhood centered on the center Given the pixel mean of the neighborhood, adaptive weights are constructed based on the noise variance at each scale. ; Nonlocal mean variant denoising is applied to images at all scales: ,in, For 11x11 search windows, The denoised image is obtained by fusing the denoising results from each scale using adaptive smoothing parameters. Where L is the total number of scales, For the first The mean value of denoising across all points at the scale; Based on prior statistics from remote sensing datasets, adaptive thresholds for clouds, fog, and shadows are determined. Morphological optimization operations are then used to remove regions, resulting in the removed image. ; right Unify to preset size: ,in For point The coordinates of the four surrounding pixels; Will The region is divided into multiple non-overlapping sub-regions, and the m-th sub-region is denoted as . The sub-region was calculated. The mean and standard deviation are used to perform a standardization operation for each pixel: ,in To avoid extremely small constants with a denominator of 0, The mean and standard deviation of the m-th region are used to obtain the preprocessed image.
3. The method for classifying remote sensing image scenes based on Transformer according to claim 1, characterized in that, The image enhancement in step S2 includes geometric enhancement and class balance enhancement, specifically implemented as follows: The preprocessed image is geometrically enhanced by rotation to obtain multiple images at different angles; class balance enhancement is then performed on all geometrically enhanced images to extract pixel features F from the images. Weighted distance calculation based on pixel features: Where D is the total dimension of pixel features. For dimension weights, The weighted distance is the pixel feature vector sum of two remote sensing image samples whose distance is to be calculated. For each sample in the smaller sample category, calculate its comparison with other samples in the same category. Choose the one with the smallest distance. Each sample yields a nearest neighbor set; Randomly select a sample from the nearest neighbor set and generate intermediate features using interpolation. ; By statistically analyzing the pixel value distribution of all original samples across all small sample categories, a reasonable region for each channel is obtained. The interpolated region is then... The image is restored. For each pixel in the restored image, the pixel variance of the 3×3 neighborhood is calculated. If the pixel variance is greater than the average local neighborhood variance of the original samples of the small sample class, the interpolation is performed again until the condition is met. For small sample categories, repeat the above steps to generate new samples until the set sample balance threshold is reached.
4. The remote sensing image scene classification method based on Transformer according to claim 1, characterized in that, The rules for sparse connections between low-significance patches include: All patches in the low saliency group are divided into non-overlapping grid cells according to image spatial coordinates, where the grid size is adaptively determined by the total number of patches in the low saliency group. For each grid cell, calculate the pixel feature similarity score of its internal patches to characterize background homogeneity: ,in, Let g be the number of patches in the g-th grid cell. These are the pixel mean and standard deviation of the g-th grid cell Patch, respectively; The corresponding pixel feature similarity scores are mapped to connection retention intervals according to preset rules. For each grid cell, anchor patch is dynamically selected at the reserved interval, and only the connection of anchor patch is retained to complete the sparse connection between low saliency patches.