Remote sensing image segmentation method based on multi-scale feature fusion and index selection mechanism

Abstract

The application discloses a remote sensing image segmentation method based on a multi-scale feature fusion and index selection mechanism, and belongs to the technical field of remote sensing image processing. The method comprises the following steps: different scale feature information in a remote sensing image is extracted by using a multi-scale convolution network, and a fusion feature map of the remote sensing image is obtained; a feature aggregation is performed on the fusion feature map of the remote sensing image by using an index selection enhancement module based on a sparse attention mechanism, and an enhanced feature map is obtained; and the enhanced feature map is decoded by using a decoder, and a remote sensing image segmentation result is obtained. The application can better focus on important areas in the remote sensing image by introducing multi-scale feature extraction and a sparse attention mechanism, so that the segmentation precision is improved.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and more specifically to a remote sensing image segmentation method based on multi-scale feature fusion and index selection mechanism. Background Technology

[0002] Remote sensing image segmentation technology plays a crucial role in remote sensing image processing and analysis, and is widely used in fields such as environmental monitoring, urban planning, land use surveys, and disaster assessment. Traditional remote sensing image segmentation methods mainly rely on pixel-level feature extraction and classification techniques, including classic methods such as threshold segmentation, edge detection, and region growing. While these methods have solved the remote sensing image segmentation problem to some extent, they face challenges such as low segmentation accuracy and low computational efficiency in complex scenes.

[0003] With the rapid development of deep learning technology, remote sensing image segmentation methods based on convolutional neural networks (CNNs) have gradually become a research hotspot. In particular, with the continuous optimization of network structures, many deep learning models have achieved significant results in remote sensing image segmentation. However, these methods still face some major challenges when processing remote sensing images: First, remote sensing images typically have complex backgrounds and diverse land cover types, and traditional convolutional neural networks have certain limitations in extracting local image features. Second, while existing attention mechanisms can capture key information in local areas, due to limitations in computational resources, it is difficult to efficiently process the large amount of feature information in large-scale remote sensing images. To improve segmentation performance with limited computational resources, how to effectively extract multi-scale information and focus on important regions has become an important research direction in remote sensing image segmentation methods. Summary of the Invention

[0004] In view of this, the present invention provides a remote sensing image segmentation method based on multi-scale feature fusion and index selection mechanism, which can not only efficiently extract multi-scale information in remote sensing images, but also effectively focus on key regions in the image through sparse attention mechanism, thereby significantly improving segmentation accuracy.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A remote sensing image segmentation method based on multi-scale feature fusion and index selection mechanism includes the following steps:

[0007] S1. Use a multi-scale convolutional network to extract feature information at different scales from remote sensing images and obtain a fused feature map of the remote sensing images;

[0008] S2. The index selection enhancement module based on sparse attention mechanism is used to perform feature aggregation on the fused feature map of the remote sensing image to obtain the enhanced feature map;

[0009] S3. Use a decoder to decode the enhanced feature map to obtain the remote sensing image segmentation result.

[0010] Furthermore, before step S1 uses a multi-scale convolutional network to extract features of different scales from the remote sensing image, it also includes preprocessing the acquired remote sensing image, specifically including:

[0011] The acquired remote sensing images are sequentially subjected to geometric correction, radiometric correction, image enhancement and denoising, size normalization and cropping operations.

[0012] Furthermore, step S1 specifically includes:

[0013] S11. Input the remote sensing image into the initial convolutional layer for preliminary feature extraction to obtain the initial feature map.

[0014] S12. Initialize the feature map A spatial global average pooling layer is input to extract global information from the remote sensing image and generate a global context feature map.

[0015] At the same time, the initial feature map Input a windmill-shaped convolutional layer to extract detailed information of local regions in a remotely sensed image, resulting in a local feature map.

[0016] S13. Transfer the global context feature map and local feature maps Perform element-wise multiplication and then combine the result with the local feature map. Perform element-wise addition to generate an initial fused feature map.

[0017] S14. Initial fused feature map By sequentially inputting convolutional layers, ReLU activation function layers, normalization layers, convolutional layers, and Sigmoid activation function layers, activation feature maps a are obtained.

[0018] S15. Compare the activated feature map a with the initial feature map. Element-wise multiplication is performed to obtain the final fused feature map.

[0019] Furthermore, step S2 specifically includes:

[0020] S21. Normalize the fused feature map obtained in step S1, and input the normalized feature map into different convolution branches to extract the query Q, key K and value Va of the multi-scale attention mechanism;

[0021] Simultaneously, the normalized feature map is sequentially input into the convolutional layer and the full-dimensional dynamic convolutional layer to obtain the normalized feature map V;

[0022] S22. Multiply the query Q and the key K to calculate the attention matrix A, where each element in the attention matrix A represents the similarity between the query and the key;

[0023] S23. Use a masking operation to set the values ​​of irrelevant positions in the attention matrix A to zero, generating an initial sparse attention matrix A′;

[0024] S24. From the initial sparse attention matrix A′, select the K largest attention scores in each row and their corresponding index positions;

[0025] S25. The attention scores of the K retained maximum values ​​are redistributed to their corresponding positions in the attention matrix using a hash operation, while the scores of the remaining positions are set to zero, resulting in a quadratic sparse attention matrix A. Sparse ;

[0026] S26. Place A Sparse Element-wise multiplication with the normalized feature map V yields the weighted feature map X:

[0027] S27. Input the weighted feature map X into a convolutional layer for convolution, and perform residual connection between the convolution operation result and the fused feature map obtained in step S1 to obtain the enhanced feature map.

[0028] Furthermore, step S3 specifically includes:

[0029] S31. Enhance the feature map through upsampling operation. The resolution is gradually restored to the original image resolution, resulting in an enhanced feature map at the original resolution.

[0030] S32. The enhanced feature map of the original resolution output in step S31 is further processed using standard convolutional layers to obtain the processed feature map.

[0031] S33. Use convolution operations to convert the feature map processed by the convolution operation in step S32 into an output map of the same size as the input remote sensing image, and use a 1x1 convolution kernel to map the converted feature map to the required number of categories. Perform softmax or sigmoid activation function operations on the mapped output to obtain the category prediction probability of each pixel.

[0032] Furthermore, the loss function used in obtaining the category prediction probability specifically includes the following loss functions:

[0033] Loss total =λ1·LossCE +λ2·Loss Dice ;

[0034] Among them, Loss CE Represents the cross-entropy loss function; Loss Dice Let λ1 and λ2 represent the Dice loss function, where λ1 and λ2 are weighting coefficients.

[0035] In another aspect, this invention discloses a remote sensing image segmentation system based on multi-scale feature fusion and index selection mechanism, comprising:

[0036] Feature extraction module: Utilizes a multi-scale convolutional network to extract feature information at different scales from the input remote sensing image and obtains a fused feature map of the remote sensing image;

[0037] The index selection enhancement module based on sparse attention mechanism is used to perform feature aggregation on the fused feature map of remote sensing images to obtain an enhanced feature map.

[0038] Feature decoding module: The decoder is used to decode the enhanced feature map to obtain the remote sensing image segmentation result.

[0039] Preferably, the above-mentioned remote sensing image segmentation system further includes a remote sensing image preprocessing module, which is used to perform geometric correction, radiometric correction, image enhancement and denoising, size normalization and cropping operations on the remote sensing image before feature extraction.

[0040] Preferably, the feature extraction module in the above-mentioned remote sensing image segmentation system specifically performs the following steps:

[0041] S11. Input the remote sensing image into the initial convolutional layer for preliminary feature extraction to obtain the initial feature map.

[0042] S12. Initialize the feature map A spatial global average pooling layer is input to extract global information from the remote sensing image and generate a global context feature map.

[0043] At the same time, the initial feature map Input a windmill-shaped convolutional layer to extract detailed information of local regions in a remotely sensed image, resulting in a local feature map.

[0044] S13. Transfer the global context feature map and local feature maps Perform element-wise multiplication and then combine the result with the local feature map. Perform element-wise addition to generate an initial fused feature map.

[0045] S14. Initial fused feature map By sequentially inputting convolutional layers, ReLU activation function layers, normalization layers, convolutional layers, and Sigmoid activation function layers, activation feature maps a are obtained.

[0046] S15. Compare the activated feature map a with the initial feature map. Element-wise multiplication is performed to obtain the final fused feature map.

[0047] Preferably, in the above-mentioned remote sensing image segmentation system, the index selection enhancement module based on the sparse attention mechanism obtains the enhanced feature map by specifically performing the following steps:

[0048] S21. Normalize the fused feature map obtained by the feature extraction module, and input the normalized feature map into different convolution branches to extract the query Q, key K and value Va of the multi-scale attention mechanism;

[0049] Simultaneously, the normalized feature map is sequentially input into the convolutional layer and the full-dimensional dynamic convolutional layer to obtain the normalized feature map V;

[0050] S22. Multiply the query Q and the key K to calculate the attention matrix A, where each element in the attention matrix A represents the similarity between the query and the key;

[0051] S23. Use a masking operation to set the values ​​of irrelevant positions in the attention matrix A to zero, generating an initial sparse attention matrix A′;

[0052] S24. From the initial sparse attention matrix A′, select the K largest attention scores in each row and their corresponding index positions;

[0053] S25. Using a hash operation, the attention scores of the K retained maximum values ​​are redistributed to their corresponding positions in the initial sparse attention matrix A′, while the scores of the remaining positions are set to zero, resulting in a quadratic sparse attention matrix A′. Sparse ;

[0054] S26. The quadratic sparse attention matrix A Sparse Element-wise multiplication with the normalized feature map V yields the weighted feature map X:

[0055] S27. Input the weighted feature map X into a convolutional layer for convolution, and perform residual connection between the convolution result and the fused feature map obtained from the feature extraction module to obtain the enhanced feature map.

[0056] Preferably, in the above-mentioned remote sensing image segmentation system, the feature decoding module uses a decoder to decode the enhanced feature map, specifically including the following steps:

[0057] S31. Enhance the feature map through upsampling operation. The resolution is gradually restored to the original remote sensing image resolution, resulting in an enhanced feature map of the original resolution.

[0058] S32. The enhanced feature map of the original resolution output in step S31 is further processed using standard convolutional layers to obtain the processed feature map.

[0059] S33. Use convolution operations to convert the feature map processed by the convolution operation in step S32 into an output map of the same size as the input remote sensing image, and use a 1x1 convolution kernel to map the converted feature map to the required number of categories. Perform softmax or sigmoid activation function operations on the mapped output to obtain the category prediction probability of each pixel.

[0060] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a remote sensing image segmentation method and related system based on multi-scale feature fusion and index selection mechanism, which has the following beneficial effects:

[0061] 1) Improve the accuracy of remote sensing image segmentation: By introducing multi-scale feature extraction and sparse attention mechanism, it is possible to better focus on important regions in remote sensing images, thereby improving segmentation accuracy.

[0062] 2) Reduced computational resource requirements: By utilizing the sparse attention mechanism, interference from unimportant information is reduced, significantly improving computational efficiency and reducing the demand for computational resources. It is suitable for image processing tasks in various computing environments. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0064] Figure 1 This is a schematic diagram of the overall process of the remote sensing image segmentation method provided by the present invention.

[0065] Figure 2 This is a schematic diagram of the feature extraction module provided by the present invention.

[0066] Figure 3 A schematic diagram of the index selection enhancement module provided by the present invention. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] This invention discloses a remote sensing image segmentation method and system based on multi-scale feature fusion and index selection mechanisms. The aim is to effectively improve segmentation accuracy by combining multi-scale feature extraction with a sparse attention mechanism to fully utilize information at different levels and scales in remote sensing images. For multi-scale feature extraction, a multi-branch convolutional structure is used to capture local details and global information, while techniques such as dilated convolution and depthwise separable convolution are introduced to enhance the perception of features at different scales. The sparse attention mechanism, on the other hand, calculates important regions in the multi-scale feature map, adaptively selects key features for weighted aggregation, suppresses irrelevant information, and improves computational efficiency. This organic combination of multi-scale and attention mechanisms not only helps the model capture richer detailed features in complex remote sensing images but also effectively improves the segmentation accuracy of the model in images with different resolutions and target sizes.

[0069] First, this invention discloses a remote sensing image segmentation method based on multi-scale feature fusion and index selection mechanism, such as... Figure 1 As shown, it includes:

[0070] S1. Use a multi-scale convolutional network to extract feature information at different scales from remote sensing images and obtain a fused feature map of the remote sensing images;

[0071] S2. The index selection enhancement module based on sparse attention mechanism is used to perform feature aggregation on the fused feature map of the remote sensing image to obtain the enhanced feature map;

[0072] S3. Use a decoder to decode the enhanced feature map to obtain pixel-level segmentation results.

[0073] Each step in the embodiments is further explained below.

[0074] Step S1 may specifically include preprocessing and feature extraction of the original remote sensing image, and the specific implementation steps are as follows.

[0075] Step S10: After a series of preprocessing steps, the acquired remote sensing images are used to obtain the input image I to be fed into the neural network. First, the image undergoes geometric correction to eliminate geometric distortions caused by sensor or platform motion. Next, radiometric correction techniques are applied to eliminate sensor noise and atmospheric interference, ensuring the accuracy of the image data. Then, image quality is improved through image enhancement and denoising processes so that the neural network can better recognize and process features. Finally, the image undergoes size normalization and cropping to ensure that the size of the input image meets the requirements of the neural network while preserving key information. After these preprocessing steps, the resulting image I is high-quality data suitable for input into the neural network for analysis and inference.

[0076] Step S11: Initial feature extraction. For example... Figure 2 The input image I obtained after preprocessing the remote sensing image shown is processed through an initial convolutional layer to extract preliminary features, resulting in an initial feature map. This step is mainly used to extract low-level features from the input image, laying the foundation for subsequent high-level feature operations.

[0077] Step S12: Process the initial feature maps respectively Perform spatial global average pooling and windmill convolution operations.

[0078] Feature map After passing through a spatial global average pooling layer, global information of the image is extracted to generate a global context feature map.

[0079]

[0080] Where Conv represents a convolutional layer and GAP represents a spatial global average pooling layer.

[0081] Meanwhile, feature map After passing through a windmill-shaped convolutional layer (PSConv), detailed information of local regions is extracted to obtain a feature map.

[0082]

[0083] Here, PSConv represents the windmill-shaped convolutional layer. This windmill-shaped convolutional layer is designed to capture multi-scale and local pattern information through specific convolution patterns, and can effectively extract complex detailed features from remote sensing images.

[0084] Step S13: Feature fusion to generate the final fused feature map. Will and Element-wise multiplication is performed to obtain the fused feature map. Element-wise multiplication can capture... and The relationship characteristics between them. Then, the result of the multiplication is combined with... The features are added together to generate the final fused feature map.

[0085]

[0086] Here, ⊙ represents element-wise multiplication. This represents the matrix addition operation.

[0087] Step S14: Nonlinear transformation and feature enhancement for fused feature maps Convolutional operations are performed, and the features are further enhanced using an activation function (ReLU) and a normalization layer. Subsequently, a second convolutional operation is performed, and finally, the features are processed using a sigmoid activation function to obtain the activated feature map a:

[0088]

[0089] Here, Sigmoid represents the Sigmoid activation function, ReLU represents the ReLU activation function, and BN is the normalization layer. This operation, through the combination of multiple convolutional layers, nonlinear activation, and normalization layers, further enhances the expressive power of features, enabling the network to better adapt to complex patterns in remote sensing images.

[0090] Step S15: Final output feature map. Compare the activation feature map a with the initial feature map. Element-wise multiplication is performed to obtain the final fused feature map. As the output of the network, this process is equivalent to enhancing the original feature map with enhanced feature information, resulting in a more accurate representation.

[0091] The feature extraction process can be achieved through Figure 2 The feature extraction module shown is implemented as described. This module effectively extracts high-level features from remote sensing images through a series of convolutional layers, pooling layers, pinwheel convolution, and feature fusion operations. In this process, global and local information are extracted through spatial global average pooling and pinwheel convolution, and the correlation between the two is strengthened through element-wise multiplication and additive fusion, ultimately generating highly expressive feature maps. These feature maps will provide richer information for subsequent classification or segmentation tasks.

[0092] Step S2: The index selection enhancement module based on sparse attention mechanism is used to aggregate features of the fused feature map of the remote sensing image to obtain the enhanced feature map. The overall framework diagram of the index selection enhancement module based on sparse attention mechanism is shown below. Figure 3 As shown, this module performs the following steps.

[0093] S21. Normalize the fused feature map obtained by the feature extraction module, and input the normalized feature map into different convolution branches to extract the query Q, key K and value Va of the multi-scale attention mechanism;

[0094] Simultaneously, the normalized feature map is sequentially input into the convolutional layer and the full-dimensional dynamic convolutional layer to obtain the normalized feature map V;

[0095] In the above steps, the normalization operation helps to standardize the input features, reduce scale differences between different features, and thus improve the training stability of the network. Then, the normalized feature maps... The inputs are fed into different convolutional branches to extract the query (Q), key (K), and value (Va). Specifically, the feature map is passed through different convolutional layers and depthwise separable convolutional layers to obtain Q, through convolutional layers and dilated convolutional layers to obtain K, and through convolutional layers and full-dimensional dynamic convolutional layers to obtain the feature map V. Here, DilatedConv represents dilated convolution, and DynamicConv represents full-dimensional dynamic convolution operation.

[0096] Then, attention and masking operations are performed to generate an initial sparse attention matrix A′. Attention scores are calculated by multiplying the query Q and key K to obtain the attention matrix A, where each element represents the similarity between the query and the key. Next, a masking operation is applied to mask out unimportant attention scores. The masking operation sets unimportant attention scores to zero, retaining only the most critical scores, thus preventing irrelevant information from interfering with subsequent feature aggregation. This masking operation generates a sparse attention matrix A′ by setting the values ​​at irrelevant positions to zero. The specific execution steps are as follows:

[0097] S22. Multiply the query Q and the key K to calculate the attention matrix A, where each element in the attention matrix A represents the similarity between the query and the key;

[0098] S23. Use a masking operation to set the values ​​of irrelevant positions in the attention matrix A to zero, generating the initial sparse attention matrix A′.

[0099] Next, index selection and hashing are performed. The top K maximum values ​​and their indices are selected. From the attention matrix A′, the K largest attention scores in each row and their corresponding index positions are selected. These strongest attention scores better represent the relationships between image features. By selecting the top K maximum values, the model can accurately retain key features relevant to the current task. Based on the selected top K maximum values ​​and their indices, these important attention scores are retained, while other unimportant parts are ignored. This step is the core of implementing the sparse attention mechanism; through index selection, the model can accurately locate the most valuable information. The hashing operation redistributes the retained K maximum attention scores to their corresponding positions in the attention matrix, setting the scores of the remaining positions to zero, resulting in A′. Sparse This operation makes the attention matrix sparser, which helps improve computational efficiency and the model's ability to focus attention. This process sparsifies the attention matrix through hashing, making the computation more efficient and focusing on important feature information. The specific execution steps are as follows:

[0100] S24. From the initial sparse attention matrix A′, select the K largest attention scores in each row and their corresponding index positions;

[0101] S25. Using a hash operation, the attention scores of the K retained maximum values ​​are redistributed to their corresponding positions in the initial sparse attention matrix A′, while the scores of the remaining positions are set to zero, resulting in a quadratic sparse attention matrix A′. Sparse ;

[0102] Then, weighted feature aggregation and residual join operations are performed. This operation includes steps S26 and S27, the specific steps of which are as follows:

[0103] S26. The sparsified attention matrix A Sparse The weighted feature map is obtained by element-wise multiplication with the feature map V.

[0104]

[0105] Here, ⊙ represents element-wise multiplication.

[0106] S27. After obtaining the weighted feature map, in order to avoid information loss, the weighted feature map... After passing through the convolutional layer, the feature map is fused with the original one. Residual connections are performed to obtain the enhanced feature map.

[0107] Residual connections ensure that input features and enhanced features can be effectively fused in the output, thereby helping the network to better extract and learn features.

[0108] Through the steps described above, the network successfully implemented a sparse attention mechanism using index selection, effectively focusing on important image regions in remote sensing image segmentation tasks. Masking, index selection, and hashing operations work together to ensure the efficiency and accuracy of feature aggregation. Furthermore, weighted feature aggregation and residual connections enhance the network's ability to represent key features, thereby improving the performance of remote sensing image segmentation.

[0109] Step S3: The enhanced feature map obtained in step S2 is processed by the decoder. Decode it to convert it into a pixel-level segmentation result.

[0110] Step S31: Upsampling of the feature map. The feature maps are gradually restored to the resolution of the original image. A series of upsampling operations are used to progressively restore the feature maps from a lower resolution to a higher resolution. Upsampling operations are performed using deconvolution or bilinear interpolation. Deconvolution operations are often performed in conjunction with learning the convolution kernel to extract more meaningful features while restoring spatial resolution.

[0111] Step S32: Convolutional operations for feature processing. The decoded features are further processed in the decoder section to accurately recover the image's structure and details. The feature map output from step S31 is further processed using standard convolutional layers. Several convolutional layers are used to extract higher-level feature information, progressively reducing the number of channels. A non-linear activation function, such as ReLU or LeakyReLU, can be added after each convolutional layer.

[0112] Step S33: Final Pixel-Level Classification. The feature map is transformed into a class prediction for each pixel. In the last layer of the decoder, a convolution operation is used to transform the feature map into an output map of the same size as the input image. A 1x1 convolution kernel is used to map the final feature map to the required number of classes (i.e., the number of output channels equals the number of classes). A softmax or sigmoid activation function is applied to the output to obtain the class prediction probability for each pixel.

[0113] In the category prediction process, the loss function is obtained through the following steps: First, cross-entropy loss is used to measure the difference between the predicted category and the true label for each pixel. Assume the network output is the category prediction probability distribution for each pixel, denoted as... This invention assigns the true label y(i) to the neural network, where i represents the pixel position. Cross-entropy loss is applied to each pixel:

[0114]

[0115] Next, Dice loss is used to optimize the overlapping region between the predicted and ground truth segmentation results, which is particularly suitable for handling class imbalance problems. The Dice coefficient between the predicted segmentation map and the ground truth label map is calculated, which measures the similarity between the two sets:

[0116]

[0117] Then calculate the Dice loss:

[0118]

[0119] Finally, the total loss function combines multiple loss functions to obtain a final loss value for optimization. The final loss function is a weighted sum of multiple losses:

[0120] Loss total =λ1·Loss CE +λ2·Loss Dice

[0121] λ1 and λ2 are weighting coefficients, which can be adjusted as needed.

[0122] The remote sensing image segmentation method of this invention first extracts information at different scales in the image through a multi-scale convolutional network during the feature extraction stage. This method can capture both global features and finely process local details. Specifically, it extracts global information of the image through a spatial global average pooling (GAP) layer and extracts detailed information of local regions through a windmill-shaped convolutional layer (PSConv). The fusion of these two features provides rich feature representation for subsequent image segmentation tasks.

[0123] Based on the above method, this invention also discloses a remote sensing image segmentation system based on multi-scale feature fusion and index selection mechanism, comprising,

[0124] Feature extraction module: Utilizes a multi-scale convolutional network to extract feature information at different scales from the input remote sensing image and obtains a fused feature map of the remote sensing image;

[0125] The index selection enhancement module based on sparse attention mechanism is used to perform feature aggregation on the fused feature map of remote sensing images to obtain an enhanced feature map.

[0126] Feature decoding module: The decoder is used to decode the enhanced feature map to obtain pixel-level segmentation results.

[0127] The specific execution process of each module can be found in the description of the method in this application, and will not be repeated here.

[0128] To address the efficiency issues of traditional attention mechanisms in large-scale remote sensing image processing, this invention employs a sparse attention mechanism. After normalizing the input features, query, key, and value feature maps are generated using convolutional branches, and an attention matrix is ​​calculated. Unimportant attention scores are masked to reduce computational cost. Next, an index selection operation retains the K strongest attention scores, further focusing on important regions in the image. Finally, weighted feature aggregation and residual connections are used to enhance the network's ability to represent key features, ensuring more accurate segmentation results.

[0129] Compared to traditional methods, this invention not only improves the accuracy of remote sensing image segmentation, but also enables more efficient large-scale image segmentation tasks under limited computational resources by combining sparse attention mechanisms and multi-scale feature extraction. Especially in complex environments, it effectively distinguishes ground features, avoiding segmentation errors caused by neglecting local features in traditional methods.

[0130] The innovation of this invention lies in its combination of multi-scale feature extraction and sparse attention mechanism, which not only overcomes the limitations of traditional methods, but also greatly improves the performance of remote sensing image segmentation through efficient computation, and has broad application prospects.

[0131] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0132] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image segmentation method based on multi-scale feature fusion and index selection mechanism, characterized in that, Includes the following steps: S1. Utilize a multi-scale convolutional network to extract feature information at different scales from remote sensing images and obtain a fused feature map of the remote sensing images; specifically including, S11. Input the remote sensing image into the initial convolutional layer for preliminary feature extraction to obtain the initial feature map; S12. Input the initial feature map into the spatial global average pooling layer to extract global information from the remote sensing image and generate a global context feature map; Simultaneously, the initial feature map is input into the windmill-shaped convolutional layer to extract detailed information of local areas of the remote sensing image, thus obtaining a local feature map; S13. Multiply the global context feature map and the local feature map element by element, and add the multiplication result to the local feature map element by element to generate the initial fused feature map; S14. Input the initial fused feature map into the convolutional layer, ReLU activation function layer, normalization layer, convolutional layer, and Sigmoid activation function layer in sequence to obtain the activation feature map; S15. Multiply the activated feature map obtained in step S14 with the initial feature map element by element to obtain the final fused feature map; S2. An index selection enhancement module based on a sparse attention mechanism is used to aggregate features from the fused feature map of the remote sensing image, resulting in an enhanced feature map; specifically, this includes... S21. Normalize the fused feature map obtained in step S1, and input the normalized feature map into different convolution branches to extract the query Q, key K and value Va of the multi-scale attention mechanism; Simultaneously, the normalized feature map is sequentially input into the convolutional layer and the full-dimensional dynamic convolutional layer to obtain the normalized feature map V; S22. Multiply the query Q and the key K to calculate the attention matrix A, where each element in the attention matrix A represents the similarity between the query and the key; S23. Use a masking operation to set the values ​​of irrelevant positions in the attention matrix A to zero, generating an initial sparse attention matrix A′; S24. From the initial sparse attention matrix A′, select the K largest attention scores in each row and their corresponding index positions; S25. Through a hash operation, the attention scores of the K retained maximum values ​​are redistributed to their corresponding positions in the initial sparse attention matrix A′, and the scores of the remaining positions are set to zero, resulting in a quadratic sparse attention matrix. ; S26. The quadratic sparse attention matrix... Multiply the weighted feature map X element-wise with the normalized feature map V to obtain the weighted feature map X; S27. Input the weighted feature map X into a convolutional layer for convolution, and perform residual connection between the convolution operation result and the fused feature map obtained in step S1 to obtain the enhanced feature map. ; S3. Use a decoder to decode the enhanced feature map to obtain the remote sensing image segmentation result.

2. The remote sensing image segmentation method based on multi-scale feature fusion and index selection mechanism according to claim 1, characterized in that, Before step S1 uses a multi-scale convolutional network to extract features at different scales from the remote sensing image, preprocessing of the acquired remote sensing image is also included, specifically: The acquired remote sensing images are sequentially subjected to geometric correction, radiometric correction, image enhancement and denoising, size normalization and cropping operations.

3. The remote sensing image segmentation method based on multi-scale feature fusion and index selection mechanism according to claim 1, characterized in that, Step S3 specifically includes: S31. Enhance the feature map through upsampling operation. The resolution is gradually restored to the original remote sensing image resolution, resulting in an enhanced feature map of the original resolution. S32. The enhanced feature map of the original resolution output in step S31 is further processed using standard convolutional layers to obtain the processed feature map. S33. Use convolution operations to convert the feature map processed by the convolution operation in step S32 into an output map of the same size as the input remote sensing image, and use a 1x1 convolution kernel to map the converted feature map to the required number of categories. Apply softmax or sigmoid activation function to the mapped output to obtain the category prediction probability of each pixel.

4. A remote sensing image segmentation system based on multi-scale feature fusion and index selection mechanism, characterized in that, include, Feature extraction module: This module uses a multi-scale convolutional network to extract feature information at different scales from the input remote sensing image and obtains a fused feature map of the remote sensing image. Specifically, the feature extraction module performs the following steps: S11. Input the remote sensing image into the initial convolutional layer for preliminary feature extraction to obtain the initial feature map. ; S12. Initialize the feature map A spatial global average pooling layer is input to extract global information from the remote sensing image and generate a global context feature map. ; At the same time, the initial feature map Input a windmill-shaped convolutional layer to extract detailed information of local regions in a remotely sensed image, resulting in a local feature map. ; S13. Transfer the global context feature map and local feature maps Perform element-wise multiplication and then combine the result with the local feature map. Perform element-wise addition to generate an initial fused feature map. ; S14. Initial fused feature map By sequentially inputting convolutional layers, ReLU activation function layers, normalization layers, convolutional layers, and Sigmoid activation function layers, activation feature maps a are obtained. S15. Compare the activated feature map a with the initial feature map. Element-wise multiplication is performed to obtain the final fused feature map. ; The index selection enhancement module based on sparse attention mechanism is used to aggregate features from the fused feature map of the remote sensing image to obtain an enhanced feature map. The specific steps performed by the index selection enhancement module based on sparse attention mechanism are as follows: S21. Normalize the fused feature map obtained by the feature extraction module, and input the normalized feature map into different convolution branches to extract the query Q, key K and value Va of the multi-scale attention mechanism; Simultaneously, the normalized feature map is sequentially input into the convolutional layer and the full-dimensional dynamic convolutional layer to obtain the normalized feature map V; S22. Multiply the query Q and the key K to calculate the attention matrix A, where each element in the attention matrix A represents the similarity between the query and the key; S23. Use a masking operation to set the values ​​of irrelevant positions in the attention matrix A to zero, generating an initial sparse attention matrix A′; S24. From the initial sparse attention matrix A′, select the K largest attention scores in each row and their corresponding index positions; S25. Through a hash operation, the attention scores of the K retained maximum values ​​are redistributed to their corresponding positions in the initial sparse attention matrix A′, and the scores of the remaining positions are set to zero, resulting in a quadratic sparse attention matrix. ; S26. The quadratic sparse attention matrix... Multiply the weighted feature map X element-wise with the normalized feature map V to obtain the weighted feature map X; S27. Input the weighted feature map X into a convolutional layer for convolution, and perform residual connection between the convolution result and the fused feature map obtained from the feature extraction module to obtain the enhanced feature map. ; Feature decoding module: The decoder is used to decode the enhanced feature map to obtain the remote sensing image segmentation result.

5. The remote sensing image segmentation system based on multi-scale feature fusion and index selection mechanism according to claim 4, characterized in that, It also includes a remote sensing image preprocessing module, which performs geometric correction, radiometric correction, image enhancement and denoising, size normalization and cropping operations on the remote sensing image before feature extraction.

6. The remote sensing image segmentation system based on multi-scale feature fusion and index selection mechanism according to claim 4, characterized in that, The feature decoding module uses a decoder to decode the enhanced feature map, specifically including the following steps: S31. Enhance the feature map through upsampling operation. The resolution is gradually restored to the original remote sensing image resolution, resulting in an enhanced feature map of the original resolution. S32. The enhanced feature map of the original resolution output in step S31 is further processed using standard convolutional layers to obtain the processed feature map. S33. Use convolution operations to convert the feature map processed by the convolution operation in step S32 into an output map of the same size as the input remote sensing image, and use a 1x1 convolution kernel to map the converted feature map to the required number of categories. Apply softmax or sigmoid activation function to the mapped output to obtain the category prediction probability of each pixel.

Images