A remote sensing image segmentation method, device, medium and equipment

Abstract

The application discloses a remote sensing image segmentation method and device, medium and equipment, and relates to the technical field of image processing. The method comprises the following steps: replacing the kernel attention module in the original MANet network with a one-dimensional convolution layer, a two-dimensional convolution layer and an attention mechanism layer connected in sequence to construct an improved MANet network. The boundary features in the convolution feature map are perceived through one-dimensional convolution operation; then, the two-dimensional convolution operation is performed on the one-dimensional feature map by using the two-dimensional convolution layer to perceive the spatial dependence relationship located in the boundary neighborhood, and the spatial perception ability of the boundary neighborhood details is enhanced; the attention mechanism is used for the output of the two-dimensional convolution layer to dynamically adjust the channel weight of the two-dimensional feature map which responds significantly to the boundary, so that the feature channel related to the segmentation boundary in the remote sensing image can be focused on, the detailed capture ability of the boundary local features is improved, and more accurate remote sensing image segmentation results are obtained.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a remote sensing image segmentation method, device, medium and equipment. Background Art

[0002] Remote sensing images contain rich information about ground objects. Segmenting them can yield surface cover classifications, providing a basis for monitoring surface changes. Semantic segmentation of remote sensing images has become a research hotspot in computer vision and remote sensing technology in recent years. It aims to segment pixels or regions in remote sensing images into categories with specific semantic labels.

[0003] MANet (Multi-Attention Network) is a network model for semantic segmentation of remote sensing images proposed by Rui Li et al. in their paper "Multi-Attention-Network for Semantic Segmentation of Fine Resolution Remote Sensing Images." The MANet network includes four attention modules, each of which has two parallel branches: the Kernel Attention Mechanism (KAM) and the Channel Attention Mechanism (CAM). The Kernel Attention Mechanism (CAM) uses a kernel-based generalized dot-product attention mechanism to address potential long-range dependencies between pixels in remote sensing images, enabling remote sensing image segmentation.

[0004] However, since the kernel attention module KAM focuses more on the long-distance global relationship between pixels in remote sensing images, it ignores the local features in remote sensing images. Local features are crucial for the segmentation of boundaries of different objects. The lack of local features may cause the network to inaccurately segment the segmentation boundaries. Summary of the Invention

[0005] Based on this, it is necessary to provide a remote sensing image segmentation method, device, medium and equipment to address the above technical problems.

[0006] The present invention adopts the following technical solutions:

[0007] The present invention provides a remote sensing image segmentation method, comprising:

[0008] Build and improve the MANet network, including:

[0009] This approach replaces the kernel attention module in the attention module connecting each convolutional layer and each deconvolutional layer in the original MANet network with a local feature extraction module consisting of a sequentially connected one-dimensional convolutional layer, a two-dimensional convolutional layer, and an attention mechanism layer. The convolution kernel size of the one-dimensional convolutional layer is 1×1, and the convolution kernel size of the two-dimensional convolutional layer is 3×3. The attention mechanism layer uses a squeeze-excitation attention mechanism. Furthermore, the attention module is directly connected to the output of each convolutional layer to transfer local boundary features within each convolutional layer.

[0010] The improved attention module includes a local feature extraction module and a global feature extraction module. The global feature extraction module is used to capture global features in convolution feature maps of different scales, including a global one-dimensional convolution layer and a global self-attention mechanism layer connected to the output end of the global one-dimensional convolution layer. The global one-dimensional convolution layer is used to perform one-dimensional convolution operations on convolution feature maps of different scales to obtain a global one-dimensional feature map; the global self-attention mechanism layer is used to capture long-distance dependencies in the global one-dimensional feature map using the dot product attention mechanism to obtain a global feature map; the outputs of the local feature extraction module and the global feature extraction module are spliced, and the spliced ​​result is used as the output of the attention module.

[0011] The improved MANet network is trained to obtain a segmentation model for segmenting remote sensing images; the remote sensing image to be segmented is input into the segmentation model, and each convolution layer performs convolution operations on the remote sensing image to be segmented to obtain convolution feature maps of different scales; the local feature extraction module performs one-dimensional convolution operations on the convolution feature maps of different scales through a one-dimensional convolution layer to perceive the local boundary features in the convolution feature maps of different scales and obtain one-dimensional feature maps of different scales; the two-dimensional convolution layer performs two-dimensional convolution on the one-dimensional feature maps of different scales to perceive the local spatial dependency of the boundary neighborhood and obtain two-dimensional feature maps of different scales; the attention mechanism layer dynamically adjusts the channel weights with significant boundary response in the two-dimensional feature maps of different scales to obtain local enhanced feature maps of different scales; each deconvolution layer performs deconvolution operations on the local enhanced feature maps of different scales to restore the size of the local enhanced feature maps to the size of the remote sensing image to be segmented, and obtains the segmented remote sensing image.

[0012] The present invention provides a remote sensing image segmentation device, comprising:

[0013] A model construction module is used to replace the core attention module in the attention module used to connect each convolution layer and each deconvolution layer of the original MANet network with a local feature extraction module to construct an improved MANet network; the local feature extraction module is composed of a one-dimensional convolution layer, a two-dimensional convolution layer, and an attention mechanism layer connected in sequence;

[0014] Model training module, used to train the improved MANet network to obtain a segmentation model for segmenting remote sensing images;

[0015] The image segmentation module is used to input the remote sensing image to be segmented into the segmentation model. Each convolution layer performs convolution operations on the remote sensing image to be segmented to obtain convolution feature maps of different scales; the local feature extraction module performs one-dimensional convolution operations on the convolution feature maps of different scales through the one-dimensional convolution layer to perceive the local boundary features in the convolution feature maps of different scales and obtain one-dimensional feature maps of different scales; the two-dimensional convolution layer performs two-dimensional convolution on the one-dimensional feature maps of different scales to perceive the local spatial dependency of the boundary neighborhood and obtain two-dimensional feature maps of different scales; the attention mechanism layer dynamically adjusts the channel weights with significant boundary response in the two-dimensional feature maps of different scales to obtain local enhanced feature maps of different scales; each deconvolution layer performs deconvolution operations on the local enhanced feature maps of different scales to restore the size of the local enhanced feature maps to the size of the remote sensing image to be segmented, and obtains the segmented remote sensing image.

[0016] The present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the remote sensing image segmentation method is implemented.

[0017] The present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the remote sensing image segmentation method is implemented.

[0018] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects:

[0019] The present invention first performs a one-dimensional convolution operation on a convolution feature map to perceive boundary features in the convolution feature map; then uses a two-dimensional convolution layer to perform a two-dimensional convolution operation on the one-dimensional feature map. Since the two-dimensional convolution operation is performed within the neighborhood of each pixel in the one-dimensional feature map, the spatial dependency relationship in the boundary neighborhood can be perceived, thereby enhancing the spatial perception ability of the boundary neighborhood details; the output of the two-dimensional convolution layer uses an attention mechanism to dynamically adjust the channel weights that have significant boundary responses in the two-dimensional feature map, which can focus attention on feature channels related to the segmentation boundary in the remote sensing image, thereby improving the ability to capture detailed boundary local features; through the above steps, more detailed boundary local features can be obtained, thereby obtaining more accurate remote sensing image segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0021] Figure 1 A schematic flow chart of a remote sensing image segmentation method provided by the present invention;

[0022] Figure 2 A schematic diagram of the structure of the original MANet network provided by the present invention;

[0023] Figure 3 The present invention provides Figure 2 Schematic diagram of the structure of attention module 1 to attention module 5;

[0024] Figure 4 A schematic diagram of the structure of the improved MANet network provided by the present invention;

[0025] Figure 5 The present invention provides Figure 4 Schematic diagram of the structures of local and global feature extraction modules-1 to local and global feature extraction modules-4. DETAILED DESCRIPTION

[0026] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.

[0027] MANet aims to address the limitations of standard semantic segmentation models in terms of feature utilization, long-distance dependency modeling, and computational complexity. It optimizes the dot product attention mechanism in the spatial domain and channel domain respectively, and proposes new attention mechanism models: KAM and CAM. The two are connected in parallel to form attention modules 1 to 4. Combined with the ResNet-50 backbone network, a semantic segmentation model that integrates multi-scale features is designed. The structure of the original MANet network is as follows: Figure 2 As shown in the figure, the ResNet-50 backbone network consists of a 1×1 convolutional layer, residual block 2, residual block 3, residual block 4 and residual block 5 connected in sequence. The output of the above convolutional layer or residual block is connected to the input of downsampling 1 to downsampling 5 respectively to obtain feature Figure 1 Features Figure 5 ;feature Figure 5 Input into the deconvolution layer 5, and the obtained results are compared with the features Figure 4 After fusion, it is input to attention module 4, and the output of attention module 4 is combined with the feature Figure 4 Fusion, and so on, until the output of attention module 1 is bilinearly interpolated to obtain the segmentation result. Figure 2 The structures of attention modules 1 to 5 in the Figure 3 As shown in the figure, it includes two parallel branches. One branch includes a 1×1 convolutional layer, batch normalization + activation function, a channel attention module, a 3×3 convolutional layer, and batch normalization + activation function, which are connected in sequence. This branch outputs a channel feature map. The other branch includes a 1×1 convolutional layer, batch normalization + activation function, a kernel attention module, a 3×3 convolutional layer, and batch normalization + activation function, which are connected in sequence. This branch outputs a kernel feature map. The channel feature map and the kernel feature map are concatenated to obtain the output feature map. However, the attention mechanism focuses primarily on the importance of local regions and channels, which may lead to a loss of global context. The lack of global information affects the network's overall understanding of remote sensing imagery, which may lead to inaccurate semantic segmentation.

[0028] Based on this, this embodiment proposes an improved MANet network, named GLFormer (Global-LocalFormer) model to solve the above problems. The corresponding attention modules in the GLFormer model are divided into GAM (Global Attention Module) and LAM (Local Attention Module). The LAM module consists of two concatenated convolutional layers with kernel sizes of 1 and 3, a batch normalization operation and a ReLU activation function, as well as a compression-excitation attention mechanism module behind this branch. The purpose of both is to extract contextual information from the global and local scales respectively.

[0029] Figure 1 The process of the remote sensing image segmentation method provided by this embodiment is shown in FIG. Figure 1 The remote sensing image segmentation method is described in detail, which specifically includes the following steps:

[0030] S1: The core attention module in the attention module used to connect each convolutional layer and each deconvolutional layer of the original MANet network is replaced with a local feature extraction module to construct an improved MANet network; the local feature extraction module is composed of a one-dimensional convolutional layer, a two-dimensional convolutional layer and an attention mechanism layer connected in sequence.

[0031] This embodiment replaces the two parallel branches of the core attention module KAM and the channel attention module CAM in the four attention modules of the MANet network with two parallel branches of the local feature extraction module LAM and the global feature extraction module GAM to construct an improved MANet network. The two parallel local feature extraction modules LAM and the global feature extraction module GAM are collectively referred to as GLAM modules. The improved MANet network structure is as follows: Figure 4As shown, it includes a 1×1 convolution layer, residual block-1, residual block-2, residual block-3 and residual block-4 connected in sequence. The output ends of the above convolution layers or residual blocks are respectively connected to the input ends of the global and local feature extraction modules-1 to the global and local feature extraction modules-4 to obtain feature maps-1 to feature maps-4 respectively; feature map-4 is input to decoding block-4, and the result is fused with feature map-3 and input to decoding block-3, and so on, until the output of decoding block-1 is subjected to one 3×3 deconvolution operation and one 3×3 convolution operation to obtain the segmentation result. Figure 4 The structure of the global and local feature extraction modules in Figure 5 As shown in the figure, it includes two parallel branches, one of which includes a 1×1 convolution layer and a global self-attention mechanism layer connected in sequence, which outputs global features; the other branch includes a 1×1 one-dimensional convolution layer, a 3×3 two-dimensional convolution layer, a batch normalization + activation function, and an attention mechanism layer connected in sequence, which outputs local features. The global features and local features are spliced ​​together to obtain fused features. Specifically:

[0032] The local feature extraction module consists of two concatenated convolutional layers with kernel sizes of 1 and 3, respectively. The convolution kernel size of the one-dimensional convolution layer is 1×1, and the convolution kernel size of the two-dimensional convolution layer is 3×3. This layer is followed by a batch normalization operation and a ReLU activation function, as well as a compression-excitation attention mechanism layer. The compression-excitation attention mechanism layer works as follows: It performs global pooling on the input feature map to compress the feature map of each channel; the compressed feature map is processed through two fully connected layers: the first fully connected layer is used to reduce the number of channels, and the second fully connected layer is used to restore the number of channels to the original number of channels to learn the nonlinear relationship between channels; the second fully connected layer uses the sigmoid activation function to generate weights between 0 and 1. These weights are used to represent the importance of each channel. The learned weight vector is multiplied by each channel of the input feature map to achieve channel weighting.

[0033] The global feature extraction module GAM consists of a global one-dimensional convolution layer and a global self-attention mechanism layer connected in sequence. The global one-dimensional convolution layer is used to perform one-dimensional convolution operations on convolution feature maps of different scales to obtain a global one-dimensional feature map. The global self-attention mechanism layer is used to capture long-range dependencies in the global one-dimensional feature map using the dot product attention mechanism to obtain a global feature map. The working principle of the dot product attention mechanism is as follows: Assume that N and C represent the length and number of channels of the input feature map sequence respectively, where N = H × W, H and W represent the height and width of the feature map respectively. Given a feature X = [x1, x2, ..., x C ]∈R N×C , using three projection matrices W Q ∈RDX×DK Generate the corresponding Q matrix, K matrix and V matrix

[0034]

[0035] Among them, D (·) Represents the dimension of (·). Note that the shapes of the Q matrix and the K matrix should be the same. Let S = QK T , S is the dot product between Q and K, which is used to calculate the similarity between Q and K. Let the normalization function ρ be The i-th Q feature can be evaluated and the jth K features The similarity between them. Let the function:

[0036] softmax row (QK T )=ρ(QK T )

[0037] Among them, softmax row Denotes the pair matrix QK T Apply the softmax function to each row of . By applying the softmax function to the result of the dot product, the attention weight can be obtained. Finally, the weighted sum of V with the obtained attention weight can be obtained to obtain the final output result. However, in the above calculation process, it is known that Then QK T ∈R N×N , resulting in computational complexity and memory complexity of O(N 2 Excessive computational and memory complexity can slow down the model's execution, consume a large amount of memory resources, and limit its scalability. Therefore, the dot-product attention mechanism is severely limited in large-scale semantic segmentation applications.

[0038] The computational complexity of the dot product attention mechanism is quadratic with the size of the input value (N = H × W), and for the channel dimension, the number of input channels C is usually much smaller than the number of pixels contained in the feature map, that is, C ≤ N. Therefore, the complexity of the softmax function corresponding to the number of channels C is O(C 2 ) is not large. Based on this, this embodiment designs this module to extract global context information while reducing computational complexity. Similar to the dot-product attention mechanism, this module has a residual connection that can add the result obtained by performing the dot product calculation to the refined features to obtain the output result of the corresponding module.

[0039] The original MANet network consists of multiple convolutional layers and multiple deconvolutional layers, forming a network structure similar to the U-ne network. The output of each convolutional layer is connected to the input of the corresponding deconvolutional layer. The convolutional layer is composed of residual modules of the residual network, and each residual module outputs convolutional feature maps of different scales. The convolutional feature maps of different scales are input to the GLAM module, which extracts local and global features through two parallel branches. Specifically:

[0040] The local feature extraction module performs one-dimensional convolution operations on convolution feature maps of different scales through a one-dimensional convolution layer to perceive the local boundary features in the convolution feature maps of different scales and obtain one-dimensional feature maps of different scales; the two-dimensional convolution layer performs two-dimensional convolution on the one-dimensional feature maps of different scales to perceive the local spatial dependency of the boundary neighborhood and obtain two-dimensional feature maps of different scales; the attention mechanism layer dynamically adjusts the channel weights that have significant boundary responses in the two-dimensional feature maps of different scales to obtain local enhanced feature maps of different scales.

[0041] The global feature extraction module GAM performs one-dimensional convolution operations on convolutional feature maps of different scales to extract several global one-dimensional feature maps; the self-attention mechanism layer obtains the global feature map by capturing the long-distance dependencies of each global one-dimensional feature map.

[0042] This embodiment also constructs an improved MANet network by relocating the four attention modules in the MANet network from the input of the deconvolution layer to the output of the convolution layer. Moving the attention modules from the input of the deconvolution layer to the output of the convolution layer allows the model to learn the importance of features earlier, thereby delivering more significant features in subsequent layers. By strengthening the focus on key features and reducing unimportant feature pairs, the model can more effectively recover or generate details in subsequent upsampling or deconvolution layers.

[0043] S2: Train the improved MANet network to obtain a segmentation model for segmenting remote sensing images.

[0044] The experiments conducted during the training process of this example were all conducted on a Windows 10 system based on the PyTorch deep learning framework. The GPU of the experimental platform was NVIDIA TITAN X, the model was built based on PyTorch 2.1.2, and the programming language used was Python 3.10. The dataset used in this example is the ISPRS Potsdam dataset, which aims to more objectively demonstrate the performance of the model. This dataset provides high-resolution remote sensing images of urban areas taken by drones. Each image has a spatial resolution of 5 cm and a size of 6000×6000. A total of 38 images are provided, including near-infrared, red, green, blue, DSM, and normalized DSM images. At the same time, each image corresponds to a real surface image. The dataset contains a total of six categories: impervious surfaces, buildings, low vegetation, trees, cars, and debris / background.

[0045] To objectively evaluate the performance of the proposed GLAM and its branch modules GAM and LAM, this example conducted ablation experiments on the ISPRSPotsdam dataset. The base model was an FCN (Fully Convolutional Network) model with Resnet-50 as the encoder. Table 1 shows examples of the ablation experiment results.

[0046] Table 1 Ablation experiment results

[0047] method Impervious surface building low vegetation trees car mF1 OA mIoU FCN 91.13 95.09 85.64 86.55 95.09 90.70 88.72 83.23 FCN+GAM 92.43 95.94 86.59 86.99 95.83 91.56 89.91 84.69 FCN+LAM 91.37 95.20 86.64 87.41 94.95 91.11 89.34 83.88 GLFormer 92.87 96.79 86.96 88.09 95.55 92.05 90.53 85.51

[0048] According to ablation experiments, the addition of the GAM module improved mF1 by 0.86%, OA by 1.19%, and mIoU by 1.46%. The addition of the LAM module increased mF1 by 0.41%, OA by 0.62%, and mIoU by 0.65%. Overall, compared to FCN, GLFormer improved mF1 by 1.35%, OA by 1.81%, and mIoU by 2.28%. Across five categories, GLFormer achieved the highest accuracy, with the exception of the FCN+GAM model for car recognition, where it was slightly lower than the FCN+GAM model. mF1 (mean F1 score) represents the average F1 score for all categories; OA represents the model's classification accuracy across all pixels, i.e., the proportion of correctly classified pixels to the total number of pixels; and mIoU (mean Intersection over Union) represents the mean intersection over union ratio.

[0049] Comparison shows that the addition of the GAM and LAM modules improves model accuracy, with the FCN+GAM module achieving a significantly greater improvement. This suggests that using global attention can help semantic segmentation models better understand the entire image, capturing the correlations and spatial distribution patterns between different objects and improving their ability to recognize complex scenes. In the semantic segmentation task of relatively complex urban scenes, models that utilize global attention have a more pronounced advantage.

[0050] To objectively evaluate the performance of the GLFormer model proposed in this example, this example quantitatively compares GLFormer with current mainstream semantic segmentation models. FCN replaces the fully connected layers in traditional CNNs with convolutional and pooling layers to preserve spatial information. DeeplabV3 utilizes dilated convolution modules and spatial pyramid pooling modules for multi-scale feature fusion. DC-Swin introduces the multi-scale feature extraction capabilities of dynamic convolution and the Swin Transformer. Table 2 shows the performance comparison results of multiple models on the ISPRS Potsdam dataset.

[0051] Table 2 Performance comparison of multiple models on the ISPRS Potsdam dataset

[0052] method Impervious surface building low vegetation trees car mF1 OA mIoU FCN 91.13 95.09 85.64 86.55 95.09 90.70 88.72 83.23 DeepLabV3 90.43 94.19 83.68 82.91 93.14 88.87 87.35 80.29 DC-Swin 90.19 93.31 83.30 82.99 91.97 88.35 86.91 79.41 GLFormer 92.87 96.79 86.96 88.09 95.55 92.05 90.53 85.51

[0053] The model comparison results shown in Table 2 show that GLFormer achieves the best accuracy compared to other models. Compared to FCN, GLFormer's mF1 is 1.35% higher, its OA is 1.81% higher, and its mIoU is 2.28% higher. Compared to DeeplabV3, GLFormer's mF1 is 3.18% higher, its OA is 3.18% higher, and its mIoU is 5.22% higher. Compared to DC-Swin, GLFormer's mF1 is 3.70% higher, its OA is 3.62% higher, and its mIoU is 6.10% higher.

[0054] S3: The remote sensing image to be segmented is input into the segmentation model, and each convolution layer performs convolution operations on the remote sensing image to be segmented to obtain convolution feature maps of different scales; the local feature extraction module performs one-dimensional convolution operations on the convolution feature maps of different scales through the one-dimensional convolution layer to perceive the local boundary features in the convolution feature maps of different scales and obtain one-dimensional feature maps of different scales; the two-dimensional convolution layer performs two-dimensional convolution on the one-dimensional feature maps of different scales to perceive the local spatial dependency of the boundary neighborhood and obtain two-dimensional feature maps of different scales; the attention mechanism layer dynamically adjusts the channel weights with significant boundary response in the two-dimensional feature maps of different scales to obtain local enhanced feature maps of different scales; each deconvolution layer performs deconvolution operations on the local enhanced feature maps of different scales to restore the size of the local enhanced feature maps to the size of the remote sensing image to be segmented, and obtains the segmented remote sensing image.

[0055] The high-resolution remote sensing image to be segmented is input into the segmentation model. Multiple residual modules are used to extract features from the high-resolution remote sensing image, with each residual module outputting a set of feature maps. Each feature map is then fed into the GLAM module, which extracts different features from it through two parallel branches. One branch uses the global feature extraction module (GAM) to extract global context information features, while the other uses the local feature extraction module (LAM) to extract local context information features. The features extracted by the two branches are fused and then decoded in decoding modules corresponding to the multiple residual modules. Each decoding module receives both the fused features output by the GLAM module and the output of the previous decoding module. The feature map output by the last decoding module is fed into the final convolutional layer, outputting the segmented high-resolution remote sensing image.

[0056] The above is a remote sensing image segmentation method provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding remote sensing image segmentation device, including:

[0057] A model construction module is used to replace the core attention module in the attention module used to connect each convolution layer and each deconvolution layer of the original MANet network with a local feature extraction module to construct an improved MANet network; the local feature extraction module is composed of a one-dimensional convolution layer, a two-dimensional convolution layer and an attention mechanism layer connected in sequence.

[0058] The model training module is used to train the improved MANet network to obtain a segmentation model for segmenting remote sensing images.

[0059] The image segmentation module is used to input the remote sensing image to be segmented into the segmentation model. Each convolution layer performs convolution operations on the remote sensing image to be segmented to obtain convolution feature maps of different scales; the local feature extraction module performs one-dimensional convolution operations on the convolution feature maps of different scales through the one-dimensional convolution layer to perceive the local boundary features in the convolution feature maps of different scales and obtain one-dimensional feature maps of different scales; the two-dimensional convolution layer performs two-dimensional convolution on the one-dimensional feature maps of different scales to perceive the local spatial dependency of the boundary neighborhood and obtain two-dimensional feature maps of different scales; the attention mechanism layer dynamically adjusts the channel weights with significant boundary response in the two-dimensional feature maps of different scales to obtain local enhanced feature maps of different scales; each deconvolution layer performs deconvolution operations on the local enhanced feature maps of different scales to restore the size of the local enhanced feature maps to the size of the remote sensing image to be segmented, and obtains the segmented remote sensing image.

[0060] The specific definition of the remote sensing image segmentation device can be found in the definition of the remote sensing image segmentation method above and will not be repeated here. The various modules in the above remote sensing image segmentation device can be implemented in whole or in part by software, hardware, or a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0061] The present invention also provides a computer-readable storage medium, which stores a computer program. The computer program can be used to execute the above-mentioned remote sensing image segmentation method.

[0062] The present invention also provides a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, memory, and non-volatile storage, and may also include other hardware required for operations. The processor reads a corresponding computer program from the non-volatile storage into the memory and then executes the program to implement the aforementioned remote sensing image segmentation method.

[0063] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this embodiment can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0064] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A remote sensing image segmentation method, characterized in that: include: The improved MANet network is constructed by replacing the core attention module in the attention module used to connect each convolutional layer and each deconvolutional layer of the original MANet network with a local feature extraction module; the local feature extraction module is composed of a one-dimensional convolutional layer, a two-dimensional convolutional layer, and an attention mechanism layer connected in sequence; The improved MANet network is trained to obtain a segmentation model for segmenting remote sensing images; The remote sensing image to be segmented is input into the segmentation model. Each convolution layer performs convolution operations on the remote sensing image to be segmented to obtain convolution feature maps of different scales. The local feature extraction module performs one-dimensional convolution operations on the convolution feature maps of different scales through the one-dimensional convolution layer to perceive the local boundary features in the convolution feature maps of different scales and obtain one-dimensional feature maps of different scales. The two-dimensional convolution layer performs two-dimensional convolution on the one-dimensional feature maps of different scales to perceive the local spatial dependency of the boundary neighborhood and obtain two-dimensional feature maps of different scales. The attention mechanism layer obtains local enhanced feature maps of different scales by dynamically adjusting the channel weights that have significant boundary responses in two-dimensional feature maps of different scales; each deconvolution layer performs deconvolution operations on the local enhanced feature maps of different scales to restore the size of the local enhanced feature maps to the size of the remote sensing image to be segmented, thereby obtaining the segmented remote sensing image.

2. The remote sensing image segmentation method according to claim 1, wherein: The attention mechanism layer uses a squeeze-excite attention mechanism.

3. The remote sensing image segmentation method according to claim 1, wherein: The convolution kernel size of the one-dimensional convolution layer is 1×1, and the convolution kernel size of the two-dimensional convolution layer is 3×3.

4. The remote sensing image segmentation method according to claim 1, wherein: The attention module is connected to the output end of each convolutional layer to transfer local boundary features in each convolutional layer.

5. The remote sensing image segmentation method according to claim 1, wherein: Each convolutional layer is composed of residual blocks connected in series.

6. The remote sensing image segmentation method according to claim 1, wherein: The attention module further includes: a global feature extraction module in parallel with the local feature extraction module; The steps of the global feature extraction module capturing global features in convolutional feature maps of different scales include: A global one-dimensional convolution layer is used to perform one-dimensional convolution operations on convolution feature maps of different scales to obtain a global one-dimensional feature map; the output end of the global one-dimensional convolution layer is connected to a global self-attention mechanism layer, which uses a dot product attention mechanism to capture long-distance dependencies in the global one-dimensional feature map to obtain a global feature map.

7. The remote sensing image segmentation method according to claim 6, wherein: The global feature map is used to be spliced ​​with the local enhanced feature map; and the spliced ​​result is used as the output of the attention module.

8. A remote sensing image segmentation device, characterized in that: include: A model construction module is used to replace the core attention module in the attention module used to connect each convolution layer and each deconvolution layer of the original MANet network with a local feature extraction module to construct an improved MANet network; the local feature extraction module is composed of a one-dimensional convolution layer, a two-dimensional convolution layer, and an attention mechanism layer connected in sequence; Model training module, used to train the improved MANet network to obtain a segmentation model for segmenting remote sensing images; The image segmentation module is used to input the remote sensing image to be segmented into the segmentation model. Each convolution layer performs convolution operations on the remote sensing image to be segmented to obtain convolution feature maps of different scales. The local feature extraction module performs one-dimensional convolution operations on the convolution feature maps of different scales through the one-dimensional convolution layer to perceive the local boundary features in the convolution feature maps of different scales and obtain one-dimensional feature maps of different scales. The two-dimensional convolution layer performs two-dimensional convolution on the one-dimensional feature maps of different scales to perceive the local spatial dependency of the boundary neighborhood and obtain two-dimensional feature maps of different scales. The attention mechanism layer obtains local enhanced feature maps of different scales by dynamically adjusting the channel weights that have significant boundary responses in two-dimensional feature maps of different scales; each deconvolution layer performs deconvolution operations on the local enhanced feature maps of different scales to restore the size of the local enhanced feature maps to the size of the remote sensing image to be segmented, thereby obtaining the segmented remote sensing image.

9. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images