Remote sensing image segmentation method and device, medium and equipment

Abstract

The invention discloses a remote sensing image segmentation method and device, a medium and equipment, and relates to the technical field of image processing. Comprising the step of constructing an improved MANet network by replacing a kernel attention module in an original MANet network with a one-dimensional convolutional layer, a two-dimensional convolutional layer and an attention mechanism layer which are connected in sequence. Sensing boundary features in the convolution feature map through a one-dimensional convolution operation; secondly, performing two-dimensional convolution operation on the one-dimensional feature map by using a two-dimensional convolution layer to perceive a spatial dependency relationship located in a boundary neighborhood, and enhancing the spatial perception capability of details of the boundary neighborhood; an attention mechanism is used for the output of the two-dimensional convolution layer to dynamically adjust the channel weight which is remarkable in boundary response in the two-dimensional feature map, and attention can be focused on the feature channel related to the segmented boundary in the remote sensing image, so that the detailed capturing capability of the boundary local features is improved, and a more accurate remote sensing image segmentation result is obtained.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a remote sensing image segmentation method, device, medium and equipment. Background Art

[0002] Remote sensing images contain rich information about ground objects. Segmenting them can obtain surface coverage classification, thus providing a basis for monitoring surface changes. In recent years, remote sensing image semantic segmentation has become a research hotspot in the field of computer vision and remote sensing technology, aiming to segment pixels or regions in remote sensing images into categories with specific semantic labels.

[0003] MANet (Multi-Attention Network) is a network model for semantic segmentation of remote sensing images proposed by Rui Li et al. in the paper "Multi-Attention-Network for Semantic Segmentation of Fine Resolution Remote Sensing Images". The MANet network includes 4 attention modules, each of which includes two parallel branches: the kernel attention module KAM (Kernel Attention Mechanism) and the channel attention module CAM (Channel Attention Mechanism). The kernel attention module KAM is based on the kernel's generalized dot product attention mechanism to process the long-distance dependencies between pixels in remote sensing images to achieve segmentation of remote sensing images.

[0004] However, since the kernel attention module KAM pays more attention to the long-distance global relationship between pixels in remote sensing images, it ignores the local features in remote sensing images. Local features are crucial to the segmentation of boundaries of different objects. The lack of local features may cause the network to inaccurately segment the segmentation boundaries. Summary of the invention

[0005] Based on this, it is necessary to provide a remote sensing image segmentation method, device, medium and equipment to address the above technical problems.

[0006] The present invention adopts the following technical solutions:

[0007] The present invention provides a remote sensing image segmentation method, comprising:

[0008] Build and improve the MANet network, including:

[0009] The kernel attention module in the attention module used to connect each convolution layer and each deconvolution layer in the original MANet network is replaced by a local feature extraction module consisting of a one-dimensional convolution layer, a two-dimensional convolution layer and an attention mechanism layer connected in sequence; the convolution kernel size of the one-dimensional convolution layer is 1×1, the convolution kernel size of the two-dimensional convolution layer is 3×3; the attention mechanism layer uses a compression-excitation attention mechanism. It also includes directly connecting the attention module to the output end of each convolution layer to transfer local boundary features in each convolution layer.

[0010] The improved attention module includes a local feature extraction module and a global feature extraction module. The global feature extraction module is used to capture global features in convolution feature maps of different scales, including a global one-dimensional convolution layer and a global self-attention mechanism layer connected to the output end of the global one-dimensional convolution layer. The global one-dimensional convolution layer is used to perform one-dimensional convolution operations on convolution feature maps of different scales to obtain a global one-dimensional feature map; the global self-attention mechanism layer is used to capture long-distance dependencies in the global one-dimensional feature map using a dot product attention mechanism to obtain a global feature map; the outputs of the local feature extraction module and the global feature extraction module are spliced, and the spliced ​​result is used as the output of the attention module.

[0011] The improved MANet network is trained to obtain a segmentation model for segmenting remote sensing images; the remote sensing image to be segmented is input into the segmentation model, and each convolution layer performs convolution operations on the remote sensing image to be segmented to obtain convolution feature maps of different scales; the local feature extraction module performs one-dimensional convolution operations on the convolution feature maps of different scales through a one-dimensional convolution layer to perceive the local boundary features in the convolution feature maps of different scales and obtain one-dimensional feature maps of different scales; the two-dimensional convolution layer performs two-dimensional convolution on the one-dimensional feature maps of different scales to perceive the local spatial dependency of the boundary neighborhood and obtain two-dimensional feature maps of different scales; the attention mechanism layer dynamically adjusts the channel weights with significant boundary response in the two-dimensional feature maps of different scales to obtain local enhanced feature maps of different scales; each deconvolution layer performs deconvolution operations on the local enhanced feature maps of different scales to restore the size of the local enhanced feature map to the size of the remote sensing image to be segmented, and obtains the segmented remote sensing image.

[0012] The present invention provides a remote sensing image segmentation device, comprising:

[0013] A model building module, used to replace the core attention module in the attention module used to connect each convolution layer and each deconvolution layer of the original MANet network with a local feature extraction module to build an improved MANet network; the local feature extraction module is composed of a one-dimensional convolution layer, a two-dimensional convolution layer and an attention mechanism layer connected in sequence;

[0014] Model training module, used to train the improved MANet network to obtain a segmentation model for segmenting remote sensing images;

[0015] The image segmentation module is used to input the remote sensing image to be segmented into the segmentation model. Each convolution layer performs convolution operations on the remote sensing image to be segmented to obtain convolution feature maps of different scales. The local feature extraction module performs one-dimensional convolution operations on the convolution feature maps of different scales through the one-dimensional convolution layer to perceive the local boundary features in the convolution feature maps of different scales and obtain one-dimensional feature maps of different scales. The two-dimensional convolution layer performs two-dimensional convolution on the one-dimensional feature maps of different scales to perceive the local spatial dependency of the boundary neighborhood and obtain two-dimensional feature maps of different scales. The attention mechanism layer dynamically adjusts the channel weights with significant boundary response in the two-dimensional feature maps of different scales to obtain local enhanced feature maps of different scales. Each deconvolution layer performs deconvolution operations on the local enhanced feature maps of different scales to restore the size of the local enhanced feature map to the size of the remote sensing image to be segmented, and obtains the segmented remote sensing image.

[0016] The present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the remote sensing image segmentation method is implemented.

[0017] The present invention provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the remote sensing image segmentation method when executing the program.

[0018] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects:

[0019] The present invention firstly performs a one-dimensional convolution operation on a convolution feature map to perceive the boundary features in the convolution feature map; then uses a two-dimensional convolution layer to perform a two-dimensional convolution operation on the one-dimensional feature map. Since the two-dimensional convolution operation is performed in each pixel neighborhood in the one-dimensional feature map, the spatial dependency relationship in the boundary neighborhood can be perceived, thereby enhancing the spatial perception ability of the boundary neighborhood details; an attention mechanism is used for the output of the two-dimensional convolution layer to dynamically adjust the channel weights in the two-dimensional feature map that have significant boundary responses, so that attention can be focused on the feature channels related to the segmentation boundary in the remote sensing image, thereby improving the ability to capture the detailed boundary local features; through the above steps, more detailed boundary local features can be obtained, thereby obtaining a more accurate remote sensing image segmentation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0021] Figure 1 A schematic diagram of a remote sensing image segmentation method provided by the present invention;

[0022] Figure 2 A schematic diagram of the structure of the original MANet network provided by the present invention;

[0023] Figure 3 The present invention provides Figure 2 Schematic diagram of the structure of attention module 1 to attention module 5;

[0024] Figure 4 A schematic diagram of the structure of the improved MANet network provided by the present invention;

[0025] Figure 5 The present invention provides Figure 4 Schematic diagram of the structures of the local and global feature extraction modules-1 to-14 in the figure. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in combination with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] In view of the limitations of the standard semantic segmentation model in terms of feature utilization, long-distance dependency modeling, and computational complexity, MANet optimizes the dot product attention mechanism in the spatial domain and channel domain respectively, and proposes new attention mechanism models: KAM and CAM. The two are connected in parallel to form attention modules 1 to 4. Combined with the ResNet-50 backbone network, a semantic segmentation model that integrates multi-scale features is designed. The structure of the original MANet network is as follows: Figure 2 As shown in FIG. 1 , the ResNet-50 backbone network includes a 1×1 convolutional layer, a residual block 2, a residual block 3, a residual block 4, and a residual block 5 connected in sequence. The output ends of the above convolutional layers or residual blocks are respectively connected to the input ends of downsampling 1 to downsampling 5 to obtain the features. Figure 1 ~Features Figure 5 ;feature Figure 5 Input to the deconvolution layer 5, and compare the obtained results with the features Figure 4 After fusion, it is input to the attention module 4, and the output of the attention module 4 is combined with the feature Figure 4Fusion, and so on, until the output of attention module 1 is bilinearly interpolated to obtain the segmentation result. Figure 2 The structures of attention modules 1 to 5 in the Figure 3 As shown in the figure, it includes two parallel branches, one of which includes a 1×1 convolution layer, batch normalization + activation function, channel attention module, 3×3 convolution layer and batch normalization + activation function connected in sequence, and the branch outputs a channel feature map; the other branch includes a 1×1 convolution layer, batch normalization + activation function, kernel attention module, 3×3 convolution layer and batch normalization + activation function connected in sequence, and the branch outputs a kernel feature map, and the channel feature map and the kernel feature map are concatenated to obtain the output feature map. However, the attention mechanism mainly focuses on the importance of local areas and channels, which may lead to the loss of global context. The loss of global information will affect the network's overall understanding of remote sensing images, which may lead to inaccurate semantic segmentation.

[0028] Based on this, this embodiment proposes an improved MANet network, which is named GLFormer (Global-LocalFormer) model to solve the above problems. The corresponding attention modules in the GLFormer model are divided into GAM (GlobalAttention Module) and LAM module (LocalAttentionModule). The LAM module is a series of convolutional layers with two kernel sizes of 1 and 3, a batch normalization operation and a ReLU activation function, and a compression-excitation attention mechanism module behind the branch. The purpose of both is to extract context information from the global and local scales respectively.

[0029] Figure 1 The process of the remote sensing image segmentation method provided by this embodiment is shown. Figure 1 The remote sensing image segmentation method is described in detail, and specifically includes the following steps:

[0030] S1: The core attention module in the attention module used to connect each convolution layer and each deconvolution layer in the original MANet network is replaced with a local feature extraction module to construct an improved MANet network; the local feature extraction module is composed of a one-dimensional convolution layer, a two-dimensional convolution layer and an attention mechanism layer connected in sequence.

[0031] In this embodiment, the two parallel branches of the core attention module KAM and the channel attention module CAM in the four attention modules in the MANet network are replaced by two parallel branches of the local feature extraction module LAM and the global feature extraction module GAM to construct an improved MANet network. The two parallel local feature extraction modules LAM and the global feature extraction module GAM are collectively referred to as GLAM modules. The improved MANet network structure is as follows: Figure 4 As shown, it includes a 1×1 convolution layer, a residual block-1, a residual block-2, a residual block-3 and a residual block-4 connected in sequence. The output ends of the above convolution layers or residual blocks are respectively connected to the input ends of the global and local feature extraction modules-1 to the global and local feature extraction modules-4 to obtain feature maps-1 to feature maps-4 respectively; the feature map-4 is input into the decoding block-4, and the result obtained is fused with the feature map-3 and input into the decoding block-3, and so on, until the output of the decoding block-1 is subjected to one 3×3 deconvolution operation and one 3×3 convolution operation to obtain the segmentation result. Figure 4 The structure of the global and local feature extraction modules in Figure 5 As shown in the figure, it includes two parallel branches, one of which includes a 1×1 convolution layer and a global self-attention mechanism layer connected in sequence, and this branch outputs global features; the other branch includes a 1×1 one-dimensional convolution layer, a 3×3 two-dimensional convolution layer, a batch normalization + activation function and an attention mechanism layer connected in sequence, and this branch outputs local features. The global features and local features are spliced ​​to obtain fused features. Specifically:

[0032] The local feature extraction module contains two concatenated convolutional layers with kernel sizes of 1 and 3, i.e., the convolution kernel size of the one-dimensional convolutional layer is 1×1, the convolution kernel size of the two-dimensional convolutional layer is 3×3, and a batch normalization operation and ReLU activation function, as well as a compression-excitation attention mechanism layer behind the branch. The working principle of the compression-excitation attention mechanism layer is as follows: globally pooling the input feature map to compress the feature map of each channel; the compressed feature map is processed through two fully connected layers: the first fully connected layer is used to reduce the number of channels, and the second fully connected layer is used to restore the number of channels to the original number of channels to learn the nonlinear relationship between channels; after the second fully connected layer, the Sigmoid activation function is used to generate weights between 0 and 1, which are used to represent the importance of each channel, and the learned weight vector is multiplied by each channel of the input feature map to achieve channel weighting.

[0033] The global feature extraction module GAM includes a global one-dimensional convolution layer and a global self-attention mechanism layer connected in sequence. The global one-dimensional convolution layer is used to perform one-dimensional convolution operations on convolution feature maps of different scales to obtain a global one-dimensional feature map. The global self-attention mechanism layer is used to capture long-distance dependencies in the global one-dimensional feature map using the dot product attention mechanism to obtain a global feature map. The working principle of the dot product attention mechanism is: Assume that N and C represent the length and number of channels of the input feature map sequence, respectively, where N = H × W, H and W represent the height and width of the feature map, respectively. Given a feature X = [x1, x2, ..., x C ]∈R N×C , using three projection matrices W Q ∈R DX×DK Generate the corresponding Q matrix, K matrix and V matrix

[0034]

[0035] Among them, D (·) Represents the dimension of (·). Note that the shapes of the Q matrix and the K matrix should be the same. Let S = QK T , S is the dot product between Q and K, which is used to calculate the similarity between Q and K. Let the normalization function ρ be The i-th Q feature can be evaluated and the jth K features The similarity between them. Let the function:

[0036] softmax row (QK T )=ρ(QK T )

[0037] Among them, softmax row Denotes the pair matrix QK T Apply the softmax function to each row of . By applying the softmax function to the result of the dot product, the attention weight can be obtained. Finally, the weighted sum of V with the obtained attention weight can be obtained to get the final output result. However, in the above calculation process, it is known that QK T ∈R N×N , resulting in a computational complexity and memory complexity of O(N 2 ). Excessive computational complexity and memory complexity will affect the running speed of the model, occupy a large amount of memory resources, and limit the scalability of the model. Therefore, the dot product attention mechanism will be severely limited in large-scale semantic segmentation applications.

[0038] The computational complexity of the dot product attention mechanism is quadratic with the size of the input value (N = H × W), and for the channel dimension, the number of input channels C is usually much smaller than the number of pixels contained in the feature map, that is, C ≤ N. Therefore, the complexity of the softmax function corresponding to the number of channels C is O(C 2 ) is not large. Based on this, the present embodiment designs this module to reduce the computational complexity while extracting global context information. Similar to the dot product attention mechanism, this module has a residual connection, which can add the result obtained by performing the dot product calculation to the refined features to obtain the output result of the corresponding module.

[0039] The original MANet network includes multiple convolutional layers and multiple deconvolutional layers, which form a network structure similar to the U-ne network. The output of each convolutional layer is connected to the input of the corresponding deconvolutional layer. Among them, the convolutional layer is composed of the residual modules of the residual network, and each residual module outputs convolutional feature maps of different scales. The convolutional feature maps of different scales are input into the GLAM module, and the GLAM module extracts local and global features through two parallel branches. Specifically:

[0040] The local feature extraction module performs one-dimensional convolution operations on convolution feature maps of different scales through a one-dimensional convolution layer to perceive the local boundary features in the convolution feature maps of different scales and obtain one-dimensional feature maps of different scales; the two-dimensional convolution layer performs two-dimensional convolution on the one-dimensional feature maps of different scales to perceive the local spatial dependency of the boundary neighborhood and obtain two-dimensional feature maps of different scales; the attention mechanism layer dynamically adjusts the channel weights with significant boundary responses in the two-dimensional feature maps of different scales to obtain local enhanced feature maps of different scales.

[0041] The global feature extraction module GAM performs one-dimensional convolution operations on convolutional feature maps of different scales to extract several global one-dimensional feature maps; the self-attention mechanism layer obtains the global feature map by capturing the long-distance dependencies of each global one-dimensional feature map.

[0042] This embodiment also modifies the positions of the four attention modules in the MANet network from the input end of the deconvolution layer to the output end of the convolution layer to construct an improved MANet network. Moving the attention module from the input end of the deconvolution layer to the output end of the convolution layer allows the model to learn the importance of features earlier, thereby delivering more significant features in subsequent layers. By strengthening the focus on key features and reducing unimportant feature pairs, the model can more effectively restore or generate details in subsequent upsampling or deconvolution layers.

[0043] S2: Train the improved MANet network to obtain a segmentation model for segmenting remote sensing images.

[0044] The experiments conducted during the training process of this embodiment are all conducted on the Windows 10 system based on the PyTorch deep learning framework. The GPU of the experimental platform is NVIDIA TITAN X, the model is built based on PyTorch2.1.2, and the programming language used is Python 3.10. The data set used in this embodiment is ISPRS Potsdam, which aims to more objectively demonstrate the performance of the model. The data set provides high-resolution remote sensing images of urban areas taken by drones. The spatial resolution of each image is 5cm and the size is 6000×6000. A total of 38 images including near-infrared, red, green, blue, DSM and normalized DSM images are provided. At the same time, each image corresponds to a real image of the surface. The data set contains a total of six categories: impermeable surfaces, buildings, low vegetation, trees, cars and debris / background.

[0045] In order to objectively evaluate the performance of the proposed GLAM and its branch modules GAM and LAM, this embodiment performs an ablation experiment on the ISPRSPotsdam dataset, and the basic model is an FCN model (Fully Convolutional Network) with Resnet-50 as the encoder. An example of the ablation experiment results is shown in Table 1.

[0046] Table 1 Ablation experiment results

[0047] method Impervious surface building Low vegetation Trees car mF1 OA QUR FCN 91.13 95.09 85.64 86.55 95.09 90.70 88.72 83.23 FCN+GAM 92.43 95.94 86.59 86.99 95.83 91.56 89.91 84.69 FCN+LAM 91.37 95.20 86.64 87.41 94.95 91.11 89.34 83.88 GLFormer 92.87 96.79 86.96 88.09 95.55 92.05 90.53 85.51

[0048] According to the results of the ablation experiment, with the addition of the GAM module, mF1 increased by 0.86%, OA increased by 1.19%, and mIoU increased by 1.46%; with the addition of the LAM module, mF1 increased by 0.41%, OA increased by 0.62%, and mIoU increased by 0.65%. Overall, compared with FCN, GLFormer has increased mF1 by 1.35%, OA by 1.81%, and mIoU by 2.28%. For the five different categories, except for the slightly lower accuracy of identifying cars than the FCN+GAM model, the accuracy of the other four categories is the highest. Among them, mF1 (mean F1 score) means the average F1 score of all categories; OA means the classification accuracy of the model on all pixels, that is, the proportion of correctly classified pixels to the total number of pixels; mIoU (mean Intersection over Union) refers to the average intersection over union ratio.

[0049] After comparison, it can be seen that after adding the GAM and LAM modules, the accuracy of the model has been improved, and relatively speaking, the results of the FCN+GAM module have been improved relatively much, indicating that for the semantic segmentation model, the use of global attention can help the model better understand the entire image, so that the model can capture the correlation and spatial distribution between different objects, thereby improving the model's ability to recognize complex scenes. In the semantic segmentation task of relatively complex urban scenes, the advantages of the semantic segmentation model that uses the global attention mechanism are relatively obvious.

[0050] In order to objectively evaluate the performance of the GLFormer model proposed in this embodiment, this embodiment quantitatively compares GLFormer with the current mainstream semantic segmentation model. FCN replaces the fully connected layer in the traditional CNN with convolutional layers and pooling layers to retain spatial information. DeeplabV3 uses a hole convolution module and a spatial pyramid pooling module for multi-scale feature fusion. DC-Swin introduces the multi-scale feature extraction capabilities of dynamic convolution and Swin Transformer. The performance comparison results of multiple models on the ISPRS Potsdam dataset are shown in Table 2.

[0051] Table 2 Performance comparison results of multiple models on the ISPRS Potsdam dataset

[0052] method Impervious surface building Low vegetation Trees car mF1 OA QUR FCN 91.13 95.09 85.64 86.55 95.09 90.70 88.72 83.23 DeepLabV3 90.43 94.19 83.68 82.91 93.14 88.87 87.35 80.29 DC-Swin 90.19 93.31 83.30 82.99 91.97 88.35 86.91 79.41 GLFormer 92.87 96.79 86.96 88.09 95.55 92.05 90.53 85.51

[0053] From the model comparison results shown in Table 2, it can be seen that GLFormer has the best performance in terms of accuracy compared with other models. Compared with FCN, GLFormer's mF1 is 1.35% higher, OA is 1.81% higher, and mIoU is 2.28% higher; compared with DeeplabV3, GLFormer's mF1 is 3.18% higher, OA is 3.18% higher, and mIoU is 5.22% higher; compared with DC-Swin, GLFormer's mF1 is 3.70% higher, OA is 3.62% higher, and mIoU is 6.10% higher.

[0054] S3: The remote sensing image to be segmented is input into the segmentation model, and each convolution layer performs convolution operations on the remote sensing image to be segmented to obtain convolution feature maps of different scales; the local feature extraction module performs one-dimensional convolution operations on the convolution feature maps of different scales through the one-dimensional convolution layer to perceive the local boundary features in the convolution feature maps of different scales and obtain one-dimensional feature maps of different scales; the two-dimensional convolution layer performs two-dimensional convolution on the one-dimensional feature maps of different scales to perceive the local spatial dependency of the boundary neighborhood and obtain two-dimensional feature maps of different scales; the attention mechanism layer dynamically adjusts the channel weights with significant boundary response in the two-dimensional feature maps of different scales to obtain local enhanced feature maps of different scales; each deconvolution layer performs deconvolution operations on the local enhanced feature maps of different scales to restore the size of the local enhanced feature map to the size of the remote sensing image to be segmented, and obtains the segmented remote sensing image.

[0055] The high-resolution remote sensing image to be segmented is input into the segmentation model, and features of the high-resolution remote sensing image are extracted through multiple residual modules. Each residual module outputs a set of feature maps. Each feature map is input into the GLAM module. The GLAM module extracts different features through two parallel branches. One branch uses the global feature extraction module GAM to extract global context information features, and the other branch uses the local feature extraction module LAM to extract local context information features. After the features extracted by the two branches are fused, they are input into the decoding modules corresponding to the multiple residual modules for decoding. The input of each decoding module includes both the fused features output by the GLAM module and the output of the previous decoding module. The feature map output by the last decoding module is input into the final convolutional layer to output the segmented high-resolution remote sensing image.

[0056] The above is a remote sensing image segmentation method provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding remote sensing image segmentation device, including:

[0057] A model building module is used to replace the core attention module in the attention module of the original MANet network used to connect each convolution layer and each deconvolution layer with a local feature extraction module to build an improved MANet network; the local feature extraction module is composed of a one-dimensional convolution layer, a two-dimensional convolution layer and an attention mechanism layer connected in sequence.

[0058] The model training module is used to train the improved MANet network to obtain a segmentation model for segmenting remote sensing images.

[0059] The image segmentation module is used to input the remote sensing image to be segmented into the segmentation model. Each convolution layer performs convolution operations on the remote sensing image to be segmented to obtain convolution feature maps of different scales. The local feature extraction module performs one-dimensional convolution operations on the convolution feature maps of different scales through the one-dimensional convolution layer to perceive the local boundary features in the convolution feature maps of different scales and obtain one-dimensional feature maps of different scales. The two-dimensional convolution layer performs two-dimensional convolution on the one-dimensional feature maps of different scales to perceive the local spatial dependency of the boundary neighborhood and obtain two-dimensional feature maps of different scales. The attention mechanism layer dynamically adjusts the channel weights with significant boundary response in the two-dimensional feature maps of different scales to obtain local enhanced feature maps of different scales. Each deconvolution layer performs deconvolution operations on the local enhanced feature maps of different scales to restore the size of the local enhanced feature map to the size of the remote sensing image to be segmented, and obtains the segmented remote sensing image.

[0060] For the specific definition of the remote sensing image segmentation device, please refer to the definition of the remote sensing image segmentation method above, which will not be repeated here. Each module in the above remote sensing image segmentation device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0061] The present invention also provides a computer-readable storage medium, which stores a computer program. The computer program can be used to execute the above-mentioned remote sensing image segmentation method.

[0062] The present invention also provides a computer device, which includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory at the hardware level, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above remote sensing image segmentation method.

[0063] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this embodiment can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0064] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A remote sensing image segmentation method, characterized in that: include: The core attention module in the attention module used to connect each convolution layer and each deconvolution layer of the original MANet network is replaced with a local feature extraction module to construct an improved MANet network; the local feature extraction module is composed of a one-dimensional convolution layer, a two-dimensional convolution layer and an attention mechanism layer connected in sequence; The improved MANet network is trained to obtain a segmentation model for segmenting remote sensing images; The remote sensing image to be segmented is input into the segmentation model, and each convolution layer performs convolution operation on the remote sensing image to be segmented to obtain convolution feature maps of different scales; the local feature extraction module performs one-dimensional convolution operation on the convolution feature maps of different scales through the one-dimensional convolution layer to perceive the local boundary features in the convolution feature maps of different scales and obtain one-dimensional feature maps of different scales; the two-dimensional convolution layer performs two-dimensional convolution on the one-dimensional feature maps of different scales to perceive the local spatial dependency of the boundary neighborhood and obtain two-dimensional feature maps of different scales; The attention mechanism layer obtains local enhanced feature maps of different scales by dynamically adjusting the channel weights with significant boundary responses in two-dimensional feature maps of different scales; each deconvolution layer performs deconvolution operations on local enhanced feature maps of different scales to restore the size of the local enhanced feature maps to the size of the remote sensing image to be segmented, thereby obtaining the segmented remote sensing image.

2. The remote sensing image segmentation method according to claim 1, characterized in that: The attention mechanism layer uses a compression-excitation attention mechanism.

3. The remote sensing image segmentation method according to claim 1, characterized in that: The convolution kernel size of the one-dimensional convolution layer is 1×1, and the convolution kernel size of the two-dimensional convolution layer is 3×3.

4. The remote sensing image segmentation method according to claim 1, characterized in that: The attention module is connected to the output end of each convolutional layer to transfer local boundary features in each convolutional layer.

5. The remote sensing image segmentation method according to claim 1, characterized in that: Each convolutional layer is composed of residual blocks connected in series.

6. The remote sensing image segmentation method according to claim 1, characterized in that: The attention module also includes: a global feature extraction module in parallel with the local feature extraction module; The steps of capturing global features in convolutional feature maps of different scales by the global feature extraction module include: A global one-dimensional convolution layer is used to perform one-dimensional convolution operations on convolution feature maps of different scales to obtain a global one-dimensional feature map; the output end of the global one-dimensional convolution layer is connected to a global self-attention mechanism layer, and the global self-attention mechanism layer uses a dot product attention mechanism to capture long-distance dependencies in the global one-dimensional feature map to obtain a global feature map.

7. The remote sensing image segmentation method according to claim 6, characterized in that: The global feature map is used to be spliced ​​with the local enhanced feature map; and the spliced ​​result is used as the output of the attention module.

8. A remote sensing image segmentation device, characterized in that: include: A model building module, used to replace the core attention module in the attention module used to connect each convolution layer and each deconvolution layer of the original MANet network with a local feature extraction module to build an improved MANet network; the local feature extraction module is composed of a one-dimensional convolution layer, a two-dimensional convolution layer and an attention mechanism layer connected in sequence; Model training module, used to train the improved MANet network to obtain a segmentation model for segmenting remote sensing images; The image segmentation module is used to input the remote sensing image to be segmented into the segmentation model. Each convolution layer performs convolution operations on the remote sensing image to be segmented to obtain convolution feature maps of different scales. The local feature extraction module performs one-dimensional convolution operations on convolution feature maps of different scales through one-dimensional convolution layers to perceive local boundary features in convolution feature maps of different scales and obtain one-dimensional feature maps of different scales. The two-dimensional convolution layer performs two-dimensional convolution on one-dimensional feature maps of different scales to perceive the local spatial dependency of the boundary neighborhood and obtain two-dimensional feature maps of different scales. The attention mechanism layer obtains local enhanced feature maps of different scales by dynamically adjusting the channel weights with significant boundary responses in two-dimensional feature maps of different scales; each deconvolution layer performs deconvolution operations on local enhanced feature maps of different scales to restore the size of the local enhanced feature maps to the size of the remote sensing image to be segmented, thereby obtaining the segmented remote sensing image.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 7 is implemented.

Images