Remote sensing image building extraction method based on edge and topology perception

Abstract

The invention belongs to the technical field of remote sensing image semantic segmentation methods, and provides a remote sensing image building extraction method based on edge and topology perception in order to overcome the defects of unclear edge information extraction and the like in practical application in the prior art. Training the high-resolution remote sensing image building extraction network, and inputting a building image to be detected into the trained high-resolution remote sensing image building extraction network to obtain a building extraction result in the high-resolution remote sensing image; the problems that in an existing high-resolution remote sensing image building extraction method, the overall topological structure and edge details of a building cannot be accurately sensed, the calculation amount is large in the global context information obtaining process, and the method is prone to being interfered by surface similar ground objects, shadows and shielding are comprehensively solved.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image semantic segmentation methods, and in particular relates to a remote sensing image building extraction method based on edge and topology perception. Background Art

[0002] Accurate building extraction plays a vital role in many applications, such as building reconstruction, mapping, population estimation, and illegal building monitoring. Manually delineating individual buildings using tools such as ArcGIS takes a lot of time and effort. With the development of remote sensing technology, high-resolution images have been widely used, providing opportunities for automatic building extraction. However, it is still a challenge to develop an effective automatic building extraction method due to the complex background information, dense distribution, and different appearances of buildings.

[0003] Traditional methods for extracting buildings from high-resolution remote sensing images usually extract artificially designed features such as spectral, spatial and texture features, and apply traditional machine learning methods (such as support vector machines and random forests) to identify buildings. However, artificial features designed based on experience limit the generalization performance of these traditional methods.

[0004] Fully convolutional neural networks based on deep learning can automatically obtain high-level semantic features from remote sensing images through multi-layer network structures and nonlinear transformations. In addition, the end-to-end, pixel-to-pixel implementation of fully convolutional neural networks can provide pixel-level road recognition and positioning. Although these fully convolutional methods have achieved encouraging results, they have encountered bottlenecks in building extraction. Specifically, fixed-size convolution kernels lead to local receptive fields of view. Therefore, fully convolutional networks are designed to extract local contexts and are inherently lacking the ability to model global contexts. However, when identifying building pixels, local contexts are often vague, and if there are global image contexts or long-range dependencies, the extraction work becomes much simpler. To capture the global context, the most common method is to incorporate attention modules into the network. Typical methods include non-local modules, dual attention modules, cross attention modules, and object context modules. In the field of building extraction, some studies have also introduced attention modules to strengthen feature representation, thereby distinguishing heterogeneous buildings from complex backgrounds in fine-resolution remote sensing images. However, these methods still use the fully convolutional network structure, which limits the global feature representation.

[0005] Transformer was originally designed for natural language processing tasks. Recently, its variant, the Visual Transformer (ViT), has become a hot topic in computer vision. Different from the convolutional neural network structure, ViT transforms a 2D image into a 1D sequence and applies a self-attention mechanism for feature description. Thanks to this design, ViT is better than convolutional neural networks in extracting global context and has achieved many breakthroughs in basic visual tasks such as image classification, object detection, and semantic segmentation. However, when ViT is used in practice, it often incurs huge memory requirements and computational costs, which seriously affects its potential in downstream tasks. In addition, ViTs mainly focus on capturing global context and ignore the preservation of spatial detail context, but spatial details are also crucial for fine-grained building segmentation. Although SwinTransformer adopts a hierarchical structure and designs a window-based multi-head self-attention mechanism to improve efficiency, its complexity still increases fourfold with the increase of window size. More importantly, the Transformer-based building extraction method loses a lot of spatial structure information due to the self-attention mechanism when converting a two-dimensional image into a one-dimensional sequence, causing the model to be overly dependent on the appearance features of the building, and thus easily interfered by objects with similar appearances, shadows, and occlusions. In addition, the existing model is affected by operations such as downsampling, and the edge information is seriously lost during the building extraction process, which seriously affects the accuracy of the extracted buildings. These defects greatly limit the practical application of these methods. Summary of the invention

[0006] In order to solve at least one of the above technical problems existing in the prior art, the present invention provides a remote sensing image building extraction method based on edge and topology perception.

[0007] The present invention is implemented by the following technical solution: a remote sensing image building extraction method based on edge and topology perception, comprising:

[0008] A high-resolution remote sensing image building extraction network is constructed, wherein the high-resolution extraction network comprises at least a two-branch encoder, a bottleneck module and a decoder, wherein the two-branch encoder comprises an edge branch and a semantic branch, wherein the edge branch is used to extract and save edge information in an input image, and the semantic branch is used to obtain global and local building features in the input image; wherein the semantic branch is sequentially composed of a 3×3 deep convolutional layer 1 and a downsampling unit 1, a downsampling unit 2 and a downsampling unit 3; wherein the downsampling unit 1, the downsampling unit 2 and the downsampling unit 3 are each composed of a large window Mamba module, an edge-semantic fusion module and a 2×2 maximum pooling layer; wherein the large window Mamba module is used to extract and save edge information in an input image, and wherein the semantic branch is used to obtain global and local building features in an ... The invention is used to extract global and local features at the same scale at the same time, the edge-semantic fusion module is used to enhance the edge features of the building edge area in the input image, and the 2×2 maximum pooling layer is used to downsample the feature map after the edge feature enhancement to obtain the feature map of the target size; the bottleneck module is composed of a large window type Mamba module; the decoder includes an upsampling unit, a transposed convolution module and a segmentation head, the upsampling unit includes a channel splicing module, a dynamic graph convolution module and a transposed convolution module, the dynamic graph convolution module is used to obtain the global topological features at different scales, and the transposed convolution module is used for upsampling; the two-branch encoder and the decoder are connected through a coordinate attention module;

[0009] Training the high-resolution remote sensing image building extraction network to obtain a trained high-resolution remote sensing image building extraction network;

[0010] The image of the building to be tested is input into the trained high-resolution remote sensing image building extraction network to obtain the building extraction result in the high-resolution remote sensing image.

[0011] Preferably, the edge branch includes edge extraction unit 1, edge extraction unit 2, edge extraction unit 3 and edge head;

[0012] The edge extraction unit 1, the edge extraction unit 2 and the edge extraction unit 3 are each composed of a 3×3 convolution layer and a semantically guided edge enhancement module; the edge feature map of the input image is extracted by the 3×3 convolution layer, and the edge feature map is then input into the semantically guided edge enhancement module to obtain a refined building edge feature map;

[0013] The edge head is composed of a first batch of normalization layers, a first ReLU activation function layer, a 3×3 convolution layer, a second batch of normalization layers, a second ReLU activation function layer, a 1×1 convolution layer and a Sigmoid activation function in sequence, and performs feature extraction on the refined building edge feature map to obtain edge information in the input image.

[0014] Preferably, the large-window Mamba module includes a pixel-level state space sequence module and a patch-level state space sequence module, wherein the pixel-level state space sequence module is used to extract local building features, and the patch-level state space sequence module is used to extract global building features;

[0015] The pixel-level state space sequence module divides the feature map of the input image into windows, and inputs the sub-windows obtained by the division into the omnidirectional state space module in the pixel-level state space sequence module to obtain local building features;

[0016] The patch-level state space sequence module pools the output of the omnidirectional state space module in the pixel-level state space sequence module, and then inputs the pooling result into the omnidirectional state space module of the patch-level state space sequence module, and then transmits the output of the omnidirectional state space module of the patch-level state space sequence module to the m×n inverse average pooling layer, and finally adds the output of the omnidirectional state space module in the pixel-level state space sequence module to the corresponding elements of the output of the m×n inverse average pooling layer to obtain the global building feature.

[0017] Preferably, the omnidirectional state space unit comprises a normalization layer, a linear transformation layer 1, a 3×3 depth convolution layer 2, an omnidirectional selection scanning module, a linear transformation layer 2, a linear transformation layer 3 and a linear transformation layer 4;

[0018] First, the input data is standardized through the standardization layer, and then transmitted to the linear transformation layer 1 for the first channel dimension transformation. The spatial features after the first channel dimension transformation are extracted by the 3×3 deep convolution layer 2. The extracted spatial features are then input into the omnidirectional selection scanning module and the linear transformation layer 2 in sequence. The output is multiplied with the corresponding pixel output by the linear transformation layer 4. After the product passes through the linear transformation layer 3, it is added to the input of the standardization layer through the residual connection to obtain the output result of the omnidirectional state space unit.

[0019] Preferably, the specific calculation of the dynamic graph convolution module is as follows:

[0020] Z = ReLU(AF'Θ)

[0021] In the formula, is the result of the dynamic graph convolution module, is the parameter matrix, is the adjacency matrix, is the graph node matrix, is a set of real numbers, H, W, C are the height, width and number of channels of the feature map respectively, and F is the feature map.

[0022] Preferably, the coordinate attention module structure is:

[0023] First, two one-dimensional global pooling operations are used to perform global pooling along the width and height directions of the input features respectively to obtain feature maps in the width direction and feature maps in the height direction. The feature maps in the width direction and the feature maps in the height direction are then subjected to channel splicing and convolution operations to obtain channel-compressed feature maps. The channel-compressed feature maps are then subjected to batch normalization operations and ReLU activation functions, and the channel-compressed feature maps are re-split into feature maps in the width and height directions. The split feature maps in the width and height directions are respectively subjected to a second convolution and Sigmoid activation function to obtain dimensionally restored width feature vectors and height feature vectors. Finally, the dimensionally restored width feature vectors and height feature vectors are weighted in the width and height directions with the input features of the coordinate attention module to generate the final coordinate attention.

[0024] Preferably, training the high-resolution remote sensing image building extraction network comprises:

[0025] The He Uniform method is used to initialize the high-resolution remote sensing image building extraction network, and then the parameters in the model are updated, and the Adam optimization function is used to optimize the loss function until the high-resolution remote sensing image building extraction network converges.

[0026] Preferably, the loss function is:

[0027] L total =20l 1 +l 2 +l 3

[0028] Where, L total is the loss function, l 1 is the weighted binary cross entropy loss function, l 2 is the cross entropy loss function, l 3 is the edge-aware loss function; l 1 belongs to the edge branch loss function, l 2 and l 3 Belongs to the semantic branch loss function.

[0029] Preferably, the image of the building to be tested is input into a trained high-resolution remote sensing image building extraction network, comprising:

[0030] The image of the building to be tested is preprocessed, and the preprocessing includes manual image annotation, image cropping, data enhancement and real building edge extraction.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] The present invention uses multiple edge extraction units and edge heads in edge branches, so that the constructed high-resolution remote sensing image building extraction network can learn more structural features, promote the precise positioning of the edges of the buildings to be tested, and adopt a large-window Mamba module in the semantic branch and the bottleneck module, which can not only simultaneously obtain the local and all building features of the building image to be tested, but also effectively reduce the amount of calculation, and the omnidirectional state space module is adopted in the large-window Mamba module to model the global context of the building in various spatial directions, which is suitable for the characteristics that the buildings in the remote sensing image may be distributed in various spatial directions, thereby improving the model performance; the dynamic graph convolution module used in the upsampling unit in the decoder of the present invention can still effectively identify the building to be tested when the building image to be tested is blocked or other appearance clues are poor; the coordinate attention module is adopted in the jump connection of the present invention, which is conducive to selecting appropriate channel information for fusion and avoiding interference and spatial structure information loss, so as to better improve the building extraction accuracy of the model. The high-resolution remote sensing image building extraction network constructed by the present invention comprehensively solves the problems existing in the existing high-resolution remote sensing image building extraction methods, such as the inability to accurately perceive the overall topological structure and edge details of the building, the large amount of calculation in obtaining global context information, and susceptibility to interference from similar-looking objects, shadows, and occlusions. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 A schematic diagram of the overall structure of a high-resolution remote sensing image building extraction network constructed in the method of the present invention;

[0035] Figure 2 It is a semantically guided edge enhancement module in the high-resolution remote sensing image building extraction network constructed in the method of the present invention;

[0036] Figure 3 A schematic diagram of the structure of a large-window Mamba module in a high-resolution remote sensing image building extraction network constructed in the method of the present invention;

[0037] Figure 4 It is a schematic diagram of the structure of the omnidirectional state space unit constructed in the method of the present invention;

[0038] Figure 5 A schematic diagram of the composition structure of the edge-semantic fusion module constructed by the method of the present invention;

[0039] Figure 6 A schematic diagram of the composition structure of the coordinate attention module in the high-resolution remote sensing image building extraction network constructed by the method of the present invention;

[0040] Figure 7 Schematic diagram of the structural composition of the dynamic graph convolution module in the high-resolution remote sensing image building extraction network constructed by the method of the present invention. DETAILED DESCRIPTION

[0041] In conjunction with the drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0042] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification so that people familiar with this technology can understand and read them. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantive technical significance. Any structural modification, change in proportional relationship or adjustment of size should fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention. It should be noted that in this specification, relational terms such as first and second are only used to distinguish one entity from several other entities, and do not necessarily require or imply any actual relationship or order between these entities.

[0043] In recent years, deep learning technology has brought revolutionary impacts to the field of image segmentation. In the past decade, methods based on convolutional neural networks have developed rapidly in remote sensing building image extraction. In general, the existing technology has been able to achieve automatic extraction of buildings through remote sensing image building extraction models based on deep learning, and the effects in all aspects are better than traditional extraction methods. However, different building bodies have the characteristics of complex shapes, rich edge information, and different sizes. The existing technology still has many defects in practical applications. Therefore, the present invention provides a high-resolution remote sensing image building extraction network, which can comprehensively solve the problems existing in the existing high-resolution remote sensing image building extraction methods, such as the inability to accurately perceive the overall topological structure and edge details of the building, the large amount of calculation in the acquisition of global context information, and the susceptibility to interference from similar-looking objects, shadows, and occlusions.

[0044] like Figure 1 As shown, an embodiment of the present invention provides a flow chart of a method for extracting buildings from remote sensing images based on edge and topology perception, comprising the following steps:

[0045] Step 1: Build a high-resolution remote sensing image building extraction network, the high-resolution extraction network includes at least a two-branch encoder, a bottleneck module and a decoder, the two-branch encoder includes an edge branch and a semantic branch, the edge branch is used to extract and save the edge information in the input image, and the semantic branch is used to obtain the global and local building features in the input image; the semantic branch is composed of a 3×3 deep convolution layer 1 and downsampling unit 1, downsampling unit 2, and downsampling unit 3 in sequence; the downsampling unit 1, downsampling unit 2, and downsampling unit 3 are each composed of a large window Mamba module, an edge-semantic fusion module and a 2×2 maximum pooling layer; the large window Mamba module The block is used to simultaneously extract global and local features at the same scale, the edge-semantic fusion module is used to enhance the edge features of the building edge area in the input image, and the 2×2 maximum pooling layer is used to downsample the feature map after the edge feature enhancement to obtain a feature map of the target size; the bottleneck module is composed of a large-window Mamba module; the decoder includes an upsampling unit, a transposed convolution one and a segmentation head, the upsampling unit includes a channel splicing module, a dynamic graph convolution module and a transposed convolution two, the dynamic graph convolution module is used to obtain global topological features at different scales, and the transposed convolution two is used for upsampling; the two-branch encoder and the decoder are connected through a coordinate attention module.

[0046] In this embodiment, the first transposed convolution and the second transposed convolution are both 3×3 transposed convolutions with a step size of 2.

[0047] Optionally, the edge branch includes an edge extraction unit 1, an edge extraction unit 2, an edge extraction unit 3 and an edge head; the edge extraction unit 1, the edge extraction unit 2 and the edge extraction unit 3 are each composed of a 3×3 convolution layer and a semantically guided edge enhancement module; the edge feature map of the input image is extracted through the 3×3 convolution layer, and the edge feature map is then input into the semantically guided edge enhancement module to obtain a refined building edge feature map; the edge head is composed of a first batch of normalization layers, a first ReLU activation function layer, a 3×3 convolution layer, a second batch of normalization layers, a second ReLU activation function layer, a 1×1 convolution layer and a Sigmoid activation function in sequence, and feature extraction is performed on the refined building edge feature map to obtain edge information in the input image.

[0048] In this embodiment, the edge head and the segmentation head are composed of the first batch of normalization layers, the first ReLU activation function layer, the 3×3 convolution layer, the second batch of normalization layers, the second ReLU activation function layer, the 1×1 convolution layer and the Sigmoid activation function in sequence, and feature extraction is performed on the refined building edge feature map to obtain the edge information in the input image.

[0049] In this embodiment, Figure 2 As shown in the figure, the semantic-guided edge enhancement module inputs the edge features extracted by the convolution layer into the Sigmoid activation function to obtain the weight coefficients indicating the importance of each part of the edge feature map, and then subtracts these weight coefficients from one to obtain the inverse weight coefficients, and then multiplies the feature map obtained by the semantic branch with the inverse weight coefficients. Finally, the obtained product is concatenated with the edge features extracted by the convolution layer in the channel dimension and the number of channels is adjusted by 1×1 convolution.

[0050] In this embodiment, the step size of the 3×3 deep convolution layer is 2, and the 2×2 maximum pooling layer downsamples the feature map size to half of the original size.

[0051] In this embodiment, Figure 5 As shown in the figure, the edge-semantic fusion module inputs the semantic features of the feature map into the Sigmoid activation function to obtain the weight coefficients of the importance of each part of the semantic feature map, and then subtracts these weight coefficients from one to obtain the inverse weight coefficients, and then multiplies the edge feature map obtained by the edge branch with the inverse weight coefficients. Finally, the product is concatenated with the semantic features in the channel dimension and the number of channels is adjusted using the 1×1 convolution in the edge-semantic fusion module.

[0052] Optionally, the large-window Mamba module includes a pixel-level state space sequence module and a patch-level state space sequence module, the pixel-level state space sequence module is used to extract local building features, and the patch-level state space sequence module is used to extract global building features; the pixel-level state space sequence module divides the feature map of the input image into windows, and inputs the divided sub-windows into the omnidirectional state space module in the pixel-level state space sequence module to obtain local building features; the patch-level state space sequence module pools the output of the omnidirectional state space module in the pixel-level state space sequence module, and then inputs the pooling result into the omnidirectional state space module of the patch-level state space sequence module, and then transmits the output of the omnidirectional state space module of the patch-level state space sequence module to the m×n inverse average pooling layer, and finally adds the output of the omnidirectional state space module in the pixel-level state space sequence module to the corresponding elements of the output of the m×n inverse average pooling layer to obtain global building features.

[0053] In this embodiment, Figure 3As shown in the figure, the large-window Mamba module is composed of a pixel-level state space sequence module and a patch-level state space sequence module in sequence. After the feature map of the building to be tested is input, the pixel-level state space sequence module and the patch-level state space sequence module extract local and global building features respectively; the pixel-level state space sequence module first divides the input feature map of size H×W into (HW) / (mn) sub-windows of size m×n, and then inputs these sub-windows into the omnidirectional state space module; then the patch-level state space sequence module inputs the feature map of size H×W into the average pooling layer of size m×n, and then inputs the result after average pooling into the omnidirectional state space module, and then transmits the output of the omnidirectional state space module of the patch-level state space sequence module to the m×n inverse average pooling layer, and finally adds the output of the omnidirectional state space module in the pixel-level state space sequence module and the corresponding elements of the output of the m×n inverse average pooling layer through residual connection to obtain the global building feature.

[0054] Optionally, the omnidirectional state space unit includes a normalization layer, a linear transformation layer one, a 3×3 deep convolution layer two, an omnidirectional selective scanning module, a linear transformation layer two, a linear transformation layer three and a linear transformation layer four; the input data is first normalized by the normalization layer, and then transmitted to the linear transformation layer one for the first channel dimension transformation, and then the spatial features after the first channel dimension transformation are extracted by the 3×3 deep convolution layer two, and then the extracted spatial features are sequentially input into the omnidirectional selective scanning module and the linear transformation layer two, and the output thereof is multiplied with the corresponding pixel output by the linear transformation layer four, and the product is added to the input of the normalization layer through a residual connection after passing through the linear transformation layer three to obtain the output result of the omnidirectional state space unit.

[0055] In this embodiment, Figure 4 As shown in the figure, the input data is first normalized by the standardization layer, and then transmitted to the linear transformation layer 1 for the first channel dimension transformation. The spatial features after the first channel dimension transformation are extracted by the 3×3 deep convolution layer 2 with fewer parameters. The obtained spatial features are sequentially input into the omnidirectional selection scanning module and the linear transformation layer 2, and the output is multiplied with the corresponding pixel output by the linear transformation layer 4. After the product passes through the linear transformation layer 3, it is added to the input of the standardization layer through the residual connection to obtain the output result of the omnidirectional state space unit.

[0056] In this embodiment, the omnidirectional selective scanning module is used to selectively scan the input features in the forward and reverse directions along the horizontal, vertical, diagonal and anti-diagonal directions, and then add all the scanning results.

[0057] Optionally, the structure of the coordinate attention module is: first, two one-dimensional global pooling operations are used to perform global pooling along the width and height directions of the input features respectively to obtain feature maps corresponding to the width direction and the height direction, and then the feature maps in the width direction and the height direction are subjected to channel splicing and convolution operations to obtain channel-compressed feature maps, and then the channel-compressed feature maps are subjected to batch normalization operations and ReLU activation functions, and then the channel-compressed feature maps are split into feature maps in the width and height directions again, and the split feature maps in the width and height directions are respectively subjected to a second convolution and Sigmoid activation function to obtain a width feature vector and a height feature vector after dimensionality restoration, and finally the width feature vector and the height feature vector after dimensionality restoration are weighted in the width and height directions with the input features of the coordinate attention module to generate the final coordinate attention.

[0058] In this embodiment, Figure 6 As shown, firstly, two one-dimensional global pooling operations are used to perform global pooling along the width and height directions of the input features respectively, and two directional perception feature maps with dimensions of C×H×1 and C×1×W are obtained. Then, the two directional perception feature maps are spliced ​​in the spatial dimension, and then the number of channels is compressed by 1×1 convolution to obtain a feature map with a dimension of (r / C)×1×(W+H) (r=2 in this embodiment). After the feature map passes through batch normalization operation and ReLU activation function in turn, the feature map is split into feature vectors in two directions again. After the splitting, the feature map in one direction is restored to C×H×1 by 1×1 convolution of H×1×C and Sigmoid activation function in turn, and the feature map in the other direction is restored to C×1×W by 1×1 convolution of 1×W×C and Sigmoid activation function in turn. Finally, the two feature vectors are weighted in the width and height directions with the input feature map of the coordinate attention module; the final coordinate attention is generated. The coordinate attention alleviates the loss of spatial topological information while filtering irrelevant features of the building.

[0059] Optionally, the specific calculation of the dynamic graph convolution module is as follows:

[0060] Z = ReLU(AF'Θ)

[0061] In the formula, is the result of the dynamic graph convolution module, is the parameter matrix, is the adjacency matrix, is the graph node matrix, is a set of real numbers, H, W, C are the height, width and number of channels of the feature map respectively, and F is the feature map.

[0062] In this embodiment, Figure 7 As shown, the graph node matrix is ​​fed into the dynamic graph convolution module as input feature F H×W×C The input transformation matrix is ​​obtained; the calculation of the adjacency matrix is ​​first performed by inputting the dynamic graph convolution module into the feature F H×W×C The input 1×1 convolution operation reduces the channel dimension to M=C / 2, and then a ReLU activation function is used for data processing to improve the fitting ability of the feature data after the channel reduction, and then two deformation functions are used to convert it into F HW×M and F M×HW ; Another way of calculation is to input the dynamic graph convolution module into the feature F H×W×C The global average pooling and 1×M 1×1 convolution operations are performed in sequence, and then the Sigmoid activation function is used for processing, and the diagonalization operation is performed at the same time to obtain the diagonalized result φ M×M ; Then F HW×M ,φ M×M and F M×HW Multiply them one by one according to matrix multiplication, and input the obtained product into the Softmax activation function to obtain the normalized proximity matrix; finally, the output of the proximity matrix and the input features of the dynamic graph convolution module are transformed by the deformation function to obtain the F C×HW Perform matrix multiplication, and the result is transformed into F through the deformation function C×H×W , and obtain the output features of the dynamic graph convolution module.

[0063] Step 2: Training the high-resolution remote sensing image building extraction network to obtain a trained high-resolution remote sensing image building extraction network.

[0064] Optionally, training the high-resolution remote sensing image building extraction network includes: initializing the high-resolution remote sensing image building extraction network using the HeUniform method, then updating the parameters in the model, and optimizing the loss function using the Adam optimization function until the high-resolution remote sensing image building extraction network converges.

[0065] In this embodiment, a historical high-resolution remote sensing dataset is obtained, and the historical high-resolution remote sensing dataset is divided into a training set and a test set, and the images in the training set and the test set are preprocessed. Then, the preprocessed training set is used to train a high-resolution remote sensing image building extraction network until the parameters in the high-resolution remote sensing image building extraction network converge. Finally, the preprocessed test set is used to perform model testing on the trained high-resolution remote sensing image building extraction network with edge heads removed, and the building extraction result of the historical high-resolution remote sensing dataset is output.

[0066] Optionally, the loss function is:

[0067] Ltotal =20l 1 +l 2 +l 3

[0068] Where, L total is the loss function, l 1 is the weighted binary cross entropy loss function, l 2 is the cross entropy loss function, l 3 is the edge-aware loss function; l 1 belongs to the edge branch loss function, l 2 and l 3 Belongs to the semantic branch loss function.

[0069] In this embodiment, l 1 It belongs to the edge branch loss function and adopts the weighted binary cross entropy loss function. The specific calculation is as follows:

[0070]

[0071] Among them, s i is the true value of the corresponding pixel, is the prediction result of the corresponding pixel, and β is the weight coefficient.

[0072] l 2 and l 3 It belongs to the semantic branch loss function, which uses the cross entropy loss function and the edge perception loss function respectively. The specific calculation is as follows:

[0073]

[0074] Where t is a predefined threshold (t=0.8), b i is the edge head output of the ith pixel, s i,c and are the true value and predicted result of the i-th pixel of category c respectively.

[0075] Step 3: Input the image of the building to be tested into the trained high-resolution remote sensing image building extraction network to obtain the building extraction result in the high-resolution remote sensing image.

[0076] Optionally, inputting the image of the building to be tested into a trained high-resolution remote sensing image building extraction network includes: preprocessing the image of the building to be tested, wherein the preprocessing includes manual image annotation, image cropping, data enhancement and extraction of real building edges.

[0077] In this embodiment, the manual image annotation specifically includes: manually annotating the buildings in the high-resolution remote sensing image at the pixel level in ArcGIS software to obtain a labeled high-resolution remote sensing image; the image cropping specifically includes: randomly cropping the labeled high-resolution remote sensing image into a sub-image of 512 pixels × 512 pixels; the data enhancement includes: performing random scale transformation, random angle image rotation, and vertical and horizontal flipping of the sub-image to obtain a high-resolution remote sensing image; the real edge extraction of the building first uses the Canny edge detection algorithm to extract the building edge from the label image, and then uses the expansion process to connect the breakpoints of the obtained building edge map to obtain a smooth and continuous real edge map of the building.

[0078] The present invention introduces a two-branch encoder, a bottleneck module and a decoder based on dynamic graph convolution in the semantic segmentation network. The additional edge branches are used to guide the feature learning of buildings, so that the model learns more structural features and promotes the precise positioning of the building edges. The large-window Mamba module can not only simultaneously obtain global and local building features, but also effectively reduce the amount of calculation. The dynamic graph convolution module can effectively identify building targets when the model building is blocked or other appearance clues are poor. The omnidirectional state space module is used in the large-window Mamba module to model the global context of the building in various spatial directions, which is suitable for the characteristics that buildings in remote sensing images may be distributed in various spatial directions, thereby improving the accuracy of the image segmentation. The model performance is improved; the edge branch adopts a semantically guided edge enhancement module to obtain edge features unique to buildings; the semantic branch adopts an edge-semantic fusion module to extract edge-enhanced building feature maps; a coordinate attention module is adopted in the jump connection, and this design is conducive to selecting appropriate channel information for fusion and avoiding interference and loss of spatial structure information, so as to better improve the building extraction accuracy of the model. The present invention constructs a high-resolution remote sensing image building extraction network, and comprehensively solves the problems existing in the existing high-resolution remote sensing image building extraction methods, such as the inability to accurately perceive the overall topological structure and edge details of the building, the large amount of calculation in the acquisition of global context information, and the susceptibility to interference from similar-looking objects, shadows, and occlusions.

[0079] What needs to be explained about the specific structure of the present invention is that the connection relationship between the various component modules adopted in the present invention is definite and feasible. Except for the special instructions in the embodiments, the specific connection relationship can bring about the corresponding technical effects, and solve the technical problems raised by the present invention without relying on the execution of the corresponding software program. Except for the specific instructions, the models and connection methods of the components, modules and specific components appearing in the present invention belong to the existing technologies such as published patents, published journal articles, or common knowledge that can be obtained by technical personnel in this field before the application date, and there is no need to elaborate, so that the technical solution provided in this case is clear, complete and feasible, and the corresponding physical products can be reproduced or obtained according to the technical means.

[0080] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A remote sensing image building extraction method based on edge and topology perception, characterized in that: include: A high-resolution remote sensing image building extraction network is constructed, wherein the high-resolution extraction network comprises at least a two-branch encoder, a bottleneck module and a decoder, wherein the two-branch encoder comprises an edge branch and a semantic branch, wherein the edge branch is used to extract and save edge information in an input image, and the semantic branch is used to obtain global and local building features in the input image; wherein the semantic branch is sequentially composed of a 3×3 deep convolutional layer 1 and a downsampling unit 1, a downsampling unit 2 and a downsampling unit 3; wherein the downsampling unit 1, the downsampling unit 2 and the downsampling unit 3 are each composed of a large window Mamba module, an edge-semantic fusion module and a 2×2 maximum pooling layer; wherein the large window Mamba module is used to extract and save edge information in an input image, and wherein the semantic branch is used to obtain global and local building features in an ... The invention is used to extract global and local features at the same scale at the same time, the edge-semantic fusion module is used to enhance the edge features of the building edge area in the input image, and the 2×2 maximum pooling layer is used to downsample the feature map after the edge feature enhancement to obtain the feature map of the target size; the bottleneck module is composed of a large window type Mamba module; the decoder includes an upsampling unit, a transposed convolution module and a segmentation head, the upsampling unit includes a channel splicing module, a dynamic graph convolution module and a transposed convolution module, the dynamic graph convolution module is used to obtain the global topological features at different scales, and the transposed convolution module is used for upsampling; the two-branch encoder and the decoder are connected through a coordinate attention module; Training the high-resolution remote sensing image building extraction network to obtain a trained high-resolution remote sensing image building extraction network; The image of the building to be tested is input into the trained high-resolution remote sensing image building extraction network to obtain the building extraction result in the high-resolution remote sensing image.

2. The method for extracting buildings from remote sensing images based on edge and topology perception according to claim 1, characterized in that: The edge branch includes an edge extraction unit 1, an edge extraction unit 2, an edge extraction unit 3 and an edge head; The edge extraction unit 1, the edge extraction unit 2 and the edge extraction unit 3 are each composed of a 3×3 convolution layer and a semantically guided edge enhancement module; the edge feature map of the input image is extracted by the 3×3 convolution layer, and the edge feature map is then input into the semantically guided edge enhancement module to obtain a refined building edge feature map; The edge head is composed of a first batch of normalization layers, a first ReLU activation function layer, a 3×3 convolution layer, a second batch of normalization layers, a second ReLU activation function layer, a 1×1 convolution layer and a Sigmoid activation function in sequence, and performs feature extraction on the refined building edge feature map to obtain edge information in the input image.

3. The method for extracting buildings from remote sensing images based on edge and topology perception according to claim 1, characterized in that: The large-window Mamba module includes a pixel-level state space sequence module and a patch-level state space sequence module, wherein the pixel-level state space sequence module is used to extract local building features, and the patch-level state space sequence module is used to extract global building features; The pixel-level state space sequence module divides the feature map of the input image into windows, and inputs the sub-windows obtained by the division into the omnidirectional state space module in the pixel-level state space sequence module to obtain local building features; The patch-level state space sequence module pools the output of the omnidirectional state space module in the pixel-level state space sequence module, and then inputs the pooling result into the omnidirectional state space module of the patch-level state space sequence module, and then transmits the output of the omnidirectional state space module of the patch-level state space sequence module to the m×n inverse average pooling layer, and finally adds the output of the omnidirectional state space module in the pixel-level state space sequence module to the corresponding elements of the output of the m×n inverse average pooling layer to obtain the global building feature.

4. The method for extracting buildings from remote sensing images based on edge and topology perception according to claim 3, characterized in that: The omnidirectional state space unit includes a normalization layer, a linear transformation layer 1, a 3×3 depth convolution layer 2, an omnidirectional selection scanning module, a linear transformation layer 2, a linear transformation layer 3 and a linear transformation layer 4; First, the input data is standardized through the standardization layer, and then transmitted to the linear transformation layer 1 for the first channel dimension transformation. The spatial features after the first channel dimension transformation are extracted by the 3×3 deep convolution layer 2. The extracted spatial features are then input into the omnidirectional selection scanning module and the linear transformation layer 2 in sequence. The output is multiplied with the corresponding pixel output by the linear transformation layer 4. After the product passes through the linear transformation layer 3, it is added to the input of the standardization layer through the residual connection to obtain the output result of the omnidirectional state space unit.

5. The method for extracting buildings from remote sensing images based on edge and topology perception according to claim 1, characterized in that: The specific calculation of the dynamic graph convolution module is as follows: Z = ReLU(AF'Θ) In the formula, is the result of the dynamic graph convolution module, is the parameter matrix, is the adjacency matrix, is the graph node matrix, i is a real number set, H, W, C are the height, width and number of channels of the feature map respectively, and F is the feature map.

6. The method for extracting buildings from remote sensing images based on edge and topology perception according to claim 1, characterized in that: The coordinate attention module structure is: First, two one-dimensional global pooling operations are used to perform global pooling along the width and height directions of the input features respectively to obtain feature maps in the width direction and feature maps in the height direction. The feature maps in the width direction and the feature maps in the height direction are then subjected to channel splicing and convolution operations to obtain channel-compressed feature maps. The channel-compressed feature maps are then subjected to batch normalization operations and ReLU activation functions, and the channel-compressed feature maps are re-split into feature maps in the width and height directions. The split feature maps in the width and height directions are respectively subjected to a second convolution and Sigmoid activation function to obtain dimensionally restored width feature vectors and height feature vectors. Finally, the dimensionally restored width feature vectors and height feature vectors are weighted in the width and height directions with the input features of the coordinate attention module to generate the final coordinate attention.

7. The method for extracting buildings from remote sensing images based on edge and topology perception according to claim 1, characterized in that: The training of the high-resolution remote sensing image building extraction network comprises: The He Uniform method is used to initialize the high-resolution remote sensing image building extraction network, and then the parameters in the model are updated, and the Adam optimization function is used to optimize the loss function until the high-resolution remote sensing image building extraction network converges.

8. The method for extracting buildings from remote sensing images based on edge and topology perception according to claim 7, characterized in that: The loss function is: L total ＝20l1+l2+l3 Where, L total is the loss function, l1 is the weighted binary cross entropy loss function, l2 is the cross entropy loss function, and l3 is the edge-aware loss function; l1 belongs to the edge branch loss function, and l2 and l3 belong to the semantic branch loss function.

9. The method for extracting buildings from remote sensing images based on edge and topology perception according to claim 1, characterized in that: The step of inputting the image of the building to be tested into a trained high-resolution remote sensing image building extraction network comprises: The image of the building to be tested is preprocessed, and the preprocessing includes manual image annotation, image cropping, data enhancement and real building edge extraction.

Images