Building extraction method and system based on cooperative encoder and uncertainty aggregation decoder

Abstract

The invention provides a building extraction method and system based on a cooperative encoder and an uncertainty aggregation decoder, and the method comprises the steps: inputting a preprocessed remote sensing image into a building extraction model, capturing the local and global features of the remote sensing image at a plurality of stages through the cooperative encoder, and obtaining the multi-scale features, inputting the features into a global-local fusion module, and performing feature fusion according to stages to obtain local aggregation features and global aggregation features; respectively obtaining the local uncertainty of the local aggregation feature and the global uncertainty of the global aggregation feature by an uncertainty aggregation decoder; and taking the local uncertainty and the global uncertainty as weights, and integrating the local aggregation features and the global aggregation features to obtain a final feature map so as to extract the building in the remote sensing image. According to the extraction method provided by the invention, the high-quality global-local representation can be constructed, the difference between multi-level features is reduced, the complementary global-local representation is fully utilized for feature integration, and the extraction precision is high.

Description

Technical Field

[0001] The present invention relates to the field of image processing and large model technology, and in particular to a building extraction method and system based on a collaborative encoder and an uncertainty aggregation decoder. Background Art

[0002] Building extraction from remote sensing images aims to distinguish buildings from high-resolution remote sensing images and has been widely used in urban planning, population forecasting, geographic information system (GIS) and other fields. With the rapid development of satellite and aerial remote sensing imaging technology, how to accurately and efficiently extract buildings from aerial images has become a focus of attention in the field of computer vision. Early building extraction algorithms used hand-crafted features for pixel-level classification, such as texture, geometry, and adjacent connections. Although intuitive and simple, hand-crafted features cannot effectively handle the complex structural changes of buildings, resulting in low accuracy and limited generalization ability. With the development of deep learning technology, a series of building extraction methods based on convolutional neural networks (CNNs) have emerged. These methods use local convolution kernels to capture discriminative features through forward-backward propagation, enabling the model to construct a hierarchical semantic representation of foreground buildings through large-scale model training. However, since the CNN-based architecture relies on stacking multiple convolutional blocks with limited receptive fields, it is difficult to utilize long-range information in different regions, resulting in poor performance in complex situations with occlusion and blur. In recent years, Vision Transformer (ViTs) has been applied to building extraction tasks. ViTs-based methods use non-local self-attention to capture long-range dependencies, which can mine informative global context for dense prediction. However, due to the large size differences of buildings in remote sensing images, the transformer has difficulty perceiving local details of multiple scales and shapes, and often produces unclear segmentation outputs. The latest methods also adopt a hybrid CNN and ViT architecture to take advantage of the complementary strengths of CNN and ViT.

[0003] Although existing methods have made some progress in the field of remote sensing image buildings, there are two potential problems with existing technologies: low quality of encoded feature extraction and insufficient multi-scale information aggregation. Features in shallow layers tend to retain more local information but are not sensitive to the global context of buildings. On the contrary, features in deep layers are more conducive to exploiting the overall global connectivity of building instances, but are prone to losing local details. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a building extraction method and system based on a collaborative encoder and an uncertainty aggregation decoder to eliminate or improve one or more defects existing in the prior art.

[0005] In one aspect, the present invention provides a building extraction method based on a collaborative encoder and an uncertainty aggregation decoder, the method comprising the following steps:

[0006] Acquire a remote sensing image, wherein the remote sensing image contains a building to be extracted;

[0007] Inputting the remote sensing image into a pre-trained building extraction model to extract buildings from the remote sensing image;

[0008] The building extraction model includes a collaborative encoder, a global-local fusion module and an uncertainty aggregation decoder connected in sequence; the remote sensing image is input into the collaborative encoder to capture the local characteristics of the remote sensing image in the early stage and the global features of the remote sensing image in the final stage; the multi-scale features extracted by the collaborative encoder are respectively input into the global-local fusion module for feature fusion in stages to obtain local aggregation features and global aggregation features; the local aggregation features and the global aggregation features are respectively input into the uncertainty aggregation decoder to obtain the local uncertainty of the local aggregation features and the global uncertainty of the global aggregation features, the local uncertainty and the global uncertainty are mapped to attenuation weights, and based on the attenuation weights, the local aggregation features and the global aggregation features are integrated to obtain a final feature map to extract the building.

[0009] In some embodiments of the present invention, before the remote sensing image is input into the pre-trained building extraction model, the remote sensing image is pre-processed, including: scaling the remote sensing image to a preset size and performing normalization processing.

[0010] In some embodiments of the present invention, the collaborative encoder includes four stages, wherein the first stage and the second stage are the early stages, the third stage is the intermediate stage, and the fourth stage is the terminal stage; in the early stage, a multi-core convolution modulation block is used to construct the local features; in the intermediate stage, an intermediate cooperative interaction block is used to perform global and local feature interaction; in the terminal stage, a multi-head self-attention block is used to enhance the global features.

[0011] In some embodiments of the present invention, when the collaborative encoder is used to extract multi-scale features of the remote sensing image, the method includes:

[0012] Flattening and linearly projecting the remote sensing image into a one-dimensional vector, and reshaping to obtain an embedded feature map;

[0013] Splitting the embedded feature map in the channel dimension to obtain multiple feature groups;

[0014] In the early stage, for each feature group, depth-wise separable convolution with different kernel sizes is used to capture local information, and point convolution is performed to combine to obtain the combined feature, which is calculated as:

[0015] Z ′ =Cat(DW 3×3 (Z 1 ),…,DW k×k (Z n ));

[0016] M=W p *Z ′ ;

[0017] Among them, Z ′ Indicates the local information; DW k×k represents a depth-wise separable convolution with a kernel size of k×k; Z n represents the nth feature group; M represents the combined feature; W p represents the point convolution;

[0018] The multi-core convolution modulation block is used to enhance the combined features, and the calculation formula is:

[0019]

[0020] Among them, MKFAM(·) represents the multi-core convolution operation; F i represents the output features of the i-th stage;

[0021] In the intermediate stage, after the input features of the intermediate cooperative interaction block are normalized, they are input into the multi-core convolution modulation block with residual connection, and the calculation formula is:

[0022]

[0023] In the final stage, the features obtained in the intermediate stage are input into the multi-head self-attention block to simulate the global dependency relationship. The calculation formula is:

[0024]

[0025] Among them, X l represents the input features of the lth intermediate cooperative interaction block; FFN(·) represents the operation of the multi-layer perceptron; MHSA(·) represents the operation of the multi-head self-attention block.

[0026] In some embodiments of the present invention, the multi-scale features extracted by the collaborative encoder are respectively input into the global-local fusion module for feature fusion in stages, further comprising:

[0027] The multi-scale features are input into the deep convolution block with residual connection to establish the information propagation path, and the calculation formula is:

[0028]

[0029] in, Indicates F i The information propagation path of F i represents the output features of the i-th stage of the collaborative encoder; DWConv(·) represents the operation of the depth convolution block.

[0030] In some embodiments of the present invention, obtaining local aggregate features and global aggregate features includes:

[0031] The output features of the early stage and the mid-stage are fused to obtain the local aggregated features, and the calculation formula is:

[0032]

[0033] The output features of the mid-term stage and the terminal stage are fused to obtain the global aggregate feature, which is calculated as follows:

[0034]

[0035] Among them, F L represents the local aggregation feature; F G represents the global aggregation feature; Indicates F i The information propagation path of F i represents the output features of the i-th stage of the collaborative encoder; UpConv(·) represents the up-convolution operation.

[0036] In some embodiments of the present invention, the local aggregate feature and the global aggregate feature are respectively input into the uncertainty aggregation decoder to obtain the local uncertainty and the global uncertainty, including:

[0037] For any pixel in the local aggregate feature or the global aggregate feature, a Gaussian distribution model is established; and reparameterization technology is used to implement uncertainty learning to obtain the local uncertainty and the global uncertainty.

[0038] In some embodiments of the present invention, based on the attenuation weight, the local aggregated features and the global aggregated features are integrated to obtain a final feature map, and the calculation formula is:

[0039] F out =(1-U G )×F G +(1-U L )×F L ;

[0040] Among them, F out represents the final feature map; U G represents the global uncertainty; U L represents local uncertainty; F G represents the global aggregation feature; F L represents the local aggregation feature.

[0041] On the other hand, the present invention also provides a building extraction system based on a collaborative encoder and an uncertainty aggregation decoder, wherein the system, when executed, implements the steps of any of the methods mentioned above, and the system comprises:

[0042] An image processing module, used for acquiring remote sensing images and preprocessing the remote sensing images;

[0043] The building extraction module inputs the remote sensing image into a pre-trained building extraction model to extract buildings from the remote sensing image.

[0044] On the other hand, the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which implements the steps of any of the methods mentioned above when executed by a processor.

[0045] The present invention provides a building extraction method and system based on a collaborative encoder and an uncertainty aggregation decoder, comprising: inputting a preprocessed remote sensing image into a building extraction model to extract the building. The building extraction model provided by the present invention comprises a collaborative encoder, a global-local fusion module and an uncertainty aggregation decoder connected in sequence. In the collaborative encoder part, unlike the traditional hybrid structure, convolutional neural networks and visual transformers are stacked in parallel or sequentially, but convolutional neural networks are used to capture local information in the early stage, and visual transformers are used to capture global information in the final stage. Subsequently, an intermediate cooperative interaction block based on a hybrid layer is introduced, and the gap between local semantics and global semantics is narrowed by using non-local operators to cooperatively merge multi-scale local features, thereby improving the extraction quality of multi-scale features and solving the problem of multi-scale information loss caused by simple stacking in the prior art. Afterwards, a global-local fusion module is used to complementarily fuse hierarchical visual representations to avoid the lack of semantic information caused by serial connection in the prior art. Finally, in order to reduce the uncertainty of segmentation, an uncertainty aggregation decoder is used to identify difficult pixels that are difficult to identify, thereby forcing the model to focus on fuzzy areas to improve the accuracy of segmentation, providing a more robust solution for building extraction in complex scenes.

[0046] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0047] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0049] Figure 1 Schematic diagram of the steps of a building extraction method based on a collaborative encoder and an uncertainty aggregation decoder in one embodiment of the present invention.

[0050] Figure 2 It is a flowchart of a building extraction method based on a collaborative encoder and an uncertainty aggregation decoder in one embodiment of the present invention.

[0051] Figure 3 Schematic diagram of the modulation process of a multi-scale feature map in one embodiment of the present invention.

[0052] Figure 4 It is a schematic diagram of stacking multiple intermediate cooperative interaction blocks to perform global-local information interaction in one embodiment of the present invention.

[0053] Figure 5 This is a comparison diagram of the effects of the building extraction method of the present application and other existing extraction methods in one embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0055] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0056] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0057] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0058] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0059] In order to solve the problems of low quality of coding feature extraction and insufficient multi-scale information aggregation in the prior art, the present invention provides a building extraction method based on a collaborative encoder and an uncertainty aggregation decoder, such as Figure 1 As shown, the method includes the following steps S101-S102:

[0060] Step S101: Acquire a remote sensing image, which contains a building to be extracted.

[0061] Step S102: inputting the remote sensing image into the pre-trained building extraction model to extract buildings from the remote sensing image.

[0062] like Figure 2As shown in FIG, the building extraction model includes a collaborative encoder, a global-local fusion module, and an uncertainty aggregation decoder connected in sequence. The remote sensing image is input into the collaborative encoder to capture the local characteristics of the remote sensing image in the early stage and the global features of the remote sensing image in the final stage; the multi-scale features extracted by the collaborative encoder are respectively input into the global-local fusion module for feature fusion in stages to obtain local aggregate features and global aggregate features; the local aggregate features and the global aggregate features are respectively input into the uncertainty aggregation decoder to obtain the local uncertainty of the local aggregate features and the global uncertainty of the global aggregate features, and the local uncertainty and the global uncertainty are mapped to attenuation weights. Based on the attenuation weights, the local aggregate features and the global aggregate features are integrated to obtain the final feature map to extract the building.

[0063] In step S101, a remote sensing image is obtained. A remote sensing image is a data image of the ground or atmosphere collected by remote sensing technology. Remote sensing technology mainly relies on various sensors and satellite / aerospace platforms to collect information such as reflection and radiation of the surface and generate images. These images are usually used in many fields such as geographic information system (GIS) analysis, environmental monitoring, agricultural management, urban planning, etc.

[0064] In some embodiments, a preprocessing operation is performed on the remote sensing image, the remote sensing image is scaled to a preset size, such as 512*512, and then the image is normalized so that the data distribution of the image is adapted to the pre-trained parameters of the backbone network of the building extraction model.

[0065] In step S102, the remote sensing image is input into the pre-trained building extraction model to extract buildings from the remote sensing image. The building extraction model includes a collaborative encoder, a global-local fusion module and an uncertainty aggregation decoder connected in sequence. The three modules are further described below.

[0066] The cooperative encoder (CE) uses its hybrid convolution and self-attention operators to capture local and global features of remote sensing images. In the early stage, the cooperative encoder adopts layered convolution blocks to build local representations of buildings, in the middle stage, cooperative interaction blocks are introduced to perform global-local information interaction, and in the terminal stage, pure transformers are used to enhance the global context.

[0067] In some embodiments, the collaborative encoder includes four stages, wherein the first and second stages are early stages, the third stage is an intermediate stage, and the fourth stage is a final stage. In the early stage, a multi-kernel convolutional modulation block (MKFM) is used to construct local features; in the intermediate stage, an intermediate cooperative interaction block (CIB) is used to perform global and local feature interaction; in the final stage, a multi-head self-attention block (MHSA) is used to enhance global features.

[0068] Based on the above description, we can know that the remote sensing image (denoted as I) is input into the collaborative encoder to capture the global and local features of the remote sensing image at different stages and generate a multi-scale feature map F i , F i represents the output features of the i-th stage of the collaborative encoder.

[0069] Specifically: In the first two stages (early stages), convolutional neural networks with different convolution kernels are used to capture local information. First, the remote sensing image I is flattened and linearly projected into a one-dimensional vector, and the embedded feature map Z is reshaped. Then the embedded feature map Z is split in the channel dimension to obtain multiple feature groups, denoted as Z = Z 1 ,Z 2 ,…,Z n For each feature group, depthwise separable convolution with different kernel sizes is used to capture local information, and pointwise convolution is performed to combine them to obtain combined features. The calculation formula is shown in formula (1) and formula (2):

[0070] Z ′ =Cat(DW 3×3 (Z 1 ),…,DW k×k (Z n )); (1)

[0071] M=W p *Z ′ ; (2)

[0072] Among them, Z ′ Indicates the captured local information; DW k×k represents a depth-wise separable convolution with a kernel size of k×k; Z n represents the nth feature group; M represents the combined feature; W p Represents point convolution.

[0073] like Figure 3 As shown in the figure, the combined feature M can be regarded as a multi-core convolutional modulation block (MKFM), which enhances the local representation through the convolution of the multi-core structure. i The modulation process can be expressed as formula (3):

[0074]

[0075] Among them, MKFA(·) represents the multi-core convolution operation; F i Represents the output features of the i-th stage.

[0076] By stacking multiple multi-kernel convolutional modulation blocks in the first two stages, diverse multi-scale local features can be captured with only a small computational cost.

[0077] Then, the features learned in the deep stage tend to focus on the discriminative global context. In order to narrow the gap between local and global features, the present invention uses an intermediate cooperative interaction block in the next stage (the third stage). Figure 4 As shown, in the third stage, multiple intermediate cooperative interaction blocks are stacked to perform global-local information interaction.

[0078] For the lth intermediate cooperative interaction block in the third stage, the input feature X l-1 First, it is normalized and then input into the multi-core convolution modulation block with residual connection, as shown in formula (4) and formula (5):

[0079]

[0080] Among them, X l represents the input feature of the lth intermediate cooperative interaction block; MKFA(·) represents the multi-core convolution operation; FFN(·) represents the operation of the multi-layer perceptron; * represents the intermediate quantity.

[0081] Then X l It is sent to the standard multi-head self-attention block to simulate the global dependency, as shown in formulas (6) and (7):

[0082]

[0083] Among them, MHSA(·) is the operation of the multi-head self-attention block.

[0084] Using multiple intermediate cooperative interaction blocks for alternative communication between the multi-core convolutional modulation block and the multi-head self-attention block, the collaborative encoder can effectively interact between global and local visual semantics to narrow the semantic gap, and in the terminal stage, multiple multi-head self-attention blocks are used to strengthen global context learning.

[0085] The global-local fusion module fuses the multi-scale features output by the collaborative encoder, supplements the fusion of hierarchical visual semantics, and gives full play to the complementary advantages of global and local features.

[0086] In some embodiments, the multi-scale feature F i Input a deep convolution block with residual connection to establish the information propagation path. The calculation formula is shown in formula (8):

[0087]

[0088] in, Indicates F i The information propagation path of F irepresents the output features of the i-th stage of the collaborative encoder; DWConv(·) represents the operation of a deep convolutional block, which is exemplarily composed of two depthwise separable convolutional layers and one pointwise convolutional layer.

[0089] Then for the multi-scale feature F i Perform feature fusion. Considering that due to the reduction of resolution, the features in the deep stage will present more global context information, while the local details will be degraded; while the features in the shallow stage will retain more local details, but lack the global understanding of identifying buildings. In the present invention, the multi-stage features are fused separately to aggregate the local representation and the global representation respectively.

[0090] For local aggregation features, the output features of the early and middle stages of the collaborative encoder are fused, and the calculation formula is shown in formula (9):

[0091]

[0092] Similarly, for the global aggregation features, the output features of the mid-stage and final stages of the collaborative encoder are fused, and the calculation formula is shown in formula (10):

[0093]

[0094] Among them, F L Represents local aggregation features; F G Represents global aggregate features; Indicates F i The information propagation path of F i represents the output features of the i-th stage of the collaborative encoder; UpConv(·) represents the up-convolution (upsampling) operation.

[0095] Based on this fusion strategy, the intermediate features generated by the intermediate cooperative interaction blocks can be supplemented and integrated into the local features and the global features respectively.

[0096] Uncertainty Aggregated Decoder (UAD) is used to quantify the uncertainty in pixels, with the goal of forcing the model to focus on uncertain regions to reduce the uncertainty of segmentation.

[0097] In some embodiments, for any pixel in the local aggregate feature or the global aggregate feature, a Gaussian distribution model is established. Taking the local aggregate feature as an example, for the local aggregate feature F L For any pixel p in the image, a probability Gaussian distribution model with a mean of μ and a variance of σ is established to quantify uncertainty. pis regarded as a random variable, and two embedding functions are introduced to predict the uncertainty of the pixel. As shown in formulas (11) and (12):

[0098] μ=Φ μ (F L ),σ=Φ σ (F L ); (11)

[0099]

[0100] Among them, F L represents local aggregation features; μ and σ represent mean and variance respectively, and the shape is H×W×1; represents the Gaussian distribution of pixel p.

[0101] Among them, σ can also measure the uncertainty of the pixel. If the response of any pixel in σ is high, it means that the model has no confidence in the predicted output at that location.

[0102] In conventional processing, T samples are randomly drawn from the probability representation to generate a series of uncertainty-aware segmentation maps, and the turbulence of the segmentation output is considered in the formula, and its variance is calculated as the uncertainty map, as shown in formula (13):

[0103] U=Norm(Var(x (1) ,x (2) ,…,x (T) )); (13)

[0104] Where U represents the obtained uncertainty map.

[0105] While the above uncertainty modeling is intuitively feasible, the sampling operation in the formula requires indiscriminate sampling to estimate the uncertainty map, which makes it challenging to train the uncertainty aggregation decoder using forward-backward propagation.

[0106] To solve this problem, the present invention adopts the reparameterization trick to convert the direct sampling operation into a trainable component. Specifically, the present invention does not directly extract samples from the probability representation, but introduces the turbulence epsilon, which is generated from the standard Gaussian distribution N(0,I), and then calculates x=σ×epsilon+μ.

[0107] By this method, the indiscriminate sampling process can be avoided and the gradient information can be effectively propagated backward, thereby achieving uncertainty learning. By the above method, the local aggregation feature F can be obtained. L The local uncertainty U L and the global aggregate feature F G The global uncertainty UG .

[0108] In order to mitigate the impact of low confidence areas, the uncertainty maps in the local and global branches can be regarded as attenuated weights, and the local aggregate features and the global aggregate features are combined for feature integration, as shown in formula (14):

[0109] F out =(1-U G )×F G +(1-U L )×F L ; (14)

[0110] Among them, F out represents the final feature map; U G represents global uncertainty; U L represents local uncertainty; F G represents the global aggregation feature; F L Represents local aggregation features.

[0111] The buildings contained in the remote sensing image are extracted based on the final feature map.

[0112] To further illustrate the advantages of the building extraction method provided by this application, as Figure 5 As shown, by comparing the extraction method of the present application with other existing technologies, it can be seen that the extraction method of the present application can also achieve high-quality building extraction in challenging scenarios.

[0113] Corresponding to the building extraction method based on the collaborative encoder and the uncertainty aggregation decoder, the present invention also provides a building extraction system based on the collaborative encoder and the uncertainty aggregation decoder, the system comprising:

[0114] The image processing module is used to acquire remote sensing images and preprocess the remote sensing images.

[0115] The building extraction module inputs the remote sensing image into the pre-trained building extraction model to extract buildings from the remote sensing image.

[0116] Corresponding to the above method, the present invention also provides an electronic device, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the electronic device implements the steps of the method described above.

[0117] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0118] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0119] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0120] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0121] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A building extraction method based on collaborative encoder and uncertainty aggregation decoder, characterized in that: The method comprises the following steps: Acquire a remote sensing image, wherein the remote sensing image contains a building to be extracted; Inputting the remote sensing image into a pre-trained building extraction model to extract buildings from the remote sensing image; The building extraction model includes a collaborative encoder, a global-local fusion module and an uncertainty aggregation decoder connected in sequence; the remote sensing image is input into the collaborative encoder to capture the local characteristics of the remote sensing image in the early stage and the global features of the remote sensing image in the final stage; the multi-scale features extracted by the collaborative encoder are respectively input into the global-local fusion module for feature fusion in stages to obtain local aggregation features and global aggregation features; the local aggregation features and the global aggregation features are respectively input into the uncertainty aggregation decoder to obtain the local uncertainty of the local aggregation features and the global uncertainty of the global aggregation features, the local uncertainty and the global uncertainty are mapped to attenuation weights, and based on the attenuation weights, the local aggregation features and the global aggregation features are integrated to obtain a final feature map to extract the building.

2. The building extraction method based on collaborative encoder and uncertainty aggregation decoder according to claim 1 is characterized in that: Before inputting the remote sensing image into the pre-trained building extraction model, the remote sensing image is pre-processed, including: The remote sensing image is scaled to a preset size and normalized.

3. The building extraction method based on collaborative encoder and uncertainty aggregation decoder according to claim 1 is characterized in that: The method further comprises: The collaborative encoder comprises four stages, wherein the first stage and the second stage are the early stages, the third stage is the middle stage, and the fourth stage is the final stage; In the early stage, a multi-core convolutional modulation block is used to construct the local features; in the middle stage, an intermediate cooperative interaction block is used to perform global and local feature interaction; and in the terminal stage, a multi-head self-attention block is used to enhance the global features.

4. The building extraction method based on collaborative encoder and uncertainty aggregation decoder according to claim 3 is characterized in that: The collaborative encoder is used to extract multi-scale features of the remote sensing image, including: Flattening and linearly projecting the remote sensing image into a one-dimensional vector, and reshaping to obtain an embedded feature map; Splitting the embedded feature map in the channel dimension to obtain multiple feature groups; In the early stage, for each feature group, depth-wise separable convolution with different kernel sizes is used to capture local information, and point convolution is performed to combine to obtain the combined feature, which is calculated as: Z ′ =Cat(DW 3×3 (Z1),…,DW k×k (Z n )); M=W p *Z ′ ; Among them, Z ′ Indicates the local information; DW k×k represents a depth-wise separable convolution with a kernel size of k×k; Z n represents the nth feature group; M represents the combined feature; W p represents the point convolution; The multi-core convolution modulation block is used to enhance the combined features, and the calculation formula is: Among them, MKFAM(·) represents the multi-core convolution operation; F i represents the output features of the i-th stage; In the intermediate stage, after the input features of the intermediate cooperative interaction block are normalized, they are input into the multi-core convolution modulation block with residual connection, and the calculation formula is: In the final stage, the features obtained in the intermediate stage are input into the multi-head self-attention block to simulate the global dependency relationship. The calculation formula is: Among them, X l represents the input features of the lth intermediate cooperative interaction block; FFN(·) represents the operation of the multi-layer perceptron; MHSA(·) represents the operation of the multi-head self-attention block.

5. The building extraction method based on collaborative encoder and uncertainty aggregation decoder according to claim 3 is characterized in that: The multi-scale features extracted by the collaborative encoder are respectively input into the global-local fusion module for feature fusion in stages, further comprising: The multi-scale features are input into the deep convolution block with residual connection to establish the information propagation path, and the calculation formula is: in, Indicates F i The information propagation path of F i represents the output features of the i-th stage of the collaborative encoder; DWConv(·) represents the operation of the depth convolution block.

6. The building extraction method based on collaborative encoder and uncertainty aggregation decoder according to claim 5 is characterized in that: Get local aggregation features and global aggregation features, including: The output features of the early stage and the mid-stage are fused to obtain the local aggregated features, and the calculation formula is: The output features of the mid-term stage and the terminal stage are fused to obtain the global aggregate feature, which is calculated as follows: Among them, F L represents the local aggregation feature; F G represents the global aggregation feature; Indicates F i The information propagation path of F i represents the output features of the i-th stage of the collaborative encoder; UpConv(·) represents the up-convolution operation.

7. The building extraction method based on collaborative encoder and uncertainty aggregation decoder according to claim 1 is characterized in that: Inputting the local aggregate feature and the global aggregate feature into the uncertainty aggregate decoder respectively to obtain local uncertainty and global uncertainty, including: For any pixel in the local aggregate feature or the global aggregate feature, establishing a Gaussian distribution model; The reparameterization technology is adopted to realize uncertainty learning and obtain the local uncertainty and the global uncertainty.

8. The building extraction method based on collaborative encoder and uncertainty aggregation decoder according to claim 1, characterized in that: Based on the attenuation weight, the local aggregation features and the global aggregation features are integrated to obtain the final feature map, and the calculation formula is: F out =(1-U G )×F G +(1-U L )×F L ; Among them, F out represents the final feature map; U G represents the global uncertainty; U L represents local uncertainty; F G represents the global aggregation feature; F L represents the local aggregation feature.

9. A building extraction system based on collaborative encoder and uncertainty aggregation decoder, characterized in that: When the system is executed, the steps of the method according to any one of claims 1 to 8 are implemented, and the system comprises: An image processing module, used for acquiring remote sensing images and preprocessing the remote sensing images; The building extraction module inputs the remote sensing image into a pre-trained building extraction model to extract buildings from the remote sensing image.

10. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images