Intelligent building extraction method based on remote sensing image

Abstract

The invention relates to an intelligent building extraction method based on a remote sensing image. The method comprises the following steps: S1, obtaining a high-resolution data set; s2, designing a remote sensing image building semantic segmentation network based on a coding-decoding structure, wherein the remote sensing image building semantic segmentation network comprises a feature extraction skeleton network and a semantic segmentation decoder; s3, a remote sensing image building DSM estimation network based on the generative adversarial network is designed and comprises a DSM generator and a DSM discriminator, and the DSM generator comprises a DSM generator encoder and a DSM generator decoder; s4, designing a feature fusion and reinforcement module; s5, designing a loss function; s6, training and optimizing a remote sensing image building intelligent extraction network according to the high-resolution data set and the loss function; and S7, performing building intelligent extraction based on the remote sensing image through the trained remote sensing image building intelligent extraction network. According to the method, the problem of high DSM acquisition cost is solved, and the performance of extracting the building in the remote sensing image is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a method for intelligently extracting buildings based on remote sensing images. Background Art

[0002] With the rapid development of satellite imaging technology, high spatial resolution remote sensing data at home and abroad are becoming increasingly abundant, playing an important role in many fields such as national economic construction, resource and environmental protection, etc. Automatically and accurately extracting the location, boundary, type and other information of objects from high-resolution remote sensing images is the key difficulty of remote sensing image processing.

[0003] In recent years, with the rise of deep neural networks, the semantic segmentation of remote sensing images based on deep neural network models has achieved high accuracy on standard data sets, and has also shown great application potential in high-resolution remote sensing image object recognition and information extraction.

[0004] The size of buildings in remote sensing images varies greatly, and they generally present weak target characteristics, which brings great challenges to the segmentation of buildings in remote sensing images. In addition, remote sensing images have large differences between classes and small differences within classes. The roof types and heights of buildings in remote sensing images present a long-tail distribution, and the edges of different classes in remote sensing images are similar, such as roofs and roads. In actual scenes, due to the influence of the shooting phase and angle, remote sensing images contain interference factors such as shadows, occlusions, and clouds. These factors will make the segmentation of buildings very difficult.

[0005] At present, the application of deep semantic segmentation models in remote sensing image semantic segmentation tasks faces the following difficulties: on the one hand, the inherent contradiction between high-level semantics and low-level details of deep convolutional networks causes a large loss of detail information; at the same time, high-resolution remote sensing images are rich in details, requiring neural networks to be able to extract more discriminative detail information and multi-scale information. Summary of the invention

[0006] In order to solve the technical problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a method, electronic device and storage medium for intelligent extraction of buildings based on remote sensing images, which can effectively improve the performance of extracting buildings from remote sensing images.

[0007] To achieve the above-mentioned object of the invention, the present invention provides a method for intelligent extraction of buildings based on remote sensing images, comprising the following steps:

[0008] Step S1, obtaining a high-resolution data set;

[0009] Step S2, designing a remote sensing image building semantic segmentation network based on an encoding-decoding structure, wherein the remote sensing image building semantic segmentation network includes a semantic segmentation encoder and a semantic segmentation decoder, wherein the semantic segmentation encoder is a feature extraction skeleton network, and the feature extraction skeleton network includes a GLFE module for fusing global features with local features;

[0010] Step S3, designing a remote sensing image building DSM estimation network based on a generative adversarial network, wherein the DSM estimation network includes a DSM generator and a DSM discriminator, wherein the DSM generator includes a DSM generator encoder and a DSM generator decoder, and the DSM generator encoder is the feature extraction skeleton network;

[0011] Step S4, designing a feature fusion and enhancement module, the feature fusion and enhancement module is used to realize the output feature fusion of the feature extraction skeleton network, the feature fusion of the semantic segmentation decoder and the DSM generator decoder, and includes a MASPP module and a DSATT module;

[0012] Step S5, designing a loss function;

[0013] Step S6, training and optimizing a remote sensing image building intelligent extraction network according to the high-resolution data set and the loss function, wherein the remote sensing image building intelligent extraction network includes the remote sensing image building semantic segmentation network, the remote sensing image DSM estimation network and the feature fusion and enhancement module;

[0014] Step S7: performing intelligent extraction of buildings based on remote sensing images through the trained remote sensing image building intelligent extraction network.

[0015] According to a technical solution of the present invention, step S2 specifically comprises the following steps:

[0016] Step S21, designing the feature extraction skeleton network;

[0017] The feature extraction skeleton network uses the ResNet101 network as the basic network, is divided into 5 stages, removes the fully connected layer on the network output side, does not perform downsampling in stage5, and uses ImageNet for pre-training; the input size of the feature extraction skeleton network is 3×512×512; the stage1 and stage2 of the feature extraction skeleton network have the same basic structure as ResNet101, and the stage3, stage4 and stage5 of the feature extraction skeleton network include the GLFE module; step S22, designing the GLFE module;

[0018] The GLFE module includes a global feature extraction module GFE and a local feature extraction module LFE. The global feature extraction module GFE is implemented by an external attention module using a two-linear layer, which is expressed as follows:

[0019] F out =Linear2(Norm(Linear1(F in )))

[0020] The local feature extraction module LFE is composed of two parallel branches. The input of the local feature extraction module LFE is subjected to 1×1 convolution for channel dimension increase, and the channel is evenly divided. The half of the channels after even division are extracted through a 1×3, 3×1 convolution cascade branch, which is expressed as follows:

[0021]

[0022] Another branch of the local feature extraction module LFE is F 1out The sum of the features of the other half of the channels after equal division is used as input, and features are extracted through pooling and 1×5 and 5×1 convolutions, as shown below:

[0023]

[0024] F 1out With F 2out Perform a 1×1 convolution on the channel dimension, expressed as follows:

[0025] F = conv 1×1 (F 1out +F 2out );

[0026] Step S23, designing the semantic segmentation decoder;

[0027] The semantic segmentation decoder is used to upsample the semantic segmentation features extracted by the feature extraction skeleton network to restore them to the input size; the semantic segmentation decoder includes two branches, and the first branch of the semantic segmentation decoder is a convolution kernel of 3×1 and 1×3 cascade, which is expressed as follows:

[0028] F branch1 =conv 1×3 (conv 3×1 (F dec ))

[0029] where F dec represents the input of the semantic segmentation decoder, F branch1 represents the output of the first branch of the semantic segmentation decoder;

[0030] The second branch of the semantic segmentation decoder takes the output features of the first branch of the semantic segmentation decoder and the fusion of the input features of the semantic segmentation decoder as input, and outputs after passing through a depthwise separable convolutional layer with a convolution kernel of 3 and a dilation rate of 2 and a densely connected layer with a convolution kernel of 3 and a dilation rate of 3. The output of the second branch of the semantic segmentation decoder is represented by F branch2 ; Then the two branches of the semantic segmentation decoder are spliced ​​and fused from the channel dimension, and the channel information is fused through a convolution with a convolution kernel of 1×1.

[0031] Step S24, designing a fusion module of the semantic segmentation decoder. For each stage of the semantic segmentation decoder, its input includes the output of the previous stage of the semantic segmentation decoder, the corresponding stage output of the DSM generator decoder, and the output of the corresponding stage of the feature extraction skeleton network.

[0032] According to a technical solution of the present invention, the design of the building DSM estimation network in step S3 specifically includes the following steps:

[0033] Step S31, designing the DSM generator decoder;

[0034] The DSM generator decoder adopts a dual-branch fusion module, whose input is divided into channels after the input channel dimension is increased by 1×1 regular convolution, that is, F c / 2,1 、F c / 2,2 ;

[0035] The first branch of the DSM generator decoder is a 3×3 regular convolution, and the input of the second branch of the DSM generator decoder is F c / 2,2 The channel fusion features of the first branch of the DSM generator decoder are output through 3×3 convolution; finally, the outputs of the two branches of the DSM generator decoder are added and fused;

[0036] The channel dimension of each stage of the DSM generator decoder is half of the corresponding stage of the building semantic segmentation decoder;

[0037] Step S32, design a fusion module of the DSM generator decoder. For each stage of the DSM generator decoder, its input includes the output of the previous stage of the DSM generator decoder, the corresponding stage output of the semantic segmentation decoder, and the output of the corresponding stage of the feature extraction skeleton network.

[0038] Step S33, designing the DSM discriminator;

[0039] The DSM discriminator is used to identify the authenticity of the DSM generated by the DSM generator, uses ResNet50 as the basic skeleton, and includes 5 stages, wherein the second stage, the fourth stage, and the fifth stage of the DSM discriminator all use a dual-branch feature extraction module, and the dual-branch feature extraction module implements channel dimension upgrading through a 1×1 conventional convolution on the input. The input of the first branch of the dual-branch feature extraction module is a 1 / 2 channel feature, and a 3×3 depth-separable convolution is used to implement feature extraction. The second branch of the dual-branch feature extraction module adds another 1 / 2 channel feature to the output of the first branch of the dual-branch feature extraction module as input, and outputs it after a depth-separable dilated convolution with a convolution kernel of 3 and a dilation rate of 2. Finally, the two branches of the dual-branch feature extraction module are channel-fused, and feature extraction is implemented through a 1×1 convolution.

[0040] According to a technical solution of the present invention, the feature fusion and enhancement module design in step S4 includes the following steps:

[0041] Step S41, designing the MASPP module after the feature extraction skeleton network output;

[0042] The MASPP module adopts multi-branch parallelism. The input of the MASPP module is F pin , the first branch of the MASPP module is a conventional convolution with a convolution kernel of 1×1, and the output of the first branch of the MASPP module is F p1 ; The second branch of the MASPP module is a conventional convolution with a convolution kernel of 3×3, and the output of the second branch of the MASPP module is F p2 ; The input of the third branch of the MASPP module is F pin With F p2 The output of the third branch of the MASPP module can be expressed as follows:

[0043] F p3_1 =Aconv 3,2 (AP 3 (F pin +F p2 ))

[0044]

[0045] F p3 =F p3_1 +F p3_2 +AP 3 (F pin +F p2 )

[0046] Among them, AP3 Indicates average pooling with a kernel of 3, Aconv 3,2 Aconv represents a dilated convolution with a convolution kernel of 3 and a dilation rate of 2. 3,3 It represents a dilated convolution with a convolution kernel of 3 and a dilation rate of 3. Indicates that features are fused based on channels;

[0047] The input of the fourth branch of the MASPP module is F pin With F p3 The fusion output can be expressed as follows:

[0048] F p4_1 =Aconv 3,4 (AP 5 (F pin +F p3 ))

[0049]

[0050] F p4 =F p4_1 +F p4_2 +AP 5 (F pin +F p3 )

[0051] Among them, AP 5 Indicates average pooling with a kernel of 5, Aconv 3,4 Aconv represents a dilated convolution with a kernel size of 3 and a dilation rate of 4. 3,5 It is represented as a dilated convolution with a convolution kernel of 3 and a dilation rate of 5; the fifth branch of the MASPP module is global average pooling, and the output of the fifth branch of the MASPP module is represented as F p5 ;

[0052] The output of the parallel branch of the MASPP module uses channel fusion to combine F p1 、F p2 、F p3 、F p4 and F p5 Fusion is performed to achieve information fusion between channels through 1×1 convolution, so that the channel dimension is consistent with the module input, and the output is represented as F piout ; By strengthening the effective features through the channel attention mechanism, the output of the MASPP module is:

[0053] F pout =F piout +SEB(F piout )

[0054] Where SEB(.) represents the fusion channel attention mechanism of mean pooling and maximum pooling;

[0055] Step S42, designing the DSATT module;

[0056] The input of the DSATT module is the output feature F of the semantic segmentation decoder seg And the output feature F of the DSM generator decoder at the corresponding stage dsm , the output process of the DSATT module can be expressed as follows:

[0057] Att seg =S(Linear(ReLU(Linear(GAP(F seg )))))

[0058] Att dsm =S(Linear(ReLU(Linear(GAP(F DSM )))))

[0059]

[0060]

[0061] Among them, F segout 、F dsmout represents the output after the DSATT module, GAP(.) represents global average pooling, Linear(.) represents linear change, S(.) represents activation function, Att seg With Att dsm Respectively represent the channel attention weights of the output features of the semantic segmentation decoder and the channel attention weights of the output features of the DSM generator decoder, Art seg With Att dsm The sum is 1.

[0062] According to a technical solution of the present invention, in step S5, it specifically includes:

[0063] Step S51, designing the loss function of the remote sensing image building semantic segmentation network, the total loss of the remote sensing image building semantic segmentation network includes pixel-level loss and regional loss, and the calculation formula of the pixel-level loss is as follows:

[0064]

[0065] Among them, yi c represents the true label at position i, expressed as one-hot. It is 1 when the true label is a building, otherwise it is 0. C is the number of categories, which is 2 here, and N is the total number of pixels in the predicted image.ic represents the probability of classifying sample i as category c; w c is the weight parameter, and the number of pixels of buildings in all samples of the training set is calculated to calculate w c , which is expressed as follows:

[0066]

[0067] The calculation formula of the area loss is as follows:

[0068]

[0069] Among them, y pred represents the probability of predicting category c, y true Represents the true one-hot label;

[0070] The total loss of the remote sensing image building semantic segmentation network is:

[0071] L seg =αL wce +βL softIoU ;

[0072] Step S52: Design a loss function of the building DSM estimation network. The total loss of the building DSM estimation network includes adversarial loss and height estimation loss. The adversarial loss is expressed as follows:

[0073] L G = -E(logD(G(x input )))

[0074] L D = -E(logD(x input_dsm ))-E(log(1-D(G(x input ))))

[0075] L adv =L G +L D

[0076] Where G and D represent the DSM generator encoder and the DSM generator decoder respectively, E(.) represents the mathematical expectation, x input_dsm represents the real DSM of the input remote sensing image, x input Represents the input remote sensing image;

[0077] The height estimation loss is expressed as follows:

[0078]

[0079] The building DSM estimates the total network loss to be:

[0080] L dsm =γ 1 L adv +γ 2 L h ;

[0081] Among them, γ 1 and γ 2 denote the loss weights of adversarial loss and height estimation loss respectively.

[0082] According to a technical solution of the present invention, in step S1, the high-resolution dataset includes a Vaihingen dataset and a Potsdam dataset, and each sample in the high-resolution dataset includes an RGB image and its corresponding DSM image; the samples of the Vaihingen dataset and the Potsdam dataset are preprocessed and cropped into slices of 521 pixels × 512 pixels.

[0083] According to one aspect of the present invention, there is provided an electronic device, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory, and when the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes a method for intelligent extraction of buildings based on remote sensing images as described in any one of the above technical solutions.

[0084] According to one aspect of the present invention, a computer-readable storage medium is provided for storing computer instructions. When the computer instructions are executed by a processor, a method for intelligently extracting buildings based on remote sensing images as described in any one of the above technical solutions is implemented.

[0085] Compared with the prior art, the present invention has the following beneficial effects:

[0086] 1. Through the optimized encoder, the network's ability to aggregate contextual information and extract multi-scale features is effectively improved;

[0087] 2. Aiming at the multi-scale problem of ground objects, the present invention adds a multi-scale densely connected hole convolution attention module between the encoder and the decoder. Through multi-branch structure, conventional convolution, pooling operation, hole pyramid convolution, dense connection and feature flow between branches, it realizes the extraction and fusion of features of different scales and enriches context information. ;

[0088] 3. Add channel attention after multi-scale densely connected atrous pyramid pooling to strengthen effective features and make the extracted building features more discriminative;

[0089] 4. To solve the problem that similar objects are difficult to distinguish, the present invention optimizes multi-task features by combining multiple tasks. By combining the building semantic segmentation task with the DSM generation task, the information complementarity between semantic features and height features is achieved, the features are enhanced, and the segmentation performance is improved by using multi-source features;

[0090] 5. In view of the time and cost of DSM acquisition, the present invention generates DSM for a single remote sensing image through generative adversarial network technology. When predicting the segmentation of buildings, only a single remote sensing image needs to be input to obtain the corresponding DSM and segmentation map at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0092] Figure 1 A design flow chart of a building intelligent extraction model based on remote sensing images according to an embodiment of the present invention is schematically shown;

[0093] Figure 2 A flowchart of a method for intelligently extracting buildings based on remote sensing images according to an embodiment of the present invention is schematically shown;

[0094] Figure 3 The overall software architecture of the method for intelligent extraction of buildings based on remote sensing images according to an embodiment of the present invention is schematically shown;

[0095] Figure 4 Schematically showing the structure of the GLFE module according to an embodiment of the present invention;

[0096] Figure 5 Schematically showing the MASPP module structure diagram according to an embodiment of the present invention;

[0097] Figure 6 The following is a schematic diagram showing the structure of a DSATT module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0098] The description of the embodiments of this specification should be combined with the corresponding drawings, which should be considered as part of the complete specification. In the drawings, the shape or thickness of the embodiments may be enlarged and indicated for simplification or convenience. Furthermore, the parts of each structure in the drawings will be described separately. It is worth noting that the elements not shown in the drawings or not described in words are in a form known to ordinary technicians in the relevant technical field.

[0099] The description of the embodiments herein and any reference to directions and orientations are only for the convenience of description and are not to be construed as any limitation on the scope of protection of the present invention. The following description of the preferred embodiments may involve combinations of features, which may exist independently or in combination, and the present invention is not particularly limited to the preferred embodiments. The scope of the present invention is defined by the claims.

[0100] This embodiment provides a method for intelligently extracting buildings based on remote sensing images based on the method of the present invention. Figures 1 to 6 As shown, the following steps are included:

[0101] Step S1, obtaining a high-resolution data set;

[0102] According to a technical solution of the present invention, in step S1, the high-resolution dataset includes a Vaihingen dataset and a Potsdam dataset, and each sample in the high-resolution dataset includes an RGB image and its corresponding DSM image; the samples of the Vaihingen dataset and the Potsdam dataset are preprocessed and cropped into slices of 521 pixels × 512 pixels.

[0103] Step S2, designing a remote sensing image building semantic segmentation network based on an encoding-decoding structure, wherein the remote sensing image building semantic segmentation network includes a semantic segmentation encoder and a semantic segmentation decoder, wherein the semantic segmentation encoder is a feature extraction skeleton network, and the feature extraction skeleton network includes a GLFE module for fusing global features with local features;

[0104] According to a technical solution of the present invention, step S2 specifically includes the following steps:

[0105] Step S21, designing a feature extraction skeleton network;

[0106] Among them, the feature extraction skeleton network takes ResNet1 01 network as the basic network, which is divided into 5 stages. The fully connected layer on the network output side is removed, stage5 does not perform downsampling, and uses ImageNet for pre-training; the input size of the feature extraction skeleton network is 3×512×512; among them, stage1 and stage2 of the feature extraction skeleton network have the same basic structure as ResNet101, and stage3, stage4 and stage5 of the feature extraction skeleton network include GLFE modules.

[0107] Step S22, designing a GLFE module;

[0108] The GLFE module includes a global feature extraction module GFE and a local feature extraction module LFE. The global feature extraction module GFE is implemented by using an external attention module with two linear layers, which is expressed as follows:

[0109] F ou t ＝ Linear2(Norm(Linear1(F in )))

[0110] The local feature extraction module LFE is composed of two parallel branches. The input of the local feature extraction module LFE is subjected to 1×1 convolution for channel dimension increase, and then the channel is evenly divided. The half of the channels after even division are extracted through a 1×3, 3×1 convolution cascade branch, which is expressed as follows:

[0111]

[0112] Another branch of the local feature extraction module LFE is based on F 1out The sum of the features of the other half of the channels after equal division is used as input, and features are extracted through pooling and 1×5 and 5×1 convolutions, as shown below:

[0113]

[0114] F 1out With F 2out Perform a 1×1 convolution on the channel dimension, expressed as follows:

[0115] F = conv 1×1 (F 1out +F 2out );

[0116] The above process adopts a global-local feature extraction branch to extract features of different receptive fields, strengthen information flow, and enrich contextual information, making the features of buildings more robust to different scales.

[0117] Step S23, designing a semantic segmentation decoder;

[0118] The semantic segmentation decoder is used to upsample the semantic segmentation features extracted by the feature extraction skeleton network to restore them to the input size; the semantic segmentation decoder needs to be upsampled 4 times to restore them to 3×512×512. The semantic segmentation decoder consists of two branches. The first branch of the semantic segmentation decoder is a cascade of convolution kernels of 3×1 and 1×3, which is expressed as follows:

[0119] F branch1 =conv 1×3 (conv 3×1 (F dec ))

[0120] where F dec represents the input of the semantic segmentation decoder, F branch1represents the output of the first branch of the semantic segmentation decoder;

[0121] The second branch of the semantic segmentation decoder takes the output features of the first branch of the semantic segmentation decoder and the fusion of the input features of the semantic segmentation decoder as input, and outputs it after passing through a depthwise separable convolutional layer with a convolution kernel of 3 and a dilation rate of 2 and a densely connected layer with a convolution kernel of 3 and a dilation rate of 3. The output of the second branch of the semantic segmentation decoder is denoted as F branch2 ; Then the two branches of the semantic segmentation decoder are spliced ​​and fused from the channel dimension, and the channel information is fused through convolution with a convolution kernel of 1×1.

[0122] This module, through dense connections of deep separable convolutions with different branches and different void ratios, expands the receptive field with fewer parameters, realizes the fusion of features of different scales, and solves the multi-scale characteristics of buildings.

[0123] Step S24, design a fusion module of the semantic segmentation decoder. For each stage of the semantic segmentation decoder, its input includes the output of the previous stage of the semantic segmentation decoder, the corresponding stage output of the DSM generator decoder, and the output of the corresponding stage of the feature extraction skeleton network. The fusion module of the semantic segmentation decoder effectively fuses multi-source features, detailed features of the encoder part, and high-level features of the previous stage of the decoder, making the extracted building features more discriminative.

[0124] Through step S2, the encoding and decoding parts of the building semantic segmentation are designed, the context features are enriched through the global-local branch, and the deep separable convolutions with different branches and different void rates are densely connected to achieve the fusion of features of different scales and alleviate the impact of the multi-scale characteristics of remote sensing images.

[0125] Step S3, designing a remote sensing image building DSM estimation network based on a generative adversarial network, wherein the DSM estimation network includes a DSM generator and a DSM discriminator, which are used for DSM generation of buildings in the reasoning stage, and the DSM generator includes a DSM generator encoder and a DSM generator decoder, and the DSM generator encoder is a feature extraction skeleton network;

[0126] Step S31, design a DSM generator decoder; in order to alleviate the problem of complex and expensive DSM generation process, a generative adversarial network technology is used to synthesize the DSM. The encoder part of the DSM generator in the present invention uses the feature extraction skeleton network of step S21.

[0127] The DSM generator decoder uses a dual-branch fusion module, whose input is divided into channels after the input channel dimension is increased by 1×1 regular convolution, that is, F c / 2,1 、F c / 2,2 ;

[0128] The first branch of the DSM generator decoder is a 3×3 regular convolution, and the input of the second branch of the DSM generator decoder is F c / 2,2 The channel fusion features of the first branch of the DSM generator decoder are output through 3×3 convolution; finally, the outputs of the two branches of the DSM generator decoder are added and fused;

[0129] The channel dimension of each stage of the DSM generator decoder is half of the corresponding stage of the building semantic segmentation decoder;

[0130] Step S32, design a fusion module of the DSM generator decoder. For each stage of the DSM generator decoder, its input includes the output of the previous stage of the DSM generator decoder, the corresponding stage output of the semantic segmentation decoder, and the output of the corresponding stage of the feature extraction skeleton network. The fusion module of the DSM generator decoder effectively fuses multi-source features, detailed features of the encoder part, and high-level features of the previous stage of the decoder to enrich the DSM features.

[0131] Step S33, designing a DSM discriminator;

[0132] The DSM discriminator is used to identify the authenticity of the DSM generated by the DSM generator. It uses ResNet50 as the basic skeleton and includes 5 stages. The second, fourth and fifth stages of the DSM discriminator all use a dual-branch feature extraction module. The dual-branch feature extraction module uses a 1×1 conventional convolution to realize channel dimension increase. The input of the first branch of the dual-branch feature extraction module is a 1 / 2 channel feature, and a 3×3 depth-separable convolution is used to realize feature extraction. The second branch of the dual-branch feature extraction module adds another 1 / 2 channel feature to the output of the first branch of the dual-branch feature extraction module as input, and outputs it after a depth-separable dilated convolution with a convolution kernel of 3 and a dilation rate of 2. Finally, the two branches of the dual-branch feature extraction module are channel-fused and feature extraction is realized through a 1×1 convolution. By using different basic modules at different stages, the extraction of features of different scales is realized, making the features of the building DSM more discriminative.

[0133] The DSM estimation network is used to generate DSM for a single RGB remote sensing image, which is used to assist in feature extraction of building semantic segmentation. In order to effectively fuse the features of the common encoder part and the decoding branches of different task branches, increase the fluidity of information, and effectively fuse the semantic segmentation branch and the DSM estimation branch, different fusion modules will be used below.

[0134] Step S4, design a feature fusion and enhancement module, which is used to realize the output feature fusion of the feature extraction skeleton network, the feature fusion of the semantic segmentation decoder and the DSM generator decoder, including the MASPP module and the DSATT module; the MASPP module realizes multi-scale feature fusion, integrates multi-scale features and enhances effective features; the fusion of different decoder branches, the DSATT module enhances the fusion of different modal features, and improves the performance of semantic segmentation.

[0135] According to a technical solution of the present invention, the feature fusion and enhancement module design in step S4 includes the following steps:

[0136] Step S41, designing a MASPP module after the output of the feature extraction skeleton network;

[0137] like Figure 4 In order to aggregate context information and enrich multi-scale features to improve segmentation performance, the MASPP module adopts multi-branch parallelism. The input of the MASPP module is F pin , the first branch of the MASPP module is a regular convolution with a convolution kernel of 1×1, and the output of the first branch of the MASPP module is F p1 ; The second branch of the MASPP module is a regular convolution with a convolution kernel of 3×3. The output of the second branch of the MASPP module is F p2 ; The input of the third branch of the MASPP module is F pin With F p2 The output of the third branch of the MASPP module can be expressed as follows:

[0138] F p3_1 =Aconv 3,2 (AP 3 (F pin +F p2 ))

[0139]

[0140] F p3 =F p3_1 +F p3_2 +AP 3 (F pin +F p2 )

[0141] Among them, AP 3 Indicates average pooling with a kernel of 3, Aconv 3,2 Aconv represents a dilated convolution with a convolution kernel of 3 and a dilation rate of 2. 3，3 It represents a dilated convolution with a convolution kernel of 3 and a dilation rate of 3. It indicates that the features are fused according to the channel; the third branch fuses the previous branch with the input, and combines average pooling and dense hole convolution to achieve the fusion of features of different scales.

[0142] The input of the fourth branch of the MASPP module is F pin With F p3 The fusion output can be expressed as follows:

[0143] F p4_1 =Aconv 3,4 (AP 5 (F pin +F p3 ))

[0144]

[0145] F p4 =F p4_1 +F p4_2 +AP 5 (F pin +F p3 )

[0146] Among them, AP 5 Indicates average pooling with a kernel of 5, Aconv 3,4 Aconv represents a dilated convolution with a kernel size of 3 and a dilation rate of 4. 3,5 It is represented as a dilated convolution with a convolution kernel of 3 and a dilation rate of 5. The fifth branch of the MASPP module is global average pooling, and the output of the fifth branch of the MASPP module is represented as F p5 ;

[0147] The output of the parallel branch of the MASPP module uses channel fusion to convert F p1 、F p2 、F p3 、F p4 and F p5 Fusion is performed to achieve information fusion between channels through 1×1 convolution, so that the channel dimension is consistent with the module input, and the output is represented as F piout ; By strengthening the effective features through the channel attention mechanism, the output of the MASPP module is:

[0148] F pout =F piout +SEB(F piout )

[0149] Where SEB(.) represents the fusion channel attention mechanism of mean pooling and maximum pooling;

[0150] The design of the MASPP module integrates parallel branches with different receptive fields, introduces pooling operations and densely connected dilated convolutional pyramids, and realizes multi-scale feature extraction and flow between features. Finally, the effective features are strengthened through the channel attention mechanism.

[0151] Step S42, designing a DSATT module;

[0152] like Figure 5 As shown in Figure 1, the DSATT module effectively integrates features from multiple sources and uses the complementary attention features of the two branches to strengthen the main task and auxiliary tasks, thereby improving the feature recovery function of the decoder. The input of the DSATT module is the output feature F of the semantic segmentation decoder. seg And the output feature F of the DSM generator decoder at the corresponding stage dsm , the output process of the DSATT module can be expressed as follows:

[0153] Att seg =S(Linear(ReLU(Linear(GAP(F seg )))))

[0154] Att dsm =S(Linear(ReLU(Linear(GAP(F DSM )))))

[0155]

[0156]

[0157] Among them, F segout 、F dsmout represents the output after the DSATT module, GAP(.) represents global average pooling, Linear(.) represents linear change, S(.) represents the activation function, Att seg With Att dsm Att represents the channel attention weight of the output feature of the semantic segmentation decoder and the channel attention weight of the output feature of the DSM generator decoder, respectively. seg With Att dsm The sum is 1. The feature enhancement of the main branch is achieved, and the constraints of the auxiliary tasks are added to improve each other.

[0158] The above steps complete the data preprocessing and network design. Subsequently, the loss function will be designed to train the network.

[0159] Step S5: design a loss function; constrain the entire network architecture, train and optimize the network, and improve network performance.

[0160] According to a technical solution of the present invention, in step S5, it specifically includes:

[0161] Step S51, designing the loss function of the remote sensing image building semantic segmentation network, taking into account that buildings account for a small proportion in remote sensing images, solving the problem of category imbalance, using weighted cross entropy to optimize the semantic segmentation network, the total loss of the remote sensing image building semantic segmentation network includes pixel-level loss and regional loss, and the calculation formula of pixel-level loss is as follows:

[0162]

[0163] Among them, yi c represents the true label at position i, expressed as one-hot. If the true label is a building, it is 1, otherwise it is 0. C is the number of categories, which is 2 here, and N is the total number of pixels in the predicted image. ic represents the probability of classifying sample i as category c; w c is the weight parameter, and the number of pixels of buildings in all samples of the training set is calculated to calculate w c , which is expressed as follows:

[0164]

[0165] In order to increase the constraints on buildings, the regional loss constraint is added on the basis of the pixel-level loss. The calculation formula of the regional loss is as follows:

[0166]

[0167] Among them, y pred represents the probability of predicting category c, y true Represents the true one-hot label;

[0168] The total loss of the remote sensing image building semantic segmentation network is:

[0169] L seg =αL wce +βL softIoU ;

[0170] Step S52: Design a loss function of the building DSM estimation network. The total loss of the building DSM estimation network includes adversarial loss and height estimation loss. The adversarial loss is expressed as follows:

[0171] L G = -E(logD(G(x input )))

[0172] L D = -E(logD(x input_dsm ))-E(log(1-D(G(x input))))

[0173] L adv =L G +L D

[0174] Where G and D represent the DSM generator encoder and DSM generator decoder respectively, E(.) represents the mathematical expectation, and x input_dsm represents the real DSM of the input remote sensing image, x input Represents the input remote sensing image;

[0175] The DSM generator and the adversary synthesize a real DSM image through game confrontation. In order to constrain the accuracy of DSM synthesis, the Huber loss with δ of 1 is used to reduce the sensitivity to DSM outliers. The height estimation loss is expressed as follows:

[0176]

[0177] The building DSM estimates the total loss of the network to be:

[0178] L dsm =γ 1 L adv +γ 2 L h ;

[0179] Among them, γ 1 and γ 2 denote the loss weights of adversarial loss and height estimation loss respectively.

[0180] Step S6, training and optimizing a remote sensing image building intelligent extraction network according to a high-resolution data set and a loss function, wherein the remote sensing image building intelligent extraction network includes a remote sensing image building semantic segmentation network, a remote sensing image DSM estimation network, and a feature fusion and enhancement module;

[0181] Step S7: performing intelligent extraction of buildings based on remote sensing images through the trained remote sensing image building intelligent extraction network.

[0182] Through the above steps, intelligent extraction of buildings from high-resolution remote sensing images can be achieved. In view of the fact that high-resolution remote sensing images have rich detailed information on objects, large changes in building scales, and similar categories are difficult to distinguish, which results in poor segmentation performance of buildings in actual scenes, the present invention designs a multi-task joint to achieve intelligent extraction of buildings from high-resolution remote sensing images. In view of the limited context information extracted by deep convolutional neural networks, the present invention optimizes the encoder by designing a global-local dual-branch module, extracts global information through a simplified self-attention mechanism, and achieves the fusion of different receptive field features through convolution cascade and different convolution kernel feature fusion. Through the optimized encoder, the network's ability to aggregate context information and multi-scale feature extraction capabilities are effectively improved. In view of the multi-scale problem of objects, the present invention adds a multi-scale densely connected hole convolution attention module between the encoder and the decoder, and realizes the extraction and fusion of features of different scales through a multi-branch structure, conventional convolution, pooling operation, hole pyramid convolution, dense connection, and feature flow between branches, and enriches context information. In addition, channel attention is added after the multi-scale densely connected hole pyramid pooling to strengthen effective features, so that the extracted building features are more discriminative.

[0183] In view of the problem that similar objects are difficult to distinguish, the present invention optimizes multi-task features by combining multiple tasks. By combining the building semantic segmentation task with the DSM generation task, the information complementarity between semantic features and height features is achieved, the features are enhanced, and the segmentation performance is improved by using multi-source features. In view of the time and cost of DSM acquisition, the present invention generates DSM for a single remote sensing image by generating adversarial network technology. In view of the fusion problem of semantic features and height features, the present invention designs a complementary attention mechanism, which fuses the branch attention enhancement features and the complementary attention features of the height features in the semantic feature branch, and fuses the branch attention enhancement features and the complementary attention features of the semantic features in the height feature branch, so as to increase the features of the trunk task, and supplemented by the complementary features of another branch, effectively enhance the fluidity and fusion of information. In order to extract multi-scale features and detail information, a dual-branch feature extraction module is introduced in the decoder and DSM discriminator of the DSM estimation network, and convolutional features with different void rates are fused. In the optimization training process, the network is optimized by pre-training methods and multiple loss functions, adversarial loss, height estimation loss, semantic segmentation cross entropy loss, and intersection-over-union loss. When predicting and segmenting buildings, only a single remote sensing image needs to be input to obtain the corresponding DSM and segmentation map at the same time.

[0184] According to one aspect of the present invention, there is provided an electronic device, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory, and when the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes a method for intelligent extraction of buildings based on remote sensing images as described in any one of the above technical solutions.

[0185] The processor may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0186] According to one aspect of the present invention, a computer-readable storage medium is provided for storing computer instructions. When the computer instructions are executed by a processor, a method for intelligently extracting buildings based on remote sensing images as described in any one of the above technical solutions is implemented.

[0187] Computer-readable storage media may include any medium capable of storing or transmitting information. Examples of computer-readable storage media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments may be downloaded via computer networks such as the Internet, intranets, etc.

[0188] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0189] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0190] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0191] It should also be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0192] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for those skilled in the art, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A method for intelligent building extraction based on remote sensing images, characterized in that: The following steps are involved: Step S1, obtaining a high-resolution data set; Step S2, designing a remote sensing image building semantic segmentation network based on an encoding-decoding structure, wherein the remote sensing image building semantic segmentation network includes a semantic segmentation encoder and a semantic segmentation decoder, wherein the semantic segmentation encoder is a feature extraction skeleton network, and the feature extraction skeleton network includes a GLFE module for fusing global features with local features; Step S3, designing a remote sensing image building DSM estimation network based on a generative adversarial network, wherein the DSM estimation network includes a DSM generator and a DSM discriminator, wherein the DSM generator includes a DSM generator encoder and a DSM generator decoder, and the DSM generator encoder is the feature extraction skeleton network; Step S4, designing a feature fusion and enhancement module, the feature fusion and enhancement module is used to realize the output feature fusion of the feature extraction skeleton network, the feature fusion of the semantic segmentation decoder and the DSM generator decoder, and includes a MASPP module and a DSATT module; Step S5, designing a loss function; Step S6, training and optimizing a remote sensing image building intelligent extraction network according to the high-resolution data set and the loss function, wherein the remote sensing image building intelligent extraction network includes the remote sensing image building semantic segmentation network, the remote sensing image DSM estimation network and the feature fusion and enhancement module; Step S7: performing intelligent extraction of buildings based on remote sensing images through the trained remote sensing image building intelligent extraction network.

2. The intelligent building extraction method based on remote sensing images according to claim 1 is characterized in that: The step S2 specifically comprises the following steps: Step S21, designing the feature extraction skeleton network; The feature extraction skeleton network uses the ResNet101 network as the basic network, is divided into 5 stages, removes the fully connected layer on the network output side, does not perform downsampling in stage5, and uses ImageNet for pre-training; the input size of the feature extraction skeleton network is 3×512×512; the stage1 and stage2 of the feature extraction skeleton network have the same basic structure as ResNet101, and the stage3, stage4 and stage5 of the feature extraction skeleton network include the GLFE module; Step S22, designing the GLFE module; The GLFE module includes a global feature extraction module GFE and a local feature extraction module LFE. The global feature extraction module GFE is implemented by an external attention module using a two-linear layer, which is expressed as follows: F out =Linear2(Norm(Linear1(F in ))) The local feature extraction module LFE is composed of two parallel branches. The input of the local feature extraction module LFE is subjected to 1×1 convolution for channel dimension increase, and the channel is evenly divided. The half of the channels after even division are extracted through a 1×3, 3×1 convolution cascade branch, which is expressed as follows: Another branch of the local feature extraction module LFE is F 1out The sum of the features of the other half of the channels after equal division is used as input, and features are extracted through pooling and 1×5 and 5×1 convolutions, as shown below: F 1out With F 2out A 1×1 convolution is performed on the channel dimension, which is expressed as follows: F=conv 1×1 (F 1out +F 2out ); Step S23, designing the semantic segmentation decoder; The semantic segmentation decoder is used to upsample the semantic segmentation features extracted by the feature extraction skeleton network to restore them to the input size; the semantic segmentation decoder includes two branches, and the first branch of the semantic segmentation decoder is a convolution kernel of 3×1 and 1×3 cascade, which is expressed as follows: F branch1 =conv 1×3 (conv 3×1 (F dec )) where F dec represents the input of the semantic segmentation decoder, F branch1 represents the output of the first branch of the semantic segmentation decoder; The second branch of the semantic segmentation decoder takes the output features of the first branch of the semantic segmentation decoder and the fusion of the input features of the semantic segmentation decoder as input, and outputs after passing through a depthwise separable convolutional layer with a convolution kernel of 3 and a dilation rate of 2 and a densely connected layer with a convolution kernel of 3 and a dilation rate of 3. The output of the second branch of the semantic segmentation decoder is represented by F branch2 ; Then the two branches of the semantic segmentation decoder are spliced ​​and fused from the channel dimension, and the channel information is fused through a convolution with a convolution kernel of 1×1. Step S24, designing a fusion module of the semantic segmentation decoder. For each stage of the semantic segmentation decoder, its input includes the output of the previous stage of the semantic segmentation decoder, the corresponding stage output of the DSM generator decoder, and the output of the corresponding stage of the feature extraction skeleton network.

3. The intelligent building extraction method based on remote sensing images according to claim 2 is characterized in that: The design of the building DSM estimation network in step S3 specifically includes the following steps: Step S31, designing the DSM generator decoder; The DSM generator decoder adopts a dual-branch fusion module, whose input is divided into channels after the input channel dimension is increased by 1×1 regular convolution, that is, F c / 2,1 、F c / 2,2 ; The first branch of the DSM generator decoder is a 3×3 regular convolution, and the input of the second branch of the DSM generator decoder is F c / 2,2 The channel fusion features of the first branch of the DSM generator decoder are output through 3×3 convolution; finally, the outputs of the two branches of the DSM generator decoder are added and fused; The channel dimension of each stage of the DSM generator decoder is half of the corresponding stage of the building semantic segmentation decoder; Step S32, design a fusion module of the DSM generator decoder. For each stage of the DSM generator decoder, its input includes the output of the previous stage of the DSM generator decoder, the corresponding stage output of the semantic segmentation decoder, and the output of the corresponding stage of the feature extraction skeleton network. Step S33, designing the DSM discriminator; The DSM discriminator is used to identify the authenticity of the DSM generated by the DSM generator, uses ResNet50 as the basic skeleton, and includes 5 stages, wherein the second stage, the fourth stage, and the fifth stage of the DSM discriminator all use a dual-branch feature extraction module, and the dual-branch feature extraction module implements channel dimension upgrading through a 1×1 conventional convolution on the input. The input of the first branch of the dual-branch feature extraction module is a 1 / 2 channel feature, and a 3×3 depth-separable convolution is used to implement feature extraction. The second branch of the dual-branch feature extraction module adds another 1 / 2 channel feature to the output of the first branch of the dual-branch feature extraction module as input, and outputs it after a depth-separable dilated convolution with a convolution kernel of 3 and a dilation rate of 2. Finally, the two branches of the dual-branch feature extraction module are channel-fused, and feature extraction is implemented through a 1×1 convolution.

4. The intelligent building extraction method based on remote sensing images according to claim 3 is characterized in that: The feature fusion and enhancement module design in step S4 includes the following steps: Step S41, designing the MASPP module after the feature extraction skeleton network output; The MASPP module adopts multi-branch parallelism. The input of the MASPP module is F pin , the first branch of the MASPP module is a conventional convolution with a convolution kernel of 1×1, and the output of the first branch of the MASPP module is F p1 ; The second branch of the MASPP module is a conventional convolution with a convolution kernel of 3×3, and the output of the second branch of the MASPP module is F p2 ; The input of the third branch of the MASPP module is F pin With F p2 The output of the third branch of the MASPP module can be expressed as follows: F p3_1 =Aconv 3，2 (AP3(F pin +F p2 )) F p3 =F p3_1+ F p3_2 +AP3(F pin +F p2 ) Among them, AP3 represents the average pooling with a kernel of 3, and Aconv 3，2 Aconv represents a dilated convolution with a convolution kernel of 3 and a dilation rate of 2. 3，3 It represents a dilated convolution with a convolution kernel of 3 and a dilation rate of 3. Indicates that features are fused based on channels; The input of the fourth branch of the MASPP module is F pin With F p3 The fusion output can be expressed as follows: F Fp4_1 =Aconv 3,4 (AP5(F pin +F p3 )) F p4 =F p4_1 +F p4_2 +AP5(Fp in +F p3 ) Among them, AP5 represents the average pooling with a kernel of 5, and Aconv 3，4 Aconv represents a dilated convolution with a kernel size of 3 and a dilation rate of 4. 3,5 It is represented as a dilated convolution with a convolution kernel of 3 and a dilation rate of 5; the fifth branch of the MASPP module is global average pooling, and the output of the fifth branch of the MASPP module is represented as F p5 ; The output of the parallel branch of the MASPP module uses channel fusion to combine F p1 、F p2 、F p3 、F p4 and F p5 Fusion is performed to achieve information fusion between channels through 1×1 convolution, so that the channel dimension is consistent with the module input, and the output is represented as F piout ; By strengthening the effective features through the channel attention mechanism, the output of the MASPP module is: F pout =F piout +SEB(F piout ) Where SEB(.) represents the fusion channel attention mechanism of mean pooling and maximum pooling; Step S42, designing the DSATT module; The input of the DSATT module is the output feature F of the semantic segmentation decoder seg And the output feature F of the DSM generator decoder at the corresponding stage dsm , the output process of the DSATT module can be expressed as follows: To seg =S(Linear(ReLU(Linear(GAP(Fse g ))))) To dsm =S(Linear(ReLU(Linear(GAP(F DSM ))))) Among them, F segout 、F dsmout represents the output after the DSATT module, GAP(.) represents global average pooling, Linear(.) represents linear change, S(.) represents activation function, Att seg With Att dsm Denote the channel attention weights of the output features of the semantic segmentation decoder and the channel attention weights of the output features of the DSM generator decoder, Att seg With Att dsm The sum is 1.

5. The intelligent building extraction method based on remote sensing images according to claim 4 is characterized in that: In the step S5, it specifically includes: Step S51, designing the loss function of the remote sensing image building semantic segmentation network, the total loss of the remote sensing image building semantic segmentation network includes pixel-level loss and regional loss, and the calculation formula of the pixel-level loss is as follows: Among them, y ic represents the true label at position i, expressed as one-hot. It is 1 when the true label is a building, otherwise it is 0. C is the number of categories, which is 2 here, and N is the total number of pixels in the predicted image. ic represents the probability of classifying sample i as category c; w c is the weight parameter, and the number of pixels of buildings in all samples of the training set is calculated to calculate w c , which is expressed as follows: The calculation formula of the area loss is as follows: Among them, y pred represents the probability of predicting category c, y true Represents the true one-hot label; The total loss of the remote sensing image building semantic segmentation network is: L seg =αL wce +βL softIoU ; Step S52: Design a loss function of the building DSM estimation network. The total loss of the building DSM estimation network includes adversarial loss and height estimation loss. The adversarial loss is expressed as follows: L G =-E(logD(G(x input ))) L D =-E(logD(x input_dsm ))-E(log(1-D(G(x iuput )))) L adv =L G +L D Where G and D represent the DSM generator encoder and the DSM generator decoder respectively, E(.) represents the mathematical expectation, x input_dsm represents the real DSM of the input remote sensing image, x input Represents the input remote sensing image; The height estimation loss is expressed as follows: The building DSM estimates the total network loss to be: L dsm =γ1L adv +γ2L h ; Among them, γ1 and γ2 represent the loss weights of adversarial loss and height estimation loss, respectively.

6. The intelligent building extraction method based on remote sensing images according to claim 1 is characterized in that: In step S1, the high-resolution dataset includes a Vaihingen dataset and a Potsdam dataset, and each sample in the high-resolution dataset includes an RGB image and its corresponding DSM image; the samples of the Vaihingen dataset and the Potsdam dataset are preprocessed and cropped into slices of 521 pixels×512 pixels.

7. An electronic device, characterized in that: include: One or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes the intelligent building extraction method based on remote sensing images as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, implement the intelligent building extraction method based on remote sensing images as described in any one of claims 1 to 6.

Images