Remote sensing image building extraction method fusing semantic and edge features

Abstract

The invention provides a remote sensing image building extraction method fusing semantic and edge features, and relates to the technical field of remote sensing image processing. On the basis of a semantic segmentation network HRNet, PSABlock is designed to reconstruct a backbone network, the backbone network serves as an image semantic information extraction branch, input features are decomposed into spatial components and channel components, spatial relations and channel dependency are captured respectively, and effective fusion of global and local features is achieved. According to the method, a module fusing multilayer edge features is designed as an edge feature extraction branch, an edge feature map of a building is obtained by generating an edge prediction result and performing weighted fusion, and an edge extraction effect is optimized through explicit edge loss. In addition, the invention provides a'main body-edge-fusion 'joint loss function. Compared with a semantic segmentation model of single feature extraction, the method has the advantages that the precision of the building extracted from the test data set is higher, the edge is more complete, and the requirements of practical application are met.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a remote sensing image building extraction method integrating semantics and edge features. Background Art

[0002] As a core component of urban space, the precise extraction of buildings is of great significance for urban planning, disaster management, and smart city construction. With the advancement of earth observation technology, the spatial resolution of remote sensing images has continued to increase. Their rich spatial details and precise geographic positioning provide new data support for the task of building extraction, but also bring new challenges. On the one hand, due to the diversity of structures and materials, different buildings have obvious intra-class differences. On the other hand, problems such as projection distortion of building clusters in complex urban scenes and spectral confusion of similar objects such as vegetation and roads also seriously restrict extraction accuracy. In recent years, the rapid development of deep learning technology has provided new solutions for the automated extraction of buildings. Thanks to the powerful feature expression capabilities of deep learning methods, deep learning semantic segmentation algorithms have achieved remarkable results in the task of building segmentation.

[0003] Although existing semantic segmentation algorithms have solved some problems to a certain extent, there are still the following deficiencies in extracting buildings from remote sensing images:

[0004] First, the existing network has insufficient receptive field and lacks the ability to understand complex scenes, resulting in loss of global information and insufficient segmentation and extraction accuracy;

[0005] Second, the detailed information of the building edges is easily lost during the network downsampling process, resulting in blurred and discontinuous building edges in the extracted results, affecting the integrity and accuracy of the extraction results.

[0006] Therefore, it is urgent to propose an improved method for building extraction, which can integrate multi-scale semantic features and multi-level edge information of buildings, thereby achieving accurate extraction of buildings in high-resolution images. Summary of the Invention

[0007] Purpose of the invention: To propose a remote sensing image building extraction method that integrates semantic and edge features to solve the above-mentioned problems existing in the prior art.

[0008] The present invention proposes a method for extracting buildings from remote sensing images by integrating semantic and edge features, comprising the following steps:

[0009] Step S1: Create a building semantic segmentation dataset and perform data enhancement, dividing the dataset into a training set, a validation set, and a test set;

[0010] Step S2: Design an improved HRNet branch network. Based on the semantic segmentation network HRNet, design a polarized self-attention module PSABlock and rebuild the backbone network as the image semantic information extraction branch.

[0011] Step S3: Design an edge detection module as an image edge feature extraction branch to extract and fuse edge information at different levels. By generating edge prediction results and weighted fusion, an edge feature map of the image is obtained. The edge extraction effect is optimized through explicit edge loss.

[0012] Step S4: constructing a high-resolution remote sensing image building semantic segmentation model HRCFNet, which simultaneously extracts multi-scale semantic information and multi-level edge information by connecting a semantic feature extraction branch and an edge feature extraction branch in parallel to form a dual-branch structure;

[0013] Step S5: Construct a joint loss function, input the training set and validation set obtained in step S1 into the dual-branch structure of step S4 for training, calculate based on the joint loss function, and iterate and optimize through forward and backward propagation to finally obtain the trained network model, thus completing the training of the building extraction model;

[0014] Step S6: Input the test set obtained in step S1 into the trained network, output the results after model inference, splice the output results, and convert them into vector format to complete the building extraction and perform accuracy assessment.

[0015] In a further embodiment, step S1 specifically includes:

[0016] Step S11: Create a vector file based on the remote sensing image of the target area, and draw vectors in sequence according to the spatial range of the building top surface in the image for sample label production;

[0017] Step S12: After the drawing is completed, the vector file is converted into a raster image consistent with the spatial range and resolution of the remote sensing image, and the pixel value of the building is set to 1 and the pixel value of the non-building is set to 0;

[0018] Step S13: Crop the high-resolution remote sensing image I and the label file T into an image of size 512×512 according to the same cropping size and naming rules, save the coordinate information, and construct the building semantic segmentation dataset D:

[0019] D={(I1,T1),(I2,T2),…,(I n ,T n )}

[0020] Step S14: perform data enhancement on the prepared dataset, including operations such as zooming in, zooming out, rotating, cropping, and color jittering, and divide the enhanced dataset into a training set, a validation set, and a test set in a ratio of 6:2:2.

[0021] In a further embodiment, step S2 specifically includes:

[0022] Based on the semantic segmentation network HRNet, a polarized self-attention module PSA Block is designed and the backbone network is rebuilt to obtain an improved HRNet as a semantic feature extraction branch. HRNet uses smaller convolution kernels and a shallow network structure, and there is a problem of insufficient receptive field. In complex scenes, the edge areas of large targets may need to be distinguished in combination with global semantic information, and insufficient receptive field will cause blurred or broken edges. The present invention redesigns a PSA Block to construct the backbone network of HRNet. The module uses three consecutive void convolution kernels (with expansion rates r of 1, 2 and 3 respectively) to expand the receptive field, and introduces a polarized self-attention mechanism in the module. PSA decomposes the input features into spatial components and channel components, capturing spatial relationships and channel dependencies respectively, thereby achieving an effective fusion of global and local features.

[0023] Specifically, let the input feature be Where C, H, and W are the number of channels, height, and width respectively;

[0024] Channel attention is calculated as:

[0025] A c =σ(W2·ReLU(W1·GAP(X)))

[0026] Where GAP(·) represents global average pooling; W1 and W2 represent learnable weights; σ(·) represents the Sigmoid activation function; and spatial attention is calculated as:

[0027] A s =σ(f(X))

[0028] Where f(·) represents the convolution operation;

[0029] The feature fusion of spatial components and channel components is expressed as:

[0030] X ′ =A c *X+A s *X

[0031] where X ′ Represents the spatial component A s and channel component A c The fused features.

[0032] In a further embodiment, in step S3, an edge detection module is designed as an edge feature extraction branch to extract and fuse edge information at different levels to obtain more building edge details. Although HRNet can capture detailed information, it has a weak response to tiny edges in complex scenes, which can easily lead to blurred edges. The module applies 1×1 convolution on each feature map to generate edge prediction results, and weightedly fuses the edge prediction results to obtain the final edge result. By making full use of multi-level edge features, the edge extraction capability of the network can be effectively improved. The prediction result S of the edge detection module at the mth edge prediction result is m Expressed as:

[0033] S m =σ(W m *F m +b m )

[0034] Among them F m is the feature map of the mth layer, W m and b m are the weight and bias of the 1×1 convolutional layer, and σ(·) is the activation function.

[0035] In a further embodiment, in step S4, the semantic feature extraction branch and the edge feature extraction branch are connected in parallel to form a dual-branch structure to construct a remote sensing image building extraction model HRCFNet, which simultaneously extracts multi-scale semantic information and multi-level edge information. Compared with the traditional semantic segmentation network, it can identify multi-scale buildings and detect more complete building edges. Specifically: the semantic feature extraction branch includes four stages, mainly composed of 4 Stages and 3 Transitions arranged alternately. Except for the first stage, each stage contains a Transition structure and a Stage structure. During the feature extraction process, the nth Stage will output n feature maps of different resolutions. The feature map before entering the first stage and the highest-dimensional feature map output by each Stage are input into the edge extraction module for edge prediction, and the prediction results are weighted fused. Finally, the output feature map of the semantic feature extraction branch and the output feature map of the edge feature extraction branch are fused to obtain the final feature map for building extraction.

[0036] In a further embodiment, step S5 specifically includes:

[0037] The dataset obtained in S1 is input into HRCFNet. During the training process, a random seed is set, the model weights are initialized for training, and a weight file is generated. At the same time, the model loss function is improved and iterative optimization is performed through forward and backward propagation until the optimal accuracy is achieved. Finally, the trained network model is obtained, completing the training of the building extraction model. The specific sub-steps include:

[0038] Step S51: The feature map before entering the first stage and the highest-dimensional feature map output by each stage enter the edge feature extraction branch at the same time as entering the next stage task, and the generated image edge prediction results are weighted fused to obtain the feature map of the building edge; the final output feature map of the semantic feature extraction branch and the building edge feature map obtained by the edge feature extraction branch are fused and input into the segmentation head for building extraction;

[0039] Step S52: The joint loss function of the building extraction task is composed of the edge extraction loss, semantic extraction loss, and weighted fusion loss. In the edge extraction task, the loss of each scale feature map is calculated separately and finally added with the weighted semantic extraction loss to calculate the loss of the entire training process. Through forward and backward propagation iterative optimization, the trained network model is finally obtained.

[0040] In a further embodiment, the loss function of each task is defined using binary cross entropy loss, which is expressed as:

[0041]

[0042] Where P and G represent the predicted label and the true label, respectively, and w and h correspond to the width and height of the image, respectively.

[0043] In a further embodiment, step S6 specifically includes:

[0044] Step S61: Input the test set obtained in step S1 into the trained model, output the results through model inference, use the coordinate information saved during cropping to perform splicing and write the coordinate information, convert it into vector format, and complete the building extraction;

[0045] Step S62: Based on the building extraction results, perform accuracy assessment on the four indicators of recall, precision, F1 score, and intersection over union (IoU):

[0046]

[0047] Where TP represents true positive examples, that is, the number of examples that are predicted to be in the current class and actually belong to the current class; FP represents false positive examples, that is, the number of examples that are predicted to be in the current class but actually do not belong to the current class; and FN represents false negative examples, that is, the number of examples that actually belong to the current class but are predicted to be in other classes.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] (1) To address the problems of insufficient receptive field, global information loss, and lack of understanding of complex scenes in existing remote sensing image building semantic segmentation networks, this paper designs PSA Block based on the semantic segmentation network HRNet to rebuild its backbone network and use it as an image semantic information extraction branch. This branch uses dilated convolution to expand the receptive field and introduces a polarized self-attention mechanism to decompose the input features into spatial components and channel components, respectively capturing spatial relationships and channel dependencies, achieving an effective fusion of global and local features, thereby improving the integrity and accuracy of feature extraction.

[0050] (2) In order to solve the problem that the detailed information of building edges is easily lost during the network downsampling process, resulting in blurred and discontinuous building edges in the extraction results, which affects the integrity and accuracy of the extraction, the present invention designs a module that fuses multiple layers of edge features as an edge feature extraction branch. By generating edge prediction results and weighted fusion, the edge feature map of the building is obtained, and the edge extraction effect is optimized through explicit edge loss.

[0051] (3) To enable the network to detect building edges more completely and accurately, the present invention constructs a high-resolution remote sensing image building extraction model HRCFNet based on the semantic information extraction branch and the edge feature extraction branch, and proposes a "subject-edge-fusion" joint loss function. The joint loss function is composed of a weighted combination of edge extraction loss, semantic extraction loss, and fusion loss. This enables the network to simultaneously extract edge shape information and semantic information at different scales, improves the network's ability to learn semantic and edge features, and enhances the network's building extraction accuracy and mapping effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a network model diagram of the remote sensing image building extraction method that integrates semantics and edge features proposed in this invention.

[0053] Figure 2 This is the structure diagram of the PSA polarized self-attention mechanism.

[0054] Figure 3 This is a performance comparison chart of the method of the present invention and the comparative method on the aerial remote sensing image dataset of Nanjing's main urban area.

[0055] Figure 4 This is a classification map of buildings extracted by the method of the present invention from an aerial remote sensing image dataset of Nanjing's main urban area.

[0056] Figure 5 This is a flowchart of the remote sensing image building extraction method that integrates semantics and edge features proposed in the present invention. DETAILED DESCRIPTION

[0057] In the following description, numerous specific details are provided to provide a more thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention can be practiced without one or more of these details. In other instances, certain technical features well known in the art have not been described to avoid confusion with the present invention.

[0058] The following will further clearly and completely describe the technical solution of the present invention through the embodiments of the present invention and in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0059] Example 1:

[0060] like Figures 1 to 3 As shown, this embodiment proposes a remote sensing image building extraction network that integrates semantic and edge features. The steps are as follows:

[0061] Step S1: Create a building semantic segmentation dataset and divide the dataset into a training set, a validation set, and a test set after data enhancement.

[0062] Step S2: Design an improved HRNet branch network. Based on the semantic segmentation network HRNet, design the PSA Block and rebuild the backbone network as the image semantic information extraction branch.

[0063] Step S3: Design an edge detection module as an image edge feature extraction branch to extract and fuse edge information at different levels. By generating edge prediction results and weighted fusion, an edge feature map of the image is obtained. The edge extraction effect is optimized through explicit edge loss.

[0064] Step S4: Constructing a high-resolution remote sensing image building semantic segmentation model HRCFNet. By connecting a semantic feature extraction branch and an edge feature extraction branch in parallel to form a dual-branch structure, multi-scale semantic information and multi-level edge information are simultaneously extracted. Compared with traditional semantic segmentation networks, it can identify multi-scale buildings and detect more complete building edges.

[0065] Step S5: Construct a joint loss function, input the training set and test set obtained in step S1 into the network model of step S4 for training, calculate based on the joint loss function and iteratively optimize through forward and backward propagation until the optimal accuracy is achieved, and finally obtain the trained network model to complete the training of the building extraction model;

[0066] Step S6: Input the test set obtained in step S1 into the trained network, output the results after model inference, splice the output results, convert the results into vector format, complete the building extraction, and perform accuracy evaluation including four indicators: Precision, Recall, IoU and F1 Score.

[0067] Example 2:

[0068] Based on the above embodiment 1, this embodiment optimizes the above technical solution and discloses some specific measures. The overall method flow chart is as follows: Figure 1 As shown, it specifically includes the following contents.

[0069] The dataset used in this example is an aerial remote sensing image dataset of Nanjing's main urban area. Its spatial resolution is 1 meter, and the image size is 20480×20480. The building density is high, representing a typical complex urban building distribution scene. The product data was acquired in 2020 and contains data from four bands. This implementation example only uses data from the RGB band. For ease of description, the product will be referred to as "Main City High Score 2020."

[0070] Step S1 is specifically as follows:

[0071] Step S11: Create a vector file based on the remote sensing image of the target area, and draw vectors in sequence according to the spatial range of the building top surface in the image for sample label production.

[0072] Step S12: After the drawing is completed, the vector file is converted into a raster image consistent with the spatial range and resolution of the remote sensing image, and the pixel value of the building is set to 1 and the pixel value of the non-building is set to 0;

[0073] Step S13: Crop the high-resolution remote sensing image I and the label file T into an image of size 512×512 according to the same cropping size and naming rules, save the coordinate information, and construct the building semantic segmentation dataset D = {(I1, T1), (I2, T2), …, (I n ,T n )};

[0074] Step S14: perform data enhancement on the prepared dataset, including operations such as zooming in, zooming out, rotating, cropping, and color jittering, and divide the amplified dataset into a training set, a validation set, and a test set in a ratio of 6:2:2.

[0075] Furthermore, step S2 is specifically as follows: based on the semantic segmentation network HRNet, a PSA Block is designed and the backbone network is rebuilt to obtain an improved HRNet as a semantic feature extraction branch. HRNet uses smaller convolution kernels and a shallow network structure, and there is a problem of insufficient receptive field. In complex scenes, the edge areas of large targets may need to be judged in combination with global semantic information, and insufficient receptive field will cause blurred or broken edges. The present invention redesigns a PSA Block to construct the backbone network of HRNet. The module uses three consecutive void convolution kernels (with expansion rates r of 1, 2, and 3, respectively) to expand the receptive field, and introduces the PSA polarization self-attention mechanism into the module. PSA decomposes the input features into spatial components and channel components, capturing spatial relationships and channel dependencies respectively, thereby achieving an effective fusion of global and local features. Specifically: suppose the input feature is Where C, H, and W are the number of channels, height, and width respectively. Channel attention is calculated as:

[0076] A c =σ(W2·ReLU(W1·GAP(X)))

[0077] Where GAP(·) represents global average pooling, W1 and W2 represent learnable weights, and σ(·) represents the Sigmoid activation function. The spatial attention is calculated as:

[0078] A s =σ(f(X))

[0079] where f(·) represents the convolution operation.

[0080] The feature fusion of spatial components and channel components can be expressed as:

[0081] X ′ =A c *X+A s *X

[0082] Furthermore, step S3 is specifically as follows: design an edge detection module as an edge feature extraction branch to extract and fuse edge information at different levels. Although HRNet can capture detailed information, it has a weak response to tiny edges in complex scenes, which easily leads to blurred edges. The module applies 1×1 convolution on each feature map to generate edge prediction results, and weightedly fuses each edge prediction result to obtain the final edge result. By making full use of multi-scale edge features, the edge extraction capability of the network can be effectively improved. Specifically: the edge detection module predicts the result S of the mth edge prediction result. m It can be expressed as:

[0083] S m =σ(W m *Fm +b m )

[0084] Among them F m is the feature map of the mth layer, W m and b m are the weight and bias of the 1×1 convolutional layer, and σ(·) is the activation function.

[0085] Furthermore, step S4 specifically involves connecting the semantic feature extraction branch and the edge feature extraction branch in parallel to form a dual-branch structure, constructing the remote sensing image building extraction model HRCFNet. This model simultaneously extracts multi-scale semantic information and multi-level edge information. Compared to traditional semantic segmentation networks, it can identify multi-scale buildings and detect more complete building edges. Specifically, the semantic feature extraction branch includes four stages, mainly composed of four stages and three transitions arranged in an alternating manner. Except for the first stage, each stage contains a transition structure and a stage structure. During the feature extraction process, the nth stage outputs n feature maps of different resolutions.

[0086] The Transition structure is responsible for generating low-resolution feature maps. Each time a Transition structure is passed, a new low-resolution feature map is generated by downsampling based on the original line. Therefore, the nth stage will output n feature maps of different resolutions. The Stage structure is responsible for the fusion of multi-scale features. The output feature map of the nth stage is converted into M={M1,…M n The upsampling or downsampling is cross-fused and fed into the Transition structure. The upsampling module in the Stage consists of a convolutional layer with a 1×1 kernel size, a batch normalization layer, and an upsample layer. Upsampling uses the nearest neighbor interpolation method by default. The downsampling module differs from the upsampling module in that each doubling of downsampling requires an additional convolutional layer with a stride of 2 and a 3×3 kernel size, a batch normalization layer, and a ReLU layer.

[0087] The feature maps before entering the first stage and the highest-dimensional feature maps output by each stage are input into the edge extraction module for edge prediction, and the prediction results are weighted and fused. Finally, the feature maps output by the semantic feature extraction branch and the edge extraction branch are fused to obtain the final feature map for building extraction.

[0088] Furthermore, step S5 is specifically as follows: inputting the dataset obtained in S1 into HRCFNet, setting a random seed during the training process, initializing the model weights for training, generating a weight file, and improving the model's loss function and optimizing iteratively through forward and backward propagation until the optimal accuracy is achieved. Finally, the trained network model is obtained, completing the training of the high-resolution image building extraction model. Specifically, it includes the following sub-steps:

[0089] In step S51, the feature map before entering the first stage and the highest-dimensional feature map output by each stage enter the edge feature extraction branch at the same time as entering the next stage task. The generated image edge prediction results are weighted fused to obtain the feature map of the building edge. The final output feature map of the semantic feature extraction branch and the building edge feature map obtained by the edge feature extraction branch are fused and input into the segmentation head for building extraction.

[0090] Step S52: The joint loss function of the building extraction task is composed of the edge extraction loss, semantic extraction loss, and fusion loss weighted. In the edge extraction task, the loss of each scale feature map is calculated separately and finally added with the weighted semantic extraction loss to calculate the loss of the entire training process. Through forward and backward propagation iterative optimization, the trained network model is finally obtained. In the present invention, the loss function of each task is defined using binary cross entropy loss, which can be expressed as:

[0091]

[0092] Where P and G represent the predicted label and the true label, respectively, and w and h correspond to the width and height of the image, respectively.

[0093] Furthermore, step S6 is specifically as follows:

[0094] Step S61: Input the test set obtained in step S1 into the model. After the model inference outputs the result, the coordinate information saved during cropping is used for splicing and the coordinate information is written. The format of the spliced ​​image is converted to obtain the building vectors of the target area, and the mapping is completed.

[0095] Step S62: Based on the building extraction results, perform accuracy assessment on the four indicators of Recall, Precision, F1 Score, and IoU. The calculation formulas for the four indicators are as follows:

[0096]

[0097] Where TP represents true positive examples, that is, the number of examples that are predicted to be in the current class and actually belong to the current class; FP represents false positive examples, that is, the number of examples that are predicted to be in the current class but actually do not belong to the current class; and FN represents false negative examples, that is, the number of examples that actually belong to the current class but are predicted to be in other classes.

[0098] Furthermore, Table 1 shows the accuracy evaluation of the comparison methods on the main city high score 2020 dataset.

[0099] Table 1 Accuracy evaluation of the comparison methods on the main city high score 2020 dataset

[0100]

[0101] Furthermore, Table 1 shows the performance indicators of each comparison method in the building extraction task. As can be seen from the table, HRCFNet shows significant advantages in all indicators, outperforming the traditional UNet and HRNet methods. Specifically, the Precision of HRCFNet reaches 93.99%, which is 1.44% higher than HRNet and 4.69% higher than UNet. At the same time, the Recall of HRCFNet reaches 93.90%, which is 1.41% higher than HRNet and 4.66% higher than UNet, achieving higher Precision and Recall. In addition, the IOU and F1score of HRCFNet are 88.34% and 93.94% respectively, which are 1.84% and 1.43% higher than HRNet, and 3.23% and 4.68% higher than UNet. Overall, HRCFNet surpasses existing methods in Precision, Recall and comprehensive performance, demonstrating stronger building extraction capabilities.

[0102] Further, Figure 3 The following shows some visualization results of the main city high score 2020 dataset. From left to right, the columns are the input image, the true value label, the prediction map of U-Net, HRNet and HRCFNet. Figure 3 (a) to Figure 3 (e) Representative buildings of different structural sizes. Figure 3 (a) is a high-rise building in the city, which has a special shape and is difficult to extract. Figure 3 (a) The buildings in the two boxes appear irregularly jagged. All three models extract the basic outlines of the buildings. HRCFNet has a finer-grained extraction at the edges, so the edges extracted by HRCFNet can also clearly express tiny corners.

[0103] Further, Figure 3 (b) Figure 3(c) is a low-rise building in the city. The houses are dense and the objects are small, but the houses are regular and uniform in shape, making the extraction easier. Therefore, all three models achieved high-precision results. However, due to the lack of edge feature fusion in UNet and HRNet, the extracted building edges are fuzzy and smooth, and clear and accurate building edges cannot be obtained. In addition, Figure 3 (b) Figure 3 (c) It can be seen that the building edges extracted by HRCFNet are more regular. Figure 3 For the buildings in the upper frame, HRCFNet can accurately segment the two buildings with adhered edges.

[0104] Further, Figure 3 (d) shows a typical large urban building. The U-Net model easily produces holes when extracting large buildings, and there are also many small spots. The edges of the buildings have a salt-and-pepper effect, and the edges of the buildings cannot be accurately restored. HRNet and HRCFNet can basically identify the subtle corners of large buildings, showing more building edge details and eliminating the internal holes of large buildings. However, they cannot distinguish some small edges inside the buildings.

[0105] Further, Figure 3 (e) shows a relatively sparse building scene with trees, buildings, and shadows. The U-Net model suffers from significant misclassification and omissions under interference from trees and shadows. When extracting adjacent buildings, significant edge adhesion occurs, preventing clear building edges and resulting in poor extraction of small target buildings. Compared to HRNet, HRCFNet exhibits slight over-extraction, which is an area worth improving.

[0106] Further, Figure 4 It shows some visualization results of the main city high score 2020 dataset. Figure 4 (a) is a screenshot of the remote sensing image of the Nanjing dataset. Figure 4 (b) is the building vector file extracted based on this method, which shows that the method of the present invention has certain stability and superiority in extracting buildings from remote sensing images, is beneficial to improving the quality and application value of the automatic mapping results of remote sensing image buildings, and can be used as an effective method for extracting buildings from remote sensing images.

[0107] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes may be made to it in form and detail without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A method for extracting buildings from remote sensing images by integrating semantic and edge features, characterized in that: The steps include: Step S1: Create a building semantic segmentation dataset and perform data enhancement, dividing the dataset into a training set, a validation set, and a test set; Step S2: Based on the semantic segmentation network HRNet, a polarized self-attention module PSABlock is designed and the backbone network is rebuilt as the image semantic information extraction branch; Step S3: Design an edge detection module as an image edge feature extraction branch to extract and fuse edge information at different levels. By generating edge prediction results and weighted fusion, an edge feature map of the image is obtained. The edge extraction effect is optimized through explicit edge loss. Step S4: constructing a high-resolution remote sensing image building semantic segmentation model HRCFNet, which simultaneously extracts multi-scale semantic information and multi-level edge information by connecting a semantic feature extraction branch and an edge feature extraction branch in parallel to form a dual-branch structure; Step S5: Construct a joint loss function, input the training set and validation set obtained in step S1 into the dual-branch structure of step S4 for training, calculate based on the joint loss function, and iterate and optimize through forward and backward propagation to finally obtain the trained network model, thus completing the training of the building extraction model; Step S6: Input the test set obtained in step S1 into the trained network, output the results after model inference, splice the output results, and convert them into vector format to complete the building extraction and perform accuracy assessment.

2. The remote sensing image building extraction method integrating semantics and edge features according to claim 1 is characterized in that: Step S1 specifically includes: Step S11: Create a vector file based on the remote sensing image of the target area, and draw vectors in sequence according to the spatial range of the building top surface in the image for sample label production; Step S12: After the drawing is completed, the vector file is converted into a raster image consistent with the spatial range and resolution of the remote sensing image, and the pixel value of the building is set to 1 and the pixel value of the non-building is set to 0; Step S13: Crop the high-resolution remote sensing image I and the label file T into an image of size 512×512 according to the same cropping size and naming rules, save the coordinate information, and construct the building semantic segmentation dataset D: D={(I1,T1),(I2,T2),…,(I n ,T n )} Step S14: perform data enhancement on the prepared dataset, including operations such as zooming in, zooming out, rotating, cropping, and color jittering, and divide the enhanced dataset into a training set, a validation set, and a test set in a ratio of 6:2:

2.

3. The remote sensing image building extraction method integrating semantics and edge features according to claim 1 is characterized in that: Step S2 specifically includes: Based on the semantic segmentation network HRNet, a polarized self-attention module PSABlock is designed and the backbone network is reconstructed to obtain an improved HRNet as the semantic feature extraction branch.

4. The remote sensing image building extraction method integrating semantics and edge features according to claim 3 is characterized in that: The polarized self-attention module PSABlock uses three consecutive dilation kernels with dilation rates of 1, 2, and 3 to expand the receptive field and introduces the polarized self-attention mechanism PSA; The polarized self-attention mechanism (PSA) decomposes the input features into spatial components and channel components, capturing spatial relationships and channel dependencies respectively, thereby achieving the fusion of global and local features.

5. The remote sensing image building extraction method integrating semantics and edge features according to claim 4 is characterized in that: The input features are Where C, H, and W are the number of channels, height, and width respectively; Channel attention is calculated as: TO c =σ(W2·ReLU(W1·GAP(X))) Where GAP(·) represents global average pooling; W1 and W2 represent learnable weights; σ(·) represents the Sigmoid activation function; The spatial attention is calculated as: A s =σ(f(X)) Where f(·) represents the convolution operation; The feature fusion of spatial components and channel components is expressed as: X ′ =A c *X+A s *X where X ′ Represents the spatial component A s and channel component A c The fused features.

6. The remote sensing image building extraction method integrating semantics and edge features according to claim 1 is characterized in that: In step S3, the edge detection module detects the prediction result S of the mth edge prediction result. m Expressed as: S m =σ(W m *F m +b m ) Among them F m is the feature map of the mth layer, W m and b m are the weight and bias of the 1×1 convolutional layer, and σ(·) is the activation function.

7. The method for extracting buildings from remote sensing images by integrating semantics and edge features according to claim 1, wherein: In step S4, the semantic feature extraction branch of the high-resolution remote sensing image building semantic segmentation model HRCFNet includes four stages, which are composed of 4 stages and 3 transitions arranged alternately; Except for the first stage, each stage contains a Transition structure and a Stage structure; During the feature extraction process, the nth stage outputs n feature maps of different resolutions; The feature map before entering the first stage and the highest-dimensional feature map output by each stage are input into the edge extraction module for edge prediction, and the prediction results are weighted fused; The output feature map of the semantic feature extraction branch and the output feature map of the edge feature extraction branch are fused to obtain the final feature map for building extraction.

8. The method for extracting buildings from remote sensing images by integrating semantics and edge features according to claim 7, wherein: Step S5 specifically includes: Step S51: The feature map before entering the first stage and the highest-dimensional feature map output by each stage enter the edge feature extraction branch at the same time as entering the next stage task, and the generated image edge prediction results are weighted fused to obtain the feature map of the building edge; the final output feature map of the semantic feature extraction branch and the building edge feature map obtained by the edge feature extraction branch are fused and input into the segmentation head for building extraction; Step S52: The joint loss function of the building extraction task is composed of the edge extraction loss, semantic extraction loss, and weighted fusion loss. In the edge extraction task, the loss of each scale feature map is calculated separately and finally added with the weighted semantic extraction loss to calculate the loss of the entire training process. Through forward and backward propagation iterative optimization, the trained network model is finally obtained.

9. The remote sensing image building extraction method integrating semantics and edge features according to claim 8, characterized in that: The loss function of each task uses binary cross entropy loss to define the loss function, which is expressed as: Where P and G represent the predicted label and the true label, respectively, and w and h correspond to the width and height of the image, respectively.

10. The remote sensing image building extraction method integrating semantics and edge features according to claim 1, characterized in that: Step S6 specifically includes: Step S61: Input the test set obtained in step S1 into the trained model, output the results through model inference, use the coordinate information saved during cropping to perform splicing and write the coordinate information, convert it into vector format, and complete the building extraction; Step S62: Based on the building extraction results, perform accuracy assessment on the four indicators of recall, precision, F1 score, and intersection over union (IoU): Where TP represents true positive examples, that is, the number of examples that are predicted to be in the current class and actually belong to the current class; FP represents false positive examples, that is, the number of examples that are predicted to be in the current class but actually do not belong to the current class; and FN represents false negative examples, that is, the number of examples that actually belong to the current class but are predicted to be in other classes.

Images