A post-disaster building image segmentation method based on boundary enhancement and feature perception

Abstract

A post-disaster building image segmentation method based on boundary enhancement and feature perception, which relates to the field of image segmentation processing, uses a dynamic perceptual encoder to extract features from remote sensing building images, introduces a Sobel operator for boundary enhancement, then fuses features using a feature-Sobel fusion module. Finally, an upsampling residual decoder is used to complete the post-disaster building segmentation. The model has improved stability, and due to the enhanced boundary features, it can easily process remote sensing images of severely damaged buildings after disasters.

Description

Technical Field

[0001] The present invention relates to the field of image segmentation processing, and in particular to a post-disaster building image segmentation method based on boundary enhancement and feature perception. Background Art

[0002] Natural disasters come in many forms, causing a wide range of damage to various types of buildings. Therefore, accurately assessing the damage to buildings is of great significance to my country and society. Traditional methods usually rely on manual assessment of building damage, which is time-consuming, labor-intensive, and a serious waste of resources. The rapid development of deep learning and artificial intelligence technologies has greatly promoted the segmentation and assessment of remote sensing building images. However, the damage to buildings caused by natural disasters varies, making remote sensing image segmentation still a challenging task. For example, remote sensing building images have complex contextual environments, including the ambiguity of building structure boundaries and the complexity of shapes, making it difficult to effectively segment remote sensing images. Summary of the Invention

[0003] In order to overcome the deficiencies of the above technologies, the present invention provides a method for analyzing images of various natural disaster scenes and providing emergency management departments with a reliable method for segmenting image data.

[0004] The technical solution adopted by the present invention to overcome the technical problems is:

[0005] A post-disaster building image segmentation method based on boundary enhancement and feature perception includes the following steps:

[0006] a) Obtaining post-disaster building remote sensing images I input ;

[0007] b) Establish a dynamic perceptual encoder network DAE to transform the post-disaster building remote sensing image I input Input into the dynamic perception encoder network DAE, and output the feature F dae1 , Feature F dae2 , Feature F dae3 , Feature F dae4 ; c) Remote sensing image of post-disaster buildings I input Perform preprocessing to obtain feature F sobel1 , Feature F sobel2 , Feature F sobel3 , Feature F sobel4 ;

[0008] d) Establish the feature-Sobel fusion encoder F-SFE, and transform the feature F dae1 , Feature F dae2 , Feature F dae3 , Feature F dae4 With feature F sobel1 , Feature Fsobel2 , Feature F sobel3 , Feature F sobel4 Input into the feature-Sobel fusion encoder F-SFE, and output the fusion feature F of image feature and Sobel feature fusion ;

[0009] e) Establish an upsampling residual decoder to fusion feature F fusion Input into the upsampled residual decoder, and output the post-disaster remote sensing building segmentation image I seg .

[0010] Furthermore, step b) comprises the following steps:

[0011] b-1) The dynamic perception encoder network DAE consists of a first dynamic perception module, a second dynamic perception module, a third dynamic perception module, and a fourth dynamic perception module. The first dynamic perception module, the second dynamic perception module, the third dynamic perception module, and the fourth dynamic perception module are all composed of a global mapping module, a local mapping module, and a Sigmoid activation function. The global mapping module is sequentially composed of a first convolutional layer, a first BatchNorm2d layer, a ReLU activation function, a second convolutional layer, and a second BatchNorm2d layer. The local mapping module is sequentially composed of an adaptive average pooling layer, a convolutional layer, a BatchNorm2d layer, and a ReLU activation function.

[0012] b-2) Post-disaster building remote sensing image I input Input into the global mapping module of the first dynamic perception module, and output the feature F G1 , the post-disaster building remote sensing image I input Input into the local mapping module of the first dynamic perception module, and output the feature F L1 , the feature F G1 With feature F L1 Perform the addition operation to obtain the feature F GL1 , the feature F GL1 Input into the Sigmoid activation function of the first dynamic perception module, and output the feature F sig1 , through formula F dae1 =F L1 *F sig1 +F G1 *(1-F sig1 ) Calculate the feature F dae1 ;

[0013] b-3) The feature F dae1 Input into the global mapping module of the second dynamic perception module, and output the feature F G2 , the feature F dae1Input into the local mapping module of the second dynamic perception module, and output the feature F L2 , the feature F G2 With feature F L2 Perform the addition operation to obtain the feature F GL2 , the feature F GL2 Input into the Sigmoid activation function of the second dynamic perception module, and output the feature F sig2 , through formula F dae2 =F L2 *F sig2 +F G2 *(1-F sig2 ) Calculate the feature F dae2 ;

[0014] b-4) Change the feature F dae2 Input into the global mapping module of the third dynamic perception module, and output the feature F G3 , the feature F dae2 Input into the local mapping module of the third dynamic perception module, and output the feature F L3 , the feature F G3 With feature F L3 Perform the addition operation to obtain the feature F GL3 , the feature F GL3 Input into the Sigmoid activation function of the third dynamic perception module, and output the feature F sig3 , through formula F dae3 =F L3 *F sig3 +F G3 *(1-F sig3 ) Calculate the feature F dae3 ;

[0015] b-5) Change the feature F dae3 Input into the global mapping module of the fourth dynamic perception module, and output the feature F G4 , the feature F dae3 Input into the local mapping module of the fourth dynamic perception module, and output the feature F L4 , the feature F G4 With feature F L4 Perform the addition operation to obtain the feature F GL4 , the feature F GL4 Input into the Sigmoid activation function of the fourth dynamic perception module, and output the feature F sig4 , through formula F dae4 =F L4 *F sig4 +F G4 *(1-F sig4 ) Calculate the feature Fdae4 .

[0016] Preferably, the convolution kernel size of the first convolution layer of the global mapping module of the first dynamic perception module, the second dynamic perception module, the third dynamic perception module, and the fourth dynamic perception module is 3×3, the step size is 1, and the padding is 1; the convolution kernel size of the second convolution layer of the global mapping module of the first dynamic perception module, the second dynamic perception module, the third dynamic perception module, and the fourth dynamic perception module is 3×3, the step size is 1, and the padding is 1; the convolution kernel size of the convolution layer of the local mapping module of the first dynamic perception module, the second dynamic perception module, the third dynamic perception module, and the fourth dynamic perception module is 1×1, the step size is 1, and the padding is 0.

[0017] Furthermore, step c) includes the following steps:

[0018] c-1) Extracting post-disaster building remote sensing images using the sobel() function in the OpenCV library input The Sobel image of the post-disaster building is obtained as image I sobel ;

[0019] c-2) Image I sobel Input into the stage1 module of the ResNet-50 network and output the feature F sobel1 ; c-3) Feature F sobel1 Input into the stage2 module of the ResNet-50 network and output the feature F sobel2 ;

[0020] c-4) Change the feature F sobel2 Input into the stage3 module of the ResNet-50 network and output the feature F sobel3 ;

[0021] c-5) Change the feature F sobel3 Input into the stage4 module of the ResNet-50 network and output the feature F sobel4 .

[0022] Furthermore, step d) comprises the following steps:

[0023] d-1) The feature-Sobel fusion encoder F-SFE consists of a first feature-Sobel fusion module, a second feature-Sobel fusion module, a third feature-Sobel fusion module, and a fourth feature-Sobel fusion module. The first feature-Sobel fusion module, the second feature-Sobel fusion module, the third feature-Sobel fusion module, and the fourth feature-Sobel fusion module all consist of a first convolutional layer, a second convolutional layer, a third convolutional layer, and a Sigmoid activation function.

[0024] d-2) Change feature F dae1 Input into the first convolutional layer of the first feature-Sobel fusion module, and output the feature F v1 , the feature F sobel1 Input to the second convolution layer of the first feature-Sobel fusion module, and output the feature F k1 , the feature F sobel1 Input to the third convolution layer of the first feature-Sobel fusion module, and output the feature F q1 , the feature F k1 With feature F v1 Perform the addition operation to obtain the feature F kv1 , the feature F kv1 Input into the Sigmoid activation function of the first feature-Sobel fusion module, and the output is F kv_sig1 , through formula F fusion1 =F q1 *(1+F kv_sig1 ) Calculate the feature F fusion1 ; d-3) Feature F dae2 Input into the first convolution layer of the second feature-Sobel fusion module, and output the feature F v2 , the feature F sobel2 Input to the second convolution layer of the second feature-Sobel fusion module, and output the feature F k2 , the feature F sobel2 Input to the third convolution layer of the second feature-Sobel fusion module, and output the feature F q2 , the feature F k2 With feature F v2 Perform the addition operation to obtain the feature F kv2 , the feature F fusion1 With feature F q2 Perform the addition operation to obtain the feature F fq2 , the feature F kv2 Input into the Sigmoid activation function of the second feature-Sobel fusion module, and the output is F kv_sig2 , through formula F fusion2 =F fq2 *(1+F kv_sig2 ) Calculate the feature F fusion2 ;

[0025] d-4) Change feature F dae3 Input into the first convolution layer of the third feature-Sobel fusion module, and output the feature F v3 , the feature F sobel3Input to the second convolution layer of the third feature-Sobel fusion module, and output the feature F k3 , the feature F sobel3 Input to the third convolution layer of the third feature-Sobel fusion module, and output the feature F q3 , the feature F k3 With feature F v3 Perform the addition operation to obtain the feature F kv3 , the feature F fusion2 With feature F q3 Perform the addition operation to obtain the feature F fq3 , the feature F kv3 Input into the Sigmoid activation function of the third feature-Sobel fusion module, and the output is F kv_sig3 , through formula F fusion3 =F fq3 *(1+F kv_sig3 ) Calculate the feature F fusion3 ;

[0026] d-5) Change feature F dae4 Input into the first convolution layer of the fourth feature-Sobel fusion module, and output the feature F v4 , the feature F sobel4 Input to the second convolution layer of the fourth feature-Sobel fusion module, and output the feature F k4 , the feature F sobel4 Input to the third convolution layer of the fourth feature-Sobel fusion module, and output the feature F q4 , the feature F k4 With feature F v4 Perform the addition operation to obtain the feature F kv4 , the feature F fusion3 With feature F q4 Perform the addition operation to obtain the feature F fq4 , the feature F kv4 Input into the Sigmoid activation function of the fourth feature - Sobel fusion module, and the output is F kv_sig4 , through formula F fusion =F fq4 *(1+F kv_sig4 ) Calculate the feature F fusion .

[0027] Preferably, the convolution kernel size of the first convolution layer of the first feature-Sobel fusion module, the second feature-Sobel fusion module, the third feature-Sobel fusion module, and the fourth feature-Sobel fusion module are all 3×3, the step size is 1, and the padding is 1; the convolution kernel size of the second convolution layer of the first feature-Sobel fusion module, the second feature-Sobel fusion module, the third feature-Sobel fusion module, and the fourth feature-Sobel fusion module are all 3×3, the step size is 1, and the padding is 1; the convolution kernel size of the third convolution layer of the first feature-Sobel fusion module, the second feature-Sobel fusion module, the third feature-Sobel fusion module, and the fourth feature-Sobel fusion module are all 3×3, the step size is 1, and the padding is 1.

[0028] Furthermore, step e) comprises the following steps:

[0029] e-1) The upsampling residual decoder is composed of a first upsampling residual module, a second upsampling residual module, a third upsampling residual module, a fourth upsampling residual module, and a convolutional layer. The first upsampling residual module, the second upsampling residual module, the third upsampling residual module, and the fourth upsampling residual module are all composed of a first upsampling branch module and a second upsampling branch module. The first upsampling branch module is sequentially composed of a first ReLU activation function, an upsampling layer, a convolutional layer, a BatchNorm2d layer, and a second ReLU activation function. The second upsampling branch module is sequentially composed of a convolutional layer and a ReLU activation function.

[0030] e-2) Fusion feature F fusion Input into the first upsampling branch module of the first upsampling residual module, and output the feature F up1_1 , the fusion feature F fusion Input to the second upsampling branch module of the first upsampling residual module, and output the feature F up1_2 , the feature F up1_1 With feature F up1_2 Perform the addition operation to obtain the feature F up1 ;

[0031] e-3) The feature F up1 Input to the first upsampling branch module of the second upsampling residual module, and output the feature F up2_1 , the feature F up1 Input to the second upsampling branch module of the second upsampling residual module, and output the feature F up2_2 , the feature F up2_1 With feature F up2_2 Perform the addition operation to obtain the feature F up2 ;

[0032] e-4) Change the feature F up2 Input to the first upsampling branch module of the third upsampling residual module, and output the feature F up3_1 , the feature F up2 Input to the second upsampling branch module of the third upsampling residual module, and output the feature F up3_2 , the feature F up3_1 With feature F up3_2 Perform the addition operation to obtain the feature F up3 ;

[0033] e-5) Change feature F up3 Input to the first upsampling branch module of the fourth upsampling residual module, and output the feature F up4_1 , the feature F up3 Input to the second upsampling branch module of the fourth upsampling residual module, and output the feature F up4_2 , the feature F up4_1 With feature F up4_2 Perform the addition operation to obtain the feature F up ;

[0034] e-6) Change feature F up Input into the convolution layer of the upsampling residual decoder, and output the post-disaster remote sensing building segmentation image I seg .

[0035] Preferably, the convolution kernel size of the convolution layer of the upsampling residual decoder is 1×1, the step size is 1, and the padding is 0; the convolution kernel size of the convolution layer of the first upsampling residual module, the second upsampling residual module, the third upsampling residual module, and the first upsampling branch module of the fourth upsampling residual module are all 3×3, the step size is 1, and the padding is 0; the convolution kernel size of the convolution layer of the second upsampling branch module of the first upsampling residual module, the second upsampling residual module, the third upsampling residual module, and the fourth upsampling residual module are all 3×3, the step size is 1, and the padding is 0.

[0036] The beneficial effects of the present invention are as follows: first, the features of the post-disaster building remote sensing image are effectively extracted through the dynamic perception encoder, and the post-disaster building remote sensing image is processed by the Sobel operator to obtain the post-disaster building Sobel image, and the semantic features of the image are extracted through the pre-trained ResNet-50 network. Secondly, a feature-Sobel fusion encoder is established to complete the fusion of Sobel features and image features. Finally, the upsampling of features is completed by the upsampling residual encoder, and the post-disaster building segmentation image is generated at the same time. Compared with other methods, this model has better stability. At the same time, due to the enhancement of boundary features, the model can easily process severely damaged post-disaster building remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of the method of the present invention;

[0038] Figure 2 1 is a structural diagram of the dynamic perceptual encoder network DAE of the present invention;

[0039] Figure 3 This is a structural diagram of the feature-Sobel fusion encoder F-SFE of the present invention;

[0040] Figure 4 2 is a structural diagram of the upsampling residual decoder of the present invention. DETAILED DESCRIPTION

[0041] The following is combined with Figure 1 To the attached Figure 4 The present invention is further described.

[0042] A post-disaster building image segmentation method based on boundary enhancement and feature perception includes the following steps:

[0043] a) Obtaining post-disaster building remote sensing images I input .

[0044] b) Establish a dynamic perceptual encoder network DAE to transform the post-disaster building remote sensing image I input Input into the dynamic perception encoder network DAE, and output the feature F dae1 , Feature F dae2 , Feature F dae3 , Feature F dae4 c) Remote sensing images of buildings after disasters input Perform preprocessing to obtain feature F sobel1 , Feature F sobel2 , Feature F sobel3 , Feature F sobel4 .

[0045] d) Establish the feature-Sobel fusion encoder F-SFE, and transform the feature F dae1 , Feature F dae2 , Feature F dae3 , Feature F dae4 With feature F sobel1 , Feature F sobel2 , Feature F sobel3 , Feature F sobel4 Input into the feature-Sobel fusion encoder F-SFE, and output the fusion feature F of image feature and Sobel feature fusion .

[0046] e) Establish an upsampling residual decoder to fusion feature F fusionInput into the upsampled residual decoder, and output the post-disaster remote sensing building segmentation image I seg .

[0047] First, to address the complex context of post-disaster building images, the present invention designs a multi-level dynamic perception encoder. The most critical component is the dynamic perception module, which effectively extracts features from remotely sensed building images through two special feature mapping networks. Secondly, a boundary enhancement module is designed, which introduces the Sobel operator to guide the model's focus on the edges of remotely sensed building images. Furthermore, a feature-Sobel fusion module is designed to effectively fuse the features of remotely sensed and Sobel images, thereby improving the model's segmentation accuracy. Multiple experiments demonstrate that the proposed model can effectively perform image segmentation tasks, analyze images from various natural disaster scenarios, and provide reliable segmented image data for emergency management departments.

[0048] In one embodiment of the present invention, step b) comprises the following steps:

[0049] b-1) The dynamic perception encoder network DAE consists of a first dynamic perception module, a second dynamic perception module, a third dynamic perception module, and a fourth dynamic perception module. The first dynamic perception module, the second dynamic perception module, the third dynamic perception module, and the fourth dynamic perception module are all composed of a global mapping module, a local mapping module, and a Sigmoid activation function. The global mapping module is sequentially composed of a first convolutional layer, a first BatchNorm2d layer, a ReLU activation function, a second convolutional layer, and a second BatchNorm2d layer. The local mapping module is sequentially composed of an adaptive average pooling layer, a convolutional layer, a BatchNorm2d layer, and a ReLU activation function.

[0050] b-2) Post-disaster building remote sensing image I input Input into the global mapping module of the first dynamic perception module, and output the feature F G1 , the post-disaster building remote sensing image I input Input into the local mapping module of the first dynamic perception module, and output the feature F L1 , the feature F G1 With feature F L1 Perform the addition operation to obtain the feature F GL1 , the feature F GL1 Input into the Sigmoid activation function of the first dynamic perception module, and output the feature F sig1 , through formula F dae1 =F L1 *F sig1 +F G1 *(1-F sig1 ) Calculate the feature F dae1 ;

[0051] b-3) Change the feature F dae1 Input into the global mapping module of the second dynamic perception module, and output the feature F G2 , the feature F dae1 Input into the local mapping module of the second dynamic perception module, and output the feature F L2 , the feature F G2 With feature F L2 Perform the addition operation to obtain the feature F GL2 , the feature F GL2 Input into the Sigmoid activation function of the second dynamic perception module, and output the feature F sig2 , through formula F dae2 =F L2 *F sig2 +F G2 *(1-F sig2 ) Calculate the feature F dae2 ;

[0052] b-4) Change the feature F dae2 Input into the global mapping module of the third dynamic perception module, and output the feature F G3 , the feature F dae2 Input into the local mapping module of the third dynamic perception module, and output the feature F L3 , the feature F G3 With feature F L3 Perform the addition operation to obtain the feature F GL3 , the feature F GL3 Input into the Sigmoid activation function of the third dynamic perception module, and output the feature F sig3 , through formula F dae3 =F L3 *F sig3 +F G3 *(1-F sig3 ) Calculate the feature F dae3 ;

[0053] b-5) Change the feature F dae3 Input into the global mapping module of the fourth dynamic perception module, and output the feature F G4 , the feature F dae3 Input into the local mapping module of the fourth dynamic perception module, and output the feature F L4 , the feature F G4 With feature F L4 Perform the addition operation to obtain the feature F GL4 , the feature F GL4 Input into the Sigmoid activation function of the fourth dynamic perception module, and output the feature Fsig4 , through formula F dae4 =F L4 *F sig4 +F G4 *(1-F sig4 ) Calculate the feature F dae4 .

[0054] In this embodiment, the convolution kernel size of the first convolution layer of the global mapping module of the first dynamic perception module, the second dynamic perception module, the third dynamic perception module, and the fourth dynamic perception module is 3×3, the step size is 1, and the padding is 1; the convolution kernel size of the second convolution layer of the global mapping module of the first dynamic perception module, the second dynamic perception module, the third dynamic perception module, and the fourth dynamic perception module is 3×3, the step size is 1, and the padding is 1; the convolution kernel size of the convolution layer of the local mapping module of the first dynamic perception module, the second dynamic perception module, the third dynamic perception module, and the fourth dynamic perception module is 1×1, the step size is 1, and the padding is 0.

[0055] In one embodiment of the present invention, step c) comprises the following steps:

[0056] c-1) Extracting post-disaster building remote sensing images using the sobel() function in the OpenCV library input The Sobel image of the post-disaster building is obtained as image I sobel ;

[0057] c-2) Image I sobel Input into the stage1 module of the ResNet-50 network and output the feature F sobel1 ; c-3) Feature F sobel1 Input into the stage2 module of the ResNet-50 network and output the feature F sobel2 ;

[0058] c-4) Change the feature F sobel2 Input into the stage3 module of the ResNet-50 network and output the feature F sobel3 ;

[0059] c-5) Change the feature F sobel3 Input into the stage4 module of the ResNet-50 network and output the feature F sobel4 .

[0060] In one embodiment of the present invention, step d) comprises the following steps:

[0061] d-1) The feature-Sobel fusion encoder F-SFE consists of a first feature-Sobel fusion module, a second feature-Sobel fusion module, a third feature-Sobel fusion module, and a fourth feature-Sobel fusion module. The first feature-Sobel fusion module, the second feature-Sobel fusion module, the third feature-Sobel fusion module, and the fourth feature-Sobel fusion module all consist of a first convolutional layer, a second convolutional layer, a third convolutional layer, and a Sigmoid activation function.

[0062] d-2) Change feature F dae1 Input into the first convolutional layer of the first feature-Sobel fusion module, and output the feature F v1 , the feature F sobel1 Input to the second convolution layer of the first feature-Sobel fusion module, and output the feature F k1 , the feature F sobel1 Input to the third convolution layer of the first feature-Sobel fusion module, and output the feature F q1 , the feature F k1 With feature F v1 Perform the addition operation to obtain the feature F kv1 , the feature F kv1 Input into the Sigmoid activation function of the first feature-Sobel fusion module, and the output is F kv_sig1 , through formula F fusion1 =F q1 *(1+F kv_sig1 ) Calculate the feature F fusion1 ; d-3) Feature F dae2 Input into the first convolution layer of the second feature-Sobel fusion module, and output the feature F v2 , the feature F sobel2 Input to the second convolution layer of the second feature-Sobel fusion module, and output the feature F k2 , the feature F sobel2 Input to the third convolution layer of the second feature-Sobel fusion module, and output the feature F q2 , the feature F k2 With feature F v2 Perform the addition operation to obtain the feature F kv2 , the feature F fusion1 With feature F q2 Perform the addition operation to obtain the feature F fq2 , the feature F kv2 Input into the Sigmoid activation function of the second feature-Sobel fusion module, and the output is F kv_sig2 , through formula F fusion2 =Ffq2 *(1+F kv_sig2 ) Calculate the feature F fusion2 ;

[0063] d-4) Change feature F dae3 Input into the first convolution layer of the third feature-Sobel fusion module, and output the feature F v3 , the feature F sobel3 Input to the second convolution layer of the third feature-Sobel fusion module, and output the feature F k3 , the feature F sobel3 Input to the third convolution layer of the third feature-Sobel fusion module, and output the feature F q3 , the feature F k3 With feature F v3 Perform the addition operation to obtain the feature F kv3 , the feature F fusion2 With feature F q3 Perform the addition operation to obtain the feature F fq3 , the feature F kv3 Input into the Sigmoid activation function of the third feature-Sobel fusion module, and the output is F kv_sig3 , through formula F fusion3 =F fq3 *(1+F kv_sig3 ) Calculate the feature F fusion3 ;

[0064] d-5) Change feature F dae4 Input into the first convolution layer of the fourth feature-Sobel fusion module, and output the feature F v4 , the feature F sobel4 Input to the second convolution layer of the fourth feature-Sobel fusion module, and output the feature F k4 , the feature F sobel4 Input to the third convolution layer of the fourth feature-Sobel fusion module, and output the feature F q4 , the feature F k4 With feature F v4 Perform the addition operation to obtain the feature F kv4 , the feature F fusion3 With feature F q4 Perform the addition operation to obtain the feature F fq4 , the feature F kv4 Input into the Sigmoid activation function of the fourth feature - Sobel fusion module, and the output is F kv_sig4 , through formula F fusion =F fq4 *(1+F kv_sig4 ) Calculate the feature Ffusion .

[0065] In this embodiment, the convolution kernel size of the first convolution layer of the first feature-Sobel fusion module, the second feature-Sobel fusion module, the third feature-Sobel fusion module, and the fourth feature-Sobel fusion module is 3×3, the step size is 1, and the padding is 1; the convolution kernel size of the second convolution layer of the first feature-Sobel fusion module, the second feature-Sobel fusion module, the third feature-Sobel fusion module, and the fourth feature-Sobel fusion module is 3×3, the step size is 1, and the padding is 1; the convolution kernel size of the third convolution layer of the first feature-Sobel fusion module, the second feature-Sobel fusion module, the third feature-Sobel fusion module, and the fourth feature-Sobel fusion module is 3×3, the step size is 1, and the padding is 1.

[0066] In one embodiment of the present invention, step e) comprises the following steps:

[0067] e-1) The upsampling residual decoder is composed of a first upsampling residual module, a second upsampling residual module, a third upsampling residual module, a fourth upsampling residual module, and a convolutional layer. The first upsampling residual module, the second upsampling residual module, the third upsampling residual module, and the fourth upsampling residual module are all composed of a first upsampling branch module and a second upsampling branch module. The first upsampling branch module is sequentially composed of a first ReLU activation function, an upsampling layer, a convolutional layer, a BatchNorm2d layer, and a second ReLU activation function. The second upsampling branch module is sequentially composed of a convolutional layer and a ReLU activation function.

[0068] e-2) Fusion feature F fusion Input into the first upsampling branch module of the first upsampling residual module, and output the feature F up1_1 , the fusion feature F fusion Input to the second upsampling branch module of the first upsampling residual module, and output the feature F up1_2 , the feature F up1_1 With feature F up1_2 Perform the addition operation to obtain the feature F up1 ;

[0069] e-3) The feature F up1 Input to the first upsampling branch module of the second upsampling residual module, and output the feature F up2_1 , the feature F up1 Input to the second upsampling branch module of the second upsampling residual module, and output the feature F up2_2 , the feature F up2_1 With feature F up2_2Perform the addition operation to obtain the feature F up2 ;

[0070] e-4) Change the feature F up2 Input to the first upsampling branch module of the third upsampling residual module, and output the feature F up3_1 , the feature F up2 Input to the second upsampling branch module of the third upsampling residual module, and output the feature F up3_2 , the feature F up3_1 With feature F up3_2 Perform the addition operation to obtain the feature F up3 ;

[0071] e-5) Change feature F up3 Input to the first upsampling branch module of the fourth upsampling residual module, and output the feature F up4_1 , the feature F up3 Input to the second upsampling branch module of the fourth upsampling residual module, and output the feature F up4_2 , the feature F up4_1 With feature F up4_2 Perform the addition operation to obtain the feature F up ;

[0072] e-6) Change feature F up Input into the convolution layer of the upsampling residual decoder, and output the post-disaster remote sensing building segmentation image I seg .

[0073] In this embodiment, the convolution kernel size of the convolution layer of the upsampling residual decoder is 1×1, the step size is 1, and the padding is 0; the convolution kernel size of the convolution layer of the first upsampling residual module, the second upsampling residual module, the third upsampling residual module, and the first upsampling branch module of the fourth upsampling residual module are all 3×3, the step size is 1, and the padding is 0; the convolution kernel size of the convolution layer of the second upsampling branch module of the first upsampling residual module, the second upsampling residual module, the third upsampling residual module, and the fourth upsampling residual module are all 3×3, the step size is 1, and the padding is 0.

[0074] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A post-disaster building image segmentation method based on boundary enhancement and feature perception, characterized in that: The steps include: a) Obtaining post-disaster building remote sensing images I input ; b) Establish a dynamic perceptual encoder network DAE to transform the post-disaster building remote sensing image I input Input into the dynamic perception encoder network DAE, and output the feature F dae1 , Feature F dae2 , Feature F dae3 , Feature F dae4 ; c) Remote sensing images of buildings after disasters I input Perform preprocessing to obtain feature F sobel1 , Feature F sobel2 , Feature F sobel3 , Feature F sobel4 ; d) Establish the feature-Sobel fusion encoder F-SFE, and transform the feature F dae1 , Feature F dae2 , Feature F dae3 , Feature F dae4 With feature F sobel1 , Feature F sobel2 , Feature F sobel3 , Feature F sobel4 Input into the feature-Sobel fusion encoder F-SFE, and output the fusion feature F of image feature and Sobel feature fusion ; e) Establish an upsampling residual decoder to fusion feature F fusion Input into the upsampled residual decoder, and output the post-disaster remote sensing building segmentation image I seg ; Step d) comprises the following steps: d-1) The feature-Sobel fusion encoder F-SFE consists of a first feature-Sobel fusion module, a second feature-Sobel fusion module, a third feature-Sobel fusion module, and a fourth feature-Sobel fusion module. The first feature-Sobel fusion module, the second feature-Sobel fusion module, the third feature-Sobel fusion module, and the fourth feature-Sobel fusion module all consist of a first convolutional layer, a second convolutional layer, a third convolutional layer, and a Sigmoid activation function. d-2) Change feature F dae1 Input into the first convolutional layer of the first feature-Sobel fusion module, and output the feature F v1 , the feature F sobel1 Input to the second convolution layer of the first feature-Sobel fusion module, and output the feature F k1 , the feature F sobel1 Input to the third convolution layer of the first feature-Sobel fusion module, and output the feature F q1 , the feature F k1 With feature F v1 Perform the addition operation to obtain the feature F kv1 , the feature F kv1 Input into the Sigmoid activation function of the first feature-Sobel fusion module, and the output is F kv_sig1 , through formula F fusion1 =F q1 *(1+F kv_sig1 ) Calculate the feature F fusion1 ; d-3) Change feature F dae2 Input into the first convolution layer of the second feature-Sobel fusion module, and output the feature F v2 , the feature F sobel2 Input to the second convolution layer of the second feature-Sobel fusion module, and output the feature F k2 , the feature F sobel2 Input to the third convolution layer of the second feature-Sobel fusion module, and output the feature F q2 , the feature F k2 With feature F v2 Perform the addition operation to obtain the feature F kv2 , the feature F fusion1 With feature F q2 Perform the addition operation to obtain the feature F fq2 , the feature F kv2 Input into the Sigmoid activation function of the second feature-Sobel fusion module, and the output is F kv_sig2 , through formula F fusion2 =F fq2 *(1+F kv_sig2 ) Calculate the feature F fusion2 ; d-4) Change feature F dae3 Input into the first convolution layer of the third feature-Sobel fusion module, and output the feature F v3 , the feature F sobel3 Input to the second convolution layer of the third feature-Sobel fusion module, and output the feature F k3 , the feature F sobel3 Input to the third convolution layer of the third feature-Sobel fusion module, and output the feature F q3 , the feature F k3 With feature F v3 Perform the addition operation to obtain the feature F kv3 , the feature F fusion2 With feature F q3 Perform the addition operation to obtain the feature F fq3 , the feature F kv3 Input into the Sigmoid activation function of the third feature-Sobel fusion module, and the output is F kv_sig3 , through formula F fusion3 =F fq3 *(1+F kv_sig3 ) Calculate the feature F fusion3 ; d-5) Change feature F dae4 Input into the first convolution layer of the fourth feature-Sobel fusion module, and output the feature F v4 , the feature F sobel4 Input to the second convolution layer of the fourth feature-Sobel fusion module, and output the feature F k4 , the feature F sobel4 Input to the third convolution layer of the fourth feature-Sobel fusion module, and output the feature F q4 , the feature F k4 With feature F v4 Perform the addition operation to obtain the feature F kv4 , the feature F fusion3 With feature F q4 Perform the addition operation to obtain the feature F fq4 , the feature F kv4 Input into the Sigmoid activation function of the fourth feature - Sobel fusion module, and the output is F kv_sig4 , through formula F fusion =F fq4 *(1+F kv_sig4 ) Calculate the feature F fusion .

2. The post-disaster building image segmentation method based on boundary enhancement and feature perception according to claim 1 is characterized in that: Step b) comprises the following steps: b-1) The dynamic perception encoder network DAE consists of a first dynamic perception module, a second dynamic perception module, a third dynamic perception module, and a fourth dynamic perception module. The first dynamic perception module, the second dynamic perception module, the third dynamic perception module, and the fourth dynamic perception module are all composed of a global mapping module, a local mapping module, and a Sigmoid activation function. The global mapping module is sequentially composed of a first convolutional layer, a first BatchNorm2d layer, a ReLU activation function, a second convolutional layer, and a second BatchNorm2d layer. The local mapping module is sequentially composed of an adaptive average pooling layer, a convolutional layer, a BatchNorm2d layer, and a ReLU activation function. b-2) Post-disaster building remote sensing image I input Input into the global mapping module of the first dynamic perception module, and output the feature F G1 , the post-disaster building remote sensing image I input Input into the local mapping module of the first dynamic perception module, and output the feature F L1 , the feature F G1 With feature F L1 Perform the addition operation to obtain the feature F GL1 , the feature F GL1 Input into the Sigmoid activation function of the first dynamic perception module, and output the feature F sig1 , through formula F dae1 =F L1 *F sig1 +F G1 *(1-F sig1 ) Calculate the feature F dae1 ; b-3) Change the feature F dae1 Input into the global mapping module of the second dynamic perception module, and output the feature F G2 , the feature F dae1 Input into the local mapping module of the second dynamic perception module, and output the feature F L2 , the feature F G2 With feature F L2 Perform the addition operation to obtain the feature F GL2 , the feature F GL2 Input into the Sigmoid activation function of the second dynamic perception module, and output the feature F sig2 , through formula F dae2 =F L2 *F sig2 +F G2 *(1-F sig2 ) Calculate the feature F dae2 ; b-4) Change the feature F dae2 Input into the global mapping module of the third dynamic perception module, and output the feature F G3 , the feature F dae2 Input into the local mapping module of the third dynamic perception module, and output the feature F L3 , the feature F G3 With feature F L3 Perform the addition operation to obtain the feature F GL3 , the feature F GL3 Input into the Sigmoid activation function of the third dynamic perception module, and output the feature F sig3 , through formula F dae3 =F L3 *F sig3 +F G3 *(1-F sig3 ) Calculate the feature F dae3 ; b-5) Feature F dae3 Input into the global mapping module of the fourth dynamic perception module, and output the feature F G4 , the feature F dae3 Input into the local mapping module of the fourth dynamic perception module, and output the feature F L4 , the feature F G4 With feature F L4 Perform the addition operation to obtain the feature F GL4 , the feature F GL4 Input into the Sigmoid activation function of the fourth dynamic perception module, and output the feature F sig4 , through formula F dae4 =F L4 *F sig4 +F G4 *(1-F sig4 ) Calculate the feature F dae4 .

3. The post-disaster building image segmentation method based on boundary enhancement and feature perception according to claim 2, characterized in that: The convolution kernel size of the first convolution layer of the global mapping module of the first dynamic perception module, the second dynamic perception module, the third dynamic perception module, and the fourth dynamic perception module is 3×3, the step size is 1, and the padding is 1; the convolution kernel size of the second convolution layer of the global mapping module of the first dynamic perception module, the second dynamic perception module, the third dynamic perception module, and the fourth dynamic perception module is 3×3, the step size is 1, and the padding is 1; the convolution kernel size of the convolution layer of the local mapping module of the first dynamic perception module, the second dynamic perception module, the third dynamic perception module, and the fourth dynamic perception module is 1×1, the step size is 1, and the padding is 0.

4. The post-disaster building image segmentation method based on boundary enhancement and feature perception according to claim 1, characterized in that: Step c) comprises the following steps: c-1) Extracting post-disaster building remote sensing images using the sobel() function in the OpenCV library input The Sobel image of the post-disaster building is obtained as image I sobel ; c-2) Image I sobel Input into the stage1 module of the ResNet-50 network and output the feature F sobel1 ; c-3) Feature F sobel1 Input into the stage2 module of the ResNet-50 network and output the feature F sobel2 ; c-4) Change the feature F sobel2 Input into the stage3 module of the ResNet-50 network and output the feature F sobel3 ; c-5) Change the feature F sobel3 Input into the stage4 module of the ResNet-50 network and output the feature F sobel4 .

5. The post-disaster building image segmentation method based on boundary enhancement and feature perception according to claim 1, characterized in that: The convolution kernel size of the first convolution layer of the first feature-Sobel fusion module, the second feature-Sobel fusion module, the third feature-Sobel fusion module, and the fourth feature-Sobel fusion module are all 3×3, the stride is 1, and the padding is 1; The convolution kernel size of the second convolution layer of the first feature-Sobel fusion module, the second feature-Sobel fusion module, the third feature-Sobel fusion module, and the fourth feature-Sobel fusion module are all 3×3, the stride is 1, and the padding is 1; The convolution kernel size of the third convolution layer of the first feature-Sobel fusion module, the second feature-Sobel fusion module, the third feature-Sobel fusion module, and the fourth feature-Sobel fusion module are all 3×3, the stride is 1, and the padding is 1.

6. The post-disaster building image segmentation method based on boundary enhancement and feature perception according to claim 1, characterized in that: Step e) comprises the following steps: e-1) The upsampling residual decoder is composed of a first upsampling residual module, a second upsampling residual module, a third upsampling residual module, a fourth upsampling residual module, and a convolutional layer. The first upsampling residual module, the second upsampling residual module, the third upsampling residual module, and the fourth upsampling residual module are all composed of a first upsampling branch module and a second upsampling branch module. The first upsampling branch module is sequentially composed of a first ReLU activation function, an upsampling layer, a convolutional layer, a BatchNorm2d layer, and a second ReLU activation function. The second upsampling branch module is sequentially composed of a convolutional layer and a ReLU activation function. e-2) Fusion feature F fusion Input into the first upsampling branch module of the first upsampling residual module, and output the feature F up1_1 , the fusion feature F fusion Input to the second upsampling branch module of the first upsampling residual module, and output the feature F up1_2 , the feature F up1_1 With feature F up1_2 Perform the addition operation to obtain the feature F up1 ; e-3) The feature F up1 Input to the first upsampling branch module of the second upsampling residual module, and output the feature F up2_1 , the feature F up1 Input to the second upsampling branch module of the second upsampling residual module, and output the feature F up2_2 , the feature F up2_1 With feature F up2_2 Perform the addition operation to obtain the feature F up2 ; e-4) Change the feature F up2 Input to the first upsampling branch module of the third upsampling residual module, and output the feature F up3_1 , the feature F up2 Input to the second upsampling branch module of the third upsampling residual module, and output the feature F up3_2 , the feature F up3_1 With feature F up3_2 Perform the addition operation to obtain the feature F up3 ; e-5) Change feature F up3 Input to the first upsampling branch module of the fourth upsampling residual module, and output the feature F up4_1 , the feature F up3 Input to the second upsampling branch module of the fourth upsampling residual module, and output the feature F up4_2 , the feature F up4_1 With feature F up4_2 Perform the addition operation to obtain the feature F up ; e-6) Change feature F up Input into the convolution layer of the upsampling residual decoder, and output the post-disaster remote sensing building segmentation image I seg .

7. The post-disaster building image segmentation method based on boundary enhancement and feature perception according to claim 6, characterized in that: The convolution kernel size of the convolution layer of the upsampling residual decoder is 1×1, the step size is 1, and the padding is 0; the convolution kernel size of the convolution layer of the first upsampling branch module of the first upsampling residual module, the second upsampling residual module, the third upsampling residual module, and the fourth upsampling residual module is 3×3, the step size is 1, and the padding is 0; the convolution kernel size of the convolution layer of the second upsampling branch module of the first upsampling residual module, the second upsampling residual module, the third upsampling residual module, and the fourth upsampling residual module is 3×3, the step size is 1, and the padding is 0.

Images