Image segmentation method for damaged buildings, electronic devices and storage media

Abstract

This invention relates to the field of aerospace information technology, providing a method, electronic device, and storage medium for image segmentation of damaged buildings. The method includes: acquiring a first remote sensing image before a geological disaster and a second remote sensing image after the disaster, and preprocessing the first and second images respectively; inputting the preprocessed images into a damaged building image segmentation model to obtain images of damaged buildings at different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolutional network. In the damaged building image segmentation model of this invention, the channel attention mechanism network enhances feature channels, and the pyramid pooling convolutional network captures contextual information at different scales, thereby making the obtained images of damaged buildings at different damage levels more accurate.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of space information technology, and in particular to a damaged building image segmentation method, an electronic device and a storage medium. BACKGROUND

[0002] Natural disasters have caused serious loss of life and property to human society. Earthquake is a phenomenon of ground vibration caused by the sudden release of internal energy of the earth, which leads to rapid rupture or dislocation of the crust or lithosphere, accompanied by the propagation of seismic waves. Among all kinds of natural disasters, earthquake is difficult to predict, has a wide range of influence and serious damage to cities and infrastructure, so it is of great significance to strengthen the monitoring, prediction and response measures of earthquake disaster and to reduce disaster loss and improve social resilience.

[0003] At present, there are various schemes for building damage detection, such as manual visual detection and detection model constructed based on deep learning technology. However, the traditional manual visual detection needs professional personnel to visually detect, although it can obtain high accuracy, but due to its time-consuming and laborious characteristics, it often cannot meet the demand of rapid extraction of damaged buildings in emergency monitoring, and the existing detection model relies on a single data source, and the accuracy of building damage evaluation is poor. SUMMARY

[0004] The present application provides a damaged building image segmentation method, an electronic device and a storage medium, which can solve the problem of poor accuracy of building damage evaluation in the prior art and accurately evaluate the damaged building.

[0005] The present application provides a damaged building image segmentation method, which comprises:

[0006] Obtaining a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, and respectively pre-processing the first remote sensing image and the second remote sensing image;

[0007] Inputting the pre-processed first remote sensing image and the second remote sensing image into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network.

[0008] According to the damaged building image segmentation method provided by the present application, the damaged building image segmentation model comprises an encoder and a decoder;

[0009] The encoder comprises a multi-level feature network, a channel attention mechanism network and a pyramid pooling convolution network connected in sequence;

[0010] The decoder comprises a feature reconstruction network, the multi-level feature network is connected with the feature reconstruction network through a skip connection, and the pyramid pooling convolutional network is connected with the feature reconstruction network;

[0011] The preprocessed first remote sensing image and the second remote sensing image are input into a damaged building image segmentation model to obtain damaged building images of different damage levels.

[0012] Based on the multi-level feature network, first convolutional features are extracted from the first remote sensing image and second convolutional features are extracted from the second remote sensing image layer by layer, target convolutional features are obtained based on the first convolutional features and the second convolutional features, and the first convolutional features and the second convolutional features obtained by each layer are fused respectively to obtain corresponding fusion features, and the fusion features are input into the feature reconstruction network based on a skip connection.

[0013] Based on the channel attention mechanism network and the pyramid pooling convolutional network, the target convolutional features are processed to obtain encoded features.

[0014] Based on the feature reconstruction network, feature maps are reconstructed through the encoded features and the fusion features to obtain damaged building images of different damage levels.

[0015] According to the damaged building image segmentation method provided by the application, the multi-level feature network comprises a first multi-level feature extraction module, a second multi-level feature extraction module, a feature splicing module and at least one feature fusion module.

[0016] Each level unit in the first multi-level feature extraction module and the second multi-level feature extraction module corresponds, and the feature fusion module is connected between other level units in the first multi-level feature extraction module and the second multi-level feature extraction module except the last level unit, the feature fusion module is connected with the feature reconstruction network through a skip connection, and the last level unit in the first multi-level feature extraction module and the second multi-level feature extraction module is connected through the feature splicing module, and the feature splicing module is connected with the feature reconstruction network.

[0017] According to the damaged building image segmentation method provided by the application, the feature reconstruction network comprises a plurality of feature reconstruction modules connected in sequence, the number of the feature reconstruction modules is equal to the number of the level units in the first multi-level feature extraction module, and the feature fusion module is connected with the corresponding feature reconstruction module through a skip connection; wherein the feature reconstruction module is constructed based on sub-pixel convolution and progressive attention structure.

[0018] According to the method for segmenting damaged building images provided by the application, the training process of the damaged building image segmentation model comprises the following steps:

[0019] Obtain a training set and a verification set; wherein the training set and the verification set respectively comprise a plurality of samples, and the samples comprise sample data and sample labels;

[0020] Input training samples in the training set into an initial damaged building image segmentation model for iterative training until training is completed, to obtain the damaged building image segmentation model;

[0021] Verify the damaged building image segmentation model trained to completion by using verification samples in the verification set.

[0022] According to the method for segmenting damaged building images provided by the application, the initial damaged building image segmentation model comprises an encoder and a decoder, the decoder comprises a feature reconstruction network, and the feature reconstruction network comprises a plurality of feature reconstruction modules connected in sequence;

[0023] The step of inputting training samples in the training set into an initial damaged building image segmentation model for iterative training until training is completed, to obtain the damaged building image segmentation model, comprises the following steps:

[0024] Input the training samples in the training set into the initial damaged building image segmentation model, to obtain feature reconstruction images output by each feature reconstruction module;

[0025] Calculate a training loss based on the feature reconstruction images;

[0026] Iteratively train the damaged building image segmentation model based on the training loss until training is completed, to obtain the damaged building image segmentation model.

[0027] According to the method for segmenting damaged building images provided by the application, the pre-processing of the first remote sensing image and the second remote sensing image respectively comprises the following steps:

[0028] The first remote sensing image and the second remote sensing image are respectively pre-processed by orthorectification, geographic matching, resolution unification and cropping.

[0029] According to the method for segmenting damaged building images provided by the application, after the pre-processed first remote sensing image and the pre-processed second remote sensing image are input into a damaged building image segmentation model to obtain damaged building images of different damage levels, the method further comprises the following steps:

[0030] Classify and optimize the damaged building images of different damage levels by using a preset damage classification standard.

[0031] The application further provides a damaged building image segmentation device, comprising:

[0032] a preprocessing module configured to acquire a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, and perform preprocessing on the first remote sensing image and the second remote sensing image respectively;

[0033] an image segmentation module configured to input the preprocessed first remote sensing image and the second remote sensing image into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network.

[0034] The application further provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the damaged building image segmentation method of any of the above when executing the computer program.

[0035] The application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the damaged building image segmentation method of any of the above.

[0036] The application further provides a computer program product comprising a computer program, wherein the computer program is executable by a processor to implement the damaged building image segmentation method of any of the above.

[0037] The damaged building image segmentation method, the electronic device and the storage medium provided by the application acquire a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, and perform preprocessing on the first remote sensing image and the second remote sensing image respectively. A damaged building image segmentation model is constructed in advance, the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network, the channel attention mechanism network can adaptively enhance the feature channels useful for the current input of the first remote sensing image, the second remote sensing image and the damaged building image segmentation task, and suppress the channels that are less important or have more noise. The pyramid pooling convolution network can capture the context information of different scales from local details to global scenes at the same time, and finally output a feature map containing more rich and comprehensive scene information by fusing multi-scale feature representations. Therefore, the damaged building images of different damage levels obtained by inputting the preprocessed first remote sensing image and the second remote sensing image into the damaged building image segmentation model can be more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to make the technical solutions in the present application or the prior art clearer, the accompanying drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and other accompanying drawings can be obtained by those skilled in the art without any creative effort on the basis of these accompanying drawings.

[0039] Figure 1 is a flowchart of the damaged building image segmentation method provided by the present application.

[0040] Figure 2 is a schematic diagram of the damaged building image segmentation model provided by the present application.

[0041] Figure 3 is a structural schematic diagram of the ConvNeXt network provided by the present application.

[0042] Figure 4 is a schematic diagram of the operation flow of the encoder provided by the present application.

[0043] Figure 5 is a structural schematic diagram of the BAM module provided by the present application.

[0044] Figure 6 is a structural schematic diagram of the feature fusion module provided by the present application.

[0045] Figure 7 is a structural schematic diagram of the channel attention mechanism network provided by the present application.

[0046] Figure 8 is a schematic diagram of the combination of the channel attention mechanism network and the pyramid pooling convolutional network provided by the present application.

[0047] Figure 9 is a schematic diagram of sub-pixel convolution upsampling provided by the present application.

[0048] Figure 10 is a schematic diagram of the remote sensing image used in the experiment provided by the present application.

[0049] Figure 11 is the output result of each network in the experiment provided by the present application.

[0050] Figure 12 is a structural schematic diagram of the damaged building image segmentation device provided by the present application.

[0051] Figure 13 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0052] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be clearly and completely described below in conjunction with the accompanying drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the protection scope of the present application.

[0053] In the prior art, although the traditional manual visual interpretation method can obtain high accuracy, due to the time-consuming and labor-intensive characteristics, it is often difficult to meet the demand of rapid extraction of damaged buildings in emergency monitoring. In the aspect of automatic extraction model, due to the implicitness of damage features and high intra-class variance (for example, buildings under the same "complete collapse" label have differences in fragment distribution, inclination angle, etc.), it is difficult to establish a stable mapping relationship between damage degree and image features. This bottleneck promotes the research paradigm to gradually transform into multi-temporal collaborative analysis: by introducing pre-earthquake high-resolution reference images, the model can construct the evolution trajectory of building structure from normal state to damaged state, and capture the mutation threshold of damage signal (such as the texture contrast transition before and after the roof collapse) by using time series difference features. For example, the dual-channel input framework proposed based on the xBD dataset effectively separates the pre-disaster and post-disaster features through the parameter sharing mechanism, realizing the pixel-level time series enhanced expression of building collapse. This spatio-temporal coupling analysis not only enhances the physical interpretability of damage identification through dynamic trajectory modeling, but also reveals the coupling rules between spectral, morphological and geometric features in the post-earthquake damage evolution process (such as the features of broken house geometric features and chaotic texture). In addition, most of the existing deep learning methods rely on a single earthquake event and a single satellite data source for training and modeling. However, in the earthquake emergency scene, the resolution of image data, the diversity of satellite data sources and the differences in the epicenter area lead to the richness of building styles and topography, which poses a huge challenge to the diversification of training data. The deep learning model of a single data source is often difficult to effectively cope with the multi-scale and multi-style post-earthquake building extraction task. In addition, the damage building detection method based on change detection can only determine whether the building is damaged, and it is difficult to classify and extract buildings of different damage types, which limits the accurate assessment of disaster damage in the post-disaster emergency rescue process. In summary, the current application of remote sensing technology in the extraction of damaged buildings in geological disasters has the following main problems: (1) traditional manual interpretation is time-consuming and labor-intensive, and it is difficult to meet the real-time needs of emergency monitoring; (2) most of the existing models rely on a single satellite data source for training, and cannot adapt to the emergency application needs of diverse data sources after the disaster; (3) the damage building extraction based on change detection lacks damage grading, which affects the accuracy of disaster damage assessment after the disaster.

[0054] The remote sensing images involved in the present application are all fully authorized data, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and corresponding operation portals are provided for users to choose authorization or rejection.

[0055] Figure 1 is a flow chart of a damaged building image segmentation method according to an exemplary embodiment. As shown in Figure 1 the damaged building image segmentation method includes steps 110 to 120, which are described in detail as follows.

[0056] Step 110, obtaining a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, and respectively pre-processing the first remote sensing image and the second remote sensing image.

[0057] In the embodiment of the present application, the first remote sensing image before the geological disaster and the second remote sensing image after the geological disaster are obtained, and the first remote sensing image and the second remote sensing image are respectively pre-processed by radiation calibration, atmospheric correction, orthorectification, geographic matching, resolution unification and cutting.

[0058] Step 120, inputting the pre-processed first remote sensing image and second remote sensing image into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network.

[0059] In the embodiment of the present application, a damaged building image segmentation model is constructed in advance, and the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network. The channel attention mechanism network can adaptively enhance the feature channels useful for the current input of the first remote sensing image, the second remote sensing image and the damaged building image segmentation task, while suppressing the channels that are less important or have larger noise. The pyramid pooling convolution network can capture the context information of different scales from local details to global scenes at the same time, and finally output a feature map containing more rich and comprehensive scene information by fusing multi-scale feature representations. Therefore, the damaged building images of different damage levels obtained can be more accurate.

[0060] In an exemplary embodiment of the present application, the damaged building image segmentation model includes an encoder and a decoder.

[0061] The encoder includes a multi-level feature network, a channel attention mechanism network and a pyramid pooling convolution network connected in sequence.

[0062] The decoder comprises a feature reconstruction network, the multi-level feature network is connected to the feature reconstruction network through a skip connection, and the pyramid pooling convolutional network is connected to the feature reconstruction network;

[0063] The preprocessed first remote sensing image and the second remote sensing image are input into a damaged building image segmentation model to obtain damaged building images of different damage levels.

[0064] Based on the multi-level feature network, first convolutional features are extracted from the first remote sensing image and second convolutional features are extracted from the second remote sensing image layer by layer, target convolutional features are obtained based on the first convolutional features and the second convolutional features, the first convolutional features and the second convolutional features obtained at each layer are fused respectively to obtain corresponding fused features, and the fused features are input into the feature reconstruction network based on a skip connection.

[0065] Based on the channel attention mechanism network and the pyramid pooling convolutional network, the target convolutional features are processed to obtain encoded features.

[0066] Based on the feature reconstruction network, feature maps are reconstructed through the encoded features and the fused features to obtain damaged building images of different damage levels.

[0067] In the embodiment of the application, the damaged building image segmentation model comprises an encoder and a decoder. In the encoding stage, a multi-level feature fusion mechanism is designed, a multi-level feature network is used to extract first convolutional features from a first remote sensing image and second convolutional features from a second remote sensing image layer by layer, and summation and difference are used to realize difference enhancement fusion of the convolutional features in different remote sensing images to obtain fused features, and channel splicing is used to realize feature interaction to obtain target convolutional features. In order to further enhance the feature representation capability, a channel attention mechanism network (Channel Attention Module, CAM) is integrated to strengthen the detail response of the damage sensitive area in the remote sensing image through feature re-labeling. A pyramid pooling convolutional network (Pyramid Pooling Module, PPM) is introduced at the bottom of the encoder to capture multi-granularity spatial context information, and then encoded features are obtained.

[0068] In the decoding stage, a cross-level feature transmission path is constructed, a skip connection is used to integrate the semantic features of different levels of the encoder, high-resolution feature map reconstruction is realized, and damaged building images of different damage levels are obtained.

[0069] In an exemplary embodiment of the application, the multi-level feature network comprises a first multi-level feature extraction module, a second multi-level feature extraction module, a feature splicing module and at least one feature fusion module.

[0070] Each level unit in the first multi-level feature extraction module and the second multi-level feature extraction module corresponds, and each level unit in the first multi-level feature extraction module and the second multi-level feature extraction module except the last level unit is connected through the feature fusion module respectively, the feature fusion module is connected with the feature reconstruction network, and the last level unit in the first multi-level feature extraction module and the second multi-level feature extraction module is connected through the feature splicing module, and the feature splicing module is connected with the feature reconstruction network.

[0071] In the embodiment of the application, the first multi-level feature extraction module and the second multi-level feature extraction module in the encoder each have N level units (ConvNeXt-BAM Stage), N is a natural number greater than or equal to 2, and the level units of the two modules correspond. Further, a double-branch architecture is formed, that is, each level unit in the first multi-level feature extraction module extracts a first convolutional feature from the first remote sensing image layer by layer, and each level unit in the second multi-level feature extraction module extracts a second convolutional feature from the second remote sensing image layer by layer. As shown in Figure 2 The first multi-level feature extraction module and the second multi-level feature extraction module are each provided with 4 level units. In the first N-1 level units, the first convolutional features and the second convolutional features extracted by each corresponding level unit are fused by the feature fusion module to obtain fused features, and the fused features are input into the feature reconstruction network through the skip connection. In the Nth level unit, the first convolutional features and the second convolutional features extracted by the two level units are channel spliced by the feature splicing module (C) to obtain target convolutional features.

[0072] The original FTN model (Feature-wise Transformation Network) adopts a parameter-shared Swin Transformer double-branch architecture, which realizes global context modeling through local self-attention and shift window mechanism, but has inherent defects of high computational complexity and redundant parameters. In the embodiment of the application, the ConvNeXt network is introduced as the core component of the encoder, that is, the first multi-level feature extraction module and the second multi-level feature extraction module are a hybrid coding architecture based on ConvNeXt-BAM, to optimize the feature extraction capability of the FTN model. The ConvNeXt network of the application inherits the advantages of the Transformer training paradigm (including the AdamW optimizer and large-scale data enhancement strategy), and through the collaborative design of large-core depth separable convolution (7x7) and inverted bottleneck structure (Inverted Bottleneck), achieves high performance in the ImageNet classification task.

[0073] As shown in Figure 3 , the ConvNeXt network adopts a residual structure design, and its core process is in turn a 7x7 large kernel depth separable convolution (96 channels), a layer normalization (Layer Norm), a 1x1 convolution to expand the channel to 384 dimensions, a GELU activation function, then a 1x1 convolution to compress back to 96 channels, and finally a residual connection with the input. The ConvNeXt network simulates the global feature interaction capability of the Transformer through a large kernel convolution, combines layer normalization and a bottleneck feedforward network (expansion-compression structure), and aligns the training strategy of the Transformer architecture while preserving the local advantages of convolution, and takes into account global perception and hardware computing efficiency.

[0074] To further strengthen the feature selection capability, the convolution block (ConvBlock) in the ConvNeXt network is used for multi-level feature extraction, and the bottleneck attention module (BAM) is used for feature enhancement, thereby constructing a dual-phase feature extraction framework with strong representation capability. As shown in Figure 4 , by embedding the BAM module in the feature generation part of the four convolution blocks (ConvBlock1-4), the feature selection efficiency of the pre-disaster and post-disaster encoding segments is strengthened through feature weight redistribution before feature cross transmission, and the corresponding features (Feature) are obtained.

[0075] The specific implementation principle of the BAM module is shown in Figure 5 , which realizes feature re-labeling through parallel spatial-channel attention paths. As shown in Figure 5 , the channel branch adopts global average pooling (Global average pool) and full connection (FC) repeated n times (Repeatn times), and then reconstructs and decodes (Expend) to generate a channel attention map. The spatial branch captures multi-scale context information through a dilated convolution, and finally realizes attention weight fusion through a Sigmoid activation. Compared with the window self-attention mechanism of SwinTransformer, the BAM module realizes equivalent feature enhancement effect with a small amount of parameter increment, significantly reducing the computational complexity.

[0076] In the embodiment of the present application, in order to weaken the difference between the first convolutional feature and the second convolutional feature, a feature fusion module (Feature Fusion Module, FFM) is constructed through a difference enhancement module based on FTN. As shown in Figure 6As shown, in the feature fusion process, first, pixel-by-pixel difference operation is performed on the first convolutional feature and the second convolutional feature to strengthen the response value of the change region to obtain a difference feature, and at the same time, pixel-by-pixel summation operation is performed to retain the common features of the unchanged region to obtain a common feature, and finally the difference feature and the common feature are spliced along the channel dimension to form a fusion feature, and then skip connection is performed, thereby constructing a complementary dual-phase feature expression system. The FFM module calculation formula is as follows:

[0077] ;

[0078] ;

[0079] ;

[0080] Wherein, F pre and F post are the first convolutional feature and the second convolutional feature, F diff is the difference feature, F sum is the common feature, and F enhance is the fusion feature.

[0081] In terms of multi-phase feature fusion, the channel attention mechanism is introduced to realize cross-time feature optimization. As shown in Figure 7 , after the dual-phase image feature fusion is realized by the combination operation of the convolution layer (conv), batch normalization (BN) and SiLU activation function, the channel attention mechanism network is then used for feature re-calibration. The dual-path channel attention includes two key stages of feature compression and feature excitation: in the compression stage, the global average pooling (Avg_pool) and the global maximum pooling (Max_pool) are used for parallel processing to compress the feature map with a spatial dimension of HxW into a channel description vector, and a channel-level global statistical feature is established; in the excitation stage, a fully connected layer with a nonlinear activation function is used to learn the complex correlation between channels to generate a normalized weight coefficient, and the mathematical expression is as follows:

[0082] ;

[0083] Wherein, represents the compressed channel statistics, and are learnable parameter matrices, C represents the number of channels, r represents the compression ratio, δ represents the SiLU activation function, is the Sigmoid function, and w represents the normalized weight coefficient of the output. Finally, the feature is re-calibrated through channel weighting to enhance the response strength of the effective feature channel.

[0084] In the embodiment of the present application, in order to improve the adaptability of the model to multi-scale image features, a multi-scale feature enhancement mechanism based on a pyramid pooling convolutional network is provided. Figure 8 As shown in the figure, the channel attention mechanism network (ChannelAttention) outputs features input into the pyramid pooling convolutional network. The pyramid pooling convolutional network adopts a multi-level pooling kernel parallel processing strategy, performs 1x1, 3x3, 6x6 and 8x8 four scale parallel adaptive pooling operations (POOL) on the input features, performs feature dimension compression through 1x1 convolution (Conv), performs feature map upsampling (Upsampling) using bilinear interpolation, and finally forms multi-scale fused encoding features through channel concatenation (CAT) of different scale feature maps. The mathematical expression of the pyramid pooling convolutional network is:

[0085]

[0086] Among them, represents the features output by the pyramid pooling convolutional network, represents the input features, represents the i x i size of the pooling operation, is a 1x1 convolution layer, and Upsample is a bilinear interpolation upsampling, represents channel dimension concatenation. The pyramid pooling convolutional network significantly enhances the model's expression ability for multi-scale features of ground objects through multi-receptive field feature fusion.

[0087] In the embodiment of the present application, in the feature encoding stage, the first convolutional features and the second convolutional features are input into the PPM module after channel dimension concatenation, processed through multi-scale feature enhancement, and finally output encoding features containing rich spatio-temporal information. The encoding features have multi-temporal change features and multi-scale spatial features, and can effectively support the subsequent building damage extraction task after geological disasters.

[0088] In an exemplary embodiment of the present application, the feature reconstruction network includes a plurality of feature reconstruction modules connected in sequence, the number of the feature reconstruction modules is equal to the number of the level units in the first multi-level feature extraction module, and the feature fusion module is jump-connected with the corresponding feature reconstruction module; wherein the feature reconstruction module is constructed based on sub-pixel convolution and progressive attention structure.

[0089] In the embodiment of the present application, as shown in the figure, Figure 2 ​As shown, the feature reconstruction network includes a plurality of feature reconstruction modules (PAM-Pixel Shuffle) connected in sequence, the number of feature reconstruction modules is equal to the number of level units in the first multi-level feature extraction module, the feature fusion module is jump-connected with the corresponding feature reconstruction module, and the feature reconstruction module is constructed based on a subpixel convolution (Pixel Shuffle) and a progressive attention module (PAM).

[0090] For the feature reconstruction and information compensation problem in the decoder architecture design, the embodiment of the present application proposes a multi-level feature optimization mechanism. The mechanism constructs a decoder framework through three core modules of reversible upsampling, feature difference attention enhancement and deep supervision, effectively improving the recovery accuracy of disaster change features and the stability of model training.

[0091] In terms of feature reconstruction, the embodiment of the present application uses subpixel convolution for inverse upsampling, that is, the upsampling operation is realized by expanding the channel into a 4*4 area, and the resolution is improved by feature channel reorganization, and the mathematical expression is:

[0092] ;

[0093] Where, PS represents a pixel reorganization operation, Conv is a 3*3 convolution layer, represents the feature output by the subpixel convolution, represents the input feature. However, the spatial context information loss caused by the upsampling process leads to incomplete feature upsampling information, thereby limiting the improvement of classification accuracy. Therefore, the embodiment of the present application constructs a cross-layer feature compensation path: the fusion feature of the kth level of the encoder and the enhanced feature after double-channel calculation of the feature reconstruction module of the k+1th level in the decoder are concatenated in the channel, and cross-scale feature fusion is realized through 3*3 convolution.

[0094] As Figure 9 shown, in the upsampling process, in order to ensure the integrity of the upsampling feature and retain the key features, the embodiment of the present application adopts the progressive attention structure PAM introduced in FTN. The structure realizes the processing of the upsampling feature by splicing the sum and difference features in the channel dimension, combining convolution operation for feature calculation, and constructing channel attention through global average pooling, and the structure can be specifically expressed as:

[0095] ;

[0096] ;

[0097] Where, respectively represent the cross-layer connection of sum and difference characteristics in the encoder, Conv represents a convolution operation, BN represents batch normalization, and ReLU represents a ReLU (Rectified Linear Unit) function, represents a Sigmoid function, and GAP represents a global average pooling operation, represents the feature of the kth layer, represents the final attention feature of the kth layer.

[0098] In an example embodiment of the present application, the training process of the damaged building image segmentation model comprises:

[0099] obtaining a training set and a validation set; wherein the training set and the validation set each comprise a plurality of samples, and each sample comprises sample data and a sample label;

[0100] inputting the training samples in the training set into an initial damaged building image segmentation model for iterative training until the training is completed, to obtain the damaged building image segmentation model;

[0101] verifying the trained damaged building image segmentation model by using the verification samples in the validation set.

[0102] In the embodiment of the present application, the training set and the validation set are obtained, and the samples in the training set and the validation set are consistent with the aforementioned remote sensing images. Each sample contains a first sample remote sensing image before a geological disaster and a second sample remote sensing image after the geological disaster. The samples in the training set are training samples, and the samples in the validation set are verification samples. The sample label is a corresponding real damaged building image. The training samples in the training set are input into an initial damaged building image segmentation model for iterative training until the training is completed, to obtain a damaged building image segmentation model. The initial damaged building image segmentation model and the damaged building image segmentation model have the same structure, and the only difference between them is that the parameters in the encoders and decoders of the two models are different. The trained damaged building image segmentation model is verified by using the verification samples in the validation set, to verify the accuracy and effectiveness of the prediction results of the model.

[0103] In an example embodiment of the present application, the initial damaged building image segmentation model comprises an encoder and a decoder, the decoder comprises a feature reconstruction network, and the feature reconstruction network comprises a plurality of feature reconstruction modules connected in sequence;

[0104] The inputting of the training samples in the training set into the initial damaged building image segmentation model for iterative training until the training is completed, to obtain the damaged building image segmentation model, comprises:

[0105] input the training sample in the training set into the initial damaged building image segmentation model to obtain a feature reconstruction image output by each feature reconstruction module;

[0106] calculate a training loss based on the feature reconstruction image;

[0107] perform iterative training on the damaged building image segmentation model based on the training loss until training is completed to obtain the damaged building image segmentation model.

[0108] In the embodiment of the application, to further optimize the model training process, a multi-level supervision mechanism is constructed, an auxiliary classifier is connected to the output end of each stage of the decoder, and gradient multi-path propagation is realized through a deep supervision strategy. That is, the training sample in the training set is input into the initial damaged building image segmentation model to obtain a feature reconstruction image output by each feature reconstruction module, as shown in Figure 2 Each feature reconstruction module (PAM-Pixel Shuffle) outputs one feature reconstruction image. The training loss is calculated based on the feature reconstruction image.

[0109] In the embodiment of the application, since the number of pixels of the target to be recognized in the remote sensing recognition task is quite different from the background, there is an imbalance problem between positive and negative samples, therefore, a loss function based on Dice coefficient (DSC) and a cross-entropy loss function are used to calculate the training loss.

[0110] The loss function of the Dice coefficient is defined as:

[0111] ;

[0112] Wherein, A and B are the predicted positive sample set and the real positive sample set respectively, represents the Dice loss. The Diss loss pays attention to the similarity of false positives (FP) and false negatives (FN), therefore, it has good effect in the segmentation task of class imbalance, and improves the problem of too large difference in the number of positive and negative samples in remote sensing recognition.

[0113] Cross-entropy loss (CELoss) is a loss function used for classification problems in machine learning and deep learning. It can be used to measure the difference between the probability distribution of model prediction and the probability distribution of real label, and the formula is as follows:

[0114] ;

[0115] In the formula, y i represents the one-hot encoding of the real label, and p iThe probability that the model predicts as the i-th class, L represents the cross-entropy loss. One-hot encoding is a method of converting a categorical variable into a form that can be better processed by a machine learning algorithm. In one-hot encoding, each category value is represented as a binary vector, except that one position representing the category is 1 and the rest are 0.

[0116] The loss of each feature reconstruction image in the embodiment of the application The mixed loss of the Dice loss and the cross-entropy loss, and the specific formula expression is as follows:

[0117] ;

[0118] Further, the training loss is:

[0119] ;

[0120] Through the cross-level link and the layer-by-layer loss function calculation, the embodiment of the application realizes the output control of the multi-layer deep supervision network, thereby controlling the output classification precision.

[0121] In an example embodiment of the application, the validation of the completed training damaged building image segmentation model through the validation samples in the validation set comprises:

[0122] The validation sample in the validation set is input into the completed training damaged building image segmentation model to obtain a validation damaged building image.

[0123] Based on the validation damaged building image and the sample label, a model evaluation value is calculated.

[0124] Based on the model evaluation value, the damaged building image segmentation model is validated.

[0125] In the embodiment of the application, the validation sample is input into the damaged building image segmentation model, and the validation damaged building image of different damage levels is obtained through the model, and any one of the precision (Precision), the recall rate (Recall), the average intersection over union (MIoU) and the balanced score (F1-score) is used as the evaluation standard of the damaged building image segmentation model.

[0126] In machine learning, the prediction results can be represented by a confusion matrix. As shown in Table 1 below, the confusion matrix describes the performance of a classification model. In this matrix, TP represents the number of samples that are both actual and predicted to be true, FP represents the number of samples that are incorrectly predicted to be true but are actually false; FN represents the number of samples that are incorrectly predicted to be false but are actually true; and TN represents the number of samples that are both actual and predicted to be false. Based on this matrix, indicators such as precision and recall are developed. Considering that the damaged building image segmentation model of the present application can output damaged building images of different damage levels, i.e. there are multiple classifications, the damaged building image segmentation task is regarded as multiple binary classifications to calculate the confusion matrix of each class:

[0127] Table 1

[0128]

[0129] The calculation formulas of precision (Precision) and recall (Recall) are as follows:

[0130] ;

[0131] ;

[0132] The F1 score is the harmonic mean of precision and recall, and the specific calculation formula is as follows:

[0133] ;

[0134] MIoU is a statistical measure widely used to evaluate target detection and segmentation models, especially in the field of computer vision and image processing. The formula is as follows:

[0135] ;

[0136] Where n represents the number of classes.

[0137] In an exemplary embodiment of the present application, the respective pre-processing of the first remote sensing image and the second remote sensing image includes:

[0138] Respective pre-processing of the first remote sensing image and the second remote sensing image includes orthorectification, geographic matching, resolution unification, and cropping.

[0139] In the emergency situation of geological disasters, the original remote sensing image data collected often has problems such as low spatial resolution, geographical coordinate offset, and inaccurate matching of pre-disaster and post-disaster data. Therefore, it is necessary to preprocess the pre-disaster and post-disaster data to ensure its applicability and effectiveness. Therefore, the embodiment of the present application carries out specific preprocessing procedures on the original first remote sensing image and the second remote sensing image to ensure the quality of the data itself and the practicality of subsequent analysis.

[0140] For the image quality and cloud cover in the original first remote sensing image and the second remote sensing image, the image quality and cloud cover will significantly affect the identification of damaged buildings. The embodiment of the present application carries out preliminary screening on the original first remote sensing image and the second remote sensing image data from different sources to ensure that only high-quality remote sensing images are retained for further analysis. For remote sensing images with both multispectral and panchromatic bands, fusion technology is used to improve the resolution of multispectral images, thereby enhancing their identifiability. When the first remote sensing image and the second remote sensing image are images taken by different sensors at different times, there will usually be a standardization problem, that is, there is a significant difference in gray value and absolute radiance between the two remote sensing images. In order to solve this problem, the embodiment of the present application carries out radiometric calibration on the collected remote sensing images, that is, converts the visible light reflectance data in the first remote sensing image and the second remote sensing image into standardized units, thereby improving the readability and reliability of the data.

[0141] Although the atmosphere usually has little effect on visible light, aerosols and water vapor can still affect the propagation of light. Therefore, the embodiment of the present application carries out atmospheric correction on the first remote sensing image and the second remote sensing image to further optimize the image quality.

[0142] There may be a problem of geographical inaccuracy between the first remote sensing image and the second remote sensing image, resulting in misalignment of the coordinates of the first remote sensing image and the second remote sensing image. By performing orthorectification on the visible light image, the geographical positioning accuracy is significantly improved, the distortion caused by the terrain is reduced, and convenience is provided for subsequent registration of the first remote sensing image and the second remote sensing image. Despite these improvements, there are still differences in details between the first remote sensing image and the second remote sensing image. After orthorectification, the first remote sensing image and the second remote sensing image still need to be geographically matched to ensure the spatial consistency between the remote sensing images.

[0143] In the case where the resolutions of the first remote sensing image and the second remote sensing image are different, the embodiment of the present application carries out grid downsampling on the remote sensing image with higher resolution to realize the unification of the resolutions of the first remote sensing image and the second remote sensing image, thereby ensuring the pixel-level correspondence between the remote sensing images.

[0144] In order to optimize the input data of subsequent deep learning analysis, the embodiment of the present application systematically crops remote sensing images into a uniform size of 512*512 pixels. In the cropping process, images lacking building features are excluded, thereby maintaining the balance between complete buildings and damaged buildings.

[0145] Compared with manual identification, the technical solution provided by the present application only needs to collect remote sensing images before and after geological disasters, automatically pre-process the remote sensing images, unify the pre-disaster and post-disaster coverage range, remove the influence of geographic distortion of the remote sensing images, unify the resolution, then input the damaged building image segmentation model, automatically obtain damaged building images of different damage levels, and finally realize the extraction of large-scale post-earthquake damaged buildings through manual correction.

[0146] In an exemplary embodiment of the present application, after the pre-processed first remote sensing image and the second remote sensing image are input into the damaged building image segmentation model to obtain damaged building images of different damage levels, the method further comprises:

[0147] The damaged building images of different damage levels are classified and optimized by a preset damage classification standard.

[0148] In the embodiment of the present application, due to the accuracy limitation of the model itself and the interference of abnormal factors such as cloud, shooting light, shooting satellite, etc., the technical solution provided by the present application cannot achieve 100% accurate prediction of damaged buildings during implementation. Therefore, the damaged building images of different damage levels obtained by the damaged building image segmentation model can be secondarily confirmed and judged by the preset damage classification standard. At the same time, when it is determined that the damaged building image obtained by the damaged building image segmentation model is incorrect, the first remote sensing image, the second remote sensing image and the confirmed damaged building image form a training sample, and then the damaged building image segmentation model is retrained.

[0149] In the embodiment of the present application, for the display effect of the damaged building image under the condition of multi-resolution, the embodiment of the present application classifies the damaged building into three categories of intact, slightly damaged, severely damaged and completely collapsed, and forms a preset damage classification standard. According to the specific characteristics of the damage, the specific description is shown in the following table 2:

[0150] Table 2

[0151]

[0152] Through the above standard, the severely damaged and completely collapsed buildings are labeled, and the misidentified areas are removed to achieve the emergency disaster reduction standard.

[0153] As a core technical means of the earth observation system, remote sensing technology has become an important data source for earth system science research due to its wide coverage, non-contact observation, and immunity to surface conditions. In the geological disaster emergency response scenario, the multi-scale feature extraction capability of high-resolution images can realize the directional detection of the edge fragmentation of damaged buildings, the roof collapse rate and other forms, providing effective emergency rescue information for rapid disaster assessment. The present application uses remote sensing means, combines the morphological features of buildings in post-disaster remote sensing images and the significant feature transformation of buildings before and after the disaster, and forms an automatic monitoring model, which has the following advantages: when an earthquake occurs, this method provides an automatic image processing, resampling, cropping and mosaicking process to ensure the standardization of the input remote sensing image; the present application improves the change detection model to realize the automatic extraction of damaged buildings based on pre-disaster and post-disaster high-resolution remote sensing images.

[0154] In the earthquake disaster scenario, the damaged building image segmentation model of the present application is trained based on remote sensing images in the earthquake scenario, and the damaged building image segmentation model can be called a seismic damage house extraction deep learning model (Building Demage Grading Inspection Net, BDGINet). In view of the complex characteristics of remote sensing image change detection in the earthquake disaster scenario, the present application constructs a multi-dimensional comparison experiment framework to systematically evaluate the detection performance of the technical solution provided by the present application in a multi-scale complex scenario. As shown in Table 3 below, five representative benchmark models are selected for comparison and analysis with the damaged building image segmentation model provided by the present application, including classical semantic segmentation networks (Unet, Unet++, DeeplabV3+), dual-phase detection networks (BDD-Net) and feature transformation networks (FTN). All models are initialized with ImageNet pre-trained weights, using ResNet101 as a unified encoder, and BDD-Net uses EfficientNet-B4 encoder. The model parameters are optimized on the same training set through a fine-tuning strategy.

[0155] The quantitative analysis results show that the traditional single-phase segmentation network has significant limitations in the post-earthquake complex scene: the F1-score of the Unet series network on the test set is less than 45%, and the MIoU index is not more than 46%. This is mainly due to the high similarity of collapsed buildings and bare ground in the spectrum after the earthquake, making it difficult for the model to establish an effective inter-class separability criterion. It is worth noting that although DeeplabV3+ expands the receptive field through dilated convolution, its recall rate of 54.8% still indicates that there are a large number of missed detections, which verifies that the complex damage features have a significant inhibitory effect on the detection performance of single-phase methods.

[0156] The comparative experiment of the dual-time detection network further reveals the key influence of the model architecture design. Although the BDD-Net adopts the dual-branch feature extraction strategy, it has a significant overfitting tendency under limited training data, and there is a significant imbalance between the accuracy of 84.7% and the recall rate of 52.4%, indicating that the model has excessive inhibition problems for negative samples. In comparison, the FTN and the model provided by the application realize the effective improvement of the feature expression ability through the deep supervision learning mechanism and the parameter sharing strategy. In particular, the multi-scale feature fusion architecture proposed by the application improves the F1-score (69.6%) and MIoU (83.3%) by 1.5 and 1.6 percentage points respectively, which verifies the synergistic optimization effect of the improved feature extraction module and the reversible upsampling structure.

[0157] Table 3

[0158]

[0159] As shown in Figure 10 and Figure 11 , (a) represents the remote sensing image before the earthquake, (b) represents the remote sensing image of the earthquake, (c) represents the ground truth, (d) represents the Unet network result, (e) represents the Unet++ network result, (f) represents the DeeplabV3+ network result, (g) represents the BDD-Net network result, (h) represents the FTN network result, and (i) represents the BDGINet result. Red represents completely collapsed buildings, orange represents severely damaged buildings, and black represents the background.

[0160] As can be seen from the pictures of Figure 10 and Figure 11 , in the experimental remote sensing images selected by the application, the architectural style, the earthquake damage degree, the image lighting condition and the image resolution difference pose great challenges to the segmentation and classification of the model. The conventional semantic segmentation model with double image input is significantly insufficient in segmentation ability for this experimental data set. The classical semantic segmentation networks Unet, Unet++, and DeeplabV3+ have poor detection effect on damaged buildings during the experiment, and can only locate a few damaged building locations. In addition, the BDD-Net network responds well to the completely collapsed area through the change detection structure, but due to the limitation of the network structure scale, it is almost impossible to detect single damaged buildings for images with limited spatial resolution. The FTN network benefits from the window displacement and self-attention mechanism of the Swin-Transformer network, and effectively extracts most of the damaged buildings.

[0161] Further analysis shows that the performance advantage of the damaged building image segmentation model provided by the application is derived from three key designs: first, the pyramid pooling convolutional network enhances the model's ability to capture multi-scale features in damaged areas through multi-receptive field feature fusion; second, the feature difference enhancement mechanism effectively amplifies the temporal change information of the dual-phase images; and finally, the deep supervision strategy optimizes the feature decoding process through multi-level gradient propagation. Experimental results show that these innovative designs significantly improve the discriminant ability of the damaged building image segmentation model in the feature confusion scene, providing a new technical approach for disaster emergency remote sensing detection.

[0162] The damaged building image segmentation device provided by the application is described below, and the damaged building image segmentation device described below can be correspondingly referred to the damaged building image segmentation method described above. It should be noted that the device provided in the following embodiments and the method provided in the above embodiments belong to the same concept, and the specific manner in which each module and unit performs operations has been described in detail in the method embodiments, which will not be described here.

[0163] In an exemplary embodiment of the application, please refer to Figure 12 , Figure 12 is a damaged building image segmentation device according to an exemplary embodiment, comprising the following modules.

[0164] The preprocessing module 1210 is configured to obtain a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, and pre-process the first remote sensing image and the second remote sensing image, respectively;

[0165] The image segmentation module 1220 is configured to input the pre-processed first remote sensing image and the second remote sensing image into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolutional network.

[0166] In an exemplary embodiment of the application, the damaged building image segmentation model includes an encoder and a decoder;

[0167] The encoder includes a multi-level feature network, a channel attention mechanism network and a pyramid pooling convolutional network connected in sequence;

[0168] The decoder includes a feature reconstruction network, the multi-level feature network is connected to the feature reconstruction network in a skip connection manner, and the pyramid pooling convolutional network is connected to the feature reconstruction network;

[0169] The image segmentation module 1220 includes:

[0170] The extraction submodule is configured to extract first convolutional features from the first remote sensing image and second convolutional features from the second remote sensing image layer by layer based on the multi-level feature network, obtain target convolutional features based on the first convolutional features and the second convolutional features, perform feature fusion on the first convolutional features and the second convolutional features obtained by each layer respectively to obtain corresponding fused features, and input the fused features into the feature reconstruction network based on a skip connection.

[0171] The processing submodule is configured to process the target convolutional features based on the channel attention mechanism network and the pyramid pooling convolutional network to obtain encoded features.

[0172] The reconstruction submodule is configured to perform feature map reconstruction based on the encoded features and the fused features by the feature reconstruction network to obtain damaged building images of different damage levels.

[0173] In an exemplary embodiment of the present application, the multi-level feature network comprises a first multi-level feature extraction module, a second multi-level feature extraction module, a feature splicing module and at least one feature fusion module.

[0174] Each level unit in the first multi-level feature extraction module and the second multi-level feature extraction module corresponds to each other, the feature fusion module is connected between other level units except the last level unit in the first multi-level feature extraction module and the second multi-level feature extraction module, the feature fusion module is connected with the feature reconstruction network in a skip connection, and the feature splicing module is connected between the last level unit in the first multi-level feature extraction module and the second multi-level feature extraction module, and the feature splicing module is connected with the feature reconstruction network.

[0175] In an exemplary embodiment of the present application, the feature reconstruction network comprises a plurality of feature reconstruction modules connected in sequence, the number of the feature reconstruction modules is equal to the number of the level units in the first multi-level feature extraction module, and the feature fusion module is connected with the corresponding feature reconstruction module in a skip connection; wherein the feature reconstruction module is constructed based on a sub-pixel convolution and a progressive attention structure.

[0176] In an exemplary embodiment of the present application, the training process of the damaged building image segmentation model comprises:

[0177] The acquisition module is configured to acquire a training set and a validation set; wherein the training set and the validation set respectively comprise a plurality of samples, and the sample comprises sample data and a sample label.

[0178] The iteration training module is configured to input the training samples in the training set into an initial damaged building image segmentation model for iteration training until training is completed, so as to obtain the damaged building image segmentation model.

[0179] The verification module is configured to verify the trained damaged building image segmentation model by using the verification samples in the verification set.

[0180] In an example embodiment of the present application, the initial damaged building image segmentation model comprises an encoder and a decoder, the decoder comprises a feature reconstruction network, and the feature reconstruction network comprises a plurality of feature reconstruction modules connected in sequence.

[0181] The iteration training module comprises:

[0182] The input submodule is configured to input the training samples in the training set into the initial damaged building image segmentation model, so as to obtain feature reconstruction images output by each feature reconstruction module.

[0183] The calculation submodule is configured to calculate a training loss based on the feature reconstruction images.

[0184] The iteration training submodule is configured to perform iteration training on the damaged building image segmentation model based on the training loss until training is completed, so as to obtain the damaged building image segmentation model.

[0185] In an example embodiment of the present application, the preprocessing module 1210 comprises:

[0186] The preprocessing submodule is configured to perform preprocessing of orthorectification, geographic matching, resolution unification and cropping on the first remote sensing image and the second remote sensing image respectively.

[0187] In an example embodiment of the present application, the damaged building image segmentation device further comprises:

[0188] The classification optimization processing module is configured to perform classification optimization processing on damaged building images of different damage levels by using a preset damage classification standard.

[0189] Figure 13 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 13As shown, the electronic device can include a processor 1310, a communications interface 1320, a memory 1330, and a communications bus 1340, wherein the processor 1310, the communications interface 1320, and the memory 1330 complete mutual communication through the communications bus 1340. The processor 1310 can invoke a logical instruction in the memory 1330 to execute a damaged building image segmentation method, which includes:

[0190] obtaining a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, and respectively pre-processing the first remote sensing image and the second remote sensing image;

[0191] inputting the pre-processed first remote sensing image and the second remote sensing image into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network.

[0192] In addition, the logical instruction in the memory 1330 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0193] On the other hand, the present application also provides a computer program product, which includes a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program is executed by a processor, and the computer can execute the damaged building image segmentation method provided by the above-mentioned method, which includes:

[0194] obtaining a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, and respectively pre-processing the first remote sensing image and the second remote sensing image;

[0195] input the preprocessed first remote sensing image and the second remote sensing image into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network.

[0196] In another aspect, the present application also provides a non-transitory computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements a damaged building image segmentation method provided by the above method, the method comprising:

[0197] obtaining a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, and respectively pre-processing the first remote sensing image and the second remote sensing image;

[0198] input the preprocessed first remote sensing image and the second remote sensing image into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network.

[0199] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0200] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary general hardware platforms, and of course, can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0201] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for segmenting a damaged building image, characterized by, The method comprises the following steps: obtaining a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, and respectively preprocessing the first remote sensing image and the second remote sensing image; inputting the preprocessed first remote sensing image and the second remote sensing image into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network; the damaged building image segmentation model comprises an encoder and a decoder; the encoder comprises a multi-level feature network, a channel attention mechanism network and a pyramid pooling convolution network connected in sequence; the decoder comprises a feature reconstruction network, the multi-level feature network is connected to the feature reconstruction network through a skip connection, and the pyramid pooling convolution network is connected to the feature reconstruction network; the step of inputting the preprocessed first remote sensing image and the second remote sensing image into the damaged building image segmentation model to obtain damaged building images of different damage levels comprises: based on the multi-level feature network, first convolution features are extracted from the first remote sensing image and second convolution features are extracted from the second remote sensing image layer by layer, target convolution features are obtained based on the first convolution features and the second convolution features, the first convolution features and the second convolution features obtained at each layer are respectively fused to obtain corresponding fusion features, and the fusion features are input into the feature reconstruction network based on the skip connection; based on the channel attention mechanism network and the pyramid pooling convolution network, the target convolution features are processed to obtain encoding features; based on the feature reconstruction network, feature maps are reconstructed through the encoding features and the fusion features to obtain damaged building images of different damage levels.

2. The method of claim 1, wherein, the multi-level feature network comprises a first multi-level feature extraction module, a second multi-level feature extraction module, a feature splicing module and at least one feature fusion module; each level unit in the first multi-level feature extraction module and the second multi-level feature extraction module corresponds, the feature fusion module is connected between other level units in the first multi-level feature extraction module and the second multi-level feature extraction module except the last level unit, the feature fusion module is connected to the feature reconstruction network through a skip connection, and the last level unit in the first multi-level feature extraction module and the second multi-level feature extraction module is connected through the feature splicing module, and the feature splicing module is connected to the feature reconstruction network.

3. The method of claim 2, wherein, the feature reconstruction network comprises a plurality of feature reconstruction modules connected in sequence, the number of the feature reconstruction modules is equal to the number of the level units in the first multi-level feature extraction module, and the feature fusion module is connected to the corresponding feature reconstruction module through a skip connection; wherein the feature reconstruction module is constructed based on a sub-pixel convolution and a progressive attention structure.

4. The method of claim 1, wherein, the training process of the damaged building image segmentation model comprises: Obtaining a training set and a validation set; wherein the training set and the validation set respectively include a plurality of samples, and the samples include sample data and sample labels; Inputting training samples in the training set into an initial damaged building image segmentation model for iterative training until training is completed, to obtain the damaged building image segmentation model; Verifying the trained damaged building image segmentation model through verification samples in the validation set.

5. The method of claim 4, wherein, The initial damaged building image segmentation model includes an encoder and a decoder, and the decoder includes a feature reconstruction network, and the feature reconstruction network includes a plurality of feature reconstruction modules connected in sequence; The inputting training samples in the training set into an initial damaged building image segmentation model for iterative training until training is completed, to obtain the damaged building image segmentation model, includes: Inputting the training samples in the training set into the initial damaged building image segmentation model, to obtain feature reconstruction images output by each feature reconstruction module; Calculating training loss based on the feature reconstruction images; Iteratively training the damaged building image segmentation model based on the training loss until training is completed, to obtain the damaged building image segmentation model.

6. The damaged building image segmentation method according to any one of claims 1 to 5, characterized by, The preprocessing the first remote sensing image and the second remote sensing image respectively includes: Respectively performing orthorectification, geographic matching, resolution unification and cropping preprocessing on the first remote sensing image and the second remote sensing image.

7. The damaged building image segmentation method according to any one of claims 1 to 5, characterized by, After inputting the preprocessed first remote sensing image and the second remote sensing image into the damaged building image segmentation model to obtain damaged building images of different damage levels, the method further includes: Classifying and optimizing the damaged building images of different damage levels through a preset damage classification standard.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to implement the damaged building image segmentation method of any one of claims 1 to 7. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the damaged building image segmentation method of any one of claims 1 to 7.

Images