Damaged building image segmentation method, electronic equipment and storage medium

Abstract

The invention relates to the technical field of aerospace information, and provides a damaged building image segmentation method, electronic equipment and a storage medium, and the method comprises the steps: obtaining a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, preprocessing the first remote sensing image and the second remote sensing image respectively; inputting the preprocessed first remote sensing image and the preprocessed second remote sensing image into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolutional network. In the damaged building image segmentation model, the channel attention mechanism network enhances the feature channel, and the pyramid pooling convolutional network captures context information of different scales, so that the obtained damaged building images of different damage levels can be more accurate.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of space information technology, and in particular to a damaged building image segmentation method, an electronic device and a storage medium. BACKGROUND

[0002] Natural disasters have caused serious loss of life and property to human society. Earthquake is a phenomenon of ground vibration caused by the sudden release of internal energy of the earth, which leads to rapid rupture or dislocation of the crust or lithosphere, accompanied by the propagation of seismic waves. Among all kinds of natural disasters, earthquake is difficult to predict, has a wide range of influence and causes serious damage to cities and infrastructure. Therefore, it is of great significance to strengthen the research and implementation of monitoring, prediction and response measures for earthquake disasters, which will reduce disaster losses and improve social resilience.

[0003] At present, there are various schemes for building damage detection, such as manual visual detection and detection model constructed based on deep learning technology. However, the traditional manual visual detection needs professional personnel to visually detect, although it can obtain high accuracy, but due to its time-consuming and laborious characteristics, it often cannot meet the demand of rapid extraction of damaged buildings in emergency monitoring, and the existing detection model relies on a single data source, and the accuracy of building damage evaluation is poor. SUMMARY

[0004] The present application provides a damaged building image segmentation method, an electronic device and a storage medium, which can solve the problem of poor accuracy of building damage evaluation in the prior art and accurately evaluate the damaged buildings.

[0005] The present application provides a damaged building image segmentation method, which comprises: obtaining a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, and respectively pre-processing the first remote sensing image and the second remote sensing image; inputting the pre-processed first remote sensing image and the second remote sensing image into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network.

[0006] According to the damaged building image segmentation method provided by the present application, the damaged building image segmentation model comprises an encoder and a decoder; The encoder comprises a multi-level feature network, a channel attention mechanism network and a pyramid pooling convolution network connected in sequence; The decoder comprises a feature reconstruction network, the multi-level feature network is connected to the feature reconstruction network by a skip connection, and the pyramid pooling convolution network is connected to the feature reconstruction network. The pre-processed first remote sensing image and the second remote sensing image are input into a damaged building image segmentation model to obtain damaged building images of different damage levels, which comprises: Based on the multi-level feature network, first convolutional features are extracted from the first remote sensing image and second convolutional features are extracted from the second remote sensing image layer by layer, target convolutional features are obtained based on the first convolutional features and the second convolutional features, and the first convolutional features and the second convolutional features obtained by each layer are respectively fused to obtain corresponding fusion features, and the fusion features are input into the feature reconstruction network based on the skip connection; Based on the channel attention mechanism network and the pyramid pooling convolutional network, the target convolutional features are processed to obtain encoding features; Based on the feature reconstruction network, feature map reconstruction is performed through the encoding features and the fusion features to obtain damaged building images of different damage levels.

[0007] According to the damaged building image segmentation method provided by the application, the multi-level feature network comprises a first multi-level feature extraction module, a second multi-level feature extraction module, a feature splicing module and at least one feature fusion module; Each level unit in the first multi-level feature extraction module and the second multi-level feature extraction module corresponds, and the feature fusion module is connected between other level units in the first multi-level feature extraction module and the second multi-level feature extraction module except the last level unit, the feature fusion module is connected with the feature reconstruction network through the skip connection, and the last level unit in the first multi-level feature extraction module and the second multi-level feature extraction module is connected through the feature splicing module, and the feature splicing module is connected with the feature reconstruction network.

[0008] According to the damaged building image segmentation method provided by the application, the feature reconstruction network comprises a plurality of feature reconstruction modules connected in sequence, the number of the feature reconstruction modules is equal to the number of the level units in the first multi-level feature extraction module, and the feature fusion module is connected with the corresponding feature reconstruction module through the skip connection; wherein the feature reconstruction module is constructed based on sub-pixel convolution and progressive attention structure.

[0009] According to the damaged building image segmentation method provided by the application, the training process of the damaged building image segmentation model comprises: Obtaining a training set and a validation set; wherein the training set and the validation set respectively comprise a plurality of samples, and the samples comprise sample data and sample labels; inputting the training samples in the training set into the initial damaged building image segmentation model for iterative training until training is completed, to obtain the damaged building image segmentation model; verify the damaged building image segmentation model trained to be completed through the verification samples in the verification set.

[0010] According to the damaged building image segmentation method provided by the application, the initial damaged building image segmentation model comprises an encoder and a decoder, the decoder comprises a feature reconstruction network, and the feature reconstruction network comprises a plurality of feature reconstruction modules connected in sequence. The inputting the training samples in the training set into the initial damaged building image segmentation model for iterative training until training is completed, to obtain the damaged building image segmentation model, comprises: inputting the training samples in the training set into the initial damaged building image segmentation model, to obtain feature reconstruction images output by each feature reconstruction module; calculating a training loss based on the feature reconstruction images; iteratively training the damaged building image segmentation model based on the training loss until training is completed, to obtain the damaged building image segmentation model.

[0011] According to the damaged building image segmentation method provided by the application, the preprocessing of the first remote sensing image and the second remote sensing image respectively comprises: The first remote sensing image and the second remote sensing image are respectively preprocessed through orthorectification, geographic matching, resolution unification and cropping.

[0012] According to the damaged building image segmentation method provided by the application, after the preprocessed first remote sensing image and second remote sensing image are input into the damaged building image segmentation model to obtain damaged building images of different damage levels, the method further comprises: Classifying and optimizing the damaged building images of different damage levels through a preset damage classification standard.

[0013] The application further provides a damaged building image segmentation device, comprising: A preprocessing module is configured to acquire a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, and to preprocess the first remote sensing image and the second remote sensing image respectively. An image segmentation module is configured to input the preprocessed first remote sensing image and second remote sensing image into a damaged building image segmentation model to obtain damaged building images of different damage levels, wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network.

[0014] The application further provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the building damage image segmentation method according to any one of the above when executing the computer program.

[0015] The application further provides a non-transitory computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the building damage image segmentation method according to any one of the above.

[0016] The application further provides a computer program product, comprising a computer program, wherein the computer program is executed by a processor to implement the building damage image segmentation method according to any one of the above.

[0017] The building damage image segmentation method, the electronic device and the storage medium provided by the application obtain a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, and pre-process the first remote sensing image and the second remote sensing image respectively. A building damage image segmentation model is constructed in advance, and the building damage image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network. The channel attention mechanism network can adaptively enhance the feature channels useful for the current input of the first remote sensing image, the second remote sensing image and the building damage image segmentation task, and suppress the channels that are less important or have more noise. The pyramid pooling convolution network can capture the context information of different scales from local details to global scenes at the same time, and finally output a feature map containing more rich and comprehensive scene information by fusing multi-scale feature representations. Therefore, the building damage images of different damage levels obtained by inputting the pre-processed first remote sensing image and second remote sensing image into the building damage image segmentation model can be more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0019] Figure 1 is a flowchart of the building damage image segmentation method provided by the application.

[0020] Figure 2 is a schematic diagram of the building damage image segmentation model provided by the application.

[0021] Figure 3FIG. 1 is a structural schematic diagram of a ConvNeXt network provided by the present application.

[0022] Figure 4 FIG. 2 is an operation flow schematic diagram of an encoder provided by the present application.

[0023] Figure 5 FIG. 3 is a structural schematic diagram of a BAM module provided by the present application.

[0024] Figure 6 FIG. 4 is a structural schematic diagram of a feature fusion module provided by the present application.

[0025] Figure 7 FIG. 5 is a structural schematic diagram of a channel attention mechanism network provided by the present application.

[0026] Figure 8 FIG. 6 is a schematic diagram of a combination of a channel attention mechanism network and a pyramid pooling convolutional network provided by the present application.

[0027] Figure 9 FIG. 7 is a schematic diagram of sub-pixel convolution upsampling provided by the present application.

[0028] Figure 10 FIG. 8 is a schematic diagram of a remote sensing image used in the experiment provided by the present application.

[0029] Figure 11 FIG. 9 is an output result of each network in the experiment provided by the present application.

[0030] Figure 12 FIG. 10 is a structural schematic diagram of a damaged building image segmentation device provided by the present application.

[0031] Figure 13 FIG. 11 is a structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0032] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0033] In the prior art, although the traditional manual visual interpretation method can obtain high accuracy, it is time-consuming and laborious, and often cannot meet the demand of rapid extraction of damaged buildings in emergency monitoring. In terms of automatic extraction model, due to the implicitness of damage features and high intra-class variance (for example, buildings under the same "complete collapse" label have differences in fragment distribution, inclination angle, etc.), it is difficult to establish a robust mapping relationship between damage degree and image features. This bottleneck promotes the research paradigm to gradually transform into multi-temporal collaborative analysis: by introducing pre-earthquake high-resolution reference images, the model can construct the evolution trajectory of building structure from normal state to damaged state, and capture the mutation threshold of damage signal (such as the texture contrast transition before and after the roof collapse) by using time series difference features. For example, the dual-channel input framework proposed based on the xBD dataset effectively separates the pre-disaster and post-disaster features through the parameter sharing mechanism, realizing the pixel-level time series enhanced expression of building collapse. This spatio-temporal coupling analysis not only enhances the physical interpretability of damage identification through dynamic trajectory modeling, but also reveals the coupling rules between spectral, morphological and geometric features in the post-earthquake damage evolution process (such as the features of broken geometric features and chaotic textures of houses). In addition, most existing deep learning methods rely on a single earthquake event and a single satellite data source for training and modeling. However, in the earthquake emergency scenario, the resolution of image data, the diversity of satellite data sources, and the differences in the epicenter area lead to the richness of building styles and topography, posing a huge challenge to the diversification of training data. The deep learning model of a single data source often cannot effectively cope with the post-earthquake building extraction task of multiple scales and styles. In addition, the damage building detection method based on change detection can only determine whether the building is damaged, and cannot classify and extract buildings of different damage types, which limits the accurate assessment of disaster damage in the post-disaster emergency rescue process. In summary, the current application of remote sensing technology in damaged building extraction in geological disasters has the following main problems: (1) traditional manual interpretation is time-consuming and laborious, and cannot meet the real-time needs of emergency monitoring; (2) most existing models rely on a single satellite data source for training and cannot adapt to the emergency application needs of diverse data sources after a disaster; (3) damage building extraction based on change detection lacks damage classification, affecting the accuracy of post-disaster damage assessment.

[0034] The remote sensing images involved in the present application are all fully authorized data, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and appropriate operation portals are provided for users to choose authorization or refusal.

[0035] Figure 1 is a flowchart of a damaged building image segmentation method according to an exemplary embodiment. As Figure 1As shown, in an exemplary embodiment, the one kind damaged building image segmentation method includes steps 110 to step 120, and details are as follows.

[0036] Step 110, a first remote sensing image before the geological disaster and a second remote sensing image after the geological disaster are acquired, and the first remote sensing image and the second remote sensing image are preprocessed respectively.

[0037] In the embodiment of the application, the first remote sensing image before the geological disaster and the second remote sensing image after the geological disaster are acquired, and the first remote sensing image and the second remote sensing image are preprocessed by radiation calibration, atmospheric correction, orthorectification, geographic matching, resolution unification and cutting.

[0038] Step 120, the preprocessed first remote sensing image and the second remote sensing image are input into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network.

[0039] In the embodiment of the application, a damaged building image segmentation model is constructed in advance, the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network, the channel attention mechanism network can adaptively enhance the feature channels useful for the current input first remote sensing image, second remote sensing image and damaged building image segmentation task, and suppress the channels less important or with larger noise. The pyramid pooling convolution network can capture the context information of different scales from local details to global scenes at the same time, and finally output a feature map containing richer and more comprehensive scene information by fusing multi-scale feature representations. Therefore, the damaged building images of different damage levels obtained can be more accurate.

[0040] In an exemplary embodiment of the application, the damaged building image segmentation model includes an encoder and a decoder. The encoder includes a multi-level feature network, a channel attention mechanism network and a pyramid pooling convolution network connected in sequence. The decoder includes a feature reconstruction network, the multi-level feature network is connected to the feature reconstruction network by jumping, and the pyramid pooling convolution network is connected to the feature reconstruction network. The damaged building image segmentation model is input into the damaged building image segmentation model by the preprocessed first remote sensing image and the second remote sensing image, and damaged building images of different damage levels are obtained, including: Based on the multi-level feature network, first convolutional features are extracted from the first remote sensing image and second convolutional features are extracted from the second remote sensing image layer by layer, target convolutional features are obtained based on the first convolutional features and the second convolutional features, and the first convolutional features and the second convolutional features obtained by each layer are fused respectively to obtain corresponding fusion features, and the fusion features are input into the feature reconstruction network based on the skip connection; Based on the channel attention mechanism network and the pyramid pooling convolutional network, the target convolutional features are processed to obtain encoded features. Based on the feature reconstruction network, feature maps are reconstructed through the encoded features and the fusion features to obtain damaged building images of different damage levels.

[0041] In the embodiment of the application, the damaged building image segmentation model includes an encoder (Encoder) and a decoder (Decoder). In the encoding stage, a multi-level feature fusion mechanism is designed, and through a multi-level feature network, first convolutional features are extracted from the first remote sensing image and second convolutional features are extracted from the second remote sensing image layer by layer. The convolutional feature difference enhancement fusion in different remote sensing images is realized by summation and difference, the fusion features are obtained, and the channel splicing is used to realize the feature interaction to obtain the target convolutional features. In order to further enhance the feature representation ability, the channel attention mechanism network (ChannelAttention Module, CAM) is integrated, and the detail response of the damaged sensitive area in the remote sensing image is strengthened through feature re-labeling. The pyramid pooling convolutional network (Pyramid Pooling Module, PPM) is introduced at the bottom of the encoder to capture the multi-granularity spatial context information, and then the encoded features are obtained.

[0042] In the decoding stage, a cross-level feature transmission path is constructed, the semantic features of different levels of the encoder are integrated by using the skip connection, the high-resolution feature map reconstruction is realized, and the damaged building images of different damage levels are obtained.

[0043] In an exemplary embodiment of the application, the multi-level feature network includes a first multi-level feature extraction module, a second multi-level feature extraction module, a feature splicing module and at least one feature fusion module. Each level unit in the first multi-level feature extraction module and the second multi-level feature extraction module corresponds, and each level unit in the first multi-level feature extraction module and the second multi-level feature extraction module is connected through the feature fusion module, the feature fusion module is connected with the feature reconstruction network in a skip connection manner, and the last level units in the first multi-level feature extraction module and the second multi-level feature extraction module are connected through the feature splicing module, and the feature splicing module is connected with the feature reconstruction network.

[0044] In the embodiment of the application, the first multi-level feature extraction module and the second multi-level feature extraction module in the encoder each have N level units (ConvNeXt-BAM Stage), N is a natural number greater than or equal to 2, and the level units of the two modules correspond. Further, a double-branch architecture is formed, that is, each level unit in the first multi-level feature extraction module extracts a first convolutional feature from the first remote sensing image layer by layer, and each level unit in the second multi-level feature extraction module extracts a second convolutional feature from the second remote sensing image layer by layer. As shown in Figure 2 In the first N-1 levels, the first convolutional features and the second convolutional features extracted by each corresponding level unit are fused by the feature fusion module to obtain fused features, and the fused features are input into the feature reconstruction network through the skip connection. In the Nth level, the first convolutional features and the second convolutional features extracted by the two level units are channel spliced by the feature splicing module (C) to obtain target convolutional features.

[0045] The original FTN model (Feature-wise Transformation Network) adopts a parameter-shared Swin Transformer double-branch architecture, which realizes global context modeling through local self-attention and shift window mechanism, but has inherent defects of high computational complexity and redundant parameters. In the embodiment of the application, the ConvNeXt network is introduced as the core component of the encoder, that is, the first multi-level feature extraction module and the second multi-level feature extraction module are a hybrid coding architecture based on ConvNeXt-BAM, to optimize the feature extraction capability of the FTN model. The ConvNeXt network of the application inherits the advantages of the Transformer training paradigm (including the AdamW optimizer and large-scale data enhancement strategy), and through the collaborative design of large-core depth separable convolution (7x7) and inverted bottleneck structure (Inverted Bottleneck), achieves high performance in the ImageNet classification task.

[0046] AsFigure 3 As shown, the ConvNeXt network adopts a residual structure design, and its core process is in turn a 7x7 large-core depth separable convolution (96 channels), a layer normalization (Layer Norm), a 1x1 convolution to expand the channel to 384 dimensions, a GELU activation function, then a 1x1 convolution to compress back to 96 channels, and finally a residual connection with the input. The ConvNeXt network simulates the global feature interaction capability of the Transformer through a large-core convolution, combines layer normalization and a bottleneck feedforward network (expansion-compression structure), aligns with the Transformer architecture training strategy while preserving the local advantage of convolution, and takes into account global perception and hardware computing efficiency.

[0047] To further strengthen the feature selection capability, the convolution block (ConvBlock) in the ConvNeXt network is used for multi-level feature extraction, and the bottleneck attention module (BAM) is used for feature enhancement, thereby constructing a dual-phase feature extraction framework with strong representation capability. As shown in Figure 4 As shown, by embedding the BAM module in the feature generation part of the four convolution blocks (ConvBlock1-4), the feature selection efficiency of the pre-disaster and post-disaster encoding segments is strengthened through feature weight redistribution before feature cross transmission, and the corresponding features (Feature) are obtained.

[0048] As shown in Figure 5 The specific implementation principle of the BAM module is shown in Figure 5 As shown, the channel branch adopts global average pooling (Global average pool) and full connection (FC) repeated n times (Repeatn times), and then reconstructs and decodes (Expend) to generate a channel attention map. The spatial branch captures multi-scale context information through a dilated convolution, and finally realizes attention weight fusion through a Sigmoid activation. Compared with the window self-attention mechanism of SwinTransformer, the BAM module realizes equivalent feature enhancement effect with a small amount of parameter increment, significantly reducing the computational complexity.

[0049] In the embodiment of the present application, in order to weaken the difference between the first convolutional feature and the second convolutional feature, a feature fusion module (Feature Fusion Module, FFM) is constructed through a difference enhancement module based on FTN. As shown in Figure 6As shown, in the feature fusion process, first, pixel-by-pixel difference operation is performed on the first convolutional feature and the second convolutional feature to strengthen the response value of the change region to obtain a difference feature, and at the same time, pixel-by-pixel summation operation is performed to retain the common features of the unchanged region to obtain a common feature, and finally the difference feature and the common feature are spliced along the channel dimension to form a fusion feature, and then skip connection is performed, thereby constructing a complementary dual-phase feature expression system. The FFM module calculation formula is as follows: ; ; ; Wherein, F pre and F post are the first convolutional feature and the second convolutional feature, F diff is the difference feature, F sum is the common feature, and F enhance is the fusion feature.

[0050] In terms of multi-time feature fusion, the channel attention mechanism is introduced to realize cross-time feature optimization. Figure 7 As shown, after the channel attention mechanism network realizes the fusion of dual-time image features through the combination operation of the convolution layer (conv), batch normalization (BN) and SiLU activation function, the feature is then re-calibrated through the double-path channel attention constructed. The double-path channel attention includes two key stages of feature compression and feature excitation: in the compression stage, the global average pooling (Avg_pool) and the global maximum pooling (Max_pool) are processed in parallel to compress the feature map with a spatial dimension of HxW into a channel description vector, and a channel-level global statistical feature is established; in the excitation stage, the complex correlation between channels is learned through a fully connected layer with a nonlinear activation function to generate a normalized weight coefficient, and the mathematical expression is: ; Wherein, represents the compressed channel statistics, and are the learnable parameter matrices, C represents the number of channels, r represents the compression ratio, δ represents the SiLU activation function, is the Sigmoid function, and w represents the normalized weight coefficient of the output. Finally, the feature is re-calibrated through channel weighting to enhance the response strength of the effective feature channel.

[0051] In the embodiment of the application, in order to improve the adaptability of the model to multi-scale image features, a multi-scale feature enhancement mechanism based on a pyramid pooling convolutional network is provided. As shown in Figure 8As shown, the channel attention mechanism network (ChannelAttention) outputs a feature input pyramid pooling convolutional network, which adopts a multi-level pooling kernel parallel processing strategy, performs 1x1, 3x3, 6x6 and 8x8 four scale parallel adaptive pooling operations (POOL) on the input features, performs feature dimension compression through 1x1 convolution (Conv), performs feature map upsampling (Upsampling) through bilinear interpolation, and finally forms multi-scale fused encoding features through channel splicing (CAT) of different scale feature maps. The mathematical expression of the pyramid pooling convolutional network is: ; Wherein, represents the feature output by the pyramid pooling convolutional network, represents the input feature, represents the i x i size of the pooling operation, is a 1x1 convolution layer, and Upsample is a bilinear interpolation upsampling, represents channel dimension splicing. The pyramid pooling convolutional network significantly enhances the expression ability of the model to multi-scale features of ground objects through multi-receptive field feature fusion.

[0052] In the embodiment of the application, in the feature encoding stage, the first convolutional feature and the second convolutional feature are input into the PPM module after channel dimension splicing, processed through multi-scale feature enhancement, and finally output encoding features containing rich spatio-temporal information. The encoding features have multi-temporal change features and multi-scale spatial features, and can effectively support the subsequent building damage extraction task after geological disasters.

[0053] In an example embodiment of the application, the feature reconstruction network includes a plurality of feature reconstruction modules connected in sequence, the number of the feature reconstruction modules is equal to the number of the level units in the first multi-level feature extraction module, and the feature fusion module is jump-connected with the corresponding feature reconstruction module; wherein the feature reconstruction module is constructed based on sub-pixel convolution and progressive attention structure.

[0054] In the embodiment of the application, as shown in Figure 2 The feature reconstruction network includes a plurality of feature reconstruction modules (PAM-Pixel Shuffle) connected in sequence, the number of the feature reconstruction modules is equal to the number of the level units in the first multi-level feature extraction module, the feature fusion module is jump-connected with the corresponding feature reconstruction module, and the feature reconstruction module is constructed based on sub-pixel convolution (Pixel Shuffle) and progressive attention structure (Progressive Attention Module, PAM).

[0055] In view of the feature reconstruction and information compensation problems in the decoder architecture design, the embodiment of the present application proposes a multi-level feature optimization mechanism. The mechanism constructs a decoder framework through three core modules of reversible upsampling, feature difference attention enhancement and deep supervision, effectively improving the recovery accuracy of disaster change features and the stability of model training.

[0056] In terms of feature reconstruction, the embodiment of the present application uses subpixel convolution for inverse upsampling, that is, the upsampling operation is realized by expanding the channel into a 4x4 area, and the resolution is improved by feature channel reorganization, which can be mathematically expressed as: ; Wherein, PS represents the pixel reorganization operation, Conv is a 3x3 convolution layer, represents the feature output by the subpixel convolution, represents the input feature. However, the spatial context information loss caused by the upsampling process leads to incomplete feature upsampling information, thereby limiting the improvement of classification accuracy. Therefore, the embodiment of the present application constructs a cross-layer feature compensation path: the fusion feature of the kth layer of the encoder and the enhanced feature after the double-channel calculation of the feature reconstruction module of the k+1th layer in the decoder are concatenated in the channel, and cross-scale feature fusion is realized through 3x3 convolution.

[0057] As shown in Figure 9 , in the upsampling process, in order to ensure the integrity of the upsampling feature and retain the key features, the embodiment of the present application adopts the progressive attention structure PAM introduced in FTN. The structure realizes the processing of the upsampling feature by splicing the sum and difference features in the channel dimension, combining convolution operation for feature calculation, and constructing channel attention through global average pooling. The structure can be specifically expressed as: ; ; Wherein, respectively represent the cross-layer connection of the sum and difference features in the encoder, Conv represents the convolution operation, BN represents the batch normalization, ReLU represents the ReLU (Rectified Linear Unit) function, represents the Sigmoid function, GAP represents the global average pooling operation, represents the feature of the kth layer, represents the final attention feature of the kth layer.

[0058] In an exemplary embodiment of the present application, the training process of the damaged building image segmentation model comprises: obtaining a training set and a verification set, wherein the training set and the verification set respectively include a plurality of samples, and each sample includes sample data and a sample label; inputting training samples in the training set into an initial damaged building image segmentation model for iterative training until training is completed, to obtain the damaged building image segmentation model; verifying the trained damaged building image segmentation model by using verification samples in the verification set.

[0059] In the embodiment of the application, the training set and the verification set are obtained, and the samples in the training set and the verification set are consistent with the remote sensing image as described above. Each sample contains a first sample remote sensing image before a geological disaster and a second sample remote sensing image after the geological disaster. The samples in the training set are training samples, and the samples in the verification set are verification samples. The sample label is a corresponding real damaged building image. The training samples in the training set are input into the initial damaged building image segmentation model for iterative training until the training is completed, to obtain the damaged building image segmentation model. The initial damaged building image segmentation model and the damaged building image segmentation model have the same structure, and the difference is only in the parameters in the encoder and the decoder. The trained damaged building image segmentation model is verified by using the verification samples in the verification set, to verify the accuracy and effectiveness of the prediction result of the model.

[0060] In an exemplary embodiment of the application, the initial damaged building image segmentation model includes an encoder and a decoder, the decoder includes a feature reconstruction network, and the feature reconstruction network includes a plurality of feature reconstruction modules connected in sequence. The inputting of the training samples in the training set into the initial damaged building image segmentation model for iterative training until the training is completed, to obtain the damaged building image segmentation model, includes: inputting the training samples in the training set into the initial damaged building image segmentation model, to obtain feature reconstruction images output by each feature reconstruction module; calculating a training loss based on the feature reconstruction images; iteratively training the damaged building image segmentation model based on the training loss until the training is completed, to obtain the damaged building image segmentation model.

[0061] In the embodiment of the application, to further optimize the model training process, a multi-level supervision mechanism is constructed, an auxiliary classifier is connected to the output end of each stage of the decoder, and gradient multi-path propagation is realized by using a deep supervision strategy. That is, the training samples in the training set are input into the initial damaged building image segmentation model, to obtain feature reconstruction images output by each feature reconstruction module, and the feature reconstruction images are input into the auxiliary classifier to obtain a feature reconstruction image label. Figure 2As shown, each feature reconstruction module (PAM-Pixel Shuffle) outputs a feature reconstruction image. The training loss is calculated based on the feature reconstruction image.

[0062] In the embodiment of the present application, since in the remote sensing recognition task, the number of pixels of the target to be recognized and the background are quite different, there is an imbalance problem between positive and negative samples, therefore, the loss function based on Dice coefficient (DSC) and cross-entropy loss function are used to calculate the training loss.

[0063] The loss function of the Dice coefficient is defined as: ; Where A and B are the predicted positive sample set and the real positive sample set respectively, The Dice loss represents the Dice loss. The Diss loss pays attention to the similarity of false positives (FP) and false negatives (FN), so it has good effect in the segmentation task of class imbalance, and improves the problem that the number difference of positive and negative samples in remote sensing recognition is too large.

[0064] Cross-entropy loss (Cross-Entropy Loss, CELoss) is a loss function used in machine learning and deep learning for classification problems. It can be used to measure the difference between the probability distribution of model prediction and the probability distribution of real label, and the formula is as follows: ; In the formula, y i represents the one-hot encoding of the real label, p i represents the probability of the model predicting the i-th class, and L represents the cross-entropy loss. One-hot encoding is a method of converting classification variables into a form that machine learning algorithms can handle better. In one-hot encoding, each category value is represented as a binary vector, except for one position representing the category, which is 1, and the rest are 0.

[0065] The loss of each feature reconstruction image in the embodiment of the present application is the mixed loss of Dice loss and cross-entropy loss, and the specific formula expression is as follows: ; Further, the training loss is: ; Through cross-level linking and layer-by-layer loss function calculation, the embodiment of the present application realizes multi-layer deep supervision network output control, thereby controlling the output classification accuracy.

[0066] In an exemplary embodiment of the present application, the trained damaged building image segmentation model is verified by the verification sample in the verification set, comprising: inputting the verification sample in the trained damaged building image segmentation model to obtain a verification damaged building image; calculating a model evaluation value based on the verification damaged building image and the sample label; verifying the damaged building image segmentation model based on the model evaluation value.

[0067] In the embodiment of the application, the damaged building image segmentation model is input with a verification sample, and a verification damaged building image of different damage levels is obtained through the model, and any one of precision, recall, mean intersection over union (MIoU) and balanced score (F1-score) is used as an evaluation standard of the damaged building image segmentation model.

[0068] In machine learning, the prediction result can be represented by a confusion matrix. As shown in Table 1 below, the confusion matrix describes the performance of a classification model. In the matrix, TP represents the number of samples that are both actual and predicted to be true, FP represents the number of samples that are incorrectly predicted to be true but are actually false; FN represents the number of samples that are incorrectly predicted to be false but are actually true; and TN represents the number of samples that are both actual and predicted to be false. Based on this matrix, indicators such as precision and recall are developed. Considering that the damaged building image segmentation model of the application can output damaged building images of different damage levels, i.e. there are multiple classifications, the damaged building image segmentation task is regarded as multiple binary classifications to calculate the confusion matrix of each class: Table 1

[0069] The calculation formulas of precision (Precision) and recall (Recall) are as follows: ; ; F1 score is the harmonic mean of precision and recall, and the specific calculation formula is as follows: ; MIoU is a statistical measure widely used to evaluate target detection and segmentation models, especially in the field of computer vision and image processing. The formula is as follows: ; where n represents the number of classes.

[0070] In an exemplary embodiment of the application, the pre-processing of the first remote sensing image and the second remote sensing image respectively comprises: The first remote sensing image and the second remote sensing image are respectively subjected to orthorectification, geographic matching, resolution unification and cutting preprocessing.

[0071] In the emergency of geological disasters, the original remote sensing image data collected often has problems such as low spatial resolution, geographic coordinate offset and inaccurate matching of pre-disaster and post-disaster data. Therefore, the data before and after the disaster need to be preprocessed to ensure their applicability and effectiveness. For this purpose, the original first remote sensing image and the second remote sensing image are subjected to specific preprocessing procedures in the embodiment to ensure the quality of the data itself and the practicality of subsequent analysis.

[0072] For the image quality and cloud cover in the original first remote sensing image and the second remote sensing image, the image quality and cloud cover can significantly affect the identification of damaged buildings. The original first remote sensing image and the second remote sensing image data from different sources are preliminarily screened in the embodiment to ensure that only high-quality remote sensing images are retained for further analysis. For remote sensing images with both multispectral and panchromatic bands, a fusion technique is used to improve the resolution of the multispectral image, thereby enhancing its recognizability. When the first remote sensing image and the second remote sensing image are images taken by different sensors at different times, there is usually a standardization problem, that is, there is a significant difference in the gray value and absolute radiance between the two remote sensing images. In order to solve this problem, the collected remote sensing images are subjected to radiometric calibration, that is, the visible light reflectance data in the first remote sensing image and the second remote sensing image are converted into standardized units, thereby improving the readability and reliability of the data.

[0073] Although the atmosphere usually has little effect on visible light, aerosols and water vapor can still affect the propagation of light. Therefore, atmospheric correction is performed on the first remote sensing image and the second remote sensing image in the embodiment to further optimize the image quality.

[0074] There may be a problem of geographic inaccuracy between the first remote sensing image and the second remote sensing image, resulting in misalignment of the coordinates of the first remote sensing image and the second remote sensing image. By performing orthorectification on the visible light image, the geographic positioning accuracy is significantly improved, the distortion caused by the terrain is reduced, and convenience is provided for the subsequent registration of the first remote sensing image and the second remote sensing image. Despite these improvements, there are still differences in details between the first remote sensing image and the second remote sensing image. After orthorectification, the first remote sensing image and the second remote sensing image still need to be subjected to geographic matching to ensure the spatial consistency between the remote sensing images.

[0075] In the case where the resolutions of the first remote sensing image and the second remote sensing image are different, the remote sensing image with higher resolution is subjected to grid decimation in the embodiment to realize the unification of the resolutions of the first remote sensing image and the second remote sensing image, thereby ensuring the pixel-level correspondence between the remote sensing images.

[0076] In order to optimize the input data of subsequent deep learning analysis, the embodiment of the present application systematically crops the remote sensing image into a uniform size of 512*512 pixels. In the cropping process, the images lacking building features are excluded, thereby maintaining the balance between complete buildings and damaged buildings.

[0077] Compared with manual identification, the technical solution provided by the present application only needs to collect remote sensing images before and after geological disasters, automatically pre-process the remote sensing images, unify the coverage range before and after the disaster, remove the influence of geographical distortion of the remote sensing images, unify the resolution, then input the damaged building image segmentation model, automatically obtain damaged building images of different damage levels, and finally realize the extraction of large-scale post-earthquake damaged buildings through manual correction.

[0078] In an exemplary embodiment of the present application, after the pre-processed first remote sensing image and the second remote sensing image are input into the damaged building image segmentation model to obtain damaged building images of different damage levels, the method further comprises: Classifying and optimizing the damaged building images of different damage levels according to a preset damage classification standard.

[0079] In the embodiment of the present application, due to the precision limitation of the model itself and the interference of abnormal factors such as cloud, shooting light, shooting satellite, etc., the technical solution provided by the present application cannot achieve 100% accurate prediction of damaged buildings during implementation. Therefore, the damaged building images of different damage levels obtained by the damaged building image segmentation model can be confirmed and judged again according to the preset damage classification standard. At the same time, when it is determined that the damaged building image obtained by the damaged building image segmentation model is incorrect, the first remote sensing image, the second remote sensing image and the confirmed damaged building image form a training sample, and then the damaged building image segmentation model is retrained.

[0080] In the embodiment of the present application, for the display effect of the damaged building image under the condition of multi-resolution, the embodiment of the present application classifies the damaged building into three categories: intact, slightly damaged, severely damaged and completely collapsed, and forms a preset damage classification standard. According to the specific characteristics of the damage, the specific description is shown in the following table 2: Table 2

[0081] Through the above standard, the severely damaged and completely collapsed buildings are labeled, and the misidentified area is removed to achieve the emergency disaster reduction standard.

[0082] As a core technical means of the earth observation system, remote sensing technology has become an important data source for earth system science research due to its wide coverage, non-contact observation, and immunity to surface conditions. In the geological disaster emergency response scenario, the multi-scale feature extraction capability of high-resolution images can realize the directional detection of the edge fragmentation of damaged buildings, the roof collapse rate and other forms, providing effective emergency rescue information for rapid disaster assessment. The present application uses remote sensing means, combines the morphological features of buildings in post-disaster remote sensing images and the significant feature transformation of buildings before and after the disaster, and forms an automatic monitoring model, which has the following advantages: when an earthquake occurs, this method provides an automatic image processing, resampling, cropping and mosaicking process to ensure the standardization of the input remote sensing image; the present application improves the change detection model to realize the automatic extraction of damaged buildings based on pre-disaster and post-disaster high-resolution remote sensing images.

[0083] In the earthquake disaster scenario, the damaged building image segmentation model of the present application is trained based on remote sensing images in the earthquake scenario, and the damaged building image segmentation model can be called a seismic damage house extraction deep learning model (Building Demage Grading Inspection Net, BDGINet). In view of the complex characteristics of remote sensing image change detection in the earthquake disaster scenario, the present application constructs a multi-dimensional comparison experiment framework to systematically evaluate the detection performance of the technical solution provided by the present application in a multi-scale complex scenario. As shown in Table 3 below, five representative benchmark models are selected for comparison and analysis with the damaged building image segmentation model provided by the present application, including classical semantic segmentation networks (Unet, Unet++, DeeplabV3+), dual-phase detection networks (BDD-Net) and feature transformation networks (FTN). All models are initialized with ImageNet pre-trained weights, using ResNet101 as a unified encoder, and BDD-Net uses EfficientNet-B4 encoder. The model parameters are optimized on the same training set through a fine-tuning strategy.

[0084] The quantitative analysis results show that the traditional single-phase segmentation network has significant limitations in the post-earthquake complex scene: the F1-score of the Unet series network on the test set is less than 45%, and the MIoU index is not more than 46%. This is mainly due to the high similarity of collapsed buildings and bare ground in the spectrum after the earthquake, making it difficult for the model to establish effective class separability criteria. It is worth noting that although DeeplabV3+ expands the receptive field through dilated convolution, its recall rate of 54.8% still indicates that there are a large number of missed detections, which verifies that the complex damage features have a significant inhibitory effect on the detection performance of single-phase methods.

[0085] The comparative experiment of the dual-time detection network further reveals the key influence of the model architecture design. Although the BDD-Net adopts the dual-branch feature extraction strategy, it has a significant overfitting tendency under limited training data, and there is a significant imbalance between the accuracy of 84.7% and the recall rate of 52.4%, indicating that the model has excessive inhibition problems for negative samples. In comparison, the FTN and the model provided by the application realize the effective improvement of the feature expression ability through the deep supervision learning mechanism and the parameter sharing strategy. In particular, the multi-scale feature fusion architecture proposed in the application improves the F1-score (69.6%) and MIoU (83.3%) by 1.5 and 1.6 percentage points respectively, which verifies the synergistic optimization effect of the improved feature extraction module and the reversible upsampling structure.

[0086] Table 3

[0087] As shown in Figure 10 and Figure 11 , (a) represents the remote sensing image before the earthquake, (b) represents the remote sensing image of the earthquake, (c) represents the ground truth, (d) represents the Unet network result, (e) represents the Unet++ network result, (f) represents the DeeplabV3+ network result, (g) represents the BDD-Net network result, (h) represents the FTN network result, and (i) represents the BDGINet result. Red represents completely collapsed buildings, orange represents severely damaged buildings, and black represents the background.

[0088] As can be seen from the pictures of Figure 10 and Figure 11 , in the experimental remote sensing images selected by the application, the architectural style, the earthquake damage degree, the image lighting condition and the image resolution difference pose great challenges to the segmentation and classification of the model. The conventional semantic segmentation model with double image input is significantly insufficient in segmentation ability for this experimental data set. The classical semantic segmentation networks Unet, Unet++, and DeeplabV3+ have poor detection effect on damaged buildings during the experiment, and can only locate a few damaged building locations. In addition, the BDD-Net network responds well to the completely collapsed area through the change detection structure, but due to the limitation of the network structure scale, it is almost impossible to detect single damaged buildings for images with limited spatial resolution. The FTN network benefits from the window displacement and self-attention mechanism of the Swin-Transformer network and effectively extracts most of the damaged buildings.

[0089] Further analysis shows that the performance advantage of the damaged building image segmentation model provided by the application is derived from three key designs: first, the pyramid pooling convolutional network enhances the model's ability to capture multi-scale features in damaged areas through multi-receptive field feature fusion; second, the feature difference enhancement mechanism effectively amplifies the temporal change information of the dual-phase images; and finally, the deep supervision strategy optimizes the feature decoding process through multi-level gradient propagation. Experimental results show that these innovative designs significantly improve the discriminant ability of the damaged building image segmentation model in the feature confusion scene, providing a new technical approach for disaster emergency remote sensing detection.

[0090] The damaged building image segmentation device provided by the application is described below, and the damaged building image segmentation device described below can be correspondingly referred to the damaged building image segmentation method described above. It should be noted that the device provided by the embodiment below and the method provided by the above embodiment belong to the same concept, and the specific manner in which each module and unit performs operations has been described in detail in the method embodiment, which will not be described here.

[0091] In an exemplary embodiment of the application, please refer to Figure 12 , Figure 12 A damaged building image segmentation device according to an exemplary embodiment is shown, which includes the following modules.

[0092] The preprocessing module 1210 is configured to obtain a first remote sensing image before a geological disaster and a second remote sensing image after a geological disaster, and pre-process the first remote sensing image and the second remote sensing image, respectively; The image segmentation module 1220 is configured to input the pre-processed first remote sensing image and the second remote sensing image into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolutional network.

[0093] In an exemplary embodiment of the application, the damaged building image segmentation model includes an encoder and a decoder; The encoder includes a multi-level feature network, a channel attention mechanism network and a pyramid pooling convolutional network connected in sequence; The decoder includes a feature reconstruction network, the multi-level feature network is connected to the feature reconstruction network through a skip connection, and the pyramid pooling convolutional network is connected to the feature reconstruction network; The image segmentation module 1220 includes: The extraction submodule is configured to extract first convolutional features from the first remote sensing image and second convolutional features from the second remote sensing image layer by layer based on the multi-level feature network, obtain target convolutional features based on the first convolutional features and the second convolutional features, perform feature fusion on the first convolutional features and the second convolutional features obtained by each layer respectively to obtain corresponding fused features, and input the fused features into the feature reconstruction network based on a skip connection. The processing submodule is configured to process the target convolutional features based on the channel attention mechanism network and the pyramid pooling convolutional network to obtain encoded features. The reconstruction submodule is configured to perform feature map reconstruction based on the encoded features and the fused features by the feature reconstruction network to obtain damaged building images of different damage levels.

[0094] In an example embodiment of the present application, the multi-level feature network comprises a first multi-level feature extraction module, a second multi-level feature extraction module, a feature splicing module, and at least one feature fusion module. Each level unit in the first multi-level feature extraction module and the second multi-level feature extraction module corresponds to each other, the feature fusion module is connected between other level units except the last level unit in the first multi-level feature extraction module and the second multi-level feature extraction module, the feature fusion module is connected to the feature reconstruction network in a skip connection, and the feature splicing module is connected between the last level unit in the first multi-level feature extraction module and the second multi-level feature extraction module, and the feature splicing module is connected to the feature reconstruction network.

[0095] In an example embodiment of the present application, the feature reconstruction network comprises a plurality of feature reconstruction modules connected in sequence, the number of the feature reconstruction modules is equal to the number of the level units in the first multi-level feature extraction module, and the feature fusion module is connected to the corresponding feature reconstruction module in a skip connection; wherein the feature reconstruction module is constructed based on sub-pixel convolution and progressive attention structure.

[0096] In an example embodiment of the present application, the training process of the damaged building image segmentation model comprises: The acquisition module is configured to acquire a training set and a validation set; wherein the training set and the validation set respectively comprise a plurality of samples, and the samples comprise sample data and sample labels. The iterative training module is configured to input training samples in the training set into an initial damaged building image segmentation model for iterative training until the training is completed to obtain the damaged building image segmentation model. The verification module is configured to verify the trained image segmentation model of the damaged building using verification samples from the verification set.

[0097] In an exemplary embodiment of the present invention, the initial damaged building image segmentation model includes an encoder and a decoder, the decoder includes a feature reconstruction network, and the feature reconstruction network includes a plurality of feature reconstruction modules connected in sequence; The iterative training module includes: The input submodule is configured to input the training samples in the training set into the initial damaged building image segmentation model to obtain the feature reconstruction images output by each feature reconstruction module; The computational submodule is configured to calculate the training loss based on the reconstructed image using the features. The iterative training submodule is configured to iteratively train the image segmentation model of the damaged building based on the training loss until the training is completed, thereby obtaining the image segmentation model of the damaged building.

[0098] In an exemplary embodiment of the present invention, the preprocessing module 1210 includes: The preprocessing submodule is configured to perform orthorectification, geographic matching, resolution unification, and cropping preprocessing on the first remote sensing image and the second remote sensing image, respectively.

[0099] In an exemplary embodiment of the present invention, the damaged building image segmentation apparatus further includes: The classification and optimization processing module is configured to classify and optimize images of damaged buildings of different damage levels based on preset damage classification standards.

[0100] Figure 13 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 13 As shown, the electronic device may include: a processor 1310, a communications interface 1320, a memory 1330, and a communication bus 1340, wherein the processor 1310, the communications interface 1320, and the memory 1330 communicate with each other via the communication bus 1340. The processor 1310 can call logical instructions in the memory 1330 to execute a damaged building image segmentation method, which includes: Acquire the first remote sensing image before the geological disaster and the second remote sensing image after the geological disaster, and preprocess the first remote sensing image and the second remote sensing image respectively; input the preprocessed first remote sensing image and the second remote sensing image into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network.

[0101] In addition, the logical instructions in the memory 1330 described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0102] On the other hand, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program is executed by a processor, so that the computer can execute the damaged building image segmentation method provided by the above-mentioned methods, and the method comprises: obtaining a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, and respectively pre-processing the first remote sensing image and the second remote sensing image; inputting the preprocessed first remote sensing image and the second remote sensing image into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network.

[0103] In another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the damaged building image segmentation method provided by the above-mentioned methods, and the method comprises: obtaining a first remote sensing image before a geological disaster and a second remote sensing image after the geological disaster, and respectively pre-processing the first remote sensing image and the second remote sensing image; The preprocessed first remote sensing image and the second remote sensing image are input into a damaged building image segmentation model to obtain damaged building images of different damage levels; wherein the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolution network.

[0104] The device embodiments described above are merely illustrative, wherein the units illustrated as separate components can or can not be physically separated, and the components illustrated as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0105] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0106] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for image segmentation of damaged buildings, characterized in that, include: Acquire the first remote sensing image before the geological disaster and the second remote sensing image after the geological disaster, and preprocess the first remote sensing image and the second remote sensing image respectively; The preprocessed first and second remote sensing images are input into the damaged building image segmentation model to obtain damaged building images of different damage levels; wherein, the damaged building image segmentation model is constructed based on a channel attention mechanism network and a pyramid pooling convolutional network.

2. The method for segmenting images of damaged buildings according to claim 1, characterized in that, The image segmentation model for the damaged building includes an encoder and a decoder; The encoder comprises a multi-level feature network, a channel attention mechanism network, and a pyramid pooling convolutional network connected in sequence. The decoder includes a feature reconstruction network, the multi-level feature network is skipped to the feature reconstruction network, and the pyramid pooling convolutional network is connected to the feature reconstruction network. The step of inputting the preprocessed first and second remote sensing images into the damaged building image segmentation model to obtain damaged building images of different damage levels includes: Based on the multi-level feature network, first convolutional features are extracted from the first remote sensing image and second convolutional features are extracted from the second remote sensing image layer by layer. Target convolutional features are obtained based on the first and second convolutional features. Feature fusion is performed on the first and second convolutional features obtained from each layer to obtain corresponding fused features. The fused features are then input into the feature reconstruction network based on skip connections. Based on the channel attention mechanism network and the pyramid pooling convolutional network, the target convolutional features are processed to obtain encoded features; Based on the feature reconstruction network, feature maps are reconstructed using the encoded features and the fused features to obtain images of damaged buildings at different damage levels.

3. The method for image segmentation of damaged buildings according to claim 2, characterized in that, The multi-level feature network includes a first multi-level feature extraction module, a second multi-level feature extraction module, a feature concatenation module, and at least one feature fusion module; Each level unit in the first multi-level feature extraction module and the second multi-level feature extraction module corresponds to another level unit. Except for the last level unit, the other level units in the first multi-level feature extraction module and the second multi-level feature extraction module are connected through the feature fusion module. The feature fusion module is connected to the feature reconstruction network in a skip connection. The last level unit in the first multi-level feature extraction module and the second multi-level feature extraction module is connected through the feature splicing module. The feature splicing module is connected to the feature reconstruction network.

4. The method for segmenting images of damaged buildings according to claim 3, characterized in that, The feature reconstruction network includes multiple feature reconstruction modules connected in sequence. The number of feature reconstruction modules is equal to the number of hierarchical units in the first multi-level feature extraction module. The feature fusion module is skipped to the corresponding feature reconstruction module. The feature reconstruction module is constructed based on sub-pixel convolution and progressive attention structure.

5. The method for image segmentation of damaged buildings according to claim 1, characterized in that, The training process of the image segmentation model for damaged buildings includes: Obtain a training set and a validation set; wherein the training set and the validation set each include multiple samples, and the samples include sample data and sample labels; The training samples in the training set are input into the initial damaged building image segmentation model for iterative training until the training is completed, and the damaged building image segmentation model is obtained. The trained image segmentation model for damaged buildings is validated using validation samples in the validation set.

6. The method for segmenting images of damaged buildings according to claim 5, characterized in that, The initial damaged building image segmentation model includes an encoder and a decoder. The decoder includes a feature reconstruction network, which includes multiple feature reconstruction modules connected in sequence. The step of inputting training samples from the training set into the initial damaged building image segmentation model for iterative training until training is complete, thereby obtaining the damaged building image segmentation model, includes: The training samples in the training set are input into the initial damaged building image segmentation model to obtain the feature reconstruction images output by each feature reconstruction module; The training loss is calculated based on the reconstructed image using the aforementioned features; The image segmentation model of the damaged building is iteratively trained based on the training loss until the training is completed, thus obtaining the image segmentation model of the damaged building.

7. The method for image segmentation of damaged buildings according to any one of claims 1 to 6, characterized in that, The preprocessing of the first remote sensing image and the second remote sensing image includes: The first and second remote sensing images are preprocessed by orthorectification, geographic matching, resolution unification, and cropping, respectively.

8. The method for image segmentation of damaged buildings according to any one of claims 1 to 6, characterized in that, After inputting the preprocessed first and second remote sensing images into the damaged building image segmentation model to obtain damaged building images of different damage levels, the method further includes: By setting up damage classification standards, images of damaged buildings of different damage levels are classified and optimized.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the damaged building image segmentation method as described in any one of claims 1 to 8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image segmentation method for damaged buildings as described in any one of claims 1 to 8.

Images