A post-earthquake damaged building identification method and system based on semantic scene migration

Abstract

The application discloses a post-earthquake damaged building identification method and system based on semantic scene migration, and the method comprises the following steps: annotating building damage samples for remote sensing images, constructing a building damage sample dataset, and performing sample expansion through a sample augmentation strategy; a general semantic segmentation network encoder is constructed, a decoder with hierarchical semantic feature extraction and dynamic adaptive identification target capability is designed, and a damaged building identification model suitable for semantic scene migration is reconstructed; the model is trained, and the newly identified damaged buildings are used for cyclic iteration training until the identification precision converges to complete the model training; a damaged building area image is input into the model for identification, pixel identification results are converted into damaged area vectors, spatial comprehensive analysis is performed on the damaged area vectors and building vector data, and a damaged building vector graph obtained through identification is generated. Through the technical scheme of the application, semantic scene migration of the model is realized, and intelligent and accurate extraction of damaged buildings is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pattern recognition, and in particular to a post-earthquake type damaged building identification method based on semantic scene migration and a post-earthquake type damaged building identification system based on semantic scene migration. BACKGROUND

[0002] Buildings are one of the most severely damaged disaster-bearing bodies by high-intensity earthquakes. The transverse wave and longitudinal wave generated by high-intensity earthquakes can jointly destroy buildings, causing the buildings to be damaged or even collapsed. Considering the characteristics of densely distributed buildings and concentrated population in residential areas in China, once a high-intensity earthquake occurs, the damaged and collapsed buildings will often cause relatively large casualties and property losses. Therefore, it is crucial to quickly and accurately identify the damaged buildings after an earthquake for the timely development of disaster prevention and mitigation work.

[0003] With the rapid development of remote sensing technology, the revisit period of remote sensing images is getting shorter and shorter, and the spatial resolution is getting higher and higher, providing a high-quality data source for post-earthquake building damage identification, and it has gradually become possible to quickly and accurately identify large-scale building damage. Based on high-resolution remote sensing images, the identification of damaged buildings can quickly grasp the building disaster situation in the post-earthquake area, and the identification efficiency is greatly improved compared with traditional field identification methods. With the rapid development of artificial intelligence technology in recent years, various deep learning models with strong feature extraction and expression ability have been proposed and applied in the field of remote sensing image interpretation, and have achieved much better performance than traditional methods. Therefore, applying this advanced technology of deep learning to the identification of damaged buildings is expected to further improve the accuracy of the identification of damaged buildings, which is of great significance.

[0004] However, there are still many problems to be overcome in the current identification of damaged buildings based on remote sensing images and deep learning technology:

[0005] First, reliable remote sensing data sets of earthquake-damaged buildings are lacking, and the sample set of damaged buildings is small in number and low in quality, which greatly limits the feature extraction ability of deep learning models, resulting in low accuracy of the identification results.

[0006] Second, classical deep learning models are usually developed to solve computer vision problems, and the training samples are usually natural images. However, remote sensing data is different from natural images, and the diversity of spatial resolution, temporal resolution, spectral resolution, and load type makes it difficult to achieve good results by directly applying deep learning large models to the identification of damaged buildings. SUMMARY

[0007] To solve the above problems, the application provides a post-earthquake damaged building identification method and system based on semantic scene migration, which increases the high-quality damaged building sample set by adopting a scene reconstruction sample augmentation strategy, effectively expands the damaged samples, realizes scene migration of the damaged target, constructs a deep learning model decoder structure integrating hierarchical semantic features, skillfully combines the advantages of remote sensing big data and visual big models, effectively improves the problems of lack of damaged building sample data and weak adaptability of general semantic segmentation models in damaged building identification tasks, makes the model dynamically adapt to the damaged building identification task, realizes intelligent and accurate extraction of damaged buildings, and has important significance for improving the intelligent level of earthquake disaster acquisition and post-disaster response.

[0008] To achieve the above purpose, the application provides a post-earthquake damaged building identification method based on semantic scene migration, which comprises the following steps:

[0009] For post-earthquake optical remote sensing image annotation of building damage samples, a building damage sample dataset is constructed, and sample expansion is performed on the damaged buildings in the building damage sample dataset based on a scene reconstruction sample augmentation strategy;

[0010] A general semantic segmentation network encoder is constructed, a decoder with hierarchical semantic feature extraction and dynamic adaptive identification target capability is designed, and a damaged building identification model for adaptive semantic scene migration is reconstructed;

[0011] The expanded building damage sample dataset is used to train the damaged building identification model, and the building damage sample dataset is updated with newly identified damaged building data during the training process, and the updated building damage sample dataset is used to iteratively train the damaged building identification model until the identification accuracy converges and the model training is completed;

[0012] The damaged building area image is input into the trained damaged building identification model for identification, the pixel recognition result is converted into a damaged area vector, and spatial comprehensive analysis is performed on the corresponding area building vector data to generate a damaged building vector graph obtained by identification.

[0013] In the above technical solution, preferably, the specific process of expanding the samples of the damaged buildings in the building damage sample dataset based on the scene reconstruction sample augmentation strategy comprises:

[0014] For the damaged building images in the building damage sample dataset, the size of the damaged building images is determined, and the damaged building images are cropped;

[0015] The horizontal number, vertical number and row and column coordinates of the cropped image are calculated to obtain a target sample, and an initial target semantic scene is generated;

[0016] The target sample is subjected to position transformation, rotation and flip operation to generate a target image pseudo-label, and the target image pseudo-label is spliced with the target sample to generate a target pseudo-image, thereby realizing sample expansion.

[0017] In the above technical solution, preferably, the general semantic segmentation network encoder is constructed, a decoder with hierarchical semantic feature extraction and dynamic adaptive target recognition capability is designed, a damaged building recognition model for adaptive semantic scene migration is reconstructed, and the specific process includes:

[0018] The encoder based on the general semantic segmentation network is constructed to be capable of extracting features of preset hierarchical semantic features of the damaged building, thereby obtaining hierarchical features.

[0019] The cross feature fusion module is used to extract deep features and cross fuse the hierarchical features, thereby obtaining hierarchical feature fusion information related to the damaged building.

[0020] The SFE is constructed by using three groups of convolution modules and three groups of EPSA modules, the input image can be down-sampled eight times through three convolution processes, the shallow information of the image can be extracted through the three groups of EPSA modules, thereby the building detail semantic information lost in the encoder due to down-sampling can be extracted, and the shallow semantic features are obtained.

[0021] The hierarchical features and the shallow semantic features are spliced in the feature dimension, the feature map after splicing can be restored to the size of the input image through two convolution layers, thereby realizing the adaptation of the model to the recognized target semantic scene.

[0022] In the above technical solution, preferably, the damaged building recognition model is trained by using the expanded damaged building sample dataset, the damaged building data newly recognized in the training process is used to update the damaged building sample dataset, the updated damaged building sample dataset is used to perform cyclic iterative training on the damaged building recognition model, and the model training is completed until the recognition accuracy converges, and the specific process includes:

[0023] The damaged building recognition model is trained by using the expanded damaged building sample dataset.

[0024] The damaged building recognition model is trained by using the sum of binary cross entropy loss and Dice loss as the total loss function.

[0025] select a preset probability threshold as a classification standard of the damaged building, and supplement the pixels with a damaged building probability value higher than the preset probability threshold in the damaged building identification model identification result to the building damage sample dataset for data augmentation;

[0026] The building damage sample dataset is repeatedly updated, and the building damage identification model is iteratively trained using the updated building damage sample dataset until the total loss function converges to complete the model training process.

[0027] In the above technical solution, preferably, the specific process of spatial comprehensive analysis of the corresponding area building vector data comprises:

[0028] The open source building contour dataset is introduced as auxiliary data to spatially constrain the building damage area identified by the damaged building identification model, and the single building contour vector having an intersection or containing spatial relationship with the building damage area is identified as a damaged building;

[0029] The proportion of the intersection area of the building contour vector and the building damage area to the area of the building contour vector is calculated to determine the damage degree information of the building.

[0030] The application also provides a post-earthquake damaged building identification system based on semantic scene migration, which applies the post-earthquake damaged building identification method based on semantic scene migration disclosed in any one of the above technical solutions, comprising:

[0031] A sample data construction module is configured to label building damage samples for post-earthquake optical remote sensing images, construct a building damage sample dataset, and perform sample augmentation on the damaged buildings in the building damage sample dataset through a sample augmentation strategy based on scene reconstruction;

[0032] An identification model construction module is configured to construct a general semantic segmentation network encoder, design a decoder with hierarchical semantic feature extraction and dynamic adaptive identification target capability, and reconstruct an adaptive semantic scene migration damaged building identification model;

[0033] An identification model training module is configured to train the damaged building identification model using the augmented building damage sample dataset, update the building damage sample dataset with newly identified damaged building data during the training process, and iteratively train the damaged building identification model using the updated building damage sample dataset until the identification accuracy converges to complete the model training.

[0034] The damaged building identification module is used for inputting a damaged building area image into the trained damaged building identification model to identify the pixel identification result, converting the pixel identification result into a damaged area vector, and performing spatial comprehensive analysis on the building vector data of the corresponding area to generate a damaged building vector graph obtained by identification.

[0035] In the technical solution, preferably, the sample data construction module is specifically used for:

[0036] For the damaged building image in the building damage sample data set, the size of the damaged building image is determined, and the damaged building image is cropped;

[0037] The horizontal number, vertical number and row-column coordinates of the cropped image are calculated to obtain a target sample, and an initial target semantic scene is generated;

[0038] The target sample is subjected to position transformation, rotation and flipping operations to generate a target image pseudo-label, and the target image pseudo-label and the target sample are spliced to generate a target pseudo-image, thereby realizing sample expansion.

[0039] In the technical solution, preferably, the identification model construction module is specifically used for:

[0040] An encoder based on a general semantic segmentation network is constructed to be capable of extracting features of a preset level of semantic features of the damaged building to obtain hierarchical features;

[0041] A cross-feature fusion module is used to extract deep features and cross-fuse the hierarchical features to obtain hierarchical feature fusion information related to the damaged building;

[0042] Three groups of convolution modules and three groups of EPSA modules are used to construct an SFE, which can perform eight times of down-sampling on the input image through three times of convolution processes, and can extract shallow information of the image through the three groups of EPSA modules, so as to extract the building detail semantic information lost in the encoder due to down-sampling, and obtain shallow semantic features;

[0043] The hierarchical features and the shallow semantic features are spliced in the feature dimension, and the spliced feature map can be restored to the size of the input image through two convolution layers, thereby realizing the adaptation of the model to the identified target semantic scene.

[0044] In the technical solution, preferably, the identification model training module is specifically used for:

[0045] The expanded building damage sample data set is used to train the damaged building identification model;

[0046] The sum of the binary cross-entropy loss and the Dice loss is taken as a total loss function to train the damaged building recognition model.

[0047] A preset probability threshold is selected as a classification standard of the damaged building, and pixels with a probability value higher than the preset probability threshold in the damaged building recognition result of the damaged building recognition model are taken as new recognized damaged samples and supplemented to the building damage sample dataset for data augmentation.

[0048] The building damage sample dataset is repeatedly updated, and the updated building damage sample dataset is used to iteratively train the damaged building recognition model until the total loss function converges to complete the model training process.

[0049] In the above technical solution, preferably, the damaged building recognition module is specifically used for:

[0050] An open-source building contour dataset is introduced as auxiliary data to spatially constrain the building damage area recognized by the damaged building recognition model, and a single building contour vector having an intersection or containing spatial relationship with the building damage area is recognized as a damaged building.

[0051] The proportion of the intersection area of the building contour vector and the building damage area to the area of the building contour vector is calculated to determine the damage degree information of the building.

[0052] Compared with the prior art, the beneficial effects of the present application are: by adopting a scene reconstruction sample augmentation strategy to increase the high-quality damaged building sample set, the damaged samples are effectively expanded, the scene migration of the damaged target is realized, the deep learning model decoder structure is constructed by fusing the hierarchical semantic features, the multi-level semantic features extracted by the general semantic segmentation large model are ingeniously fused with the image shallow features, the semantic scene migration is realized, the advantages of remote sensing big data and visual big model are ingeniously combined, the problems of lack of damaged building sample data and weak adaptability of the general semantic segmentation model in the damaged building recognition task are effectively improved, the model can dynamically adapt to the damaged building recognition task, the intelligent and accurate extraction of the damaged building is realized, and it is of great significance to improve the intelligent level of earthquake disaster acquisition and post-disaster response. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 A flowchart of a post-earthquake damaged building recognition method based on semantic scene migration disclosed by an embodiment of the present application is disclosed.

[0054] Figure 2 A flowchart of image overlapping and cropping disclosed by an embodiment of the present application is disclosed.

[0055] Figure 3A schematic diagram of a scene reconstruction process disclosed for an embodiment of the present application;

[0056] Figure 4 A network structure schematic diagram of a damaged building identification model disclosed for an embodiment of the present application;

[0057] Figure 5 A basic process schematic diagram of damaged building information extraction disclosed for an embodiment of the present application;

[0058] Figure 6 A module schematic diagram of a post-earthquake damaged building identification system based on semantic scene migration disclosed for an embodiment of the present application.

[0059] In the figure, the correspondence between each component and the reference numeral is:

[0060] 1. Sample data construction module, 2. Identification model construction module, 3. Identification model training module, 4. Damaged building identification module. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0062] The present application will be described in further detail below with reference to the drawings:

[0063] As shown in the figure, according to a post-earthquake damaged building identification method based on semantic scene migration provided by the present application, the method comprises: Figure 1

[0064] For labeling building damage samples in post-earthquake optical remote sensing images, a building damage sample dataset is constructed, and sample expansion of damaged buildings in the building damage sample dataset is performed through a sample augmentation strategy based on scene reconstruction;

[0065] A general semantic segmentation network encoder is constructed, a decoder with the ability of hierarchical semantic feature extraction and dynamic adaptive identification target is designed, and a damaged building identification model adaptive to semantic scene migration is reconstructed;

[0066] ​The building damage sample data set is expanded, the building damage recognition model is trained, and the building damage sample data set is updated during the training process, and the building damage recognition model is iteratively trained until the recognition accuracy converges to complete model training.

[0067] The damaged building area image is input into the trained building damage recognition model for recognition, the pixel recognition result is converted into a damaged area vector, and spatial comprehensive analysis is performed on the corresponding area building vector data to generate a damaged building vector graph obtained by recognition.

[0068] In this embodiment, the scene reconstruction sample augmentation strategy is adopted to increase the high-quality damaged building sample set, effectively expanding the damage sample, realizing the scene migration of the damage target, constructing a deep learning model decoder structure that fuses hierarchical semantic features, and skillfully combining the advantages of remote sensing big data and visual big models to effectively improve the problems of lack of damaged building sample data and weak adaptability of general semantic segmentation models in damaged building recognition tasks, so that the model can dynamically adapt to the damaged building recognition task, and the intelligent and accurate extraction of damaged buildings is realized, which is of great significance to improve the intelligent level of earthquake disaster acquisition and post-disaster response.

[0069] Specifically, first, the building damage samples are labeled based on post-earthquake optical images, a damaged building data set is constructed, a scene reconstruction sample augmentation strategy is adopted to increase the high-quality damaged building sample set, the Segment Anything Model (SAM) visual big model structure is adjusted to adapt to the building damage recognition task, semantic scene migration is realized, the model is trained using self-labeled damaged building samples, and automatic recognition of damaged buildings in high-resolution remote sensing images is completed.

[0070] In the implementation process, the following steps are performed:

[0071] Step one, based on post-earthquake optical high-resolution remote sensing images, building damage samples are labeled in the images, a building damage sample data set is constructed, sample data is geometrically transformed, color transformed, and overlapped and cropped for data enhancement processing, sample augmentation of damaged buildings is performed through a scene reconstruction-based sample augmentation strategy, semantic scene migration of the damage target is realized, the problem of few and low-quality damaged building sample sets is solved, and more abundant training data is provided for feature extraction of a deep learning model.

[0072] Step two, build a general semantic segmentation network model encoder, through the design of a model decoder with hierarchical semantic feature extraction and dynamic adaptive target recognition capability, realize the semantic scene migration of the basic visual model to the damaged building recognition model, and use the pre-trained model weight on the large-scale natural scene dataset for parameter initialization.

[0073] Step three, fine-tune the model with the building damage sample dataset generated in step one, update the training samples with newly recognized damaged buildings during the model training process, retrain the model with updated training samples to improve the robustness and accuracy of the model in recognizing damaged buildings, and continuously iterate this process until the optimal accuracy is achieved, realizing model training based on semantic scene migration.

[0074] Step four, use the trained model in step three to perform pixel-level recognition on the damaged building area, convert the pixel recognition result to a damaged area vector, and perform spatial comprehensive analysis with the latest building vector data collected in the post-earthquake disaster area to generate a damaged building vector and obtain the area of the damaged building. Further, classic precision evaluation indicators can be used to evaluate the model recognition accuracy from three aspects: pixel-level damage area, number of damaged buildings, and area.

[0075] In the above embodiment, preferably, the specific process of sample expansion of the damaged buildings in the building damage sample dataset through the sample augmentation strategy based on scene reconstruction includes:

[0076] As shown in Figure 2 , for the damaged building images in the building damage sample dataset, the size A x B of the damaged building images is determined, and the damaged building images are cropped, and the size of the cropped small images is C x C, and the overlap rate between adjacent small images is P;

[0077] The horizontal number m, vertical number n, and row and column coordinates (left upper corner X coordinate and Y coordinate) of the cropped images are calculated to obtain the target sample and generate the initial target semantic scene, and the specific formula is as follows:

[0078]

[0079] x = i x C x (1-P)

[0080] y = j x C x (1-P)

[0081] Wherein, i∈(0,m-1), j∈(0,n-1), i and j represent the row index and column index of the small image in the original remote sensing image respectively.

[0082] Q i,j = Q[x:x+C-1,y:y+C-1]

[0083] wherein Q represents original remote sensing image grid data;

[0084] As shown in the following formula (1), the target sample is subjected to position transformation, rotation and flip operation to generate a target image pseudo label, and the target image pseudo label is spliced with the target sample to generate a target pseudo image, thereby realizing sample expansion. Figure 3 new =f1(f2(f3(L)))

[0085]

[0086]

[0087] wherein L new represents a target image pseudo label, L represents a target image pseudo label, f1, f2 and f3 represent position transformation, rotation and flip operation respectively, T new represents a target pseudo image, T represents a target image, S represents a source sample image, and i and j represent row and column values of a grid image.

[0088] As shown in the following formula (1), the target sample is subjected to position transformation, rotation and flip operation to generate a target image pseudo label, and the target image pseudo label is spliced with the target sample to generate a target pseudo image, thereby realizing sample expansion. Figure 4 In the above embodiment, preferably, a general semantic segmentation network encoder is constructed, a decoder with hierarchical semantic feature extraction and dynamic adaptive target recognition capability is designed, and a damaged building recognition model adapted to semantic scene migration is reconstructed, and the specific process includes:

[0089] In the original SAM model, the deep feature maps output by multiple Vision Transformer (ViT) encoders lack rich semantic information. In order to obtain more semantic features related to damaged buildings, an encoder based on a general semantic segmentation network is constructed to be able to extract features of a preset level of semantic features of damaged buildings, such as four levels of semantic features output in the 6th, 12th, 18th and 24th Vit blocks, to obtain hierarchical features.

[0090] A cross feature fusion module (CFFM) is used to extract deep features and cross fuse the hierarchical features to obtain hierarchical feature fusion information related to damaged buildings. The CFFM is composed of three feature cross fusion modules. In each module, the four input features are divided into four groups according to the arrangement rule C4 3 Each group is composed of three different features. Cross fusion of features of different depths by the feature cross fusion module can realize multi-level feature information fusion of damaged buildings and effectively improve the recognition performance of the network.

[0091] ​​The SFE is constructed by three groups of convolution modules and three groups of EPSA modules, the input image can be down-sampled eight times through three convolution processes, and the shallow information of the image can be extracted through the three groups of EPSA modules, so that the building detail semantic information lost in the ViT encoder due to down-sampling can be extracted, and the shallow semantic feature can be obtained.

[0092] The hierarchical feature and the shallow semantic feature are spliced in the feature dimension, and the spliced feature map can be restored to the size of the input image through two convolution layers, so as to realize the adaptation of the model to the identified target semantic scene.

[0093] In the above embodiment, preferably, the damaged building recognition model is trained by using the expanded building damage sample data set, and the building damage sample data set is updated by using the newly recognized damaged building data during the training process, and then the updated building damage sample data set is used for cyclic iterative training of the damaged building recognition model until the recognition accuracy converges to complete the model training, and the specific process includes:

[0094] The damaged building recognition model is trained by using the expanded building damage sample data set;

[0095] The sum of binary cross-entropy loss and Dice loss is used as the total loss function for training the damaged building recognition model to adapt to the class imbalance between damaged buildings and undamaged buildings, and to improve the stability of the model, and the specific formula is:

[0096]

[0097]

[0098]

[0099] wherein, represents the total loss, represents the cross-entropy loss, represents the Dice loss, y i represents the label of sample i, the positive class is 1 and the negative class is 0, N represents the number of pixels, p i represents the probability of sample i being predicted as a positive class;

[0100] A preset probability threshold is selected as the classification standard of the damaged building, the pixels with a damaged building probability value higher than the preset probability threshold in the damaged building recognition model recognition result are taken as new recognized damaged samples, and are supplemented to the building damage sample data set for data expansion, so as to realize the adaptive training of the model to the semantic scene transfer;

[0101] By repeatedly updating the building damage sample dataset and using the updated dataset to iteratively train the damaged building identification model until the total loss function converges and the model training process is completed, the optimal identification accuracy is gradually achieved.

[0102] like Figure 5 As shown, in the above embodiment, preferably, the specific process of performing spatial comprehensive analysis with the building vector data of the corresponding area includes:

[0103] An open-source building outline dataset is introduced as auxiliary data to spatially constrain the damaged areas identified by the damaged building recognition model. Individual building outline vectors that intersect with or contain the damaged areas are then identified as damaged buildings. The specific formula is as follows:

[0104]

[0105] In the formula, dmg i Represents the i-th building vector. i The damage identification result is represented by 1, 0, and Result, where Result represents the damage vector of the building area identified by the model, and ∩ represents the spatial intersection operation.

[0106] The proportion of the intersection area between the building's outline vector and the damaged area to the total area of ​​the building's outline vector is used to determine the degree of damage to the building. The specific formula is as follows:

[0107]

[0108] In the formula, dmg_class i The vector represents the degree of damage to the i-th building, S represents the spatial area represented by a single pixel in the raster result of the vector, m represents the number of pixels damaged in the i-th building, and n represents the number of pixels covered by the i-th building.

[0109] like Figure 6 As shown, this invention also proposes a post-earthquake damaged building identification system based on semantic scene transfer, applying the post-earthquake damaged building identification method based on semantic scene transfer disclosed in any of the above embodiments, including:

[0110] The sample data construction module 1 is used to annotate building damage samples in post-earthquake optical remote sensing images, perform data augmentation processing such as geometric transformation, color transformation, and overlap cropping on the sample data, construct a building damage sample dataset, and amplify the damaged buildings in the building damage sample dataset through a sample augmentation strategy based on scene reconstruction.

[0111] The recognition model construction module 2 is configured to construct a general semantic segmentation network encoder, design a model decoder with the ability of hierarchical semantic feature extraction and dynamic adaptation of recognition targets, and reconstruct a damaged building recognition model adapted to semantic scene migration;

[0112] The recognition model training module 3 is configured to train the damaged building recognition model by using the expanded building damage sample dataset, update the building damage sample dataset with newly recognized damaged buildings during the training process, and iteratively train the damaged building recognition model by using the updated building damage sample dataset until the recognition accuracy converges to complete the model training based on semantic scene migration.

[0113] The damaged building recognition module 4 is configured to input the damaged building region image into the trained damaged building recognition model for recognition, convert the pixel recognition result into a damaged region vector, perform spatial comprehensive analysis on the damaged region vector and the building vector data of the corresponding region, and generate a damaged building vector graph obtained by recognition.

[0114] In the embodiment, the scene migration of the damaged building target is realized by using the sample augmentation strategy based on scene reconstruction, which effectively improves the problem of lack of sample data in the damaged building remote sensing recognition task; a deep learning model decoder structure capable of fusing hierarchical semantic features is designed, the multi-level semantic features extracted by the general semantic segmentation model are effectively fused with the shallow features which are difficult to obtain, the semantic scene migration of the model is realized, and the problem that the general semantic segmentation model cannot dynamically adapt to the damaged building recognition task is solved.

[0115] In the above embodiment, preferably, the sample data construction module 1 is specifically configured to:

[0116] The size of the damaged building image in the building damage sample dataset is determined, and the damaged building image is cropped;

[0117] The horizontal number, vertical number and row-column coordinates of the cropped image are calculated to obtain a target sample, and an initial target semantic scene is generated;

[0118] The target sample is subjected to position transformation, rotation and flipping operations to generate a target image pseudo-label, and the target image pseudo-label and the target sample are spliced to generate a target pseudo-image, so as to realize sample augmentation.

[0119] In the above embodiment, preferably, the recognition model construction module 2 is specifically configured to:

[0120] The encoder based on the general semantic segmentation network is constructed to be capable of extracting features of a preset level of semantic features of the damaged building to obtain hierarchical features.

[0121] The cross feature fusion module is adopted to perform deep feature extraction and cross fusion on the hierarchical features, so as to obtain hierarchical feature fusion information related to the damaged building.

[0122] The SFE is constructed by adopting three groups of convolution modules and three groups of EPSA modules, eight times of down-sampling of the input image can be performed through three times of convolution processes, the shallow information of the image can be extracted through the three groups of EPSA modules, so that the building detail semantic information lost in the encoder due to down-sampling can be extracted, and the shallow semantic feature is obtained.

[0123] The hierarchical fusion feature and the shallow semantic feature are spliced in the feature dimension, the spliced feature map can be restored to the size of the input image through two convolution layers, and the adaptation of the model to the identified target semantic scene is realized.

[0124] In the above embodiment, preferably, the identification model training module 3 is specifically used for:

[0125] The damaged building identification model is trained by using the expanded building damage sample dataset;

[0126] The sum of the binary cross entropy loss and the Dice loss is used as the total loss function to train the damaged building identification model;

[0127] A preset probability threshold is selected as the classification standard of the damaged building, and the pixels with a damaged building probability value higher than the preset probability threshold in the damaged building identification model identification result are taken as new identified damaged samples and supplemented to the building damage sample dataset for data expansion;

[0128] The building damage sample dataset is repeatedly updated, and the updated building damage sample dataset is used to iteratively train the damaged building identification model until the total loss function converges to complete the model training process.

[0129] In the above embodiment, preferably, the damaged building identification module 4 is specifically used for:

[0130] The open source building contour dataset is introduced as auxiliary data to spatially constrain the building damage area identified by the damaged building identification model, and the single building contour vector having an intersection or containing spatial relationship with the building damage area is identified as a damaged building;

[0131] The proportion of the intersection area of the building contour vector and the building damage area to the area of the building contour vector is calculated to determine the damage degree information of the building.

[0132] According to the post-earthquake damaged building identification system based on semantic scene migration disclosed by the above embodiment, the functions to be realized by each module correspond to each step in the post-earthquake damaged building identification method based on semantic scene migration disclosed by the above embodiment respectively, and in the implementation process, the operation is performed with reference to the above embodiment, and here is not described again.

[0133] The above only is the preferred embodiment of the present application, and is not used to limit the present application, and for the person skilled in the art, the present application can have various changes and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1.A method for identifying post-earthquake damaged buildings based on semantic scene migration, characterized in that, The application relates to a building damage recognition method based on a scene reconstruction semantic segmentation network. The application comprises the following steps: A general semantic segmentation network encoder is constructed, a decoder with hierarchical semantic feature extraction and dynamic adaptive target recognition capability is designed, a damaged building recognition model suitable for semantic scene migration is reconstructed, and the specific process comprises the following steps: A general semantic segmentation network encoder is constructed to extract the preset hierarchical semantic features of the damaged building, and hierarchical features are obtained. Cross-feature fusion modules are used to extract and cross-fuse deep features of the hierarchical features, and hierarchical fusion feature information related to the damaged building is obtained. Three groups of convolution modules and three groups of efficient pyramid compression attention EPSA modules are used to construct a shallow feature extractor SFE, eight times of down-sampling of the input image can be realized through three times of convolution, and shallow information of the image can be extracted through the three groups of EPSA modules, so that the building detail semantic information lost in the encoder due to down-sampling can be extracted, and shallow semantic features are obtained. The hierarchical features and the shallow semantic features are spliced in the feature dimension, the spliced feature map can be restored to the size of the input image through two convolution layers, and the adaptation of the model to the recognized target semantic scene is realized. The damaged building recognition model is trained by using the expanded building damage sample dataset, the building damage sample dataset is updated by using the newly recognized damaged building data during the training process, and the damaged building recognition model is iteratively trained by using the updated building damage sample dataset until the recognition accuracy converges and the model training is completed. The damaged building region image is input into the trained damaged building recognition model for recognition, the pixel recognition result is converted into a damaged region vector, spatial comprehensive analysis is performed on the corresponding region building vector data, and a damaged building vector graph obtained by recognition is generated. 2.The method of claim 1, wherein, The specific process of sample expansion of the damaged building in the building damage sample dataset based on the scene reconstruction sample expansion strategy comprises the following steps: The size of the damaged building image in the building damage sample dataset is determined, and the damaged building image is cropped; The horizontal number, vertical number and row-column coordinates of the cropped image are calculated to obtain a target sample, and an initial target semantic scene is generated; The target sample is subjected to position transformation, rotation and flip operation, a target image pseudo-label is generated, the target image pseudo-label is spliced with the target sample to generate a target pseudo-image, and sample expansion is realized. 3.The method of claim 1, wherein, The building damage sample data set after the expansion is used to train the damaged building recognition model, and in the training process, the building damage sample data set is updated with newly recognized damaged building data, and the building damage sample data set after the update is used to iteratively train the damaged building recognition model until the recognition accuracy converges to complete model training, and the specific process includes: The building damage sample data set after the expansion is used to train the damaged building recognition model; The sum of binary cross-entropy loss and Dice loss is used as the total loss function to train the damaged building recognition model; A preset probability threshold is selected as the classification standard of the damaged building, and the pixels in the damaged building recognition model recognition result whose probability value is higher than the preset probability threshold are taken as new damaged samples and supplemented to the building damage sample data set for data expansion; The building damage sample data set is repeatedly updated, and the building damage sample data set after the update is used to iteratively train the damaged building recognition model until the total loss function converges to complete the model training process. 4.The method of claim 3, wherein, The specific process of the spatial comprehensive analysis of the building vector data of the corresponding region includes: The open source building contour data set is introduced as auxiliary data to spatially constrain the building damage area recognized by the damaged building recognition model, and the single building contour vector that has intersection or inclusion spatial relationship with the building damage area is recognized as a damaged building; The proportion of the intersection area of the building contour vector and the building damage area to the area of the building contour vector is calculated to determine the damage degree information of the building. 5.A post-earthquake damaged building recognition system based on semantic scene migration, characterized in that, The post-earthquake damaged building recognition method based on semantic scene migration according to any one of claims 1 to 4 comprises: A sample data construction module is configured to label building damage samples for post-earthquake optical remote sensing images, construct a building damage sample data set, and expand samples of damaged buildings in the building damage sample data set through a sample augmentation strategy based on scene reconstruction; An identification model construction module is configured to construct a general semantic segmentation network encoder, design a decoder with hierarchical semantic feature extraction and dynamic adaptive identification target capability, and reconstruct a damaged building recognition model with adaptive semantic scene migration; An identification model training module is configured to train the damaged building recognition model using the expanded building damage sample data set, update the building damage sample data set with newly recognized damaged building data during the training process, and iteratively train the damaged building recognition model using the updated building damage sample data set until the recognition accuracy converges to complete model training; A damaged building recognition module is configured to input a damaged building area image into the trained damaged building recognition model for recognition, convert the pixel recognition result into a damaged area vector, and perform spatial comprehensive analysis on the damaged area vector and the building vector data of the corresponding region to generate a recognized damaged building vector graph; The identification model construction module is specifically configured to: An encoder based on a general semantic segmentation network is constructed to be able to extract features of preset hierarchical semantic features of the damaged building, and obtain hierarchical features; A cross feature fusion module is used to extract deep features and cross fuse the hierarchical features, and obtain hierarchical fusion feature information related to the damaged building; Three groups of convolution modules and three groups of EPSA modules are used to construct an SFE, which can perform eight times of down-sampling on the input image through three times of convolution process, and extract shallow information of the image through the three groups of EPSA modules, so as to extract the semantic information of the building details lost in the encoder due to down-sampling, and obtain shallow semantic features; The hierarchical features and the shallow semantic features are spliced in the feature dimension, and the spliced feature map can be restored to the size of the input image through two convolution layers, realizing the adaptation of the model to the identified target semantic scene. 6.The post-earthquake damaged building identification system based on semantic scene migration of claim 5, wherein, The sample data construction module is specifically configured to: For the damaged building images in the building damage sample data set, the size of the damaged building images is determined, and the damaged building images are cropped; The horizontal number, vertical number and row and column coordinates of the cropped image are calculated to obtain a target sample, and an initial target semantic scene is generated; The target sample is subjected to position transformation, rotation and flip operation to generate a target image pseudo label, and the target image pseudo label and the target sample are spliced to generate a target pseudo image, realizing sample expansion. 7.The post-earthquake damaged building identification system based on semantic scene migration of claim 5, wherein, The identification model training module is specifically configured to: The expanded building damage sample data set is used to train the damaged building identification model; The sum of binary cross entropy loss and Dice loss is used as a total loss function to train the damaged building identification model; A preset probability threshold is selected as a classification standard of the damaged building, and pixels with a damaged building probability value higher than the preset probability threshold in the damaged building identification model recognition result are regarded as new identified damaged samples and supplemented to the building damage sample data set for data expansion; The building damage sample data set is repeatedly updated, and the updated building damage sample data set is used to iteratively train the damaged building identification model until the total loss function converges to complete the model training process. 8.The post-earthquake damaged building identification system based on semantic scene migration of claim 7, wherein, The damaged building identification module is specifically configured to: An open-source building contour data set is introduced as auxiliary data to spatially constrain the building damage area identified by the damaged building identification model, and monomer building contour vectors having intersection or containing spatial relationship with the building damage area are identified as damaged buildings; The proportion of the intersection area of the building contour vector and the building damage area to the area of the building contour vector is calculated to determine the damage degree information of the building.

Images