Building damage detection method and device based on coupling SAR image and optical image multi-features

Abstract

The present invention provides a building damage detection method and device that couples multiple features of SAR and optical images, including: obtaining SAR images, optical images, and drone images corresponding to a study area in pre-disaster and post-disaster scenarios, respectively; constructing a change detection image set for the study area based on the SAR and optical images, and constructing a vector dataset corresponding to the study area based on the optical and drone images; extracting multi-scale features corresponding to building vector units from the change detection image set based on the vector dataset to construct a multi-scale feature library; and using the multi-scale feature library to train a damaged building classification network, which is then used to perform damage detection in the study area. This invention can effectively address issues in existing technologies, such as a single data source, limited feature extraction, weak model generalization, and insufficient real-time and robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a building damage detection method and device that couples multiple features of SAR images and optical images. Background Art

[0002] Earthquakes, one of the most destructive natural disasters, can trigger chain reactions of building collapses, landslides, and secondary disasters, resulting in significant casualties and property losses. Building damage is the core factor leading to casualties, and rapid and accurate damage assessment plays a decisive role in post-disaster emergency rescue and reconstruction. Traditional methods that rely on manual field surveys require experts to verify the internal and external features of buildings on site, which has significant drawbacks such as low efficiency, high risk, and limited coverage. Remote sensing technology, with its advantages of non-contact, wide-area coverage, and high spatiotemporal resolution, has become a key means of assessing building damage after disasters. Existing technologies mainly focus on the coordinated application of optical and synthetic aperture radar (SAR) sensors: optical imaging is easily restricted by cloud cover, while SAR, with its all-weather observation capabilities, shows greater applicability in damage detection. SAR-based assessment methods can be divided into two categories: (1) change detection methods: through pre-disaster / post-disaster data comparative analysis, using block unit (intensity, coherence and polarization feature analysis) or building unit (single building feature extraction) strategies; (2) single-phase assessment methods: using the polarization and texture features of post-disaster SAR images, combined with machine learning or deep learning classifiers to achieve damage classification. In addition, the method of fusing optical images with geographic information system (GIS) data is used to improve detection accuracy, but such methods have problems such as a single data source, strong feature extraction limitations, weak model generalization ability, and insufficient real-time and robustness. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a building damage detection method and device that couples multiple features of SAR images and optical images, which can effectively improve the problems of the existing technology such as single data source, strong feature extraction limitations, weak model generalization ability, insufficient real-time performance and robustness, and provide a high-precision, multi-scale solution for post-disaster building damage assessment.

[0004] In a first aspect, the present invention provides a building damage detection method that couples multiple features of SAR images and optical images, comprising:

[0005] Obtain SAR images, optical images, and UAV images of the study area before and after the disaster;

[0006] Construct a change detection image set for the study area based on SAR images and optical images, and construct a vector dataset corresponding to the study area based on optical images and UAV images. The vector dataset contains a vector unit for each building in the study area.

[0007] Based on the vector dataset, multi-scale features corresponding to building vector units are extracted from the change detection image set to construct a multi-scale feature library. The multi-scale features include one or more of H / A / Alpha polarization decomposition features, backscatter coefficient correlation, backscatter coefficient difference, InSAR coherence and normalized coherence index, NDBI index, and InSAR three-dimensional displacement.

[0008] A damaged building classification network is trained using a multi-scale feature library. The trained damaged building classification network is used to detect damage in the study area. The damaged building classification network includes a variational autoencoder and an improved multi-layer perceptron classifier. The improved multi-layer perceptron classifier is used to alternately perform local convolution and global attention fusion on the token dimension of the input latent space vector, and perform feedforward enhancement and attention selection on the channel dimension.

[0009] In one embodiment, the building vector units are divided into damaged building vector units and intact building vector units; and a damaged building classification network is trained using a multi-scale feature library, including:

[0010] From the multi-scale features corresponding to the building vector units, target multi-scale features whose correlation with the damaged building vector units is higher than a preset threshold are selected;

[0011] The target multi-scale features corresponding to the building vector units are input into the damaged building classification network to obtain the binary building damage classification type output by the damaged building classification network. The joint loss value is determined by combining the labels carried by the building vector units. The joint loss value includes the feature decoupling loss value and the classification accuracy loss value.

[0012] The joint loss value is used to adjust the network parameters of the damaged building classification network.

[0013] In one embodiment, the improved multi-layer perceptron classifier includes an input linear mapping layer, a multi-layer TokenMixer network, a multi-layer channel Mixer network, a pooling layer, and an output layer. The target multi-scale features corresponding to the building vector units are input into the damaged building classification network, and the damaged building classification network outputs a binary classification of building damage, including:

[0014] Through the variational autoencoder, the latent space vector of the target multi-scale features corresponding to the building vector unit is extracted;

[0015] By inputting the linear mapping layer, the latent space vector is mapped to the specified token dimension;

[0016] Through a multi-layer Token Mixer network, local convolution and global attention fusion are alternately performed on the latent space vector in the token dimension;

[0017] Through a multi-layer channel mixer network, feedforward enhancement and attention selection are performed on the feature vector output by the token mixer network in the channel dimension;

[0018] Through the pooling and output layers, the binary classification of building damage is generated based on the feature vector output by the channel Mixer network.

[0019] In one embodiment, each Token Mixer network includes a depthwise separable convolution layer, a pointwise convolution layer, an improved Transformer encoder, and a layer normalization layer. Through the multi-layer Token Mixer network, local convolution and global attention fusion are alternately performed on the latent space vector in the token dimension, including:

[0020] For any layer of the Token Mixer network, the following operations are performed on the input features using the Token Mixer network:

[0021] Through the depth-wise separable convolution layer and the point-wise convolution layer, local convolution is performed on the input features in the token dimension to extract token-level local features;

[0022] Through the improved Transformer encoder, global attention fusion is performed on the input features in the token dimension to extract token-level global features;

[0023] Through the layer normalization layer, the token-level local features and token-level global features are fused.

[0024] In one embodiment, the improved Transformer encoder includes a multi-head attention layer, a layer normalization layer, a fully connected layer, an activation function layer, another fully connected layer, and another normalization layer;

[0025] The improved Transformer encoder adopts two forms of residual connection structures; among them, the first form of residual connection structure is as follows: a residual connection is made between the input end of the multi-head attention layer and the output end of the multi-head attention layer; the second form of residual connection structure is as follows: a residual connection is made between the input end of the multi-head attention layer and the output end of another fully connected layer.

[0026] In one embodiment, each channel Mixer network includes a layer normalization layer, a feedforward network, and an ECA channel attention layer, and a residual connection is formed between the input end of the layer normalization layer and the output end of the ECA channel attention layer.

[0027] In one embodiment, determining a joint loss value in combination with labels carried by building vector units includes:

[0028] Determining a feature reconstruction loss value based on the target multi-scale features corresponding to the building vector units and the reconstructed features generated by the damaged building classification network during the damage detection process; and determining a latent KL divergence loss value based on the feature mean and feature variance of each latent dimension in the latent space vector output by the variational autoencoder; and determining a latent regularization loss value based on the feature variance of each latent dimension in the latent space vector output by the variational autoencoder and the feature covariance between any two latent dimensions;

[0029] The feature reconstruction loss value, potential KL divergence loss value, and potential regularization loss value are weightedly fused to obtain the feature decoupling loss value;

[0030] Determine the classification accuracy loss value based on the building damage binary classification type corresponding to the building vector unit and the label carried by the building vector unit;

[0031] The feature decoupling loss value and the classification accuracy loss value are weightedly fused to obtain the joint loss value.

[0032] In a second aspect, the present invention further provides a building damage detection device that couples multiple features of SAR images and optical images, comprising:

[0033] Image acquisition module, used to obtain SAR images, optical images and UAV images of the study area in pre-disaster and post-disaster scenarios respectively;

[0034] An image processing module is used to construct a change detection image set for the study area based on SAR images and optical images, and to construct a vector dataset corresponding to the study area based on optical images and UAV images. The vector dataset contains a vector unit for each building in the study area.

[0035] A feature library construction module is used to extract multi-scale features corresponding to building vector units from the change detection image set based on the vector data set to construct a multi-scale feature library. The multi-scale features include one or more of H / A / Alpha polarization decomposition features, backscatter coefficient correlation, backscatter coefficient difference, InSAR coherence and normalized coherence index, NDBI index, and InSAR three-dimensional displacement;

[0036] The network training module is used to train the damaged building classification network using a multi-scale feature library. The trained damaged building classification network is used to detect damage in the study area. The damaged building classification network includes a variational autoencoder and an improved multi-layer perceptron classifier. The improved multi-layer perceptron classifier is used to alternately perform local convolution and global attention fusion on the token dimension of the input latent space vector, and perform feedforward enhancement and attention selection on the channel dimension.

[0037] In a third aspect, the present invention further provides an electronic device comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement any one of the methods provided in the first aspect.

[0038] In a fourth aspect, the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement any one of the methods provided in the first aspect.

[0039] The present invention provides a building damage detection method and device that couples SAR images and optical images with multiple features. First, SAR images, optical images, and drone images corresponding to the study area in pre-disaster and post-disaster scenarios are obtained. Then, a change detection image set of the study area is constructed based on the SAR images and optical images, and a vector data set corresponding to the study area is constructed based on the optical images and drone images. The vector data set contains each building vector unit in the study area. Then, based on the vector data set, multi-scale features corresponding to the building vector units are extracted from the change detection image set to construct a multi-scale feature library. The multi-scale features include H / A / Alp One or more of ha polarization decomposition features, backscatter coefficient correlation, backscatter coefficient difference, InSAR coherence and normalized coherence index, NDBI index, and InSAR three-dimensional displacement; finally, the damaged building classification network is trained using a multi-scale feature library. The trained damaged building classification network is used to detect damage in the study area. The damaged building classification network includes a variational autoencoder and an improved multi-layer perceptron classifier. The improved multi-layer perceptron classifier is used to alternately perform local convolution and global attention fusion on the token dimension for the input latent space vector, and perform feedforward enhancement and attention selection on the channel dimension. The above method collects SAR images, optical images and drone images corresponding to the study area in pre-disaster and post-disaster scenarios, and on this basis constructs a multi-scale feature library containing multi-scale features corresponding to each building vector unit to improve the problems of single data source and strong feature extraction limitations in the existing technology. In addition, the multi-scale feature library is used to train a damaged building classification network, which contains a variational autoencoder and an improved multi-layer perceptron classifier. The trained damaged building classification network is used to detect damage in the study area. Through this network, the problems of weak model generalization ability, lack of real-time performance and robustness in the existing technology can be effectively improved, thereby providing a high-precision, multi-scale solution for post-disaster building damage assessment.

[0040] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or understood by practicing the present invention. The purposes and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0041] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 A schematic flow chart of a building damage detection method that couples multiple features of SAR images and optical images, provided in an embodiment of the present invention;

[0044] Figure 2 A technical framework diagram of a building damage detection method that couples multiple features of SAR images and optical images, provided by an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of the structure of a damaged building detection network provided by an embodiment of the present invention;

[0046] Figure 4 A schematic structural diagram of a DIPVAE encoder provided in an embodiment of the present invention;

[0047] Figure 5 A schematic diagram of the structure of a Token Mixer network provided by an embodiment of the present invention;

[0048] Figure 6 A schematic diagram of the structure of a channel mixer network provided by an embodiment of the present invention;

[0049] Figure 7 A schematic diagram of the structure of an ECA channel attention layer provided by an embodiment of the present invention;

[0050] Figure 8 A schematic structural diagram of a building damage detection device that couples multiple features of SAR images and optical images, provided by an embodiment of the present invention;

[0051] Figure 9 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0053] Currently, several existing technologies address building damage detection methods. For example, the patent application "Building Damage Detection Method Using Sequential InSAR Coherence Images Based on a Generative Model" (Publication No. CN119516384A) proposes a generative model-based InSAR surface deformation anomaly detection method. This method relies on a single InSAR data source and does not incorporate optical imagery or other features, resulting in limited ability to discriminate complex damage types (such as localized collapses). The generative model also has high computational complexity, making it difficult to meet the needs of rapid post-disaster assessments. Another patent application, "A Post-Earthquake Building Damage Detection Method Based on Drone Video" (Publication No. CN111126184A), proposes a post-earthquake building damage detection method based on drone video. This method utilizes drone video streams for real-time detection, identifying damaged areas through dynamic target tracking and image segmentation. However, this method primarily relies on drone video data, which can be limited by weather and lighting conditions and lacks SAR deformation feature fusion. Furthermore, the patent "Method and Apparatus for Building Damage Detection Based on Image Superpixel Fusion" (Authorization Announcement No.: CN109360190B) proposes a building damage detection method based on image superpixel fusion. However, this method does not integrate InSAR deformation and coherence features, resulting in a single feature library dimension and weak generalization of the classification model. Another patent, "Method for Building Damage Detection in Aerial Images Based on Shadow and Texture Features" (Authorization Announcement No.: CN105631892B), proposes a building damage detection method based on shadow and texture features in aerial images. However, this method primarily relies on shadow and texture features, which are susceptible to interference from the shooting angle, resulting in a high false detection rate, and does not quantify SAR physical characteristics (such as the backscatter coefficient).

[0054] Furthermore, Synthetic Aperture Radar (InSAR) can accurately measure millimeter-level surface deformation by interferometry of multi-temporal SAR images and has been widely used in earthquake disaster monitoring. Recent studies have shown that InSAR deformation information can indirectly reflect building damage characteristics (Du Y et al. 2024; Wang C et al. 2024), but its single deformation parameter is not robust enough to handle complex damage patterns.

[0055] In summary, the above studies still have the following problems: (1) Single data source: Most methods rely only on single optical or SAR data, and do not integrate multi-source remote sensing features (such as InSAR deformation, coherence, NDBI index, etc.), resulting in incomplete feature information. (2) Feature extraction limitations: Traditional methods rely on artificially designed features (such as shadows and textures), and have poor adaptability to complex damage patterns (such as multi-scale deformation), resulting in low accuracy of building damage detection. (3) Weak model generalization ability: The classifier design is simple (such as threshold segmentation, traditional machine learning), and does not combine the feature self-extraction advantages of deep learning, resulting in limited accuracy. (4) Insufficient real-time and robustness: Some methods rely on high-resolution image processing, have low computational efficiency, and are greatly affected by environmental interference (such as cloud cover and light changes).

[0056] Based on this, the present invention provides a building damage detection method and device that couples multiple features of SAR images and optical images. By integrating multi-source remote sensing data (including SAR images, optical images and drone images) and combining variational autoencoders and multi-layer perceptron neural networks for building damage detection, it can effectively improve the problems of single data source, strong feature extraction limitations, weak model generalization ability, insufficient real-time and robustness in existing technologies, and provide a high-precision, multi-scale solution for post-disaster building damage assessment, which has significant academic value and engineering application potential.

[0057] To facilitate understanding of this embodiment, a building damage detection method that couples SAR images and optical images with multiple features disclosed in an embodiment of the present invention is first described in detail. Figure 1 The flowchart of a building damage detection method coupling SAR images and optical image multi-features is shown. The method mainly includes the following steps S102 to S108:

[0058] Step S102: Obtain SAR images, optical images, and UAV images corresponding to the study area in pre-disaster and post-disaster scenarios, respectively.

[0059] In one example, taking an earthquake as an example, SAR images, optical images, and UAV images corresponding to the study area in pre-earthquake and post-earthquake scenarios can be obtained.

[0060] Step S104: constructing a change detection image set for the study area based on the SAR image and the optical image, and constructing a vector data set corresponding to the study area based on the optical image and the UAV image.

[0061] Among them, the change detection image set includes one or more of the following: backscatter coefficient difference map, NDBI index difference map, entropy difference map, scattering angle difference map, anisotropy difference map, coherence map and interferometry pair; the vector data set contains each building vector unit in the study area. The building vector units are divided into damaged building vector units and intact building vector units. The building vector units are used to describe the spatial range of the building.

[0062] In one example, an enhanced difference map algorithm was used to generate difference maps for optical images and SAR images of pre-earthquake and post-earthquake scenes, and coherence maps and interference pairs were generated based on SAR images of pre-earthquake and post-earthquake scenes for subsequent multi-scale feature analysis of damaged buildings. At the same time, high-resolution drone images and optical images were used, and visual interpretation was used to construct vector datasets of damaged building areas and intact building areas.

[0063] Step S106 : extracting multi-scale features corresponding to building vector units from the change detection image set according to the vector data set to construct a multi-scale feature library.

[0064] Multi-scale features include one or more of H / A / Alpha polarization decomposition features, backscatter coefficient correlation, backscatter coefficient difference, InSAR coherence and normalized coherence index, NDBI (Naturalistic Developmental Behavioral Interventions) index, and InSAR three-dimensional displacement. In one example, for any image in a change detection image set, a vector dataset can be used to extract sub-images corresponding to building vector units from the image. Features corresponding to the building vector units can then be extracted from the sub-images. Based on this, multi-scale features corresponding to each building vector unit can be extracted, thereby constructing a multi-scale feature library.

[0065] Step S108: Using the multi-scale feature library to train a damaged building classification network, the trained damaged building classification network is used to perform damage detection in the study area.

[0066] The damaged building classification network includes a variational autoencoder and an improved multi-layer perceptron classifier. The variational autoencoder is used to extract latent space vectors, while the improved multi-layer perceptron classifier alternately performs local convolution and global attention fusion on the token dimension of the input latent space vector, and performs feedforward enhancement and attention selection on the channel dimension. In one example, the correlation between each feature in the multi-scale feature library and intact and damaged buildings is first analyzed, and features with high correlation with damaged buildings are selected to obtain the target multi-scale features corresponding to each building vector unit. The target multi-scale features corresponding to the building vector units are then input into the damaged building classification network to obtain the binary damage type of the building output by the damaged building classification network. The joint loss value is determined by combining the label carried by the building vector unit. The binary damage type of the building is used to describe whether the building unit is damaged. The joint loss value includes feature decoupling loss and classification accuracy loss. The joint loss value is used to adjust the network parameters of the damaged building classification network to achieve training of the damaged building classification network.

[0067] The embodiment of the present invention provides a building damage detection method that couples SAR images and optical image multi-features. SAR images, optical images, and drone images corresponding to the study area in pre-disaster and post-disaster scenarios are collected, and on this basis, a multi-scale feature library containing multi-scale features corresponding to each building vector unit is constructed to improve the problems of a single data source and strong feature extraction limitations in the existing technology. In addition, the multi-scale feature library is used to train a damaged building classification network, which includes a variational autoencoder and an improved multi-layer perceptron classifier. The trained damaged building classification network is used to detect damage in the study area. This network can effectively improve the problems of weak model generalization ability, insufficient real-time performance, and insufficient robustness in the existing technology, thereby providing a high-precision, multi-scale solution for post-disaster building damage assessment.

[0068] For ease of understanding, the present invention provides a specific implementation of a building damage detection method that couples SAR images and optical images with multiple features. Figure 2The technical framework diagram of a building damage detection method that couples SAR images and optical images with multiple features is shown. The core idea of ​​this method is: S1, obtaining multi-source remote sensing data, including SAR images, optical images and UAV images, and preprocessing them; S2, making a difference map of multi-source remote sensing images, and constructing vector data of damaged and intact building areas; S3, constructing a multi-scale feature library of building vector units; S4, analyzing and selecting the correlation between damaged buildings and the multi-scale feature library, and dividing the training, verification and test data sets; S5, constructing a damaged building detection network based on variational autoencoders and multi-layer perceptrons, and performing network training and inference to output the detection results of damaged buildings in the earthquake-stricken area, that is, the result includes the two-classification type of building damage corresponding to each building vector unit in the study area.

[0069] The embodiment of the present invention further explains the above S1 to S5.

[0070] S1, acquires multi-source remote sensing data, including SAR images, optical images, and UAV images, and preprocesses them.

[0071] High-resolution SAR, optical, and UAV imagery of the earthquake-stricken area were acquired and preprocessed. Optical images underwent radiometric calibration, atmospheric correction, and orthorectification. UAV imagery underwent regional adjustment, aerial triangulation, 3D TIN (Triangulated Irregular Network) construction, and 3D reconstruction and visualization. SAR images in GRD (Grid Data Format) underwent orbit correction, thermal noise removal, radiometric calibration, multi-look, speckle filtering, range-Doppler terrain correction, and registration. SAR images in SLC format underwent data import, precise orbit file updates, and registration.

[0072] Specifically, data acquisition: According to the latitude and longitude geographical location of the study area, high-resolution SAR images, optical images, drone images and digital elevation model (DEM) data before and after the earthquake are obtained. Taking the Jishishan earthquake study area in Gansu Province on December 18, 2023 as an example, the collected Sentinel-1A, Sentinel-2 and Jilin-1 satellite data information is shown in Table 1.

[0073] Table 1: Detailed information of the multi-source remote sensing dataset for the Jishishan earthquake in Gansu Province

[0074]

[0075] On this basis, the embodiment of the present invention further provides a specific process of data preprocessing.

[0076] In one example, the Sentinel-1A preprocessing pipeline is as follows:

[0077] For Sentinel-1 data in GRD format, the Sentinel-1 GRD format data before and after the earthquake in the study area were used to perform orbit correction, thermal noise removal, radiation calibration, multi-viewing, coherent speckle filtering, range Doppler terrain correction, and alignment using SNAP software to obtain a backscatter coefficient map. Then, a difference map of the backscatter coefficients of the two scenes acquired before the earthquake on December 14, 2023 and after the earthquake on December 26, 2023 was made for the subsequent extraction of backscatter coefficient difference and correlation coefficient features of building damage in the Jishishan earthquake.

[0078] For IW-mode Sentinel-1 data, three pre- and post-earthquake views covering the study area were used using SNAP software for data import, precise orbit file update, TOPS (Terrain Observation by Progressive Scans) data segmentation, deskewing, DEM registration, deskewing, burst stitching, and sub-swath mosaicking. Notably, the Enhanced Spectral Divergence (ESD) registration method was used for precise azimuthal offset correction. The deskewing, interpolation, and deskewing steps were all performed at the burst level. The resulting registered SLC-format SAR images were used for subsequent coherence, normalized coherence, and InSAR displacement generation.

[0079] In one example, the Sentinel-2 preprocessing pipeline is as follows:

[0080] Sentinel-2 images were subjected to radiometric calibration, atmospheric correction, and orthorectification using ENVI software, and the processed images were used for subsequent NDBI index extraction.

[0081] For example, the preprocessing process for Jilin-1 optical satellite imagery is as follows: First, the multispectral imagery undergoes radiometric calibration, atmospheric correction, and orthorectification (using a 12.5-meter DEM). The panchromatic imagery is then radiometrically calibrated and orthorectified using the same methods. The orthorectified panchromatic and multispectral bands are then fused using the Gram-Schmidt Panchromatic Sharpening (GS) image fusion algorithm. The resulting optical satellite imagery has a spatial resolution of 0.75 meters.

[0082] In one example, the preprocessing process for drone imagery is as follows: ContextCapture (CC) software was used. First, the raw drone image data, including the original image, camera files, and POS (Position and Orientation System) files, was imported. Regional adjustment and aerial triangulation were performed to generate a dense point cloud. Based on this, a 3D TIN grid was constructed through dense matching of multi-view images, generating a preliminary 3D model. Subsequently, automatic texture mapping technology was used to refine and texture map the 3D model, ultimately generating a high-precision 3D scene, enabling 3D reconstruction and visualization of the drone imagery in the Jishishan area.

[0083] S2: Production of multi-source remote sensing image difference maps and construction of vector data of damaged and intact building areas. Specifically, an enhanced difference map algorithm is used to generate change detection difference maps from pre- and post-earthquake optical and SAR images for subsequent multi-scale feature analysis of damaged buildings. High-resolution drone and optical images are then visually interpreted to construct vector datasets of damaged and intact building areas. This includes:

[0084] (2.1) After the above preprocessing, differential maps of Sentinel-2 and Sentinel-1 GRD format data are generated, including backscatter coefficient differential maps, NDBI index differential maps, entropy (H) differential maps, scattering angle (α) differential maps, and anisotropy (A) differential maps.

[0085] In one example, based on preprocessed Sentinel-2 and Sentinel-1 GRD format data, the backscatter coefficient map, NDBI index map, and H / A / Alpha polarization decomposition feature maps (including entropy (H) map, scattering angle (α) map, and anisotropy (A) map) before and after the earthquake were determined. The value of each pixel in the backscatter coefficient map is the backscatter coefficient, the value of each pixel in the NDBI index map is the NDBI index, the value of each pixel in the entropy (H) map is the entropy, the value of each pixel in the scattering angle (α) map is the scattering angle, and the value of each pixel in the anisotropy (A) map is the anisotropy. Then, the difference between the pre- and post-earthquake images is taken to obtain the corresponding difference map.

[0086] The embodiment of the present invention further provides a specific process for generating an NDBI index map, an entropy (H) map, a scattering angle (α) map, and an anisotropy (A) map:

[0087] (2.11) NDBI index: The NDBI index is extracted using the spectral reflectance of the near-infrared (NIR) and short-wave infrared (SWIR) bands of the optical image. The NDBI calculation formula is as follows:

[0088] ;

[0089] Among them, NIR is the near-infrared band, B8 (center wavelength is about 842 nm) and SWIR is the short-wave infrared band: B11 (center wavelength is about 1610 nm) or B12 (2190 nm).

[0090] (2.12) H / A / Alpha polarization decomposition characteristic diagram:

[0091] Taking Sentinel-1 GRD format data as an example, the H / A / Alpha decomposition method is used to extract the scattering mechanism of buildings after earthquakes using the registered VH and VV dual-polarization GRD Sentinel-1 data. The three parameters of entropy (H), scattering angle (α), and anisotropy (A) can be calculated based on the eigenvalues ​​and eigenvectors of the polarization coherence matrix. The H / A / Alpha parameters are obtained using the SNAP software. First, the scattering matrix is ​​defined as:

[0092] ;

[0093] Where, 、 represent the complex scattering coefficients of different polarization channels, where is the scattering coefficient of vertical transmission and horizontal reception, is the scattering coefficient of vertical emission and vertical reception.

[0094] The corresponding target scattering vector k can be expressed as:

[0095] ;

[0096] Therefore, the dual-polarization SAR data can be represented by the following polarization covariance matrix:

[0097] ;

[0098] in Denotes conjugate transpose, ∗ is the conjugate operator. The covariance matrix C is a positive semidefinite matrix, which can be decomposed into the weighted sum of two mutually orthogonal covariance matrices, expressed as:

[0099] ;

[0100] Among them, real numbers represents the eigenvalue of the polarization covariance matrix C; represents the eigenvector; z is the rank of the matrix.

[0101] Entropy (H) describes the disorder of various scattering types (i.e., the randomness of the scattering). Entropy (H) ranges from 0 to 1, indicating the randomness of the scattering. The formula is as follows:

[0102] ;

[0103] ;

[0104] This can be called the eigenvalue weight or the scattering mechanism probability.

[0105] Anisotropy (A), as a complementary parameter to entropy, can provide information about surface roughness, as shown in the following formula:

[0106] ;

[0107] A low anisotropy (A) value indicates that the dominant scattering mechanism is single scattering, while a medium or high anisotropy (A) value indicates that the dominant scattering mechanism is double scattering.

[0108] The scattering angle (α) is directly related to the average physical scattering mechanism. Its value is between 0 and 90 degrees.

[0109] ;

[0110] The above process can be used to calculate the three parameters of entropy (E), scattering angle (α), and anisotropy (A) used to describe the SAR images before and after the earthquake, which are used to analyze the damage detection of buildings after the earthquake.

[0111] (2.2) The above difference image is enhanced using an enhanced difference image algorithm to improve the contrast of the changed area in the difference image.

[0112] Change detection technology has become an important tool for detecting building damage after earthquakes. The quality of difference maps directly affects the accuracy of detecting and segmenting building change areas. This embodiment of the present invention provides a specific process for processing any generated difference map using an enhanced difference map algorithm, including the following steps (2.21) to (2.24):

[0113] (2.21) In view of the advantages of neighborhood difference difference maps and neighborhood logarithmic ratio difference maps, an embodiment of the present invention proposes a difference map enhancement method based on Chebyshev distance and grayscale similarity weighting, aiming to improve the contrast of the changed areas in the difference image while reducing the pseudo-changes caused by speckle noise.

[0114] Use one The neighborhood window calculates the difference value for each pixel in the difference map and the surrounding 8 neighboring pixels Chebyshev distance Grayscale similarity , its mathematical expression is shown in the formula:

[0115] ;

[0116] ;

[0117] in, Represents the row and column coordinates of a single pixel a, Represents the pixel relative to the center The row and column offsets of , Represents the value of the pixel in the difference difference image. The smaller the similarity metric value, the closer the grayscale value of the adjacent pixels is to the center pixel, and the more likely they are to belong to similar change or background feature areas.

[0118] (2.22) Then apply Gaussian function weighting to Chebyshev distance and grayscale similarity respectively. Set Gaussian parameters , its mathematical expression is:

[0119] ;

[0120] ;

[0121] is the index of the pixel in the 3*3 neighborhood, used to distinguish 8 neighborhood pixels. is the standard deviation of the Gaussian distribution, which is used to control the weighted spatial and grayscale similarity sensitivity. is the Gaussian weighting factor for distance. is a Gaussian weighting factor for similarity. This weighting assigns higher weights to pixels that are close and have similar change characteristics, while lower weights are assigned to pixels that are far from the center and have significantly different change characteristics. This allows the enhanced results to better highlight the change signals in similar regions of local space.

[0122] (2.23) Multiply the distance weight and the similarity weight and sum over the neighborhood:

[0123] ;

[0124] is the combined weight, which is the total weight integrating the contributions of all neighborhood pixels under the distance and similarity weights.

[0125] (2.24) The position ( ) corresponds to the initial difference value of the pixel at Multiply by the portfolio weight To enhance the change information of the pixel, and then normalize it to ensure that the value of the final result image is within a unified range, which is conducive to subsequent visualization and analysis. The calculation formula is as follows:

[0126] ;

[0127] is the position in the difference map ( ) and the enhanced differential value corresponding to the pixel at . The above calculation process can complete the production of the multi-source remote sensing data differential map.

[0128] (2.3) Using the processed 3D-reconstructed drone imagery and Jilin-1 satellite imagery, a multi-scale feature dataset was constructed for each building unit. ArcGIS software was used to label damaged and intact buildings and create a vector map of sample buildings. Ultimately, this embodiment of the present invention extracted 3,774 houses, including 1,637 damaged and 2,137 intact.

[0129] S3, construct a multi-scale feature library of building vector units, including multi-scale image data such as H / A / Alpha polarization decomposition characteristics, backscatter coefficient correlation, backscatter coefficient difference, InSAR coherence and normalized coherence index, NDBI index and InSAR three-dimensional displacement, and establish a multi-scale feature library of building vector units.

[0130] Specifically, ArcGIS software was used to extract multi-scale features within these building vector units and establish a multi-scale feature library of building vector units. This process included data import, raster data conversion, raster data clipping, feature extraction, feature library establishment, data storage, and management.

[0131] (3.1) Data import: Import the above remote sensing data into the ArcGIS environment to ensure the spatial reference consistency of the data for subsequent analysis.

[0132] (3.2) Raster data conversion: Convert various types of remote sensing data into raster format to facilitate spatial analysis.

[0133] (3.3) Raster data clipping: Use the boundaries of building vector units to clip the raster data and extract relevant feature data within each building unit.

[0134] (3.4) Feature extraction: Within each building vector unit, the following multi-scale features are calculated:

[0135] (3.41) H / A / Alpha Polarization Decomposition Characteristics: Using the polarization decomposition method, the H / A / Alpha polarization characteristic parameters of each building unit are extracted. In practice, the three parameters (entropy (H), scattering angle (α), and anisotropy (A)) can be directly extracted from the entropy (H) difference map, scattering angle (α) difference map, and anisotropy (A) difference map generated in S2.

[0136] (3.42) Backscatter coefficient correlation: Calculate the correlation between the backscatter coefficients of different polarization channels within each building unit to evaluate the consistency of their scattering characteristics. Specifically, the backscatter coefficient correlation calculation method is as follows: Create a difference map of the backscatter coefficients for the two scenes acquired on December 14, 2023, before the earthquake, and on December 26, 2023, after the earthquake. The backscatter coefficient correlation calculation formula is as follows:

[0137] ;

[0138] Where r is the backscatter coefficient correlation, and N represents the total number of pixels in the target window. , Represent the value of the i-th pixel before and after the earthquake, respectively. , Represents the average value of the 13 × 13 neighborhood pixels around pixel i.

[0139] (3.43) Backscatter coefficient difference: Calculate the difference in backscatter coefficients for different polarization channels and analyze the differences in their scattering characteristics. Specifically, create a backscatter coefficient difference map for two scenes acquired before the earthquake (December 14, 2023) and after the earthquake (December 26, 2023). The backscatter coefficient difference calculation formula is as follows:

[0140] ;

[0141] Where, represents the backscatter coefficient difference, , Represents the average value of the 13 × 13 neighborhood pixels around pixel i in the difference image.

[0142] (3.45) InSAR coherence and normalized coherence index: Extract the InSAR coherence value of each building unit and calculate the normalized coherence index to evaluate its coherence level. Specifically, the InSAR coherence and normalized coherence index calculation method is as follows:

[0143] The coherence calculation is performed using the SLC files with the acquisition dates of 20231214 and 20231226 of the registered images, as shown in the following formula:

[0144] ;

[0145] is the computational coherence, and are complex values ​​corresponding to the pixels of the SAR images before and after the earthquake, express The complex conjugate of , where E represents the expected value. The coherence ranges from 0 to 1, with lower values ​​indicating greater ground changes (severe damage in building damage assessment).

[0146] In this embodiment, an improved method can also be used to calculate the coherence difference to distinguish damaged and undamaged buildings. The normalized difference between two coherent image pairs (i.e., the pre-earthquake coherence map and the post-earthquake coherence map) is used as a parameter for distinguishing building damage in this earthquake (the normalized coherence index is the normalized difference between two coherence image pairs (i.e., the pre-earthquake coherence map and the post-earthquake coherence map) as a parameter for distinguishing building damage in this earthquake). See the following formula for details:

[0147] ;

[0148] Where ∆γ is the normalized coherence index, and are the coherence of the interferometer pair before the earthquake, and the coherence values ​​of the interferometer pair before and after the earthquake, respectively), for example, and They are the interference pair coherence before the event, i.e., 20231202 and 20231214, and the interference pair coherence after the event, i.e., 20231214 and 20231226.

[0149] (3.46) NDBI Index: Calculate the Normalized Difference Building Index (NDBI) for each building unit to assess its building density. In practice, the NDBI index can be directly extracted from the NDBI index difference map generated in S2.

[0150] (3.47) InSAR 3D displacement: Extract the InSAR 3D displacement data for each building unit and analyze its displacement characteristics. Specifically, the InSAR LOS direction and 3D displacement calculation method is as follows:

[0151] The LOS displacement map was generated by differential interferometry using the registered SAR images. The ascending orbit data used the post-earthquake main image 20231226 and the pre-earthquake secondary image 20231027 to generate a differential interferometry pair with a spatial baseline of 63.880 meters. The descending orbit data used the post-earthquake main image 20231226 and the pre-earthquake secondary image 20231202 to generate a differential interferometry pair with a spatial baseline of 118.716 meters. The two-orbit method was used for the differential interferometry process. For any point on the interferogram, its deformation phase can be expressed as:

[0152] ;

[0153] Where, is the interference phase, represents the flat Earth phase, represents the terrain phase, represents the deformation phase, represents the atmospheric delay phase, represents the coherence loss noise phase or decorrelation noise phase, It also includes the orbit error phase caused by inaccurate orbit parameters , is the wavelength of the radar signal, B is the distance between the satellite orbits during the two observations, and B is called the spatial baseline. are parallel baselines, is the vertical baseline, R is the slant distance from the radar to the target during satellite observation, and h is the elevation of the point. Assuming that the surface deformation phase at this point ignores noise, the surface deformation can be expressed as:

[0154] ;

[0155] Using the error propagation law, the surface deformation is obtained The partial derivative of can be derived as follows:

[0156] ;

[0157] From the above, we can see that the basic principle of D-InSAR is to use the established high-precision external DEM data to obtain the differential interferometric phase information in the study area through the differential interferometric measurement process. Then, all irrelevant phases in the differential interferometric phase, such as the flat ground phase, are removed. , terrain phase , atmospheric phase and noise phase , and finally retain the deformation phase , and then convert the obtained deformation phase into LOS deformation information. The embodiment of the present invention uses GAMMA software to perform interference processing on the Sentinel-1A ascending and descending orbit data. During the interference processing, COP-DEM is used to remove the terrain phase, and the precise orbit ephemeris provided by ESA (European Space Agency) is used to reduce the orbit error. In order to suppress phase noise, the interference pattern is subjected to 1 (azimuth) × 5 (range) multi-view processing, and the signal-to-noise ratio of the interference pattern is improved by Goldstein filtering, and then the MCF (Minimum Cost Flow) phase unwrapping is completed to obtain the original interference pattern. At the same time, the tropospheric delay in the filtered interference pattern is corrected by GACOS software before unwrapping, and finally the phase is transformed into deformation to obtain the displacement of the ascending and descending orbit data in the LOS (Line of Sight) direction. Among them, the LOS displacement Interferometric phase with unwrapping The relationship can be expressed as:

[0158] ;

[0159] The three-dimensional deformation field is solved using MAI and DInSAR technology. SAR images before and after the earthquake are obtained after registration of the study area. The MAI method obtains the azimuth displacement through azimuth band splitting and interferometry technology. First, the GAMMA software is used to obtain the forward and backward images from the original SAR single-look complex (SLC) image through the azimuth common band filtering method. For one interference pair, four new SLCs can be generated, including the forward and backward master images, and the forward and backward slave images. The forward and backward pairs can then be used to generate the forward interferogram and the backward interferogram. Subsequently, the MAI phase can be generated based on the phase difference between the forward and backward interferograms. In order to obtain the azimuth displacement of the ascending orbit, it is necessary to use the polynomial fitting method to remove the flat ground trend phase and terrain phase from the ascending orbit MAI interferogram. Therefore, at a pixel, the azimuth displacement of the ascending orbit interferogram is The phase relationship with MAI is as follows.

[0160] ;

[0161] in is the azimuthal displacement of the ascending interferometer pair, is the MAI phase, l is the effective antenna length, n is the fraction of the full aperture width, and the same method is used to process the descending orbit interferometer to extract the descending orbit azimuth. displacement.

[0162] After geocoding the ascending and descending LOS and along-track displacement data obtained by DInSAR and resampling them to the same geographic grid, a 3D displacement map is generated using the least squares method. For any point, the relationship between the 3D displacement components and the LOS and along-track displacements can be expressed as:

[0163] ;

[0164] Where, is the deformation in the LOS direction calculated by DInSAR, 、 、 are the decomposed components of the LOS deformation in the vertical, north-south, and east-west directions, respectively. Subscripts a and d denote ascending and descending orbits, respectively. θ is the radar incident angle; is the azimuth of the satellite along the flight direction. Based on the above observation value equation, the least squares algorithm is used to solve the three-dimensional displacement map of the study area.

[0165] (3.5) Feature library establishment: Summarize the multi-scale feature data extracted above to establish a multi-scale feature library of building vector units.

[0166] (3.6) Data storage and management: The feature library is stored in the ArcGIS geographic database to facilitate subsequent query and analysis.

[0167] S4, correlation analysis and selection between damaged buildings and the multi-scale feature library (that is, from the multi-scale features corresponding to the building vector units, select the target multi-scale features whose correlation with the damaged building vector units is higher than the preset threshold), and divide the training, validation, and test data sets.

[0168] In one embodiment, the correlation between each feature in the multi-scale feature library of building vector units and intact and damaged buildings is analyzed. The importance of each feature in the dataset is evaluated using a Bayesian-optimized XGBoost classifier. Features with a high correlation with damaged buildings are selected as target multi-scale features for subsequent damaged building classification. Training, validation, and test datasets are created, and data augmentation techniques are used to increase the diversity of the datasets.

[0169] Specifically, you can use the Python programming language and related machine learning libraries (such as XGBoost and Hyperopt) to perform Bayesian optimization on the hyperparameters of the XGBoost classifier. The specific steps are as follows:

[0170] (4.1) Define the objective function: Build a function that accepts the XGBoost hyperparameters as input, evaluates the model’s performance (such as accuracy, AUC, etc.), and returns the evaluation result.

[0171] (4.2) Choose a Bayesian optimization tool: Use a Bayesian optimization library such as Hyperopt to automate the hyperparameter tuning optimization process.

[0172] (4.3) Setting the hyperparameter space: Define the range of hyperparameters to be optimized, including the learning rate (learning_rate), the maximum tree depth (max_depth), the subsample ratio (subsample), the feature sampling ratio (colsample_bytree), etc. In the above code, the space dictionary defines the search space for each hyperparameter.

[0173] (4.4) Perform Baye Optimization: Using the Baye Optimizer, we search within the defined hyperparameter space to find the optimal hyperparameter combination. In the above code, the fmin function performs the Baye optimization process, and the best variable stores the optimal hyperparameter combination.

[0174] (4.5) Train the final model: Use the optimal hyperparameters obtained through optimization to train the XGBoost model and evaluate its performance on the test set.

[0175] (4.6) Use XGBoost to train the model and obtain the importance score of each feature through its built-in feature importance evaluation function.

[0176] Based on the key feature scores, relevant features related to building damage are screened. This embodiment of the present invention selects the LOS displacement of descending and ascending data, the coherence and normalized coherence index, the backscatter coefficient difference and correlation coefficient, the NDBI index, the entropy (H) map, the scattering angle (α) and anisotropy (A), and InSAR three-dimensional displacement (east-west displacement, north-south displacement, and vertical displacement) as features for subsequent building damage detection. These multi-scale target features within building vector units are then used to create training, validation, and test datasets, and data augmentation techniques are used to increase the diversity of these datasets.

[0177] S5, based on the variational autoencoder and multi-layer perceptron, builds a damaged building detection network, performs network training and inference, and outputs the detection results of damaged buildings in the earthquake-stricken area.

[0178] The embodiment of the present invention proposes a building damage detection model based on feature extraction of variational autoencoder combined with improved multi-layer perceptron classifier. Figure 3 The schematic diagram of the structure of a damaged building detection network shown in FIG, includes a variational autoencoder and an improved multi-layer perceptron classifier (i.e. Figure 3The model first uses the encoder portion of the Disentangled Inferred Prior Variational Auto-Encoder (DIPVAE) model (the DIPVAE encoder for short) to learn high-quality latent representations of the input features, achieving fine-grained control of the latent feature distribution. This means that the variational autoencoder extracts the latent space vectors of the target multi-scale features corresponding to the building vector units. Subsequently, an improved multi-layer perceptron classifier architecture, incorporating hybrid convolutions, Transformer Token mixers, and channel-wise attention (ECA), achieves global and local information fusion in the feature space. The model takes as input the target multi-scale features of the building unit structure, and outputs a binary classification of building damage.

[0179] For ease of understanding, the embodiments of the present invention explain the variational autoencoder and the improved multi-layer perceptron classifier respectively.

[0180] (5.1) DIPVAE encoder-constrained feature representation learning model. The DIPVAE encoder is designed to effectively learn and decouple latent representations of high-dimensional input features. Building on the conventional VAE, the DIPVAE encoder introduces a regularization term in the latent space feature covariance matrix, making feature dimensions closer to independence and variance normalization. It also incorporates the Huber reconstruction loss, KL divergence constraint, and DIP regularization term (which constrains the diagonal and off-diagonal elements of the covariance matrix) into the classic VAE loss. This results in more reasonable reconstruction, more independent and interpretable latent representations, and higher feature discriminability, providing high-quality input features for subsequent classifiers (building damage detection).

[0181] See also Figure 4 Figure 1 shows a schematic diagram of the DIPVAE encoder architecture. The DIPVAE encoder comprises multiple hidden layers, each composed of multiple fully connected layers, batch normalization, and SiLU activation functions. The encoder, a fully connected network structure, maps target multi-scale features into a latent space. Two fully connected layers output the mean and log-variance of the latent space vector distribution parameters, which are then used to determine the parameters of the latent distribution. Latent features are sampled using a reparameterization technique. In this process, DIPVAE introduces the Disentangled Inferred Prior (DIP) regularization term, which imposes weighted constraints on the diagonal and off-diagonal elements of the covariance matrix between latent variables to enhance feature decoupling and interpretability in the latent space. Furthermore, by using the Huber Loss as a reconstruction loss and a KL divergence constraint for joint optimization, the model achieves a smoother and more robust latent representation while maintaining a compact feature representation.

[0182] (5.2) The improved multilayer perceptron classifier is the backbone of the entire classification network. Its core function is to perform subsequent feature integration and classification decisions based on the latent features provided by DIPVAE. Drawing on the multilayer perceptron network, the present invention shifts its data processing and feature extraction strategy from directly processing raw input to first learning latent feature representations using DIPVAE. Token mixing and channel mixing are also organically combined. The Token Mixer layer is used to fuse local and global features in the token dimension, and the Channel Mixer layer module is used to enhance features in the channel dimension.

[0183] Compared with the original multi-layer perceptron module, the improved multi-layer perceptron classifier uses DIPVAE to extract features and learn potential representations of input features (such as data representations from CSV). The features have been represented at a high level of abstraction before entering the classifier. At the same time, the Token Mixer layer is used to replace the pure MLP in the Token mixing stage to achieve local convolution pattern extraction and global attention fusion. In the channel mixing stage, the channel Mixer layer module and ECA channel attention are used to replace the simple MLP to make feature selection and emphasis more targeted. Therefore, the improved MLP_Mixer network proposed in the embodiment of the present invention flexibly integrates convolution, Transformer and channel attention to achieve efficient fusion of multi-dimensional feature information at lower complexity, thereby providing a more discriminative feature representation for the final binary classification task (building damaged or intact).

[0184] Please continue to see Figure 3 , Figure 3 It shows that the improved multi-layer perceptron classifier includes an input linear mapping layer (i.e. Figure 3 The fully connected layer at the top of the improved MLP module), multi-layer Token Mixer network, multi-layer channel Mixer network, pooling and output layer (that is, Figure 3 The global pooling layer, Dropout layer, fully connected layer and Sigmoid layer in the network are used to achieve full fusion of features in multiple dimensions.

[0185] Based on this structure, an embodiment of the present invention provides a process for processing an input latent space vector by an improved multilayer perceptron classifier, including:

[0186] (a) The latent space vector is mapped to a specified token dimension through the input linear mapping layer. For example, the input feature is mapped to a specified token dimension (such as 128 dimensions) to obtain the initial token representation.

[0187] (b) Through a multi-layer Token Mixer network, local convolution and global attention fusion are alternately performed on the latent space vector in the token dimension to construct a rich token-level representation. In practice, the multi-layer Token Mixer network fuses local and global information on the token sequence, enriching the contextual semantics of token-level features.

[0188] The Token Mixer Network organically integrates convolution operations with the Transformer attention mechanism to achieve local and global information fusion of features at the token dimension. By hybridizing depthwise separable convolutions (to capture local feature patterns) with multi-head self-attention (to establish global dependencies) within the same layer, the Token Mixer Network innovatively incorporates both spatial neighborhood features and cross-token relationship information, improving the expressive power of feature sequences.

[0189] See also Figure 5 The following is a schematic diagram of the structure of a Token Mixer network. The Token Mixer network includes a permutation layer (Permute), a depthwise separable convolution layer (Depthwise Convolution), a pointwise convolution layer (Pointwise Convolution), an improved Transformer encoder, and a layer normalization layer (LayerNorm). It enhances token-level feature representation by simultaneously focusing on local feature patterns and global correlations. Based on this, for any layer of the Token Mixer network, the Token Mixer network performs the following operations on the input features: through the depthwise separable convolution layer and the pointwise convolution layer, local convolution is performed on the input features in the token dimension to extract token-level local features, thereby capturing and enhancing token-level local patterns; through the improved Transformer encoder, global attention fusion is performed on the input features in the token dimension to capture the global dependencies and contextual information between tokens to extract token-level global features; through the layer normalization layer, feature fusion of token-level local features and token-level global features is performed, and stable gradient propagation is maintained. This hybrid strategy enables Token Mixer to focus on short-range dependencies within feature sequences while establishing global semantic associations, thereby achieving more expressive feature representation at the token level.

[0190] For further information, please see Figure 5The improved Transformer encoder provided by an embodiment of the present invention includes a multi-head attention layer (Multi-Head_Self-Attention), a layer normalization layer (LayerNorm), a fully connected layer (Fullyconnected layer), an activation function layer (GELU), another fully connected layer (Fully connected layer) and another layer normalization layer (LayerNorm).

[0191] In the Token Mixer layer, two residual connection structures are designed to enhance the transferability of feature representations and network optimization capabilities. The first residual connection structure is shown below: a residual connection is established between the input and output of the multi-head attention layer. Specifically, in the Token Mixer layer, the input features are processed through depthwise separable convolution and pointwise convolution before being fused with the initial input features through a residual connection. This residual path allows the model to capture spatial correlation while preserving the global feature representation of the input, alleviating the vanishing gradient problem caused by excessive information transformation. The second residual connection structure is shown below: a residual connection is established between the input of the multi-head attention layer and the output of another fully connected layer. Specifically, after the LayerNorm operation, the second residual connection is established, and the feature output of the Transformer Encoder is superimposed with the Transformer Encoder input features through a residual connection. The introduction of this residual connection facilitates the model's context-sensitive adaptive enhancement of input features while ensuring gradient smoothness during training, further improving training stability and model expressiveness.

[0192] (c) Through a multi-layer channel mixer network, feedforward enhancement and attention selection are performed on the feature vectors output by the token mixer network in the channel dimension, improving the discriminability and stability of the channel space. In the specific implementation, several layers of channel mixer networks further enhance and filter features in the channel dimension.

[0193] The Channel Mixer network performs nonlinear transformation and enhancement in the channel dimension, improving the discriminative power of feature representation through a feedforward fully connected structure and channel attention mechanism. It also integrates the residual structure with Efficient Channel Attention (ECA) to improve representation capabilities without compromising the stability of the original distribution, and effectively mitigates gradient vanishing and overfitting in multi-layer stacking. Figure 6The diagram shows the structure of a channel mixer network. Each channel mixer network consists of a layer normalization layer, two feed-forward fully connected layers (with GELU activation and dropout), and an ECA channel attention layer. A residual connection is used between the input of the layer normalization layer and the output of the ECA channel attention layer. LayerNorm is used to normalize input features to ensure training stability; the feed-forward network (two fully connected layers + GELU activation + dropout) is used to perform nonlinear mapping and enhancement on features, thereby improving feature abstraction; the ECA channel attention (optional) is used to selectively enhance the influence of key feature channels in the channel dimension; and the residual connection is used to add the input features to the processed features to maintain the continuity of the feature distribution and the stability of model training.

[0194] Among them, the ECA channel attention layer is a lightweight and efficient channel attention mechanism that captures the local interaction relationship between channels by performing one-dimensional convolution operations on features in the channel dimension (rather than complex fully connected layer mapping). This allows key feature channels to be highlighted without complex operations with too many additional parameters, weakening redundant or noisy features and achieving more efficient feature expression. For building damage detection tasks, features often come from multiple data sources and feature types (such as coherence, backscatter difference, deformation, etc.). The ECA channel attention layer can highlight the most discriminative features at the channel level, strengthen feature channels that are strongly correlated with damage patterns, and suppress channels that contribute less to decision-making, thereby improving the model's sensitivity and robustness to building damage details. For details, please refer to Figure 7 A schematic diagram of the structure of an ECA channel attention layer is shown.

[0195] On this basis, the input features are first normalized using LayerNorm, then nonlinearly mapped and enhanced using a feedforward fully connected network. The ECA channel attention layer weights and emphasizes channel information at this stage. Finally, the input features are added to the processed features via residual connections, improving feature expressiveness while maintaining model training stability and continuity of feature distribution. The channel mixer module effectively mixes information between channels and, in conjunction with the token mixer, enables comprehensive feature interaction across multiple dimensions.

[0196] (d) Through pooling and output layers, a binary classification of building damage is generated based on the feature vector output by the channel mixer network. In practice, the processed features are aggregated (e.g., through mean pooling) and mapped to a single output node through a linear layer to achieve the binary classification decision output.

[0197] On this basis, training of damaged building classification network is carried out.

[0198] All experiments were conducted using PyTorch on devices equipped with high-performance GPUs (CUDA) or CPUs (depending on availability). Prior to training, input features were normalized using StandardScaler to ensure numerical stability and promote model convergence. For DIPVAE and classification models, training data was fed into the model in mini-batches (e.g., batch size = 32), enabling efficient stochastic gradient descent updates.

[0199] For the DIPVAE encoder, this embodiment uses the AdamW optimizer with a learning rate of 1×10e5 and a total of 100 training epochs. The DIPVAE encoder is first trained to learn the disentangled latent representation. Then, a multilayer perceptron-based classifier is trained using the encoder parameters, also using the AdamW optimizer with a learning rate of 1×10e4. This embodiment employs cosine annealing with warm restarts or a step-by-step scheduler to dynamically adjust the learning rate, mitigating overfitting and improving model generalization. Intermediate model checkpoints are saved after each training session to ensure reproducibility and monitoring of results.

[0200] In addition, an embodiment of the present invention proposes determining a joint loss value based on the binary building damage classification type output by the damaged building classification network and the labels carried by the building vector units to simultaneously optimize feature decoupling and classification performance. The joint loss value includes a feature decoupling loss value and a classification accuracy loss value. The feature decoupling loss value is also the loss of the DIPVAE encoder, which is composed of a feature reconstruction loss value, a potential KL divergence loss value, and a potential regularization loss value. The calculation process of the joint loss value is as follows:

[0201] (I) Determining a feature reconstruction loss based on the target multi-scale features corresponding to the building vector units and the reconstructed features generated by the damaged building classification network during damage detection; and determining a latent KL divergence loss based on the feature mean and feature variance of each latent dimension in the latent space vector output by the variational autoencoder; and determining a latent regularization loss based on the feature variance of each latent dimension in the latent space vector output by the variational autoencoder and the feature covariance between any two latent dimensions.

[0202] In one example, to ensure that the damaged building classification network generates reconstruction features during damage detection, Compared with the original target multi-scale features Similarly, using Huber loss, feature reconstruction loss value The specific calculation formula is as follows:

[0203] ;

[0204] Where, is the threshold that determines the transition between the quadratic and linear regions of the Huber loss.

[0205] In one example, the KL divergence will be the underlying distribution Restricted to be close to the prior distribution p(z) (assumed to be Gaussian), the potential KL divergence loss value The specific calculation formula is as follows:

[0206] ;

[0207] Where Z is the latent space vector, d is the dimension of the latent space vector, and are the feature mean and feature variance of the i-th latent dimension in the latent space vector, respectively.

[0208] In one example, DIP regularization enforces decoupling by penalizing the covariance matrix of the latent variables, regularizing the loss The calculation is shown as follows:

[0209] ;

[0210] Where, is the covariance between the i-th latent dimension and the j-th latent dimension in the latent space vector, Var( ) is the variance of the ith latent dimension in the latent space vector, and Hyperparameters controlling off-diagonal and diagonal penalties.

[0211] (II) The feature reconstruction loss value, the potential KL divergence loss value, and the potential regularization loss value are weighted and fused to obtain the feature decoupling loss value. Among them, the feature decoupling loss value The calculation formula is as follows:

[0212] ;in, is the KL divergence loss value The weighting factor of .

[0213] (III) Determine the classification accuracy loss value based on the binary building damage type corresponding to the building vector unit and the label carried by the building vector unit. The classification accuracy loss value, also known as the binary cross entropy loss value, is used to evaluate the classification performance. The classification accuracy loss value is calculated as follows:

[0214] ;

[0215] Among them is No. The building damage binary classification type of building vector units, is the sigmoid function, N is the number of building vector units, It is The ground truth labels of the building vector units.

[0216] (IV) Perform weighted fusion of the feature decoupling loss and the classification accuracy loss to obtain the joint loss. The calculation formula for the joint loss is as follows:

[0217] ;

[0218] Where: is the feature decoupling loss value, which includes the feature reconstruction loss value , potential KL divergence loss value , potential regularization loss value , is the binary cross entropy loss for the binary classification task (i.e., the classification accuracy loss value). is a hyperparameter used to balance feature decoupling ( ) and classification accuracy ( ). This formulation ensures that the learned latent representations are disentangled and interpretable while optimizing classification accuracy.

[0219] Finally, the trained model was applied to damage detection of buildings in the study area, achieving automatic identification and detection of damaged buildings in the study area. In practical applications, while the classification results of damaged buildings obtained using the method provided by this embodiment of the present invention differed somewhat from the ground truth, the overall performance demonstrated the potential of this method for extracting damaged buildings in earthquake-stricken areas such as Jishishan County. These findings demonstrate that the model successfully captures the damage patterns of buildings caused by the earthquake and provides reliable classification results for practical post-earthquake building damage detection applications.

[0220] In summary, this method integrates the encoding part of the variational encoder with the multi-layer perceptron neural network, comprehensively utilizes multi-source remote sensing data to effectively extract the multi-dimensional features of damaged buildings, and significantly improves the performance of building damage classification. Based on this, the present invention provides an efficient damage detection solution for buildings in earthquake-stricken areas. Through an automated damage detection process, it quickly obtains building damage information, providing timely data support for post-disaster emergency rescue and building reconstruction. In addition, the embodiments of the present invention exhibit the advantages of high precision, multi-scale and high efficiency in building damage detection, which is of great significance for quickly obtaining building damage information, post-disaster emergency rescue and building reconstruction, and has important application value in post-disaster emergency response, reconstruction planning and other aspects, and can provide strong support for disaster management and decision-making. Furthermore, the embodiments of the present invention quickly obtain building damage information through an automated damage detection process, which is expected to be further promoted in the application of rapid assessment of earthquake building damage, providing more scientific data support and decision-making basis for disaster risk management and emergency response.

[0221] Based on the above embodiments, the present invention provides a building damage detection device that couples multiple features of SAR images and optical images. Figure 8 The structure diagram of a building damage detection device that couples SAR images and optical images with multiple features is shown. The device mainly includes the following parts:

[0222] Image acquisition module 802, used to acquire SAR images, optical images and UAV images corresponding to the study area in pre-disaster and post-disaster scenarios respectively;

[0223] An image processing module 804 is configured to construct a change detection image set for the study area based on the SAR image and the optical image, and to construct a vector dataset corresponding to the study area based on the optical image and the UAV image, wherein the vector dataset includes a vector unit for each building in the study area;

[0224] A feature library construction module 806 is configured to extract multi-scale features corresponding to building vector units from the change detection image set based on the vector data set to construct a multi-scale feature library, where the multi-scale features include one or more of H / A / Alpha polarization decomposition features, backscatter coefficient correlation, backscatter coefficient difference, InSAR coherence and normalized coherence index, NDBI index, and InSAR three-dimensional displacement;

[0225] The network training module 808 is used to train the damaged building classification network using a multi-scale feature library. The trained damaged building classification network is used to detect damage in the study area. The damaged building classification network includes a variational autoencoder and an improved multi-layer perceptron classifier. The improved multi-layer perceptron classifier is used to alternately perform local convolution and global attention fusion on the token dimension of the input latent space vector, and perform feedforward enhancement and attention selection on the channel dimension.

[0226] The embodiment of the present invention provides a building damage detection device that couples SAR images and optical image multi-features. It collects SAR images, optical images, and drone images corresponding to the study area in pre-disaster and post-disaster scenarios, and on this basis constructs a multi-scale feature library containing multi-scale features corresponding to each building vector unit, so as to improve the problems of single data source and strong feature extraction limitations in the existing technology. In addition, the multi-scale feature library is used to train a damaged building classification network, which includes a variational autoencoder and an improved multi-layer perceptron classifier. The trained damaged building classification network is used to detect damage in the study area. Through this network, the problems of weak model generalization ability, insufficient real-time performance and robustness in the existing technology can be effectively improved, thereby providing a high-precision, multi-scale solution for post-disaster building damage assessment.

[0227] In one embodiment, the building vector units are divided into damaged building vector units and intact building vector units; the network training module 808 is specifically configured to:

[0228] From the multi-scale features corresponding to the building vector units, target multi-scale features whose correlation with the damaged building vector units is higher than a preset threshold are selected;

[0229] The target multi-scale features corresponding to the building vector units are input into the damaged building classification network to obtain the binary building damage classification type output by the damaged building classification network. The joint loss value is determined by combining the labels carried by the building vector units. The joint loss value includes the feature decoupling loss value and the classification accuracy loss value.

[0230] The joint loss value is used to adjust the network parameters of the damaged building classification network.

[0231] In one embodiment, the improved multi-layer perceptron classifier includes an input linear mapping layer, a multi-layer TokenMixer network, a multi-layer channel Mixer network, a pooling and output layer; the network training module 808 is specifically used to:

[0232] Through the variational autoencoder, the latent space vector of the target multi-scale features corresponding to the building vector unit is extracted;

[0233] By inputting the linear mapping layer, the latent space vector is mapped to the specified token dimension;

[0234] Through a multi-layer Token Mixer network, local convolution and global attention fusion are alternately performed on the latent space vector in the token dimension;

[0235] Through a multi-layer channel mixer network, feedforward enhancement and attention selection are performed on the feature vector output by the token mixer network in the channel dimension;

[0236] Through the pooling and output layers, the binary classification of building damage is generated based on the feature vector output by the channel Mixer network.

[0237] In one embodiment, each Token Mixer network includes a depthwise separable convolution layer, a pointwise convolution layer, an improved Transformer encoder, and a layer normalization layer; the network training module 808 is specifically configured to:

[0238] For any layer of the Token Mixer network, the following operations are performed on the input features using the Token Mixer network:

[0239] Through the depth-wise separable convolution layer and the point-wise convolution layer, local convolution is performed on the input features in the token dimension to extract token-level local features;

[0240] Through the improved Transformer encoder, global attention fusion is performed on the input features in the token dimension to extract token-level global features;

[0241] Through the layer normalization layer, the token-level local features and token-level global features are fused.

[0242] In one embodiment, the improved Transformer encoder includes a multi-head attention layer, a layer normalization layer, a fully connected layer, an activation function layer, another fully connected layer, and another normalization layer;

[0243] The improved Transformer encoder adopts two forms of residual connection structures; among them, the first form of residual connection structure is as follows: a residual connection is made between the input end of the multi-head attention layer and the output end of the multi-head attention layer; the second form of residual connection structure is as follows: a residual connection is made between the input end of the multi-head attention layer and the output end of another fully connected layer.

[0244] In one embodiment, each channel Mixer network includes a layer normalization layer, a feedforward network, and an ECA channel attention layer, and a residual connection is formed between the input end of the layer normalization layer and the output end of the ECA channel attention layer.

[0245] In one embodiment, the network training module 808 is specifically configured to:

[0246] Determining a feature reconstruction loss value based on the target multi-scale features corresponding to the building vector units and the reconstructed features generated by the damaged building classification network during the damage detection process; and determining a latent KL divergence loss value based on the feature mean and feature variance of each latent dimension in the latent space vector output by the variational autoencoder; and determining a latent regularization loss value based on the feature variance of each latent dimension in the latent space vector output by the variational autoencoder and the feature covariance between any two latent dimensions;

[0247] The feature reconstruction loss value, potential KL divergence loss value, and potential regularization loss value are weightedly fused to obtain the feature decoupling loss value;

[0248] Determine the classification accuracy loss value based on the building damage binary classification type corresponding to the building vector unit and the label carried by the building vector unit;

[0249] The feature decoupling loss value and the classification accuracy loss value are weightedly fused to obtain the joint loss value.

[0250] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those in the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference can be made to the corresponding content in the aforementioned method embodiment.

[0251] An embodiment of the present invention provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned embodiments.

[0252] Figure 9 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 90, a memory 91, a bus 92 and a communication interface 93, wherein the processor 90, the communication interface 93 and the memory 91 are connected via the bus 92; the processor 90 is used to execute an executable module stored in the memory 91, such as a computer program.

[0253] Memory 91 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between the system network element and at least one other network element is achieved through at least one communication interface 93 (which may be wired or wireless), and may utilize the Internet, a wide area network, a local area network, a metropolitan area network, or the like.

[0254] The bus 92 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 9 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0255] Among them, the memory 91 is used to store programs, and the processor 90 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 90 or implemented by the processor 90.

[0256] The processor 90 may be an integrated circuit chip with signal processing capabilities. During implementation, the steps of the above-described method may be performed by hardware integrated logic circuits or software instructions within the processor 90. The processor 90 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention may be directly executed by a hardware decoding processor or by a combination of hardware and software modules within the decoding processor. The software modules may be located in storage media well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or the like. The storage medium is located in the memory 91 , and the processor 90 reads the information in the memory 91 and completes the steps of the above method in combination with its hardware.

[0257] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can be referred to the previous method embodiment and will not be repeated here.

[0258] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0259] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A building damage detection method that couples SAR image and optical image multi-features, characterized by: include: Obtain SAR images, optical images, and UAV images of the study area before and after the disaster; constructing a change detection image set for the study area based on the SAR image and the optical image, and constructing a vector dataset corresponding to the study area based on the optical image and the UAV image, wherein the vector dataset includes a vector unit for each building in the study area; Extracting, based on the vector data set, multi-scale features corresponding to the building vector units from the change detection image set to construct a multi-scale feature library, wherein the multi-scale features include one or more of H / A / Alpha polarization decomposition features, backscatter coefficient correlation, backscatter coefficient difference, InSAR coherence and normalized coherence index, NDBI index, and InSAR three-dimensional displacement; The multi-scale feature library is used to train a damaged building classification network. The trained damaged building classification network is used to detect damage in the study area. The damaged building classification network includes a variational autoencoder and an improved multi-layer perceptron classifier. The improved multi-layer perceptron classifier is used to alternately perform local convolution and global attention fusion on the token dimension of the input latent space vector, and perform feedforward enhancement and attention selection on the channel dimension.

2. The building damage detection method based on coupling SAR image and optical image multi-features according to claim 1 is characterized in that: The building vector units are divided into damaged building vector units and intact building vector units; The damaged building classification network is trained using the multi-scale feature library, including: Filtering, from the multi-scale features corresponding to the building vector units, target multi-scale features whose correlation with the damaged building vector units is higher than a preset threshold; Inputting the target multi-scale features corresponding to the building vector units into a damaged building classification network to obtain a binary building damage classification type output by the damaged building classification network, and determining a joint loss value in combination with the labels carried by the building vector units, wherein the joint loss value includes a feature decoupling loss value and a classification accuracy loss value; The network parameters of the damaged building classification network are adjusted using the joint loss value.

3. The building damage detection method based on coupling SAR image and optical image multi-features according to claim 2 is characterized in that: The improved multi-layer perceptron classifier includes an input linear mapping layer, a multi-layer token mixer network, a multi-layer channel mixer network, a pooling layer, and an output layer. The target multi-scale features corresponding to the building vector unit are input into a damaged building classification network to obtain a binary building damage classification type output by the damaged building classification network, including: Extracting the latent space vector of the target multi-scale feature corresponding to the building vector unit through the variational autoencoder; Mapping the latent space vector to a specified token dimension through the input linear mapping layer; By using multiple layers of the Token Mixer network, local convolution and global attention fusion are alternately performed on the latent space vector in the token dimension; Performing feedforward enhancement and attention selection on the feature vector output by the Token Mixer network in the channel dimension through multiple layers of the channel Mixer network; Through the pooling and output layers, a binary classification of building damage is generated based on the feature vector output by the channel Mixer network.

4. The building damage detection method based on coupling SAR image and optical image multi-features according to claim 3 is characterized in that: Each of the Token Mixer networks includes a depthwise separable convolution layer, a pointwise convolution layer, an improved Transformer encoder, and a layer normalization layer. Through multiple layers of the Token Mixer network, local convolution and global attention fusion are alternately performed on the latent space vector in the token dimension, including: For any layer of the Token Mixer network, the following operations are performed on the input features using the Token Mixer network: Performing local convolution on the input features in the token dimension through the depth-wise separable convolution layer and the point-wise convolution layer to extract token-level local features; Performing global attention fusion on the input features in the token dimension through the improved Transformer encoder to extract token-level global features; The token-level local features and the token-level global features are fused through the layer normalization layer.

5. The building damage detection method based on coupling SAR image and optical image multi-features according to claim 4 is characterized in that: The improved Transformer encoder includes a multi-head attention layer, a layer normalization layer, a fully connected layer, an activation function layer, another fully connected layer, and another normalization layer; The improved Transformer encoder adopts two forms of residual connection structures; among them, the first form of the residual connection structure is as follows: a residual connection between the input end of the multi-head attention layer and the output end of the multi-head attention layer; the second form of the residual connection structure is as follows: a residual connection between the input end of the multi-head attention layer and the output end of another fully connected layer.

6. The building damage detection method based on coupling SAR image and optical image multi-features according to claim 3, characterized in that: Each of the channel Mixer networks includes a layer normalization layer, a feedforward network, and an ECA channel attention layer, and a residual connection is formed between an input end of the layer normalization layer and an output end of the ECA channel attention layer.

7. The building damage detection method based on coupling SAR image and optical image multi-features according to claim 2, characterized in that: Determining a joint loss value in combination with the label carried by the building vector unit includes: Determining a feature reconstruction loss value based on the target multi-scale features corresponding to the building vector unit and the reconstructed features generated by the damaged building classification network during damage detection; and determining a potential KL divergence loss value based on the feature mean and feature variance of each latent dimension in the latent space vector output by the variational autoencoder; and determining a potential regularization loss value based on the feature variance of each latent dimension in the latent space vector output by the variational autoencoder and the feature covariance between any two of the latent dimensions; Performing weighted fusion on the feature reconstruction loss value, the potential KL divergence loss value, and the potential regularization loss value to obtain the feature decoupling loss value; Determining a classification accuracy loss value according to the binary building damage type corresponding to the building vector unit and the label carried by the building vector unit; The feature decoupling loss value and the classification accuracy loss value are weightedly fused to obtain a joint loss value.

8. A building damage detection device that couples SAR images and optical images with multiple features, characterized by: include: Image acquisition module, used to obtain SAR images, optical images and UAV images of the study area in pre-disaster and post-disaster scenarios respectively; an image processing module, configured to construct a change detection image set for the study area based on the SAR image and the optical image, and to construct a vector dataset corresponding to the study area based on the optical image and the UAV image, wherein the vector dataset includes a vector unit for each building in the study area; a feature library construction module, configured to extract, based on the vector data set, multi-scale features corresponding to the building vector units from the change detection image set to construct a multi-scale feature library, wherein the multi-scale features include one or more of H / A / Alpha polarization decomposition features, backscatter coefficient correlation, backscatter coefficient difference, InSAR coherence and normalized coherence index, NDBI index, and InSAR three-dimensional displacement; A network training module is used to train a damaged building classification network using the multi-scale feature library. The trained damaged building classification network is used to detect damage in the study area. The damaged building classification network includes a variational autoencoder and an improved multi-layer perceptron classifier. The improved multi-layer perceptron classifier is used to alternately perform local convolution and global attention fusion on the token dimension of the input latent space vector, and perform feedforward enhancement and attention selection on the channel dimension.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 7.

Images