Building damage detection method and device for coupling multiple features of SAR image and optical image

Abstract

The invention provides a building damage detection method and device for coupling multiple features of an SAR image and an optical image. The method comprises the following steps: acquiring an SAR image, an optical image and an unmanned aerial vehicle image corresponding to a research area in a pre-disaster scene and a post-disaster scene respectively; constructing a research area change detection image set based on the SAR image and the optical image, and constructing a vector data set corresponding to the research area based on the optical image and the unmanned aerial vehicle image; according to the vector data set, extracting multi-scale features corresponding to the building vector units from the change detection image set to construct a multi-scale feature library; and training the damaged building classification network by using the multi-scale feature library, wherein the trained damaged building classification network is used for carrying out damage detection on the research area. According to the method, the problems of single data source, relatively high feature extraction limitation, weak model generalization ability, insufficient real-time performance and robustness and the like in the prior art can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular, to a method and device for detecting building damage by coupling multi-features of SAR images and optical images. Background Art

[0002] As one of the most destructive natural disasters, earthquakes can trigger building collapses, landslides, and secondary disaster chain reactions, causing heavy casualties and property losses. Among them, building damage is the core factor leading to casualties, and rapid and accurate damage assessment plays a decisive role in post-disaster emergency rescue and reconstruction. Traditional methods relying on manual on-site investigations require experts to verify the internal and external features of buildings on-site, which have significant drawbacks such as low efficiency, high risk, and limited coverage. Remote sensing technology, with its advantages of non-contact, wide-area coverage, and high spatio-temporal resolution, has become a key means for post-disaster building damage assessment. Existing technologies mainly focus on the collaborative application of optical and synthetic aperture radar (SAR) sensors: optical images are easily restricted by cloud cover, while SAR, with its all-weather observation ability, shows higher applicability in damage detection. SAR-based assessment methods can be divided into two categories: (1) change detection methods: by comparing pre-disaster / post-disaster data and analyzing block units (intensity, coherence, and polarization features) or building units (extracting single-building features); (2) single-temporal assessment methods: using the polarization and texture features of post-disaster SAR images and combining machine learning or deep learning classifiers to achieve damage classification. In addition, methods that fuse optical images and geographic information system (GIS) data are used to improve detection accuracy, but such methods have problems such as single data source, strong limitations in feature extraction, weak model generalization ability, and insufficient real-time performance and robustness. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method and device for detecting building damage by coupling multi-features of SAR images and optical images, which can effectively improve problems in the prior art such as single data source, strong limitations in feature extraction, weak model generalization ability, and insufficient real-time performance and robustness, and provide a high-precision and multi-scale solution for post-disaster building damage assessment.

[0004] In the first aspect, the present invention provides a method for detecting building damage by coupling multi-features of SAR images and optical images, including: Obtaining SAR images, optical images, and UAV images corresponding to the study area in the pre-disaster scenario and the post-disaster scenario respectively; Constructing a change detection image set of the study area based on the SAR images and optical images, and constructing a corresponding vector data set of the study area based on the optical images and UAV images, where the vector data set includes each building vector unit in the study area; According to the vector dataset, multi-scale features corresponding to the building vector units are extracted from the change detection image set to construct a multi-scale feature library. The multi-scale features include one or more of H / A / Alpha polarization decomposition features, backscattering coefficient correlation, backscattering coefficient difference, InSAR coherence and normalized coherence index, NDBI index, and InSAR three-dimensional displacement. The damaged building classification network is trained using the multi-scale feature library. The trained damaged building classification network is used for damage detection in the study area. The damaged building classification network includes a variational autoencoder and an improved multi-layer perceptron classifier. The improved multi-layer perceptron classifier is used to alternately perform local convolution and global attention fusion on the input latent space vector in the token dimension and perform feed-forward enhancement and attention selection in the channel dimension.

[0005] In one implementation, the building vector units are divided into damaged building vector units and intact building vector units. Training the damaged building classification network using the multi-scale feature library includes: From the multi-scale features corresponding to the building vector units, target multi-scale features with a correlation higher than a preset threshold with the damaged building vector units are screened out. The target multi-scale features corresponding to the building vector units are input into the damaged building classification network to obtain the binary classification type of building damage output by the damaged building classification network. The joint loss value is determined in combination with the label carried by the building vector unit. The joint loss value includes a feature decoupling loss value and a classification accuracy loss value. The network parameters of the damaged building classification network are adjusted using the joint loss value.

[0006] In one implementation, the improved multi-layer perceptron classifier includes an input linear mapping layer, multiple layers of TokenMixer networks, multiple layers of channel Mixer networks, a pooling and output layer. Inputting the target multi-scale features corresponding to the building vector units into the damaged building classification network to obtain the binary classification type of building damage output by the damaged building classification network includes: Through the variational autoencoder, the latent space vector of the target multi-scale features corresponding to the building vector units is extracted. Through the input linear mapping layer, the latent space vector is mapped to the specified token dimension. Through multiple layers of Token Mixer networks, local convolution and global attention fusion are alternately performed on the latent space vector in the token dimension. Through multiple layers of channel Mixer networks, feed-forward enhancement and attention selection are performed on the feature vector output by the Token Mixer network in the channel dimension. Generate the binary classification type of building damage based on the feature vector output by the channel Mixer network through pooling and output layers.

[0007] In one implementation, each Token Mixer network includes a depthwise separable convolutional layer, a pointwise convolutional layer, an improved Transformer encoder, and a layer normalization layer; through multiple Token Mixer networks, local convolution and global attention fusion are alternately performed on the latent space vector in the token dimension, including: For any layer of the Token Mixer network, perform the following operations on the input features using this Token Mixer network: Perform local convolution on the input features in the token dimension through the depthwise separable convolutional layer and the pointwise convolutional layer to extract token-level local features; Perform global attention fusion on the input features in the token dimension through the improved Transformer encoder to extract token-level global features; Perform feature fusion on the token-level local features and the token-level global features through the layer normalization layer.

[0008] In one implementation, the improved Transformer encoder includes a multi-head attention layer, a layer normalization layer, a fully connected layer, an activation function layer, another fully connected layer, and another layer normalization layer; The improved Transformer encoder adopts two forms of residual connection structures; among them, the first form of the residual connection structure is as follows: there is a residual connection between the input end and the output end of the multi-head attention layer; the second form of the residual connection structure is as follows: there is a residual connection between the input end of the multi-head attention layer and the output end of another fully connected layer.

[0009] In one implementation, each channel Mixer network includes a layer normalization layer, a feed-forward network, and an ECA channel attention layer, and there is a residual connection between the input end of the layer normalization layer and the output end of the ECA channel attention layer.

[0010] In one implementation, determine the joint loss value in combination with the label carried by the building vector unit, including: Determine the feature reconstruction loss value based on the target multi-scale features corresponding to the building vector unit and the reconstruction features generated during the damage detection process by the damaged building classification network; and, determine the latent KL divergence loss value according to the feature mean and feature variance of each latent dimension in the latent space vector output by the variational autoencoder; and, determine the latent regularization loss value according to the feature variance of each latent dimension in the latent space vector output by the variational autoencoder and the feature covariance between any two latent dimensions; Perform weighted fusion on the feature reconstruction loss value, the latent KL divergence loss value, and the latent regularization loss value to obtain the feature decoupling loss value; Determine the classification accuracy loss value according to the binary classification type of building damage corresponding to the building vector unit and the label carried by the building vector unit; Perform weighted fusion on the feature decoupling loss value and the classification accuracy loss value to obtain the joint loss value.

[0011] In a second aspect, the present invention further provides a building damage detection device that couples multi-features of SAR images and optical images, including: An image acquisition module, configured to acquire SAR images, optical images, and drone images corresponding to the study area in the pre-disaster scenario and the post-disaster scenario respectively; An image processing module, configured to construct a change detection image set of the study area based on the SAR image and the optical image, and construct a vector data set corresponding to the study area based on the optical image and the drone image, where the vector data set includes each building vector unit in the study area; A feature library construction module, configured to extract multi-scale features corresponding to the building vector unit from the change detection image set according to the vector data set to construct a multi-scale feature library, where the multi-scale features include one or more of H / A / Alpha polarization decomposition features, backscattering coefficient correlation, backscattering coefficient difference, InSAR coherence and normalized coherence index, NDBI index, and InSAR three-dimensional displacement; A network training module, configured to train the damaged building classification network using the multi-scale feature library, and the trained damaged building classification network is used to perform damage detection on the study area. The damaged building classification network includes a variational autoencoder and an improved multi-layer perceptron classifier. The improved multi-layer perceptron classifier is configured to alternately perform local convolution and global attention fusion on the token dimension for the input latent space vector, and perform feed-forward enhancement and attention selection on the channel dimension.

[0012] In a third aspect, the present invention further provides an electronic device, including a processor and a memory. The memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the method according to any one of the first aspect.

[0013] In a fourth aspect, the present invention further provides a computer-readable storage medium storing computer-executable instructions, which, when called and executed by a processor, cause the processor to implement the method according to any one of the first aspect.

[0014] A building damage detection method and device for coupling multi-features of SAR images and optical images provided by the present invention first obtain SAR images, optical images, and UAV images corresponding to a study area in pre-disaster and post-disaster scenarios respectively; then construct a change detection image set of the study area based on the SAR images and optical images, and construct a vector data set corresponding to the study area based on the optical images and UAV images, where the vector data set includes each building vector unit within the study area; then, according to the vector data set, multi-scale features corresponding to the building vector units are extracted from the change detection image set to construct a multi-scale feature library, and the multi-scale features include one or more of H / A / Alpha polarization decomposition features, backscattering coefficient correlation, backscattering coefficient difference, InSAR coherence, normalized coherence index, NDBI index, and InSAR three-dimensional displacement; finally, the multi-scale feature library is used to train a damaged building classification network, and the trained damaged building classification network is used to detect damage in the study area. The damaged building classification network includes a variational autoencoder and an improved multi-layer perceptron classifier. The improved multi-layer perceptron classifier is used to alternately perform local convolution and global attention fusion on the input latent space vector in the token dimension, and perform feed-forward enhancement and attention selection in the channel dimension. The above method collects SAR images, optical images, and UAV images corresponding to a study area in pre-disaster and post-disaster scenarios respectively, and constructs a multi-scale feature library containing multi-scale features corresponding to each building vector unit on this basis to improve problems such as single data source and strong limitations in feature extraction in the prior art. In addition, the multi-scale feature library is used to train a damaged building classification network, which includes a variational autoencoder and an improved multi-layer perceptron classifier. The trained damaged building classification network is used to detect damage in the study area. Through this network, problems such as weak model generalization ability, lack of real-time performance and robustness in the prior art can be effectively improved, thereby providing a high-precision and multi-scale solution for post-disaster building damage assessment.

[0015] Other features and advantages of the present invention will be described in the following specification, and in part will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification, claims, and drawings.

[0016] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following specifically presents preferred embodiments and, in conjunction with the accompanying drawings, provides a detailed description as follows. Description of the Drawings

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 Schematic flowchart of a building damage detection method for coupling multi - features of SAR images and optical images provided by an embodiment of the present invention; Figure 2 Technical framework diagram of a building damage detection method for coupling multi - features of SAR images and optical images provided by an embodiment of the present invention; Figure 3 Schematic structural diagram of a damaged building detection network provided by an embodiment of the present invention; Figure 4 Schematic structural diagram of a DIPVAE encoder provided by an embodiment of the present invention; Figure 5 Schematic structural diagram of a Token Mixer network provided by an embodiment of the present invention; Figure 6 Schematic structural diagram of a channel Mixer network provided by an embodiment of the present invention; Figure 7 Schematic structural diagram of an ECA channel attention layer provided by an embodiment of the present invention; Figure 8 Schematic structural diagram of a building damage detection device for coupling multi - features of SAR images and optical images provided by an embodiment of the present invention; Figure 9 Schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed Embodiments

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0020] Currently, there are multiple existing technologies related to building damage detection methods. For example, the patent application "Sequential InSAR Coherence Map Building Damage Detection Method Based on Generative Model" (Publication No.: CN119516384A) proposes an InSAR surface deformation anomaly detection method based on a generative model. This method relies on a single InSAR data source and does not combine optical images or other features, resulting in limited discrimination ability for complex damage types (such as local collapse); the computational complexity of the generative model is high, making it difficult to meet the requirements of rapid post-disaster assessment. Another patent application, "A Post-earthquake Building Damage Detection Method Based on UAV Video" (Publication No.: CN111126184A), proposes a post-earthquake building damage detection method based on UAV video. It uses the UAV video stream for real-time detection and identifies damaged areas through dynamic target tracking and image segmentation. However, this method mainly relies on UAV video data, may be limited by weather and lighting conditions, and lacks SAR deformation feature fusion. In addition, the patent "Building Damage Detection Method and Device Based on Image Superpixel Fusion" (Grant Publication No.: CN109360190B) proposes a building damage detection method based on image superpixel fusion. However, this method does not integrate InSAR deformation and coherence features, has a single-dimensional feature library, and the classification model has weak generalization ability. Another patent, "An Aerial Image Building Damage Detection Method Based on Shadow and Texture Features" (Grant Publication No.: CN105631892B), proposes an aerial image building damage detection method based on shadow and texture features. However, this method mainly relies on shadow and texture features. The shadow features are easily interfered by the shooting angle, resulting in a high false detection rate, and do not quantify SAR physical features (such as backscattering coefficient).

[0021] In addition, Interferometry Synthetic Aperture Radar (InSAR) can accurately measure millimeter-level surface deformation through multi-temporal SAR image interferometry processing and has been widely used in earthquake disaster monitoring. Recent research shows that InSAR deformation information can indirectly reflect building damage characteristics (Du Y et al 2024; Wang C et al 2024), but the robustness of its single deformation parameter is insufficient to cope with complex damage patterns.

[0022] In summary, the above research still has the following problems: (1) Data source singularity: Most methods only rely on single data such as optical or SAR data, without comprehensively considering multi-source remote sensing features (such as InSAR deformation, coherence, NDBI index, etc.), resulting in incomplete feature information. (2) Limitations in feature extraction: Traditional methods rely on manually designed features (such as shadows, textures), and have poor adaptability to complex damage patterns (such as multi-scale deformation), resulting in low accuracy in building damage detection. (3) Weak model generalization ability: The classifier design is simple (such as threshold segmentation, traditional machine learning), without combining the feature self-extraction advantages of deep learning, resulting in limited accuracy. (4) Insufficient real-time performance and robustness: Some methods rely on high-resolution image processing, with low computational efficiency and are greatly affected by environmental interference (such as cloud occlusion, light changes).

[0023] Based on this, the embodiments of the present invention provide a building damage detection method and device that couples multi-features of SAR images and optical images. By integrating multi-source remote sensing data (including SAR images, optical images, and UAV images) and combining variational autoencoders and multi-layer perceptron neural networks for building damage detection, it can effectively improve problems such as single data source, strong limitations in feature extraction, weak model generalization ability, and insufficient real-time performance and robustness in the prior art, providing a high-precision and multi-scale solution for post-disaster building damage assessment, and having significant academic value and engineering application potential.

[0024] For the convenience of understanding this embodiment, first, a building damage detection method that couples multi-features of SAR images and optical images disclosed in the embodiments of the present invention will be introduced in detail. Refer to Figure 1 the schematic flowchart of a building damage detection method that couples multi-features of SAR images and optical images shown in the figure. This method mainly includes the following steps S102 to S108: Step S102, obtain the SAR images, optical images, and UAV images corresponding to the pre-disaster scenario and post-disaster scenario of the research area respectively.

[0025] In one example, taking an earthquake as an example, the SAR images, optical images, and UAV images corresponding to the pre-earthquake scenario and post-earthquake scenario of the research area can be obtained.

[0026] Step S104, construct a change detection image set of the research area based on the SAR images and optical images, and construct a corresponding vector data set of the research area based on the optical images and UAV images.

[0027] Among them, the change detection image set includes one or more of the backscattering coefficient difference map, NDBI index difference map, entropy difference map, scattering angle difference map, anisotropy difference map, coherence map, and differential interferometric pair; the vector data set contains each building vector unit within the study area, and the building vector units are divided into damaged building vector units and intact building vector units, and the building vector units are used to describe the spatial range of the buildings.

[0028] In one example, for the optical images and SAR images before and after the earthquake, the enhanced difference map algorithm is used to generate difference maps, and based on the SAR images before and after the earthquake, coherence maps and differential interferometric pairs are generated for subsequent multi-scale feature analysis of damaged buildings; at the same time, high-resolution drone images and optical images are used, and visual interpretation is adopted to construct the vector data set of damaged building areas and intact building areas.

[0029] Step S106, according to the vector data set, extract the multi-scale features corresponding to the building vector units from the change detection image set to construct a multi-scale feature library.

[0030] Among them, the multi-scale features include one or more of the H / A / Alpha polarization decomposition features, backscattering coefficient correlation, backscattering coefficient difference, InSAR coherence and normalized coherence index, NDBI (Naturalistic Developmental Behavioral Interventions, normalized building) index, and InSAR three-dimensional displacement. In one example, for any image in the change detection image set, the vector data set can be used to extract the sub-image corresponding to the building vector unit from this image, and then extract the features corresponding to the building vector unit from the sub-image. On this basis, the multi-scale features corresponding to each building vector unit can be extracted, and then a multi-scale feature library can be constructed.

[0031] Step S108, use the multi-scale feature library to train the damaged building classification network, and the trained damaged building classification network is used to perform damage detection on the study area.

[0032] Among them, the damaged building classification network includes a variational autoencoder and an improved multi-layer perceptron classifier. The variational autoencoder is used to extract latent space vectors, and the improved multi-layer perceptron classifier is used to alternately perform local convolution and global attention fusion on the input latent space vectors in the token dimension and perform feed-forward enhancement and attention selection in the channel dimension. In one example, first, the correlation between each feature in the multi-scale feature library and intact buildings and damaged buildings is analyzed, and the features with higher correlation with damaged buildings are selected to obtain the target multi-scale features corresponding to each building vector unit; then the target multi-scale features corresponding to the building vector units are input into the damaged building classification network to obtain the binary classification type of building damage output by the damaged building classification network, and the joint loss value is determined in combination with the labels carried by the building vector units. The binary classification type of building damage is used to describe whether the building unit is damaged. The joint loss value includes a feature decoupling loss value and a classification accuracy loss value. The network parameters of the damaged building classification network are adjusted using the joint loss value to realize the training of the damaged building classification network.

[0033] The building damage detection method that couples multi-features of SAR images and optical images provided by the embodiments of the present invention collects the SAR images, optical images, and UAV images corresponding to the pre-disaster scenario and the post-disaster scenario of the research area respectively, and on this basis, constructs a multi-scale feature library containing multi-scale features corresponding to each building vector unit to improve problems such as single data source and strong limitations in feature extraction in the prior art. In addition, the multi-scale feature library is used to train the damaged building classification network, which includes a variational autoencoder and an improved multi-layer perceptron classifier. The trained damaged building classification network is used to detect damage in the research area. Through this network, problems such as weak model generalization ability, lack of real-time performance and robustness in the prior art can be effectively improved, and thus a high-precision and multi-scale solution for post-disaster building damage assessment is provided.

[0034] For ease of understanding, the embodiments of the present invention provide a specific implementation manner of a building damage detection method that couples multi-features of SAR images and optical images. Please refer to Figure 2The technical framework diagram of a building damage detection method that couples multi-features of SAR images and optical images. The core idea of this method is as follows: S1, obtain multi-source remote sensing data, including SAR images, optical images, and UAV images, and preprocess them; S2, create difference maps of multi-source remote sensing images, and construct vector data of damaged and intact building areas; S3, construct a multi-scale feature library of building vector units; S4, analyze and select the correlation between damaged buildings and the multi-scale feature library, and divide the training, validation, and test data sets; S5, construct a damaged building detection network based on a variational autoencoder and a multi-layer perceptron, and perform network training and inference to output the detection results of damaged buildings in the earthquake-stricken area, that is, the results include the binary classification types of building damage corresponding to each building vector unit in the study area.

[0035] The embodiments of the present invention further explain the foregoing S1 to S5.

[0036] S1, obtain multi-source remote sensing data, including SAR images, optical images, and UAV images, and preprocess them.

[0037] Obtain high-resolution SAR images, optical images, and UAV images of the earthquake-stricken area, and preprocess them. Among them, the optical images are subjected to steps such as radiometric calibration, atmospheric correction, and orthorectification; the UAV images are subjected to steps such as regional overall adjustment, aerial triangulation, three-dimensional TIN (Triangulated Irregular Networ) network construction, three-dimensional reconstruction and visualization, etc., and the GRD (Grid Data Format) SAR images are subjected to steps such as orbit correction, thermal noise removal, radiometric calibration, multi-look, speckle filtering, range-Doppler terrain correction, registration, etc.; the SLC format SAR images are subjected to steps such as data import, update of precise orbit files, registration, etc.

[0038] Specifically, obtain data: Obtain pre-earthquake and post-earthquake high-resolution SAR images, optical images, UAV images, and digital elevation model (DEM) data according to the geographical location of the study area in terms of longitude and latitude. Taking the study area of the Jishishan earthquake in Gansu on December 18, 2023 as an example, the Sentinel-1A, Sentinel-2, and Jilin-1 satellite data information collected is shown in Table 1.

[0039] Table 1: Detailed information of the multi-source remote sensing data set of the Jishishan earthquake in Gansu

[0040] On this basis, the embodiments of the present invention further provide a specific process of data preprocessing.

[0041] In one example, the Sentinel-1A preprocessing process is as follows: For Sentinel-1 data in GRD format, Sentinel-1 GRD format data before and after the earthquake in the study area coverage are used. Steps such as orbit correction, thermal noise removal, radiometric calibration, multi-looking, speckle filtering, range-Doppler terrain correction, and registration are performed using SNAP software to obtain the backscatter coefficient map. Then, a difference map of the backscatter coefficients of two scenes with a pre-earthquake acquisition time of 20231214 and a post-earthquake acquisition time of 20231226 is made for subsequent extraction of the backscatter coefficient difference and correlation coefficient characteristics of the damaged buildings in the Jishishan earthquake.

[0042] For Sentinel-1 data in IW mode, three scenes of Sentinel-1 in IW mode before and after the earthquake in the study area coverage are used. Data import, precise orbit file update, TOPS (Terrain Observation by Progressive Scans) data segmentation, de-slanting, DEM registration, anti-de-slanting, Burst stitching, and sub-swath mosaicking are performed using SNAP software. It should be noted that the ESD (Enhanced Spectral Divergence) registration method is used for precise correction of azimuthal offset. The above steps such as de-slanting, interpolation, and anti-de-slanting are all performed at the burst level. After the above processing, the registered SLC format SAR image is obtained for subsequent generation of coherence, normalized coherence, and InSAR displacement.

[0043] In one example, the Sentinel-2 preprocessing process is as follows: Radiometric calibration, atmospheric correction, and orthorectification processes are performed on the Sentinel-2 image using ENVI software to obtain the processed image for subsequent extraction of the NDBI index.

[0044] In one example, the preprocessing process of Jilin-1 optical satellite images is as follows: First, radiometric calibration, atmospheric correction, and orthorectification (using a 12.5 m DEM for correction) are performed on the multispectral image. Subsequently, radiometric calibration and orthorectification are performed on the panchromatic band image using the same method. Then, the GS (Gram-Schmidt Panchromatic Sharpening) panchromatic sharpening image fusion algorithm is used to fuse the orthorectified panchromatic band and multispectral band. Finally, an optical satellite image with a spatial resolution of 0.75 m is obtained.

[0045] In one example, the preprocessing process of UAV images is as follows: ContextCapture (CC) software is adopted. First, the original image data captured by the UAV is imported, including the original images, camera files, and POS (Position and Orientation System) files, for regional overall adjustment and aerial triangulation to solve the dense point cloud. On this basis, through multi-view image dense matching, a three-dimensional TIN grid is constructed to generate a preliminary three-dimensional model. Subsequently, using the automatic texture mapping technology, the refinement processing and texture mapping of the three-dimensional model are completed, and finally a high-precision three-dimensional scene is generated to realize the three-dimensional reconstruction and visualization of the UAV images in the Jishishan area.

[0046] S2. Production of multi-source remote sensing image difference maps and construction of vector data for damaged and intact building areas. Specifically: The enhanced difference map algorithm is used for the pre-earthquake and post-earthquake optical images and SAR images obtained to generate difference maps for change detection, which are used for subsequent multi-scale feature analysis of damaged buildings. And visual interpretation is used with high-resolution UAV images and optical images to construct vector data sets for damaged and intact building areas. Specifically included are: (2.1) Production of difference maps for the Sentinel-2 and Sentinel-1 GRD format data after the above preprocessing, including backscatter coefficient difference maps, NDBI index difference maps, entropy (H) difference maps, scattering angle (α) difference maps, and anisotropy (A) difference maps.

[0047] In one example, first, based on the preprocessed Sentinel-2 and Sentinel-1 GRD format data, the pre-earthquake and post-earthquake backscatter coefficient maps, NDBI index maps, and H / A / Alpha polarization decomposition feature maps (including entropy (H) maps, scattering angle (α) maps, and anisotropy (A) maps) are respectively determined. The value of each pixel in the backscatter coefficient map is the backscatter coefficient, the value of each pixel in the NDBI index map is the NDBI index, the value of each pixel in the entropy (H) map is the entropy, the value of each pixel in the scattering angle (α) map is the scattering angle, and the value of each pixel in the anisotropy (A) map is the anisotropy; then the corresponding difference maps can be obtained by taking the difference between the pre-earthquake and post-earthquake images.

[0048] The embodiments of the present invention further provide the specific processes for generating NDBI index maps, entropy (H) maps, scattering angle (α) maps, and anisotropy (A) maps: (2.11) NDBI index map: The extraction of the NDBI index is completed using the spectral reflectance of the near-infrared (NIR) and short-wave infrared (SWIR) bands of the optical image. The NDBI calculation formula is as follows: ; Among them, NIR is the near-infrared band, B8 (central wavelength is about 842 nm), and SWIR is the short-wave infrared band: B11 (central wavelength is about 1610 nm) or B12 (2190 nm).

[0049] (2.12) H / A / Alpha polarization decomposition feature map: Taking the Sentinel-1 GRD format data as an example, the GRD Sentinel-1 data in VH and VV dual-polarization modes after registration is used to extract the scattering mechanism of buildings after the earthquake by using the H / A / Alpha decomposition method. According to the eigenvalues and eigenvectors of the polarization coherence matrix, three parameters, entropy (H), scattering angle (α), and anisotropy (A), can be calculated. The H / A / Alpha parameters are obtained by the SNAP software. First, the scattering matrix is defined as: ; In the formula, 、 respectively represent the complex scattering coefficients of different polarization channels, where is the scattering coefficient of vertical transmission and horizontal reception, is the scattering coefficient of vertical transmission and vertical reception.

[0050] The corresponding target scattering vector k can be expressed as: ; Therefore, the dual-polarization SAR data can be represented by the following polarization covariance matrix: ; Among them represents the conjugate transpose, and ∗ is the conjugate operator. The covariance matrix C is a positive semi-definite matrix, which can be decomposed into the weighted sum of two mutually orthogonal covariance matrices, expressed as: ; Among them, the real number represents the eigenvalue of the polarization covariance matrix C; represents the eigenvector; z is the rank of the matrix.

[0051] Entropy (H) describes the disorder of various different scattering types (i.e., the randomness of scattering). The range of entropy (H) is from 0 to 1, indicating the randomness of scattering. The formula is as follows: ; ; can be called the eigenvalue weight or the scattering mechanism probability.

[0052] Anisotropy (A), as a supplementary parameter to entropy, can provide information about surface roughness. See the following formula: ; Lower anisotropy (A) values indicate that the main scattering mechanism is single scattering, while medium or higher anisotropy (A) values indicate that the main scattering mechanism is double scattering.

[0053] The scattering angle (α) is directly related to the average physical scattering mechanism. Its value ranges from 0 to 90 degrees.

[0054] ; Through the above process, images of three parameters, namely entropy (E), scattering angle (α), and anisotropy (A), used to describe pre-earthquake and post-earthquake SAR images, can be calculated and are used for the analysis of post-earthquake building damage detection.

[0055] (2.2) Use the enhanced difference map algorithm to enhance the above difference map to improve the contrast of the changed area in the difference image.

[0056] Change detection technology has become an important means for building damage detection after an earthquake, and the quality of the generated difference map directly affects the detection and segmentation accuracy of the building change area. The embodiments of the present invention provide a specific process for processing any generated difference map using the enhanced difference map algorithm, including the following (2.21) to (2.24): (2.21) In view of the advantages of the neighborhood difference difference map and the neighborhood logarithm ratio difference map, the embodiments of the present invention propose a difference map enhancement method based on the weighted Chebyshev distance and gray-scale similarity, aiming to improve the contrast of the changed area in the difference image while reducing the pseudo-changes caused by speckle noise.

[0057] Use a neighborhood window to calculate the Chebyshev distance between each pixel in the difference difference map and its surrounding 8-neighborhood pixels and the gray-scale similarity , and its mathematical expression is as shown in the formula: ; ; where represents the row and column coordinates of a single pixel a, represents the row and column offsets relative to the central pixel , , represents the value of the pixel point in the difference difference map. The smaller this similarity metric value is, the closer the gray-scale values of the adjacent pixel and the central pixel are, and the more likely they belong to similar changed or background feature regions.

[0058] (2.22) Then, apply Gaussian function weighting to the Chebyshev distance and the gray-scale similarity respectively. Set the Gaussian parameter , and its mathematical expression is: ; ; is the index of the pixel in the 3*3 neighborhood, used to distinguish 8 neighboring pixels, is the standard deviation of the Gaussian distribution, used to control the spatial and gray-scale similarity sensitivity of the weighting. is the Gaussian weighting factor for the distance. is the Gaussian weighting factor for the similarity. Through Gaussian weighting, pixels that are close in distance and have similar change characteristics are given higher weights, and pixels that are far from the center and have large differences in change characteristics are given lower weights. This makes the enhanced result more able to highlight the change signals in the local spatial similarity regions.

[0059] (2.23) Multiply the distance weight and the similarity weight and sum over the neighborhood: ; is the combined weight, which is the total weight that integrates the contributions of all neighboring pixels under the distance and similarity weights.

[0060] (2.24) Multiply the initial difference value corresponding to the pixel at position ( ) in the difference map by the combined weight to enhance the change information of this pixel, and then normalize to ensure that the values of the final result image are within a unified range, which is beneficial for subsequent visualization and analysis. The calculation formula is as follows: ; is the enhanced difference value corresponding to the pixel at position ( ) in the difference map. Through the above calculation process, the production of the difference map of multi-source remote sensing data can be completed.

[0061] (2.3) Use the processed 3D reconstructed UAV images combined with Jilin-1 satellite images to construct a multi-scale feature dataset for each building unit, and use ArcGIS software to mark damaged and intact buildings and produce the vector map of house samples. Finally, 3774 houses are extracted in the embodiment of the present invention, including 1637 damaged houses and 2137 intact houses.

[0062] S3. Construct a multi-scale feature library for building vector units, including multi-scale image data such as H / A / Alpha polarization decomposition features, backscattering coefficient correlation, backscattering coefficient difference, InSAR coherence, normalized coherence index, NDBI index, and InSAR three-dimensional displacement, and establish a multi-scale feature library for building vector units.

[0063] Specifically, use ArcGIS software to extract the multi-scale features within these building vector units and establish a multi-scale feature library for building vector units. This includes processes such as data import, raster data conversion, raster data clipping, feature extraction, feature library establishment, data storage, and management.

[0064] (3.1) Data import: Import the above remote sensing data into the ArcGIS environment to ensure the consistency of the spatial reference of the data for subsequent analysis.

[0065] (3.2) Raster data conversion: Convert various types of remote sensing data into raster format for convenient spatial analysis.

[0066] (3.3) Raster data clipping: Use the boundaries of the building vector units to clip the raster data and extract the relevant feature data within each building unit.

[0067] (3.4) Feature extraction: Calculate the following multi-scale features within each building vector unit: (3.41) H / A / Alpha polarization decomposition features: Use the polarization decomposition method to extract the H / A / Alpha polarization feature parameters of each building unit. In specific implementation, the three parameters of entropy (H), scattering angle (α), and anisotropy (A) can be directly extracted from the entropy (H) difference map, scattering angle (α) difference map, and anisotropy (A) difference map made in the aforementioned S2.

[0068] (3.42) Backscattering coefficient correlation: Calculate the correlation between the backscattering coefficients of different polarization channels within each building unit to evaluate the consistency of its scattering characteristics. Specifically, the calculation method of the backscattering coefficient correlation is as follows: Make a difference map of the backscattering coefficients of two scenes with the pre-earthquake acquisition time of December 14, 2023, and the post-earthquake acquisition time of December 26, 2023. The calculation formula for the backscattering coefficient correlation is as follows: ; In the formula, r is the backscattering coefficient correlation, and N represents the total number of pixels within the target window. , respectively represent the values of the i-th pixel before and after the earthquake, , represents the average value of the pixel values in the 13×13 neighborhood around pixel i.

[0069] (3.43)Backscattering coefficient difference: Calculate the difference in backscattering coefficients between different polarization channels and analyze the differences in their scattering characteristics. Specifically: Create a difference map of the backscattering coefficients for two scenes, one with a pre-earthquake acquisition time of 20231214 and the other with a post-earthquake acquisition time of 20231226. The formula for calculating the backscattering coefficient difference is as follows: ; In the formula, represents the backscattering coefficient difference, , represents the average value of the pixel values in the 13×13 neighborhood around pixel i in the difference map.

[0070] (3.45)InSAR coherence and normalized coherence index: Extract the InSAR coherence value for each building unit and calculate the normalized coherence index to evaluate its coherence level. Specifically, the calculation methods for InSAR coherence and normalized coherence index are as follows: Use the registered images with acquisition dates of 20231214 and 20231226 SLC files to calculate the coherence. The specific formula is as follows: ; is the calculated coherence, and are the complex values of the corresponding pre-earthquake and post-earthquake SAR image pixels, represents the complex conjugate of, and E represents the expected value. The value range of coherence is 0 to 1, and the lower the value, the greater the ground change (severe damage in the field of building damage assessment).

[0071] In this embodiment, an improved method can also be used to calculate the coherence difference to distinguish damaged and undamaged buildings. The normalized difference between two pairs of coherence images (i.e., the pre-earthquake coherence map and the post-earthquake coherence map) is used as a parameter to determine building damage in this earthquake (the normalized coherence index is the normalized difference between two pairs of coherence images (i.e., the pre-earthquake coherence map and the post-earthquake coherence map) as a parameter to determine building damage in this earthquake). The specific formula is as follows: ; where ∆γ is the normalized coherence index, and are the coherence of the pre-earthquake interference pair and the coherence values of the pre-earthquake and post-earthquake interference pairs, respectively). Exemplarily, and are the coherence of the interference pairs on December 2, 2023 and December 14, 2023 before the event and the coherence of the interference pairs on December 14, 2023 and December 26, 2023 during the event, respectively.

[0072] (3.46) NDBI Index: Calculate the Normalized Difference Built-up Index (NDBI) for each building unit to evaluate its building density. In specific implementation, the NDBI index can be directly extracted from the NDBI index difference map produced in the aforementioned S2.

[0073] (3.47) InSAR 3D Displacement: Extract the InSAR 3D displacement data for each building unit and analyze its displacement characteristics. Specifically, the calculation methods for InSAR LOS direction and 3D displacement are as follows: Use the registered SAR images to perform the differential interferometry process to generate the displacement map in the LOS direction. Among them, for the ascending orbit data, the differential interferometry pair is generated using the post-earthquake main image 20231226 and the pre-earthquake sub-image 20231027, with a spatial baseline of 63.880 meters. For the descending orbit data, the differential interferometry pair is generated using the post-earthquake main image 20231226 and the pre-earthquake sub-image 20231202, with a spatial baseline of 118.716 m. The two-track method is used for the differential interferometry process. For any point on the interferogram, its deformation phase can be expressed as: ; In the formula, is the interferometric phase, represents the flat-earth phase, represents the topographic phase, represents the deformation phase, represents the atmospheric delay phase, represents the coherence loss noise phase or decorrelation noise phase, also includes the orbital error phase caused by inaccurate orbital parameters , is the wavelength of the radar signal, B is the satellite orbital spacing during the two observations, B is called the spatial baseline, is the parallel baseline, is the vertical baseline, R is the slant range from the radar to the target during satellite observation, and h is the elevation of this point. Assuming that the surface deformation phase at this point ignores the noise, the surface deformation can be expressed as: ; Using the error propagation law, the partial derivative of the surface deformation can be obtained and deduced as: ; As can be seen from the above, the basic principle of D-InSAR is to use the established high-precision external DEM data to obtain the differential interferometric phase information in the study area through the differential interferometry process. Subsequently, all irrelevant phases in the differential interferometric phase are removed, such as the flat ground phase , the topographic phase , the atmospheric phase and the noise phase , and finally retain the deformation phase , and then convert the obtained deformation phase into LOS deformation information. In the embodiment of the present invention, GAMMA software is used to perform interferometric processing on Sentinel-1A ascending and descending orbit data. During the interferometric processing, the topographic phase is removed using COP-DEM, and the precise orbit ephemeris provided by ESA (European Space Agency) is used to weaken the orbit error. In order to suppress the phase noise, the interferogram is subjected to 1 (azimuth) × 5 (range) multi-look processing, and the signal-to-noise ratio of the interferogram is improved through Goldstein filtering, and then the MCF (Minimum Cost Flow) phase unwrapping is completed to obtain the original interferogram. At the same time, the tropospheric delay in the filtered interferogram is corrected using GACOS software before unwrapping, and finally the phase is converted into deformation to obtain the displacement in the LOS (Line of Sight) direction of the ascending and descending orbit data. Among them, the LOS displacement and the unwrapped interferometric phase The relationship can be expressed as: ; Use MAI and DInSAR techniques to solve the three-dimensional deformation field. Obtain the pre-earthquake and post-earthquake SAR images after registration in the study area. The MAI method obtains the azimuth displacement through azimuth band splitting and interferometric measurement techniques. First, use GAMMA software to obtain the forward-looking and backward-looking images from the original SAR single-look complex (SLC) image through the azimuth co-band filtering method. For an interferometric pair, four new SLCs can be generated, including the forward-looking and backward-looking master images, and the forward-looking and backward-looking slave images. Then, the forward-looking and backward-looking pairs can be used to generate the forward-looking interferogram and the backward-looking interferogram. Subsequently, the MAI phase can be generated according to the phase difference between the forward-looking and backward-looking interferograms . In order to obtain the ascending orbit azimuth displacement, it is necessary to use the polynomial fitting method to remove the flat earth trend phase and the topographic phase from the ascending orbit MAI interferogram. Therefore, at a pixel, the ascending orbit interferometric azimuth displacement The relationship with the MAI phase is as follows.

[0074] ; where is the azimuth displacement of the ascending orbit interferometric pair, is the MAI phase, l is the effective antenna length, n is the fraction of the full aperture width, and the descending orbit interferometric pair is processed in the same way to extract the descending orbit azimuth displacement.

[0075] After geocoding the LOS displacements and along-track displacement data of ascending and descending orbits solved by the above DInSAR and resampling them into the same geographic grid, a three-dimensional displacement map is generated using the least squares method. For any point, the relationship between the three-dimensional displacement components and the LOS and along-track displacements can be expressed as: ; In the formula, is the deformation in the LOS direction calculated by DInSAR, , , are the decomposition components of the LOS deformation in the vertical, north-south, and east-west directions respectively. The subscripts a and d represent ascending and descending orbits respectively. θ is the radar incident angle; is the azimuth angle of the satellite along the flight direction. Based on the above observation value equation, the least squares algorithm is used to solve the three-dimensional displacement map of the study area.

[0076] (3.5) Feature library establishment: Summarize the above-extracted multi-scale feature data to establish a multi-scale feature library of building vector units.

[0077] (3.6) Data storage and management: Store the feature library in the geodatabase of ArcGIS for convenient subsequent query and analysis.

[0078] S4. Correlation analysis and selection between damaged buildings and the multi-scale feature library (that is, select the target multi-scale features with a correlation higher than the preset threshold with the damaged building vector units from the multi-scale features corresponding to the building vector units), and divide the training, validation, and test data sets.

[0079] In one implementation, analyze the correlation between each feature in the multi-scale feature library of building vector units and intact and damaged buildings, use the XGBoost classifier optimized by Bayesian to evaluate the importance of each feature in the data set, and select the features with a higher correlation with damaged buildings as the target multi-scale features for subsequent damaged building classification. Make training, validation, and test data sets, and increase the diversity of the data set through data augmentation techniques; Specifically, the hyperparameters of the XGBoost classifier can be optimized by Bayesian optimization using the Python programming language and related machine learning libraries (such as XGBoost, Hyperopt). The specific steps are as follows: (4.1) Define the objective function: Construct a function that accepts the hyperparameters of XGBoost as input, models and evaluates its performance metrics (such as accuracy, AUC, etc.), and returns the evaluation results.

[0080] (4.2)Select the Bayesian optimization tool: Use a Bayesian optimization library (such as Hyperopt) to automate the hyperparameter tuning and optimization process.

[0081] (4.3)Set the hyperparameter space: Define the range of hyperparameters to be optimized, including the learning rate (learning_rate), the maximum depth of the tree (max_depth), the subsample ratio (subsample), the feature sampling ratio (colsample_bytree), etc. In the above code, the space dictionary defines the search space for each hyperparameter.

[0082] (4.4)Perform Bayesian optimization: Use the Bayesian optimization tool to search within the defined hyperparameter space to find the optimal combination of hyperparameters. In the above code, the fmin function performs the Bayesian optimization process, and the best variable stores the optimized combination of hyperparameters.

[0083] (4.5)Train the final model: Use the best hyperparameters obtained from the optimization to train the XGBoost model and evaluate its performance on the test set.

[0084] (4.6)Use the XGBoost training model and obtain the importance score of each feature through its built-in feature importance evaluation function.

[0085] According to the feature importance scores, filter out the relevant features related to building damage. In the embodiments of the present invention, the LOS displacement of the descending orbit data, the LOS displacement of the ascending orbit data, the coherence and the normalized coherence index, the difference in backscattering coefficient and the correlation coefficient, the NDBI index, the entropy (H) map, the scattering angle (α) and the anisotropy (A), and the InSAR three-dimensional displacement (east-west displacement, north-south displacement, vertical displacement) are selected as the features for subsequent building damage detection. Then, the target multi-scale features within these building vector units are used to create training, validation, and test data sets, and data augmentation techniques are used to increase the diversity of the data sets.

[0086] S5. Construct a damaged building detection network based on a variational autoencoder and a multi-layer perceptron, and perform network training and inference to output the detection results of damaged buildings in the earthquake-stricken area.

[0087] The embodiments of the present invention propose a building damage detection model that combines feature extraction based on a variational autoencoder and an improved multi-layer perceptron classifier. Specifically, refer to Figure 3 the structural schematic diagram of a damaged building detection network shown, including a variational autoencoder and an improved multi-layer perceptron classifier (i.e., Figure 3In the improved MLP module), the model first uses the encoder part of the Disentangled Inferred Prior Variational Auto-Encoder (DIPVAE) model (abbreviated as DIPVAE encoder) to perform high-quality latent representation learning on the input features, achieving fine control of the latent feature distribution (that is, through the variational auto-encoder, extracting the latent space vectors of the target multi-scale features corresponding to the building vector units); subsequently, by adding an improved multi-layer perceptron classifier structure with a hybrid convolution and Transformer Token mixer and channel attention ECA, the global and local information fusion of the feature space is achieved. Among them, the input of the model is the target multi-scale features of the building unit structure, and the output is the binary classification type of building damage.

[0088] For ease of understanding, the embodiments of the present invention will separately explain the variational auto-encoder and the improved multi-layer perceptron classifier.

[0089] (5.1) Feature representation learning model constrained by DIPVAE encoder. The DIPVAE encoder aims to perform effective latent representation learning and decoupling on high-dimensional input features. Based on the conventional VAE, the DIPVAE encoder introduces a regularization term for the covariance matrix of latent space features, making the feature dimensions closer to independence and variance normalization, and integrating the Huber reconstruction loss, KL divergence constraint, and DIP regularization term (restricting the diagonal and non-diagonal elements of the covariance matrix) into the classical VAE loss, thereby achieving more reasonable reconstruction, more independent and interpretable latent representations, and higher feature discriminability, providing high-quality input features for the subsequent classifier (building damage detection).

[0090] See Figure 4 A schematic structural diagram of a DIPVAE encoder as shown. The DIPVAE encoder includes multiple hidden layers, each hidden layer is composed of multiple fully connected layers, Batch Normalization, and SiLU activation functions. Through the encoder of the fully connected network structure, the target multi-scale features are mapped to the latent space, and the parameters of the mean and log variance of the latent space vector distribution are output through two fully connected layers, and the latent distribution parameters are determined using the mean and log variance. The latent features are sampled through the reparameterization trick. During this process, the DIPVAE introduces the DIP (Disentangled InferredPrior) regularization term, and by weighting and constraining the diagonal and non-diagonal elements of the covariance matrix between latent variables, the decoupling and interpretability of features in the latent space are enhanced. At the same time, using the Huber Loss as the joint optimization of the reconstruction loss and KL divergence constraint enables the model to obtain a smoother and more robust latent representation while maintaining the compact representation of features.

[0091] (5.2) The improved multi-layer perceptron classifier is the main framework of the entire classification network. Its core function is to perform subsequent feature integration and classification decision-making on the latent features provided by DIPVAE. In the embodiments of the present invention, referring to the multi-layer perceptron network, the data processing and feature extraction strategy changes from directly processing the original input to first using DIPVAE for latent feature representation learning. At the same time, token mixing and channel mixing are organically combined. The Token Mixer layer is used to perform local and global feature fusion in the token dimension, and then the feature is enhanced in the channel dimension through the channel Mixer layer module.

[0092] Compared with the original multi-layer perceptron module, the improved multi-layer perceptron classifier uses DIPVAE to perform feature extraction and latent representation learning on the input features (such as data representations from CSV). High-level abstract representations of the features have been performed before entering the classifier. At the same time, in the token mixing stage, the Token Mixer layer is used to replace the pure MLP to achieve the fusion of local convolutional mode extraction and global attention. In the channel mixing stage, the channel Mixer layer module and ECA channel attention are used to replace the simple MLP, making feature selection and emphasis more targeted. Therefore, the improved MLP_Mixer network proposed in the embodiments of the present invention flexibly integrates convolution, Transformer, and channel attention, and realizes the efficient fusion of multi-dimensional feature information with lower complexity, thereby providing a more discriminative feature representation for the final binary classification task (building damage or intact).

[0093] Please continue to refer to Figure 3 , Figure 3 which shows that the improved multi-layer perceptron classifier includes an input linear mapping layer (i.e., Figure 3 the fully connected layer at the beginning of the improved MLP module in Figure 3 ), a multi-layer Token Mixer network, a multi-layer channel Mixer network, a pooling and output layer (i.e.,

[0094] the global pooling layer, Dropout layer, fully connected layer, and Sigmoid layer in (a) Through the input linear mapping layer, map the latent space vector to the specified token dimension. For example, map the input feature to the specified token dimension (such as 128 dimensions) to obtain the initial token representation.

[0095] (b)Through the multi-layer Token Mixer network, local convolution and global attention fusion are alternately performed on the latent space vectors in the token dimension to construct rich token-level representations. In specific implementation, the multi-layer Token Mixer network performs local and global information fusion on the token sequence, thereby enriching the context semantics of the token-level features.

[0096] The Token Mixer network organically combines the convolution operation and the Transformer attention mechanism to achieve local and global information fusion of features in the token dimension. By using a hybrid method of depthwise separable convolution (capturing local feature patterns) and multi-head self-attention (establishing global dependencies) in the same layer, the Token Mixer network innovatively covers both spatial neighborhood features and cross-token relationship information, improving the expressive ability of the feature sequence.

[0097] See Figure 5 Refer to the structural schematic diagram of a Token Mixer network shown in

[0098] Furthermore, please continue to refer to Figure 5, the improved Transformer encoder provided by the embodiments of the present invention includes a multi-head attention layer (Multi-Head_Self-Attention), a layer normalization layer (LayerNorm), a fully connected layer (Fullyconnected layer), an activation function layer (GELU), another fully connected layer (Fully connected layer), and another layer normalization layer (LayerNorm).

[0099] In the Token Mixer Layer, in order to enhance the transitivity of feature representation and the optimization ability of the network, two forms of residual connection structures are designed. The first form of the residual connection structure is as follows: a residual connection is made between the input end and the output end of the multi-head attention layer. Specifically, in the Token Mixe layer, after the input features are processed by depthwise convolution and pointwise convolution, they are fused with the initial input features through a residual connection. This residual path allows the model to capture spatial correlations while retaining the global feature representation of the input, alleviating the problem of gradient disappearance caused by excessive information transformation. The second form of the residual connection structure is as follows: a residual connection is made between the input end of the multi-head attention layer and the output end of another fully connected layer. Specifically, after the LayerNorm operation, the feature output of the Transformer Encoder and the input features of the Transformer Encoder are superimposed through a residual connection. The introduction of this residual connection can promote the model to perform context-related adaptive enhancement on the input features, while ensuring the gradient smoothness during the training process, further improving the training stability and the expressive ability of the model.

[0100] (c) Through the multi-layer channel Mixer network, feedforward enhancement and attention selection are performed on the feature vectors output by the Token Mixer network in the channel dimension to improve the discriminability and stability of the channel space. In specific implementation, several layers of channel Mixer networks further strengthen and screen features in the channel dimension.

[0101] The channel Mixer network performs non-linear transformation and enhancement in the channel dimension, and improves the discriminability of feature representation through a feedforward fully connected structure and a channel attention mechanism. And the residual structure is fused with the ECA (Efficient ChannelAttention) channel attention, so that while improving the representation ability, the stability of the original distribution is not damaged, and the gradient disappearance and overfitting can be effectively alleviated in the multi-layer stacking. See Figure 6Schematic diagram of the structure of a channel Mixer network. Each channel Mixer network includes a layer normalization layer, two layers of feed-forward fully connected networks (including GELU activation and Dropout), and an ECA channel attention layer. There is a residual connection between the input end of the layer normalization layer and the output end of the ECA channel attention layer. Among them, LayerNorm is used to: normalize the input features to ensure the stability of the training process; the feed-forward network (two fully connected layers + GELU activation + Dropout) is used to: perform non-linear mapping and enhancement on the features to improve the abstract ability of the features; ECA channel attention (optional) is used to: selectively enhance the influence of key feature channels in the channel dimension; the residual connection is used to: add the input features and the processed features to maintain the continuity of the feature distribution and the stability of model training.

[0102] Among them, the ECA channel attention layer is a lightweight and efficient channel attention mechanism. It captures the local interaction relationships between channels by performing one-dimensional convolution operations on the features in the channel dimension (instead of complex fully connected layer mappings), so that key feature channels can be highlighted without excessive complex operations with too many additional parameters, redundant or noisy features can be weakened, and more efficient feature expression can be achieved. For the building damage detection task, the features often come from multiple data sources and feature types (such as coherence, backscatter difference, deformation, etc.). The ECA channel attention layer can highlight the most discriminative features at the channel level, strengthen the feature channels strongly related to the damage pattern, and at the same time suppress the channels that contribute less to the decision-making, thereby improving the sensitivity and robustness of the model to the details of building damage. For specific details, please refer to Figure 7 the schematic diagram of the structure of an ECA channel attention layer shown.

[0103] On this basis, first normalize the input features through LayerNorm, and then perform non-linear mapping and feature enhancement through the feed-forward fully connected network; the ECA channel attention layer weights and emphasizes the channel information at this stage; finally, add the input features and the processed features through the residual connection, so as to improve the feature expressiveness while maintaining the stability of model training and the continuity of the feature distribution. The channel Mixer module effectively mixes the information between channels and cooperates with the token mixer to jointly achieve the all-round interaction of features in multiple dimensions.

[0104] (d) Through pooling and the output layer, generate the binary classification type of building damage based on the feature vector output by the channel Mixer network. In specific implementation, aggregate the processed features (such as average pooling) and map them to a single output node through a linear layer to achieve the decision output of the binary classification task.

[0105] On this basis, carry out the training of the damage building classification network.

[0106] All experiments were conducted using PyTorch on a device equipped with a high-performance GPU (CUDA) or CPU (depending on availability). Before training, the input features were standardized using StandardScaler to ensure numerical stability and facilitate model convergence. For DIPVAE and the classification model, the training data was input into the model in small batches (e.g., batch size = 32), enabling efficient stochastic gradient descent updates.

[0107] For the DIPVAE encoder, the embodiments of the present invention employed the AdamW optimizer with a learning rate of 1×10e5 and a total of 100 training epochs. First, the DIPVAE encoder was trained to learn the disentangled latent representation. Then, a multi-layer perceptron-based classifier was trained in combination with the parameters of the encoder, also using the AdamW optimizer with a learning rate of 1×10e4. The embodiments of the present invention adopted a cosine annealing warm restart or a step scheduler to dynamically adjust the learning rate, mitigate overfitting, and improve the model's generalization ability. Intermediate model checkpoints were saved after each training to ensure result reproducibility and monitoring.

[0108] In addition, the embodiments of the present invention propose to determine the joint loss value according to the binary classification type of building damage output by the damaged building classification network and the label carried by the building vector unit, so as to optimize the feature decoupling and classification performance simultaneously. Among them, the joint loss value includes the feature decoupling loss value and the classification accuracy loss value. The feature decoupling loss value, which is also the loss of the DIPVAE encoder, consists of the feature reconstruction loss value, the latent KL divergence loss value, and the latent regularization loss value. The calculation process of the joint loss value is as follows: (I) Determine the feature reconstruction loss value according to the target multi-scale features corresponding to the building vector unit and the reconstruction features generated during the damage detection process by the damaged building classification network; and, determine the latent KL divergence loss value according to the feature mean and feature variance of each latent dimension in the latent space vector output by the variational autoencoder; and, determine the latent regularization loss value according to the feature variance of each latent dimension in the latent space vector output by the variational autoencoder and the feature covariance between any two latent dimensions.

[0109] In one example, in order to ensure that the reconstruction features generated during the damage detection process by the damaged building classification network are similar to the original target multi-scale features the Huber loss is used, and the specific calculation formula for the feature reconstruction loss value ; In the formula, is the threshold for determining the transition between the quadratic region and the linear region of the Huber loss.

[0110] In one example, the KL divergence restricts the latent distribution to be close to the prior distribution p(z) (assumed to be Gaussian), and the specific calculation formula for the latent KL divergence loss value is as follows: ; where Z is the latent space vector, d is the dimension of the latent space vector, and are the characteristic mean and characteristic variance of the i-th latent dimension in the latent space vector, respectively.

[0111] In one example, DIP regularization enforces decoupling by penalizing the latent variable covariance matrix, and the calculation formula for the regularization loss value is as follows: ; where is the covariance between the i-th and j-th latent dimensions in the latent space vector, Var( ) is the variance of the i-th latent dimension in the latent space vector, and are hyperparameters that control the off-diagonal and diagonal penalties.

[0112] (II) Weighted fusion of the feature reconstruction loss value, the latent KL divergence loss value, and the latent regularization loss value is performed to obtain the feature decoupling loss value. Among them, the feature decoupling loss value has the following calculation formula: ; where is the weighted factor of the KL divergence loss value .

[0113] (III) According to the binary classification type of building damage corresponding to the building vector unit and the label carried by the building vector unit, the classification accuracy loss value is determined. The classification accuracy loss value can also be called the binary cross-entropy loss value and is used to evaluate the classification performance. The calculation formula for the classification accuracy loss value is as follows: ; where is the binary classification type of building damage of the -th building vector unit, is the sigmoid function, N is the number of building vector units, is the ground truth label of the -th building vector unit.

[0114] (IV) The characteristic decoupling loss value and the classification accuracy loss value are weighted and fused to obtain a combined loss value. The calculation formula of the combined loss value is as follows: ; In the formula: is the characteristic decoupling loss value, which includes the feature reconstruction loss value , the latent KL divergence loss value , the latent regularization loss value , is the binary cross-entropy loss for the binary classification task (i.e., the classification accuracy loss value). is a hyperparameter used to balance the characteristic decoupling ( ) and the classification accuracy ( ). This formula ensures that the learned latent representation is disentangled and interpretable, while optimizing the classification accuracy.

[0115] Finally, the trained model is applied to detect the damaged buildings in the study area, realizing the automatic identification and detection of the damaged buildings in the study area. In practical applications, although there are certain differences between the classification results of the damaged buildings obtained by using the method provided in the embodiments of the present invention and the ground truth, the overall performance proves the potential of this method in extracting damaged buildings in earthquake-stricken areas (such as Jishishan County). These findings indicate that the model has successfully captured the patterns of building damage caused by earthquakes and provided reliable classification results for practical post-earthquake building damage detection applications.

[0116] In summary, this method integrates the encoding part of the variational encoder and the multi-layer perceptron neural network, effectively extracts the multi-dimensional features of damaged buildings by comprehensively using multi-source remote sensing data, and significantly improves the performance of building damage classification. Based on this, the present invention provides an efficient damaged building detection scheme for earthquake-stricken areas. Through an automated damaged building detection process, the earthquake damage information of buildings can be quickly obtained, providing timely data support for post-disaster emergency rescue and building reconstruction. In addition, the embodiments of the present invention show the advantages of high precision, multi-scale and high efficiency in building damage detection, which are of great significance for quickly obtaining building earthquake damage information, post-disaster emergency rescue and building reconstruction, and have important application values in aspects such as post-disaster emergency response and reconstruction planning, and can provide strong support for disaster management and decision-making. Further, through the automated damaged building detection process, the earthquake damage information of buildings can be quickly obtained, and it is expected to be further promoted in the rapid assessment application of earthquake building damage, providing more scientific data support and decision-making basis for disaster risk management and emergency response.

[0117] On the basis of the foregoing embodiments, the embodiments of the present invention provide a building damage detection device that couples multi-features of SAR images and optical images. SeeFigure 8 Schematic diagram of the structure of a building damage detection device that couples multi - features of SAR images and optical images. The device mainly includes the following parts: An image acquisition module 802, configured to acquire SAR images, optical images, and UAV images corresponding to the study area in the pre - disaster scenario and the post - disaster scenario respectively; An image processing module 804, configured to construct a change detection image set of the study area based on the SAR image and the optical image, and construct a vector data set corresponding to the study area based on the optical image and the UAV image. The vector data set includes each building vector unit within the study area; A feature library construction module 806, configured to extract multi - scale features corresponding to the building vector units from the change detection image set according to the vector data set to construct a multi - scale feature library. The multi - scale features include one or more of H / A / Alpha polarization decomposition features, backscattering coefficient correlation, backscattering coefficient difference, InSAR coherence and normalized coherence index, NDBI index, and InSAR three - dimensional displacement; A network training module 808, configured to train a damaged building classification network using the multi - scale feature library. The trained damaged building classification network is used to detect damage in the study area. The damaged building classification network includes a variational auto - encoder and an improved multi - layer perceptron classifier. The improved multi - layer perceptron classifier is configured to alternately perform local convolution and global attention fusion on the input latent space vector in the token dimension and perform feed - forward enhancement and attention selection in the channel dimension.

[0118] The building damage detection device that couples multi - features of SAR images and optical images provided by the embodiments of the present invention collects SAR images, optical images, and UAV images corresponding to the study area in the pre - disaster scenario and the post - disaster scenario respectively, and constructs a multi - scale feature library containing multi - scale features corresponding to each building vector unit on this basis to improve problems such as single data source and strong limitations in feature extraction in the prior art. In addition, the damaged building classification network is trained using the multi - scale feature library. The network includes a variational auto - encoder and an improved multi - layer perceptron classifier. The trained damaged building classification network is used to detect damage in the study area. Through this network, problems such as weak model generalization ability, lack of real - time performance and robustness in the prior art can be effectively improved, thereby providing a high - precision and multi - scale solution for post - disaster building damage assessment.

[0119] In one implementation, the building vector units are divided into damaged building vector units and intact building vector units; the network training module 808 is specifically configured to: From the multi-scale features corresponding to the building vector units, filter out the target multi-scale features whose correlation with the damaged building vector units is higher than a preset threshold; Input the target multi-scale features corresponding to the building vector units into the damaged building classification network to obtain the binary classification type of building damage output by the damaged building classification network, and determine the joint loss value in combination with the label carried by the building vector units. The joint loss value includes the feature decoupling loss value and the classification accuracy loss value; Use the joint loss value to adjust the network parameters of the damaged building classification network.

[0120] In one implementation, the improved multi-layer perceptron classifier includes an input linear mapping layer, multiple layers of TokenMixer networks, multiple layers of channel Mixer networks, and a pooling and output layer; the network training module 808 is specifically used for: Extract the latent space vector of the target multi-scale features corresponding to the building vector units through a variational autoencoder; Map the latent space vector to a specified token dimension through the input linear mapping layer; Through multiple layers of Token Mixer networks, alternately perform local convolution and global attention fusion on the latent space vector in the token dimension; Through multiple layers of channel Mixer networks, perform feed-forward enhancement and attention selection on the feature vector output by the Token Mixer network in the channel dimension; Generate the binary classification type of building damage based on the feature vector output by the channel Mixer network through the pooling and output layer.

[0121] In one implementation, each Token Mixer network includes a depthwise separable convolutional layer, a point convolutional layer, an improved Transformer encoder, and a layer normalization layer; the network training module 808 is specifically used for: For any layer of Token Mixer network, use this Token Mixer network to perform the following operations on the input features: Through the depthwise separable convolutional layer and the point convolutional layer, perform local convolution on the input features in the token dimension to extract token-level local features; Through the improved Transformer encoder, perform global attention fusion on the input features in the token dimension to extract token-level global features; Through the layer normalization layer, perform feature fusion on the token-level local features and the token-level global features.

[0122] In one embodiment, the improved Transformer encoder includes a multi-head attention layer, a layer normalization layer, a fully-connected layer, an activation function layer, another fully-connected layer, and another layer normalization layer; The improved Transformer encoder adopts two forms of residual connection structures; among them, the first form of the residual connection structure is as follows: a residual connection is made between the input end and the output end of the multi-head attention layer; the second form of the residual connection structure is as follows: a residual connection is made between the input end of the multi-head attention layer and the output end of another fully-connected layer.

[0123] In one embodiment, each channel Mixer network includes a layer normalization layer, a feed-forward network, and an ECA channel attention layer, and a residual connection is made between the input end of the layer normalization layer and the output end of the ECA channel attention layer.

[0124] In one embodiment, the network training module 808 is specifically configured to: Determine a feature reconstruction loss value according to the target multi-scale features corresponding to the building vector unit and the reconstruction features generated during the damage detection by the damaged building classification network; and determine a latent KL divergence loss value according to the feature mean and feature variance of each latent dimension in the latent space vector output by the variational autoencoder; and determine a latent regularization loss value according to the feature variance of each latent dimension in the latent space vector output by the variational autoencoder and the feature covariance between any two latent dimensions; Perform weighted fusion on the feature reconstruction loss value, the latent KL divergence loss value, and the latent regularization loss value to obtain a feature decoupling loss value; Determine a classification accuracy loss value according to the binary classification type of the building damage corresponding to the building vector unit and the label carried by the building vector unit; Perform weighted fusion on the feature decoupling loss value and the classification accuracy loss value to obtain a joint loss value.

[0125] The device provided by the embodiments of the present invention has the same implementation principle and the same technical effects as those of the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in the device embodiments, reference may be made to the corresponding contents in the foregoing method embodiments.

[0126] The embodiments of the present invention provide an electronic device. Specifically, the electronic device includes a processor and a storage device; a computer program is stored on the storage device, and the computer program executes the method according to any one of the above-mentioned embodiments when being run by the processor.

[0127] Figure 9A structural schematic diagram of an electronic device provided by an embodiment of the present invention. The electronic device 100 includes: a processor 90, a memory 91, a bus 92, and a communication interface 93. The processor 90, the communication interface 93, and the memory 91 are connected through the bus 92. The processor 90 is configured to execute an executable module stored in the memory 91, such as a computer program.

[0128] Among them, the memory 91 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. Through at least one communication interface 93 (which can be wired or wireless), a communication connection is realized between this system network element and at least one other network element, and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used.

[0129] The bus 92 may be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 9 only a bidirectional arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0130] Among them, the memory 91 is used to store a program. After receiving an execution instruction, the processor 90 executes the program. The method executed by the device defined by the flow process disclosed in any embodiment of the foregoing embodiments of the present invention can be applied to the processor 90 or implemented by the processor 90.

[0131] The processor 90 may be an integrated circuit chip with the ability to process signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 90 or the instructions in the form of software. The above-mentioned processor 90 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field-programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present invention can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by the combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 91, and the processor 90 reads the information in the memory 91 and combines its hardware to complete the steps of the above method.

[0132] The computer program product of the readable storage medium provided by the embodiments of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For the specific implementation, reference can be made to the foregoing method embodiments and will not be elaborated here.

[0133] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0134] Finally, it should be noted that the above-mentioned embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting them. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A building damage detection method coupling SAR image and optical image multi-features, characterized in that: include: Obtain SAR images, optical images and UAV images of the study area in pre-disaster and post-disaster scenarios; constructing a change detection image set of the study area based on the SAR image and the optical image, and constructing a vector data set corresponding to the study area based on the optical image and the drone image, wherein the vector data set includes each building vector unit in the study area; According to the vector data set, extracting multi-scale features corresponding to the building vector unit from the change detection image set to construct a multi-scale feature library, wherein the multi-scale features include one or more of H / A / Alpha polarization decomposition features, backscatter coefficient correlation, backscatter coefficient difference, InSAR coherence and normalized coherence index, NDBI index, and InSAR three-dimensional displacement; The damaged building classification network is trained using the multi-scale feature library. The trained damaged building classification network is used to detect damage in the study area. The damaged building classification network includes a variational autoencoder and an improved multi-layer perceptron classifier. The improved multi-layer perceptron classifier is used to alternately perform local convolution and global attention fusion on the token dimension for the input latent space vector, and perform feedforward enhancement and attention selection on the channel dimension.

2. The building damage detection method based on coupling SAR image and optical image multi-features according to claim 1 is characterized in that: The building vector units are divided into damaged building vector units and intact building vector units; The damaged building classification network is trained using the multi-scale feature library, including: Filtering out, from the multi-scale features corresponding to the building vector units, target multi-scale features whose correlation with the damaged building vector units is higher than a preset threshold; Inputting the target multi-scale features corresponding to the building vector units into a damaged building classification network, obtaining a binary classification type of building damage output by the damaged building classification network, and determining a joint loss value in combination with the labels carried by the building vector units, wherein the joint loss value includes a feature decoupling loss value and a classification accuracy loss value; The joint loss value is used to adjust the network parameters of the damaged building classification network.

3. The building damage detection method based on coupling SAR image and optical image multi-features according to claim 2 is characterized in that: The improved multi-layer perceptron classifier includes an input linear mapping layer, a multi-layer Token Mixer network, a multi-layer channel Mixer network, a pooling and output layer; the target multi-scale features corresponding to the building vector unit are input into the damaged building classification network to obtain the building damage binary classification type output by the damaged building classification network, including: Extracting the latent space vector of the target multi-scale feature corresponding to the building vector unit through the variational autoencoder; Mapping the latent space vector to a specified token dimension through the input linear mapping layer; By using multiple layers of the Token Mixer network, alternately performing local convolution and global attention fusion on the latent space vector in the token dimension; Through the multi-layer channel mixer network, feed-forward enhancement and attention selection are performed on the feature vector output by the token mixer network in the channel dimension; Through the pooling and output layer, a binary classification type of building damage is generated based on the feature vector output by the channel Mixer network.

4. The building damage detection method based on coupling SAR image and optical image multi-features according to claim 3 is characterized in that: Each of the Token Mixer networks includes a depth-separable convolution layer, a point convolution layer, an improved Transformer encoder, and a layer normalization layer; through multiple layers of the Token Mixer network, local convolution and global attention fusion are alternately performed on the latent space vector in the token dimension, including: For any layer of the Token Mixer network, the Token Mixer network is used to perform the following operations on the input features: Performing local convolution on the input features in the token dimension through the depth-wise separable convolution layer and the point-wise convolution layer to extract token-level local features; By improving the Transformer encoder, global attention fusion is performed on the input features in the token dimension to extract token-level global features; The token-level local features and the token-level global features are fused through the layer standardization layer.

5. The building damage detection method of coupling SAR image and optical image multi-features according to claim 4 is characterized in that: The improved Transformer encoder includes a multi-head attention layer, a layer normalization layer, a fully connected layer, an activation function layer, another fully connected layer and another normalization layer; The improved Transformer encoder adopts two forms of residual connection structures; among them, the first form of the residual connection structure is as follows: a residual connection between the input end of the multi-head attention layer and the output end of the multi-head attention layer; the second form of the residual connection structure is as follows: a residual connection between the input end of the multi-head attention layer and the output end of another fully connected layer.

6. The building damage detection method based on coupling SAR image and optical image multi-features according to claim 3 is characterized in that: Each of the channel Mixer networks includes a layer normalization layer, a feedforward network and an ECA channel attention layer, and a residual connection is formed between an input end of the layer normalization layer and an output end of the ECA channel attention layer.

7. The building damage detection method of coupling SAR image and optical image multi-features according to claim 2 is characterized in that: The joint loss value is determined in combination with the label carried by the building vector unit, including: Determine a feature reconstruction loss value according to the target multi-scale features corresponding to the building vector unit and the reconstruction features generated by the damaged building classification network during the damage detection process; and determine a potential KL divergence loss value according to the feature mean and feature variance of each potential dimension in the latent space vector output by the variational autoencoder; and determine a potential regularization loss value according to the feature variance of each potential dimension in the latent space vector output by the variational autoencoder and the feature covariance between any two of the latent dimensions; The feature reconstruction loss value, the potential KL divergence loss value, and the potential regularization loss value are weightedly fused to obtain the feature decoupling loss value; Determine a classification accuracy loss value according to the binary classification type of building damage corresponding to the building vector unit and the label carried by the building vector unit; The feature decoupling loss value and the classification accuracy loss value are weightedly fused to obtain a joint loss value.

8. A building damage detection device that couples SAR images and optical images with multiple features, characterized in that: include: Image acquisition module, used to obtain SAR images, optical images and UAV images of the study area in pre-disaster and post-disaster scenarios respectively; An image processing module, configured to construct a change detection image set of the study area based on the SAR image and the optical image, and to construct a vector data set corresponding to the study area based on the optical image and the UAV image, wherein the vector data set includes each building vector unit in the study area; A feature library construction module is used to extract multi-scale features corresponding to the building vector unit from the change detection image set according to the vector data set to construct a multi-scale feature library, wherein the multi-scale features include one or more of H / A / Alpha polarization decomposition features, backscatter coefficient correlation, backscatter coefficient difference, InSAR coherence and normalized coherence index, NDBI index, and InSAR three-dimensional displacement; A network training module is used to train a damaged building classification network using the multi-scale feature library. The trained damaged building classification network is used to perform damage detection on the study area. The damaged building classification network includes a variational autoencoder and an improved multi-layer perceptron classifier. The improved multi-layer perceptron classifier is used to alternately perform local convolution and global attention fusion on the token dimension for an input latent space vector, and perform feedforward enhancement and attention selection on the channel dimension.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 7.

Images