A method, apparatus, device, and readable storage medium for detecting changes in a building

Abstract

The application relates to the field of remote sensing application technology, and specifically provides a method, device and equipment for detecting building changes and a readable storage medium, the method comprising the following steps: extracting multi-scale features of two target remote sensing images by two CNN encoders respectively to obtain two groups of multi-scale features; extracting a difference enhancement map of the two target remote sensing images by a feature difference module; extracting a difference feature of the difference enhancement map by a VIT encoder; respectively fusing the difference feature and one group of multi-scale features in the two groups of multi-scale features by the two CNN encoders to obtain two groups of fusion features; and extracting and interacting part of the features in the two groups of fusion features by a cross-branch feature interaction module and a segmentation head to obtain a building change feature. The method can accurately detect the process of building changes.

Description

Technical Field

[0001] This application relates to the field of remote sensing application technology, and more specifically, to a method, apparatus, device, and readable storage medium for detecting changes in buildings. Background Technology

[0002] With the acceleration of urbanization and the diversification of land resource utilization, the dynamic monitoring of building changes has become an important requirement in fields such as urban planning, disaster assessment, and environmental management. Traditional remote sensing image building change detection methods typically rely on hand-designed feature extraction algorithms (such as texture analysis, spectral differences, etc.) and rule-based classification strategies.

[0003] However, these methods have significant limitations when dealing with complex scenarios (such as diverse building shapes, shadow interference, and sensor noise), and are difficult to adapt to the processing needs of remote sensing data under different resolutions and imaging conditions. Furthermore, traditional methods often require a large number of manually labeled samples for model training and have weak generalization ability for small sample data, resulting in detection accuracy and efficiency that fail to meet practical application requirements. The rapid development of deep learning technology has provided new solutions for remote sensing image processing. Convolutional Neural Networks (CNNs), due to their powerful feature learning capabilities, are widely used in semantic segmentation and target detection tasks of remote sensing images. For example, CNNs can effectively capture the spatial neighborhood relationships and contextual information of buildings by processing multi-scale features in parallel. However, feature extraction at a single scale is difficult to fully characterize their spatial distribution characteristics. Alignment bias across different periods of images: Due to changes in satellite revisit cycles, imaging angles, and lighting conditions, there may be significant geometric and spectral differences between two periods of remote sensing images, leading to difficulties in feature matching. Class imbalance problem: In real-world scenarios, the pixel proportion of the "unchanged" category is usually much higher than that of the "new" or "reduced" categories, and traditional segmentation models are prone to bias due to uneven class distribution. Small sample size makes it difficult for traditional models to learn robust feature representations with limited data: the amount of remote sensing data in some areas is limited.

[0004] Therefore, how to accurately detect the process of building changes is a technical problem that needs to be solved. Summary of the Invention

[0005] The purpose of this application is to provide a method for detecting changes in buildings. The technical solution of this application can achieve the effect of accurately detecting the process of changes in buildings.

[0006] In a first aspect, embodiments of this application provide a method for detecting building changes, applied to a building detection model, comprising: extracting multi-scale features from two target remote sensing images of two periods using two CNN encoders to obtain two sets of multi-scale features; extracting a difference enhancement map of the two target remote sensing images using a feature difference module; extracting difference features from the difference enhancement map using a VIT encoder; fusing the difference features and one set of multi-scale features from the two sets of multi-scale features using two CNN encoders to obtain two sets of fused features; and extracting and interacting with some features from the two sets of fused features using a cross-branch feature interaction module and a segmentation head to obtain building change features.

[0007] In the above embodiments, this application extracts multi-scale features from two sets of remote sensing images through two parallel convolutional branches, extracts difference features between the two sets of remote sensing images through a feature difference module and a VIT encoder, and finally extracts and interacts with some features from the two sets of fused features through a cross-branch feature interaction module and a segmentation head to obtain building change features. Through the synergistic effect of the pre-trained building detection model and the dual-branch structure, high-precision detection of building changes can be achieved with limited samples, while also considering computational efficiency. This achieves the effect of accurately detecting the process of building changes.

[0008] In some embodiments, the building detection model includes: two CNN encoders, a VIT encoder, a feature difference module, a cross-branch feature interaction module, and a segmentation head; the two CNN encoders each include a neck, a multi-layer residual connection block, and a multi-layer feature fusion module; the VIT encoder includes a block coding layer and a multi-layer coding layer.

[0009] In the above embodiments, the building detection model mechanism can achieve dual-branch extraction of multi-scale and differential features from different remote sensing images, and finally achieve the effect of building change detection by fusing multi-scale and differential features.

[0010] In some embodiments, two sets of multi-scale features are obtained by extracting multi-scale features of two target remote sensing images from two different CNN encoders, including: extracting multi-scale features of two target remote sensing images from two different CNN encoders through the neck and residual connection blocks of the two CNN encoders to obtain two sets of multi-scale features.

[0011] Two sets of fused features are obtained by fusing differential features and one set of multi-scale features from two sets of multi-scale features using two CNN encoders. This includes fusing differential features and one set of multi-scale features from two sets of multi-scale features using the multi-layer feature fusion modules of the two CNN encoders.

[0012] In the above embodiments, this application can accurately detect changes in buildings by extracting and fusing multi-scale features and difference features from two favorable images.

[0013] In some embodiments, the building detection model further includes: a low-rank fine-tuning module; extracting difference features of the difference enhancement map through a VIT encoder, including: extracting difference features of the difference enhancement map through a block coding layer, a multi-layer coding layer, and a low-rank fine-tuning module inserted in the multi-layer coding layer.

[0014] In the above embodiments, the low-rank fine-tuning module of the building detection model can accurately extract the difference features between two remote sensing images.

[0015] In some embodiments, before extracting multi-scale features from two target remote sensing images at two different times using two CNN encoders to obtain two sets of multi-scale features, the method further includes: inputting training data composed of remote sensing images taken at different times and feature difference annotations into the basic VIT encoder to obtain feature differences; comparing the feature differences with the feature differences corresponding to the feature difference annotations to obtain comparison results; and adjusting the parameters of the low-rank fine-tuning module according to the comparison results to obtain the VIT encoder.

[0016] In the above embodiments, this application trains the VIT encoder using remote sensing images taken at different times and image annotations, which can accurately extract the difference features of different remote sensing images.

[0017] In some embodiments, two sets of multi-scale features are obtained by extracting multi-scale features from two target remote sensing images using two CNN encoders, including: extracting local detail features and global context features from two target remote sensing images using two CNN encoders, and adjusting the local detail features and global context features by adjusting the convolution stride and dilated convolution parameters to obtain two sets of multi-scale features.

[0018] In the above embodiments, this application extracts local detail features and global context features from two remote sensing images through two parallel convolutional branches, and the extracted multi-scale features can be fused and injected into local detail features and global context features.

[0019] In some embodiments, building change features are obtained by extracting and interacting some features from two sets of fused features through a cross-branch feature interaction module and a segmentation head. This includes: interacting some features from two sets of fused features through the dual attention mechanism of the cross-branch feature interaction module to obtain interactive features; and obtaining building change features through multiple convolution operations and upsampling operations on the interactive features by the segmentation head.

[0020] In the above embodiments, this application can accurately obtain the final building price change features by performing feature interaction, convolution operations, and upsampling through a cross-branch feature interaction module and a segmentation head.

[0021] Secondly, embodiments of this application provide an apparatus for detecting changes in a building, comprising:

[0022] The first extraction module is used to extract multi-scale features from two phases of remote sensing images of the target through two CNN encoders, resulting in two sets of multi-scale features.

[0023] The second extraction module is used to extract the difference enhancement map of the target remote sensing images of the two periods through the feature difference module;

[0024] The third extraction module is used to extract the differential features of the differential enhancement map through the VIT encoder;

[0025] The fusion module is used to fuse the difference features and one set of multi-scale features from two sets of multi-scale features through two CNN encoders to obtain two sets of fused features.

[0026] The fourth extraction module is used to extract and interact with some features from the two sets of fused features through the cross-branch feature interaction module and the segmentation head to obtain building change features.

[0027] Optional building inspection models include:

[0028] Two CNN encoders, a VIT encoder, a feature difference module, a cross-branch feature interaction module, and a segmentation head;

[0029] The two CNN encoders each include a neck section, a multi-layer residual connection block, and a multi-layer feature fusion module.

[0030] The VIT encoder includes a block coding layer and a multi-layer coding layer.

[0031] Optionally, the fusion module is specifically used for:

[0032] Multi-scale features of the target remote sensing images from two periods were extracted by using the neck and residual connection blocks of two CNN encoders, resulting in two sets of multi-scale features.

[0033] By fusing the difference features and one set of multi-scale features from two sets of multi-scale features using two CNN encoders, two sets of fused features are obtained, including:

[0034] By fusing the differential features and one set of multi-scale features from two sets of multi-scale features through the multi-layer feature fusion modules of two CNN encoders, two sets of fused features are obtained.

[0035] Optional building inspection models also include:

[0036] Low-rank fine-tuning module;

[0037] The third extraction module is specifically used for:

[0038] Discrete features of the discrepancy enhancement map are extracted through block coding layers, multi-layer coding layers, and low-rank fine-tuning modules inserted in multi-layer coding layers.

[0039] Optionally, the device further includes:

[0040] The training module is used to input training data consisting of remote sensing images taken at different times and feature difference annotations into the basic VIT encoder before the first extraction module extracts multi-scale features of the target remote sensing images of the two periods through two CNN encoders to obtain two sets of multi-scale features, in order to obtain feature differences.

[0041] By comparing the feature differences with the feature differences labeled with the feature differences, the comparison results are obtained;

[0042] Based on the comparison results, the parameters of the low-rank fine-tuning module are adjusted to obtain the VIT encoder.

[0043] Optionally, the first extraction module is specifically used for:

[0044] Two CNN encoders were used to extract local detail features and global context features from the two target remote sensing images, respectively. The local detail features and global context features were adjusted by adjusting the convolution stride and dilated convolution parameters to obtain two sets of multi-scale features.

[0045] Optionally, the fusion module is specifically used for:

[0046] Interactive features are obtained by using the dual attention mechanism of the cross-branch feature interaction module to interact with some features in the two sets of fused features.

[0047] The building variation features are obtained by performing multiple convolution and upsampling operations on the interactive features using the segmentation head.

[0048] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the method provided in the first aspect above are performed.

[0049] Fourthly, embodiments of this application provide a readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method provided in the first aspect above.

[0050] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. Attached Figure Description

[0051] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A flowchart illustrating a method for detecting changes in a building, as provided in this application embodiment;

[0053] Figure 2 A schematic diagram of a self-attention structure for inserting a low-rank fine-tuning module is provided in an embodiment of this application;

[0054] Figure 3 A schematic diagram of a feature fusion module provided in an embodiment of this application;

[0055] Figure 4 A schematic diagram illustrating an implementation method for detecting changes in a building, provided in an embodiment of this application;

[0056] Figure 5 A schematic block diagram of a device for detecting changes in a building, provided as an embodiment of this application;

[0057] Figure 6 This is a schematic block diagram of a device for detecting changes in a building, provided as an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0059] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0060] First, some of the terms used in the embodiments of this application will be explained to facilitate understanding by those skilled in the art.

[0061] The Vision Transformer (ViT) encoder is a model that applies the Transformer architecture to computer vision tasks, achieving global feature modeling by processing image patches. Its core lies in segmenting the image into fixed-size patches and capturing the relationships between these patches through a self-attention mechanism, thereby overcoming the limitations of the local receptive field in traditional convolutional neural networks.

[0062] CNN usually refers to Convolutional Neural Network, which is a core model in deep learning specifically designed to process grid-structured data such as images and videos.

[0063] Difference enhancement maps (usually referring to difference map enhancement techniques) are a technique in image processing and computer vision that aims to enhance the visual features or semantic information of specific regions by comparing the differences between different images, thereby improving the recognition quality or analysis efficiency of images.

[0064] This application is applied to the scenario of building change detection. Specifically, the scenario involves extracting multi-scale features and differential features from different remote sensing images through the multi-branch structure of the building detection model, and finally fusing the differential features and a set of multi-scale features to obtain the building change detection results.

[0065] With the acceleration of urbanization and the diversification of land resource utilization, monitoring the dynamic changes of buildings has become an important requirement in fields such as urban planning, disaster assessment, and environmental management. Traditional methods for detecting building changes in remote sensing images typically rely on manually designed feature extraction algorithms (such as texture analysis and spectral differences) and rule-based classification strategies. However, these methods have significant limitations when dealing with complex scenarios (such as diverse building morphologies, shadow interference, and sensor noise), and are difficult to adapt to the processing needs of remote sensing data under different resolutions and imaging conditions. Furthermore, traditional methods often require a large number of manually labeled samples for model training and have weak generalization ability for small sample data, resulting in detection accuracy and efficiency that fail to meet the requirements of practical applications. The rapid development of deep learning technology has provided new solutions for remote sensing image processing. Convolutional Neural Networks (CNNs), due to their powerful feature learning capabilities, are widely used in semantic segmentation and object detection tasks in remote sensing images. For example, CNNs can effectively capture the spatial neighborhood relationships and contextual information of buildings by processing multi-scale features in parallel. However, feature extraction at a single scale is insufficient to fully characterize their spatial distribution characteristics. Alignment bias across different image periods: Due to variations in satellite revisit cycles, imaging angles, and lighting conditions, significant geometric and spectral differences may exist between two remote sensing images, leading to difficulties in feature matching. Class imbalance: In real-world scenarios, the proportion of pixels in the "unchanged" category is usually much higher than that of the "new" or "reduced" categories, making traditional segmentation models prone to bias due to uneven class distribution. Weak adaptability with small samples: In some areas, the amount of remote sensing data is limited, making it difficult for traditional models to learn robust feature representations from limited data.

[0066] To this end, this application extracts multi-scale features from two sets of target remote sensing images using two CNN encoders, resulting in two sets of multi-scale features. A feature difference module extracts difference enhancement maps from the two sets of target remote sensing images. A VIT encoder extracts the difference features from the difference enhancement maps. The two CNN encoders then fuse the difference features and one set of multi-scale features from the two sets of multi-scale features, resulting in two sets of fused features. A cross-branch feature interaction module and a segmentation head extract and interact with some features from the two sets of fused features to obtain building change features. Specifically, the application extracts multi-scale features from two sets of remote sensing images using two parallel convolutional branches, extracts difference features from the two sets of remote sensing images using a feature difference module and a VIT encoder, and finally extracts and interacts with some features from the two sets of fused features using a cross-branch feature interaction module and a segmentation head to obtain building change features. Through the synergistic effect of the pre-trained building detection model and the dual-branch structure, high-precision detection of building changes can be achieved with limited samples, while also considering computational efficiency. This achieves the effect of accurately detecting the process of building changes.

[0067] In this embodiment of the application, the executing entity can be a building change detection device in a building change detection system. In practical applications, the building change detection device can be electronic devices such as terminal devices and servers, and there are no restrictions here.

[0068] The following is combined Figure 1 The method for detecting changes in buildings according to embodiments of this application will be described in detail.

[0069] Please refer to Figure 1 , Figure 1 A flowchart illustrating a method for detecting changes in a building, provided in an embodiment of this application, is applied to a building detection model, such as... Figure 1 The methods for detecting changes in buildings shown include:

[0070] Step 110: Extract multi-scale features from the two target remote sensing images using two CNN encoders to obtain two sets of multi-scale features.

[0071] The CNN encoder can employ a ResNet18 residual network. By setting the convolution stride and dilation ratio, it avoids reducing the feature map resolution, providing a foundation for accurately identifying the edges of changing patches. Two CNN encoders correspond to two different image phases, generating two sets of multi-scale features. These multi-scale features can include various image characteristics, such as the shape, size, and color of buildings, as well as the overall shape of the image. The CNN encoder includes a neck layer and four sets of residual connection blocks. Each residual connection block contains multiple convolutional layers. The last convolutional layer in each set adjusts the downsampling size and receptive field by combining the convolution stride and dilation ratio. For example, the four sets of convolution strides are 1, 2, 1, 1, and the dilation ratios are 1, 1, 2, 4, corresponding to four feature map sizes of the input image. , , , .

[0072] In some embodiments of this application, the building detection model includes: two CNN encoders, a VIT encoder, a feature difference module, a cross-branch feature interaction module, and a segmentation head; the two CNN encoders each include a neck, a multi-layer residual connection block, and a multi-layer feature fusion module; the VIT encoder includes a block coding layer and a multi-layer coding layer.

[0073] In the above process, the building detection model can extract multi-scale and differential features from different remote sensing images through a dual-branch mechanism, and finally achieve the effect of building change detection by fusing multi-scale and differential features.

[0074] The ViT encoder can be a large pre-trained visual model with a network structure of ViT-B, containing 12 Transformer layers and 1 block encoder layer, and achieving efficient parameter fine-tuning through a low-rank adaptation module. For example, its input is the output of the feature difference module. H and W represent the image's width and height, respectively, and 3 channels. The block coding layer will... according to The size is converted into a visual token of dimension 768. This process is implemented using a convolutional layer with a kernel size of 16 and a stride of 16. After the block encoding layer, the feature map size is... The output of the ViT encoder is then obtained after passing through 12 Transformer layers. ViT has a barrel-shaped structure, and the Transformer layers do not change the size and dimension of the feature maps. The 12 layers are equally divided into 4 groups, and each group interacts with the CNN features through the Feature Fusion Module (FFM).

[0075] In some embodiments of this application, before extracting multi-scale features from two target remote sensing images using two CNN encoders to obtain two sets of multi-scale features, Figure 1 The method also includes: inputting training data consisting of remote sensing images taken at different times and feature difference annotations into the basic VIT encoder to obtain feature differences; comparing the feature differences with the feature differences corresponding to the feature difference annotations to obtain comparison results; and adjusting the parameters of the low-rank fine-tuning module according to the comparison results to obtain the VIT encoder.

[0076] In the above process, this application trains the VIT encoder using remote sensing images taken at different times and image annotations, which can accurately extract the difference features of different remote sensing images.

[0077] The remote sensing images can be images stored in a database or remote sensing images taken by satellite. Feature difference annotations can be obtained by relevant personnel annotating the differences in the remote sensing images in conjunction with field investigations. The basic VIT encoder can be a basic image recognition convolutional network framework. The comparison results include the actual difference features identified by the basic VIT encoder in different remote sensing images.

[0078] In some embodiments of this application, two sets of multi-scale features are obtained by extracting multi-scale features from two target remote sensing images using two CNN encoders, including: extracting local detail features and global context features from two target remote sensing images using two CNN encoders, and adjusting the local detail features and global context features by adjusting the convolution stride and dilated convolution parameters to obtain two sets of multi-scale features.

[0079] In the above process, this application extracts local detail features and global context features from two remote sensing images in two parallel convolutional branches, respectively. The extracted multi-scale features can be fused and injected into the local detail features and global context features.

[0080] Local detail features include the type, location, and actual size of buildings and objects in the surrounding environment. Global contextual features include the overall resolution, size, and building type of the image.

[0081] In some embodiments of this application, two sets of multi-scale features are obtained by extracting multi-scale features of two target remote sensing images from two different CNN encoders, including: extracting multi-scale features of two target remote sensing images from two different CNN encoders through the neck and residual connection blocks of the two CNN encoders to obtain two sets of multi-scale features;

[0082] Two sets of fused features are obtained by fusing differential features and one set of multi-scale features from two sets of multi-scale features using two CNN encoders. This includes fusing differential features and one set of multi-scale features from two sets of multi-scale features using the multi-layer feature fusion modules of the two CNN encoders.

[0083] In the above process, this application can accurately detect changes in buildings by extracting and fusing multi-scale features and difference features from two favorable images.

[0084] Step 120: Extract the difference enhancement map of the target remote sensing images from the two periods using the feature difference module.

[0085] The feature difference module takes two remote sensing images as input, amplifies the pixel semantic differences between the two images, and suppresses pseudo-changes caused by color, illumination, registration errors, etc. Its output is used as the input of the ViT encoder for calculation.

[0086] Step 130: Extract the differential features of the differential enhancement map using the VIT encoder.

[0087] The differential features include the differential features of buildings in the image and the differential features of the surrounding environment.

[0088] In some embodiments of this application, a building inspection model, Figure 1 The method shown also includes: a low-rank fine-tuning module; extracting difference features of the difference enhancement map through a VIT encoder, including: extracting difference features of the difference enhancement map through a block coding layer, a multi-layer coding layer and a low-rank fine-tuning module inserted in the multi-layer coding layer.

[0089] In the above process, this application can accurately extract the difference features between two remote sensing images through the low-rank fine-tuning module of the building detection model.

[0090] In this case, the ViT parameters are relatively large, while downstream scenarios typically have limited sample sizes and computational resources, making it impossible to support full-parameter training of ViT. Therefore, a low-rank fine-tuning strategy is used to reduce the amount of parameter updates in downstream scenarios. During training, the ViT parameters are frozen, and only the parameters of the low-rank fine-tuning module are updated, thereby reducing sample requirements and model update time. The low-rank fine-tuning module consists of two fully connected layers, inserted into the self-attention calculation process of the Transformer layer. The self-attention structure of the low-rank fine-tuning module is as follows: Figure 2 As shown, for the fully connected layer with insertion positions at Q and V, the forward process after insertion is as follows:

[0091]

[0092] in, Indicates the parameters of the fully connected layer. and These are the input and output channel dimensions, respectively. and The low-rank fine-tuning module parameters are implemented through a fully connected layer. Because... much smaller and This significantly reduces training time.

[0093] Please refer to Figure 2 , Figure 2 A schematic diagram of a self-attention structure for inserting a low-rank fine-tuning module is provided in this application, as shown below. Figure 2 The self-attention structure shown includes:

[0094] Low-rank adaptation layer, layer normalization layer (LN), self-attention layer (Attn), and fully connected layer (Mlp).

[0095] Specifically, the feature difference map X is extracted from the two remote sensing images through multi-layer fully connected layers of the low-rank fine-tuning module and layer normalization. The obtained difference features are then added element-wise with the remote sensing image features extracted by the low-rank adaptation layer to obtain features Q, K, and V. Features Q and K are then processed by vector dot product and Softmax function, and then multiplied by vector dot product with feature V. Finally, the initial difference features are output through the fully connected layer. After multiple rounds of the above operations, the final difference feature X' is obtained.

[0096] also, Figure 2 The specific methods and processes shown can be referred to Figure 1 The methods and steps shown will not be elaborated further here.

[0097] Step 140: The difference features and one set of multi-scale features from the two sets of multi-scale features are fused by two CNN encoders to obtain two sets of fused features.

[0098] The process involves fusing differential features and one set of multi-scale features from two sets of multi-scale features using two CNN encoders. The resulting fused features are achieved through a feature fusion module. This module is inserted after each residual connection block of the CNN encoder, establishing a connection between ViT features and convolutional features using a bidirectional attention mechanism, as shown in Figure 3. This module updates the CNN and ViT features through two cross-attention calculations. The updated CNN features are then fed into the next residual connection block, while the updated ViT features are used for the next feature fusion calculation.

[0099] Please refer to Figure 3 , Figure 3 A schematic diagram of a feature fusion module provided in this application is shown below. Figure 2 The feature fusion structure shown includes:

[0100] Self-attention layer Attn, cross-attention layer CrossAttn, layer normalization LN, and fully connected layer Linear.

[0101] Specifically, feature Q is obtained through a self-attention layer and layer normalization query operation, and feature K is obtained through a key / value pair operation. These features are then subjected to a cross-attention layer and layer normalization, and finally through a fully connected layer and layer normalization, to obtain the updated CNN and ViT feature correspondences Q` and K`.

[0102] also, Figure 3 The specific methods and processes shown can be referred to Figure 1 The methods and steps shown will not be elaborated further here.

[0103] Step 150: Extract and interact with some features from the two sets of fused features through the cross-branch feature interaction module and the segmentation head to obtain building change features.

[0104] The cross-branch feature interaction module takes multi-scale features generated by two CNN encoders as input and achieves feature interaction through a bidirectional attention method. The segmentation head takes the output of the cross-branch feature interaction module as input and generates the final pixel-level segmentation result through several layers of convolution and upsampling operations.

[0105] In some embodiments of this application, building change features are obtained by extracting and interacting some features from two sets of fused features through a cross-branch feature interaction module and a segmentation head. This includes: interacting some features from two sets of fused features through the dual attention mechanism of the cross-branch feature interaction module to obtain interactive features; and obtaining building change features through multiple convolution operations and upsampling operations on the interactive features by the segmentation head.

[0106] In the above process, this application uses a cross-branch feature interaction module and a segmentation head to perform feature interaction, convolution operations, and upsampling, which can accurately obtain the final building price change features.

[0107] The module utilizes attention computation to perform interactive computation of deep features from two CNN branches. First, self-attention computation is performed on two sets of CNN features (multi-scale features), completed by two self-attention layers. Then, bidirectional attention is used to complete feature interaction. The two features output by bidirectional attention are then concatenated to obtain the final semantic features usable for segmentation. The segmentation head mainly consists of convolution and deconvolution. Two deconvolutions upsample the image to the input size, and the final output uses simple image interpolation to obtain a segmentation map with the same size as the input.

[0108] In the above Figure 1 In the process described, this application extracts multi-scale features from two sets of target remote sensing images using two CNN encoders, resulting in two sets of multi-scale features. A feature difference module extracts difference enhancement maps from the two sets of target remote sensing images. A VIT encoder extracts the difference features from the difference enhancement maps. The two CNN encoders fuse the difference features and one set of multi-scale features from the two sets of multi-scale features, resulting in two sets of fused features. A cross-branch feature interaction module and a segmentation head extract and interact with some features from the two sets of fused features to obtain building change features. Specifically, two parallel convolutional branches extract multi-scale features from the two sets of remote sensing images, a feature difference module and a VIT encoder extract the difference features from the two sets of remote sensing images, and finally, a cross-branch feature interaction module and a segmentation head extract and interact with some features from the two sets of fused features to obtain building change features. Through the synergistic effect of the pre-trained building detection model and the dual-branch structure, high-precision detection of building changes can be achieved with limited samples, while also considering computational efficiency. This achieves the effect of accurately detecting the process of building changes.

[0109] The following is combined Figure 4 The implementation method for detecting changes in buildings according to embodiments of this application will be described in detail.

[0110] Please refer to Figure 4 , Figure 4 This is a schematic diagram illustrating an implementation method for detecting changes in a building, as provided in an embodiment of this application. Figure 4 The implementation method for detecting changes in buildings, as shown, includes:

[0111] Multi-scale features are extracted from two remote sensing images from two different periods using two CNN encoders, a VIT encoder, a feature difference module, a cross-branch feature interaction module, and an UpperNet segmentation head. The feature difference module and VIT encoder extract the difference features between the two periods of remote sensing images. Finally, the cross-branch feature interaction module and segmentation head extract and interact with some features from the two sets of fused features to obtain the building change features. The two CNN encoders each include a neck layer, a multi-layer residual connection block, and a multi-layer feature fusion module; the VIT encoder includes a block coding layer and a 12-layer coding layer.

[0112] also, Figure 4 The specific methods and steps shown can be found in [reference]. Figure 1 The method shown will not be elaborated further here.

[0113] The previous text passed Figure 1 The method for detecting changes in buildings is described below. Figures 5-6 Describes a device for detecting changes in buildings.

[0114] Please refer to Figure 5 This is a schematic block diagram of a device 500 for detecting changes in a building, provided in an embodiment of this application. The device 500 can be a module, program segment, or code on an electronic device. This device 500 is similar to the one described above. Figure 1 The method implementation corresponds to this and can be executed. Figure 1 The various steps involved in the method embodiment, and the specific functions of the device 500, can be found in the description below. To avoid repetition, detailed descriptions are appropriately omitted here.

[0115] Optionally, the device 500 includes:

[0116] The first extraction module 510 is used to extract multi-scale features of two target remote sensing images through two CNN encoders to obtain two sets of multi-scale features.

[0117] The second extraction module 520 is used to extract the difference enhancement map of the two target remote sensing images through the feature difference module;

[0118] The third extraction module 530 is used to extract the differential features of the differential enhancement map through the VIT encoder;

[0119] The fusion module 540 is used to fuse the difference features and one set of multi-scale features from two sets of multi-scale features through two CNN encoders to obtain two sets of fused features.

[0120] The fourth extraction module 550 is used to extract and interact with some features from the two sets of fused features through the cross-branch feature interaction module and the segmentation head to obtain building change features.

[0121] Optional building inspection models include:

[0122] The system consists of two CNN encoders, a VIT encoder, a feature difference module, a cross-branch feature interaction module, and a segmentation head. The two CNN encoders each include a neck, a multi-layer residual connection block, and a multi-layer feature fusion module. The VIT encoder includes a block coding layer and a multi-layer coding layer.

[0123] Optionally, the fusion module is specifically used for:

[0124] Multi-scale features of the target remote sensing images from two periods are extracted by the neck and residual connection blocks of two CNN encoders to obtain two sets of multi-scale features. The difference features and one set of multi-scale features from the two sets of multi-scale features are fused by the two CNN encoders respectively to obtain two sets of fused features. This includes fusing the difference features and one set of multi-scale features from the two sets of multi-scale features through the multi-layer feature fusion modules of the two CNN encoders to obtain two sets of fused features.

[0125] Optionally, the building inspection model also includes: a low-rank fine-tuning module;

[0126] The third extraction module is specifically used to extract the differential features of the differential enhancement map through the block coding layer, multi-layer coding layer, and low-rank fine-tuning module inserted in the multi-layer coding layer.

[0127] Optionally, the device further includes:

[0128] The training module is used to input training data consisting of remote sensing images taken at different times and feature difference annotations into the basic VIT encoder before the first extraction module extracts multi-scale features of the target remote sensing images from two periods through two CNN encoders to obtain two sets of multi-scale features. The training module obtains feature differences by comparing the feature differences with the feature differences corresponding to the feature difference annotations. The parameters of the low-rank fine-tuning module are adjusted according to the comparison results to obtain the VIT encoder.

[0129] Optionally, the first extraction module is specifically used for:

[0130] Two CNN encoders were used to extract local detail features and global context features from the two target remote sensing images, respectively. The local detail features and global context features were adjusted by adjusting the convolution stride and dilated convolution parameters to obtain two sets of multi-scale features.

[0131] Optionally, the fusion module is specifically used for:

[0132] The interaction features are obtained by using the dual attention mechanism of the cross-branch feature interaction module to interact some features in the two sets of fused features; the building change features are obtained by performing multiple convolution operations and upsampling operations on the interaction features through the segmentation head.

[0133] Please refer to Figure 6 This is a schematic block diagram of a device for detecting changes in a building, provided in an embodiment of this application. The device may include a memory 610 and a processor 620. Optionally, the device may further include a communication interface 630 and a communication bus 640. This device is similar to the one described above. Figure 1 The method implementation corresponds to this and can be executed. Figure 1 The specific functions of the device involved in the method embodiments can be found in the following description.

[0134] Specifically, memory 610 is used to store computer-readable instructions.

[0135] Processor 620 is used to process readable instructions stored in memory and is capable of executing... Figure 1 Each step in the method.

[0136] The communication interface 630 is used for signaling or data communication with other node devices. For example, it is used for communication with a server or terminal, or for communication with other device nodes, but the embodiments of this application are not limited thereto.

[0137] Communication bus 640 is used to enable direct communication between the above components.

[0138] In this embodiment, the communication interface 630 of the device is used for signaling or data communication with other node devices. The memory 610 can be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 610 can also be at least one storage device located remotely from the aforementioned processor. The memory 610 stores computer-readable instructions, which, when executed by the processor 620, enable the electronic device to perform the aforementioned... Figure 1 The method process is shown. The processor 620 can be used on the device 500 and is used to perform the functions in this application. Exemplarily, the processor 620 described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, and the embodiments of this application are not limited thereto.

[0139] This application embodiment also provides a readable storage medium, wherein when the computer program is executed by a processor, it performs the following... Figure 1The method process executed by the electronic device in the illustrated method embodiment.

[0140] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the aforementioned method, and will not be elaborated further here.

[0141] In summary, this application provides a method, apparatus, device, and readable storage medium for detecting building changes. The method includes: extracting multi-scale features from two target remote sensing images using two CNN encoders to obtain two sets of multi-scale features; extracting a difference enhancement map from the two target remote sensing images using a feature difference module; extracting difference features from the difference enhancement map using a VIT encoder; fusing the difference features and one set of multi-scale features from the two sets of multi-scale features using two CNN encoders to obtain two sets of fused features; and extracting and interacting with partial features from the two sets of fused features using a cross-branch feature interaction module and a segmentation head to obtain building change features. This method can accurately detect the process of building changes.

[0142] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0143] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0144] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0145] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0146] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0147] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for detecting changes in a building, characterized in that, Applied to building inspection models, including: Two sets of multi-scale features were obtained by extracting multi-scale features from two target remote sensing images using two CNN encoders. The difference enhancement map of the two target remote sensing images is extracted using the feature difference module; The difference features of the difference enhancement map are extracted using a VIT encoder; The two CNN encoders respectively fuse the difference features and one set of multi-scale features from the two sets of multi-scale features to obtain two sets of fused features; By extracting and interacting with some features from the two sets of fused features through the cross-branch feature interaction module and the segmentation head, the building change features are obtained.

2. The method according to claim 1, characterized in that, The building inspection model includes: The two CNN encoders, the VIT encoder, the feature difference module, the cross-branch feature interaction module, and the segmentation head; The two CNN encoders each include a neck, a multi-layer residual connection block, and a multi-layer feature fusion module; The VIT encoder includes a block coding layer and a multi-layer coding layer.

3. The method according to claim 2, characterized in that, The process involves extracting multi-scale features from two target remote sensing images using two CNN encoders, resulting in two sets of multi-scale features, including: Multi-scale features of the two target remote sensing images are extracted through the neck and residual connection blocks of the two CNN encoders to obtain the two sets of multi-scale features; The two sets of fused features are obtained by fusing the difference features and one set of multi-scale features from the two sets of multi-scale features through the two CNN encoders, including: The two sets of fused features are obtained by fusing the difference features and one set of multi-scale features from the two sets of multi-scale features through the multi-layer feature fusion modules of the two CNN encoders.

4. The method according to claim 2, characterized in that, The building inspection model also includes: Low-rank fine-tuning module; The extraction of differential features from the differential enhancement map using the VIT encoder includes: The difference features of the difference enhancement map are extracted through the block coding layer, the multi-layer coding layer, and the low-rank fine-tuning module inserted in the multi-layer coding layer.

5. The method according to claim 4, characterized in that, Before extracting multi-scale features from the two target remote sensing images using two CNN encoders to obtain two sets of multi-scale features, the method further includes: The training data, consisting of remote sensing images taken at different times and feature difference annotations, is input into the basic VIT encoder to obtain the feature differences. By comparing the feature differences with the feature differences corresponding to the feature difference annotations, the comparison results are obtained; The parameters of the low-rank fine-tuning module are adjusted based on the comparison results to obtain the VIT encoder.

6. The method according to any one of claims 1-4, characterized in that, The process involves extracting multi-scale features from two target remote sensing images using two CNN encoders, resulting in two sets of multi-scale features, including: The two CNN encoders extract local detail features and global context features from the two target remote sensing images, respectively. The local detail features and global context features are then adjusted by adjusting the convolution stride and dilated convolution parameters to obtain the two sets of multi-scale features.

7. The method according to any one of claims 1-4, characterized in that, The process involves extracting and interacting with partial features from the two sets of fused features through a cross-branch feature interaction module and a segmentation head to obtain building change features, including: The interaction features are obtained by using the dual attention mechanism of the cross-branch feature interaction module to interact with some features in the two sets of fused features. The building variation features are obtained by performing multiple convolution and upsampling operations on the interactive features using the segmentation head.

8. A device for detecting changes in a building, characterized in that, include: The first extraction module is used to extract multi-scale features from two phases of remote sensing images of the target through two CNN encoders, resulting in two sets of multi-scale features. The second extraction module is used to extract the difference enhancement map of the two target remote sensing images through the feature difference module; The third extraction module is used to extract the differential features of the differential enhancement map through the VIT encoder; The fusion module is used to fuse the difference features and one set of multi-scale features from the two sets of multi-scale features respectively through the two CNN encoders to obtain two sets of fused features; The fourth extraction module is used to extract and interact with some features from the two sets of fused features through the cross-branch feature interaction module and the segmentation head to obtain building change features.

9. An electronic device, characterized in that, include: A memory and a processor, the memory storing computer-readable instructions that, when executed by the processor, perform the steps of the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, include: A computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1-7.

Images