Building change area detection method and device, electronic equipment and storage medium

Abstract

The application provides a building change area detection method and device, electronic equipment and a storage medium, which are applied to the technical field of computers, and the method comprises the following steps: acquiring a historical image and a current image of a building area; inputting the historical image and the current image into a detection network for change area detection processing to determine a target change area corresponding to the current image; wherein the detection network is obtained by training a plurality of training image pairs, each training image pair comprises a first training image and a second training image, and each second training image is labeled with a labeled change area that changes relative to the corresponding first training image; the detection network comprises an attention module, and the attention module is used to give a higher weight to a region with a position change in each training image pair than to a region without a position change. The technical scheme of the application can improve the accuracy of building change area detection.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting changes in building areas. Background Technology

[0002] Building change detection (such as road change detection) primarily involves observing or detecting subtle changes in buildings (such as roads) within the same scene across different time phases / series of remote sensing images. This detection technology is widely used in urban and rural planning. However, accurately detecting building changes typically relies on expert experience, which is time-consuming and expensive.

[0003] With the continuous development of deep learning technology, there are now cases where deep learning technology is used to detect building changes. Taking road change detection as an example, convolutional networks can be used to learn similar regions of roads in remote sensing images at different time series, and the features of these similar regions can be subtracted to obtain information on road changes in the same scene.

[0004] However, the above-mentioned technologies suffer from low accuracy in change detection results. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for detecting building change areas, which addresses the shortcomings of existing technologies that obtain building change information by subtracting features from similar areas, resulting in low accuracy of change detection results. The invention achieves accurate detection of building change areas through a detection network trained with an attention module, thereby improving the detection accuracy of the obtained change areas with location changes.

[0006] This invention provides a method for detecting changes in building areas, comprising:

[0007] Acquire historical images of the building area taken at a historical time and current images of the building area taken at the current time;

[0008] Both historical and current images are input into a preset detection network to detect and process changed regions, thereby identifying the corresponding target changed regions in the current image;

[0009] The detection network is trained on multiple sets of training image pairs. Each set of training image pairs includes a first training image and a second training image. Each second training image is labeled with a region that has changed relative to the corresponding first training image. The detection network includes an attention module, which assigns a higher weight to regions with positional changes in each set of training image pairs than to regions without positional changes.

[0010] According to the building change area detection method provided by the present invention, the training process of the above-mentioned detection network includes:

[0011] The first and second training images in each training image pair are input into the initial detection network to detect change regions, and the prediction result of the change region corresponding to each training image pair is determined; the prediction result of the change region includes the predicted change region in the corresponding second training image.

[0012] Multiple loss functions are used to calculate the predicted change region and the corresponding labeled change region for each training image pair. The initial detection network is then trained using the values ​​of these multiple losses to obtain a well-trained detection network.

[0013] According to the present invention, a method for detecting changes in building areas includes an initial detection network comprising a feature extraction module, an encoding module, a decoding module, and a fusion module. The method involves inputting the first and second training images from each training image pair into the initial detection network for changes region detection processing, and determining the predicted changes region for each training image pair, including:

[0014] For each pair of training images, the first training image and the second training image are both input into the feature extraction module of the initial detection network for feature extraction processing to obtain the first initial feature corresponding to the first training image and the second initial feature corresponding to the second training image.

[0015] The first and second initial features are input into the encoding module of the initial detection network for multi-scale encoding feature extraction processing to determine the first encoding feature map at different scales corresponding to the first training image and the second encoding feature map at different scales corresponding to the second training image.

[0016] The first encoded feature maps at different scales and the second encoded feature maps at different scales are input into the decoding module of the initial detection network for decoding processing to determine the first decoded feature maps at different scales corresponding to the first training image and the second decoded feature maps at different scales corresponding to the second training image.

[0017] The first encoded feature map, the second encoded feature map, the first decoded feature map, and the second decoded feature map at different scales are input into the fusion module of the initial detection network for feature fusion processing to determine the prediction result of the corresponding change region for the training image.

[0018] According to a method for detecting building change areas provided by the present invention, the above-mentioned coding module includes an attention module, which includes a location attention module and a spatial channel attention module;

[0019] The aforementioned positional attention module and spatial channel attention module are used to assign higher weights to the initial features of regions with positional changes in the first initial features than to the initial features of regions without positional changes, in both the temporal and spatial dimensions.

[0020] According to the building change area detection method provided by the present invention, the above-mentioned inputting first encoded feature maps, second encoded feature maps, first decoded feature maps, and second decoded feature maps at different scales into the fusion module of the initial detection network for feature fusion processing, and determining the prediction result of the training image for the corresponding change area, includes:

[0021] The fusion module of the initial detection network is used to fuse the first encoded feature map and the first decoded feature map at different scales to determine the first fused feature map corresponding to the first training image.

[0022] The fusion module of the initial detection network is used to fuse the second encoded feature maps and the second decoded feature maps of different scales to determine the second fused feature map corresponding to the second training image.

[0023] The first and second fused feature maps are subjected to matrix difference processing to determine the prediction result of the corresponding change region for the training image.

[0024] According to the building change area detection method provided by the present invention, the fusion module includes a local attention embedding module, and the above-mentioned fusion processing of first encoded feature maps and first decoded feature maps of different scales to determine the first fused feature map corresponding to the first training image includes:

[0025] A local attention embedding module is used to perform local attention embedding on the first encoded feature map and the first decoded feature map of the same scale to determine the key features in the first encoded feature map and the key features in the first decoded feature map of the same scale.

[0026] Adaptive pooling is performed on key features in the first encoded feature map and the first decoded feature map of the same scale to determine the first fused feature map corresponding to the first training image.

[0027] According to the building change area detection method provided by the present invention, the fusion module further includes a mapping module, wherein the matrix difference processing of the first fused feature map and the second fused feature map to determine the prediction result of the training image for the corresponding change area includes:

[0028] The first and second fused feature maps are subjected to matrix difference processing to determine the corresponding change feature maps for the training image pairs.

[0029] A mapping module is used to map the values ​​of each pixel in the change feature map to determine the prediction result of the training image for the corresponding change region. The prediction result of the change region specifically includes the prediction result of each pixel in the corresponding second training image. The prediction result is used to characterize whether the corresponding pixel is a predicted change region.

[0030] The present invention also provides a building change area detection device, comprising the following modules:

[0031] The acquisition module is used to acquire historical images of the building area taken at historical times and current images of the building area taken at the current time.

[0032] The detection module is used to input both historical images and the current image into a preset detection network to detect and process the changed regions, and to determine the corresponding target changed regions in the current image;

[0033] The detection network is trained on multiple sets of training image pairs. Each set of training image pairs includes a first training image and a second training image. Each second training image is labeled with a region that has changed relative to the corresponding first training image. The detection network includes an attention module, which assigns a higher weight to regions with positional changes in each set of training image pairs than to regions without positional changes.

[0034] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the building change area detection method as described above.

[0035] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the building change area detection method as described above.

[0036] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the building change area detection method as described above.

[0037] The present invention provides a method, apparatus, electronic device, and storage medium for detecting building change areas. This involves acquiring historical and current images of the same building area taken at historical and current times, respectively, and inputting both images into a pre-trained detection network for change area detection processing to determine the target change area in the current image. The detection network is trained on multiple sets of training image pairs. Each training image pair includes a first training image and a second training image, and each second training image is labeled with marked change areas relative to the corresponding first training image. The detection network also includes an attention module, which assigns a higher weight to areas with positional changes than to areas without positional changes in each training image pair. In this method, because the attention module can assign higher weights to areas with positional changes during network training, the detection network focuses more on detecting positional changes during training. Therefore, when using the trained detection network to detect building change areas, more accurate positional change area detection results can be obtained, thus improving the accuracy of the final detected change areas. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 This is one of the flowcharts of the building change area detection method provided by the present invention.

[0040] Figure 2 This is the second flowchart of the building change area detection method provided by the present invention.

[0041] Figure 3 This is the third flowchart of the building change area detection method provided by the present invention.

[0042] Figure 4 This is a schematic diagram illustrating the visualization effect of road change detection provided by the present invention.

[0043] Figure 5 This is a schematic diagram of the building change area detection device provided by the present invention.

[0044] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0046] To better illustrate the technical solutions of the embodiments of the present invention, the technical background of the embodiments of the present invention will be described below first.

[0047] Building change detection, taking road change detection as an example, mainly involves observing or detecting subtle changes in roads in the same scene in remote sensing images at different times / sequences. Two remote sensing road images from different times but in the same area present a huge challenge for detecting road changes due to the influence of various factors such as the surrounding ecological environment, road paving materials, and traffic signs.

[0048] When precise detection of road changes is required, early change detection methods relied heavily on expert experience, but this was time-consuming and expensive. Other change detection methods are generally divided into two parts: single-pixel analysis and region change identification. Single-pixel analysis aims to construct feature information from raw remote sensing image data, centered on individual pixels. Image pixels and image objects are the two main categories of analyzed pixels. Different types of analyzed pixels share similar feature extraction techniques. Then, to further improve change detection accuracy, spectral and spatial features have been extensively studied. Region change identification typically uses manual or learned rules to compare the representations of analyzed pixels to determine the change category or region. Additionally, a simpler method is to calculate feature difference maps and separate changed regions using thresholds. These change detection methods often involve manual feature design and selection, leaving significant room for improvement in detection accuracy.

[0049] With the rapid development of deep learning and remote sensing imaging technologies, the use of deep learning for change detection has been proposed to mitigate errors caused by manually generated features. However, current deep learning technologies still fall short in capturing deep semantic information and subtle changes. Therefore, how to accurately and efficiently detect corresponding road change regions from remote sensing images has become a research hotspot. For example, some techniques propose using convolutional networks to learn similar regions of roads in remote sensing images from different time series and subtracting the features of similar regions to obtain information on road changes in the same scene. Other techniques combine convolutional and long short-term networks to obtain spatial local and global temporal information of corresponding regions in a set of images and fuse these feature information to accurately determine change information, thereby improving the overall detection performance of the framework. Still other techniques use Transformer models to obtain local details and global semantics of roads in images, as well as corresponding contextual information, thereby improving the network's ability to perceive subtle changes. While these techniques can effectively improve the network's ability to detect road changes in remote sensing images, they neglect the ability of low-level semantic information to represent attributes such as the physical appearance of the target road. At the same time, the feature flow causes a large amount of redundant information to be reused during inter-layer transmission, resulting in insufficient attention to salient features and thus a problem of insufficient representation of salient information.

[0050] The aforementioned techniques reveal that roads are crucial transportation hubs within cities or between cities, characterized by complex backgrounds and a dense, widespread distribution. The appearance of roads in remote sensing images can drastically change in color and shape at different times, leading to the easy overlooking or misclassification of simple road variations. Simply put, relying solely on deep features or physical appearance to directly locate changed areas is insufficient; noise and other pseudo-information can also compromise the model's robustness. Furthermore, the quality and class balance of road samples are critical. Current techniques lack adequate sample quality screening, mistakenly using non-positive samples as positive ones, or failing to effectively locate and label boundary pixels during sample annotation. Additionally, the number of changed classes may be significantly smaller than the number of unchanged classes, resulting in severe class imbalance and ultimately decreased change detection accuracy. Moreover, roads exhibit diversity and a wide scale range, rendering traditional single-scale feature extractors inadequate for extracting effective target information from remote sensing road images, resulting in insufficient detail description of roads. Similarly, when using a simple feature extractor to obtain deep features of a road, redundant information is repeatedly used, and the ability to represent context, global and local details is ignored. The feature is fused using a simple stitching strategy, which leads to a serious loss of target detail information and ultimately affects the performance of road area change detection.

[0051] Based on this, embodiments of the present invention provide a method, apparatus, electronic device, and storage medium for detecting changes in building areas, which can solve the above-mentioned technical problems.

[0052] The following is combined Figures 1-4 This invention describes a method for detecting changes in building areas according to an embodiment of the present invention.

[0053] It should be noted that the execution subject of the embodiments of the present invention may be a building change area detection device, an electronic device, or other devices or systems. No specific limitation is made here. The following embodiments will use an electronic device as the execution subject for illustration.

[0054] Figure 1 This is one of the flowcharts illustrating the building change area detection method provided by the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0055] S102, acquire historical images of the building area taken at a historical time and current images of the building area taken at the current time.

[0056] The building area can be a large area, such as an area containing one or more buildings, including structures like buildings and roads. The building area can typically include an area with multiple roads.

[0057] Historical time can be a time earlier than the current time. There is no specific limitation on the time interval between historical time and the current time. For example, historical time can be one year ago or half a year ago.

[0058] At both historical and current times, the same building area can be photographed using image acquisition devices to obtain images of the building area at the corresponding moments, which are recorded as the historical image and the current image, respectively. These image acquisition devices can be, for example, cameras, video cameras, snapshot cameras, drones, or other acquisition devices equipped with cameras.

[0059] Since buildings may change over time (e.g., planned roads may change), it is necessary to detect areas of change in the current image of the building area for subsequent data processing.

[0060] S104, both the historical image and the current image are input into a preset detection network for detecting changed regions to determine the corresponding target changed regions in the current image; wherein, the detection network is trained based on multiple sets of training image pairs, each set of training image pairs includes a first training image and a second training image, and each second training image is labeled with a marked changed region relative to the corresponding first training image; the detection network includes an attention module, and the attention module is used to assign a higher weight to regions with positional changes in each set of training image pairs than to regions without positional changes.

[0061] In this step, a pre-trained detection network can be used to detect changing regions in the current building area image. This detection network can be a neural network, optionally a neural network based on the U-net architecture. Furthermore, this detection network focuses on detecting changes in the spatial location of the region; for example, it could be a spatially enhanced U-shaped detection network.

[0062] The detection network can be pre-trained, and its training process is briefly described as follows: obtain multiple sets of training image pairs and the labeled change regions corresponding to each set of training image pairs, use each set of training image pairs as the reference input of the initial detection network, use the labeled change regions of each set of training image pairs as the reference output of the initial detection network, and train the initial detection network to obtain the trained detection network.

[0063] Each training image pair consists of two training images, designated as the first training image and the second training image. Both the first and second training images in each pair are taken from the same sample building area, but their capture events / time sequences differ. Specifically, the first training image may have been captured earlier than the second training image. Using the first training image as a reference, the primary focus is on detecting changes in the second training image relative to the first. Furthermore, each second training image can be labeled with areas that have changed or not changed compared to the corresponding first training image. This labeling of changed areas in the second training image can be done at the pixel level. For example, every pixel in the second training image can be labeled; a label of 1 indicates a changed area, while a label of 0 indicates an unchanged area. This labeling process allows us to obtain the labeled changed and unchanged areas in each first training image, which can serve as the gold standard / reference for network training.

[0064] In addition, the above detection network may include an attention module, which can be composed of network layers or logic algorithms. During the training process of the detection network, the attention module can assign higher weights to the features of regions where the position has changed, and lower weights to the features of regions where the position has not changed. This allows the detection network to focus more on learning the significant feature of position change, and ultimately improve the fine detection of the feature of position change.

[0065] The detection network can be trained using the above method. The training process of the detection network will be described in detail in the following embodiments, and only a brief explanation is given here. After obtaining the trained detection network, both historical images and current images of the same building area can be input into the trained detection network for change region detection processing, so as to detect the regions in the current image that have changed relative to the historical images. The detected changed regions can be recorded as target changed regions. In addition, the change here generally refers to a change in spatial location.

[0066] In this embodiment, historical and current images of the same building area, captured at historical and current times respectively, are acquired and input into a pre-trained detection network for detecting changed regions, thereby identifying the target changed regions in the current image. This detection network is trained on multiple sets of training image pairs, each pair including a first and a second training image. Each second training image is labeled with marked changed regions relative to the corresponding first training image. The detection network also includes an attention module, which assigns higher weights to regions with positional changes than to regions without positional changes in each training image pair. Because the attention module assigns higher weights to regions with positional changes during network training, the detection network focuses more on detecting positional changes during training. Therefore, when detecting changed regions of a building using the trained detection network, more accurate positional change region detection results can be obtained, thus improving the accuracy of the final detected changed regions.

[0067] The following examples illustrate the specific training process of the detection network.

[0068] Figure 2 This is the second flowchart of the building change area detection method provided by the present invention, as shown below. Figure 2 As shown, the training process / method of the above detection network may include the following steps:

[0069] S202, the first training image and the second training image in each training image pair are input into the initial detection network to perform change region detection processing, and the change region prediction result corresponding to each training image pair is determined; the change region prediction result includes the predicted change region in the corresponding second training image.

[0070] In this step, as mentioned above, multiple sets of training image pairs can be acquired during network training. When acquiring these sets, one could obtain a set of training image pairs representing different time periods of the building area (e.g., a set of remote sensing road images from different time periods). Then, using LabelMe annotation software, the changed regions and unchanged regions in the second training image of this set are labeled. For example, changed regions can be labeled as 1, and unchanged regions as 0. Alternatively, multiple sets of training image pairs can be acquired and labeled separately. Then, the same preprocessing operation can be performed on both training images in each set. This yields more sets of training image pairs. Preprocessing ensures the effectiveness of the training samples and highlights the differences and diversity between them, while also increasing the number of training samples. Preprocessing operations can include translation, rotation, color transformation, inversion, and random cropping.

[0071] Here, various preprocessing techniques are applied to the training samples to expand them, highlighting the differences between buildings (such as roads) and the surrounding background. This helps to alleviate semantic ambiguity between buildings and the surrounding organizational structure, especially the boundary pixel attributes between buildings and the surrounding organizational structure.

[0072] Furthermore, considering that buildings within the changing region have varying sizes and shapes, extracting features from a suitable range can better represent this changing region. Therefore, to ensure that the training images in each training image pair can be smoothly input into the designed detection network framework, each training image in the original and amplified training image pairs can be cropped to a preset size according to a preset cropping ratio r. This preset size can be determined based on the input size of the detection network, for example, it could be 512×512 pixels. The preset cropping ratio r could be, for example, 0.1. By using this preset cropping ratio, more detail information can be preserved in the cropped image for accurate detection of the changing region.

[0073] Furthermore, in order to demonstrate the superiority of the designed detection network on small samples, in this embodiment, the multiple training image pairs finally amplified above can be divided into three parts: training set, test set and validation set, with each accounting for 40%, 10% and 50% of the total training image pairs, respectively.

[0074] After dividing the training set as described above, multiple training image pairs in the training set can be input into the initial detection network for change region detection processing to obtain the change region prediction result corresponding to each training image pair, which may include the change region predicted in the second training image, denoted as the predicted change region.

[0075] S204 employs multiple loss functions to calculate various losses between the predicted change region and the corresponding labeled change region for each training image pair, and uses the values ​​of these multiple losses to train the initial detection network, thereby obtaining a trained detection network.

[0076] In this step, after obtaining the predicted change region for each training image pair, various loss functions can be used to calculate the loss between the predicted change region and its corresponding labeled change region. These various loss functions may include, for example, the binary cross-entropy loss function, the Focal Loss loss function, and the Dice loss function.

[0077] After calculating the various losses between the predicted change region and its corresponding labeled change region for each training image pair using various loss functions, the values ​​of the various losses for each training image pair can be summed (including weighted summation or direct summation) to obtain the total loss value for that training image pair. Then, this total loss value can be used for back-parameter tuning to train the initial detection network, ultimately obtaining a trained detection network.

[0078] The method for summing the various losses for each training image pair described above can be found in the following formula:

[0079] .

[0080] in, λ , α , β This represents the learnable balance factor, and its values ​​can be, for example, 0.25, 0.4, or 1. L bce , L fl , L dice denoted as binary cross-entropy loss, Focal Loss, and dice loss, respectively.

[0081] In this embodiment, multiple loss functions are used to calculate the predicted and labeled change regions for each training image pair, and the calculated losses are used to train the initial detection network. This can alleviate the problem of reduced detection accuracy caused by class imbalance, and thus improve the detection accuracy of the trained detection network for change regions.

[0082] In some embodiments, the initial detection network includes a feature extraction module, an encoding module, a decoding module, and a fusion module. The following embodiments illustrate how these modules of the initial detection network specifically perform change region detection processing and obtain predicted change regions.

[0083] Figure 3 This is the third flowchart of the building change area detection method provided by the present invention, as shown below. Figure 3 As shown, the above S202 may include the following steps:

[0084] S302, for each pair of training images, the first training image and the second training image are both input into the feature extraction module of the initial detection network for feature extraction processing to obtain the first initial feature corresponding to the first training image and the second initial feature corresponding to the second training image.

[0085] Taking a set of training image pairs as an example, the detection network has two input channels, which are used to input the two training images in each set of training image pairs. Specifically, the first and second training images in the set of training image pairs can be input into the feature extraction module of the initial detection network through these two channels respectively.

[0086] Optionally, the initial detection network may include two identical feature extraction modules. The first training image and the second training image can be respectively input into a feature extraction module for feature extraction processing to obtain their respective extracted initial features. Here, the features extracted from the first training image are denoted as the first initial features, and the features extracted from the second training image are denoted as the second initial features.

[0087] For example, the first initial features can be obtained by performing initial feature extraction on the first training image using the following formula.

[0088] .

[0089] in, X A This represents the first training image. X B The second training image is represented as the range of the two training images. Where R represents a real number, and C, H, and W represent the spatial channel, image height, and image width, respectively; This represents the first initial feature of the first training image; This represents the max pooling operation; This represents a 7×7 convolution operation; This represents the ReLU activation function, i.e., the function that performs the activation operation. This indicates the batch normalization layer, which is where batch normalization operations are performed.

[0090] Similarly, the second initial features of the second training image can be obtained according to the above formula. .

[0091] S304, the first initial feature and the second initial feature are input into the encoding module of the initial detection network for multi-scale encoding feature extraction processing to determine the first encoding feature map at different scales corresponding to the first training image and the second encoding feature map at different scales corresponding to the second training image.

[0092] In this step, the initial detection network may include two sets of identical encoding modules (e.g., ResNet residual modules). Each set of encoding modules is used to perform encoding feature extraction processing at different scales on an initial feature. After obtaining the first and second initial features, the first and second initial features can be respectively input into the corresponding set of encoding modules for encoding feature extraction processing at different scales, to obtain multiple encoding feature maps at different scales corresponding to the first initial feature / first training image (all denoted as the first encoding feature map) and multiple encoding feature maps at different scales corresponding to the second initial feature / second training image (all denoted as the second encoding feature map).

[0093] Each set of encoding modules may include multiple encoding modules of different scales, used to extract local spatial details and global contextual temporal information of different scales from the training image / initial features.

[0094] As an optional embodiment, the above-described encoding module may further include an attention module, which includes a positional attention module and a spatial channel attention module. The positional attention module and the spatial channel attention module are used to assign higher weights to the initial features of regions with positional changes in the first initial features, both in the temporal and spatial dimensions, than to the initial features of regions without positional changes. Specifically, each scale's encoding module may include both an attention module and a spatial channel attention module, so that when each encoding module extracts the encoded feature map at the corresponding scale, it focuses more on extracting local details and global features of regions with positional changes.

[0095] For example, taking the first training image as an example, for the first training image X A The multi-scale first coding feature map, taking four first coding feature maps of different scales as an example, can be defined as follows: ,in, Let s represent the first encoded feature map at a certain scale, and s represent different scales. Furthermore, the input to the first encoding module in each group of encoding modules is the first initial feature, and the input to each subsequent encoding module is the first encoded feature at a different scale output by the previous encoding module. For example, the first encoded feature map can be obtained by extracting encoded features from the first initial feature of the first training image using the following formula:

[0096] .

[0097] in express The first encoded feature map output by the previous encoding module, This represents encoding modules of different scales.

[0098] The feature extraction process for the first coding module of the first training image can be performed using the following formula:

[0099] .

[0100] in, This indicates a positional attention module; This indicates a spatial channel attention module; This represents the feature matrix concatenation operation; , These represent 1×1 convolution operations and 3×3 convolution operations, respectively. This represents the addition operation of the elements of the characteristic matrix.

[0101] Specifically, the first initial feature is input to the first encoding module, and the encoding feature is extracted by different encoding modules in a set of encoding modules corresponding to the first training image. Each module can obtain an encoding feature map of a different scale. In this way, multiple encoding feature maps of different scales can be obtained for each training image.

[0102] As described above, this embodiment embeds a positional attention module and a spatial channel attention module in each encoding module to enhance feature extraction processing on a set of training image pairs. This allows for a focus on the correlation between the same positions within a set of training image pairs, establishing positional associations between the first and second training images at different time sequences. Without additional auxiliary information, feature sharing can be achieved between the same and cross-layer levels. Simultaneously, an adaptive average pooling layer is used to aggregate the global context, effectively perceiving subtle changes between a set of training image pairs and improving the change detection performance of the detection network. Furthermore, an excitation squeezing layer can be introduced into the change detection head module (i.e., the initial input module of the detection network) to reduce computation while maintaining detection accuracy.

[0103] S306, input the first encoded feature maps of different scales and the second encoded feature maps of different scales into the decoding module of the initial detection network for decoding processing, and determine the first decoded feature maps of different scales corresponding to the first training image and the second decoded feature maps of different scales corresponding to the second training image.

[0104] In this step, the initial detection module may also include two identical decoding modules, each used to decode the multi-scale encoded features of a training image. After obtaining the multi-scale encoded features of each training image, they can be input into the corresponding decoding modules for decoding to obtain the multi-scale decoded features for each of the two training images. Here, the multi-scale decoded features correspond one-to-one with the scale of the corresponding multi-scale encoded features, thus facilitating enhanced feature representation.

[0105] In addition, each decoding module can also include multiple decoding modules, which are used to decode the encoded features at the corresponding scale to obtain the decoded feature map at the corresponding scale; or, according to the upsampling or downsampling structure of the U-shaped network, the input of the first decoding module is the encoded feature map output by the last encoding module of the corresponding training image, and then the decoding modules continuously perform upsampling processing to obtain decoded feature maps at different scales.

[0106] Here, the multi-scale decoded feature maps obtained after decoding the multi-scale encoded feature maps of the first training image can all be denoted as the first decoded feature maps, and the multi-scale decoded feature maps obtained after decoding the multi-scale encoded feature maps of the second training image can all be denoted as the second decoded feature maps.

[0107] For example, the first decoded feature map of the first training image can be denoted as: The second decoded feature map of the second training image can be denoted as... .

[0108] S308, input the first encoded feature map, the second encoded feature map, the first decoded feature map, and the second decoded feature map at different scales into the fusion module of the initial detection network for feature fusion processing, and determine the prediction result of the corresponding change region for the training image.

[0109] In this step, the initial detection network may also include a fusion module. The first encoded feature maps, second encoded feature maps, first decoded feature maps, and second decoded feature maps of different scales obtained above can all be input into this fusion module for feature fusion processing. Specifically, during feature fusion processing, the encoded and decoded features of the same scale for each training image can be fused first to obtain the fused features for each training image. Then, the fused features of two training images can be fused to obtain the final fused features. The prediction results for the corresponding change regions of the training images can then be obtained using these final fused features.

[0110] It is understandable that for each pair of training images, the predicted change region for each pair of training images can be obtained by following the steps described above.

[0111] As described above, the building change area detection data in this embodiment has time and space dimensions. In this embodiment, spatial and positional attention mechanisms are used to jointly enhance the encoding and decoding features of a set of training image pairs. By leveraging the hidden correlation between the space and position of the two training images, the effectiveness of building change area detection can be improved.

[0112] In this embodiment, the feature extraction module, multi-scale encoding module, multi-scale decoding module, and feature fusion module in the detection network perform initial feature extraction, multi-scale encoding processing, multi-scale decoding processing, and feature fusion processing on two training images, respectively, and finally obtain the change region prediction results of the training image pair. In this way, during the multi-scale feature, multi-level local and global feature fusion process, the encoded and decoded features at different scales can be modeled, and the model with different weights can be used to flexibly focus on salient features. It can also learn spatial heterogeneous semantics from multi-temporal training image data, pay more attention to context, global and local detail information, and can collaboratively infer and discover rich architectural detail information in the image, thereby improving the accuracy of change region detection.

[0113] The following examples illustrate the specific process of feature fusion of the multi-scale encoded feature map and the decoded feature map.

[0114] In some embodiments, S308 may include the following steps:

[0115] Step A1: Using the fusion module of the initial detection network, the first encoded feature map and the first decoded feature map of different scales are fused to determine the first fused feature map corresponding to the first training image.

[0116] In this process, the encoded feature maps and decoded feature maps of the same scale of the first training image can be fused to obtain a fused feature map at each scale. Then, the fused feature maps of multiple different scales can be further fused to obtain the final fused feature map, which is denoted as the first fused feature map.

[0117] In addition, to avoid fusing all features during the feature fusion process and repeatedly using redundant information, which would lead to a serious loss of target detail information and ultimately affect the performance of building area change detection, as an optional embodiment, the above-mentioned fusion module may include a local attention embedding module. Specifically, in the feature fusion process, the local attention embedding module can be used to perform local attention embedding processing on the first encoded feature map and the first decoded feature map of the same scale to determine the key features in the first encoded feature map and the first decoded feature map of the same scale; adaptive pooling processing is then performed on the key features in the first encoded feature map and the first decoded feature map of the same scale to determine the first fused feature map corresponding to the first training image.

[0118] In other words, during the feature fusion process of encoded and decoded feature maps of the same scale, a local attention embedding module can be used to extract key details from both the encoded and decoded feature maps, and then perform feature fusion processing on the key details to retain more local details and improve the detection accuracy of the changing areas of the building.

[0119] For example, the feature fusion process of the first training image using the local attention embedding module can be represented by the following formula:

[0120] .

[0121] in, This represents the first fused feature map; This indicates an adaptive pooling operation; Represents a local attention embedding module for the space; These represent the first multi-scale encoding feature map and the first multi-scale decoding feature map of the first training image, respectively.

[0122] Step A2: Using the fusion module of the initial detection network, the second encoded feature maps and the second decoded feature maps of different scales are fused to determine the second fused feature map corresponding to the second training image.

[0123] In this step, the second fusion feature map corresponding to the second training image can be obtained in the manner described in step A1 above, which will not be repeated here.

[0124] Step A3: Perform matrix difference processing on the first fused feature map and the second fused feature map to determine the prediction result of the corresponding change region for the training image.

[0125] In this step, the fusion feature maps of the two training images can be processed using the following formula:

[0126] .

[0127] in, O represents the matrix interpolation operation; O represents the final change feature map obtained.

[0128] The difference between the first and second fused feature maps can be calculated using the matrix interpolation operation described above. This difference represents the feature differences or changes between the two training images and is denoted as the change feature map. The change feature map can then be used to obtain the prediction result for the changed region.

[0129] As an optional embodiment, the above-mentioned fusion module further includes a mapping module, and this step may include the following steps:

[0130] The first fused feature map and the second fused feature map are subjected to matrix difference processing to determine the corresponding change feature map of the training image pair; the value of each pixel in the change feature map is mapped by the mapping module to determine the prediction result of the change region corresponding to the training image pair; the above change region prediction result specifically includes the prediction result of each pixel in the corresponding second training image, and the above prediction result is used to characterize whether the corresponding pixel is a predicted change region.

[0131] After obtaining the change feature map, its size is generally the same as the size of the first and second training images input to the detection network, and it can include multiple pixels. Here, the mapping module in the detection network can be used to map the values ​​of each pixel in the change feature map. Specifically, the values ​​of each pixel in the change feature map can be normalized. For example, each pixel value can be compared with a threshold. If a pixel value is greater than the threshold, its value is set to 1; if a pixel value is not greater than the threshold, its value is set to 0. This method yields a binarized change feature map. This binarized change feature map is the prediction result of the corresponding change region for the training image. The value of each pixel in this prediction result indicates whether the pixel is a change region. For example, a pixel value of 1 indicates that the pixel is a predicted change region, and a pixel value of 0 indicates that the pixel is a predicted unchanged region.

[0132] It is understood that, based on the explanation of this embodiment, the target change region obtained in S104 above can be the change region detection result output by the detection network. The change region detection result is actually a binary change feature map. The change region can be obtained by the value of each pixel in the binary change feature map. Finally, the target change region can be obtained by combining these change regions.

[0133] In this embodiment, during the fusion of multi-scale features, multi-level local and global features, the local spatial embedding module models the encoded and decoded features at different scales. It also utilizes networks with different weights to flexibly focus on salient features in areas of building change and learns spatial heterogeneous semantics from multi-temporal building training image pairs to collaboratively infer and discover rich architectural details in the images, thereby improving the detection accuracy of areas of building change.

[0134] In this embodiment, the multi-scale decoding features and decoding features of each training image are first fused, and then the fused features of the two training images are subjected to matrix difference operation to obtain the change region prediction result. This multi-scale encoding and decoding feature fusion and matrix difference operation allows for a description of the building area from different angles, providing richer information and improving the accuracy of the final detected change region. Furthermore, a local attention embedding module can be used to extract key features during feature fusion, preserving more detailed information for feature fusion and ultimately improving the accuracy and efficiency of change region detection. Further, by mapping the pixel values ​​in the change feature map after the fused feature difference operation, a binarized change feature map is obtained. This provides a more intuitive understanding of the change region detection results, improving the readability and accuracy of the final change region detection results.

[0135] Furthermore, the detection network can be trained and obtained through the above embodiments. At the same time, the trained detection network can be tested and verified through the above test set and validation set, and efficiency and performance tests were performed on large-scale images. The test verification can prove that the detection network finally obtained in this embodiment is relatively effective and robust in detecting changing regions.

[0136] Taking the change area detection of a specific road area as an example, see [link / reference]. Figure 4 The diagram shows the visualization effect of road change detection. In the first row, the left image (a) is the historical image before the change, and the right image (b) is the current image after the change. In the second row, the left image (c) is the generated change mask (i.e., the binarized change feature map mentioned above), and the right image (d) is the visualization area, which is the road change area displayed in the current image in a visual way.

[0137] As can be seen, the change area detection method in this embodiment can effectively detect subtle changes in the road and improve the accuracy of road area detection.

[0138] The following describes the building change area detection device provided by the present invention. The building change area detection device described below can be referred to in correspondence with the building change area detection method described above.

[0139] Figure 5 This is a structural schematic diagram of the building change area detection device provided by the present invention. (See attached diagram) Figure 5 As shown, the device may include:

[0140] The acquisition module 510 is used to acquire historical images of the building area taken at a historical time and current images of the building area taken at the current time.

[0141] The detection module 520 is used to input both historical images and the current image into a preset detection network to perform change region detection processing and determine the corresponding target change region in the current image;

[0142] The detection network is trained on multiple sets of training image pairs. Each set of training image pairs includes a first training image and a second training image. Each second training image is labeled with a region that has changed relative to the corresponding first training image. The detection network includes an attention module, which assigns a higher weight to regions with positional changes in each set of training image pairs than to regions without positional changes.

[0143] In some embodiments, the above-described apparatus may further include a training module, the training module comprising:

[0144] The prediction unit is used to input the first training image and the second training image in each training image pair into the initial detection network to perform change region detection processing and determine the change region prediction result corresponding to each training image pair; the change region prediction result includes the predicted change region in the corresponding second training image.

[0145] The training unit is used to calculate multiple losses between the predicted change region and the corresponding labeled change region for each training image pair using various loss functions, and to train the initial detection network using the values ​​of these multiple losses to obtain a trained detection network.

[0146] In some embodiments, the initial detection network includes a feature extraction module, an encoding module, a decoding module, and a fusion module. Specifically, the prediction unit is configured to: input both the first and second training images into the feature extraction module of the initial detection network for feature extraction processing to obtain a first initial feature corresponding to the first training image and a second initial feature corresponding to the second training image; input the first and second initial features into the encoding module of the initial detection network for multi-scale encoding feature extraction processing to determine a first encoded feature map at different scales corresponding to the first training image and a second encoded feature map at different scales corresponding to the second training image; input the first encoded feature maps at different scales and the second encoded feature maps at different scales into the decoding module of the initial detection network for decoding processing to determine a first decoded feature map at different scales corresponding to the first training image and a second decoded feature map at different scales corresponding to the second training image; and input the first encoded feature maps at different scales, the second encoded feature maps at different scales, the first decoded feature maps at different scales, and the second decoded feature maps at different scales into the fusion module of the initial detection network for feature fusion processing to determine the prediction result of the change region corresponding to the training image pair.

[0147] Optionally, the above encoding module includes an attention module, which includes a positional attention module and a spatial channel attention module; the positional attention module and the spatial channel attention module are used to assign a higher weight to the initial features of regions with positional changes in the first initial features than to the initial features of regions without positional changes in the first initial features in the time and spatial dimensions.

[0148] In some embodiments, the prediction unit is specifically used to perform fusion processing on first encoded feature maps and first decoded feature maps of different scales using the fusion module of the initial detection network to determine the first fused feature map corresponding to the first training image; to perform fusion processing on second encoded feature maps and second decoded feature maps of different scales using the fusion module of the initial detection network to determine the second fused feature map corresponding to the second training image; and to perform matrix difference processing on the first fused feature map and the second fused feature map to determine the prediction result of the training image for the corresponding change region.

[0149] Optionally, the fusion module includes a local attention embedding module. The prediction unit is specifically used to perform local attention embedding processing on the first encoded feature map and the first decoded feature map of the same scale using the local attention embedding module to determine the key features in the first encoded feature map and the first decoded feature map of the same scale; and to perform adaptive pooling processing on the key features in the first encoded feature map and the first decoded feature map of the same scale to determine the first fusion feature map corresponding to the first training image.

[0150] Optionally, the fusion module further includes a mapping module. The prediction unit is specifically used to perform matrix difference processing on the first fused feature map and the second fused feature map to determine the corresponding change feature map of the training image pair; the mapping module is used to map the values ​​of each pixel in the change feature map to determine the prediction result of the corresponding change region of the training image pair; the prediction result of the change region specifically includes the prediction result of each pixel in the corresponding second training image, and the prediction result is used to characterize whether the corresponding pixel is a predicted change region.

[0151] It should be noted that the apparatus provided in this embodiment of the invention can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0152] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a building change area detection method. This method includes: acquiring historical images of the building area taken at a historical time and a current image of the building area taken at the current time; inputting both the historical and current images into a preset detection network for change area detection processing to determine the corresponding target change area in the current image; wherein the detection network is trained based on multiple sets of training image pairs, each set of training image pairs including a first training image and a second training image, and each second training image is labeled with a marked change area that has changed relative to the corresponding first training image; the detection network includes an attention module, and the attention module assigns a higher weight to regions with positional changes in each set of training image pairs than to regions without positional changes.

[0153] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0154] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the building change area detection method provided by the above methods. The method includes: acquiring historical images of a building area taken at a historical time and current images of the building area taken at the current time; inputting both the historical images and the current images into a preset detection network for change area detection processing to determine the target change area corresponding to the current image; wherein the detection network is trained based on multiple sets of training image pairs, each set of training image pairs includes a first training image and a second training image, and each second training image is marked with a marked change area that has changed relative to the corresponding first training image; the detection network includes an attention module, and the attention module is used to assign a higher weight to the regions with positional changes in each set of training image pairs than to the regions without positional changes.

[0155] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the building change area detection method provided by the above methods. The method includes: acquiring historical images of a building area taken at a historical time and a current image of the building area taken at the current time; inputting both the historical image and the current image into a preset detection network for change area detection processing to determine the target change area corresponding to the current image; wherein the detection network is trained based on multiple sets of training image pairs, each set of training image pairs includes a first training image and a second training image, and each second training image is marked with a marked change area that has changed relative to the corresponding first training image; the detection network includes an attention module, and the attention module is used to assign a higher weight to the regions with positional changes in each set of training image pairs than to the regions without positional changes.

[0156] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0157] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting changes in building areas, characterized in that, include: Acquire historical images of the building area taken at a historical time and current images of the building area taken at the current time; Both historical and current images are input into a preset detection network to detect and process changed regions, thereby identifying the corresponding target changed regions in the current image; The detection network is trained on multiple sets of training image pairs. Each set of training image pairs includes a first training image and a second training image. Each second training image is labeled with a region that has changed relative to the corresponding first training image. The regions with changes are labeled in pixels. The detection network includes an attention module, which assigns a higher weight to regions with positional changes than to regions without positional changes in each set of training image pairs. The attention module includes a positional attention module and a spatial channel attention module. The detection network includes multiple encoding modules of different scales, and each encoding module includes a positional attention module and a spatial channel attention module. The initial detection network includes a feature extraction module, an encoding module, a decoding module, and a fusion module. The fusion module includes a local attention embedding module. The training process of the detection network includes: For each training image pair, the training image pair is input into the feature extraction module for feature extraction processing to obtain the first initial feature and the second initial feature corresponding to the training image pair; The first and second initial features are input into the encoding module for multi-scale encoding feature extraction to determine the first encoding feature map and the second encoding feature map at different scales for the training image pairs. The first encoded feature maps and the second encoded feature maps of different scales are input into the decoding module for decoding processing to determine the first decoded feature maps and the second decoded feature maps of different scales corresponding to the training image pairs. Different scales of first encoded feature maps, second encoded feature maps, first decoded feature maps, and second decoded feature maps are input into the fusion module for feature fusion processing to determine the prediction results of the corresponding change regions for the training image. The feature fusion processing includes: using a local attention embedding module to perform local attention embedding processing on the first encoded feature maps and the first decoded feature maps of the same scale to determine the key features in the first encoded feature maps and the first decoded feature maps of the same scale; and performing adaptive pooling processing on the key features in the first encoded feature maps and the first decoded feature maps of the same scale to determine the first fused feature map corresponding to the first training image.

2. The method for detecting building change areas according to claim 1, characterized in that, The training process of the detection network also includes: Multiple loss functions are used to calculate the various losses between the predicted change region and the corresponding labeled change region for each training image pair, and the initial detection network is trained using the values ​​of the multiple losses to obtain the trained detection network.

3. The method for detecting building change areas according to claim 1, characterized in that, The step of inputting first encoded feature maps, second encoded feature maps, first decoded feature maps, and second decoded feature maps of different scales into the fusion module for feature fusion processing to determine the prediction result of the corresponding change region for the training image includes: The fusion module of the initial detection network is used to fuse the first encoded feature maps and the first decoded feature maps of different scales to determine the first fused feature map corresponding to the first training image. The fusion module of the initial detection network is used to fuse the second encoded feature maps and the second decoded feature maps of different scales to determine the second fused feature map corresponding to the second training image. The first fused feature map and the second fused feature map are subjected to matrix difference processing to determine the prediction result of the corresponding change region of the training image.

4. The method for detecting building change areas according to claim 3, characterized in that, The fusion module further includes a mapping module, wherein the matrix difference processing of the first fused feature map and the second fused feature map to determine the prediction result of the corresponding change region of the training image includes: The first fused feature map and the second fused feature map are subjected to matrix difference processing to determine the change feature map corresponding to the training image pair; The mapping module is used to map the values ​​of each pixel in the change feature map to determine the prediction result of the corresponding change region for the training image; the prediction result of the change region specifically includes the prediction result of each pixel in the corresponding second training image, and the prediction result is used to characterize whether the corresponding pixel is a predicted change region.

5. A device for detecting changes in building areas, characterized in that, include: The acquisition module is used to acquire historical images of the building area taken at historical times and current images of the building area taken at the current time. The detection module is used to input both historical images and the current image into a preset detection network to detect and process the changed regions, and to determine the corresponding target changed regions in the current image; The detection network is trained on multiple sets of training image pairs. Each set of training image pairs includes a first training image and a second training image. Each second training image is labeled with a region that has changed relative to the corresponding first training image. The regions with changes are labeled in pixels. The detection network includes an attention module, which assigns a higher weight to regions with positional changes than to regions without positional changes in each set of training image pairs. The attention module includes a positional attention module and a spatial channel attention module. The detection network includes multiple encoding modules of different scales, and each encoding module includes a positional attention module and a spatial channel attention module. The initial detection network includes a feature extraction module, an encoding module, a decoding module, and a fusion module. The fusion module includes a local attention embedding module. The device also includes a training module, which includes a prediction unit. The prediction unit is used to: input the training image pairs into the feature extraction module for feature extraction processing to obtain the first initial feature and the second initial feature corresponding to the training image pairs; input the first initial feature and the second initial feature into the encoding module for multi-scale encoding feature extraction processing to determine the first encoding feature map and the second encoding feature map at different scales corresponding to the training image pairs; input the first encoding feature map and the second encoding feature map at different scales into the decoding module for decoding processing to determine the first decoding feature map and the second decoding feature map at different scales corresponding to the training image pairs; and input the first encoding feature map, the second encoding feature map, the first decoding feature map, and the second decoding feature map at different scales into the fusion module for feature fusion processing to determine the prediction result of the change region corresponding to the training image pairs. The prediction unit is specifically used to perform local attention embedding processing on the first encoded feature map and the first decoded feature map of the same scale using the local attention embedding module, to determine the key features in the first encoded feature map and the first decoded feature map of the same scale; and to perform adaptive pooling processing on the key features in the first encoded feature map and the first decoded feature map of the same scale to determine the first fused feature map corresponding to the first training image.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the building change area detection method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the building change area detection method as described in any one of claims 1 to 4.

Images