Building change detection method and system, electronic device and storage medium

Abstract

This application provides a method, system, electronic device, and storage medium for detecting building changes, belonging to the field of high-resolution remote sensing image processing technology. The method includes: acquiring a first image to be detected; acquiring a reference image and a reference building annotation map; preprocessing the first image; sampling the reference image and the reference building annotation map to form a training sample set; training a semantic segmentation model based on the training sample set; inputting the first image into the trained semantic segmentation model to obtain a probability distribution map; labeling pixels with probability values ​​greater than a first preset threshold in the probability distribution map as buildings, generating a first building annotation map; acquiring multiple building patches within the intersection of the reference image and the first image; performing similarity matching on the building patches in the probability distribution map to obtain a matched building patch map; and obtaining the building change detection result of the first image based on the matched building patch map and the first building annotation map.

Description

Technical Field

[0001] This application relates to the field of high-resolution remote sensing image processing technology, and in particular to a method, system, electronic device and storage medium for detecting building changes. Background Technology

[0002] In related technologies, the new image is typically processed to ensure it matches the resolution of the original image, thus reducing the impact of temporal and sensor differences on the building change detection process. Then, a change detection algorithm is used to directly compare the annotation results of the entire new image with those of the entire original image to obtain the change annotations. However, considering only resolution cannot truly eliminate imaging angle differences. In two images, differences in imaging angle often cause differences in local parts of the image, resulting in differences in building change annotations. Such differences often increase the difficulty of comparing the new and original images and reduce the accuracy of the annotation results. Summary of the Invention

[0003] The main objective of this application is to propose a building change detection method, system, electronic device, and storage medium that can reduce the difficulty of comparing the first image and the reference image while minimizing the impact of differences in imaging angles, thereby improving the accuracy of the annotation results.

[0004] To achieve the above objectives, a first aspect of this application proposes a method for detecting building changes. The method includes: acquiring a first image to be detected; wherein the first image is an image formed by taking a picture of a region from a first angle; acquiring a reference image and a reference building annotation map corresponding to the reference image; wherein the reference image is an image formed by taking a picture of a region from a second angle; preprocessing the first image to make the image attribute parameters of the first image and the reference image consistent; sampling the reference image and the reference building annotation map corresponding to the reference image to obtain multiple sample blocks, and forming a training sample set based on the multiple sample blocks; wherein the sample block includes an image block within a preset range in the reference image and a reference building annotation map. The process involves: labeling buildings corresponding to the image patches; training a semantic segmentation model based on the training sample set to obtain a trained semantic segmentation model; inputting the first image into the trained semantic segmentation model to obtain a probability distribution map; labeling pixels with probability values ​​greater than a first preset threshold in the probability distribution map as buildings to generate a first building label map; acquiring multiple building patches of the reference image within the intersection of the reference image and the first image; performing similarity matching on the probability distribution map to obtain a matching result, and obtaining a matched building patch map based on the matching result; comparing the matched building patch map with the first building label map to obtain the building change detection result of the first image.

[0005] According to some embodiments of this application, after obtaining the building change detection result of the first image, the method further includes: performing a sample augmentation process, which includes: extracting the sample blocks within the intersection range of the reference image and the first image in the sample set; matching the sample blocks as building patches in the probability distribution map to obtain the rectangular region with the highest similarity to the building patches as the target rectangular region; calculating the proportion of the changed portion in the target rectangular region to the change in the target rectangular region based on the building change detection result; if the change proportion is less than a preset threshold, extracting a first image sub-block based on the preprocessed first image, and extracting a building label sub-block based on the first image sub-block at the position corresponding to the matched building patch; forming multiple augmentation pairs by combining the first image sub-block, the building label sub-block, the sample block, and the building label corresponding to the sample block; performing mixed sample augmentation on the first image sub-block and the building label sub-block based on the augmentation pairs to generate multiple augmented samples; and repeating the sample augmentation process.

[0006] According to some embodiments of this application, after generating multiple augmented samples, the method further includes: forming a sample training set based on the augmented samples, and training the semantic segmentation model; during the training process, adjusting the parameters of the semantic segmentation model to obtain a trained semantic segmentation model; performing building change detection on the first image and the reference image based on the semantic segmentation model to generate a building change verification result; and comparing and verifying the building change verification result with the building change detection result.

[0007] According to some embodiments of this application, the image attribute parameters include coordinate positioning, resolution, band, and spectral band; the preprocessing of the first image to make the image attribute parameters of the first image consistent with those of the reference image includes: adjusting the first image based on the reference image so that the coordinate positioning and resolution of the first image are consistent with those of the reference image; determining whether the first image contains infrared and near-infrared bands; if the first image contains infrared and near-infrared bands, extracting the green area of ​​the first image corresponding to the infrared and near-infrared bands to form a green area mask, and removing the green area mask; determining whether the band and spectral band of the first image and the reference image are the same; if the band and spectral band of the first image and the reference image are the same, performing relative radiometric correction on the first image; if the band or spectral band of the first image and the reference image are different, linearly quantizing the effective pixel value range of the first image so that the effective pixel value range of the first image is the same as that of the reference image.

[0008] According to some embodiments of this application, after making the effective pixel value range of the first image the same as the effective pixel value range of the reference image, the method further includes: checking the image attribute parameters of the first image and the reference image; if the image attribute parameters of the first image and the reference image are still different, taking image sample blocks from the first image and the reference image respectively to form a set, and training a style transfer network model; inputting the first image into the trained style transfer network model so that the image attribute parameters of the first image and the reference image are consistent.

[0009] According to some embodiments of this application, the step of marking pixels with probability values ​​greater than a first preset threshold in the probability distribution map as buildings and generating a first building annotation map includes: processing the probability distribution map with a conditional random field algorithm to obtain a continuously distributed probability distribution map; and selecting pixels with probability values ​​greater than the first preset threshold from the probability distribution map.

[0010] The pixels are expanded using morphological dilation filtering; pixels in the probability distribution map that are greater than a first preset threshold are labeled as buildings to obtain a first building label map.

[0011] According to some embodiments of this application, the step of performing similarity matching on the probability distribution map to obtain a matching result, and obtaining a matched building patch map based on the matching result, includes: within the intersection range, matching the building patches on the probability distribution map to obtain multiple candidate matching points; calculating the matching similarity between the building patches and the candidate matching points; constructing a weighted bipartite graph based on the matching similarity; and selecting the candidate matching point that uniquely corresponds to the building patch from the weighted bipartite graph to obtain the matched building patch map of the first image.

[0012] According to some embodiments of this application, within the intersection range, matching the building patches in the probability distribution map to obtain multiple candidate matching points includes: setting local search parameters and a minimum matching distance; taking an outer rectangle as a matching template block for each building patch, and selecting a matching reference point from the matching template block; within the intersection range, moving the matching template block in the probability distribution map according to the local search parameters, and calculating the similarity between the matching template block and the corresponding rectangular area in the probability distribution map in real time based on the matching reference point as a first similarity; and normalizing the value of the first similarity so that the measurement of each first similarity in the probability distribution map is consistent. The range is the same; in the probability distribution map, the rectangular regions with the first similarity greater than the similarity threshold are obtained, and the rectangular regions are divided according to a preset number of divisions to obtain multiple candidate matching positions; the similarity between the matching reference point and each candidate matching position is calculated as the second similarity, and the point with the largest second similarity is selected for each candidate matching position as the candidate matching point for each candidate matching position; for the candidate matching point, the point with the largest second similarity is selected as the reference point; the reference distance between the remaining candidate matching points and the reference point is calculated, and the candidate matching points with the reference distance less than the minimum matching distance are filtered out to obtain multiple filtered candidate matching points.

[0013] According to some embodiments of this application, after obtaining multiple filtered candidate matching points, the method further includes: arranging the multiple candidate matching points and performing distance-shift matching in the probability distribution map; if there is a first matching point in the probability distribution map whose distance to each candidate matching point is less than the minimum matching distance, then the first matching point is used as a new candidate matching point, and the matching template block corresponding to the first matching point is merged with the matching template block corresponding to the candidate matching point to obtain a second template block; the merged similarity between the first matching point and the second template block is calculated, and the value of the merged similarity is normalized so that the metric range of each merged similarity is the same in the probability distribution map.

[0014] According to some embodiments of this application, the step of calculating the similarity between the building patch and the candidate matching point, constructing a weighted bipartite graph based on the similarity, and selecting the candidate matching point uniquely corresponding to the building patch from the weighted bipartite graph to obtain a matched building patch map of the first image includes: calculating the similarity between the matching template block corresponding to the building patch and the candidate matching point; using the similarity as a weight, setting the building patch and the candidate matching point together to construct a weighted bipartite graph; filling the weighted bipartite graph with virtual weights so that the number of matching template blocks corresponding to the building patch and the number of candidate matching points are consistent; using the maximum weight as the matching, performing maximum weight matching on the weighted bipartite graph, and during the matching process, assigning each matching template block to a unique candidate matching point to generate a matched building patch map.

[0015] According to some embodiments of this application, the step of comparing the matched building patch map and the first building annotation map to obtain the building change detection result of the first image includes: mapping the matched building patch map to the first building annotation map, and matching the building patches in the matched building patch map with the building patches in the first building annotation map; if the building patch does not have a candidate matching point in the matched building patch map, then selecting the reference patch closest to the building patch in the matched building patch map, and calculating the offset of the building patch relative to the reference patch; in the first building annotation map, finding the region corresponding to the candidate matching point of the reference patch, and marking the corresponding region as a demolished building according to the offset; or, if the similarity between the building patch and the candidate matching point in the matched building patch map is less than a preset similarity threshold, then marking the region corresponding to the candidate matching point as a demolished building in the first building annotation map; or, if the candidate matching point of the building patch does not find a corresponding region in the first building annotation map, then marking the corresponding building in the first building annotation map as a newly added building.

[0016] A second aspect of this application proposes a building change detection system, the system comprising: a first image acquisition module for acquiring a first image to be detected; wherein the first image is an image formed by taking a picture of a region from a first angle; a reference image acquisition module for acquiring a reference image and a reference building annotation map corresponding to the reference image; wherein the reference image is an image formed by taking a picture of a region from a second angle; a preprocessing module for preprocessing the first image to make the image attribute parameters of the first image and the reference image consistent; a training sample set generation module for sampling the reference image and the reference building annotation map corresponding to the reference image to obtain multiple sample blocks, and forming a training sample set based on the multiple sample blocks; wherein the sample blocks include image blocks within a preset range in the reference image and building annotations in the reference building annotation map corresponding to the image blocks; and a semantic segmentation model acquisition module. The system includes the following modules: a module for training a semantic segmentation model based on the training sample set to obtain a trained semantic segmentation model; a probability distribution map acquisition module for inputting the first image into the trained semantic segmentation model to obtain a probability distribution map; a first building annotation map generation module for labeling pixels with probability values ​​greater than a first preset threshold in the probability distribution map as buildings to generate a first building annotation map; a building patch acquisition module for acquiring multiple building patches of the reference image within the intersection range of the reference image and the first image; a matching building patch map acquisition module for performing similarity matching on the probability distribution map to obtain a matching result and obtaining a matching building patch map based on the matching result; and a building change detection result acquisition module for comparing the matching building patch map and the first building annotation map to obtain the building change detection result of the first image.

[0017] A third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the building change detection method according to any one of the first aspects of this application.

[0018] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the building change detection method described in any one of the first aspects of this application.

[0019] The building change detection method, system, electronic device, and storage medium proposed in this application can obtain an accurate probability distribution map by training a semantic segmentation model. Within the intersection of the reference image and the first image, multiple building patches are selected, and each building patch is locally matched against the probability distribution map. A matching building patch map is obtained based on the matching results. This matching map is then matched against the first building annotation map to obtain the building change detection result for the first image, thus achieving an accurate building change detection result. The technical solution adopted in this application can significantly reduce the impact of large local differences caused by global image matching, reduce the difficulty of comparing the first image and the reference image, and improve the accuracy of the annotation results. Attached Figure Description

[0020] Figure 1-a This is a matching angle diagram provided in an embodiment of this application;

[0021] Figure 1-b This is yet another matching angle diagram provided in the embodiments of this application;

[0022] Figure 2 This is a schematic diagram of the building change detection system provided in the embodiments of this application;

[0023] Figure 3 This is a flowchart of the provided building change detection method;

[0024] Figure 4 This is a flowchart of the main process steps S101 to S110 provided in the embodiments of this application;

[0025] Figure 5 This is a flowchart provided in an embodiment of this application after obtaining the building change detection results of the first image;

[0026] Figure 6 This is a flowchart provided in the embodiments of this application after generating multiple expanded samples;

[0027] Figure 7 yes Figure 2 The flowchart of step S103 in the process;

[0028] Figure 8 This is a flowchart provided in an embodiment of the present application, showing how the effective range of the first image is linearly quantized to be the same as the effective range of the reference image;

[0029] Figure 9 yes Figure 2 The flowchart of step S107 in the process;

[0030] Figure 10 yes Figure 2 The flowchart of step S109 in the document;

[0031] Figure 11 This is a flowchart of step S701 provided in the embodiments of this application;

[0032] Figure 12 This is a schematic diagram of building patch matching provided in an embodiment of this application;

[0033] Figure 13 This is a flowchart provided in an embodiment of this application after obtaining multiple candidate matching points;

[0034] Figure 14 This is a flowchart of step S702 provided in the embodiments of this application;

[0035] Figure 15 This is a flowchart illustrating the process of selecting alternative matching points provided in an embodiment of this application;

[0036] Figure 16 This is a flowchart of step S1010 provided in the embodiments of this application;

[0037] Figure 17 This is a schematic diagram of the functional modules of the building change detection system provided in the embodiments of this application;

[0038] Figure 18 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0040] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0042] In high-resolution remote sensing monitoring in fields such as urban planning and management, a crucial approach is to utilize high-resolution remote sensing imagery to monitor the distribution and changes of buildings. Due to the wide scope and long time span of urban monitoring, a large amount of historical imagery and annotation results are often accumulated over long-term remote sensing applications. Using this historical imagery and annotation results, combined with new observational imagery for change updates, is an effective method for monitoring the distribution and changes of buildings.

[0043] In related technologies, often only the resolution and texture of the first image and the reference image are converted. Then, annotations are directly applied to the first and reference images, and the overall image is compared based on these annotations. However, differences in imaging angles often affect the overall image. For example, an image taken vertically downwards from the zenith may only show the tops of buildings, with the relative positions of the buildings and their bases aligned. Conversely, an image taken at a large angle from the zenith may show the side facades of multiple buildings, with significant differences in the relative positions of the tops and bases of buildings of different heights. Therefore, directly annotating buildings on the reference and first images and then comparing them to identify changing buildings often results in substantial errors.

[0044] Reference Figure 1-a and Figure 1-b , Figure 1-a and Figure 1-b The images show the matching results based on images captured from different shooting angles. As can be seen from the figures, due to differences in building height and shooting angle, the top offsets of each building vary. The overall matching results show offsets of different directions and magnitudes for each building, especially for the lower building in the lower right corner. Since matching for most taller buildings requires a larger offset, this lower building exhibits an even greater offset after matching. This indicates that overall matching alone produces a large error; therefore, local matching is necessary to reduce the error in detecting building changes.

[0045] Based on this, embodiments of this application provide a building change detection method, system, electronic device, and storage medium, which can greatly reduce the impact of large local differences caused by global image matching, reduce the difficulty of comparing the first image and the reference image, and improve the accuracy of the annotation results.

[0046] The building change detection method, system, electronic device, and storage medium provided in this application are specifically described through the following embodiments. First, the building change detection system in this application is described.

[0047] Please refer to Figure 2In some embodiments, the building change detection system includes: a control module 201, an image preprocessing module 202, a probability calculation module 203, a similarity calculation module 204, a candidate matching point selection module 205, and an image output module 206.

[0048] In some embodiments, the control module 201 can be the nerve center and command center of the system. The control module 201 can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions. Specifically, the control module 201 can control the image preprocessing module 202, the probability calculation module 203, the similarity calculation module 204, the alternative matching point selection module 205, and the image output module 206 according to the operation instructions.

[0049] In some embodiments, the image preprocessing module 202 is used to make the image attribute parameters of the first image consistent with those of the reference image, so as to facilitate subsequent analysis. In some embodiments, the image preprocessing module 202 may be a deep learning network model for style transfer, which can automatically make the image attribute parameters of two or more inputs consistent.

[0050] In some embodiments, the probability calculation module 203 is used to calculate the probability of buildings in an image. In some embodiments, the probability calculation module 203 can be a semantic segmentation model, which can input an image into the probability calculation module 203, input the probability of each part in the image, and then output a probability distribution map.

[0051] In some embodiments, the similarity calculation module 204 is used to calculate the similarity between the matching template block corresponding to the building patch and the rectangular region in the probability distribution map. Alternatively, it calculates the similarity between each candidate matching region and the matching template block.

[0052] In some embodiments, the candidate matching point selection module 205 calculates the minimum matching distance between the reference point and multiple candidate matching regions, and filters out candidate matching regions that do not meet the conditions based on the minimum matching distance, ultimately obtaining multiple candidate matching points. In some embodiments, the candidate matching point selection module 205 can also generate a weighted bipartite graph based on similarity, building patches, and multiple candidate matching points, ultimately obtaining a unique candidate matching point for each building patch.

[0053] In some embodiments, the image output module 206 is used to output a change-annotated map, or to output other change images.

[0054] The building change detection method in this application can be illustrated by the following examples.

[0055] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. For example, when obtaining user-stored data and user cached data access requests, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application need to obtain sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to a confirmation page. Only after obtaining the user's separate permission or consent is the necessary user-related data for the normal operation of embodiments of this application obtained.

[0056] Figure 3 This is an optional flowchart of the building change detection method provided in the embodiments of this application. Figure 2 The method may include, but is not limited to, steps S101 to S110.

[0057] Step S101: Obtain the first image to be detected; wherein, the first image is an image formed by taking a picture of the region from a first angle;

[0058] Step S102: Obtain the reference image and the corresponding reference building annotation; wherein, the reference image is an image formed by taking a picture of the area from a second angle;

[0059] Step S103: Preprocess the first image to make the image attribute parameters of the first image consistent with those of the reference image;

[0060] Step S104: Sample the reference image and the reference building annotation map corresponding to the reference image to obtain multiple sample blocks, and form a training sample set based on the multiple sample blocks; wherein, the sample block includes the image block within a preset range in the reference image and the building annotation map corresponding to the image block;

[0061] Step S105: Train the semantic segmentation model based on the training sample set to obtain the trained semantic segmentation model;

[0062] Step S106: Input the first image into the trained semantic segmentation model to obtain the probability distribution map;

[0063] Step S107: Mark the pixels with probability values ​​greater than the first preset threshold in the probability distribution map as buildings, and generate the first building annotation map;

[0064] Step S108: Within the intersection of the reference image and the first image, obtain multiple building patches of the reference image;

[0065] Step S109: Perform similarity matching on the probability distribution map to obtain the matching results, and obtain the matching building patch map based on the matching results;

[0066] Step S110: Compare the matched building patch map and the first building annotation map to obtain the building change detection result of the first image.

[0067] Please refer to Figure 4 , Figure 4 This is a flowchart illustrating the main processes of steps S101 to S110 of this application. In some embodiments, the reference image is a historically acquired image of the building, and the first image is a newly captured image of the building to be changed. Alternatively, the first image is a historically acquired image of the building, and the reference image is also a historically acquired image of the building; this application does not impose specific limitations on this. It is understood that the areas captured by the first image and the second image should be the same. For example, if the area captured by the first image is cell 1, then the area captured by the reference image should also be cell 1. It is understood that, to facilitate comparison between the first image and the reference image, the shooting angles can be made as similar as possible.

[0068] In some embodiments, image attribute parameters include coordinate positioning, resolution, band, and spectral band. It is understood that the first image can be transformed to make its spectral, texture, and other image attribute parameters consistent with those of the reference image, thereby facilitating the analysis of both the first and reference images.

[0069] In some embodiments, a deep learning network model for style transfer can be trained by extracting corresponding sample blocks from a first image and a reference image. During training, forward propagation is used to ensure that the image attribute parameters of the first image and the reference image are consistent, and the model performance is improved by continuously optimizing the model parameters. After the model is trained, the first image and the reference image can be directly input into the model, and the model can process the other image based on the image attribute parameters of one image. For example, the first image can be processed based on the attribute parameters of the reference image.

[0070] In some embodiments, the reference image and the corresponding reference building annotation can be sampled to form multiple sample blocks, and a training set can be formed based on the multiple sample blocks. The sample blocks include image blocks within a preset range in the reference image and building annotations in the reference building annotation.

[0071] In some embodiments, the semantic segmentation model can be trained using a training sample set to obtain a trained semantic segmentation model. In some embodiments, a first image can be input into the trained semantic segmentation model to obtain a probability distribution map. Specifically, the semantic segmentation model can be FCN (Fully Convolutional Networks), UNet, DeepLab, Mask R-CNN, etc., selected according to the characteristics of the data. For example, when the range of the reference image is large enough (e.g., the effective range is greater than 30000*30000 pixels), a large number of samples can be selected, and a semantic segmentation model with more parameters can be used, or a semantic segmentation algorithm using the Transformer model can be selected. In some embodiments, a sample set selected from the reference image and the original building annotation image can be input into the semantic segmentation model to train the semantic segmentation model.

[0072] Understandably, during the training of the semantic segmentation model based on the sample set, the original building annotation image can be compared with the reference image, the loss function can be calculated, and backpropagation can be continuously performed to optimize the model parameters. In some embodiments, after the semantic segmentation model is trained, the first image can be directly input into the model to obtain the model's output. Understandably, the model's output is the result of extracting and annotating the building pixels, i.e., a binary mask. Specifically, color encoding or grayscale encoding can be used to map the probability values ​​of the building pixels onto the image to generate a probability distribution map.

[0073] Understandably, the probability distribution map labels each target area with a corresponding probability, such as labeling a building area as 1 and a non-building area as 0. In some embodiments, a first preset threshold can be set, labeling areas with a probability greater than the first preset threshold as buildings and areas with a probability less than the first preset threshold as non-buildings. Understandably, the first preset threshold can be adjusted as needed. For example, the first preset threshold can be 0.5, and in the probability map, areas with a probability greater than 0.5 are labeled as buildings, and areas with a probability less than 0.5 are labeled as non-buildings, thereby generating a first building label map.

[0074] In some embodiments, to improve matching accuracy, the matching range can be limited. Specifically, building areas can be selected as the effective range in the first image and the reference image, and the intersection of the effective ranges of the first image and the reference image can be used as the intersection range, i.e., the detection range. This avoids interference from invalid ranges and saves computational resources. In some embodiments, multiple building patches of the reference image can be obtained within the intersection range. Specifically, if it is necessary to detect changes in the top of a building, then the building patch represents the top of the building, and one building patch represents one top of a building.

[0075] In some embodiments, building patches can be matched for similarity in a probability distribution map to determine whether there are similar regions in the first image. In some embodiments, similarity can be calculated in real time during the similarity matching process, and matching regions with similarity values ​​less than a similarity threshold can be filtered out.

[0076] In some embodiments, to improve the accuracy of matching, the rectangular region matched by the matching template block corresponding to the building patch can be divided. It is understood that the rectangular region is the area whose similarity to the matching template block is greater than a similarity threshold during the matching process. Specifically, the rectangular region can be divided to obtain multiple candidate matching regions, and the similarity between the multiple candidate matching regions and the matching template block can be calculated. The candidate matching region with the highest similarity is selected as the reference point from among the multiple candidate matching regions.

[0077] Understandably, candidate matching regions whose distance to the reference point is less than the minimum matching distance have high similarity to the reference point and can be filtered out, ultimately resulting in multiple candidate matching points. Understandably, a unique candidate matching point can be obtained from these multiple candidate matching points based on the weighted bipartite graph, and a matching building patch map can be generated in the image corresponding to the intersection range of the building patch and the candidate matching point.

[0078] In some embodiments, a matching result can be obtained by mapping a matching building patch map to a first building annotation map and matching the building patches in the matching building patch map to the first building annotation map. Specifically, if there is no candidate matching point for a building patch in the matching building patch map, a reference patch closest to the building patch is selected in the matching building patch map, the offset of the building patch relative to the reference patch is calculated, and in the first building annotation map, the region corresponding to the candidate matching point of the reference patch is found, and the corresponding region is marked as a demolished building according to the offset. Alternatively, if the similarity between a building patch and a candidate matching point in the matching building patch map is less than a preset similarity threshold, the region corresponding to the candidate matching point is marked as a newly added building in the first building annotation map. Alternatively, if no corresponding region is found for the candidate matching point of the building patch in the first building annotation map, the building in the first building annotation map is marked as a newly added building in the reference image.

[0079] Understandably, a sample augmentation process can be performed on the first image and the reference image. By iteratively training the semantic segmentation model, the parameters of the semantic segmentation model can be adjusted according to the training situation to generate building change verification results. The building change verification results are then compared with the original building change detection results to verify the building change detection results. In other words, the effectiveness of the semantic segmentation model is verified, thereby continuously improving the accuracy of detection.

[0080] Understandably, this application significantly reduces the impact of large local differences caused by global image matching by performing local matching on architectural images, reduces the difficulty of comparing the first image with the reference image, and improves the accuracy of the annotation results.

[0081] Please refer to Figure 5 In some embodiments, after obtaining the building change detection result of the first image, the process may include, but is not limited to, steps S201 to S207:

[0082] Step S201: Perform the sample augmentation process, which includes: extracting sample blocks within the intersection range of the reference image and the first image from the sample set;

[0083] Step S202: Match the sample block as a building patch in the probability distribution map to obtain the rectangular region with the highest similarity to the building patch as the target rectangular region.

[0084] Step S203: Based on the building change detection results, calculate the proportion of the changed portion in the target rectangular area to the total change in the target rectangular area;

[0085] Step S204: If the change ratio is less than a preset threshold, then extract the first image sub-block based on the preprocessed first image, and extract the building annotation sub-block based on the position of the first image sub-block corresponding to the matching building patch.

[0086] Step S205: The first image sub-block, the building annotation sub-block, the sample block, and the building annotations corresponding to the sample block are combined into multiple augmentation pairs;

[0087] Step S206: Based on the augmentation pair, perform mixed sample augmentation on the first image sub-block and the building annotation sub-block to generate multiple augmented samples;

[0088] Step S207: Repeat the sample expansion process.

[0089] Please refer to Figure 4 In some embodiments, the sample set can be expanded to increase the number of training iterations of the semantic segmentation model, thereby improving the model's accuracy and enabling it to output the first building annotation map more accurately. It is understood that a sample block refers to a sample formed by the building annotation of a sample and the corresponding patch in the reference image.

[0090] It is understood that the intersection of the first image and the reference image includes the building area shared by both the first image and the reference image. In some embodiments, sample blocks falling within the intersection of the reference image and the first image can be taken, and the coordinates of the sample blocks can be mapped to a probability distribution map. Each sample block corresponds to a building patch, and the building patches are matched in the probability distribution map. The rectangular areas matched in the probability distribution map are then divided into four, six, eight, or other divisions.

[0091] The rectangular regions matched by building patches in the probability distribution map are summed to obtain a total rectangular block. The proportion of each changed region in the total rectangular block is calculated from the change annotation map. If the proportion is less than a preset value, it indicates that the change in that region is small, and a new sample block can be extracted. In some embodiments, a first image sub-block can be extracted from the preprocessed first image as a new sample block, and a building annotation sub-block corresponding to the first image sub-block can be extracted from the second building annotation map as the original sample block. Then, an augmentation pair is formed based on the new sample block and the building annotation sub-block.

[0092] In some embodiments, the coordinates of the first image sub-block and the building annotation sub-block can be obtained, a parameter λ can be selected according to the beta distribution, and then the augmented pair can be augmented, such as: the first image sub-block is (x s ,y s The building label sub-block is (x n ,y n ), taking the parameter λ according to the beta distribution, generate

[0093]

[0094]

[0095] Understandably, by adjusting the parameter λ of the beta distribution, multiple new samples can be obtained. In some embodiments, the sample set can be expanded in the manner described above to continuously train the semantic segmentation model, optimize the model's parameters, and improve the model's accuracy in image recognition.

[0096] Please refer to Figure 6 In some embodiments, after generating multiple augmented samples, steps S301 to S304 are included, but are not limited to:

[0097] Step S301: Based on the expanded samples, form a sample training set and train the semantic segmentation model;

[0098] Step S302: During the training process, the parameters of the semantic segmentation model are adjusted to obtain the trained semantic segmentation model.

[0099] Step S303: Based on the semantic segmentation model, perform building change detection on the first image and the reference image to generate building change verification results;

[0100] Step S304: Compare and verify the building change verification results with the building change detection results.

[0101] In some embodiments, the semantic segmentation model can be trained using expanded samples, and the parameters of the semantic segmentation model can be continuously adjusted during training to obtain a well-trained semantic segmentation model. It is understood that the sample training set can be divided into a training set, a validation set, and a test set for training the semantic segmentation model and verifying its training effect. It is understood that during the training of the semantic segmentation model based on the sample set, a loss function is calculated, and backpropagation is continuously performed to optimize the model parameters. In some embodiments, after the semantic segmentation model is trained, the first image can be directly input into the semantic segmentation model to obtain the output result. It is understood that the output result of the semantic segmentation model is the result of extracting and labeling the pixels of the building, i.e., a binary mask. Specifically, color encoding or grayscale encoding can be used to map the probability values ​​of the building pixels onto the image to obtain the probability distribution map output by the semantic segmentation model after sample expansion.

[0102] In some embodiments, building patches of the reference image can be matched in a probability distribution map, and a matching building patch map can be generated by calculating similarity. The matching building patch map is then compared with a first building annotation map to obtain the building change verification result. It is understood that the building change verification result can be compared with the building change detection result for analysis. If the difference between the building change verification result and the building change detection result is large, it indicates that the original semantic segmentation model needs further training and parameter tuning. If the difference between the building change verification result and the building change detection result is small, it indicates that the building change verification result is accurate. It is understood that multiple verifications can be performed based on expanded samples to ensure the accuracy of the building change detection result.

[0103] In some embodiments, image attribute parameters include coordinate positioning, resolution, band, and spectral band.

[0104] Please refer to Figure 7 In some embodiments, step S103 includes, but is not limited to, steps S401 to S403:

[0105] Step S401: Using the reference image as a reference, adjust the first image so that the coordinate positioning and resolution of the first image are consistent with those of the reference image;

[0106] Step S402: Determine whether the first image contains infrared and near-infrared bands. If the first image contains infrared and near-infrared bands, extract the green area of ​​the first image corresponding to the infrared and near-infrared bands to form a green area mask, and remove the green area mask.

[0107] Step S403: Determine whether the bands and spectral bands of the first image and the reference image are the same. If the bands and spectral bands of the first image and the reference image are the same, perform relative radiometric correction on the first image. If the bands or spectral bands of the first image and the reference image are not the same, perform linear quantization on the effective pixel value range of the first image so that the effective pixel value range of the first image is the same as the effective pixel value range of the reference image.

[0108] In some embodiments, to facilitate comparative analysis between the first image and the reference image, the first image can be adjusted to ensure that its coordinate positioning and resolution are consistent with those of the reference image. Specifically, computer vision algorithms, such as SIFT, SURF, or ORB, can be used to extract feature points from the reference image and the first image, and these feature points can be matched. It is understood that these feature points can be corner points, edges, or textures in the image. Then, based on the matched feature points, a registration algorithm is used to calculate the geometric transformation relationship between the reference image and the first image, such as translation, rotation, scaling, and distortion. Further, based on the calculated geometric transformation relationship, the first image is corrected, and image processing techniques are used to map the first image into a space consistent with the coordinate positioning of the reference image. In some embodiments, if the resolutions of the first image and the reference image are inconsistent, image interpolation methods can be used to adjust the first image so that its resolution is consistent with that of the reference image.

[0109] In some embodiments, if the first image contains standard red and near-infrared bands, it indicates that the first image contains green land, and due to seasonal or other reasons, the growth status of the green land differs greatly from that shown in the reference image. The green land should be extracted so that the green land portion can be excluded in the subsequent process.

[0110] In some embodiments, the vegetation index can be used to measure the proportion of vegetation. The formula for calculating the vegetation index is as follows:

[0111] Vegetation Index = (Near-infrared - Infrared) / (Near-infrared + Infrared)

[0112] In some embodiments, green areas can be extracted based on vegetation indices to form a green area mask, where green areas are set to 1 and non-green areas to 0, or green areas are white and non-green areas are black. In some embodiments, morphological operations or edge detection methods can be used to further process the green areas to obtain a more accurate green area mask.

[0113] It is understandable that the wavebands of an image can be interpreted as different frequency ranges within the image, which can be used to describe the spectral information in the image; while the spectral bands of an image refer to different wavelength ranges contained in the image. In some embodiments, if the first image and the reference image have the same number of wavebands and the same or similar spectral bands, relative radiometric correction is directly performed on the first image. Specifically, this can be achieved by analyzing the grayscale distribution in the reference image, calculating the average value of each grayscale level, and then correcting the first image based on the grayscale values ​​to make the grayscale distribution of the first image similar to that of the reference image. In some embodiments, histograms can be used for relative radiometric correction. In some embodiments, the first image can also be sharpened, denoised, or otherwise processed.

[0114] Understandably, if a green area mask is extracted from the first image, it can be excluded before amplitude correction. Specifically, the portion of the green area mask marked as green area can be excluded.

[0115] In some embodiments, if the first image and the reference image have different bands, or the first image and the reference image have different spectral bands, or the difference is large, then the effective range of the first image can be linearly quantized to be the same as the effective range of the reference image.

[0116] Specifically, the quantification method is expressed as Where V is the initial pixel value of the first image, V scaled N represents the pixel transformation result of the first image. max N represents the maximum value of the first image band. min B is the minimum value of the first image band. max To refer to the maximum effective value of the corresponding band of the image, B min This refers to the minimum effective value of the corresponding band in the reference image. It is understood that if a certain band in the first image does not have a corresponding band in the reference image, then the band in the first image will not be processed. Alternatively, the bands in both the first image and the reference image can be normalized to between 0 and 1. This application embodiment does not impose specific limitations on this.

[0117] Please refer to Figure 8 After linearly quantizing the effective range of the first image to be the same as the effective range of the reference image, the method further includes:

[0118] Step S501: Check the image attribute parameters of the first image and the reference image. If the image attribute parameters of the first image and the reference image are still different, take image sample blocks from the first image and the reference image respectively to form a set and train the style transfer network model.

[0119] Step S502: Input the first image into the trained style transfer network model so that the image attribute parameters of the first image and the reference image are consistent.

[0120] In some embodiments, if the first image and the reference image still have significant differences, sample block sets are taken from the first image and the reference image respectively, a deep learning network for style transfer is trained, and the model parameters are continuously adjusted during the training process to optimize the model performance.

[0121] In some embodiments, a trained model can be used to transform the first image so that the spectral and texture representation of the first image is similar to that of the reference image.

[0122] It is understood that the sample block is a rectangular block within the effective range of the first image and the reference image, which can be randomly selected or selected with overlap at a fixed step size. In some embodiments, blocks of the same size can be taken from the same position in the first image and the reference image to form a sample pair.

[0123] Understandably, during model training, each pair of sample blocks uses only the corresponding sample block as the forward or reverse generation target, and different sample pairs are not used interchangeably to ensure the accuracy of model training. For example, in a first image, there is a first sample block; in a reference image, there is a second sample block corresponding to the first sample block and a third sample block not corresponding to the first sample block. During model training, only the first and second sample blocks are used to form sample pairs, with the first and second sample blocks serving as mutual generation targets for training, to ensure consistent resolution and texture representation between them. However, the first sample block is not paired with the third sample block, and they are not combined for training, to guarantee the accuracy of model training.

[0124] In some embodiments, the style transfer network model can be selected based on the sample blocks of the model. For example, if the sample block has few data bands and a small quantization range, such as only three bands (red, green, and blue) and is quantized to byte size, then a generative adversarial network (GAN) that only converts colors can be selected. Under different sample conditions, different style transfer network models can be selected, such as a dual adversarial network (DualGAN) or a Diffusion network. This application embodiment does not impose specific limitations on this.

[0125] Please refer to Figure 9 In some embodiments, step S107 includes, but is not limited to, steps S601 to S604:

[0126] Step S601: Process the probability distribution map using the conditional random field algorithm to obtain a continuous probability distribution map;

[0127] Step S602: Select pixels with probability values ​​greater than the first preset threshold from the probability distribution map;

[0128] Step S603: Dilate the pixels using morphological dilation filtering;

[0129] Step S604: Mark the pixels in the probability distribution map that are greater than the first preset threshold as buildings to obtain the first building annotation map.

[0130] In some embodiments, the first image can be input into a trained semantic segmentation model to obtain a probability distribution map. Specifically, the semantic segmentation model can be FCN (Fully Convolutional Networks), UNet, DeepLab, Mask R-CNN, etc., selected based on the characteristics of the data. For example, when the range of the reference image is large enough (e.g., the effective range is greater than 30000*30000 pixels), a large number of samples can be selected, and a semantic segmentation model with more parameters can be used, or a semantic segmentation algorithm using the Transformer model (mutual inductance model) can be chosen. In some embodiments, a sample set selected from the reference image and the original building annotation image can be input into the semantic segmentation model to train the semantic segmentation model.

[0131] Understandably, during the training of the semantic segmentation model based on the sample set, the original building annotation image can be compared with the reference image, the loss function can be calculated, and backpropagation can be continuously performed to optimize the model parameters. In some embodiments, after the semantic segmentation model has been trained, the first image can be input into the model to obtain the model's output. Understandably, the model's output is the result of extracting and annotating the building pixels, i.e., a binary mask. Specifically, color encoding or grayscale encoding can be used to map the probability values ​​of the building pixels onto the image to generate a probability distribution map.

[0132] Understandably, in the probability distribution map, pixel values ​​between 0 and 1 represent the probability that the corresponding area is the target building. The closer the probability is to 1, the higher the probability that the corresponding area is the target building. Understandably, the target building can refer to multiple buildings, a single building, or a portion of a building, such as the top or bottom of a building. In some embodiments, a preset threshold can be set to label areas in the probability distribution map with values ​​greater than the preset threshold as buildings. Understandably, the preset threshold can be set according to actual conditions; for example, it can be set to less than 0.5. Labeling the probability distribution map according to the preset threshold yields the first building label map of the first image. Understandably, if green areas are labeled and extracted, since green areas do not belong to building areas, the probability value corresponding to the green area mask needs to be modified to 0.

[0133] Understandably, since the first image and the reference image may differ in resolution, texture, etc., the resulting probability distribution map after inputting the first image into the model may still differ significantly from the image representation of the sample. The predicted probability will generally be lower, and the building area will appear coarse and fragmented. Therefore, the probability distribution map can be processed using a standard binary potential conditional random field algorithm to obtain a more spatially continuous probability distribution map. In some embodiments, grayscale morphological dilation filtering can also be applied to the probability distribution map to appropriately expand areas with larger probability values. It is understood that applying conditional random field algorithms and dilation filtering to the probability distribution map can remove noise, resulting in a clearer and more accurate probability distribution map.

[0134] In some embodiments, solving the conditional random field of the binary potential can only obtain the probability of the first image without calculating the annotation result; in some embodiments, the pydensecrf algorithm library (dense conditional random field) can also be used to assign a label to each pixel of the probability distribution map, and use the image annotation model of the conditional random field to annotate the probability distribution map and return the probability matrix to obtain a more spatially continuous probability distribution map.

[0135] Understandably, the size and shape of the structuring element in dilation filtering determine the filtering effect. Specifically, choosing a structuring element with a smaller side length better preserves the detailed information in the probability distribution map while also better removing noise and small interferences, while a larger side length makes the building areas in the probability distribution map more continuous and complete. Therefore, the structuring element can be a square with a side length between 3 and 9, allowing the probability distribution map to enhance the overall shape of the building areas while preserving details. Understandably, the structuring element can also be adjusted according to needs, such as choosing a rectangle or square with a side length less than 3 or greater than 9.

[0136] In some embodiments, after processing the probability distribution map, pixels with a probability greater than a first preset threshold can be labeled as buildings to obtain a first building label map. It is understood that the first preset threshold can be set empirically; for example, the first preset threshold can be 0.4. Therefore, when generating the first building label map, pixels with a probability greater than 0.4 can be labeled as buildings, and pixels with a probability less than 0.4 can be labeled as non-buildings.

[0137] Please refer to Figure 10 In some embodiments, step S109 includes, but is not limited to, steps S701 to S702:

[0138] Step S701: Within the intersection range, match the building patches on the probability distribution map to obtain multiple candidate matching points;

[0139] Step S702: Calculate the matching similarity between the building patch and the candidate matching point, construct a weighted bipartite graph based on the matching similarity, and select the candidate matching point that uniquely corresponds to the building patch from the weighted bipartite graph to obtain the matching building patch map of the first image.

[0140] In some embodiments, the effective range of the reference image and the first image is obtained, that is, the building range of the reference image and the first image is obtained. In some embodiments, the intersection of the reference image and the first image can be calculated based on the effective range of the reference image and the first image. It is understood that the intersection generally represents the same part of the reference image and the first image, and can be used as the effective range for updating building changes.

[0141] In some embodiments, the building patch can be a patch formed from a building area where changes are to be detected. For example, if changes in the building roof are to be detected, then the building patch can be a patch formed from the building roof. It is understood that the building patch can be defined, such as defining the entire building or other parts thereof. In this application, the building roof is used as the building patch to illustrate embodiments.

[0142] In some embodiments, building patches can be matched on a probability map within the intersection range to obtain multiple candidate matching points. It is understood that during the matching process, the similarity between building patches and candidate matching points should be calculated in real time, and points with similarity greater than a similarity threshold should be selected as candidate matching points.

[0143] Understandably, setting multiple alternative matching points instead of a single one can avoid randomness during the matching process, prevent matching errors, and improve the accuracy of the final alternative matching point selection.

[0144] In some embodiments, similarity can be used as a weight to form a weighted bipartite graph together with building patches and candidate matching points. The candidate matching point that uniquely corresponds to the building patch is selected through the weighted bipartite graph. Based on the building patch and the uniquely corresponding candidate matching point, a matching building patch map of the first image is generated for subsequent change detection of the first image.

[0145] Please refer to Figure 11 In some embodiments, step S701 includes, but is not limited to, steps S801 to S808:

[0146] Step S801: Set local search parameters and minimum matching distance;

[0147] Step S802: Take the outer rectangle of each building patch as the matching template block, and select the matching reference point from the matching template block;

[0148] Step S803: Within the intersection range, the matching template block is moved and matched in the probability distribution map according to the local search parameters, and the similarity between the matching template block and the corresponding rectangular area in the probability distribution map is calculated in real time based on the matching reference point as the first similarity.

[0149] Step S804: Normalize the value of the first similarity so that each first similarity has the same measurement range in the probability distribution map.

[0150] Step S805: In the probability distribution map, obtain the first rectangular region with a similarity greater than the similarity threshold, and divide the rectangular region according to the preset number of divisions to obtain multiple candidate matching positions;

[0151] Step S806: Calculate the similarity between the matching reference point and each candidate matching position as the second similarity. Select the point with the largest second similarity for each candidate matching position as the candidate matching point for each candidate matching position.

[0152] Step S807: Select the point with the second highest similarity as the reference point from the candidate matching points;

[0153] Step S808: Calculate the baseline distance between the remaining candidate matching points and the baseline point, and filter out the candidate matching points whose baseline distance is less than the minimum matching distance to obtain multiple filtered candidate matching points.

[0154] In some embodiments, local search parameters and a minimum matching distance are set. Specifically, the local search parameters are used to limit the matching step size, and the minimum matching distance is used to filter candidate matching points. The values ​​of the local search parameters and the minimum matching distance can be set as needed.

[0155] In some embodiments, the matching template block is matched in the probability distribution map according to local search parameters. The matching method can be to use the upper left corner of the matching template block as the matching reference point and translate from left to right, from top to bottom, or from right to left, from bottom to top, or to spread and translate from the center to the surrounding areas, etc. The embodiments of this application do not impose specific limitations on this.

[0156] In some embodiments, to improve matching speed, the probability distribution map can be adjusted to have the same orientation as the image containing the building patch. For example, if the top of the building is at the top and the bottom at the bottom in the image containing the building patch, then the top of the building should also be at the top and the bottom at the bottom in the probability distribution map. It is understood that, to improve matching efficiency, the translation method during similarity matching should correspond to the order in which building patches are taken from the reference image. Specifically, assuming the building patches are taken from left to right and top to bottom, starting from the top left corner, then the probability distribution map should also be matched from left to right and top to bottom, starting from the top left corner.

[0157] Understandably, during the matching process, the similarity of the matching template block to the rectangular region matched in the probability distribution map can be calculated in real time, and all calculated similarities can be normalized to make the similarity calculation in the probability distribution map have a unified standard, so as to facilitate the comparison between similarities and filter out the matching rectangular regions.

[0158] Understandably, since the cosine similarity algorithm is calculated based on the length and angle of the vectors, regions with probabilities all close to 0 are also considered highly similar. Therefore, this application does not use the cosine similarity algorithm to calculate the similarity between the matching template block and the rectangular region. It is also understandable that in similarity calculations, 0 represents no correlation, positive values ​​represent positive correlation, and negative values ​​represent negative correlation. Therefore, the correlation coefficient cannot be used to calculate similarity either.

[0159] In some embodiments, similarity can be calculated using methods such as absolute difference sum, Euclidean distance, or similar metrics.

[0160] In some embodiments, the process of using absolute difference and calculating similarity is described. First, the pixels of the labeled patch mask are summed, calculated using the formula: ∑ i |1-p i | where a valid pixel value is labeled as 1, and the probability value corresponding to a valid pixel is labeled as p. i .

[0161] In some embodiments, the matching template block can search the entire region of the probability distribution map to find rectangular regions with high similarity. In some embodiments, in the probability distribution map, the closer the probability value is to 1, the more likely the corresponding pixel is a building area or a specific part of a building, such as the top of a building. Therefore, to improve matching efficiency, regions with probability values ​​close to 1 can be matched first, and after no matching of the corresponding rectangular region is found, regions with lower probability values ​​can be matched. In some embodiments, a matching threshold can be set, such as prioritizing matching in regions with probability values ​​greater than 0.4. The matching threshold can be adjusted according to actual needs, such as 0.3, 0.5, etc.

[0162] In some embodiments, successfully matched rectangular regions can be marked as matched to prevent another matching template block from matching again, thus improving matching efficiency. Alternatively, no marking can be performed, and each matching template block can be searched across the entire probability analysis graph.

[0163] In some embodiments, the similarity values ​​calculated during the matching process can be normalized to eliminate the influence of image size and background differences on the similarity. Specifically, the similarity can be uniformly transformed to values ​​within [0,1] to facilitate comparison and understanding between different similarity values. It can be understood that a similarity of 1 indicates that the matching template block and the corresponding rectangular region are completely similar, while a similarity of 0 indicates that the matching template block and the corresponding rectangular region are completely dissimilar.

[0164] For example, if the absolute difference is used to calculate similarity, then the similarity can be normalized using the following formula:

[0165]

[0166] In the formula, N is the number of valid pixels, the valid pixel value is marked as 1, and the probability value corresponding to the valid pixel is marked as p. i .

[0167] In some embodiments, to find the region that best matches the matching template block in each rectangular region and use it as a candidate matching point to improve matching accuracy, the rectangular region can be divided into 4 blocks, each of which is a candidate matching region. The similarity between each position in each of the 4 candidate matching regions and the matching template block is calculated. That is, the similarity between each position in each of the 4 candidate matching regions and the matching reference point is calculated, and the point with the highest similarity greater than a preset threshold in the candidate matching region is selected as the candidate matching point. It can be understood that a rectangular region is generally a pixel block, which consists of 2r+1 rows and 2r+1 columns of pixels, where r represents the number of pixels used to limit the search range. Therefore, the number of pixels in each candidate matching region should not be less than r rows and r columns, and the row and column where the center pixel is located are assigned to any neighboring block. In some embodiments, the rectangular region can also be divided into more or fewer than 4 blocks, and this application embodiment does not impose specific limitations on this.

[0168] In some embodiments, the similarity between each position in each candidate matching region and the matching template block can be calculated. The most similar (i.e., the one with the highest similarity) is selected as the candidate matching point, and the similarity of all candidate matching regions is compared. The candidate matching point with the highest similarity is selected as the reference point. Specifically, during the similarity calculation process, the similarity value is still normalized. The method of normalization has been described above and will not be repeated here. It can be understood that if multiple candidate matching points have the same similarity to the matching template block, the candidate matching region closest to the center point is selected as the reference position.

[0169] In some embodiments, after selecting a reference point from multiple candidate matching points, the reference point is used as one candidate matching point. The reference distances between the remaining candidate matching points and the reference point are calculated, and the calculated reference distances are compared with the minimum matching distance. It is understood that candidate matching points with distances less than the minimum matching distance typically overlap with the reference point and can therefore be filtered out. Candidate matching points with distances greater than the minimum matching distance are retained.

[0170] Please refer to Figure 12In some embodiments, r is a local search parameter. An outer rectangle is taken for each building patch (i.e., the patch itself) to facilitate matching with rectangular areas of the same size in the probability distribution map based on the outer rectangle, thus obtaining a matching result. In some embodiments, within the outer rectangle, the building portion is set to 1, and the non-building portion is set to 0, forming a matching template block. It is understood that the number of candidate matching regions can be set according to actual needs. For example, if there are four candidate matching regions, namely candidate matching region 1, candidate matching region 2, candidate matching region 3, and candidate matching region 4, and point 2 in candidate matching region 2 has the highest similarity to the matching template block, then point 2 is used as the reference position. In some embodiments, assuming the minimum matching distance is 0.1297mm (for reference only), if the reference distances between point 1 in candidate matching region 1, point 3 in candidate matching region 3, and point 4 in candidate matching region 4 and the reference position point 2 are calculated to be 0.2297mm, 0.3297mm, and 0.0996mm respectively, and the distances between points 1 and 3 and the reference position are greater than the minimum matching distance, then they are retained as candidate matching points, and the distance between point 4 and the reference position is less than the minimum matching distance, then point 4 is filtered out.

[0171] Understandably, after filtering out all candidate matching positions whose baseline distance is less than the minimum matching distance, points 1, 2, and 3 are retained as candidate matching points.

[0172] Please refer to Figure 13 In some embodiments, after obtaining multiple candidate matching points, the process further includes, but is not limited to, steps S901 to S903:

[0173] Step S901: Arrange multiple candidate matching points and perform distance movement matching in the probability distribution map;

[0174] Step S902: If there is a first matching point in the probability distribution map whose distance to each candidate matching point is less than the minimum matching distance, then the first matching point is taken as a new candidate matching point, and the matching template block corresponding to the first matching point is merged with the matching template block corresponding to the candidate matching point to obtain the second template block.

[0175] Step S903: Calculate the merged similarity between the first matching point and the second template block, and normalize the value of the merged similarity so that the merged similarity has the same measurement range in the probability distribution map.

[0176] In some embodiments, candidate matching points can be sorted in descending or ascending order of similarity. Then, distance judgment and merging of candidate matching points are performed on the probability distribution graph. Specifically, the distance between a candidate matching point and subsequent candidate matching points is calculated. If the calculated distance is less than the minimum matching distance, the candidate matching point is taken as the first matching point and merged into the current candidate matching point. It can be understood that the distance between the current candidate matching point and subsequent candidate matching points is calculated using the position in the probability graph, i.e., the distance between the top-left corner points.

[0177] For example, if the three candidate matching points are sorted from highest to lowest similarity to the matching reference points of the matching template block, the sorted result is candidate matching point 2, candidate matching point 3, and candidate matching point 1. After candidate matching point 1, the distance between each candidate matching point and candidate matching point 1 can be calculated. If there is a candidate matching point whose distance to candidate matching point 1 is less than the minimum matching distance, such as candidate matching point 2, then candidate matching point 2 is merged into the matching template block containing candidate matching point 1 as the first matching point, becoming a new candidate matching point for the matching template block. After merging, candidate matching point 1 corresponds to both the original matching template block of candidate matching point 1 and the original matching template block of candidate matching point 2.

[0178] In some embodiments, after using the first matching point as a candidate matching point for the matching template block, the similarity between the first matching point and the matching template block can be calculated, and the similarity values ​​can be normalized to ensure that the measurement range of each similarity is the same in the probability distribution graph. It is understood that the distance judgment and merging of candidate matching points is to find overlapping or nearly overlapping candidate matching positions and remove inconsistent and redundant candidate matching positions. After merging, one template block can correspond to multiple candidate matching points, and multiple template blocks can correspond to one candidate matching point. The set of template blocks and the set of candidate matching points, weighted by similarity, constitute a weighted bipartite graph.

[0179] Please refer to Figure 14 In some embodiments, step S702 includes, but is not limited to, steps S1001 to S1003:

[0180] Step S1001: Calculate the similarity between the matching template block corresponding to the building patch and the candidate matching point. Using the similarity as the weight, the building patch and the candidate matching point set are combined to form a weighted bipartite graph.

[0181] Step S1002: Fill virtual weights into the weighted bipartite graph to make the number of matching template blocks and candidate matching points corresponding to building patches consistent.

[0182] Step S1003: Using the maximum weight as the match, perform maximum weight matching on the weighted bipartite graph, and during the matching process, make each matching template block correspond to a unique alternative matching point to generate a matching building patch map.

[0183] In some embodiments, similarity can be used as the weight of edges, and the relationship between building patches and candidate matching points can be represented as weighted edges. For example, suppose there are 3 building patches and 4 candidate matching points. The similarities between building patch 1 and its corresponding 4 candidate matching points are 0.9, 0.5, 0.6, and 0.8, respectively; the similarities between building patch 2 and its corresponding 4 candidate matching points are 0.71, 0.7, 0.92, and 0.65, respectively; and the similarities between building patch 3 and its corresponding 4 candidate matching points are 0.6, 0.6, 0.8, and 0.5, respectively. Then, the building patches and candidate matching points can be used as vertices of a weighted bipartite graph, and the similarity can be used as edges to form a weighted bipartite graph.

[0184] In some embodiments, the maximum weight matching of a weighted bipartite graph can be calculated using the Kuhn-Munkres algorithm (Hungarian algorithm) or the augmenting path algorithm, based on the weight matrix of the weighted bipartite graph.

[0185] Taking the Kuhn-Munkres algorithm as an example, to maintain data consistency and balance, virtual weights and the number of building patches and candidate matching points are generally set to be equal. Specifically, the matching state of all edges is first initialized to unmatched. Then, starting from any unmatched building patch, depth-first search (DFS) or breadth-first search (BFS) is used to find augmenting paths. The matching on the augmenting path is adjusted so that each building patch on the path alternates between matching candidate matching points. In essence, an augmenting path is a path that starts from an unmatched building patch, passes through unmatched candidate matching points, then through matched candidate matching points, and finally returns to an unmatched building patch. While searching for augmenting paths, the matching weights are updated: the edge weights at odd-numbered positions on the path are added to the minimum weight in the path, and the edge weights at even-numbered positions are subtracted from the minimum weight in the path. This search is repeated until no augmenting path can be found. At this point, each building patch corresponds to only one candidate matching point, and the overall weight reaches its maximum. The matched building patch graph is then output.

[0186] Understandably, since virtual weights do not represent true similarity, they should be removed from the final result. It is also understandable that some building patches may have very low or zero weights, resulting in insufficient alternative matching points. Therefore, some building patches may not have corresponding alternative matching points. In some embodiments, if a building patch has multiple alternative matching points, once this building patch corresponds to the unique alternative matching point, other alternative matching points may not have corresponding building patches.

[0187] Figure 15 This paper introduces the process of matching building patches in a probability distribution map to obtain a matching patch map. An example is provided below. Figure 15 The following explanation is provided. In some embodiments, building patches (one patch corresponding to the top of each building) of the reference image can be matched by moving distance in a probability distribution map. This matching can be performed from top to bottom and from left to right, using pre-set local search parameters (r) and minimum matching distance (d), which can be set as needed. In some embodiments, a bounding rectangle can be defined for each building patch to facilitate matching and similarity calculation with rectangular areas in the probability distribution map.

[0188] In some embodiments, similarity can be calculated using absolute difference sums, Euclidean distance, etc., and then the calculated similarity can be normalized to unify the similarity measurement value. Further, rectangular regions with similarity greater than a similarity threshold can be selected in the probability distribution map, and the rectangular regions can be divided to obtain candidate matching positions, such as dividing them into 4 regions or 8 regions, etc. The embodiments of this application do not impose specific limitations on this.

[0189] Understandably, the top-left corner of the matching template block can be used as a matching reference point, and similarity can be calculated in the probability distribution map. Alternatively, the bottom-right corner, top-right corner, etc., can also be used as matching reference points. Taking the rectangular area divided into 4 regions as an example, the similarity of the matching reference point in these 4 regions (4 candidate matching positions) is calculated to obtain the similarity between the matching reference point and multiple points in each candidate matching position. For example, candidate matching position 1 has: point 1, similarity 0.2, point 2, similarity 0.9; candidate matching position 2 has: point 3, similarity 0.6, point 4, similarity 0.8; candidate matching position 3 has: point 5, similarity 0.7, point 6, similarity 0.6; candidate matching position 4 has: point 7, similarity 0.35, point 8, similarity 0.95.

[0190] Understandably, for each candidate matching position, the one with the highest similarity is taken as the candidate matching point for the corresponding candidate matching position. That is, point 2 of candidate matching position 1 has a similarity of 0.9; point 4 of candidate matching position 2 has a similarity of 0.8; point 5 of candidate matching position 3 has a similarity of 0.7; and point 8 of candidate matching position 4 has a similarity of 0.95.

[0191] For the multiple candidate matching positions mentioned above, the one with the highest similarity is selected as the reference point, i.e., point 8 of candidate matching position 4 is selected as the reference point. Assuming that after filtering by the minimum matching distance based on the reference position, three candidate matching points are obtained: point 2, point 3, and point 4.

[0192] In some embodiments, points whose distance to a candidate matching point is less than the minimum matching distance can be merged, and the template blocks corresponding to the points can be merged with the matching template blocks containing the candidate matching points. For example, if the distance between candidate matching point 1 and point a is less than the minimum matching distance, and the matching template block containing candidate matching point 1 is A, and the matching template block containing point a is B, then matching template blocks A and B are merged, so that both candidate matching point a and point a are considered candidate matching points and correspond to the merged matching template block.

[0193] In some embodiments, similarity can be used as a weight to combine building patches with candidate matching points to form a weighted bipartite graph. Then, by performing maximum weight matching on the weighted bipartite graph, a matching building patch map is obtained. Finally, the matching building patch map is compared with the first building annotation map to obtain the change detection result.

[0194] Please refer to Figure 16 In some embodiments, step S1010 includes, but is not limited to, steps S1101 to S1103:

[0195] Step S1101: Map the matching building patch map to the first building annotation map, and match the building patches in the matching building patch map to the first building annotation map.

[0196] Step S1102: If there is no alternative matching point for the building patch in the matching building patch map, then select the reference patch that is closest to the building patch in the matching building patch map and calculate the offset of the building patch relative to the reference patch.

[0197] Step S1103: In the first building annotation map, find the area corresponding to the candidate matching point of the reference patch, and mark the corresponding area as the demolished building according to the offset.

[0198] Alternatively, if the similarity between a building patch and a candidate matching point is less than a preset similarity threshold in the matching building patch map, then in the first building labeling map, the area corresponding to the candidate matching point is labeled as a demolished building.

[0199] Alternatively, if no corresponding area is found for the candidate matching point of a building patch in the first building annotation map, the corresponding building in the first building annotation map will be marked as a new building.

[0200] In some embodiments, the matching building patch map can be compared with the first building annotation map to detect whether the building has changed. For example, detecting whether a building has changed can determine whether the building has been demolished or newly constructed. In some embodiments, the matching building patch map with building patches can be mapped to the first building annotation map of the first image for combined observation.

[0201] In some embodiments, if a building patch does not have a corresponding candidate matching point in the matching building patch map, it means that the building patch did not find a corresponding candidate matching point during the probability distribution map matching process. That is to say, the building patch does not have a corresponding matching position in the first image. Therefore, the building where the building patch is located in the reference image is a demolished building. At this time, the demolished building needs to be marked in the first image simultaneously. Specifically, the building patch without a corresponding matching position can be called building patch 1. Then, in the matching building patch map, the building patch that is closest to building patch 1 and has a candidate matching point (hereinafter referred to as building patch 2 for ease of explanation) can be found. Building patch 2 is used as the reference patch for building patch 1. The offset is calculated, and in the first building annotation map, the position of building patch 1 is calculated based on building patch 2 and the offset. The building where building patch 1 is located is marked as a demolished building.

[0202] In some embodiments, if a building patch finds a corresponding matching point in the first building annotation map, the similarity between the candidate matching point and the matching point in the building patch can be calculated. If the calculated similarity is lower than a preset threshold, it means that the two are not the same and the area corresponding to the matching point is a demolished building.

[0203] In some embodiments, if each region in the first building annotation map does not overlap with the matching building patch map, that is, if no corresponding region is found for the candidate matching point of the building patch in the first building annotation map, then the building in the first building annotation map is a newly added building and can be annotated in the first building annotation map. It is understood that a change annotation map can be generated separately for the changed parts of the first image, or a second building annotation map can be generated based on the first building annotation map.

[0204] Please see Figure 17 This application also provides a building change detection system that can implement the above-described building change detection method. The building change detection system includes:

[0205] The first image acquisition module 1701 is used to acquire a first image to be detected; wherein, the first image is an image formed by taking a picture of the region from a first angle;

[0206] The reference image acquisition module 1702 is used to acquire a reference image and a reference building annotation drawing corresponding to the reference image; wherein, the reference image is an image formed by taking a picture of the area from a second angle;

[0207] Preprocessing module 1703 is used to preprocess the first image so that the image attribute parameters of the first image are consistent with those of the reference image;

[0208] The training sample set generation module 1704 is used to sample the reference image and the reference building annotation map corresponding to the reference image to obtain multiple sample blocks, and to form a training sample set based on the multiple sample blocks; wherein, the sample block includes the image block within a preset range in the reference image and the building annotation map corresponding to the image block.

[0209] The semantic segmentation model acquisition module 1705 is used to train the semantic segmentation model based on the training sample set to obtain the trained semantic segmentation model.

[0210] The probability distribution map acquisition module 1706 is used to input the first image into the trained semantic segmentation model to obtain the probability distribution map;

[0211] The first building annotation map generation module 1707 is used to annotate the pixels with probability values ​​greater than the first preset threshold in the probability distribution map as buildings, and generate the first building annotation map.

[0212] The building patch acquisition module 1708 is used to acquire multiple building patches of the reference image within the intersection range of the reference image and the first image;

[0213] The matching building patch map acquisition module 1709 is used to perform similarity matching of building patches in the probability distribution map, obtain matching results, and obtain matching building patch map based on the matching results;

[0214] The building change detection result acquisition module 1710 is used to compare the matching building patch map and the first building annotation map to obtain the building change detection result of the first image.

[0215] Please refer to Figure 4 , Figure 4This is a flowchart illustrating the main processes of steps S101 to S110 of this application. In some embodiments, the reference image is a historically acquired image of the building, and the first image is a newly captured image of the building to be changed. Alternatively, the first image can be a historically acquired image of the building, and the reference image can also be a historically acquired image of the building; this application does not impose specific limitations on either approach. It is understood that the areas captured by the first image and the second image should be the same. For example, if the area captured by the first image is cell 1, then the area captured by the reference image should also be cell 1.

[0216] In some embodiments, image attribute parameters include coordinate positioning, resolution, band, and spectral band. It is understood that the first image can be transformed to make its spectral, texture, and other image attribute parameters consistent with those of the reference image, thereby facilitating the analysis of both the first and reference images.

[0217] In some embodiments, a deep learning network model for style transfer can be trained by extracting corresponding sample blocks from a first image and a reference image. During training, forward propagation is used to ensure that the image attribute parameters of the first image and the reference image are consistent, and the model performance is improved by continuously optimizing the model parameters. After the model is trained, the first image and the reference image can be directly input into the model, and the model can process the other image based on the image attribute parameters of one image. For example, the first image can be processed based on the attribute parameters of the reference image.

[0218] In some embodiments, the reference image and the corresponding reference building annotation can be sampled to form multiple sample blocks, and a training set can be formed based on the multiple sample blocks. The sample blocks include image blocks within a preset range in the reference image and building annotations in the reference building annotation.

[0219] In some embodiments, the semantic segmentation model can be trained using a training sample set to obtain a trained semantic segmentation model. In some embodiments, a first image can be input into the trained semantic segmentation model to obtain a probability distribution map. Specifically, the semantic segmentation model can be FCN (Fully Convolutional Networks), UNet, DeepLab, Mask R-CNN, etc., selected according to the characteristics of the data. For example, when the range of the reference image is large enough (e.g., the effective range is greater than 30000*30000 pixels), a large number of samples can be selected, and a semantic segmentation model with more parameters can be used, or a semantic segmentation algorithm using the Transformer model can be selected. In some embodiments, a sample set selected from the reference image and the original building annotation image can be input into the semantic segmentation model to train the semantic segmentation model.

[0220] Understandably, during the training of the semantic segmentation model based on the sample set, the original building annotation image can be compared with the reference image, the loss function can be calculated, and backpropagation can be continuously performed to optimize the model parameters. In some embodiments, after the semantic segmentation model is trained, the first image can be directly input into the model to obtain the model's output. Understandably, the model's output is the result of extracting and annotating the building pixels, i.e., a binary mask. Specifically, color encoding or grayscale encoding can be used to map the probability values ​​of the building pixels onto the image to generate a probability distribution map.

[0221] Understandably, the probability distribution map labels each target area with a corresponding probability, such as labeling a building area as 1 and a non-building area as 0. In some embodiments, a first preset threshold can be set, labeling areas with a probability greater than the first preset threshold as buildings and areas with a probability less than the first preset threshold as non-buildings. Understandably, the first preset threshold can be adjusted as needed. For example, the first preset threshold can be 0.5, and in the probability map, areas with a probability greater than 0.5 are labeled as buildings, and areas with a probability less than 0.5 are labeled as non-buildings, thereby generating a first building label map.

[0222] In some embodiments, to improve matching accuracy, the matching range can be limited. Specifically, building areas can be selected as the effective range in the first image and the reference image, and the intersection of the effective ranges of the first image and the reference image can be used as the intersection range, i.e., the detection range. This avoids interference from invalid ranges and saves computational resources. In some embodiments, multiple building patches of the reference image can be obtained within the intersection range. Specifically, if it is necessary to detect changes in the top of a building, then the building patch represents the top of the building, and one building patch represents one top of a building.

[0223] In some embodiments, building patches can be matched for similarity in a probability distribution map to determine whether there are similar regions in the first image. In some embodiments, similarity can be calculated in real time during the similarity matching process, and matching regions with similarity values ​​less than a similarity threshold can be filtered out.

[0224] In some embodiments, to improve the accuracy of matching, the rectangular region matched by the matching template block corresponding to the building patch can be divided. Specifically, the rectangular region can be divided to obtain multiple candidate matching regions, and the similarity between each position in the multiple candidate matching regions and the matching template block can be calculated. The point with the highest similarity in each candidate region is selected as the candidate point, and the candidate point with the highest similarity in the multiple candidate matching regions is selected as the reference point.

[0225] Understandably, candidate points whose distance to the reference point is less than the minimum matching distance have high similarity to the reference point and can be eliminated, ultimately resulting in multiple candidate matching points. Understandably, based on the weighted bipartite graph, a unique candidate matching point can be obtained from these multiple candidate matching points, and a matching building patch map can be generated in the image corresponding to the intersection range of the building patch and the candidate matching point.

[0226] In some embodiments, a matching result can be obtained by mapping a matching building patch map to a first building annotation map and matching the building patches in the matching building patch map to the first building annotation map. Specifically, if there is no candidate matching point for a building patch in the matching building patch map, a reference patch closest to the building patch is selected in the matching building patch map, the offset of the building patch relative to the reference patch is calculated, and in the first building annotation map, the region corresponding to the candidate matching point of the reference patch is found, and the corresponding region is marked as a demolished building according to the offset. Alternatively, if the similarity between a building patch and a candidate matching point in the matching building patch map is less than a preset similarity threshold, the region corresponding to the candidate matching point in the first building annotation map is marked as a demolished building. Alternatively, if no corresponding region is found for the candidate matching point of a building patch in the first building annotation map, the building in the first building annotation map is marked as a newly added building.

[0227] Understandably, this application significantly reduces the impact of large local differences caused by global image matching by performing local matching on architectural images, reduces the difficulty of comparing the first image with the reference image, and improves the accuracy of the annotation results.

[0228] The specific implementation of this building change detection system is basically the same as the specific embodiment of the building change detection method described above, and will not be repeated here. Subject to meeting the requirements of the embodiments of this application, the building change detection system may also be equipped with other functional modules to implement the building change detection method described above.

[0229] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described building change detection method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0230] Please see Figure 18 , Figure 18 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0231] The processor 1801 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0232] The memory 1802 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1802 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1802 and is called and executed by the processor 1801 to execute the building change detection method of the embodiments of this application.

[0233] The input / output interface 1803 is used to implement information input and output;

[0234] The communication interface 1804 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0235] Bus 1805 transmits information between various components of the device (e.g., processor 1801, memory 1802, input / output interface 1803, and communication interface 1804);

[0236] The processor 1801, memory 1802, input / output interface 1803 and communication interface 1804 are connected to each other within the device via bus 1805.

[0237] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described building change detection method.

[0238] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0239] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0240] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0241] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0242] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0243] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0244] It should be understood that in this application, "at least one" and "several" refer to one or more, and "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0245] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0246] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0247] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0248] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0249] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for detecting building changes, characterized in that, The method includes: Acquire a first image to be detected; wherein the first image is an image formed by taking a picture of the region from a first angle; Obtain a reference image and a corresponding architectural annotation of the reference image; wherein the reference image is an image formed by taking a picture of the area from a second angle; The first image is preprocessed to make its image attribute parameters consistent with those of the reference image; The reference image and the corresponding reference building annotation are sampled to obtain multiple sample blocks, and a training sample set is formed based on the multiple sample blocks; wherein, the sample block includes image blocks within a preset range in the reference image and building annotations in the reference building annotation map corresponding to the image blocks; Based on the training sample set, the semantic segmentation model is trained to obtain a trained semantic segmentation model; The first image is input into the trained semantic segmentation model to obtain a probability distribution map; Pixels with probability values ​​greater than a first preset threshold in the probability distribution map are marked as buildings, and a first building annotation map is generated. Within the intersection of the reference image and the first image, multiple building patches of the reference image are obtained; The building patches are matched for similarity in the probability distribution map to obtain matching results, and a matching building patch map is obtained based on the matching results; The building change detection result of the first image is obtained by comparing the matched building patch map with the first building annotation map.

2. The building change detection method according to claim 1, characterized in that, After obtaining the building change detection result of the first image, the method further includes: Perform a sample augmentation process, which includes: extracting the sample block within the intersection range of the reference image and the first image from the sample set; The sample block is used as a building patch and matched in the probability distribution map to obtain the rectangular area with the highest similarity to the building patch as the target rectangular area; Based on the building change detection results, calculate the proportion of the changed portion in the target rectangular area to the total change in the target rectangular area; If the change ratio is less than a preset threshold, then a first image sub-block is extracted based on the preprocessed first image, and a building annotation sub-block is extracted based on the first image sub-block at the position corresponding to the matching building patch. The first image sub-block, the building annotation sub-block, the sample block, and the building annotations corresponding to the sample block are combined into multiple augmentation pairs; Based on the augmentation pair, the first image sub-block and the building annotation sub-block are augmented with mixed samples to generate multiple augmented samples; Repeat the sample augmentation process.

3. The building change detection method according to claim 2, characterized in that, After generating multiple augmented samples, the process also includes: The semantic segmentation model is trained by forming a sample training set based on the augmented samples. During the training process, the parameters of the semantic segmentation model are adjusted to obtain a trained semantic segmentation model. Based on the semantic segmentation model, architectural change detection is performed on the first image and the reference image to generate architectural change verification results; The building change verification results are compared and verified with the building change detection results.

4. The building change detection method according to claim 1, characterized in that, The image attribute parameters include coordinate positioning, resolution, band, and spectral band; the preprocessing of the first image to make its image attribute parameters consistent with those of the reference image includes: Using the reference image as a reference, the first image is adjusted so that the coordinate positioning and resolution of the first image are consistent with the reference image; Determine whether the first image contains infrared and near-infrared bands. If the first image contains infrared and near-infrared bands, extract the green area of ​​the first image corresponding to the infrared and near-infrared bands to form a green area mask, and remove the green area mask. Determine whether the wavebands and spectral bands of the first image and the reference image are the same. If the wavebands and spectral bands of the first image and the reference image are the same, perform relative radiometric correction on the first image. If the wavebands or spectral bands of the first image and the reference image are not the same, perform linear quantization on the effective pixel value range of the first image so that the effective pixel value range of the first image is the same as the effective pixel value range of the reference image.

5. The building change detection method according to claim 4, characterized in that, After ensuring that the effective pixel value range of the first image is the same as the effective pixel value range of the reference image, the method further includes: The image attribute parameters of the first image and the reference image are checked. If the image attribute parameters of the first image and the reference image are still different, image sample blocks are taken from the first image and the reference image respectively and set together to train the style transfer network model. The first image is input into the trained style transfer network model so that the image attribute parameters of the first image and the reference image are consistent.

6. The building change detection method according to claim 1, characterized in that, The step of labeling pixels with probability values ​​greater than a first preset threshold in the probability distribution map as buildings and generating a first building label map includes: The probability distribution map is processed by the conditional random field algorithm to obtain a continuous probability distribution map; From the probability distribution map, select pixels whose probability value is greater than a first preset threshold; The pixels are expanded using morphological dilation filtering; Pixels in the continuous probability distribution map that are greater than the first preset threshold are labeled as buildings to obtain the first building label map.

7. The building change detection method according to claim 1, characterized in that, The step of performing similarity matching on the probability distribution map of the building patches to obtain matching results, and obtaining a matching building patch map based on the matching results, includes: Within the intersection range, multiple candidate matching points are obtained by matching the building patches on the probability distribution map. Calculate the matching similarity between the building patch and the candidate matching point, construct a weighted bipartite graph based on the matching similarity, and select the candidate matching point that uniquely corresponds to the building patch from the weighted bipartite graph to obtain the matching building patch map of the first image.

8. The building change detection method according to claim 7, characterized in that, Within the intersection range, the building patches are matched on the probability distribution map to obtain multiple candidate matching points, including: Set local search parameters and minimum matching distance; For each building patch, take the outer rectangle as the matching template block, and select the matching reference point from the matching template block; Within the intersection range, the matching template block is moved and matched in the probability distribution map according to the local search parameters, and the similarity between the matching template block and the corresponding rectangular area in the probability distribution map is calculated in real time based on the matching reference point as the first similarity. The values ​​of the first similarity are normalized so that each of the first similarities has the same measurement range in the probability distribution map. In the probability distribution map, the rectangular regions with the first similarity greater than the similarity threshold are obtained, and the rectangular regions are divided according to a preset number of divisions to obtain multiple candidate matching positions; The similarity between the matching reference point and each of the candidate matching positions is calculated as a second similarity. For each candidate matching position, the point with the largest second similarity is selected as the candidate matching point for each candidate matching position. Of the candidate matching points, the point with the highest second similarity is selected as the reference point; Calculate the baseline distance between the remaining candidate matching points and the baseline point, and filter out the candidate matching points whose baseline distance is less than the minimum matching distance to obtain multiple filtered candidate matching points.

9. The building change detection method according to claim 8, characterized in that, After obtaining multiple filtered candidate matching points, the process further includes: The multiple candidate matching points are arranged, and distance movement matching is performed in the probability distribution map; If there exists a first matching point in the probability distribution map whose distance to each of the candidate matching points is less than the minimum matching distance, then the first matching point is taken as a new candidate matching point, and the matching template block corresponding to the first matching point is merged with the matching template block corresponding to the candidate matching point to obtain a second template block; Calculate the merged similarity between the first matching point and the second template block, and normalize the value of the merged similarity so that each merged similarity has the same measurement range in the probability distribution map.

10. The building change detection method according to claim 7, characterized in that, The process of calculating the matching similarity between the building patch and the candidate matching points, constructing a weighted bipartite graph based on the matching similarity, and selecting the candidate matching point that uniquely corresponds to the building patch from the weighted bipartite graph to obtain the matching building patch map of the first image includes: Calculate the similarity between the matching template block corresponding to the building patch and the candidate matching point, and use the similarity as the weight to form a weighted bipartite graph by combining the building patch and the candidate matching point. Virtual weights are filled into the weighted bipartite graph to make the number of matching template blocks corresponding to the building patches consistent with the number of candidate matching points; Using the maximum weight as the matching, the weighted bipartite graph is subjected to maximum weight matching, and during the matching process, each matching template block is associated with a unique candidate matching point to generate a matching building patch map.

11. The building change detection method according to claim 1, characterized in that, The step of comparing the matched building patch map and the first building annotation map to obtain the building change detection result of the first image includes: The matching building patch map is mapped onto the first building annotation map, and the building patches in the matching building patch map are matched in the first building annotation map. If the building patch does not have a candidate matching point in the matching building patch map, then the reference patch closest to the building patch is selected in the matching building patch map, and the offset of the building patch relative to the reference patch is calculated; In the first building annotation map, find the area corresponding to the candidate matching point of the reference patch, and mark the corresponding area as the demolished building according to the offset; Alternatively, if the similarity between the building patch and the candidate matching point in the matching building patch map is less than a preset similarity threshold, then in the first building annotation map, the area corresponding to the candidate matching point is marked as a demolished building. Alternatively, if no corresponding area is found for the candidate matching point of the building patch in the first building annotation map, the corresponding building in the first building annotation map is marked as a new building.

12. A building change detection system, characterized in that, The system includes: A first image acquisition module is used to acquire a first image to be detected; wherein, the first image is an image formed by taking a picture of the region from a first angle; A reference image acquisition module is used to acquire a reference image and a corresponding reference building annotation drawing; wherein, the reference image is an image formed by taking a picture of the area from a second angle; The preprocessing module is used to preprocess the first image so that the image attribute parameters of the first image are consistent with those of the reference image. The training sample set generation module is used to sample the reference image and the reference building annotation map corresponding to the reference image to obtain multiple sample blocks, and to form a training sample set based on the multiple sample blocks; wherein, the sample block includes image blocks within a preset range in the reference image and building annotations in the reference building annotation map corresponding to the image blocks; The semantic segmentation model acquisition module is used to train the semantic segmentation model based on the training sample set to obtain the trained semantic segmentation model. The probability distribution map acquisition module is used to input the first image into the trained semantic segmentation model to obtain the probability distribution map; The first building annotation map generation module is used to annotate the pixels with probability values ​​greater than a first preset threshold in the probability distribution map as buildings, and generate the first building annotation map. The building patch acquisition module is used to acquire multiple building patches of the reference image within the intersection range of the reference image and the first image; The matching building patch map acquisition module is used to perform similarity matching on the probability distribution map of the building patches, obtain matching results, and obtain a matching building patch map based on the matching results; The building change detection result acquisition module is used to compare the matched building patch map and the first building annotation map to obtain the building change detection result of the first image.

13. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the building change detection method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the building change detection method according to any one of claims 1 to 11.

Images