A local background semantic information enhancement method for building change detection

Abstract

The application relates to a local background semantic information enhancement method for building change detection, that is, two original building maps of before and after time phases are compared, building change regions are labeled on the two original maps, a label vector file is generated, the original maps are cut into N small maps with a height of H and a width of W, the label vector file is converted into a binary map with the same width and height as the original maps, then N label blocks with a height of h and a width of w are cut out, wherein h=H / 2 and w=W / 2, the formed map is used as a training sample, a deep learning model is constructed, the deep learning model is trained, and an arbitrary selected remote sensing image to be detected is input into the trained model, the method realizes local background semantic information enhancement, that is, the background pixels of the difference value parts between W and w and the difference value parts between H and h provide information enhancement for the detection of edge regions, decisive context features are provided for the deep learning model discrimination from the source, and the detection accuracy is significantly improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for enhancing local background semantic information for building change detection. Background Technology

[0002] China's rapid socio-economic development has accelerated urban spatial expansion. In the process of urbanization, buildings, as active urban elements, undergo significant renewal. Accurate and efficient extraction of building change information is crucial for urban planning and other related operations.

[0003] The development of satellite remote sensing technology has brought a wealth of remote sensing data to various research fields, among which remote sensing images have become an important data source for building change detection technology. High-resolution remote sensing images are rich in ground feature information, but the problem of "different spectra for the same object, and different objects with the same spectrum" is serious, posing a significant challenge to effectively utilizing this information for building change detection. With the innovation and development of deep learning technology, its characteristics of eliminating the need for expert manual design and strong generalization capabilities have brought new possibilities to building change detection.

[0004] In practical applications, due to the towering buildings in modern cities and the different shooting postures of cameras at different times, buildings may have different projections and problems that existing algorithms cannot handle. It is necessary to manually detect the areas of building changes. However, the low efficiency and high cost are unavoidable pain points in this field when faced with the massive drawing tasks caused by the number of buildings in a city.

[0005] Local background semantic information enhancement algorithms enable deep learning algorithms to effectively handle projection difference phenomena, further improving the accuracy of building change detection. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method for enhancing local background semantic information for building change detection that is highly accurate, efficient and low cost.

[0007] The technical solution adopted in this invention is a method for enhancing local background semantic information for building change detection, which includes the following steps:

[0008] Step 1: Compare the two original building images from different time periods. Mark the areas where the building has changed on the two original building images, that is, draw the areas where the building has changed, and generate a marked vector file. The height of the original building image is set to OH and the width is set to OW.

[0009] Step 2: Cut the two original building images from the previous and next time phases into N smaller images with height H and width W, where OH is greater than H and OW is greater than W.

[0010] Step 3: Convert the annotation vector file generated in Step 1 into a binary image with height OH and width OW. Then, cut the resulting binary image into N labels with height h and width w. Each label is located in the center of the corresponding small image, where h = H / 2 and w = W / 2.

[0011] Step 4: Combine the N small images from the previous time phase in Step 2 with the N small images from the subsequent time phase. Figure 1 One-to-one stitching is performed, and the resulting image is used as a training sample. A training sample set is generated, which includes N stitched images.

[0012] Step 5: Construct a deep learning model, which is a hook network structure;

[0013] Step 6: Train the deep learning model: Input the training sample set generated in Step 4 into the deep learning model built in Step 5 to obtain the output of the deep learning model. Input the output and the label from Step 3 into the loss function to calculate the loss value. Then, use the backpropagation algorithm to update the network weights with the loss value equal to 0. Iterate continuously to achieve the training of the deep learning model.

[0014] Step 7: Randomly select a remote sensing image to be detected, and apply the deep learning model trained in Step 6 to the remote sensing image.

[0015] The beneficial effects of this invention are as follows: The above-mentioned method for enhancing local background semantic information for building change detection achieves local background semantic information enhancement. That is, the background pixels in the difference between W and w and the background pixels in the difference between H and h enhance the information for building change detection in the edge region. This provides decisive contextual features for deep learning model discrimination from the source. In this field, this is a technology that has come from nothing, representing a qualitative leap. It significantly improves the accuracy, precision, and efficiency of detection, and is also low in cost.

[0016] Preferably, in step five, the hook network structure includes a feature extraction module and a feature mapping module located on one side of the feature extraction module. The feature extraction module has a four-layer structure, and the feature mapping module has a three-layer structure. The feature extraction module includes a first module, a second module, a third module, and a fourth module. The first module, the second module, the third module, and the fourth module all include max pooling and convolutional layers. The feature mapping module includes a fifth module, a sixth module, and a seventh module. The fifth module includes bilinear interpolation upsampling, merging, and dilated convolutional layers. The sixth module and the seventh module both include bilinear interpolation upsampling, merging, and convolutional layers.

[0017] As a preferred method, the specific process of inputting the training sample set into the hook network structure to obtain the output result of the deep learning model is as follows: the training samples in the training sample set are input into the hook network structure at full size, and after passing through the feature extraction module, a feature matrix is ​​generated. Then, after passing through the feature mapping module, a two-channel result is output. In the two channels of this result, one channel represents that the building has not changed, and the other channel represents that the building has changed.

[0018] Preferably, in step six, the loss function is defined as: Where k is the pixel position, P is the total number of pixels, and targ et k Let x be the true value of the pixel at position k, and let the deep learning model be f(), where x is the input image. Output values ​​for the model. This is the model output value at position k.

[0019] As a preferred option, in step seven, the specific process of applying the deep learning model trained in step six to any selected remote sensing image to be detected is as follows: set the height of any selected remote sensing image to be detected as OH and the width as OW, crop the remote sensing image into N small images with a height of H and a width of W, input the N small images into the trained deep learning model to calculate the output result, and then stitch the obtained output results into an image with a height of OH and a width of OW, thereby completing the application of the remote sensing image in the deep learning model.

[0020] As a preferred method, a sliding window is used to achieve full coverage of the remote sensing image to be detected when applying it to the trained deep learning model.

[0021] As a preferred method, the specific approach for achieving full coverage of the remote sensing image to be detected using a sliding window is as follows: The image to be detected is used as the bottom bounding box. The image to be detected includes two original building images from consecutive time periods, both with a height of OH and a width of OW. On both the previous and subsequent original building images, the following steps are taken: The first bounding box is used as the image input into the deep learning model at time t1. The height of the first bounding box is H and the width is W. The deep learning model calculates and outputs the result, which is the second bounding box. The height of the second bounding box is h and the width is w, where h = H / 2 and w = W / 2. The output result at time t1 is temporarily stored, thus completing the model calculation at time t1. At time t2, the detection box moves to the right, i.e., slides horizontally. The sliding step is the width w of the label. This sliding only occurs in the horizontal direction, but when changing lines, it needs to slide in the vertical direction, which is the height h of the label. After the sliding is complete, let the third block diagram be the image input into the model at time t2, with a height H and a width W. Let the fourth block diagram be the output result of the model at time t2, with a height h and a width w, where h = H / 2 and w = W / 2. The output result at time t2 is temporarily stored. At this time, the model calculation at time t2 is completed. This process is repeated iteratively to finally complete the deep learning model calculation of the two original building images of the preceding and following time phases. The temporarily stored output results are then stitched together in sequence to form the same size as the remote sensing image to be detected, thus completing the application of the remote sensing image to be detected in the trained deep learning model. Attached Figure Description

[0022] Figure 1 This is an image of a certain area in two original building images taken at different times in this invention, with the field of view set to v;

[0023] Figure 2 To and Figure 1 Images of the same area at two different time phases are given by a field of view of V, where V > v;

[0024] Figure 3 This is a schematic diagram of the input image and labels used when building a deep learning model. The area within the middle box represents the labels, i.e., the detection area for changes in buildings, while the entire image represents the input image.

[0025] Figure 4 This is a flowchart of a method for enhancing local background semantic information for building change detection according to the present invention;

[0026] Figure 5 This is a schematic diagram illustrating the annotation of training samples for building change detection provided in an embodiment of the present invention;

[0027] Figure 6 This is a schematic diagram of the structure of the deep learning model in this invention;

[0028] Figure 7 This is a schematic diagram illustrating the application of the deep learning model provided in this invention;

[0029] Reference numerals: 1. First module; 2. First arrow; 3. Second module; 4. Third module; 5. Fourth module; 6. Seventh module; 7. Second arrow; 8. Sixth module; 9. Fifth module; 10. Bottom block diagram; 11. First block diagram; 12. Second block diagram; 13. Third block diagram; 14. Fourth block diagram. Detailed Implementation

[0030] The invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can implement it based on the description. The scope of protection of the invention is not limited to these specific embodiments.

[0031] Those skilled in the art should understand that, in the disclosure of this invention, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the above terms should not be construed as limiting this invention.

[0032] exist Figure 1 In the context of both human and model analysis, it's possible to perceive a change in the building's position in the upper left corner of either image. However, when the field of view is magnified... Figure 2 As shown, it was discovered that the appearance of some buildings was due to projection differences. Figure 1 In reality, no changes occurred to the building. In traditional methods, deep learning models detect both the labeled region and the input image region, and these two regions are of identical size. However, in… Figure 1 and Figure 2 In this situation, traditional methods have obvious shortcomings; it is clear that traditional methods cannot be used in... Figure 1 While human input yields interpretable and accurate results, if humans encounter similar situations, they would incorporate more information and common sense to aid their judgment, such as scaling the image or comparing it with images from other regions.

[0033] This invention is of great significance for change detection and semantic segmentation in remote sensing. It breaks through the limitations of past algorithms that relied on "two images" or "one image" for change detection, bringing a completely new approach. It introduces more information, providing the model with contextual information that exceeds the information limits of "two images" or "one image," achieving a much wider receptive field than traditional methods. It provides semantic information support for detecting changes in buildings at image edges, enabling more accurate change perception. For example... Figure 3 As shown, the entire image is input into a deep learning model, with the middle box representing the detection area, i.e., the label area.

[0034] Please see Figure 4 , Figure 4 A flowchart of a local background semantic information enhancement method for building change detection provided by the present invention includes the following steps:

[0035] Step 1: Compare the two original building images from different time periods. Mark the areas where the building has changed on the two original building images, that is, draw the areas where the building has changed, and generate a labeled vector file. The height of the original building image is set to OH and the width is set to OW.

[0036] Step 2: Cut the two labeled vector files of the preceding and following time phases into N small images with height H and width W, where OH is greater than H and OW is greater than W;

[0037] Step 3: Convert the labeled vector file generated in Step 1 into a binary image with the same width and height as the original image, i.e., the height of the binary image is OH and the width is OW. Then, crop the binary image into N labels with a height of h and a width of w. Each label is located in the center of the corresponding small image, where h = H / 2 and w = W / 2. For example, if the size of the small image is 512*512, then the size of the label is 256*256. The specific difference between the two can be optimized according to the size occupied by the building in an image of a certain pixel size. The background pixels in the difference between W and w and the background pixels in the difference between H and h enhance the information for edge detection.

[0038] Step 4: Combine the N small images from the previous time phase in Step 2 with the N small images from the subsequent time phase. Figure 1 One-to-one stitching is performed, and the resulting image is used as a training sample to generate a training sample set, which includes N stitched images. For example, two small images of size 512*512 are stitched together to form an image of size 512*1024 as a training sample.

[0039] Step 5: Construct a deep learning model;

[0040] Step Six: Train the deep learning model: Input the training sample set generated in Step Four into the hook network structure. The training sample set includes N pieces of the image formed by piecing together. Then, input all N pieces of the image formed by piecing together into the hook network structure to obtain the output result of the deep learning model. Input the output result and the label from Step Three into the loss function to calculate the loss value. Then, use the backpropagation algorithm to update the network weights with the loss value equal to 0. Iterate continuously to achieve the training of the deep learning model.

[0041] Step 7: Randomly select a remote sensing image to be detected, and apply the deep learning model trained in Step 6 to the remote sensing image.

[0042] This patent proposes a hook net structure, such as... Figure 6 As shown, it consists of two parts: a feature extraction module on the left and a feature mapping module on the right. Traditional algorithms use a symmetrical structure to achieve similar functionality, meaning the number of feature extraction modules and feature mapping modules is the same. However, in... Figure 6 In this network structure, different numbers of layers are used: the feature extraction module has 4 layers, while the feature mapping module has only 3 layers. The "hook" signifies the alternating lengths of its two sides. Figure 6 middle:

[0043] The first module 1 includes convolution, max pooling, and convolutional layers;

[0044] The second module 3, the third module 4, and the fourth module 5 include max pooling and convolutional layers;

[0045] Module 9 of the fifth module includes bilinear interpolation upsampling, merging, and dilated convolutional layers;

[0046] Module 8 includes bilinear interpolation upsampling, merging, and convolutional layers;

[0047] The first arrow 2 represents forward calculation;

[0048] The second arrow 7 represents the copy operation.

[0049] The first module 1, the second module 3, the third module 4, and the fourth module 5 together constitute the feature extraction module, while the fifth module 9, the sixth module 8, and the seventh module 6 together constitute the feature mapping module.

[0050] The input training sample image is 3-channel 512x1024, composed of two consecutive 3-channel 512x512 images stitched together. After feature extraction, a feature size of 512 channels (32x64) is generated. The feature mapping module then outputs a 2-channel 256x256 result. In this output, one channel represents unchanged buildings, and the other represents changed buildings. The input and output image sizes are different; the input image is 512x1024, and the output image is 256x256. Figure 3 As shown, the detection area is the box area in the figure, i.e. the label. The image information of the undetected area is used to enhance the semantic information of the edge side, thereby enhancing the model's ability to perceive changes in buildings.

[0051] In step six, the loss function is defined as: Where k is the pixel position, P is the total number of pixels, and targ et k Let x be the true value of the pixel at position k, and let the deep learning model be f(), where x is the input image. Output values ​​for the model. This is the model output value at position k.

[0052] In step seven, the specific process of applying the deep learning model trained in step six to any selected remote sensing image to be detected is as follows: set the height of the randomly selected remote sensing image to OH and the width to OW, crop the remote sensing image into N small images with a height of H and a width of W, input the N small images into the trained deep learning model for calculation to obtain the output result, and then stitch the obtained output results together into an image with a height of OH and a width of OW, thereby completing the application of the remote sensing image in the deep learning model.

[0053] In the process of applying the remote sensing image to be detected to the trained deep learning model, a sliding window is used to achieve full coverage of the remote sensing image to be detected. The specific process is as follows: the image to be detected is used as the bottom bounding box 10, with a height of OH and a width of OW. Let the first bounding box 11 be the image input into the deep learning model at time t1, with a height of H and a width of W. The deep learning model calculates and obtains the output result, which is the second bounding box 12, with a height of h and a width of w, where h = H / 2 and w = W / 2. The output result at time t1 is temporarily stored, thus completing the model calculation at time t1. At time t2, the detection box... Move to the right, that is, slide horizontally, with a sliding step of the label width w. This slide only occurs in the horizontal direction, but when changing lines, it needs to slide vertically, which is the label height h. After the slide is completed, let the third block diagram 13 be the image input into the deep learning model at time t2, and the fourth block diagram 14 be the output result of the model at time t2. The result at time t2 is temporarily stored. At this time, the model calculation at time t2 is completed. Iterate in this way until the deep learning model calculation of the whole image is completed. The temporarily stored output results are stitched together in order to form the same size as the image to be detected, that is, the application of the remote sensing image to be detected in the trained deep learning model is completed.

[0054] Existing technologies cannot handle false detections caused by differences in building projections, and mainly rely on manual visual judgment. This invention enhances local background semantic information, providing decisive contextual features for model discrimination from the source. This is a qualitative leap from nothing and significantly improves detection accuracy.

[0055] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. All equivalent changes and modifications made in accordance with the scope of the present invention and the contents of the specification are within the scope of the present invention.

Claims

1. A method for enhancing local background semantic information for building change detection, characterized in that: The method includes the following steps: Step 1: Compare the two original building images from different time periods. Mark the areas where the building has changed on the two original building images and generate a labeled vector file. Set the height of the original building image to OH and the width to OW. Step 2: Crop the two original images from the preceding and following time phases into N smaller images with height H and width W, where OH is greater than H and OW is greater than W. Step 3: Convert the annotation vector file generated in Step 1 into a binary image with height OH and width OW. Then, cut the resulting binary image into N labels with height h and width w. Each label is located in the exact center of the corresponding small image, where h = H / 2 and w = W / 2. Step 4: Patch the N small images from the previous time phase in Step 2 one by one with the N small images from the subsequent time phase. The resulting image is used as a training sample, and a training sample set is generated. Step 5: Construct a deep learning model, which is a hook network structure. The hook network structure includes a feature extraction module and a feature mapping module located on one side of the feature extraction module. The feature extraction module has a four-layer structure, and the feature mapping module has a three-layer structure. Step 6: Train the deep learning model: Input the training sample set generated in Step 4 into the deep learning model in Step 5 to obtain the output of the deep learning model. Input the output and the label from Step 3 into the loss function to calculate the loss value. Then, use the backpropagation algorithm to update the network weights with the loss value equal to 0. Iterate continuously to achieve the training of the deep learning model. During the training of the deep learning model, the background pixels in the difference between W and w and the difference between H and h of each small image enhance the information of the detection region detected by the deep learning model. Step 7: Randomly select a remote sensing image to be detected, and apply the deep learning model trained in Step 6 to the remote sensing image.

2. The method for enhancing local background semantic information for building change detection according to claim 1, characterized in that: In step five, the feature extraction module includes a first module (1), a second module (3), a third module (4), and a fourth module (5). The first module (1), the second module (3), the third module (4), and the fourth module (5) all include max pooling and convolutional layers. The feature mapping module includes a fifth module (9), a sixth module (8), and a seventh module (6). The fifth module (9) includes bilinear interpolation upsampling, merging, and dilated convolutional layers. The sixth module (8) and the seventh module (6) both include bilinear interpolation upsampling, merging, and convolutional layers.

3. The method for enhancing local background semantic information for building change detection according to claim 2, characterized in that: The specific process of inputting the training sample set into the hook network structure to obtain the output result of the deep learning model is as follows: the training samples in the training sample set are input into the hook network structure at full size, and after passing through the feature extraction module, a feature matrix is ​​generated. Then, after passing through the feature mapping module, the output result is obtained. The output result is a two-channel output result. In the two channels of the output result, one channel represents that the building has not changed, and the other channel represents that the building has changed.

4. The method for enhancing local background semantic information for building change detection according to claim 1, characterized in that: In step six, the loss function is defined as: Where k is the pixel position, P is the total number of pixels, and target k Let x be the true value of the pixel at position k, and let the deep learning model be f(), where x is the input image. Output values ​​for the model. This is the model output value at position k.

5. A method for enhancing local background semantic information for building change detection according to claim 3, characterized in that: In step seven, the specific process of applying the deep learning model trained in step six to any selected remote sensing image to be detected is as follows: set the height of any selected remote sensing image to be detected as OH and the width as OW, crop the remote sensing image into N small images with a height of H and a width of W, input the N small images into the trained deep learning model to calculate the output result, and then stitch the obtained output results into an image with a height of OH and a width of OW, thereby completing the application of the remote sensing image in the deep learning model.

6. The method for enhancing local background semantic information for building change detection according to claim 5, characterized in that: In the process of applying the trained deep learning model to the remote sensing image to be detected, a sliding window is used to achieve full coverage of the remote sensing image to be detected.

7. A method for enhancing local background semantic information for building change detection according to claim 6, characterized in that: The specific process of using a sliding window to complete the full coverage of the remote sensing image to be detected is as follows: The image to be detected is used as the bottom frame (10). The image to be detected includes two original images of buildings in the previous and subsequent time phases. The height of the two original images of buildings is OH and the width is OW. On the original images of buildings in the previous and subsequent time phases, the following is adopted: The first frame (11) is used as the image input into the deep learning model at time t1. The height of the first frame (11) is H and the width is W. The deep learning model calculates the output result, and the output result is the second frame (12). The height of the second frame (12) is h and the width is w, where h = H / 2 and w = W / 2. The output result at time t1 is temporarily stored, and the model calculation at time t1 is completed. At time t2, the detection box moves to the right, that is, slides in the horizontal direction. The step length is the width w of the label. This slide only occurs in the horizontal direction, but when changing lines, it needs to slide in the vertical direction, which is the height h of the label. After the slide is completed, let the third block diagram (13) be the image input into the model at time t2. The height of the third block diagram (13) is H and the width is W. The fourth block diagram (14) is the output result of the model at time t2. The height of the fourth block diagram (14) is h and the width is w, where h = H / 2 and w = W / 2. The output result at time t2 is temporarily stored. At this time, the model calculation at time t2 is completed. Iteration is performed in this way to finally complete the deep learning model calculation of the two original building images of the previous and next time phases. The temporarily stored output results are stitched together in order to form the same size as the remote sensing image to be detected, thus completing the application of the remote sensing image to be detected in the trained deep learning model.

Images