A correction and extraction method for semantic segmentation results of buildings in remote sensing images

Abstract

The present invention discloses a method for correcting and extracting semantic segmentation results of remote sensing images of buildings, comprising the following steps: performing mean shift on the source buildings to obtain a color cluster map; obtaining a semantic cluster overlay map; calculating the percentage of the area of ​​different color pixels in the semantic cluster overlay map to the area of ​​the corresponding color pixels in the color cluster map, judging whether it is greater than a given color pixel area ratio threshold, retaining the color if it is greater than, and discarding it if it is less than, and then binarizing to obtain a color cluster extraction map; calculating the degree of conformity between the color cluster extraction map and the semantic segmentation result map under different color pixel area thresholds to obtain the optimal color pixel area ratio threshold; and re-obtaining the building semantic segmentation correction extraction map using the optimal color pixel area ratio threshold. The present invention has the beneficial effect of effectively extracting a building semantic segmentation correction map, providing effective assistance in the post-processing field of remote sensing image building semantic segmentation results.

Description

Technical Field

[0001] The present invention relates to the field of image target recognition, and in particular to a method for correcting and extracting semantic segmentation results of remote sensing image buildings. Background Art

[0002] In the field of deep learning semantic segmentation of remote sensing images, building extraction is often inaccurate because the model is affected by differences between the training samples and the actual images in terms of color, lighting, image acquisition hardware, etc., resulting in missing or redundant parts of the extracted buildings. Therefore, the accurate correction and extraction of building semantic segmentation results is a difficult problem that urgently needs to be solved in the field of post-processing of remote sensing image building semantic segmentation. Since most of the methods currently used in the post-processing process of correcting and extracting building semantic segmentation results use mathematical geometry algorithms and do not incorporate the building features in the original image for processing, the results of the correction and extraction of building semantic segmentation images are not accurate enough, requiring a lot of manpower and material resources to manually process the results, which greatly reduces the efficiency of remote sensing image building extraction. Summary of the Invention

[0003] In order to solve the above technical problems, the present invention provides a method for correcting and extracting semantic segmentation results of remote sensing images of buildings, comprising the following steps:

[0004] S1, read the source building image and its semantic segmentation result image;

[0005] S2. Applying mean shift algorithm to the source building image to obtain a color clustering image;

[0006] S3. Setting an initial color pixel area ratio threshold W for the color cluster map, superimposing the outline of the semantic segmentation result map and the color cluster map, taking the intersection, and obtaining an overlay map;

[0007] S4, determining the percentage w of the area of ​​each color pixel in the overlay image to the area of ​​the pixel of the corresponding color in the color cluster image; if w is greater than W, retain the color; otherwise, discard the color to obtain a preliminary cluster extraction image;

[0008] S5. Binarize the preliminary cluster extraction map to obtain a color cluster extraction map.

[0009] S6. Use the representation value to calculate the consistency of the color cluster extraction map and the semantic segmentation result map, and pair the current consistency calculation result and the color pixel area ratio threshold in ascending order and put them into the set S;

[0010] S7. Determine whether W is greater than a preset threshold a. If so, traverse the set S, find the absolute value threshold of the difference between the compliance calculation results corresponding to the adjacent color pixel proportion threshold, and put it into the set Q until the traversal is completed, and then go to step S8. Otherwise, W is incremented by Δt each time, and steps S4-S6 are repeated.

[0011] S8. Traverse the set Q to obtain the minimum threshold. When the number of times the minimum threshold value appears is greater than the preset threshold b, select the first threshold that appears, and determine the left boundary of the color pixel area ratio threshold range corresponding to it as the optimal color pixel area ratio threshold bestW;

[0012] S9, determining the percentage w of the area of ​​each color pixel in the overlay image to the area of ​​the pixel of the corresponding color in the color clustering image. If w is greater than bestW, retain the color; otherwise, discard the color to obtain a preliminary building semantic segmentation correction extraction image;

[0013] S10. Binarize the preliminary building semantic segmentation correction extraction map to obtain the final building semantic segmentation correction extraction map.

[0014] The present invention provides the beneficial effect of effectively extracting the correct building outlines from the building cluster extraction map and the semantic segmentation result map, solving the problem of inaccurate building semantic segmentation results and providing effective assistance in the field of post-processing of remote sensing image building semantic segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic flow chart of the method of the present invention;

[0016] Figure 2 is the source building map;

[0017] Figure 3 It is the semantic segmentation result map;

[0018] Figure 4 It is the building semantic segmentation correction extraction map. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0020] Please refer to Figure 1 , Figure 1 The flowchart of the method of the present invention is a method for correcting and extracting semantic segmentation results of remote sensing images of buildings, comprising the following steps:

[0021] S1, read the source building image and its semantic segmentation result image;

[0022] It should be noted that the present invention uses the OpenCV library API: imread() function to read the source building image and the semantic segmentation result image; please refer to Figure 2 , Figure 2 This is a schematic diagram of the source building in the embodiment of the present invention; please refer to Figure 3 , Figure 3 is a semantic segmentation result graph in an embodiment of the present invention;

[0023] S2. Applying mean shift algorithm to the source building image to obtain a color clustering image;

[0024] S3. Setting an initial color pixel area ratio threshold W for the color cluster map, superimposing the outline of the semantic segmentation result map and the color cluster map, taking the intersection, and obtaining an overlay map; in the embodiment of the present invention, W is 0.2;

[0025] It should be noted that the present invention uses the GDALWarp tool of the GDAL library to rasterize the semantic segmentation results, and converts the obtained vector boundaries into wkt strings through the API of the GDAL library: GetLayer(), GetFeature(), GetGeometryRef() and other functions, and uses the wkt string to crop the clustering map to obtain the overlay map.

[0026] S4, determining the percentage w of the area of ​​each color pixel in the overlay image to the area of ​​the pixel of the corresponding color in the color cluster image; if w is greater than W, retain the color; otherwise, discard the color to obtain a preliminary cluster extraction image;

[0027] S5. Binarize the preliminary cluster extraction map to obtain a color cluster extraction map.

[0028] S6. Use the representation value to calculate the consistency of the color cluster extraction map and the semantic segmentation result map, and pair the current consistency calculation result and the color pixel area ratio threshold in ascending order and put them into the set S;

[0029] It should be noted that, in step S6, the representation value includes an image space representation value, an image kernel representation value, an image normalization kernel representation value, and an image stabilization representation value.

[0030] The image space representation value is calculated as follows:

[0031]

[0032] Where I(i, j) is the pixel value at pixel (i, j), and the value of m+n is z, that is, the z-degree representation value.

[0033] The calculation formula of the image kernel representation value is as follows:

[0034]

[0035] in

[0036] The image normalization kernel representation values ​​are as follows:

[0037]

[0038] Image stabilization characterization values ​​include 7, and the calculation formula is as follows:

[0039] V1=qs 20 +qs 02

[0040] V2=(qs 20 -qs 02 ) 2 +4qs 11 2

[0041] V3=(qs 30 -3qs 12 ) 2 +(3qs 21 -qs 03 ) 2

[0042] V4=(qs 30 +qs 12 ) 2 +(qs 21 +qs 03 ) 2

[0043] V5=(qs 30 -3qs 12 )(qs 30 +qs 12 )[(qs 30 +qs 12 ) 2 -3(qs 21 +qs 03 ) 2 ]+(3qs 21 -qs 03 )(qs 21 +qs 03 )[3(qs 30 +qs 12 ) 2 -(qs 21 +qs 03 ) 2 ]

[0044] V6=(qs 20-qs 02 )[(qs 30 +qs 12 ) 2 -(qs 21 +qs 03 ) 2 ]+4qs 11 (qs 30 +qs 12 )(qs 21 +qs 03 )

[0045] V7=(3qs 21 -qs 03 )(qs 21 +qs 03 )[3(qs 30 +qs 12 ) 2 -(qs 21 +qs 03 ) 2 ]-(qs 30 -3qs 12 )(qs 21 +qs 03 )[3(qs 30 +qs 12 ) 2 -(qs 21 +qs 03 ) 2 ].

[0046] In step S6, the calculation formula for the consistency of the two images is as follows:

[0047]

[0048] in are the i-th stable representation values ​​of the two images A and B respectively.

[0049] S7. Determine whether W is greater than a preset threshold a. If so, traverse the set S, calculate the absolute value threshold of the difference between the compliance calculation results corresponding to the adjacent color pixel ratio threshold, and put it into the set Q until the traversal is completed, and then proceed to step S8. Otherwise, W is incremented by Δt each time, and steps S4-S6 are repeated. In this embodiment of the present invention, the threshold a is 0.9 and Δt is 0.1.

[0050] S8. Traverse the set Q to obtain the minimum threshold. When the number of times the minimum threshold value appears is greater than the preset threshold b, select the first threshold that appears, and determine the left boundary of the color pixel area ratio threshold range corresponding to it as the optimal color pixel area ratio threshold bestW;

[0051] S9, determining the percentage w of the area of ​​each color pixel in the overlay image to the area of ​​the pixel of the corresponding color in the color clustering image. If w is greater than bestW, retain the color; otherwise, discard the color to obtain a preliminary building semantic segmentation correction extraction image;

[0052] S10, binarize the preliminary building semantic segmentation correction extraction map to obtain the final building semantic segmentation correction extraction map. Figure 4 , Figure 4 This is the building semantic segmentation correction result image in the present invention.

[0053] It should be noted that, in the present invention, the preliminary building semantic segmentation correction extraction map is binarized through the API of the OpenCV library: threshold() function to obtain the building semantic segmentation correction extraction map.

[0054] The beneficial effects of the present invention are: by means of the calculation of the corresponding color proportion and image conformity, the redundant non-building parts are effectively eliminated, and the missing building parts are filled in at the same time, the outline of the building can be effectively extracted, and the correction and extraction of the semantic segmentation results can be achieved, providing an effective method for the post-processing field of semantic segmentation, correction and extraction of remote sensing image buildings.

[0055] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for correcting and extracting semantic segmentation results of buildings in remote sensing images, characterized by: The following steps are involved: S1, read the source building image and its semantic segmentation result image; S2. Applying mean shift algorithm to the source building image to obtain a color clustering image; S3. Set an initial color pixel area ratio threshold W for the color cluster map, superimpose the outline of the semantic segmentation result map and the color cluster map, and take the intersection to obtain an overlay map; S4, determining the percentage w of the area of ​​each color pixel in the overlay image to the area of ​​the pixel of the corresponding color in the color cluster image; if w is greater than W, retain the color; otherwise, discard the color to obtain a preliminary cluster extraction image; S5. Binarize the preliminary cluster extraction map to obtain a color cluster extraction map; S6. Use the representation value to calculate the consistency of the color cluster extraction map and the semantic segmentation result map, and pair the current consistency calculation result and the color pixel area ratio threshold in ascending order and put them into the set S; S7. Determine whether W is greater than a preset threshold a. If so, traverse the set S, find the absolute value threshold of the difference between the compliance calculation results corresponding to the adjacent color pixel proportion threshold, and put it into the set Q until the traversal is completed, and then go to step S8. Otherwise, W is incremented by Δt each time, and steps S4-S6 are repeated. S8. Traverse the set Q to obtain the minimum threshold. When the number of times the minimum threshold value appears is greater than the preset threshold b, select the first threshold that appears, and determine the left boundary of the color pixel area ratio threshold range corresponding to it as the optimal color pixel area ratio threshold bestW; S9, determining the percentage w of the area of ​​each color pixel in the overlay image to the area of ​​the pixel of the corresponding color in the color clustering image. If w is greater than bestW, retain the color; otherwise, discard the color to obtain a preliminary building semantic segmentation correction extraction image; S10, binarizing the preliminary building semantic segmentation correction extraction map to obtain a final building semantic segmentation correction extraction map; In step S6, the representation value includes an image space representation value, an image kernel representation value, an image normalization kernel representation value, and an image stabilization representation value; The image space representation value is calculated as follows: Where i and j represent the row and column numbers of the image respectively, I(i, j) is the pixel value at pixel (i, j), m is the index of image row number i, n is the index of image column number j, and the value of m+n is z, qm n That is, the z-degree representation value of the image; The calculation formula of the image kernel representation value is as follows: in The image normalization kernel representation values ​​are as follows: Image stabilization characterization values ​​include 7, and the calculation formula is as follows: V1=qs 20 +qs 02 <h2 style=";text-align:left;direction:ltr">V2=(qs<h2 style=";text-align:left;direction:ltr"> 20 <h2 style=";text-align:left;direction:ltr"> -qs<h2 style=";text-align:left;direction:ltr"> 02 <h2 style=";text-align:left;direction:ltr"> )<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +4qs<h2 style=";text-align:left;direction:ltr"> 11 <h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr">V3=(qs<h2 style=";text-align:left;direction:ltr"> 30 <h2 style=";text-align:left;direction:ltr"> -3qs<h2 style=";text-align:left;direction:ltr"> 12 <h2 style=";text-align:left;direction:ltr"> )<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +(3qs<h2 style=";text-align:left;direction:ltr"> 21 <h2 style=";text-align:left;direction:ltr"> -qs<h2 style=";text-align:left;direction:ltr"> 03 <h2 style=";text-align:left;direction:ltr"> )<h2 style=";text-align:left;direction:ltr"> 2 V4=(qs 30 +qs 12 ) 2 +(qs 21 +qs 03 ) 2 V5=(qs 30 -3qs 12 )(qs 30 +qs 12 )[(qs 30 +qs 12 ) 2 -3(qs 21 +qs 03 ) 2 ]+(3qs 21 -qs 03 )(qs 21 +qs 03 )[3(qs 30 +qs 12 ) 2 -(qs 21 +qs 03 ) 2 ] V6=(qs 20 -qs 02 )[(qs 30 +qs 12 ) 2 -(qs 21 +qs 03 ) 2 ]+4qs 11 (qs 30 +qs 12 )(qs 21 +qs 03 ) V7=(3qs 21 -qs 03 )(qs 21 +qs 03 )[3(qs 30 +qs 12 ) 2 -(qs 21 +qs 03 ) 2 ]-(qs 30 -3qs 12 )(qs 21 +qs 03 )[3(qs 30 +qs 12 ) 2 -(qs 21 +qs 03 ) 2 ]。 2. The method for correcting and extracting semantic segmentation results of remote sensing images of buildings according to claim 1, wherein: In step S6, the calculation formula for the consistency of the two images is as follows: in are the i-th stable representation values ​​of the two images A and B respectively.

Images