A Fast Regularization Method for Remote Sensing Building Interpretation

Abstract

In view of the limitations of the prior art, the present invention proposes a rapid regularization method for remote sensing building interpretation; the present invention can rapidly regularize the irregular masks predicted and output by the instance segmentation algorithm, and can obtain the optimal matching rectangular frame of the remote sensing building without cumbersome calculations, providing information such as angle, position, length and width, providing data support for subsequent application requirements, and effectively solving the technical problem that it is difficult to perform subsequent analysis on the irregular masks.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image segmentation, and specifically, to a fast regularization method for remote sensing building interpretation. Background Art

[0002] Semantic segmentation and instance segmentation algorithms are widely used in remote sensing ground object interpretation, especially in the extraction of buildings. Through the instance segmentation algorithm, the mask of a single building can be obtained from a remote sensing image, but the result is irregular and does not conform to the regular rectangular surface of the roof contour, making it difficult to directly apply to building mapping such as planning and disaster assessment.

[0003] In the Chinese invention patent with the publication date of August 6, 2019: A method for vectorizing houses for multi-source remote sensing data, using the detected or extracted house patches (or house point sets, edge lines) as known data, and based on the multi-class segmentation of house edge line segments using the α-expansion theory, corresponding processing is carried out respectively on the house extraction results of different remote sensing data: for the case with high house extraction accuracy, using the prior knowledge of the house shape, combining line fitting to find the corner points; for the case with low house extraction accuracy, using the feature that the image data has a high sampling rate at the edge, constructing an adaptive edge template to accurately locate each edge line segment, and on the basis of obtaining accurate edge line segments, combining line fitting to find the corner points; in order to obtain house contour information that is more consistent, effective, and regular with the house shape. However, the existing technology still has certain limitations. Summary of the Invention

[0004] Aiming at the limitations of the existing technology, the present invention proposes a fast regularization method for remote sensing building interpretation. The technical solution adopted by the present invention is as follows:

[0005] A fast regularization method for remote sensing building interpretation includes the following steps:

[0006] S1, input the original segmentation result obtained by the instance segmentation algorithm, and read the instance mask in the original segmentation result;

[0007] S2, obtain the original pixel area and the minimum bounding rectangle of the instance mask;

[0008] S3, taking the minimum bounding rectangle as the initial rectangle, keeping the aspect ratio of the rectangle, and taking the minimum absolute difference between the pixel area of the rectangle and the original pixel area as the optimization goal, using the long side of the rectangle as the iteration benchmark, and iteratively obtaining the optimal rectangle of the instance mask by continuously reducing the length and width values of the rectangle.

[0009] Compared with the prior art, the present invention can regularize the irregular mask predicted and output by the instance segmentation algorithm, and can obtain the best matching rectangular frame of the remote sensing building without cumbersome calculations, providing information such as angle, position, length, and width, providing data support for subsequent application requirements, and effectively solving the technical problem that it is difficult to perform subsequent analysis on the irregular mask.

[0010] As a preferred solution, the following steps are further included:

[0011] S4. Calculate the overlap degree between the optimal rectangles; filter the optimal rectangles whose overlap degree exceeds the preset overlap degree threshold.

[0012] As a preferred solution, the following steps are further included:

[0013] S5. Convert the optimal rectangle from the grid space coordinates to the geographic coordinates.

[0014] Furthermore, the following process is included in the step S2:

[0015] S21. Extract the outermost contour of the instance mask;

[0016] S22. Obtain the original pixel area of the instance mask according to the outermost contour;

[0017] S23. Obtain the center point, length, width, and rotation angle of the minimum circumscribed rectangle of the instance mask according to the outermost contour, and calculate the pixel area of the minimum circumscribed rectangle.

[0018] Furthermore, the following method is used to calculate the length and width values of the rectangle during the iteration process in the step S3:

[0019]

[0020]

[0021] Among them, w * , h * respectively represent the length and width values updated in this round of iteration; w and h respectively represent the length and width values of the input rectangle in this round of iteration; i represents the field edge trimming value; long_size represents the long side of the input rectangle in this round of iteration.

[0022] Furthermore, the following process is included in the step S4:

[0023] S41. Obtain the vertex coordinates of each optimal rectangle, and screen out the overlapping rectangle pairs with overlapping coordinate regions according to the vertex coordinates;

[0024] S42. Create a binary image for the optimal rectangle involved in the overlapping rectangle pair according to the vertex coordinates; obtain the overlapping area image by performing an AND operation on the binary image.

[0025] S43. Calculate the overlapping area of the overlapping rectangle pair according to the overlapping area image, and calculate the proportions iou1 and iou2 that the overlapping area occupies in the two optimal rectangles rect1 and rect2 in the overlapping rectangle pair respectively.

[0026] S44. Screen the two optimal rectangles in the overlapping rectangle pair through the following formula:

[0027]

[0028] Further, the step S5 is performed according to the following formula:

[0029]

[0030] Among them, X p , Y p represent geographic coordinates, x and y represent grid space coordinates; geoTransform[] is the affine transformation coefficient for converting between the grid space and the projected coordinate space in the GDAL library.

[0031] The present invention further includes the following content:

[0032] A fast regularization system for remote sensing building interpretation, including an instance mask reading module, a raw pixel area and minimum bounding rectangle obtaining module, and a rectangle iteration module connected in sequence; wherein:

[0033] The instance mask reading module is used to input the original segmentation result obtained by the instance segmentation algorithm and read the instance mask in the original segmentation result.

[0034] The raw pixel area and minimum bounding rectangle obtaining module is used to obtain the raw pixel area and the minimum bounding rectangle of the instance mask.

[0035] The rectangle iteration module is used to use the minimum bounding rectangle as the initial rectangle, keep the aspect ratio of the rectangle, take the minimum absolute difference between the pixel area of the rectangle and the raw pixel area as the optimization goal, take the long side of the rectangle as the iteration benchmark, and iteratively obtain the optimal rectangle of the instance mask by continuously reducing the length and width values of the rectangle.

[0036] Compared with the prior art, the present invention can regularize the irregular masks predicted and output by the instance segmentation algorithm, and can obtain the optimal matching rectangular frame of the remote sensing building without cumbersome calculations, providing information such as angle, position, length, and width, providing data support for subsequent application requirements, and effectively solving the technical problem that it is difficult to perform subsequent analysis on irregular masks.

[0037] As a preferred solution, it further includes an overlapping filtering module; the overlapping filtering module is connected to the rectangular iteration module;

[0038] The overlapping filtering module is used to calculate the overlapping degree between the optimal rectangles; and filter the optimal rectangles whose overlapping degree exceeds the preset overlapping degree threshold.

[0039] As a preferred solution, it further includes a coordinate conversion module; the coordinate conversion module is connected to the rectangular iteration module or the overlapping filtering module:

[0040] The coordinate conversion module is used to convert the optimal rectangle from the grid space coordinate to the geographic coordinate.

[0041] A computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the steps of the fast regularization method for remote sensing building interpretation as described above are implemented.

[0042] A computer device includes a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor. When the computer program is executed by the processor, the steps of the fast regularization method for remote sensing building interpretation as described above are implemented. Description of the Drawings

[0043] Figure 1 It is a schematic flowchart of a fast regularization method for remote sensing building interpretation provided in Embodiment 1 of the present invention;

[0044] Figure 2 It is an example of the original remote sensing image;

[0045] Figure 3 For Figure 2 The original segmentation result obtained after being processed by the instance segmentation algorithm;

[0046] Figure 4 It is a single instance mask read in step S1 in Embodiment 1 of the present invention;

[0047] Figure 5 It is another schematic flowchart of a fast regularization method for remote sensing building interpretation provided in Embodiment 1 of the present invention;

[0048] Figure 6 It is a schematic flowchart of step S2 in Embodiment 1 of the present invention;

[0049] Figure 7 This is an example of the outermost contour of a single instance mask in Embodiment 1 of the present invention;

[0050] Figure 8 This is an example of the minimum bounding rectangle of a single instance mask in Embodiment 1 of the present invention;

[0051] Figure 9 This is an example of the minimum bounding rectangle of each instance mask from the overall perspective in Embodiment 1 of the present invention;

[0052] Figure 10 This is an example of the optimal rectangle of a single instance mask in Embodiment 1 of the present invention;

[0053] Figure 11 This is an example of the optimal rectangle of each instance mask from the overall perspective in Embodiment 1 of the present invention;

[0054] Figure 12 This is a schematic flow diagram of step S4 in Embodiment 1 of the present invention;

[0055] Figure 13 and Figure 14 are examples of the optimal rectangle before and after overlap filtering;

[0056] Figure 15 This is a schematic diagram of a rapid regularization system for remote sensing building interpretation provided in Embodiment 2 of the present invention;

[0057] Figure 16 This is a schematic diagram of another rapid regularization system for remote sensing building interpretation provided in Embodiment 2 of the present invention. Detailed implementation manners

[0058] The accompanying drawings are only for illustrative purposes and should not be construed as a limitation of this patent;

[0059] It should be clear that the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts belong to the scope protected by the embodiments of the present application.

[0060] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application. The singular forms "a", "the" and "said" used in the embodiments of the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0061] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numerals in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances.

[0062] In addition, in the description of the present application, unless otherwise specified, "a plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after. The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0063] To solve the limitations of the prior art, this embodiment provides a technical solution, and the technical solution of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0064] Embodiment 1

[0065] Please refer to Figure 1 , a rapid regularization method for remote sensing building interpretation, comprising the following steps:

[0066] S1. Input the original segmentation result obtained by the instance segmentation algorithm and read the instance mask in the original segmentation result;

[0067] S2. Obtain the original pixel area and the minimum bounding rectangle of the instance mask;

[0068] S3. Taking the minimum bounding rectangle as the initial rectangle, keeping the aspect ratio of the rectangle, and taking the minimum absolute difference between the pixel area of the rectangle and the original pixel area as the optimization goal, and taking the long side of the rectangle as the iteration benchmark, iteratively obtain the optimal rectangle of the instance mask by continuously reducing the length and width values of the rectangle.

[0069] Compared with the prior art, the present invention can quickly regularize the irregular mask predicted and output by the instance segmentation algorithm, and can obtain the best matching rectangle frame of the remote sensing building without cumbersome calculations, providing information such as angle, position, length and width, etc., providing data support for subsequent application requirements, and effectively solving the technical problem that it is difficult to perform subsequent analysis on the irregular mask.

[0070] After being processed by the instance segmentation algorithm, as Figure 2 shown, the original remote sensing image will be converted into an original mask image as Figure 3 shown; it can be seen that Figure 3 contains a large number of irregular instance masks.

[0071] Specifically, in the step S1, the GDAL (Geospatial Data Abstraction Library) library can be used to read the original segmentation result (TIFF format) of the instance segmentation algorithm, and convert the GDAL data format into the Opencv data format. Calculate the histogram of the Opencv data format, with the pixel range from 0 to 255, where the connected component with a pixel value of 0 is the background, and each connected component with a non-zero pixel value represents an instance. Represent the mask images of all instances separately (as Figure 4 shown), and store all the mask images into an array.

[0072] In the step S3, during the iteration of each instance mask, the absolute value of the difference between the pixel area of the minimum bounding rectangle and the original pixel area can be used as the basic difference. By iteratively calculating for each instance mask, a set of absolute values of the differences between the pixel area of the new rectangle and the original pixel area is obtained, and the length and width corresponding to the minimum absolute difference value in each set are selected as the optimal rectangle length and width of the corresponding instance mask.

[0073] Please refer to Figure 5 , as a preferred embodiment, it further includes the following steps:

[0074] S4. Calculate the overlap degree between the optimal rectangles; filter the optimal rectangles whose overlap degree exceeds the preset overlap degree threshold.

[0075] Specifically, through the step S4, the instance masks with a high overlap degree (i.e., the obtained optimal rectangles) can be filtered out. As a preferred embodiment, the overlap degree threshold can be set to 0.4.

[0076] As a preferred embodiment, it further includes the following steps:

[0077] S5. Convert the optimal rectangles from the grid space coordinates to the geographic coordinates.

[0078] Specifically, by converting the optimal rectangles from the grid space coordinates to the geographic coordinates, spatial information can be given to the regularized rectangle results, providing better data support for subsequent application requirements.

[0079] Furthermore, please refer to Figure 6 , the following process is included in the step S2:

[0080] S21. Extract the outermost contour of the instance mask;

[0081] S22. Obtain the original pixel area of the instance mask according to the outermost contour;

[0082] S23. Obtain the center point, length, width and rotation angle of the minimum bounding rectangle of the instance mask according to the outermost contour, and calculate the pixel area of the minimum bounding rectangle.

[0083] Specifically, in step S21, the outermost contour (contours) of the building mask in the mask image array obtained in step S1 can be obtained by traversing the mask image array and using the findContours function in Opencv.

[0084] According to the outermost contour, in step S22, the original pixel area (area) of the building mask can be obtained by using the contourArea function in Opencv.

[0085] According to the outermost contour, in step S23, the minimum bounding rectangle (rect) of the building mask can be obtained by using the minAreaRect function in Opencv. The format of rect is ((x,y),(w,h),angle), where (x,y) is the center point of rect, (w,h) are the length and width of rect, and angle is the rotation angle of rect.

[0086] There is no specific order between step S22 and S23, and they can be carried out simultaneously.

[0087] As a preferred embodiment, after obtaining the minimum bounding rectangles of each mask, the masks (i.e., the obtained rectangles) with w or h less than 5 pixels in the minimum bounding rectangles can be filtered to exclude some non-building masks.

[0088] The outermost contour (i.e., the edge corner point image) obtained from the instance mask through step S21 is as Figure 7 shown; the minimum bounding rectangle obtained through step S23 is as Figure 8 , Figure 9 shown; among them, Figure 8 is the minimum bounding rectangle of a single instance mask, Figure 9 is the minimum bounding rectangle of each instance mask from an overall perspective.

[0089] Furthermore, step S3 calculates the length and width values of the rectangle in the iterative process in the following way:

[0090]

[0091]

[0092] Among them, w * and h * respectively represent the length and width values updated in this round of iteration; w and h respectively represent the length and width values of the input rectangle in this round of iteration; i represents the field edge trimming value; long_size represents the long side of the rectangle of the input rectangle in this round of iteration.

[0093] The optimal rectangle obtained by the instance mask through the step S3 is as Figure 10 and Figure 11 shown; among them, Figure 10 is the optimal rectangle of a single instance mask, Figure 11 is the optimal rectangle of each instance mask from an overall perspective.

[0094] Furthermore, please refer to Figure 12 , the following process is included in the step S4:

[0095] S41. Obtain the vertex coordinates of each optimal rectangle, and filter out the overlapping rectangle pairs with overlapping coordinate regions according to the vertex coordinates;

[0096] S42. According to the vertex coordinates, create a binary image for the optimal rectangles involved in the overlapping rectangle pairs; by performing an AND operation on the binary image, obtain an overlapping region image;

[0097] S43. Calculate the overlapping area of the overlapping rectangle pairs according to the overlapping region image, and calculate the proportions iou1 and iou2 of the overlapping area in the two optimal rectangles rect1 and rect2 in the overlapping rectangle pairs respectively;

[0098] S44. Screen the two optimal rectangles in the overlapping rectangle pairs through the following formula:

[0099]

[0100] Specifically, examples before and after overlap degree filtering are as Figure 13 and 14 shown.

[0101] In the step S41, the points function in Opencv can be used to calculate the 4 vertex coordinates of each optimal rectangle. By first screening all rectangles whose overlapping degrees are calculated pairwise, it is judged whether the coordinate regions of the two overlap, and the rectangle pairs with non-overlapping coordinates are filtered out, and those with overlap will be subject to subsequent operations. This can speed up the calculation speed and reduce the calculation amount.

[0102] In step S42, according to the vertex coordinates, the fillPoly function in Opencv can be used to create a binary image of each optimal rectangle, where the non-zero value area is the area corresponding to the optimal rectangle and other areas are the background.

[0103] In step S43, the bitwise_and function in Opencv can be used to perform an AND operation on the two binary images to obtain an overlapping area image.

[0104] Furthermore, step S5 is carried out according to the following formula:

[0105]

[0106] where X p and Y p represent geographical coordinates, x and y represent grid space coordinates; geoTransform[] is the affine transformation coefficient for converting between the grid space and the projected coordinate space in the GDAL library.

[0107] Embodiment 2

[0108] Please refer to Figure 15 , a fast regularization system for remote sensing building interpretation, including an instance mask reading module 1, an original pixel area and minimum bounding rectangle obtaining module 2, and a rectangle iteration module 3 connected in sequence; where:

[0109] The instance mask reading module 1 is used to input the original segmentation result obtained by the instance segmentation algorithm and read the instance mask in the original segmentation result;

[0110] The original pixel area and minimum bounding rectangle obtaining module 2 is used to obtain the original pixel area and minimum bounding rectangle of the instance mask;

[0111] The rectangle iteration module 3 uses the minimum bounding rectangle as the initial rectangle, keeps the aspect ratio of the rectangle, takes the minimum absolute difference between the pixel area of the rectangle and the original pixel area as the optimization goal, takes the long side of the rectangle as the iteration reference, and iteratively obtains the optimal rectangle of the instance mask by continuously reducing the length and width values of the rectangle.

[0112] Compared with the prior art, the present invention can quickly regularize the irregular mask predicted and output by the instance segmentation algorithm, obtain the best matching rectangle frame of the remote sensing building without cumbersome calculations, provide information such as angle, position, length and width, provide data support for subsequent application requirements, and effectively solve the technical problem that it is difficult to perform subsequent analysis on the irregular mask.

[0113] Please refer to Figure 16, as a preferred embodiment, it further includes an overlapping filtering module 4; the overlapping filtering module 4 is connected to the rectangular iteration module 3;

[0114] The overlapping filtering module 4 is used to calculate the overlapping degree between each optimal rectangle; and filter the optimal rectangles whose overlapping degree exceeds a preset overlapping degree threshold.

[0115] As a preferred embodiment, it further includes a coordinate conversion module 5; the coordinate conversion module 5 is connected to the rectangular iteration module 3 or the overlapping filtering module 4:

[0116] The coordinate conversion module is used to convert the optimal rectangle from grid space coordinates to geographic coordinates.

[0117] Embodiment 3

[0118] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the rapid regularization method for remote sensing building interpretation as described in Embodiment 1.

[0119] Embodiment 4

[0120] A computer device, including a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor, and when the computer program is executed by the processor, it implements the steps of the rapid regularization method for remote sensing building interpretation as described in Embodiment 1.

[0121] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, rather than limitations on the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation manners here. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A rapid regularization method for remote sensing building interpretation, characterized in that, It includes the following steps: S1. Input the original segmentation result obtained by the instance segmentation algorithm, and read the instance mask in the original segmentation result; S2. Obtain the original pixel area and the minimum bounding rectangle of the instance mask; S3. Use the minimum bounding rectangle as the initial rectangle, keep the aspect ratio of the rectangle, and take the minimum absolute difference between the pixel area of the rectangle and the original pixel area as the optimization goal. Take the long side of the rectangle as the iteration benchmark, and iteratively obtain the optimal rectangle of the instance mask by continuously reducing the length and width values of the rectangle.

2. The rapid regularization method for remote sensing building interpretation according to claim 1, characterized in that, It further includes the following steps: S4. Calculate the overlap degree between the optimal rectangles; filter the optimal rectangles whose overlap degree exceeds the preset overlap degree threshold.

3. The rapid regularization method for remote sensing building interpretation according to claim 1, characterized in that It further includes the following steps: S5. Convert the optimal rectangle from the grid space coordinates to the geographic coordinates.

4. The rapid regularization method for remote sensing building interpretation according to any one of claims 1 to 3, characterized in that, The following process is included in step S2: S21. Extract the outermost contour of the instance mask; S22. Obtain the original pixel area of the instance mask according to the outermost contour; S23. Obtain the center point, length, width and rotation angle of the minimum bounding rectangle of the instance mask according to the outermost contour, and calculate the pixel area of the minimum bounding rectangle.

5. The rapid regularization method for remote sensing building interpretation according to any one of claims 1 to 3, characterized in that, In step S3, the length and width values of the rectangle are calculated in the following way during the iteration process: Among them, w * and h * respectively represent the length and width values updated in this round of iteration; w and h respectively represent the length and width values of the input rectangle in this round of iteration; i represents the field edge trimming value; long_size represents the long side of the rectangle of the input rectangle in this round of iteration.

6. The rapid regularization method for remote sensing building interpretation according to claim 2, characterized in that, The following process is included in step S4: S41. Obtain the vertex coordinates of each optimal rectangle, and screen out the overlapping rectangle pairs with overlapping coordinate regions according to the vertex coordinates; S42. Create a binary image for the optimal rectangles involved in the overlapping rectangle pairs according to the vertex coordinates; obtain the overlapping area image by performing an AND operation on the binary image; S43. Calculate the overlapping area of the overlapping rectangle pairs according to the overlapping area image, and calculate the proportions iou1 and iou2 of the overlapping area in the two optimal rectangles rect1 and rect2 in the overlapping rectangle pairs respectively; S44. Screen the two optimal rectangles in the overlapping rectangle pairs through the following formula:

7. The rapid regularization method for remote sensing building interpretation according to claim 3, characterized in that Step S5 is performed according to the following formula: Among them, X p , Y p represent geographic coordinates, and x, y represent raster space coordinates; geoTransform[] is the affine transformation coefficient for converting between the raster space and the projected coordinate space in the GDAL library.

8. A rapid regularization system for remote sensing building interpretation, characterized in that, It includes an instance mask reading module (1), an original pixel area and minimum bounding rectangle obtaining module (2), and a rectangle iteration module (3) connected in sequence; where: The instance mask reading module (1) is used to input the original segmentation result obtained by the instance segmentation algorithm and read the instance mask in the original segmentation result; The original pixel area and minimum bounding rectangle obtaining module (2) is used to obtain the original pixel area and the minimum bounding rectangle of the instance mask; The rectangle iteration module (3) is used to use the minimum bounding rectangle as the initial rectangle, keep the aspect ratio of the rectangle, take the minimum absolute difference between the pixel area of the rectangle and the original pixel area as the optimization goal, take the long side of the rectangle as the iteration benchmark, and iteratively obtain the optimal rectangle of the instance mask by continuously reducing the length and width values of the rectangle.

9. A computer-readable storage medium storing a computer program thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the fast regularization method for remote sensing building interpretation according to any one of claims 1 to 7.

10. A computer device, characterized in that: It includes a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor. When the computer program is executed by the processor, it implements the steps of the rapid regularization method for remote sensing building interpretation according to any one of claims 1 to 7.

Images