Building facade right-angle polygon element graph recognition method based on intersection-to-union ratio

Abstract

The invention discloses a building facade right-angle polygon element graph recognition method based on intersection-to-union ratio, and belongs to the technical field of building modeling, and the method comprises the following steps: obtaining an initial image of one element of a building facade to form a recognition graph; identifying a minimum enclosing rectangle of the pattern; identifying a maximum inscribed pattern of the pattern; all candidate frames are taken between the external frame and the internal frame, and the intersection-to-union ratio of each candidate frame and the recognition graph is calculated; selecting the candidate frame with the maximum intersection-to-union ratio as a reduction frame of the identified graph; repeating the steps to obtain a reduction frame of all elements of the building facade; the obtained reduction frames are stacked on the same layer, the optimized graph is a rectangle or polygon with very smooth edges, and the contour of the optimized graph is more consistent with the contour of a real facade element.

Description

Technical Field

[0001] The invention relates to the technical field of building modeling, in particular to a method for optimizing irregular building facade element recognition graphics into regular graphics only according to intersection-and-union ratios. Background Art

[0002] Currently, most elements on building facades (doors, windows, etc.) are composed of rectangles.

[0003] Semantic segmentation of building facades is an important prerequisite for building pattern recognition, 3D reconstruction and other tasks. However, when semantic segmentation is performed directly on building facades, the recognition result is often an irregular graph with jagged edges of building facade elements, which has too much interference information, such as Figure 1 To this end, some related technologies have proposed some methods for optimizing irregular graphics into regular graphics with smooth edges. Currently, the main technical routes are as follows:

[0004] 1. Approximate polygon based method

[0005] Approximate polygon refers to the reconstruction of the outline of a figure using its points, area, main axis and other features, where:

[0006] The vertex-based method is for irregular contours with multiple vertices. The Douglas-Peucker algorithm (DP algorithm) uses as few vertices as possible to represent the contour line, and gradually eliminates unimportant vertices according to the distance between different vertices to achieve the effect of smoothing the edge of the figure. The elevation information of the figure is taken into account through the digital surface model (DSM). Although it makes up for the shortcomings of the DP algorithm that is sensitive to noise and starting point position, it is too dependent on the elevation parameters of the image.

[0007] Region-based methods usually use three approximation strategies, namely, external minimum area polygon approximation (EMAPA), internal maximum area polygon approximation (IMAPA) and minimum area deviation polygon approximation (MADPA), and can flexibly adjust the number of edges of the graph to adapt to different application scenarios. By using the minimum enclosing rectangle of the irregular contour combined with the Hausdorff distance method to regularize the building contour, it has a better effect on small missing parts caused by vegetation occlusion, shadows and other noise.

[0008] The main axis-based method finds the dominant direction of the building based on the parallelism and orthogonality of the building's contour lines. Based on the main axis, the building's edge lines are decomposed, angle-corrected, streamlined, and spliced ​​to achieve regular line segments and right-angle turns.

[0009] 2. Method based on active contour model

[0010] The active contour model (also known as the Snake model) is a deformable continuous curve whose shape is controlled by internal forces (implicit model) and external forces (data). The internal force acts as a smoothness constraint, and the external force guides the active contour to move in the direction of the image features, gradually optimizing the irregular contour. On this basis, the b-spline curve is used to optimize the contour, and the limited number of processing points of the local control curve improves the convergence speed and the reliability of segmentation. The gradient vector flow (GVF) expands the external force range of the classic snake, strengthens the attraction to the edge of the target concave contour, and improves the traditional snake model. The generalized gradient vector flow (GGVF) improves the convergence ability of GVF for long thin indentations (LTI) and its robustness to noise. GGVF is combined with iterative optimal minimum bounding rectangle (recursive BMBR, RBMBR) to optimize the contours of each part of the building using point projection. Compared with GGVF, the aliasing of the edges is reduced and the contour is made more regular.

[0011] 3. Morphology-based methods

[0012] Morphological operations are simple transformations of binary or grayscale images that can increase or decrease the size of objects in the image, and can be used to narrow or open the gaps between objects. Using erosion and dilation operations to optimize the marking of object contours can reduce over-segmentation and under-segmentation problems.

[0013] 4. Other methods

[0014] The two-dimensional initial building outline is converted into a one-dimensional signal, and the jagged shape in the two-dimensional initial building outline is converted into noise in the signal, so that the regularization problem becomes a signal denoising problem. The improved PointNet generates a detailed building graphic vector result by learning the prior shape and predicting the deformation of polygon vertices.

[0015] However, the existing solutions usually have defects / deficiencies: the above technical route is very sensitive to the quality of the initial image, and the optimization effect is better on graphics with less irregularity. However, the recognition results of semantic segmentation are often very irregular. When optimizing the contours of very irregular graphics, the optimization results of the existing technical route are often still relatively irregular and severely deformed; the existing technical route is more inclined to retain small concave and convex structures on the edges of graphics. On the facades of buildings, most graphics are regular shapes with smooth edges, so it is inappropriate to retain small concave and convex structures. Summary of the invention

[0016] The core reason why the current technical route has the above problems is that it only considers how to optimize the shape contour based on the recognition results of semantic segmentation, but does not consider the geometric characteristics of the semantic segmentation object itself. As a result, the current solution can only make the shape contour gradually approach the optimal value based on some abstract geometric indicators, rather than making the shape contour approach its corresponding real facade element.

[0017] In view of at least one of the above technical problems, the present invention provides a method for identifying right-angled polygonal elements of building facades based on intersection-and-union ratio, and uses the intersection-and-union ratio of the identified irregular figure and the "rectangular frame coincident with it" as the basis for contour optimization. The optimized figure is a rectangle or polygon with very smooth edges, and the contour of the optimized figure is more consistent with the contour of the real facade element. The specific technical solution is as follows:

[0018] A method for identifying right-angled polygonal elements of building facades based on intersection-and-union ratio, comprising the following steps:

[0019] S1, obtain the initial image of one element of the building facade into a recognition figure; identify the minimum circumscribed rectangle of the figure; identify the maximum inscribed figure of the figure; take all candidate frames between the circumscribed frame and the inscribed frame, calculate the intersection and union ratio of each candidate frame and the recognition figure; select the candidate frame with the largest intersection and union ratio as the restoration frame of the recognized figure;

[0020] S2, repeat the step of S1 to obtain the restored frame of all elements of the building facade;

[0021] S3, stacking the restored frames obtained in S2 onto the same layer in the order of intersection ratio from small to large.

[0022] In some embodiments of the present disclosure, in step S1, a pixel set is established when recognizing a building facade image, and the pixel set includes recognized building pixels and recognized background pixels, and the outline of the recognized graphic is an outline mark with building pixels on the inside and background pixels on the outside.

[0023] In some embodiments of the present disclosure, the contour identifier includes straight line identifiers and / or corner point identifiers; a rectangular contour established by overlapping the outermost straight line identifiers and / or corner point identifiers is the minimum circumscribed rectangle, and a rectangular contour established by overlapping the innermost straight line identifiers and / or corner point identifiers is the maximum inscribed rectangle.

[0024] In some embodiments of the present disclosure, the contour identification also includes a manual correction identification.

[0025] In some embodiments of the present disclosure, in step S3, the restoration frames are stacked on the same layer in the order of intersection-over-union ratio from small to large.

[0026] Compared with the prior art, the above-mentioned method for identifying right-angled polygonal elements of building facades based on intersection-and-union ratio has the following beneficial effects:

[0027] This method uses the intersection and union ratio of the identified irregular shape and the "rectangular frame overlapping with it" as the basis for contour optimization. The optimized shape is a rectangle or polygon with very smooth edges, and the contour of the optimized shape is more consistent with the contour of the real facade element. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is an example of the result of direct semantic segmentation of the building facade;

[0029] Figure 2 Two exemplary schematic diagrams of the method of the present invention for generating non-rectangular facade elements by superimposing multiple rectangular elements;

[0030] Figure 3 Schematic diagram of the intersection and union ratio of four forms of graphics of different shapes and their own overlapping rectangular frames in the method of the present invention;

[0031] Figure 4 Schematic diagram of four overlapping frames;

[0032] Figure 5 The optimized flow chart of the invention. DETAILED DESCRIPTION

[0033] In order to better understand the purpose, structure and function of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below. It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the present application. The terms used herein in this application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0034] As shown in the attached figure Figures 1 to 5 As shown in the figure, a method for identifying right-angled polygonal elements of building facades based on intersection-and-union ratio is designed. The principle of this scheme is based on the following two premises:

[0035] First of all, most of the elements on the building facade (doors, windows, etc.) are composed of rectangles. For some L-shaped and U-shaped ones, they can also be understood from the perspective of layers as the result of several rectangles superimposed front and back, such as Figure 2 shown.

[0036] Secondly, if a rectangular element is occluded by another rectangular element and becomes non-rectangular, such as L-shaped or U-shaped, then the Intersection over Union (IoU) between this element and the "rectangle that overlaps best with it" will be lower. The more severe the occlusion, the lower the IoU. Figure 3 shown.

[0037] Under the above premise, for the identification graphics of any facade element;

[0038] Among them, when recognizing the building facade image, a pixel set is established, and the pixel set includes the recognition building pixels and the recognition background pixels. The outline of the recognition figure is the outline mark with the building pixels on the inside and the background pixels on the outside, and specifically, the 8 pixels around each building pixel are screened. If the adjacent pixel points of the building pixel point close to the inside of the recognition figure are building pixels and the adjacent pixel points close to the outside of the recognition figure are background pixels, it can be determined that the building pixel point is a contour pixel, which is used as the basis for forming the contour; if the 8 pixel points around the building pixel point are all background pixels, they can be directly screened out.

[0039] The outline identifier includes a straight line identifier and / or a corner point identifier. If two pixel points above and below or to the left and right of the above-mentioned building pixel point are building pixels and the outer pixel is a background pixel, then this point is a component of the straight line identifier. If one pixel point above and below and to the left and right of the above-mentioned pixel point is a building pixel and forms an L-shaped right angle with the building pixel point, and at the same time, the inner side of the building pixel point is a building pixel and the outer side is a background pixel, then this building pixel point is a corner point identifier; a rectangular outline established by overlapping the outermost straight line identifiers and / or corner point identifiers is the minimum circumscribed rectangle, and a rectangular outline established by overlapping the innermost straight line identifiers and / or corner point identifiers is the maximum inscribed rectangle;

[0040] In this technical solution, an artificial correction mark can be established as needed as the basis for the contour mark.

[0041] This scheme uses four kinds of rectangular frames that have overlapping relationships with it, namely overlapping frames, such as Figure 4 As shown; optimize it into a regular graph:

[0042] ①Bounding box: Identify the smallest bounding rectangle of a graphic, such as Figure 4 The red frame in the

[0043] ② Inscribed box: Identify the largest inscribed rectangle of a graphic, such as Figure 4 The blue inner frame in the

[0044] ③Candidate box: any rectangular box between the outer box and the inner box, such as Figure 4 The yellow rectangle in the middle;

[0045] ④Restored box: The candidate box with the largest intersection-over-union ratio with the recognized figure, such as Figure 4 The green rectangular frame in the

[0046] With the help of the above four boxes, the optimization process proceeds as follows:

[0047] a. For any facade element identified as an irregular shape, assign an external frame and an internal frame to the identified shape.

[0048] Frame, such as Figure 5 As shown in part b;

[0049] b. Traverse all candidate boxes between the external box and the internal box, and calculate the intersection of each candidate box and the recognition figure

[0050] for example Figure 5 As shown in part c;

[0051] c. Among all the candidate frames, select the one with the largest intersection-union ratio as the restoration frame for the recognized graph, such as Figure 5 As shown in part d;

[0052] d. Repeat the above process until the restoration frame of the facade element identification graphic is found, such as Figure 5 As shown in part e;

[0054] e. According to the size of the intersection-and-union ratio between each restoration frame and its corresponding recognition figure, stack the restoration frames one by one on the same layer. Stack the ones with smaller intersection-and-union ratio first, and stack the ones with larger intersection-and-union ratio later. Until all the restoration frames are stacked,

[0055] After the optimization is completed, a regular building facade semantic segmentation map is obtained, such as Figure 5 As shown in part f.

[0056] It can be understood that the above description is only for illustrating the specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the present application should be included in the disclosure scope of the present application.

Claims

1. A method for identifying right-angled polygonal elements of building facades based on intersection-and-union ratio, comprising the following steps: S1, obtain the initial image of one element of the building facade into a recognition figure; identify the minimum circumscribed rectangle of the figure; identify the maximum inscribed figure of the figure; take all candidate frames between the circumscribed frame and the inscribed frame, calculate the intersection and union ratio of each candidate frame and the recognition figure; select the candidate frame with the largest intersection and union ratio as the restoration frame of the recognized figure; S2, repeat the step of S1 to obtain the restoration frame of all elements of the building facade; S3, stacking the restored frames obtained in S2 on the same layer.

2. The method for identifying right-angled polygonal elements of building facades based on intersection-and-union ratio according to claim 1, characterized in that: In the step S1, a pixel set is established when the building facade image is recognized, and the pixel set includes recognized building pixels and recognized background pixels. The outline of the recognized figure is an outline mark with building pixels on the inside and background pixels on the outside.

3. The method for identifying right-angled polygonal elements of building facades based on intersection-and-union ratio according to claim 2, characterized in that: The contour identifier includes straight line identifiers and / or corner point identifiers; a rectangular contour established by overlapping the outermost straight line identifiers and / or corner point identifiers is the minimum circumscribed rectangle, and a rectangular contour established by overlapping the innermost straight line identifiers and / or corner point identifiers is the maximum inscribed rectangle.

4. The method for identifying right-angled polygonal elements of building facades based on intersection-and-union ratio according to claim 3, characterized in that: The outline marking also includes a manual correction marking.

5. The method for identifying right-angled polygonal elements of building facades based on intersection-and-union ratio according to claim 1, characterized in that: In step S3, the restoration frames are stacked on the same layer in the order of the intersection-union ratio from small to large.

Images