Interactive extraction method for right-angle building roof under star shape constraint

Abstract

The application relates to a right-angle building roof interactive extraction method under star shape constraint, which comprises the following steps: estimating the initial position of a building based on a seed line; the seed line is a line marked by a user to indicate the position of the building; extracting a line segment on a building image, calculating the main direction of the building according to the direction histogram of the line segment, and rotating the building image to the main direction; dividing the building image into a series of rectangular blocks; integrating star shape constraint on the basis of OneCut algorithm, synchronously performing extraction and regularization of the rectangular building, and merging the extracted multiple rectangles to obtain the complex rectangular contour of the building; and the ordinary rectangular building and the complex right-angle building can be quickly and effectively extracted.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image extraction, and in particular to a star shape constraint under the right angle building roof interactive extraction method. BACKGROUND

[0002] Building occupies a very important position in remote sensing application, and its extraction method has been a research hotspot. As a kind of artificial object, building has its unique regularity, but due to various disturbances, it is very difficult to accurately extract the building roof profile.

[0003] And the graph model has a strong corresponding relationship with the image, such as the pixels or objects in the image can be regarded as the vertices in the graph model, the adjacent pixels or objects can form edges, and the association between pixels or objects can be regarded as the edge weight in the graph model, so the optimization theory in the graph model can be often applied to image processing. The prior art has various segmentation methods for different problems of image segmentation, and common graph optimization theory segmentation methods include shortest path based segmentation method, minimum spanning tree based segmentation method, and graph cut based segmentation method. The most famous algorithm of the shortest path based segmentation method is Dijkstra algorithm, in which the user selects several key points on the target contour, and the algorithm automatically finds the best segmentation line between the two points. The minimum spanning tree based segmentation method includes top-down splitting segmentation method and bottom-up merging segmentation method, and the segmentation process relies on the edge weight to achieve global optimization. The graph cut based segmentation method adds different functions and constraints to map the binary segmentation problem to the minimum cut maximum flow problem in graph theory, and realizes foreground extraction under global optimization.

[0004] The shortest path based segmentation method needs the user to select several key points on the target contour, and the interactive operation is relatively complex. The minimum spanning tree based segmentation method only relies on the edge weight, and is easily affected by noise, so it is necessary to add region consistency criterion to the segmentation process to improve the segmentation effect. The graph cut based segmentation method is relatively advanced, but the current mainstream method cannot be directly applied to extract the right angle building roof, and further integration and optimization are needed. SUMMARY

[0005] The present application provides a star shape constraint under the right angle building roof interactive extraction method, which can quickly and effectively extract ordinary rectangular buildings and complex right angle buildings.

[0006] According to the first aspect of the present application, a star shape constraint under the right angle building roof interactive extraction method is provided, which comprises:

[0007] Step 1, estimating the initial position of the building based on a seed line; the seed line is a line input by a user to mark the position of the building;

[0008] Step 2, extracting line segments on the building image, calculating the main direction of the building according to a direction histogram of the line segments, and rotating the building image to the main direction;

[0009] Step 3, dividing the building image into a series of rectangular blocks;

[0010] Step 4, integrating star shape constraints on the basis of One Cut algorithm, synchronously performing extraction and regularization of the rectangular building, and merging the extracted multiple rectangles to obtain a complex rectangular contour of the building.

[0011] On the basis of the above technical solutions, the application can also be improved as follows.

[0012] Optionally, the rules of the seed line input in step 1 include:

[0013] The seed line should be as much as possible in the middle position of the building;

[0014] The direction of the seed line is consistent with the direction of the building;

[0015] The length of the minimum bounding rectangle of the seed line is longer than two-thirds of the length of the building.

[0016] Optionally, the calculation formula of the line segment direction histogram in step 2 is:

[0017]

[0018] Wherein, θ∈[0,180) represents the direction of the line segment and the horizontal axis of the histogram; n θ is the intensity at the direction value θ of the histogram direction; l i ∈L represents a line segment; represents the set of all line segments with the direction θ in L; D represents the normalized distance map of the seed line, and D(x,y) represents the distance value at the point (x,y); represents the midpoint coordinates of the line segment l i ; Len(l i ) represents the length of the line segment l i .

[0019] Optionally, the formula for calculating the main direction of the building in step 2 is:

[0020]

[0021] After the main direction is calculated, the building image is rotated to the main direction by using the bilinear interpolation method.

[0022] Optionally, step 3 includes:

[0023] Step 301: Use the line segments with the same direction as the main direction as the reference dividing lines to perform preliminary segmentation of the building image;

[0024] Step 302: When the distance between any two adjacent reference dividing lines exceeds a set distance, a new dividing line is found between the two reference dividing lines. The new dividing line satisfies the maximum difference in the average pixel grayscale value within the buffer area on both sides. The building image is segmented using each of the new dividing lines.

[0025] Optionally, in step 302, the saliency of the dividing line is used to represent the difference in the average grayscale value of pixels within the buffer areas on both sides of the dividing line. When the direction of the new dividing line is vertical, its saliency calculation formula is:

[0026]

[0027] Where x is the x-coordinate of the dividing line, x∈[0,W); D(x,y) is the distance value at coordinate (x,y); W*H is the image size; L y This represents the set of pixels with a y-coordinate within the buffer to the left of the dividing line. L represents y Average grayscale value of medium pixels; R y This represents the set of pixels with a y-coordinate within the buffer to the right of the dividing line. R represents y Average grayscale value of mid-pixel.

[0028] Optionally, the energy function to be optimized for the One Cut algorithm integrating star-shaped constraints in step 4 is:

[0029]

[0030] Where S represents the prospective goal, Represents background; θ S and They are S and Histogram within; The L1 measure represents the overlap between the foreground and background models; p and q represent any two rectangular blocks; N represents the set of four neighboring pixel pairs; S p ∈{0,1} represents the label of rectangular block p; V pq (S p ,S q ) represents the boundary penalty term; Shape pq (S p ,S q) is a star shape constraint term; β, λ, γ represent the relative weight of each term.

[0031] Optionally, the implementation method of the star shape constraint term is: taking the circumscribed rectangle of the seed line as the center, dividing the building image into different blocks, and limiting the direction of the cut line in GraphCut in different blocks.

[0032] Optionally, the calculation formula of the star penalty term is:

[0033]

[0034] Where (x p ,y q ) and (x q ,y q ) represent the center coordinates of the rectangular block p and the rectangular block q respectively.

[0035] Optionally, the process of extraction in step 4 further includes:

[0036] Step 401: According to the parameters of each term of the energy function, an optimal result under the current shape constraint is obtained as an intermediate result;

[0037] Step 402: Calculate the maximum inscribed rectangle of the intermediate result, take the maximum inscribed rectangle as the center of the new shape constraint, update the region division, and recalculate the shape constraint parameters;

[0038] Step 403: Re-execute step 401 under the new shape constraint to obtain a new extraction result until the algorithm converges, thereby completing the building contour extraction and regularization.

[0039] The present application provides a kind of star shape constraint under the interactive extraction method of right-angle building roof, proposes a model-driven interactive extraction method of right-angle building, fast and effective extraction ordinary rectangle building and complex right-angle building, through user interactive input, iteration extraction optimal rectangular profile, can make user utilize small amount of simple interactive operation, obtain more reliable right-angle building roof profile, greatly reduce the time needed for manual drawing building, improve building acquisition efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 It is the flow chart of the interactive extraction method of right-angle building roof under the star shape constraint provided by the present application;

[0041] Figure 2 It is the schematic diagram of the specific embodiment of the interactive extraction method of right-angle building roof under the star shape constraint provided by the present application;

[0042] Fig. 3(a) is a schematic diagram of an embodiment of a general case of star shape constraint provided by the embodiment of the present application;

[0043] Fig. 3(b) is a schematic diagram of an embodiment of iterative shape constraint in the general case of star shape constraint provided by the embodiment of the present application;

[0044] Fig. 4(a) is a schematic diagram of one method of complex right-angle building extraction provided by the present application;

[0045] Fig. 4(b) is a schematic diagram of another method of complex right-angle building extraction provided by the present application. DETAILED DESCRIPTION

[0046] The principles and features of the present application are described below in conjunction with the accompanying drawings, which are only used to explain the present application and are not used to limit the scope of the present application.

[0047] Figure 1 Fig. 1 is a schematic diagram of an embodiment of a right-angle building roof interactive extraction method under star shape constraint provided by the present application, as shown in the figure, the interactive extraction method comprises the following steps: Figure 1

[0048] Step 1, estimating the initial position of the building based on a seed line; the seed line is a line input by a user to mark the position of the building.

[0049] Step 2, extracting a line segment on the building image, calculating the main direction of the building according to the direction histogram of the line segment, rotating the building image to the main direction, so that the building becomes a horizontal and vertical state for easy processing.

[0050] Step 3, dividing the building image into a series of rectangular blocks.

[0051] Step 4, integrating star shape constraint on the basis of One Cut algorithm, simultaneously performing extraction and regularization of the rectangular building, and merging the extracted multiple rectangles to obtain the complex rectangular contour of the building.

[0052] The right-angle building roof interactive extraction method under star shape constraint provided by the present application can quickly and effectively extract ordinary rectangular buildings and complex right-angle buildings.

[0053] Embodiment 1

[0054] Embodiment 1 provided by the present application is an embodiment of the right-angle building roof interactive extraction method under star shape constraint provided by the present application, as shown in the figure, which is a specific embodiment of the right-angle building roof interactive extraction method under star shape constraint provided by the present application, combined with Figure 2 and Figure 1 Figure 2 ​​It can be seen that the embodiment extracted by the interactive method comprises:

[0055] Step 1, estimating the initial position of the building based on a seed line; the seed line is a line input by a user to mark the position of the building.

[0056] In a possible embodiment mode, the rules of the input seed line comprise:

[0057] The seed line should be as much as possible in the middle position of the building.

[0058] The direction of the seed line is consistent with the direction of the building.

[0059] The length of the minimum bounding rectangle of the seed line is longer than two-thirds of the length of the building.

[0060] Step 2, extracting line segments on the building image, calculating the main direction of the building according to the direction histogram of the line segments, and rotating the building image to the main direction.

[0061] In a possible embodiment mode, the calculation formula of the value of the line segment direction histogram in step 2 is:

[0062]

[0063] Wherein, θ ∈ [0, 180) represents the direction of the line segment and the horizontal axis of the histogram; n θ is the intensity at the direction value θ of the histogram direction; l i ∈ L represents a line segment; represents the set of all line segments with the direction θ in L; D represents the normalized distance map of the seed line, and D(x, y) represents the distance value at the point (x, y); represents the midpoint coordinates of the line segment l i ; Len(l i ) represents the length of the line segment l i .

[0064] In the specific implementation, the EDLines algorithm can be used to extract the line segments L = (l1, l2,..., l N ) on the image.

[0065] In a possible embodiment mode, the formula for calculating the main direction of the building is:

[0066]

[0067] After the main direction is calculated, the building image is rotated to the main direction by using the bilinear interpolation method.

[0068] Step 3, dividing the building image into a series of rectangular blocks.

[0069] Considering the regularity of the building, this invention over-divides it into a series of small rectangular blocks along the main direction of the building. Specifically, in one possible embodiment, step 3 includes:

[0070] Step 301: Use line segments with the same direction as the main direction as the reference dividing lines to perform preliminary segmentation of the building image.

[0071] After obtaining the main direction of the building in step 2, the line segments in the line segment set L that are in the same direction as the main direction are retained, that is, the line segments in the horizontal and vertical directions are retained, and these line segments are used as the reference dividing lines to realize the initial division of the image.

[0072] Step 302: When the distance between any two adjacent baseline dividing lines exceeds a set distance, find a new dividing line between the two baseline dividing lines. The new dividing line satisfies the condition that the average pixel grayscale value difference in the buffer areas on both sides is the largest. Use each new dividing line to divide the building image.

[0073] In practice, let S represent the expected size of the divided small rectangles, specifically requiring S∈[5,9]. Then the distance between the parallel dividing lines should be greater than S / 2, which can be set to S / 2.

[0074] This invention defines a significance coefficient to describe the significance of a dividing line. The dividing line with the highest significance is taken as a newly added dividing line. Specifically, in step 302, the significance of the dividing line is used to represent the difference in the average grayscale value of pixels in the buffer areas on both sides of the dividing line. When the direction of the new dividing line is vertical, its significance calculation formula is:

[0075]

[0076] Where x is the x-coordinate of the dividing line, x∈[0,W); D(x,y) is the distance value at coordinate (x,y); W*H is the image size; L y This represents the set of pixels with a y-coordinate within the buffer to the left of the dividing line. L represents y Average grayscale value of medium pixels; R y This represents the set of pixels with a y-coordinate within the buffer to the right of the dividing line. R represents y Average grayscale value of mid-pixel.

[0077] In practice, the process begins by finding two adjacent baseline segmentation lines with a distance greater than S / 2. The segmentation line with the highest saliency is then identified and used as a new baseline segmentation line for image segmentation. This process is repeated until no adjacent segmentation lines are found with a distance greater than S / 2. These segmentation lines are then used to over-segment the image.

[0078] Step 4: On the basis of One Cut algorithm, the star shape constraint is integrated, and the extraction and regularization of rectangular buildings are simultaneously carried out, and the extracted multiple rectangles are combined to obtain the complex rectangular contour of the building.

[0079] In a possible embodiment, the energy function to be optimized of the One Cut algorithm integrated with the star shape constraint in step 4 is as follows:

[0080]

[0081] Wherein, S represents a foreground target, represents a background; θ S and are histograms in S and respectively, that is, foreground and background models; represents an L1 measure of the overlap degree of the foreground and background models; p and q represent any two rectangular blocks; N represents a set of four-neighbor adjacent pixel pairs; S p ∈{0,1} represents a label of the rectangular block p; V pq (S p ,S q ) is a boundary penalty term; Shape pq (S p ,S q ) is a newly added star shape constraint term; β, λ, γ represent the relative weights of the terms.

[0082] The interactive right-angle building extraction algorithm provided in the application is based on One Cut, and simultaneously adds a star shape constraint, and uses a model overlap degree penalty factor, so that the building extraction and contour regularization are carried out in the same framework. By introducing a virtual node, the global optimal solution of the formula can be efficiently obtained by using GraphCut.

[0083] In a possible embodiment, the implementation method of the star shape constraint term is as follows: taking the outer package rectangle of a seed line as the center, the building image is divided into different blocks, and the direction of the cut line in GraphCut is limited in different blocks, so as to realize the star shape constraint.

[0084] In a possible embodiment, the calculation formula of the star penalty term is as follows:

[0085]

[0086] Wherein, (x p ,x p ) and (x q ,y q ) represent the center point coordinates of the rectangular block p and the rectangular block q respectively.

[0087] As Figure 3(a) and3(b) The outermost rectangular frame in the figure represents the real building contour, and Fig. 3(a) is a general case in practical application; Fig. 3(b) is an iterative update of the shape constraint, and the dashed polygon represents a possible result after one max-flow / min-cut, and the rectangular frame is the black polygon maximum inscribed rectangle.

[0088] In a possible implementation mode, the process of extraction in step 4 further includes:

[0089] Step 401, assuming that the user interaction is just in the general case shown in Fig. 3(a), an optimal result under the current shape constraint is obtained as an intermediate result according to the parameters of each term of the energy function.

[0090] Step 402, as shown in the dashed polygon in Fig. 3(b). Then the maximum inscribed rectangle of the intermediate result is calculated, as shown by the dashed rectangle in Fig. 3(b). Then the maximum inscribed rectangle is taken as the center of the new shape constraint, the region is updated, and the shape constraint parameters are recalculated.

[0091] Step 403, re-execute step 401 under the new shape constraint to obtain a new extraction result, and iterate until the algorithm converges to complete the building contour extraction and regularization.

[0092] In a possible implementation mode, Figs. 4(a) and 4(b) are schematic diagrams of one method and another method for complex right-angled building extraction provided by the present application, respectively, one interaction and multiple interactions are performed, as shown in Fig. 4(a), the user marks multiple rectangles with a seed line, and then uses the Douglas-Peucker algorithm to divide the seed line into multiple independent curves, each curve is applied to the rectangular house extraction algorithm, and the multiple rectangles are combined after extraction to obtain the contour of the complex rectangle. The second method is shown in Fig. 4(b), multiple interactions, and one rectangle is extracted each time.

[0093] The embodiment of the present application provides an interactive extraction method for right-angled building roof under star-shaped shape constraint, proposes a model-driven interactive extraction method for right-angled building, quickly and effectively extracts ordinary rectangular buildings and complex right-angled buildings, iteratively extracts the optimal rectangular contour through user interaction input, can make the user obtain a relatively reliable right-angled building roof contour through a small amount of simple interactive operation, greatly reduces the time required for manual drawing of buildings, and improves the building acquisition efficiency.

[0094] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0095] Those skilled in the art will appreciate that embodiments of the present application can be devised for a variety of applications. It is intended that the present application be limited only by the scope of the appended claims, and it is intended that various modifications and alterations made by those skilled in the art be considered as within the scope of the present application. The embodiments of the present application will be described with reference to the attached drawings, wherein:

[0096] The present application is described in reference to the drawings using a flowchart and / or a block diagram of the method, apparatus (system) and computer program product according to embodiments of the application. It will be understood that each block of the flowchart and / or block diagram, and combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 means for performing each of the functions specified in the flowchart and / or block diagram block or blocks.

[0097] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 means for performing each of the functions specified in the flowchart and / or block diagram block or blocks.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 means for performing each of the functions specified in the flowchart and / or block diagram block or blocks.

[0099] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments by those skilled in the art once they learn of the basic inventive concepts. Therefore, the appended claims are intended to cover all such modifications and alterations as fall within the true spirit and scope of the application.

[0100] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. An interactive method for extracting the roof of a right-angled building under star-shaped constraints, characterized in that, The interactive extraction method includes: Step 1: Estimate the initial position of the building based on seed lines; the seed lines are lines input by the user to mark the location of the building. Step 2: Extract line segments from the building image, calculate the main direction of the building based on the direction histogram of the line segments, and rotate the building image to the main direction; Step 3: Divide the building image into a series of rectangular blocks; Step 4: Based on the One Cut algorithm, integrate star-shaped constraints, simultaneously extract and regularize rectangular buildings, and merge the extracted rectangles to obtain the complex rectangular outline of the building. Step 3 includes: Step 301: Use the line segments with the same direction as the main direction as the reference dividing lines to perform preliminary segmentation of the building image; Step 302: When the distance between any two adjacent reference dividing lines exceeds a set distance, a new dividing line is found between the two reference dividing lines. The new dividing line satisfies the maximum difference in the average pixel grayscale value within the buffer areas on both sides. The building image is then segmented using each of the new dividing lines. In step 302, the saliency of the dividing line is used to represent the difference in the average grayscale value of pixels within the buffer areas on both sides of the dividing line. When the direction of the new dividing line is vertical, its saliency calculation formula is as follows: in, The x-coordinate of the dividing line is... ; coordinates Distance value at; Image size; The vertical coordinate of the buffer zone to the left of the dividing line is... The set of pixels, express Average grayscale value of mid-pixels; The vertical coordinate of the buffer zone to the right of the dividing line is... The set of pixels, express Average grayscale value of mid-pixel.

2. The interactive extraction method according to claim 1, characterized in that, The rules for the seed line input in step 1 include: The seed line should be positioned as close to the center of the building as possible; The direction of the seed line is consistent with the direction of the building; The length of the minimum outer rectangle of the seed line is longer than two-thirds of the building length.

3. The interactive extraction method according to claim 1, characterized in that, The formula for calculating the line segment direction histogram value in step 2 is as follows: ； in, Indicates the direction of the line segment and the horizontal axis of the histogram; The histogram direction value is Strength at the location; Represents a line segment; express China is The set of all line segments; Represents the seed line normalized distance plot. Then it represents a point Distance value at; Represents line segment The coordinates of the midpoint; Represents line segment The length.

4. The interactive extraction method according to claim 1, characterized in that, The formula for calculating the main direction of the building in step 2 is: ； After calculating the main direction, the building image is rotated to the main direction using bilinear interpolation.

5. The interactive extraction method according to claim 1, characterized in that, The energy function to be optimized in the One Cut algorithm with integrated star-shaped constraints in step 4 is: ； in, Indicates future goals, Represents background; and They are and Histogram within; Indicates the overlap between the foreground and background models. measure; and Represents any two rectangular blocks; Represents the set of four neighboring pixel pairs; Represents a rectangular block The mark; For boundary penalty terms; For star-shaped constraints; , , This indicates the relative weight of each item.

6. The interactive extraction method according to claim 5, characterized in that, The method for implementing the star-shaped constraint is as follows: taking the outer rectangle of the seed line as the center, the building image is divided into different blocks, and the direction of the cut line in GraphCut is restricted in different blocks.

7. The interactive extraction method according to claim 5, characterized in that, The extraction process in step 4 also includes: Step 401: Based on the parameters of the energy function, obtain an optimal result as an intermediate result under the current shape constraint; Step 402: Calculate the maximum inscribed rectangle of the intermediate result, use the maximum inscribed rectangle as the center of the new shape constraint, update the region division, and recalculate the shape constraint parameters; Step 403: Under the new shape constraints, step 401 is executed again to obtain new extraction results until the algorithm converges, thereby completing the building outline extraction and regularization.

Images