A sky region segmentation method based on bidirectional constraint

Abstract

The application discloses a sky region segmentation method based on bidirectional constraint, and belongs to the technical field of image processing. First, the method carries out sky region edge extraction on the input image based on an edge detection operator, and carries out filtering optimization on the processed image. Then, based on the features of the sky region edge, the position of the first batch of marked seeds is found, and the filling direction is judged, and then the found seeds are subjected to horizontal constraint and vertical constraint. Next, based on the marked seed position screened by the bidirectional constraint, the scanning and filling are carried out in sections. Finally, the segmented sky region is output through linear and operation between images. In the method, the first batch of marked seed information is screened and supplemented by adopting the bidirectional constraint, the segmentation accuracy of the sky region and the non-sky region is improved, and the method can be effectively applied to scene images with fuzzy boundaries. Meanwhile, the segmentation processing time is greatly improved by adopting the sectional scanning and filling.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a sky region segmentation method based on bidirectional constraint. BACKGROUND

[0002] In the field of intelligent monitoring, intelligent transportation and unmanned aerial vehicle, the camera as the main sensor is responsible for collecting the environment information, which is generally in the form of image input to the control system. The image structure collected by the camera can be divided into the region of interest where the target object is located and the non-region of interest including the sky region. Generally, in order to focus on the region of interest, the method of segmenting the sky region in the whole image is adopted.

[0003] However, in the prior art, when the camera is collecting, due to weather (such as overcast or foggy day) or special angle and other factors, the features between the sky region and the non-sky region (region of interest) of the image to be processed are similar, and after edge extraction, there will be great sparseness or even discontinuity at the boundary, which will greatly affect the accuracy of the sky region segmentation and filling.

[0004] The traditional region segmentation and flood filling method can process the image with continuous boundary line, but when applied to the image with sparse or discontinuous boundary line, the following shortcomings exist:

[0005] (1) Since the sparseness or discontinuity of the boundary line after edge extraction is not processed, the segmentation accuracy is greatly affected;

[0006] (2) In the filling process, the traversal of pixel points is adopted, which causes the filling time to be too long, thereby prolonging the time of the whole segmentation and filling task. SUMMARY

[0007] In view of the defects of the prior art, the present application provides a sky region segmentation method based on bidirectional constraint, which is suitable for more complex scene segmentation tasks and can improve the accuracy and processing efficiency of sky region segmentation.

[0008] To this end, the present application adopts the following technical solutions:

[0009] The present application provides a sky region segmentation method based on bidirectional constraint, comprising:

[0010] Edge extraction of the sky region based on the edge detection operator is performed on the input image I1, and the filtered and optimized image I2 is obtained after the extracted image is filtered and optimized;

[0011] For the binary image I2, based on the connection features between the sky region and the non-sky region, the position of the first batch of marker seed is found, and the filling direction is judged;

[0012] If the filling direction is vertical, the first batch of seed points are vertically constrained to obtain the seed position information after vertical constraint, and the seed points after vertical constraint are horizontally constrained to obtain the seed position information after bidirectional constraint;

[0013] If the filling direction is horizontal, the first batch of seed points are horizontally constrained to obtain the seed position information after horizontal constraint, and the seed points after horizontal constraint are vertically constrained to obtain the seed position information after bidirectional constraint;

[0014] The bidirectionally constrained seed position is taken as an end point to segmentally scan and fill the image to obtain the filled binary image I3.

[0015] The linear and operation between the input image I1 and the filled binary image I3 is performed to output the image I4 of only the sky region.

[0016] Further, the edge detection operator is a Laplace operator.

[0017] Further, the filtering optimization on the extracted image comprises:

[0018] The median filtering is adopted to filter and optimize the extracted image.

[0019] Further, the position of the first batch of seed points is found, comprising:

[0020] It is assumed that the position of the seed to be found is [X, Y], and the pixel values inside and outside the edge region in the binary image I2 are opposite, so the position of the first batch of seed points is found.

[0021] Further, the filling direction is determined, comprising: if the area ratio of the sky region in the whole image is small, or the sky region is not connected, horizontal filling is selected; otherwise, the area ratio of the sky region is large and the sky region is a connected region, and vertical filling is selected.

[0022] Further, the horizontal constraint on the seed points comprises: setting a horizontal constraint threshold value as m, assuming that the coordinate of the current traversed seed point is [x1, y1], and the next seed point is [x2, y2], if the absolute value of y1-y2 is less than m, the coordinate of the seed point [x2, y2] is reserved, and the next seed point is continuously traversed; if the absolute value of y1-y2 is greater than m or the value of y2-y1 is greater than m, the coordinate of the seed point [x2, y2] is replaced by [x2, y1+m] and [x2, y1-m] respectively, and the next seed point is continuously traversed;

[0023] The longitudinal constraint of the marking point seed comprises: setting a longitudinal constraint threshold value as n, assuming that a current traversed marking point seed coordinate is [x1, y1], a next marking point seed is [x2, y2], if an absolute value of x1-x2 is less than n, a position coordinate of the [x2, y2] marking point seed is reserved, and the next marking point seed is continuously traversed; if the value of x1-x2 is greater than n or the value of x2-x1 is greater than n, the position coordinate of the [x2, y2] marking point seed is replaced by [x2+n, y1] and [x2-n, y1] respectively, and the next marking point seed is continuously traversed.

[0024] Further, if the segmentation filling direction is longitudinal filling, the first batch of seeds is processed by traversing column by column in the bidirectional constraint; on the basis of the longitudinal constraint, the transverse constraint threshold value is 1.

[0025] Further, if the segmentation filling direction is transverse filling, the first batch of seeds is processed by traversing row by row in the bidirectional constraint; on the basis of the transverse constraint, the longitudinal constraint threshold value is 1.

[0026] Advantages and positive effects of the present application:

[0027] The sky region segmentation method based on bidirectional constraint provided by the present application firstly extracts the sky region edge of the input image based on an edge detection operator, and optimizes the filtered image, then finds the position of the first batch of marking seeds based on the characteristics of the sky region edge, and judges the filling direction, and then transversely and longitudinally constrains the found seeds, then scans and fills the marking seeds filtered by the bidirectional constraint in sections. Finally, the segmented sky region is output by linear and operation between images. In this method, the first batch of marking seed information is filtered and supplemented by bidirectional constraint, which can handle the sparseness or even discontinuity of the boundary line after edge extraction, thereby improving the segmentation accuracy of the sky region and the non-sky region, and especially being effectively applicable to scene images with blurred boundaries. Meanwhile, the processing time of segmentation is greatly improved by using segmented scanning filling. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0029] Figure 1 The flowchart of the sky region segmentation method based on bidirectional constraint in the embodiments of the present application is shown in the figure.

[0030] Figure 2 The output result graph of a sky region segmentation method based on bidirectional constraint in an embodiment of the present application;

[0031] Figure 3 The comparison graph of the instance segmentation results of the two methods. DETAILED DESCRIPTION

[0032] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative efforts shall fall within the protection scope of the present application.

[0033] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device including a series of steps or units does not necessarily have to include only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product, or device.

[0034] As shown in Figure 1 , an embodiment of the present application provides a sky region segmentation method based on bidirectional constraint, comprising the following steps:

[0035] S1: performing edge extraction of a sky region based on an edge detection operator on an input image I1, and performing filter optimization on the extracted image to obtain a binary image I2;

[0036] The edge detection operator used is a Laplacian operator, and the operator kernel thereof is The method used for the extracted image is 3x3 median filtering, so as to ensure that there are no unnecessary interference points around the edges after edge extraction, and the output result is as shown in Figure 2 .

[0037] S2: for the binary image I2, finding the position of the first batch of marker point seeds based on the connection features between the sky region and the non-sky region, and judging the filling direction;

[0038] In the embodiment, the sky region of the input image I1 accounts for a large area in the whole image. Based on the principle of relatively less processing time, the vertical filling is selected. Since the obtained image in S1 is a binary image, the pixel values inside and outside the edge region are opposite, and the positions of the first batch of marker point seeds can be found as [X, Y] 2047 groups. Figure 2 The output result shows that due to the blurred scene boundary of the original input image I1, the boundary after edge extraction appears discontinuous and faulted, and the positions of the first batch of marker point seeds need to be screened and supplemented.

[0039] S3: The first batch of marker point seeds found are vertically constrained to obtain the seed position information after vertical constraint.

[0040] Due to the discontinuous faulted boundary after edge extraction, the first batch of marker point seeds obtained are relatively rough and even deviate from the original boundary position too far. The positions of the marker point seeds are first vertically constrained. Specifically, the vertical constraint threshold is set to 35. Assuming that the coordinate of the current traversed marker point seed is [x1, y1], the next marker point seed is [x2, y2], if the absolute value of x1-x2 is less than 35, the position coordinate of the marker point seed [x2, y2] is retained, the next marker point seed is continued to be traversed, otherwise, if the value of x1-x2 is greater than 35 or the value of x2-x1 is greater than 35, the position coordinates of the marker point seed [x2, y2] are replaced by [x2+35, y1] and [x2-35, y1] respectively, and the next marker point seed is continued to be traversed.

[0041] S4: The first batch of vertically constrained seeds found are horizontally constrained to obtain the seed position information after bidirectional constraint.

[0042] In this embodiment, the vertical filling is determined as the filling direction of the original image I1, so the next marker point seed is traversed and processed column by column when the first batch of seeds are bidirectionally constrained. Therefore, based on the vertical constraint in the last step, the horizontal constraint threshold is 1 in this step. Assuming that the coordinate of the current traversed marker point seed is [x1, y1], the next marker point seed is [x2, y2], if the value of y2-y1 is equal to 1, the position coordinate of the marker point seed [x2, y2] is retained, the next marker point seed is continued to be traversed, otherwise, if the value of y2-y1 is greater than 1, it means that a marker point needs to be supplemented between the two marker points, a new marker point position [x10, y10] is supplemented, and the value of [x1, y1] is assigned to [x10, y10], then [x10, y10] is compared with the next marker point seed [x2, y2], that is, the next marker point seed is continued to be traversed.

[0043] S5: Taking the seed position obtained by the above bidirectional constraint as an end point 1 and the initial point of the row and column of the whole image as an end point 2, the sky region is scanned and filled in sections to obtain a filled binary image I3.

[0044] The segmentation process is completed in the early stage of the screening of the seeds, and each seed is an end point, so that subsequent filling in sections can be performed. Figure 2 As shown in the filled binary image I3, the sky region is assigned a value of 1.

[0045] The traditional method is to process pixel by pixel, and the segmentation filling is equivalent to calculating and processing a group of pixel points at a time, so that the segmentation filling can improve the segmentation filling time and effectively improve the processing efficiency.

[0046] S6: By performing a linear AND operation on the original input image I1 and the filled binary image I3 obtained above, an image of only the sky region or an image of only the region of interest I4 can be output.

[0047] Specifically, the linear AND operation is I4 = I1 & I3.

[0048] As shown in the filled binary image I3, the sky region is assigned a value of 1. Figure 2

[0049] In order to more clearly compare the effectiveness of the method of the present application, the example segmentation results of the traditional segmentation filling method and the method of the present application are compared for the same image, as shown in Figure 3 The filled image of the traditional method is an output image obtained by processing the boundary image based on unidirectional constraint. It can be seen that the method based on bidirectional constraint can effectively handle the sparseness and fault conditions of the boundary line, thereby improving the segmentation accuracy of the whole image.

[0050] In the above embodiment, by using a bidirectional constraint to screen and supplement the first batch of marked seeds, the segmentation accuracy of the sky region and the non-sky region is improved, and the effect on the scene image with a blurred boundary is good. At the same time, by using the segmented scanning filling, the processing time of segmentation is greatly improved.

[0051] ​In several embodiments provided by the present application, it should be understood that the disclosed technology can be implemented in other manners. For example, the described unit embodiments can be divided into other ways, for example, the division of the units can be a logical function division, and there can be another division manner in actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, or the logical couplings or communication connections between different units, can be implemented by using some interfaces, and the indirect couplings or communication connections between different units can be implemented in electric or other forms.

[0052] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0053] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present separately, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0054] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0055] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A sky region segmentation method based on bidirectional constraints, characterized in that, The method comprises the following steps: Edge extraction of a sky region based on an edge detection operator is performed on an input image I1, and the extracted image is filtered and optimized to obtain a binary image I2; For the binary image I2, the positions of the first batch of marker seeds are found based on the connection features between the sky region and the non-sky region, and the filling direction is determined; If the filling direction is vertical, the first batch of marker seeds are vertically constrained to obtain the position information of the vertically constrained seeds, and the vertically constrained seeds are horizontally constrained to obtain the position information of the bidirectionally constrained seeds; If the filling direction is horizontal, the first batch of marker seeds are horizontally constrained to obtain the position information of the horizontally constrained seeds, and the horizontally constrained seeds are vertically constrained to obtain the position information of the bidirectionally constrained seeds; The bidirectionally constrained seed positions are taken as end points, and the image is scanned and filled in segments to obtain a filled binary image I3; The linear sum operation between the input image I1 and the filled binary image I3 is performed to output an image I4 containing only the sky region.

2. The sky region segmentation method based on bidirectional constraints according to claim 1, wherein, The edge detection operator is a Laplace operator.

3. The sky region segmentation method based on bidirectional constraints according to claim 1, wherein, The filtering and optimization of the extracted image comprises the following steps: The extracted image is filtered and optimized by using a median filter.

4. The method of claim 1, wherein the sky region is segmented based on bi-directional constraints. The positions of the first batch of marker seeds are found by assuming that the position of the seed to be found is [X, Y] and the pixel values inside and outside the edge region of the binary image I2 are opposite. The filling direction is determined by selecting horizontal filling if the sky region accounts for a small proportion of the whole image or the sky region is not connected, otherwise, vertical filling is selected.

5. The method of claim 1, wherein the sky region is segmented based on bi-directional constraints. The horizontal constraint of the marker seeds comprises the following steps: a horizontal constraint threshold m is set, the current marker seed coordinate is assumed to be [x1, y1], the next marker seed is [x2, y2], if the absolute value of y1-y2 is less than m, the position coordinate of the marker seed [x2, y2] is retained, and the next marker seed is continuously traversed; if the absolute value of y1-y2 is greater than m or the value of y2-y1 is greater than m, the position coordinates of the marker seed [x2, y2] are replaced by [x2, y1+m] and [x2, y1-m] respectively, and the next marker seed is continuously traversed; 6. The method of claim 1, wherein the sky region is segmented based on bi-directional constraints. The vertical constraint of the marker seeds comprises the following steps: a vertical constraint threshold n is set, the current marker seed coordinate is assumed to be [x1, y1], the next marker seed is [x2, y2], if the absolute value of x1-x2 is less than n, the position coordinate of the marker seed [x2, y2] is retained, and the next marker seed is continuously traversed; if the value of x1-x2 is greater than n or the value of x2-x1 is greater than n, the position coordinates of the marker seed [x2, y2] are replaced by [x2+n, y1] and [x2-n, y1] respectively, and the next marker seed is continuously traversed. If the segmentation and filling direction is vertical, the first batch of seeds are bidirectionally constrained by column-by-column traversal; on the basis of the vertical constraint, the horizontal constraint threshold is 1.

7. The method of claim 1, wherein the sky region is segmented based on bi-directional constraints. ​ 8. The method of claim 1, wherein the sky region is segmented based on bi-directional constraints. If the segmentation filling direction is horizontal filling, the first batch of seeds is traversed and processed row by row when being bidirectional constrained; on the basis of horizontal constraint, the vertical constraint threshold is 1.

Images