A superpixel segmentation method based on Voronoi partitioning

Abstract

The present invention designs a superpixel segmentation method based on Voronoi partitioning, which belongs to the field of superpixel segmentation with spectral consistency constraints. The present invention regards a sub-region of the Voronoi partitioning as a superpixel, so that the local edge of the superpixel located at the boundary of the object coincides with the boundary of the target, and the generated superpixel shape is relatively regular. Since the position of the seed point is the only factor that determines the Voronoi partitioning, the superpixel segmentation problem can be converted into a seed point determination problem, that is, by continuously moving the seed point position, each sub-region of the Voronoi partitioning meets the superpixel requirements. Compared with the existing superpixel segmentation method, the present invention uses superpixels with better boundary fitting and relatively regular shape, and provides a practical technical solution for superpixel segmentation.

Description

Technical Field

[0001] The present invention belongs to the field of spectral consistency constrained superpixel segmentation, and in particular relates to a superpixel segmentation method based on Voronoi partitioning. Background Art

[0002] With the advancement of image acquisition technology, high-resolution images have become widely used in various fields. High-resolution images allow us to observe a wealth of information that cannot be seen with medium- and low-resolution images. In high-resolution images, each pixel corresponds to a very small spatial unit, often a small portion of a real-world object. However, higher spatial resolution comes with a significant increase in data volume and the emphasis on spatial information. This results in pixel-based image processing being time-consuming, inefficient, and prone to "salt and pepper" noise. Image processing using superpixels as the basic processing unit can avoid these problems.

[0003] In 2003, Ren et al. first proposed the concept of superpixels. Superpixels are composed of many adjacent pixels with similar characteristics such as color, texture, and brightness. They are atomic regions with certain visual perceptual significance and can serve as the basic unit of image processing. Image processing methods based on superpixels not only effectively overcome the effects of noise such as "salt and pepper," but also, because each superpixel contains multiple pixels, can significantly reduce the amount of data, thereby reducing image processing time. Generally, a good superpixel segmentation algorithm should possess the following two properties: first, the superpixel boundaries should coincide with the boundaries of the ground object as closely as possible; second, the generated superpixels should be relatively uniform in scale and as regular in shape as possible. These favorable properties enable the convenient and effective application of superpixels in many image processing tasks, such as image classification, target tracking, image parsing, stereo vision matching, and image segmentation.

[0004] Many superpixel segmentation algorithms have been proposed. For example, the Entropy Rate Superpixel (ERS) algorithm proposed by Liu et al. generates superpixels by using the entropy rate and balance term of a random walk in the image domain. While the resulting superpixels are uniform in scale, their shapes are irregular. Such irregular superpixels make it difficult to define spatial relationships. Levinshetein et al. proposed the TurboPixel (TP) segmentation algorithm. This algorithm generates regular superpixels by evolving a geometric flow from seeds uniformly placed in the image domain. This algorithm generates significantly more regular superpixels than the ERS algorithm. These relatively regular superpixels can be used to reconstruct the superpixel spatial grid by introducing virtual nodes. However, the superpixels generated by this algorithm have poor fit to the target boundary. Hu et al. proposed the Spatial-Constrained Watershed (SCoW) algorithm. This algorithm builds on the watershed algorithm by introducing spatial constraints and uniform labeling to obtain uniformly scaled and relatively regular superpixels. This algorithm achieves better boundary fit than the TP algorithm, but due to the spatial constraints, the boundary fit for some details needs improvement.

[0005] In summary, the existing superpixel segmentation algorithms have poor boundary fitting when generating regular superpixels, and it is difficult to strike a balance between boundary fitting and superpixel shape regularity. Summary of the Invention

[0006] In response to the shortcomings of the prior art, the present invention provides a superpixel segmentation method based on Voronoi partitioning. The subregions of the Voronoi partitioning are considered as superpixels, and the local edges of the superpixels located at the boundary of the feature are made to coincide with the boundary of the target, resulting in a relatively regular superpixel shape. Since the position of the seed point is the only factor that determines the Voronoi partitioning, the superpixel segmentation problem can be transformed into a seed point determination problem. That is, by continuously moving the seed point position, each subregion of the Voronoi partitioning is made to meet the superpixel requirements.

[0007] A superpixel segmentation method based on Voronoi partitioning specifically includes the following steps:

[0008] Step 1: Obtain a high-resolution image defined in the Lab color space and express it; the resolution image is expressed as z = {z i (x i ,y i ),i=1,...,N}, where i is the pixel index, N is the total number of pixels, (x i ,y i(x, y) ∈ D is the grid coordinates of pixel i, D is the image domain, and D is expressed as the area covered by image z, or the set of all grid coordinates of image z, that is, D = {(x i , y i ), i = 1,..., N}, z i = (z ib , b = 1, 2, 3) T is the spectral measurement vector of pixel i, b is the band index, b ∈ {L, a, b}, and T is the transpose operator;

[0009] Step 2: Obtain the initial seed point set and get the initial superpixel segmentation; divide the image domain D into M squares of size S × S, that is, the initial superpixels. M is the number of superpixels and also the number of seed points, where S = (N / M) 1 / 2 , select the geometric center of the square as the initial superpixel seed point, and obtain the initial seed point set V (0) = {(u j (0) , v j (0) ), j = 1,..., M}, where j is the seed point index, (u j , v j ) ∈ D is the coordinate of seed point j. The Voronoi partition with V as the seed point set decomposes the image domain D into M superpixels, denoted as D = {D j , j = 1,..., M};

[0010] Step 3: Calculate the variance of each superpixel where J is the iteration index; set the superpixel seed point set to be updated and the superpixel seed point set not to be updated in the J - th iteration;

[0011]

[0012] where, z bl represents the value of the l - th pixel in the b - th channel in superpixel D j , represents the mean value of all pixels in the b - th channel in superpixel D j , k is the total number of pixels in superpixel D j ; if then the position of the superpixel seed point needs to be updated, otherwise the seed point is retained. T1 is the set threshold, min(δ) < T1 < max(δ), min(δ) is the minimum value of the initial superpixel variance, and max(δ) is the maximum value of the initial superpixel variance;

[0013] Let the superpixel seed point set to be updated in the J - th iteration be V (J) unq={V (J) t ,t=1,...,T (J)}, t is the superpixel seed point index that needs to be updated in the Jth iteration, T is the number of seed points that need to be updated; the superpixel seed point set that does not need to be updated is V (J) con ={V (J) t' ,t'=1,…,MT (J)}, t' is the index of the superpixel seed point that does not need to be updated in the Jth iteration;

[0014] Step 4: Update the seed point set V (J) unq According to the seed point update rule, V (J+1) unq , no need to update the superpixel seed point set V (J) con The seed point position remains unchanged to obtain V (J+1) con Go directly to the next iteration, that is, V (J+1) =V (J+1) unq +V (J +1) con ; Among them, the seed point update rules are as follows:

[0015] (1) Record any seed point V that needs to be updated j The neighborhood superpixel set is V j' ={V j' ,j'∈N j},N j Superpixel D j Neighborhood superpixel index of ;

[0016] (2) Calculate the seed point V j The color distance d from its neighboring superpixel seed point is d (Vj,Vj') ,j'∈N j}Find the j V with the largest color distance j' ; Color distance is defined as the Euclidean distance between two color points in the color space, that is:

[0017]

[0018] Among them, z ib and z i'b represent the spectral intensity values of the b-th band of pixels i and i' respectively;

[0019] (3) Record V j The coordinates of (u j ,v j), V j' The coordinates of (u j' ,v j' ), and obtain line segment V according to Bresenham's straight line algorithm j V j' Passing through the pixel coordinate point set P;

[0020] (4) Calculate the color distance between any two adjacent pixels in the point set P, and record the two pixels with the largest color distance as P. * and P *+1 , move the seed point V j Get the updated seed point make On the straight line V j V j' Upper and line segments The perpendicular bisector of * or P *+1 ;

[0021] But there may be a line segment V j V j' The perpendicular bisector of * or P *+1 In this case, the seed point V j It is necessary to update its position according to the neighboring superpixel seed point with the second largest color distance, and execute steps (2)-(3) above; if the above situation still occurs, continue to execute steps (2)-(3) above; in any iteration process, V unq This operation is performed on all seed points in ;

[0022] When there is a line segment V j V j' The perpendicular bisector of * or P *+1 In this case, the seed point V j It is necessary to update its position according to the neighboring superpixel seed point with the second largest color distance from it and re-execute steps (3)-(4); if the above situation still occurs, continue to execute steps (2)-(3) in sequence until all neighboring superpixels have performed this operation and then stop;

[0023] Step 5: By V (J+1) Get the J-th iteration of the image domain division D (J+1) ; Area D j By (u j ,v j ) is the sub-region constructed as the center, namely D j Any point (x i ,y i ) distance (u j ,v j) is less than the distance to other seed points, that is:

[0024] D j ={(x i ,y i );d((x i ,y i ),(u j ,v j ))≤d((x i ,y i ),(u j' ,v j' )),(u j ,v j ),(u j' ,v j' )∈V,j≠j' (3)

[0025] Where d is the Euclidean distance between two points in the image domain; a Voronoi partition of the seed point set V on the region D is:

[0026]

[0027] Step 6: Iterate steps 3 to 5 until J reaches the specified number of iterations. The Jth iteration divides the image domain into (J+1) This is the final superpixel segmentation result.

[0028] Beneficial technical effects of the present invention:

[0029] This paper proposes a superpixel segmentation method based on Voronoi partitioning. By moving seed points, the local edges of superpixels located at the target boundary coincide with the target boundary, making the superpixel shape as regular as possible. Compared with existing superpixel segmentation methods, this method achieves better boundary fit and relatively regular superpixel shape, providing a practical technical solution for superpixel segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A flowchart of a superpixel segmentation method based on Voronoi partitioning according to an embodiment of the present invention;

[0031] Figure 2 Experimental data of a superpixel segmentation method based on Voronoi partitioning in an embodiment of the present invention;

[0032] Figure 3 Experimental results of a superpixel segmentation method based on Voronoi partitioning in an embodiment of the present invention;

[0033] Figure 4 Quantitative evaluation of experimental results of superpixel segmentation method based on Voronoi partitioning in embodiments of the present invention. DETAILED DESCRIPTION

[0034] The specific embodiments of the present invention are described in further detail below with reference to the accompanying drawings and examples;

[0035] The present invention proposes a new superpixel segmentation algorithm, which uses Voronoi partitioning to divide the image into several sub-regions. By moving the seed point, the local edge of the sub-region located at the boundary of the ground object coincides with the target boundary, and the shape of the sub-region is as regular as possible to meet the superpixel requirements. The sub-region is regarded as a superpixel to complete the superpixel segmentation of the image.

[0036] A superpixel segmentation method based on Voronoi partitioning, such as Figure 1 The following steps are shown:

[0037] Step 1: Get a high-resolution image Figure 2 And express it. The high-resolution image is expressed as z={z i (x i ,y i ),i=1,...,N}, where i is the pixel index, N is the total number of pixels, (x i ,y i )∈D is the grid coordinate of pixel i, D is the image domain (in the absence of confusion, D is expressed as the area covered by image z, or the set of all grid coordinates of image z, that is, D={(x i ,y i ),i=1,...,N}),z i =(z ib ,b=1,2,3) T is the spectral measurement vector of pixel i, b is the band index (this paper takes the high-resolution image defined in the Lab color space as the research object, and for this image, b∈{L,a,b}) and T is the transpose operator.

[0038] Step 2: Get the initial seed point set and get the initial superpixel segmentation. Divide the image domain D into M squares (initial superpixels) of size S×S, where M is the number of superpixels and also the number of seed points, where S = (N / M) 1 / 2 , select the geometric center of the square as the initial superpixel seed point, and get the initial seed point set V (0) ={(u j (0) ,v j (0) ),j=1,...,M}, where j is the seed point index, (u j ,v j) ∈ D is the coordinate of the seed point j. The Voronoi partition with V as the seed point set decomposes the image domain D into M superpixels, denoted as D = {D j , j = 1, ..., M};

[0039] Step 3: Obtain the variance of each superpixel according to formula (1) J is the iteration index;

[0040]

[0041] where, z bl represents the value of the l-th pixel in the b-th channel in the superpixel D j , represents the mean value of all pixels in the b-th channel included in the superpixel D j , k is the total number of pixels included in the superpixel D j . If δ (J) j > T1, then the seed point position of the superpixel D (J) j needs to be updated, otherwise the seed point is retained. T1 is the set threshold, min(δ) < T1 < max(δ), min(δ) is the minimum initial superpixel variance value, and max(δ) is the maximum initial superpixel variance value. Let the superpixel seed point set to be updated in the J-th iteration be V (J) unq = {V (J) t , t = 1, ..., T (J)}, t is the index of the superpixel seed point to be updated in the J-th iteration, and T is the number of seed points to be updated. The superpixel seed point set that does not need to be updated is V (J) con = {V (J) t' , t' = 1, …, M - T (J)}, t' is the index of the superpixel seed point that does not need to be updated in the J-th iteration.

[0042] Step 4: The seed point set V (J) unq to be updated. Obtain V (J+1) unq according to the seed point update rule, and keep the seed point position in the superpixel seed point set V (J) con unchanged to obtain V (J+1) con . Directly enter the next iteration, that is, V (J+1) = V (J+1) unq + V (J +1) con; Among them, the seed point update rules are as follows:

[0043] (1) Record any seed point V that needs to be updated j The neighborhood superpixel set is V j' ={V j' ,j'∈N j},N j Superpixel D j Neighborhood superpixel index of ;

[0044] (2) Calculate the seed point V j The color distance d from its neighboring superpixel seed point is d (Vj ,V j' ),j'∈N j}Find the j V with the largest color distance j' ; Color distance is defined as the Euclidean distance between two color points in the color space, that is:

[0045]

[0046] Among them, z ib and z i'b represent the spectral intensity values of the b-th band of pixels i and i' respectively;

[0047] (3) Record V j The coordinates of (u j ,v j ), V j' The coordinates of (u j' ,v j' ), and obtain line segment V according to Bresenham's straight line algorithm j V j' Through the pixel coordinate point set P; Bresenham line algorithm is shown in Table 1

[0048] Table 1 Bresenham line algorithm

[0049]

[0050]

[0051] (4) Calculate the color distance between any two adjacent pixels in the point set P, and record the two pixels with the largest color distance as P. * and P *+1 , move the seed point V j Get the updated seed point make On the straight line V j V j' Upper and line segments The perpendicular bisector of* or P *+1 ;

[0052] But there may be a line segment V j V j' The perpendicular bisector of * or P *+1 In this case, the seed point V j It is necessary to update its position according to the neighboring superpixel seed point with the second largest color distance, and execute steps (2)-(3) above; if the above situation still occurs, continue to execute steps (2)-(3) above; in any iteration process, V unq This operation is performed on all seed points in ;

[0053] When there is a line segment V j V j' The perpendicular bisector of * or P *+1 In this case, the seed point V j It is necessary to update its position according to the neighboring superpixel seed point with the second largest color distance from it and re-execute steps (3)-(4); if the above situation still occurs, continue to execute steps (2)-(3) in sequence until all neighboring superpixels have performed this operation and then stop;

[0054] Step 5: According to formula (3)-(4), V (J+1) Get the J-th iteration of the image domain division D (J+1) . Area D j By (u j ,v j ) is the sub-region constructed as the center, that is, D j Any point (x i ,y i ) distance (u j ,v j ) is less than the distance to other seed points, that is:

[0055] D j ={(x i ,y i );d((x i ,y i ),(u j ,v j ))≤d((x i ,y i ),(u j' ,v j' )),(u j ,v j ),(u j' ,v j' )∈V,j≠j' (3)

[0056] Where d is the Euclidean distance between two points in the image domain; the Voronoi partition of the seed point set V on the region D is:

[0057]

[0058] Step 6: Iterate steps 3 to 5 until J reaches the specified number of iterations. The Jth iteration divides the image domain into (J+1) This is the final superpixel division result. The experimental results are as follows Figure 3 shown.

[0059] To verify the authenticity of this experiment, under the same experimental conditions, the eddy current super pixel (TurboPixel, TP) segmentation algorithm, the spatial-constrained watershed (Spatial-Constrained Watershed, SCoW) algorithm and the super pixel segmentation based on Voronoi partitioning (Superpixel Segmentation Based on Voronoi Partitioning, SSBOV) algorithm were used to perform the segmentation. Figure 2 Superpixel segmentation is performed, and evaluation indicators such as boundary recall rate, under-segmentation error, achievable segmentation accuracy and compactness measurement are used to quantitatively evaluate the superpixel segmentation results generated by the three algorithms. The evaluation results are as follows Figure 4 Experimental results show that the superpixels generated by the proposed algorithm have improved boundary recall, under-segmentation error, achievable segmentation accuracy, and compactness compared to those generated by the TP algorithm and the SCoW algorithm. They can better fit the boundaries of the target objects and have good shape regularity.

Claims

1. A superpixel segmentation method based on Voronoi partitioning, characterized in that: The specific steps include: Step 1: Obtain a high-resolution image defined in the Lab color space and express it; the resolution image is expressed as z = {z i (x i ,y i ),i=1,...,N}, where i is the pixel index, N is the total number of pixels, (x i ,y i )∈D is the grid coordinate of pixel i, D is the image domain, D is expressed as the area covered by image z, or the set of all grid coordinates of image z, that is, D={(x i ,y i ),i=1,...,N}),z i =(z ib ,b=1,2,3) T is the spectral measurement vector of pixel i, b is the band index, b∈{L,a,b}, T is the transpose operator; Step 2: Obtain the initial seed point set and obtain the initial superpixel segmentation; divide the image domain D into M squares of size S×S, namely the initial superpixels, where M is the number of superpixels and also the number of seed points, where S = (N / M) 1 / 2 , select the geometric center of the square as the initial superpixel seed point, and get the initial seed point set V (0) ={(u j (0) ,v j (0) ),j=1,...,M}, where j is the seed point index, (u j ,v j )∈D is the coordinate of seed point j, and the Voronoi partition with V as the seed point set decomposes the image domain D into M superpixels, denoted as D={D j ,j=1,...,M}; Step 3: Calculate the variance δ of each superpixel (J) j , where J is the iteration index; set the superpixel seed point set that needs to be updated and the superpixel seed point set that does not need to be updated in the J-th iteration; The step 3 is specifically as follows: where, zbl represents the value of the l-th pixel in the b-th channel of the superpixel D j and, represents the mean value of all pixels included in the superpixel D j in the b-th channel, k is the total number of pixels included in the superpixel D j ; if δ (J) j > T1, then the position of the seed point of the superpixel D (J) j needs to be updated, otherwise the seed point is retained, T1 is a set threshold, min(δ) < T1 < max(δ), min(δ) is the minimum initial superpixel variance value, and max(δ) is the maximum initial superpixel variance value; Assume that the superpixel seed point set to be updated in the Jth iteration is V (J) unq ={V (J) t ,t=1,...,T (J) }, t is the index of superpixel seed points that need to be updated in the Jth iteration, T is the number of seed points that need to be updated; the set of superpixel seed points that do not need to be updated is V (J) con ={V (J) t' ,t'=1,…,MT (J) }, t' is the superpixel seed point index that does not need to be updated in the Jth iteration. Step 4: Update the seed point set V (J) unq According to the seed point update rule, V (J+1) unq , no need to update the superpixel seed point set V (J) con The seed point position remains unchanged to obtain V (J+1) con Go directly to the next iteration, that is, V (J+1) =V (J+1) unq +V (J+1) con ; Step 5: By V (J+1) Get the J-th iteration of the image domain division D (J+1) ; Area D j By (u j ,v j ) is the sub-region constructed as the center, that is, D j Any point (x i ,y i ) distance (u j ,v j ) is less than the distance to other seed points; Step 6: Iterate steps 3 to 5 until J reaches the specified number of iterations. The Jth iteration divides the image domain into (J+1) This is the final superpixel segmentation result.

2. The superpixel segmentation method based on Voronoi partitioning according to claim 1, characterized in that: The seed point update rules in step 4 are as follows: (1) Record any seed point V that needs to be updated j The neighborhood superpixel set is V j' ={V j' ,j'∈N j },N j Superpixel D j Neighborhood superpixel index of ; (2) Calculate the seed point V j The color distance d from its neighboring superpixel seed point is d (Vj ,V j' ),j'∈N j }Find the j V with the largest color distance j' ; Color distance is defined as the Euclidean distance between two color points in the color space, that is: Among them, z ib and z i'b represent the spectral intensity values of the b-th band of pixels i and i' respectively; (3) Record V j The coordinates of (u j ,v j ), V j' The coordinates of (u j' ,v j' ), and obtain line segment V according to Bresenham's straight line algorithm j V j' Passing through the pixel coordinate point set P; (4) Calculate the color distance between any two adjacent pixels in the point set P, and record the two pixels with the largest color distance as P. * and P *+1 , move the seed point V j Get the updated seed point V * j , so that V * j On the straight line V j V j' On and line segment V * j V j' The perpendicular bisector of * or P *+1 ; But there may be a line segment V j V j' The perpendicular bisector of * or P *+1 In this case, the seed point V j It is necessary to update its position according to the neighboring superpixel seed point with the second largest color distance, and execute steps (2)-(3) above; if the above situation still occurs, continue to execute steps (2)-(3) above; in any iteration process, V unq This operation is performed on all seed points in ; When there is a line segment V j V j' The perpendicular bisector of * or P *+1 In this case, the seed point V j It is necessary to update its position according to the neighboring superpixel seed point with the second largest color distance from it and re-execute steps (3)-(4); if the above situation still occurs, continue to execute steps (2)-(3) in sequence until all neighboring superpixels have performed this operation and then stop.

3. The superpixel segmentation method based on Voronoi partitioning according to claim 1, characterized in that: Step 5 is as follows: D j ={(x i ,y i );d((x i ,y i ),(u j ,v j ))≤d((x i ,y i ),(u j' ,v j' )),(u j ,v j ),(u j' ,v j' )∈V,j≠j'(3) Where d is the Euclidean distance between two points in the image domain; a Voronoi partition of the seed point set V on the region D is:

Images