A method for segmenting elliptical objects based on ellipse fitting

Abstract

The invention discloses an ellipse object segmentation method based on ellipse fitting. The method obtains the object of the target area in the image by extracting the maximum connected area of ​​the G component value in the RGB gradient value of the image, and then uses the RGB super green segmentation algorithm to remove the interference of background factors in the picture, and uses the top hat transformation method to remove the non-target area; combines the fast radial symmetry method based on distance transformation and global threshold to extract the center point of the ellipse object; uses the concave point detection and contour estimation method to segment the edge part of the ellipse object; finally uses the least squares based ellipse fitting algorithm to complete the blocked part of the overlapping object; the method ensures the accurate extraction of the edge points of the overlapping targets; avoids the problem of incomplete object edge extraction and unsatisfactory segmentation effect caused by similar RGB color gradient values ​​in the image; and solves the problem that the complete contour of partially overlapping ellipse targets cannot be obtained in a two-dimensional image under the camera perspective.

Description

Technical Field

[0001] The invention relates to the technical field of image segmentation, and in particular to an ellipse object segmentation method based on ellipse fitting. Background Art

[0002] Machine vision is a hot research topic in artificial intelligence. In machine vision research, object segmentation and parameter calculation based on images is a key issue. Object detection and segmentation in images also have many applications in reality, such as medical diagnosis, industrial parts quality inspection, fruit sorting, etc.

[0003] Researchers have conducted a lot of research on object detection and segmentation in images. Common methods for segmenting target objects from background images include edge detection, threshold segmentation, and morphological segmentation. In addition, some researchers use the improved watershed segmentation algorithm to extract surface leaves in images. The specific method is to first use the RGB three-channel gradient entropy to extract the color gradient in the image, and then calculate the matrix variance in the gradient histogram in the local direction. After screening with a certain threshold, the gradient reconstruction is realized. Then, the watershed transform is used according to the image gray value gradient to achieve the segmentation of different objects in the image. In recent years, with the development of deep learning technology, some researchers have proposed using algorithm frameworks such as Mask R-CNN to segment irregular objects. These methods generally collect a large amount of image data of the target object and then manually process it, manually calibrate the target to be segmented in the image, and then put the preprocessed data into the self-built or improved model for training until the model converges, so that the trained model can predict and segment the target object in the picture. For elliptical objects, Liu Xinlei et al. took elliptical plant leaves as an example and proposed using a greedy algorithm to find the overlapping edges of the leaves, and then used a convex hull filling algorithm to obtain the complete edges of the leaves, thereby completing the obscured parts of the lower layer of overlapping elliptical leaves.

[0004] One shortcoming of the existing overlapping object segmentation method is that it is not possible to extract the edges of objects with similar RGB color gradient values ​​in the image, resulting in the object segmentation results not meeting the expected effect; another shortcoming is that it is impossible to complete the missing parts of the occluded objects, resulting in inaccurate subsequent calculation results for the target object. When using deep learning methods to segment overlapping objects, not only will the above two problems occur, but also the need to manually annotate the data set in advance, resulting in excessive labor costs and other problems. When using the convex hull filling algorithm to complete the occluded parts of overlapping objects, considering that the edge points of the overlapping objects in the image will deviate, the algorithm cannot guarantee the correct prediction of the missing parts of such objects. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an elliptical object segmentation method based on ellipse fitting. The method obtains the object of the target area in the image by extracting the maximum connected area of ​​the G component value in the RGB gradient value of the image, and then uses the RGB super green segmentation algorithm to remove the interference of background factors in the picture, and uses the top hat transformation method to remove the non-target area; combines the fast radial symmetry method based on distance transformation and global threshold to extract the center point of the elliptical object; uses the method of concave point detection and contour estimation to segment the edge part of the elliptical object; finally, uses the least squares based ellipse fitting algorithm to fill the blocked part of the overlapping object.

[0006] The technical solution for achieving the purpose of the present invention is:

[0007] A method for detecting elliptical objects based on ellipse fitting comprises the following steps:

[0008] S1. Image acquisition:

[0009] Use image acquisition equipment to collect images from above the target object at an angle perpendicular to the object, then use image processing software to cut out the target area to be segmented, and convert the image format into JPG image format;

[0010] S2. Image preprocessing:

[0011] S2-1, maximum connected area extraction: Since the overlapping objects in the intercepted image are always concentrated in one area of ​​the image, only the maximum value of the G component area is extracted from the RGB composition of the entire image, and the pixel values ​​of the remaining green blocks are changed to 0;

[0012] S2-2, super green segmentation algorithm: Based on the image obtained in step S2-1, suppose there are i pixels in the image, S i is the pixel value of each pixel, R, G, and B are the flux values ​​of red, green, and blue in the image respectively; by adjusting the image saturation under different lighting conditions, the threshold T is determined using the maximum inter-class variance calculation method graythresh in Matlab, according to:

[0013]

[0014] When the grayscale factor of the leaves in the target area of ​​the image is less than or equal to the threshold T, the pixel value of the marked image at the corresponding position is equal to 255, otherwise it is equal to 0, and the non-green part of the image is removed and the image is binarized;

[0015] S2-3, top hat transformation: in the binary image obtained in step S2-2, the extracted target area includes the target object and its attached parts, and the attached parts (non-target areas) are removed by opening operation of the grayscale image g to avoid deviation in subsequent image processing work;

[0016] S3. Extract the center point of the target object:

[0017] The center point of the elliptical object is extracted using the fast radial symmetry method. The fast radial symmetry method is a local feature extraction technology for highlighting the image gradient. It detects points with a specific distance from the edge pixel points of the target area one by one, and selects the center point that meets the requirements. The specific distance is the distance between the center point and the edge pixel points of the target area.

[0018] Given a range distance value i∈[R min ,R max ], for the pixel value of any point in the target area of ​​the image, the positive and negative influence factors of the pixel are determined by the fast radial symmetry method + 、p - , and the directional projection image O i With the amplitude projection image M i : The expression is as follows:

[0019]

[0020] In the region i∈R min ,R max ], the symmetric radial S i Depends on i 、M i and the two-dimensional Gaussian A i :

[0021]

[0022] Where k is the number of different radii on O i and M i The radial limit parameter for the normalization process, α is the scaling factor in the process, Defined as:

[0023]

[0024] Let S be the min ,R max ], the average value determined by the complete rapid radial symmetry method for the symmetry of interest:

[0025]

[0026] According to the obtained average value S, the centroid of the symmetrical area of ​​the elliptical target is detected to extract the center point of the target object;

[0027] S4. Extract the edge of the target object:

[0028] S4-1. Use the traversal method to detect concave points on the edge of the overlapping part of the target object. Given a circular template with a radius of r, the total number of pixels in the circle is P. The template is traversed over the edge points of the overlapping target. For each edge point, the number of pixels in the circular template that overlap with the overlapping area is calculated as A. The jth point E is determined according to the following formula (8): j Is it a concave point?

[0029]

[0030] Where t is the edge point E j Whether it is a concave point threshold parameter;

[0031] S4-2, edge feature extraction: The concave point set E = {e1, e2, ..., e m} and the center point set S = {s1,s2,...,s n}, the edge feature extraction function f(e k ,s l ) is defined as:

[0032]

[0033] Where λ is the weight, the distance function dis(e k ,s l ) Calculate from the concave point e k To the nearest center point s l The distance l, the divergence function div(e k ,s l ) Calculate e k to l The difference between the straight line direction and the gradient direction at that location is estimated according to the following formula (10):

[0034]

[0035] Using the function f(e k ,s l ) represents the target edges with different center points respectively;

[0036] S5. Predict the contour of the target object:

[0037] After obtaining the edge contour of the target object, the target part of the visible area is modeled in the form of an elliptical object, and the edge of the occluded part of the target object is inferred by the ellipse fitting method based on least squares; for the ellipse equation, given a point (x, y), parameter vector a = [a0 a1 a2 a3 a4 a5], the objective function d(a, (x, y)) of the second-order polynomial algebraic distance is expressed as:

[0038] d(a,(x,y))=a0x 2 +a1xy+a2y 2 +a3y+a4y+a5 (11)

[0039] For all points {(x i ,y i )|i=1,2,...,n} The constructed linear regression matrix D is expressed as:

[0040]

[0041] According to the arc fitting algorithm, the constraint matrix C is expressed as:

[0042]

[0043] According to the Lagrange multiplier method, given the Lagrange multiplier λ and the differential:

[0044] D T D a+2λC a=0(14)

[0045] Solve the system of equations as a minimization problem:

[0046]

[0047] The ellipse is fitted by solving all characteristic solutions that satisfy the conditions of equation (15) to predict the missing edges of the occluded parts of the overlapping elliptical targets.

[0048] The present invention provides an elliptical object segmentation method based on ellipse fitting. The method segments overlapping elliptical objects based on center point detection, edge extraction, and ellipse fitting. Users can segment the target object and calculate the area, azimuth, and other parameters of a single target through this method. Compared with the prior art, the present invention has the following advantages:

[0049] (1) When extracting the target edge, the concave point detection method and contour estimation scheme are combined to ensure the accurate extraction of overlapping target edge points.

[0050] (2) When segmenting the target object, avoid the problem of incomplete object edge extraction and unsatisfactory segmentation effect caused by similar RGB color gradient values ​​in the image.

[0051] (3) The missing parts of overlapping targets are predicted by the ellipse fitting method, which solves the problem that the complete outline of partially overlapping elliptical targets cannot be obtained in the two-dimensional image under the camera's perspective.

[0052] (4) Compared with the object segmentation algorithm based on deep learning, it saves labor costs while ensuring the segmentation effect and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of an elliptical object segmentation method based on ellipse fitting of the present invention;

[0054] Figure 2 Schematic diagram of image preprocessing;

[0055] Figure 3 This is a schematic diagram of the center point of a plant leaf obtained using the fast radial symmetry method;

[0056] Figure 4 Schematic diagram of the plant leaf contour obtained by concave point detection and edge extraction;

[0057] Figure 5 Schematic diagram of the complete outline of the overlapping elliptical plant leaves obtained by fitting. DETAILED DESCRIPTION

[0058] The following takes an elliptical plant leaf as an example, and further describes the content of the present invention in combination with the accompanying drawings and embodiments, but does not limit the present invention.

[0059] Example:

[0060] like Figure 1 As shown, a method for segmenting an elliptical object based on ellipse fitting comprises the following steps:

[0061] S1. Image acquisition:

[0062] The user uses a digital camera, mobile phone or other device to capture plant leaf images from above the plant at an angle perpendicular to the leaves, then uses image processing software to extract the leaf area to be segmented, and converts the image format into a JPG image format;

[0063] S2, image preprocessing, the preprocessed image is as follows Figure 2 As shown:

[0064] S2-1, maximum connected area extraction: Since the plant leaves in the intercepted image are always concentrated in an area in the image, only the maximum value of the G component area of ​​the plant leaves is extracted from the RGB composition of the entire image, and the pixel values ​​of the remaining green blocks are changed to 0;

[0065] S2-2, super green segmentation algorithm: Based on the image obtained in step S2-1, suppose there are i pixels in the image, S i is the pixel value of each pixel, R, G, and B are the flux values ​​of red, green, and blue in the image respectively; by adjusting the saturation of the plant leaf image under different lighting conditions, the threshold T is determined using the maximum inter-class variance calculation method graythresh in Matlab, according to:

[0066]

[0067] When the grayscale factor of the leaves in the target area of ​​the image is less than or equal to the threshold T, the pixel value of the marked image at the corresponding position is equal to 255, otherwise it is equal to 0, and the non-green part of the image is removed and the image is binarized;

[0068] S2-3, top hat transformation: in the binary image obtained in step S2-2, the extracted target area includes the plant leaf and its petiole, and the petiole is removed by opening the grayscale image g to avoid deviation in subsequent image processing;

[0069] S3. Extract the center point of plant leaves:

[0070] The center point of the elliptical plant leaf is extracted using the fast radial symmetry method. The fast radial symmetry method is a local feature extraction technology for highlighting the image gradient. It detects points with a specific distance from the edge pixel points of the plant leaf in the target area one by one, and selects the center point that meets the requirements. The specific distance is the distance between the center point and the edge pixel points of the target area.

[0071] Given a range distance value i∈[R min ,R max ], for the pixel value of any point in the target area of ​​the image, the positive and negative influence factors of the pixel are determined by the fast radial symmetry method + 、p - , and the directional projection image O i With the amplitude projection image M i : The expression is as follows:

[0072]

[0073]

[0074] In the region i∈R min ,Rmax ], the symmetric radial S i Depends on i 、M i and the two-dimensional Gaussian A i :

[0075]

[0076] Where k is the number of different radii on O i and M i The radial limit parameter for the normalization process, α is the scaling factor in the process, Defined as:

[0077]

[0078] Let S be the min ,R max ], the average value determined by the complete rapid radial symmetry method for the symmetry of interest:

[0079]

[0080] The centroid of the symmetrical region of the elliptical plant leaf is detected according to the obtained average value S, and the center point of the plant leaf is extracted. The center point of the plant leaf is as follows: Figure 3 As shown;

[0081] S4. Extract the edge of plant leaves:

[0082] S4-1. Use the traversal method to detect concave points on the edges of the overlapping parts of the plant leaves. Given a circular template with a radius of r, the total number of pixels in the circle is P. The template is traversed over the edge points of the overlapping leaves. For each edge point, the number of pixels in the circular template that overlap with the overlapping plant leaf area is calculated as A. The jth point E is determined according to the following formula (8): j Is it a concave point?

[0083]

[0084] Where t is the edge point E j Whether it is a concave point threshold parameter;

[0085] S4-2, edge feature extraction: The concave point set E = {e1, e2, ..., e m} and the center point set S = {s1,s2,...,s n}, the edge feature extraction function f(e k ,s l ) is defined as:

[0086]

[0087] Where λ is the weight, the distance function dis(e k ,s l ) Calculate from the concave point e k To the nearest center point s l The distance l, the divergence function div(e k ,s l ) Calculate e k to l The difference between the straight line direction and the gradient direction at that location is estimated according to the following formula (10):

[0088]

[0089] Using the function f(e k ,s l ) to represent the edges of plant leaves with different center points respectively;

[0090] The contour of the plant leaf obtained by concave point detection and edge extraction is as follows Figure 4 As shown;

[0091] S5. Prediction of the outline of the obscured part of the plant leaf:

[0092] After obtaining the leaf edge contour, the plant leaf part in the visible area is modeled in the form of an elliptical object, and the missing part of the overlapping leaves is inferred by the ellipse fitting method based on least squares; for the ellipse equation, given a point (x, y), the parameter vector a = [a0 a1 a2 a3 a4 a5], the objective function d(a, (x, y)) of the second-order polynomial algebraic distance is expressed as:

[0093] d(a,(x,y))=a0x 2 +a1xy+a2y 2 +a3y+a4y+a5 (11)

[0094] For all points {(x i ,y i )|i=1,2,...,n} The constructed linear regression matrix D is expressed as:

[0095]

[0096] According to the arc fitting algorithm, the constraint matrix C is expressed as:

[0097]

[0098] According to the Lagrange multiplier method, given the Lagrange multiplier λ and the differential:

[0099] D T Da+2λCa=0(14)

[0100] Solve the system of equations as a minimization problem:

[0101]

[0102] The ellipse is fitted by solving all characteristic solutions that satisfy the conditions of equation (15) to predict the missing edge of the occluded part of the overlapping elliptical leaves; the complete outline of the plant leaf obtained by ellipse fitting is as follows: Figure 5 shown.

Claims

1. A method for detecting elliptical objects based on ellipse fitting, characterized in that: The steps include: S1. Image acquisition: Use image acquisition equipment to collect images from above the target object at an angle perpendicular to the object, then use image processing software to cut out the target area to be segmented, and convert the image format into JPG image format; S2. Image preprocessing: S2-1, maximum connected area extraction: Since the overlapping objects in the intercepted image are always concentrated in one area of ​​the image, only the maximum value of the G component area is extracted from the RGB composition of the entire image, and the pixel values ​​of the remaining green blocks are changed to 0; S2-2, super green segmentation algorithm: Based on the image obtained in step S2-1, suppose there are i pixels in the image, S i is the pixel value of each pixel, R, G, and B are the flux values ​​of red, green, and blue in the image respectively; by adjusting the image saturation under different lighting conditions, the threshold T is determined using the maximum inter-class variance calculation method graythresh in Matlab, according to: When the grayscale factor of the leaves in the target area of ​​the image is less than or equal to the threshold T, the pixel value of the marked image at the corresponding position is equal to 255, otherwise it is equal to 0, and the non-green part of the image is removed and the image is binarized; S2-3, top hat transformation: in the binary image obtained in step S2-2, the extracted target area includes the target object and its attached parts, and the attached parts, i.e., non-target areas, are removed by opening operation of the grayscale image g to avoid deviation in subsequent image processing; S3. Extract the center point of the target object: The center point of the elliptical object is extracted using the fast radial symmetry method. The fast radial symmetry method is a local feature extraction technology for highlighting the image gradient. It detects points with a specific distance from the edge pixel points of the target area one by one, and selects the center point that meets the requirements. The specific distance is the distance between the center point and the edge pixel points of the target area. Given a range distance value i∈[R min ,R max ], for the pixel value of any point in the target area of ​​the image, the positive and negative influence factors of the pixel are determined by the fast radial symmetry method + 、p - , and the directional projection image O i With the amplitude projection image M i : The expression is as follows: In the region i∈R min ,R max ], the symmetric radial S i Depends on i 、M i and the two-dimensional Gaussian A i : Where k is the number of different radii on O i and M i The radial limit parameter for the normalization process, α is the scaling factor in the process, Defined as: Let S be the min ,R max ], the average value determined by the complete rapid radial symmetry method for the symmetry of interest: According to the obtained average value S, the centroid of the symmetrical area of ​​the elliptical target is detected to extract the center point of the target object; S4. Extract the edge of the target object: S4-1. Use the traversal method to detect concave points on the edge of the overlapping part of the target object. Given a circular template with a radius of r, the total number of pixels in the circle is P. The template is traversed over the edge points of the overlapping target. For each edge point, the number of pixels in the circular template that overlap with the overlapping area is calculated as A. The jth point E is determined according to the following formula (8): j Is it a concave point? Where t is the edge point E j Whether it is a concave point threshold parameter; S4-2, edge feature extraction: The concave point set E = {e1, e2, ..., e m } and the center point set S = {s1,s2,...,s n }, the edge feature extraction function f(e k ,s l ) is defined as: Where λ is the weight, the distance function dis(e k ,s l ) Calculate from the concave point e k To the nearest center point s l The distance l, the divergence function div(e k ,s l ) Calculate e k to l The difference between the straight line direction and the gradient direction at that location is estimated according to the following formula (10): Using the function f(e k ,s l ) represents the target edges with different center points respectively; S5. Predict the contour of the target object: After obtaining the edge contour of the target object, the target part of the visible area is modeled in the form of an elliptical object, and the edge of the occluded part of the target object is inferred by the ellipse fitting method based on least squares; for the ellipse equation, given a point (x, y), parameter vector a = [a0 a1 a2 a3 a4 a5], the objective function d(a, (x, y)) of the second-order polynomial algebraic distance is expressed as: <h2 style=";text-align:left;direction:ltr">d(a,(x,y)) = a0x<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +a1xy+a2y<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +a3y+a4y+a5 (11) For all points {(x i ,y i )|i=1,2,...,n} The constructed linear regression matrix D is expressed as: According to the arc fitting algorithm, the constraint matrix C is expressed as: According to the Lagrange multiplier method, given the Lagrange multiplier λ and the differential: D T Da+2λCa=0(14) Solve the system of equations as a minimization problem: The ellipse is fitted by solving all characteristic solutions that satisfy the conditions of equation (15) to predict the missing edges of the occluded parts of the overlapping elliptical targets.

Images