Machine vision ellipse detection method based on geometric constraint and hierarchical clustering

Abstract

The invention discloses a geometric constraint and hierarchical clustering-based machine vision ellipse detection method. The method comprises the following steps of: 1, preprocessing an image edge, and performing edge detection weighting; step 2, screening edge contours meeting candidate ellipse conditions to obtain a first ellipse and an ellipse arc segment set; step 3, grouping the elliptic arc segments in the elliptic arc segment set to obtain a quadrant constraint arc group; 4, obtaining a candidate ellipse according to the quadrant constraint arc group, and calculating a feature vector of the candidate ellipse; step 5, performing hierarchical clustering on the candidate ellipses by adopting a hierarchical clustering method to obtain a clustered ellipse set, and taking all ellipses in the set as second ellipses; and 6, taking a set of the first ellipse and the second ellipse as an ellipse detection result. According to the method, high detection efficiency and low execution time are achieved, in addition, the purity and accuracy of the detected ellipse are improved through strict ellipse verification, and high positioning precision and robustness are ensured.

Description

Technical Field

[0001] The present invention belongs to the technical fields of computer vision and image processing, and particularly relates to a machine vision ellipse detection method based on geometric constraints and hierarchical clustering. Background Art

[0002] In machine vision, elliptical objects often appear within the region of interest (ROI) of a camera, and the ellipse is one of the most common non-linear geometric shapes in the real world. Therefore, ellipse detection is crucial for shape detection and geometric measurement based on machine vision. These technologies can solve a wide range of real-world problems, including industrial inspection, pose measurement, camera calibration, medical research, traffic sign recognition, face recognition, and object tracking on robotic platforms. However, real-world images usually have characteristics such as complex backgrounds, motion blur, different lighting conditions, occlusions, and noise. Therefore, there are still few general-purpose ellipse detection solutions. These challenges result in existing methods performing poorly in terms of detection accuracy, execution time, or both. In addition, the ellipse is not always completely visible and may be partially occluded or hidden by other objects, resulting in incomplete elliptical curves in the image. In the region where the boundaries of two objects intersect, the edge contours may be disrupted, making the detection of elliptical geometries in real images more complex.

[0003] Existing ellipse detection algorithms often face problems such as high computational complexity, strong dependence on initial conditions, sensitivity to noise, and lack of robustness to occlusions. Currently, the mainstream ellipse detection methods are mainly divided into three categories: model-based methods, machine learning-based methods, and edge-linking-based methods. Model-based ellipse detection encounters real-time obstacles and is sensitive to initialization, while machine learning methods rely on a large amount of training data. Edge segment detection methods are considered the most effective methods for detecting multiple ellipses in digital images and have relatively lower computational costs. However, they are relatively dependent on the accuracy of arc detection, and when these methods are applied to images containing complex occlusions or incomplete contours, errors are likely to occur in arc detection. Summary of the Invention

[0004] The purpose of the present invention is to provide a machine vision ellipse detection method based on geometric constraints and hierarchical clustering to solve the problem that it is difficult to accurately detect the incomplete elliptical shapes in the pictures taken by existing industrial cameras.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions:

[0006] A machine vision ellipse detection method based on geometric constraints and hierarchical clustering specifically includes the following steps:

[0007] Step 1, perform image edge preprocessing and edge detection weighting to obtain a weighted edge image;

[0008] Step 2, screen the edge contours that meet the candidate ellipse conditions to obtain the first ellipse and the set of elliptical arc segments;

[0009] Step 3, group the elliptical arc segments in the set of elliptical arc segments obtained in Step 2 to obtain the quadrant constraint arc groups;

[0010] Step 4, based on the quadrant constraint arc groups obtained in Step 3, obtain candidate ellipses, and calculate the eigenvectors of the candidate ellipses, including the ellipse center coordinates, major axis, minor axis, and rotation angle;

[0011] Step 5, according to the eigenvectors of the candidate ellipses obtained in Step 4, use the hierarchical clustering method to perform hierarchical clustering on the candidate ellipses to obtain the clustered ellipse set, and use all the ellipses in this set as the second ellipse;

[0012] Step 6, use the set of the first ellipse and the second ellipse as the ellipse detection result.

[0013] Compared with the prior art, the present invention has the following technical effects:

[0014] (1) In steps 24 of the method of the present invention, the sector verification and voting mechanism are combined to effectively improve the accuracy and robustness of the first ellipse detection. In this step, the RANSAC method is first used to fit potential ellipses from each closed curve, and then it is divided into multiple sectors. The gradient direction of each sector is calculated and compared with the actual edge points to verify the effectiveness of the potential ellipses. Then, the vote count is calculated based on the continuous set of effective sectors, and a threshold is set to determine whether the potential ellipse is effective. If the verification fails, the parameters of the potential ellipse are adjusted and refitted until the best match is found. Step 24 can efficiently verify in complex backgrounds and noise environments, reduce false ellipses, and improve the purity and accuracy of the detection results.

[0015] (2) In step 25 of the method of the present invention, the improved KRDP algorithm with a tolerance threshold is used to process non-closed curves and divide them into elliptical arc segments. The algorithm first performs line segment fitting and angle change detection to identify the turning points and inflection points of the curve, and then accurately divides them according to the curvature change and the set threshold, retaining the segmentation points with large curvature changes and removing the small change points. Finally, the original curve is divided into multiple arc segments, and at the same time, the edge points near the turning points or inflection points are removed to ensure that the segmented elliptical arc segments are smoother, more continuous, and closer to the shape of the original curve.

[0016] (3) Step 3 of the method of the present invention classifies the concave and convex of the elliptical arc segments, divides them into quadrants, screens out the arc segments that meet the elliptical shape requirements, and combines them into six groups of quadrant-constrained arc groups. First, determine whether an arc segment is a concave arc or a convex arc according to the curvature value of the endpoints of the elliptical arc segment, and then classify the arc segments into different quadrants through quadrant division. Then, select arc segments belonging to different quadrants to form an arc group, and by analyzing their relative positional relationships, apply geometric constraints to screen out the arc groups that meet the elliptical conditions. Step 3 ensures that the screened arc segments can form an effective elliptical shape through strict geometric conditions and quadrant constraints, avoiding the interference of invalid or unqualified arc segments.

[0017] (4) Step 4 of the method of the present invention extracts the characteristic parameters of the candidate ellipse from the quadrant-constrained arc group through a detailed calculation process, including the center of the ellipse, the major axis, the minor axis, and the rotation angle. First, accurately obtain the geometric relationships of each pair of arcs by calculating the midpoints, tangent intersection points, and the positions of vertical lines of each arc segment; then, based on this geometric information, estimate the center of the candidate ellipse, and obtain the parameters in the elliptical coordinate system through coordinate transformation. If the distance between the center points of the candidate ellipses exceeds the set threshold, then this group of arc segments is excluded. Finally, calculate the characteristic vectors of all candidate ellipses that meet the conditions. Step 4 ensures that the obtained candidate ellipse parameters are more accurate through complex geometric construction and constraint conditions. The characteristic vectors of the candidate ellipses obtained by calculation provide a basis for subsequent ellipse clustering, further improving the stability and effect of detection.

[0018] (5) Step 5 of the method of the present invention classifies the candidate ellipses through a hierarchical clustering method, decomposes the elliptical parameter space into three independent clustering spaces (center of the ellipse, semi-axis length, rotation angle), and refines the clustering process layer by layer. First, perform the first-layer clustering based on the center coordinates of the ellipse, then within each center clustering, perform the second-layer clustering based on the semi-axis length, and finally within each semi-axis length clustering, perform the third-layer clustering according to the rotation angle. Through this hierarchical clustering, ellipses with similar characteristics can be more accurately identified and excluded, and finally an accurate set of ellipse clusters is obtained. Step 5 improves the clustering efficiency and accuracy by decomposing the complex 5D elliptical parameter space problem into three independent low-dimensional clustering problems.

[0019] In summary, the method of the present invention can effectively identify and robustly detect elliptical shapes with defective parts, especially half-elliptical shapes. The method of the present invention achieves high detection efficiency and low execution time. In addition, strict ellipse verification improves the purity and accuracy of the detected ellipses, ensures high positioning accuracy and robustness, and at the same time rejects false alarms. Description of the Drawings

[0020] Figure 1It is a schematic diagram for initially distinguishing quadrants according to gradients in step 2 of the method of the present invention.

[0021] Figure 2 It is a schematic diagram for segmenting inflection points and turning points by the improved KRDP algorithm with a tolerance threshold in step 2 of the method of the present invention.

[0022] Figure 3 It is a schematic diagram for dividing arcs into six arc groups according to endpoint coordinates and curvature in step 3 of the method of the present invention.

[0023] Figure 4 It is a schematic diagram for calculating the center of an ellipse based on two arcs through geometric constraints in step 4 of the method of the present invention.

[0024] Figure 5 It is the test result on two natural data sets.

[0025] Figure 6 It is the test result on two industrial data sets.

[0026] Figure 7 It is the test result on three synthetic data sets.

[0027] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Specific Embodiments

[0028] The following is in combination with a specific implementation process, but the implementation manner is not limited thereto.

[0029] The machine vision ellipse detection method based on geometric constraints and hierarchical clustering given by the present invention specifically includes the following steps:

[0030] Step 1, preprocess the image edge and perform edge detection weighting to obtain a weighted edge image. It includes the following sub-steps:

[0031] Step 11, preprocess the original image captured by an industrial camera by denoising and grayscaling to obtain a preprocessed image, so as to improve the recognition accuracy of subsequent algorithms.

[0032] Step 12, edge detection weighting. Specifically: use the Canny edge detector with an automatic threshold and the Sobel operator to respectively obtain two edge images from the preprocessed image, and then fuse the pixel values and gradients τ of each pixel point of the two edge images through mean weighting i , so as to obtain the pixel p i =(x i ,y i ,τ i ) of the weighted edge image, where the weighting value is selected as an empirical value according to a specific image.

[0033] Step 2, screening edge contours that meet the candidate ellipse conditions to obtain the first ellipse and an ellipse arc segment set. Specifically includes the following sub-steps:

[0034] Step 21: Calculate the pixel point p on each edge line in the weighted edge image obtained in step 1. i The gradient symbol Τ(p i ), divide the contour lines composed of multiple continuous pixel points with positive gradient signs into the first quadrant or the third quadrant, and divide the contour lines composed of multiple continuous pixel points with negative gradient signs into the second quadrant or the fourth quadrant, and obtain two contour quadrant sets D(C T )∈{ArcI∪ArcIII}, D(C T )∈{ArcII∪ArcIV}.

[0035]

[0036] In step 21, in order to effectively cut invalid contour line combinations in post-processing, the edge lines in the weighted edge image are divided into two groups corresponding to contour lines distributed in two quadrants, such as Figure 1 As shown, these contour lines are called contour quadrant sets.

[0037] Step 22, retain the contour lines with a length greater than 12 pixels in the two contour quadrants to exclude noise, and delete the remaining contour lines. The sth contour line retained Where (x i ,y i ) represents the coordinates of the i-th pixel point on the contour line, n is the number of pixels on the contour line; calculate the curvature of the pixel points on each contour line.

[0038] Specifically, the curvature K of the i-th pixel i The calculation formula is as follows:

[0039]

[0040] Among them, x i ,y i Respectively represent the horizontal and vertical coordinates of pixel point i on the continuous contour line. i The first derivative of and the second derivative The calculation formula is as follows (the same for the vertical axis):

[0041]

[0042] Among them, x i-2 、x i-1 、x i+1 、x i+2They are the abscissas of pixel points i - 2, i - 1, i + 1, and i + 2 respectively. Specifically, if pixel point i = 1 or 2 (i.e., the starting endpoints of the continuous contour line), the forward difference method is used, and the two pixels behind the starting pixel are used to estimate the derivative. If pixel point i = n - 1 or n - 2 (i.e., the ending endpoints of the continuous contour line), the backward difference is used, and the two pixels in front of the ending pixel are used to estimate the derivative.

[0043] Step 23: Connect all the pixel points with similar curvatures on each contour line to obtain a continuous curve, and get the continuous curves corresponding to all the contour lines. Then, divide all the continuous curves into closed curves and non-closed curves according to the distance between the two endpoints of the curve (specifically, if the distance between the two endpoints is less than the preset threshold, it is regarded as a closed curve, otherwise it is regarded as a non-closed curve).

[0044] Step 24: Take each closed curve obtained in Step 23 as a potential ellipse for sector verification, and save the potential ellipses corresponding to all the closed curves that pass the verification as the first ellipses.

[0045] Specifically, the specific process of sector verification for each closed curve is as follows:

[0046] Step 1: Use the method of Random Sample Consensus (RANSAC) to fit the parameters of a potential ellipse that the closed curve may correspond to.

[0047] Step 2: Divide the potential ellipse into multiple sectors equally.

[0048] Step 3: Based on the parameters of the potential ellipse, calculate the theoretical gradient direction of the center point of each sector on the circumference of the potential ellipse. Compare the actual gradient direction of the pixel points on the potential ellipse with the theoretical gradient direction. If the actual gradient direction is aligned with the theoretical direction, mark the edge point and its corresponding sector as valid.

[0049] Step 4: According to the order of the valid sectors, find a set of continuous sectors composed of at least three adjacent valid sectors.

[0050] Step 5: Calculate the voting numbers contributed by each found set of continuous sectors. Use these voting numbers to verify whether the potential ellipse meets the conditions.

[0051] Specifically, the voting number contributed by the set of continuous sectors = the number of sectors in the set of continuous sectors × the total number of sectors, where the total number of sectors refers to the number of sectors divided in Step 1.

[0052] Step 6: Set the voting threshold according to the detection experience in different scenarios; if the voting number of the potential ellipse (i.e., the sum of the voting numbers of all continuous sector sets) reaches the preset threshold, mark the potential ellipse as successfully verified, and remove the pixels contributing to the voting from the edge data of the closed curve to avoid repeated calculation. If the preset threshold is not reached, adjust the parameter model of the potential ellipse (such as the center, major and minor axes, rotation angle), and then re - perform RANSAC fitting based on the new parameter model of the potential ellipse, that is, randomly sample again from the edge data of each closed curve obtained in step 23, estimate the new potential ellipse parameters according to the sampling results, and return to step 2 until the predetermined termination condition is reached (reaching the maximum number of iterations or obtaining the verification result).

[0053] Among them, the setting of the maximum number of iterations is to prevent excessive computational consumption. In this step, when the verification of the potential ellipse fails, it indicates that some of the current fitted potential ellipse parameters (such as the center, major and minor axes, rotation angle) do not match the actual closed curve well. Therefore, these parameters need to be adjusted to generate a new potential ellipse for new verification until an ellipse that can match the closed curve is found.

[0054] Step 7: Save the potential ellipses corresponding to all the verified closed curves as the first ellipses.

[0055] In this step, through sector - by - sector verification, the combination of geometric characteristics and voting mechanism is utilized, enabling efficient ellipse detection performance to be maintained when dealing with complex backgrounds and noises. It not only reduces the appearance of false ellipses but also improves the purity and accuracy of the detection results. This improvement is of great significance for enhancing the robustness and practicality of the ellipse detection algorithm.

[0056] Step 25: Process each non - closed curve obtained in step 23 using an improved KRDP algorithm with a tolerance threshold to obtain the elliptical arc segments corresponding to each non - closed curve, as Figure 2 shown. The elliptical arc segments corresponding to all non - closed curves form an elliptical arc segment set.

[0057] Among them, processing each non - closed curve using an improved KRDP algorithm with a tolerance threshold includes the following process:

[0058] Step 1: Fit each non - closed curve C s with line segments and save the fitted line segments as the line segment set L corresponding to the non - closed curve C s ; sequentially calculate the angles between every two line segments in the line segment set L to obtain the angle set θ 1 , θ 2 , …, θ m-1 , where θ 1 is the angle between line segment L 1 and line segment L2 The included angle between them, θ m-1 is the line segment L m-1 and the line segment L m The included angle between them, m is the number of line segments in the line segment set L;

[0059] Step 2: Initialize the point set P, and the point set P contains the two endpoints of the non-closed curve C s and their curvature values; traverse the included angle set. If |θ 1 -θ i | exceeds the preset included angle change threshold T θ , search for the original non-closed curve C i corresponding to the line segment L s and add the point and its curvature value to the point set P until all line segments in the line segment set L are traversed; traverse the point set P, compare the curvature changes of adjacent points, and retain the points with curvature changes exceeding the preset curvature threshold T k in the point set P, otherwise delete the point. If |θ 1 -θ i | corresponding to a non-closed curve is less than the threshold T θ , and the curvature changes are all zero or less than the curvature threshold T k , then mark the non-closed curve as a straight line and filter it out.

[0060] Step 3: Divide each remaining non-closed curve after filtering in Step 2 into arc segments;

[0061] Specifically: According to the segmentation points retained in the point set P, all points between two adjacent segmentation points on the non-closed curve form the current arc segment, thereby dividing the non-closed curve into multiple arc segments.

[0062] In step 25, through curvature and pixel screening, the turning points and inflection points in the non-closed curve are identified and divided into elliptical arc segments with a high similarity to the candidate ellipse shape. At the same time, the edge points near the turning points are removed to make the generated elliptical arc segments more continuous and smooth and as close as possible to the original curve.

[0063] In step 2, first screen the edge contours that meet the candidate ellipse conditions to obtain closed curves and non-closed curves. By verifying the obtained closed curves, the first ellipse is determined; for non-closed curves, through the segmentation of turning points and inflection points, they are split into multiple elliptical arc segments to obtain an elliptical arc segment set.

[0064] Step 3, group the elliptical arc segments in the elliptical arc segment set obtained in step 2 to obtain a quadrant constraint arc group, including the following sub-steps:

[0065] Step 31: Find all concave arcs and convex arcs in the set of elliptical arc segments. Specifically, if the curvatures of the two endpoints of an elliptical arc segment are both positive, it is a concave arc, and its concavity / convexity symbol Ω(C) is -, if the curvatures of the two endpoints are both negative, it is a convex arc, and its concavity / convexity symbol Ω(C) is +. Other elliptical arc segments in the set of elliptical arc segments that cannot be classified as convex arcs or concave arcs will be excluded from subsequent geometric analysis or recognition processes.

[0066] Step 32: Perform quadrant division on all concave arcs and convex arcs respectively. For example, consider an arc segment D(C T ) ∈ {ArcI ∪ ArcIII} in the set of quadrant contours. If the curvatures of the two endpoints are positive, then this arc segment is a concave arc, and its concavity / convexity symbol Ω(C) is -, and it will be classified into the third quadrant (ArcIII), otherwise it will be classified into the first quadrant (ArcI).

[0067] Step 32: Select two arcs C a , C b with different quadrants from all the arc segments obtained in Step 31 to form an arc group, as shown in Figure 3 .

[0068] Step 33: According to the quadrants of the two arcs in the arc group and the coordinate values of the arc endpoints, screen out the quadrant-constrained arc group from the arc groups obtained in Step 32.

[0069] Specifically, any arc group that conforms to any one of the following 6 formulas is a quadrant-constrained arc group:

[0070]

[0071] Among them, and represent the coordinate values of the left endpoint of arc i, and represent the coordinate values of the right endpoint of arc i, where i is a or b.

[0072] The above 6 formulas describe the relative positions of 6 types of arcs. Among them, the first formula means: when a quadrant-constrained arc group C a , C b belong to ArcI and ArcII (the first and second quadrants) respectively, and , that is, when the x value of the left endpoint of arc C a is greater than the x value of the right endpoint of arc C b , this quadrant-constrained arc group is retained.

[0073] The purpose of Step 33 is to screen out arc groups that may belong to the same ellipse and discard arc groups that do not meet the conditions for forming an ellipse.

[0074] Step 4. Based on the quadrant constraint arc groups obtained in Step 3, obtain candidate ellipses and calculate the eigenvectors of the candidate ellipses, including the center coordinates of the ellipse, the major axis, the minor axis, and the rotation angle. It includes the following sub-steps:

[0075] Step 41. For each quadrant constraint arc group ε ab =(C a , C b ), as shown in Figure 4 , denote the midpoints of arc C a and C b as point M a and point M b respectively; make tangents (a total of 3) at both ends and the midpoint of arc C a , and the tangent at the midpoint intersects the tangents at both ends at point P 1 , point P 2 ; the corresponding intersection points on arc C b are point P 3 , point P 4 ; calculate the xy coordinates of point P 1 , P 2 , P 3 , P 4 respectively.

[0076] Specifically, the x and y coordinates of point P 1 and point P 2 are calculated by the following formula, and the coordinate calculation of P 3 , P 4 is the same:

[0077]

[0078] Among them, respectively represent the left and right endpoints of arc C a and arc C b ; M a .x, M a .y, M a .τ are the x, y coordinates and gradient of the midpoint M a of arc C a .

[0079] Step 42. Connect the two endpoints of each arc in the quadrant constraint arc group to the midpoint of the arc respectively, and calculate the coordinates of the midpoints m 1 , point m 2 of the two connected line segments respectively. The calculation formula is as follows:

[0080]

[0081] Step 43. Construct two vertical lines on arc C a , one of which passes through point P1 and point m 1 the straight line l 1 , and the other is the straight line l that passes through point P 2 and point m 2 the straight line l 2 ; Similarly, construct two perpendicular lines l b on the arc C 3 、l 4 , and calculate the left and right endpoints of the arc C a respectively to the slope of the midpoint M : a The slope of the straight lines l

[0082]

[0083] is calculated using the following formula 1 、l 2 :

[0084]

[0085] Step 44, calculate the coordinates of the center points of a pair of arcs C a , C b in the quadrant constraint arc group;

[0086] Specifically, the coordinates of the center point of the arc C a (C a .x, C a .y) are calculated as follows, and similarly, the center of the arc C b is obtained, and the center of a pair of arcs C a , C b in the quadrant constraint arc group.

[0087]

[0088] Step 45, if the distance between the center points of a pair of arcs in the quadrant constraint arc group exceeds the preset threshold T c , then delete the two arcs of this quadrant constraint arc group; retain the center coordinates of the remaining quadrant constraint arc groups as the center coordinates of the candidate ellipse corresponding to the quadrant constraint arc group.

[0089] In addition, the intersection coordinates of the four straight lines l 1 , l 2 , l 3 , l 4 intersecting pairwise, and the average value of all intersection coordinates are also regarded as the center of the candidate ellipse for further estimating other parameters.

[0090] All the calculated candidate ellipse centers mentioned above constitute the candidate ellipse center set Ο c corresponding to the current quadrant constraint arc group.

[0091] Step 46, calculate parameter K and the rotation angle of the candidate ellipse

[0092]

[0093] where α = q 1 q 2 -q 3 q 4 , β = q 2 q 4 (q 3 -q 1 ) + q 1 q 3 (q 4 -q 2 ) + (q 1 +q 2 -q 3 -q 4 );

[0094] Step 47, calculate operator N:

[0095]

[0096] Parameters K and operator N are used to decompose the parameter space of the candidate ellipse and implement the aggregation operation on the parameter space of the candidate ellipse.

[0097] Step 48, convert the center coordinates of each candidate ellipse in the ellipse center set Ο c from the coordinates (x i , y i ) in the world coordinate system to the coordinates (x e , y e ) in the ellipse coordinate system. The formula is as follows:

[0098]

[0099] where (x c , y c ) is the center of a candidate ellipse in the ellipse center set Ο c .

[0100] According to the following equation can be obtained:

[0101]

[0102] Calculate the major semi - axis where Calculate the minor semi - axis B = A·N.

[0103] Through the above calculations, a pair of arcs C a, C b Corresponding to the five parameters of a candidate ellipse

[0104]

[0105] Step 49, traverse the set Ο of candidate ellipse centers obtained in step 45 c , to obtain the feature vectors of all candidate ellipses corresponding to a set of quadrant constraint arcs (i.e., a pair of arcs C a , C b ) (x i , y i ) represents the coordinates of the center of a candidate ellipse, (a i , b i ) represents the major and minor axes, is the rotation angle of the candidate ellipse.

[0106] Step 5, according to the feature vectors of the candidate ellipses obtained in step 4, use the hierarchical clustering method to perform hierarchical clustering on the candidate ellipses to obtain the clustered ellipse set, and use all the ellipses in this set as the second ellipse.

[0107] This method decomposes the 5D ellipse parameter space clustering problem into two 2D spaces (center and semi-axes) and one 1D space (orientation) clustering problems. The similarity between two ellipses is evaluated by comparing the differences in ellipse parameters.

[0108] Step 5 specifically includes the following sub-steps:

[0109] Step 1: Perform the first layer of clustering based on the ellipse center coordinates:

[0110] Assign all candidate ellipse feature vectors V i to an initial ellipse set E I , and use the Gaussian Mean Shift method to cluster the ellipse center coordinates in the initial ellipse set E I to obtain the clustering centers (x, y) of N C ellipse centers 1 , (x, y) 2 ,..., (x, y) j ,..., (x, y) Nc ; for each clustering center point (x, y) j , calculate its Euclidean distance d i from the center coordinates of each ellipse in the ellipse feature vector V min , if d min is the current minimum distance, then use the current feature vector (i.e., the clustering center point and the ellipse feature vector V iThe one with the shortest Euclidean distance between the center coordinates of each ellipse is assigned to the first clustering set A j ;

[0111] Step 2: Perform the second-layer clustering based on the semi-axis lengths (major axis and minor axis):

[0112] For each first clustering set A j , perform clustering based on the semi-axis lengths of the ellipses therein to obtain N C,a cluster centers (a, b) of the semi-axis lengths 1 , (a, b) 2 ...(a, b) Nc,a , for each semi-axis length clustering center point (a, b) k , calculate the Euclidean distance d between its major and minor semi-axes and the eigenvector of each ellipse min . If d min is the current minimum distance, then assign the eigenvector of this ellipse to the second clustering set A j,k ;

[0113] Step 3: Perform the third-layer clustering based on the ellipse orientation angle:

[0114] For the second clustering set A j,k , perform clustering based on the rotation angle of the ellipses therein to obtain cluster center points of the rotation angles For each orientation angle clustering center point calculate the Euclidean distance d between it and the eigenvector of each ellipse min , if d min is the current minimum distance, then assign the eigenvector of the ellipse to the third clustering set A j,k,s ;

[0115] Step 4: Merge all the third clustering sets A j,k,s to obtain the ellipse clustering set E C , and all the ellipses in this set are used as the second ellipses.

[0116] Step 6, use the sets of the first ellipses and the second ellipses as the ellipse detection results.

[0117] Experimental test:

[0118] The present invention conducts a series of comprehensive tests on synthetic and real image datasets to evaluate the performance of the proposed method and compares it with six publicly available methods. Five publicly available datasets downloaded from the Internet (including two natural datasets, one industrial dataset, and two synthetic datasets) and two datasets created during the industrial camera shooting process (one industrial dataset and one synthetic dataset) are used to evaluate the performance of the proposed ellipse detection method. The demonstrations of different algorithms on two natural datasets, two industrial datasets, and three synthetic datasets are as shown in Figure 5 , Figure 6 , Figure 7 . Table 1 shows the execution times of seven different algorithms on seven different types of datasets.

[0119] Table 1

[0120]

[0121] Liu's hierarchical method consumes a relatively long computation time. Prasad's detector searches for arcs that match all potential ellipses within the search area of the arc, resulting in a significant increase in the execution time. The methods of Fornaciari and Jia et al. have relatively short running times, but their quadrant division methods are not suitable for small arcs and semi-circular arcs. ELSD has difficulty in detecting complex scenes because its line segment detector is vulnerable to image noise during the grouping process. Lu's method achieves high detection accuracy but sacrifices execution time. In summary, this embodiment conducts extensive experiments, fine-tunes, and validates the method parameters to achieve the best performance. The thresholds of the present invention on the industrial camera shooting dataset are set to (T c = 20, T t = 10, T θ = 35). Compared with the six publicly available methods, the method of the present invention shows significant advantages in terms of detection efficiency and execution time, and successfully detects ellipses with varying degrees of incompleteness in the images captured by the industrial camera.

Claims

1. A machine vision ellipse detection method based on geometric constraints and hierarchical clustering, characterized in that: The specific steps include: Step 1: preprocess the image edge and perform edge detection weighting to obtain a weighted edge image; Step 2, screening edge contours that meet the candidate ellipse conditions to obtain a first ellipse and an ellipse arc segment set; Step 3, grouping the elliptical arc segments in the elliptical arc segment set obtained in step 2 to obtain a quadrant constrained arc group; Step 4, obtaining a candidate ellipse according to the quadrant constraint arc group obtained in step 3, and calculating the feature vector of the candidate ellipse, including the coordinates of the ellipse center, the major axis, the minor axis, and the rotation angle; Step 5, hierarchical clustering is performed on the candidate ellipses according to the feature vectors of the candidate ellipses obtained in step 4, to obtain a clustered ellipse set, and all ellipses in the set are used as the second ellipse; Step 6: Take the set of the first ellipse and the second ellipse as the ellipse detection result.

2. The machine vision ellipse detection method based on geometric constraints and hierarchical clustering as claimed in claim 1, characterized in that: Step 1 includes the following sub-steps: Step 11, performing denoising and grayscale preprocessing on the original image taken by the industrial camera to obtain a preprocessed image; Step 12, edge detection weighting, specifically: using the Canny edge detector with automatic threshold and the Sobel operator to obtain two edge images from the preprocessed image respectively, and then fusing the pixel value and gradient of each pixel point of the two edge images by mean weighting, so as to obtain the pixels of the weighted edge image.

3. The machine vision ellipse detection method based on geometric constraints and hierarchical clustering as claimed in claim 1, characterized in that: Step 2 specifically includes the following sub-steps: Step 21: Calculate the pixel point p on each edge line in the weighted edge image obtained in step 1. i The gradient symbol Τ(p i ), divide the contour lines composed of multiple continuous pixel points with positive gradient signs into the first quadrant or the third quadrant, and divide the contour lines composed of multiple continuous pixel points with negative gradient signs into the second quadrant or the fourth quadrant, and obtain two contour quadrant sets D(C T )∈{ArcI∪ArcIII}, D(C T )∈{ArcII∪ArcIV}; Step 22, retain the contour lines with a length greater than 12 pixels in the two contour quadrants, and delete the remaining contour lines; the retained contour line s Where (x i ,y i ) represents the coordinates of the i-th pixel point on the contour line, and n is the number of pixels on the contour line; Calculate the curvature of the pixel points on each contour line; Step 23, connecting all pixel points with similar curvature on each contour line to obtain a continuous curve, and obtaining continuous curves corresponding to all contour lines; and dividing all continuous curves into closed curves and non-closed curves according to the distance between the two end points of the curve, specifically: if the distance between the two ends is less than a preset threshold, it is regarded as a closed curve, otherwise it is regarded as a non-closed curve; Step 24, taking each closed curve obtained in step 23 as a potential ellipse for sector verification, and saving the potential ellipses corresponding to all the closed curves that pass the verification as the first ellipse; Step 25, using the improved KRDP algorithm with tolerance threshold to process each non-closed curve obtained in step 23, to obtain the elliptical arc segment corresponding to each non-closed curve; the elliptical arc segments corresponding to all non-closed curves constitute an elliptical arc segment set.

4. The machine vision ellipse detection method based on geometric constraints and hierarchical clustering as claimed in claim 3, characterized in that: In step 24, the specific process of verifying each closed curve by sector is as follows: Step 1, use a random sampling consistent method to fit the parameters of a potential ellipse that the closed curve may correspond to; Step 2, divide the potential ellipse into multiple sectors; Step 3, based on the parameters of the potential ellipse, calculate the theoretical gradient direction of the center point of each sector on the circumference of the potential ellipse; The actual gradient direction of the pixel point on the potential ellipse is compared with the theoretical gradient direction. If the actual gradient direction is aligned with the theoretical direction, the edge point and its corresponding sector are marked as valid; Step 4, according to the order of valid sectors, find a continuous sector set consisting of at least three adjacent valid sectors; Step 5, calculate the number of votes contributed by each found set of consecutive sectors; Step 6: Set a vote threshold based on the detection experience of different scenarios; if the vote count of the potential ellipse reaches the preset threshold, the potential ellipse is marked as successfully verified, and the pixels that contribute to the vote are removed from the edge data of the closed curve to avoid repeated calculations; The number of votes for the potential ellipse is equal to the sum of the number of votes for all continuous sector sets; if the preset threshold is not reached, the parameter model of the potential ellipse is adjusted, and then the RANSAC fitting is re-performed based on the parameter model of the new potential ellipse, that is, random sampling is performed again from the edge data of each closed curve obtained in step 23, and the new potential ellipse parameters are estimated according to the sampling results, and the process returns to step 2 until the predetermined termination condition is reached; Step 7, save all potential ellipses corresponding to the verified closed curves as the first ellipse.

5. The machine vision ellipse detection method based on geometric constraints and hierarchical clustering as claimed in claim 3, characterized in that: In step 25, each non-closed curve is processed using an improved KRDP algorithm with a tolerance threshold, including the following process: Step 1: Use line segments to separate each non-closed curve C s Perform fitting and save the fitted line segment as the non-closed curve C s The corresponding line segment set L; calculate the angle between each two line segments in the line segment set L in turn to obtain the angle set θ1, θ2, …, θ m-1 , where θ1 is the angle between line segment L1 and line segment L2, θ m-1 For line segment L m-1 With line segment L m The angle between them, m is the number of line segments in the line segment set L; Step 2: Initialize the point set P, which contains the non-closed curve C s The two endpoints and their curvature values; traverse the angle set, if |θ1-θ i |Exceeds the preset angle change threshold T θ , find line segment L i The corresponding original non-closed curve C s , and add the point and its curvature value to the point set P until all line segments in the line segment set L are traversed; traverse the point set P, compare the curvature changes of two adjacent points, and add the points whose curvature changes exceed the preset curvature threshold T k The point is retained as a segmentation point in the point set P, otherwise the point is deleted; if a non-closed curve corresponds to |θ1-θ i | are both less than the threshold T θ , and the curvature changes are all zero or less than the curvature threshold T k , then mark the non-closed curve as a straight line and filter it out; Step 3: Split each non-closed curve remaining after filtering in step 2 into arc segments.

6. The machine vision ellipse detection method based on geometric constraints and hierarchical clustering as claimed in claim 1, characterized in that: Step 3 includes the following sub-steps: Step 31, find all concave arcs and convex arcs in the elliptical arc segment set; specifically, if the curvatures of the two endpoints of an elliptical arc segment are both positive, it is a concave arc, and its concave-convex symbol Ω(C) is -; if the curvatures of the two endpoints are both negative, it is a convex arc, and its concave-convex symbol Ω(C) is +; Step 32, divide all concave arcs and convex arcs into quadrants. For example, consider a line D(C T )∈{ArcI∪ArcIII} If the curvature of the two endpoints is positive, then the arc segment is a concave arc, and its concave-convex symbol Ω(C) is -, then it is classified into the third quadrant (ArcIII), otherwise it is classified into the first quadrant (ArcI); Step 32: Select two arcs C in different quadrants from all the arc segments obtained in step 31. a ,C b Form an arc group; Step 33, based on the quadrants of the two arcs in the arc group and the coordinate values ​​of the arc endpoints, the arc group obtained in step 32 is screened to obtain a quadrant-constrained arc group.

7. The machine vision ellipse detection method based on geometric constraints and hierarchical clustering as claimed in claim 1, characterized in that: Step 4 includes the following sub-steps: Step 41, for each quadrant constraint arc group ε ab =(C a ,C b ), the arc C a and C b The midpoints of a and point M b ; For arc C a Draw tangents at the two endpoints and the midpoint, and the tangent at the midpoint intersects the tangents at the two endpoints at points P1 and P2 respectively; arc C b The corresponding intersection points on are points P3 and P4; calculate the xy coordinates of points P1, P2, P3, and P4 respectively; Specifically, the x and y coordinates of points P1 and P2 are calculated by the following formula, and the coordinates of P3 and P4 are calculated in the same way: in, Represent arc C a and arc C b The left and right endpoints of M a .x, M a .y、M a .τ is the arc C a The midpoint M a The x, y coordinates and gradient of Step 42, connect the two end points of each arc in the quadrant constraint arc group to the midpoint of the arc, and calculate the coordinates of the midpoint m1 and the midpoint m2 of the two connected line segments: Step 43, at arc C a Construct two perpendicular lines on the arc C, one of which is the straight line l1 passing through point P1 and point m1, and the other is the straight line l2 passing through point P2 and point m2; b Similarly, construct two vertical lines l3 and l4, and calculate arc C a The left and right endpoints To the midpoint M a The slope of: The slopes of the straight lines l1 and l2 are calculated using the following formula: Step 44, calculate a pair of arcs C in the quadrant constraint arc group a ,C b The coordinates of the center point; Specifically, arc C a The coordinates of the center point (C a .x,C a .y) is calculated as follows, and arc C is obtained in the same way b The center and quadrant constrained arc group is a pair of arcs C a ,C b the center of Step 45: If the distance between the center points of a pair of arcs in the quadrant constraint arc group exceeds a preset threshold value T c , then delete the two arcs of the quadrant constraint arc group; retain the center coordinates of the remaining quadrant constraint arc groups as the center coordinates of the candidate ellipse corresponding to the quadrant constraint arc group; in addition, the coordinates of the intersection points where the four straight lines l1, l2, l3, and l4 intersect each other, and the average value of all the intersection coordinates, are also regarded as the center of the candidate ellipse; All the candidate ellipse centers calculated above constitute the candidate ellipse center set O corresponding to the current quadrant constraint arc group. c ; Step 46, calculate the parameter K and the rotation angle of the candidate ellipse Among them, α=q1q2-q3q4, β=q2q4(q3-q1)+q1q3(q4-q2)+(q1+q2-q3-q4); Step 47, calculate the operator N: Step 48, set the ellipse center set O c The center coordinates of each candidate ellipse are given by the coordinates in the world coordinate system (x i ,y i ) is converted to the coordinates (x) in the elliptical coordinate system e ,y e ), the formula is as follows: Among them, (x c ,y c ) is the set of ellipse centers Ο c A candidate ellipse center in ; according to The following equation can be obtained: Calculate the major semiaxis in Calculate the minor semi-axis B = A·N; Through the above calculation, we get a pair of arcs C a ,C b Corresponding to a candidate ellipse with five parameters Step 49, traverse the candidate ellipse center set O obtained in step 45 c , get the feature vectors of all candidate ellipses corresponding to a quadrant constraint arc group (x i ,y i ) represents the coordinates of a candidate ellipse center, (a i ,b i ) represents the major axis and minor axis, is the rotation angle of the candidate ellipse.

8. The machine vision ellipse detection method based on geometric constraints and hierarchical clustering as claimed in claim 1, characterized in that: Step 5 specifically includes the following sub-steps: Step 1: Perform the first level of clustering based on the ellipse center coordinates: All candidate ellipse feature vectors V i Assign to an initial ellipse set E I , use the Gaussian mean shift method to adjust the initial ellipse set E I Cluster the ellipse center coordinates in N C The cluster centers of the ellipse circles are (x,y)1, (x,y)2, ..., (x,y) j , ..., (x, y) Nc ; For each cluster center point (x,y) j , calculate its difference with the ellipse eigenvector V i The Euclidean distance d between the coordinates of the center of each ellipse min , if d min is the current minimum distance, then the current feature vector is assigned to the first clustering set A j ; Step 2: Perform the second level of clustering based on the semi-axis length: For each first clustering set A j , clustering is performed based on the semi-axis length of the ellipse, and N C,a The cluster centers with semi-axis lengths (a,b)1, (a,b)2... (a,b) Nc,a , for each semi-axis length cluster center point (a, b) k , calculate the Euclidean distance d between the major and minor axes of each ellipse eigenvector min ; if d min is the current minimum distance, then the ellipse feature vector is assigned to the second cluster set A j,k ; Step 3: Perform the third layer of clustering based on the ellipse direction angle: For the second clustering set A j,k , clustering is performed based on the ellipse rotation angle, and the Cluster center points with rotation angles For each direction cluster center Calculate the Euclidean distance d from each ellipse eigenvector min , if d min is the current minimum distance, then the ellipse feature vector is assigned to the third cluster set A j,k,s ; Step 4: Set all third clusters into A j,k,s Merge to get the ellipse cluster set E C , all the ellipses in this set are taken as the second ellipse.

Images