A sub-pixel ellipse extraction method

Abstract

The present invention discloses a sub-pixel ellipse extraction method, the main process of which includes the following main steps: ellipse tangent extraction, tangent weight allocation and ellipse parameter estimation based on least squares. In essence, it is an ellipse parameter estimation method based on a dual quadratic curve theoretical model. In the solution of the ellipse parameters, constraints such as threshold segmentation, weight allocation, line normalization, and parameter normalization are added. Compared with simple point constraints, the constraint method of this scheme is easier to constrain the ellipse parameters to the real parameter value range, which greatly improves the accuracy of ellipse parameter estimation. There is no edge extraction link in the entire algorithm process, which effectively avoids the error introduced by edge extraction.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a sub-pixel ellipse extraction method. Background Art

[0002] Circular features are widely used in machine vision, including camera calibration based on circular hole targets, robot automatic guidance based on circular features, object spatial position and posture measurement, visual navigation, and target object recognition. A circle is a special type of ellipse, and its perspective projection often appears as an ellipse. Therefore, studying methods for extracting elliptical features from images is of great significance.

[0003] Detection accuracy and computational performance are two important issues in ellipse image feature extraction. Numerous scholars have conducted research around these two main issues and proposed a large number of ellipse feature extraction methods. Currently, the commonly used ellipse extraction and positioning methods include the centroid method, the Hough transform method, and methods based on boundary point fitting. Among them, the invariant moment method and the least squares method are the most common algorithms for fitting based on ellipse boundary points. However, although the moment method can obtain the ellipse center at the sub-pixel level, it is easily affected by image grayscale noise. The least squares ellipse fitting algorithm based on ellipse boundary points requires that the ellipse boundary must be extracted before ellipse fitting is performed. This not only increases the complexity of the ellipse fitting algorithm, but is also extremely susceptible to boundary noise, greatly reducing the accuracy of ellipse fitting.

[0004] In practical applications of machine vision, the key to the system's detection accuracy lies in the precise extraction of the pixel coordinates of the ellipse center. This is because the system's detection function is usually achieved through the pixel coordinates of multiple ellipse centers on one or more images, while other ellipse features are relatively unimportant. This solution proposes an ellipse parameter estimation method based on a dual quadratic curve theoretical model. There is no edge extraction step in the entire algorithm process, effectively avoiding the errors introduced by edge extraction. This is mainly because edge extraction is a complex process involving multiple steps such as gradient estimation, non-maximum suppression, and threshold selection. There are many uncertainties in these steps, which seriously affect the accuracy of the final parameter estimation. In this method, the point ellipse parameter estimation method is converted into a line ellipse parameter estimation method in the dual space, and multiple parameter constraints are used during parameter estimation to ensure the accuracy of ellipse parameter estimation. Summary of the Invention

[0005] The purpose of the present invention is to provide a sub-pixel ellipse extraction method to solve the problem raised in the above background art that the existing ellipse extraction method has a complex edge extraction process, thereby greatly reducing the final parameter estimation accuracy.

[0006] To achieve the above object, the present invention provides the following technical solution: a sub-pixel ellipse extraction method, characterized in that it includes the following steps:

[0007] Step 1: Ellipse tangent extraction, convert the original input image into a gradient image, and derive its gradient based on the ellipse edge area in the gradient image

[0008]

[0009] The corresponding ellipse tangent is

[0010]

[0011] In the above two formulas, g xi ,g yi are the gradient amplitudes in the x-axis and y-axis directions of the image respectively;

[0012] Step 2: Tangent weight distribution. The ellipse parameters are solved by constructing the ellipse tangent through the gradient. When the pixel point p in the image i The gradient at When there is, there is a straight line The direction of the line points to pixel p i The gradient direction of the ellipse is calculated, and the focus of the ellipse is on this line. When added to the corresponding ellipse tangent, the tangent with a large gradient modulus is more likely to be on the ellipse boundary. Moreover, the ellipse boundary with a large gradient modulus is much more important for ellipse parameter estimation than the ellipse boundary with a small gradient modulus. Therefore, the modulus of the gradient value of each pixel point is used as a weight to enhance noise resistance and correlation accuracy.

[0013] Step 3: Ellipse parameter estimation based on least squares; given a set of straight lines l i , parameter vector Θ={A * ,B * ,C * ,D * ,E * ,F *} is the ellipse parameter matrix C * The elements in , the parameter vector can be obtained by least squares estimation, including the following steps:

[0014] a) The principle is to minimize Φ(Θ) to obtain the optimal Θ parameter estimate:

[0015]

[0016] Among them, R represents the set of ellipse tangents, i represents a tangent line in the tangent set, ω i represents the weight of the i-th tangent,

[0017] b) For the uncertainty of the linear scale, for the lineari =[a i ,b i ,c i ] T , so that ‖a i ,b i ‖=1, after normalizing the straight line, the above formula becomes:

[0018]

[0019] Among them, K i is composed of line parameters, and

[0020] c) Under the constraint condition ‖Θ‖=1, the required parameters are directly obtained through the SVD decomposition method.

[0021] Preferably, in step 1, the gradient amplitude g is selected xi ,g yi The area that is 2 to 3 times larger than its average value is used as the candidate area for ellipse parameter estimation.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] (1) Based on the dual quadratic curve theoretical model, in the solution of ellipse parameters, constraints such as threshold segmentation, weight distribution, line normalization, and parameter normalization are added. Compared with simple point constraints, the constraint method of this scheme is easier to constrain the ellipse parameters to the real parameter value range, which greatly improves the accuracy of ellipse parameter estimation;

[0024] (2) When estimating ellipse parameters, there is no need to extract ellipse contour data in advance, which greatly improves the efficiency of ellipse parameter estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flow chart of the method of the present invention;

[0026] Figure 2 The result of extracting the simulated ellipse parameters in the embodiment of the present invention;

[0027] Figure 3 for Figure 2 The local effect diagram;

[0028] Figure 4 is the calibration plate image;

[0029] Figure 5 Extract the results of calibration plate parameters;

[0030] Figure 6 3D reconstruction of feature points of the calibration plate;

[0031] Figure 7The distance from the feature point to the plane is the Z-direction accuracy of the feature point;

[0032] Figure 8 It is the Z-direction distance statistical histogram of the feature point Z-direction accuracy. DETAILED DESCRIPTION

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0034] See also Figure 1 The present invention provides a technical solution: a sub-pixel ellipse extraction method. The main process includes: ellipse tangent extraction, tangent weight assignment, and ellipse parameter estimation based on least squares. Essentially, this is an ellipse parameter estimation method based on a dual quadratic curve theoretical model. The entire algorithm process does not include edge extraction, effectively avoiding the errors introduced by edge extraction. Specifically, the steps include:

[0035] 1. Ellipse tangent extraction

[0036] Traditional ellipse parameter estimation methods often extract the boundary of the ellipse first, and then use numerical methods to fit the extracted boundary to the required ellipse parameters. The reason for the existence of the edge extraction link is that the definition of the ellipse feature adopts the form of a point quadratic curve.

[0037]

[0038] A little bit on the p i =[x i ,y i ,1] T ,exist

[0039]

[0040] Where x i 、y i For p i The x-axis and y-axis coordinates of the ellipse; A, B, C, D, E, F are the coefficients of formula (2). According to the dual relationship between points and lines, the ellipse can also be represented by tangents in its dual space. For every straight line l on the ellipse i =[a,b i ,c i ] T , where a, b i ,c i is the tangent line l iThe homogeneous coordinate value of

[0041]

[0042] In the formula

[0043]

[0044] is the inverse matrix of C. The corresponding A * ,B * ,C * ,D * ,E * ,F * are the coefficients in Equation (3). Theoretically, as long as the five ellipse tangents at general locations are known, all ellipse parameters can be uniquely determined. However, in real images, many interference factors are common, and it is necessary to develop a more robust ellipse parameter estimation method based on the dual quadratic curve theory.

[0045] According to the dual quadratic curve theory, to determine the parameters of an ellipse, we must first obtain the tangent of the ellipse. The original input image is converted into a gradient image, and the gradient of the ellipse edge area in the gradient image is obtained. where g xi ,g yi are the gradient amplitudes in the x-axis and y-axis directions of the image, and their corresponding ellipse tangents are

[0046]

[0047] At the same time, in order to avoid interference from human factors and camera noise, this scheme adopts a more direct threshold segmentation method, that is, taking the area where the gradient amplitude is 2 to 3 times greater than its average value as the candidate area for ellipse parameter estimation.

[0048] 2. Tangent weight distribution

[0049] The image gradient at the ellipse boundary can not only distinguish the image, but also represent the normal of the image edge. This patent constructs the ellipse tangent by gradient to solve the ellipse parameters. And, when the pixel point p in the image i The gradient at When there is, there is a straight line The direction of the line points to pixel p i The gradient direction of the ellipse is centered on this line.

[0050] In order to improve the accuracy and robustness of the patented method, the modulus of the gradient value of each pixel point is used as a weight and added to the corresponding ellipse tangent. Generally speaking, the tangent with a large gradient modulus is more likely to be located on the ellipse boundary; and the ellipse boundary with a large gradient modulus is much more important to the ellipse parameter estimation than the ellipse boundary with a small gradient modulus; therefore, this paper adds the gradient modulus as a weight value to the ellipse parameter estimation problem, which can greatly enhance the noise resistance and related accuracy of the patented algorithm.

[0051] 3. Ellipse parameter estimation based on least squares

[0052] Given a set of straight lines l i , parameter vector Θ={A * ,B * ,C * ,D * ,E * ,F *} is the ellipse parameter matrix C * The elements in , the parameter vector can be obtained by least squares estimation, the principle of which is to minimize Φ(Θ) to obtain the optimal Θ parameter estimate.

[0053]

[0054] Among them, R represents the set of ellipse tangents, i represents a tangent line in the tangent set, ω i Indicates the weight of the i-th tangent. In view of the uncertainty of the linear scale, this patent i =[a i ,b i ,c i ] T , so that ‖a i ,b i ‖=1, after normalizing the straight line, the above formula becomes:

[0055]

[0056] Among them, K i is composed of line parameters, and Under the constraint condition ‖Θ‖=1, the required parameters can be directly obtained by SVD decomposition method.

[0057] Simulation experiment verification

[0058] In order to verify the accuracy and robustness of the algorithm proposed in this patent, a camera calibration platform was built to verify the algorithm of this patent. The hardware of the system includes: 1 Hikvision MV-CE200-11GM industrial camera and a circular calibration plate. Among them, the camera pixel size is 2.4μm×2.4μm; the resolution is 2688pixel×1520pixel. The lens size is 4.8mm. Equipped with a megapixel industrial camera lens with a focal length of 25mm and a blue ring light source with an outer diameter of 120mm. There are 99 dots in 11 rows and 9 columns with different sizes on the calibration plate. The distance between dots is 4.2mm, such as Figure 3 shown.

[0059] This patent sets up two groups of experiments to analyze the algorithm: Experiment 1: By extracting the parameters of the simulated ellipse, the accuracy and robustness of the algorithm of this patent are evaluated; Experiment 2: Use the above platform equipment to capture images of the same ellipse calibration plate at different positions, and extract all the ellipse parameters in the calibration plate, calibrate the camera using Zhang Zhengyou calibration method, and then reconstruct the position of the ellipse pixel center in the calibration plate in the real-world coordinate system, and then analyze the accuracy and robustness of this algorithm.

[0060] The experimental steps are as follows:

[0061] 1) Ellipse fitting and calibration experiment

[0062] Before the calibration experiment, an ellipse fitting experiment was conducted. First, the ellipse parameters of the simulated image were extracted and compared with the traditional Sobel operator. The simulated image was drawn with Photoshop software on four large circles on the calibration plate. Considering the smoothing effect of the actual CMOS sensor and optical system on the image, the generated image was smoothed and then the ellipse parameters were estimated to verify the accuracy of the algorithm. The simulation experiment image and the results of the ellipse parameter estimation are shown in Figure 2. Figure 2 Figure a shows the parameter extraction effect of the simulated ellipse, and figure b shows the local magnification effect of the ellipse in the upper left corner of figure a.

[0063] To quantitatively verify the accuracy of the present invention, the results of ellipse center extraction were compared with the true values. The comparison results are shown in Table 1. As can be seen from the table, the average X-coordinate error of the four ellipse centers is 0.028 pixels, and the average Y-coordinate error is 0.027 pixels. Therefore, the accuracy of the ellipse parameter extraction method proposed in this patent can reach below 0.03 pixels. Compared with the extraction results of the Sobel operator, this patent greatly improves the extraction accuracy of ellipse parameters.

[0064] Table 1 Simulation image center extraction results

[0065]

[0066] In the actual extraction experiment, the captured calibration plate image is processed to obtain Figure 4 The results are shown in the figure. It is not difficult to see that the major and minor axes, tilt directions, etc. of the extracted circles and ellipses are all correct. Through simulation and actual experiments, the accuracy and stability of the ellipse parameter estimation algorithm proposed in the invention are verified.

[0067] 2) 3D reconstruction and accuracy analysis

[0068] In order to further verify the accuracy and stability of the ellipse parameter estimation method proposed in this invention, the calibration plate images taken during the calibration process were 3D reconstructed and compared with the standard values ​​given by the calibration plate manufacturer. The 3D reconstructed point cloud of the calibration plate at a certain position is shown in Figure 2. Figure 5 As shown in the figure, the accuracy of the 3D reconstructed point cloud is analyzed, and the position accuracy (X, Y direction) and Z direction accuracy of the reconstructed point cloud are analyzed respectively. Figure 4 For each of the ten ellipse center points near the mid-diagonal line, the average distance from each point to the two nearest center points is calculated. Points 1 and 10 are located at the upper left and lower right corners of the calibration plate, respectively. The remaining points are evenly distributed along the diagonal line. The resulting point cloud position accuracy is shown in Table 2. The calculated distance values ​​in the table indicate that the reconstructed point cloud error in the edge regions is greater than that in the center region, with a maximum error of 2.1 μm. Using the maximum error of the reconstructed 3D points on the calibration plate as the evaluation indicator for calibration accuracy, the system achieves a position accuracy of 2.1 μm.

[0069] Table 2 Position accuracy of three-dimensional feature points of the calibration plate

[0070]

[0071] In order to intuitively evaluate the reconstruction accuracy of the feature points in the Z direction, the reconstructed feature points of the calibration plate are fitted into a plane, the distance from each feature point to the plane is calculated, and the number of points at different distances is counted to generate a histogram, as shown in Figure 6 As shown, it can be seen that the distance from the point to the plane is between -1μm and 1μm, and most of the distances from the point to the plane are concentrated in the range of -0.4μm to 0.4μm. The maximum error of the three-dimensional points of the reconstructed calibration plate is used as the evaluation of the calibration accuracy. The Z-direction calibration accuracy of the system can reach 1μm.

[0072] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A sub-pixel ellipse extraction method, characterized in that: The following steps are involved: Step 1: Ellipse tangent extraction, convert the original input image into a gradient image, and derive its gradient based on the ellipse edge area in the gradient image The corresponding ellipse tangent is In the above two formulas, g xi ,g yi are the gradient amplitudes in the x-axis and y-axis directions of the image respectively; Step 2: Tangent weight distribution. The ellipse tangent is constructed by gradient to solve the ellipse parameters. When the pixel point p in the image i The gradient at When there is, there is a straight line The direction of the line points to pixel p i The gradient direction of the ellipse is calculated, and the focus of the ellipse is on this line. When added to the corresponding ellipse tangent, the tangent with a large gradient modulus is more likely to be on the ellipse boundary. Moreover, the ellipse boundary with a large gradient modulus is much more important for ellipse parameter estimation than the ellipse boundary with a small gradient modulus. Therefore, the modulus of the gradient value of each pixel point is used as a weight to enhance noise resistance and correlation accuracy. Step 3: Ellipse parameter estimation based on least squares; given a set of straight lines l i , parameter vector Θ={A * ,B * ,C * ,D * ,E * ,F * } is the ellipse parameter matrix C * The elements in , the parameter vector can be obtained by least squares estimation, including the following steps: a) The principle is to minimize Φ(Θ) to obtain the optimal Θ parameter estimate: Among them, R represents the set of ellipse tangents, i represents a tangent line in the tangent set, ω i represents the weight of the i-th tangent, b) For the uncertainty of the linear scale, for the linear i =[a i ,b i ,c i ] T , so that ‖a i ,b i ‖=1, after normalizing the straight line, the above formula becomes: Among them, K i is composed of line parameters, and c) Under the constraint condition ‖Θ‖=1, the required parameters are directly obtained through the SVD decomposition method.

2. A sub-pixel ellipse extraction method according to claim 1, characterized in that: In step 1, select the gradient amplitude g xi ,g yi The area that is 2 to 3 times larger than its average value is used as the candidate area for ellipse parameter estimation.

Images