High-precision ellipse fitting method based on multi-scale smoothing and key point searching
By optimizing the selection of fitting points through multi-scale smoothing and key point search, the problem of ellipse fitting being easily affected by outliers is solved, achieving high-precision and robust ellipse fitting, which is suitable for ellipse detection in complex scenarios.
Patent Information
- Application Number
- CN202211650633.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-21
- Publication Date
- 2025-12-09
- Estimated Expiration
- 2042-12-21
AI Technical Summary
Existing ellipse fitting methods are susceptible to outliers, resulting in low fitting accuracy. Furthermore, artificial intelligence methods require large datasets for training, and their fitting accuracy is also affected by insufficient training samples and variable objects.
We employ a multi-scale smoothing and keypoint search approach. We remove outliers using multi-scale operators, optimize the selection of fitting points using a keypoint search algorithm, and improve fitting accuracy by combining the smoothing algorithm with the method.
It improves the accuracy and robustness of ellipse fitting, enabling more accurate fitting of ellipse edges and making it suitable for ellipse detection in complex scenarios.
Smart Images
Figure CN115830098B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application relates to a high-precision ellipse fitting method based on multi-scale smoothing and key point searching, and belongs to the technical field of computer vision and digital image processing, and mainly relates to feature parameter identification and contour analysis technology. BACKGROUND
[0002] Ellipse fitting is a key problem in the field of computer vision. As one of the most common conic curves, all objects with quadratic curve structures such as circles, arcs and ellipses in life can be accurately represented as an ellipse or a combination of multiple ellipses in an image. In short, ellipse fitting can be widely applied to face recognition, medical image segmentation, unmanned driving and other fields. Ellipse fitting is the key to computer vision and target detection to adapt to various use scenarios and to fit an ellipse from an image in real time and with high precision. Taking the unmanned driving technology as an example, since the recognition image is a complex actual street or natural scene, the existing non-artificial intelligence ellipse fitting method is susceptible to outliers, resulting in low fitting precision, and even a wrong ellipse may be fitted; the existing artificial intelligence ellipse fitting method needs to establish a large number of data sets for the target to be detected for recognition training to ensure accuracy, and is affected by insufficient training samples and indefinite objects involved, resulting in low fitting precision. The method designs a multi-scale operator to reduce the input image to remove isolated or discontinuous pixels, then searches for key points through a key point searching algorithm, and finally smoothes the key points for ellipse fitting, so that the fitting precision is improved. SUMMARY
[0003] The application proposes a high-precision ellipse fitting method based on multi-scale smoothing and key point searching for the bottleneck of the ellipse fitting method in the non-artificial intelligence field. At present, ellipse fitting methods mainly include algebra-based ellipse fitting methods, geometry-based ellipse fitting methods and robustness-based ellipse fitting methods. Since the algebra-based ellipse fitting method and the geometry-based ellipse fitting method are susceptible to outliers, the fitting precision is affected, and the purpose of the application is to improve the fitting precision by optimizing the selection of fitting points.
[0004] The specific steps of the technical solution provided by the application are as follows:
[0005] Step S1: An image to be fitted with an ellipse is obtained, and the image to be fitted with an ellipse is obtained by using an image storage module or an image sensor of a device in an application scenario. The camera image can be compatible with grayscale images and RGB, HSV, CMYK and other multi-color images.
[0006] Step S2: image processing, using Canny edge detection algorithm to extract the ellipse arc segment of the input image to be fitted, then segmenting each arc according to the concave-convex property of the ellipse arc segment to obtain an ellipse arc, and finally using the ellipse combination algorithm provided in the edge connection method-based ellipse detection algorithm AAMED to combine the segmented ellipse arcs to obtain an ellipse arc combination image.
[0007] Step S3: multi-scale feature pixel labeling, taking k*k pixels as a k-order cell, mapping the k-order cell in the ellipse arc combination image obtained in step S2 into a pixel point in the k-order scale feature pixel image, and reducing the size of the ellipse arc combination image from m*n to k*k k-order scale feature pixel images are obtained. If the multi-scale feature value of the k-order cell is greater than a given threshold θ, the pixel point corresponding to the k-order cell in the k-order scale feature pixel image is marked as an edge point; otherwise, it is marked as a non-edge point.
[0008] The multi-scale feature value of the k-order cell is defined as the number of edge points in the k-order cell.
[0009] Step S4: smoothing the k-order scale feature pixel image, using a smoothing operator to perform Zigzag scanning calculation on the k-order scale feature pixel image, and using an 8-neighborhood operator The k-order scale feature pixel image is smoothed, and isolated points in the k-order scale feature pixel image are deleted.
[0010] The definition of the isolated point is that the isolated point is an edge point with a degree of 0, i.e., there is no edge point in the 8-neighborhood of the point.
[0011] Step S5: k-order scale feature pixel image division, the pixel point position of the k-order scale feature pixel image is represented as P(i,j), i represents the column number of the pixel in the image, j represents the row number of the pixel in the image, i∈(a,b), j∈(c,d), and the four extreme points of the k-order scale feature pixel image are taken as division boundary points. Connecting the upper and lower extreme points and the left and right extreme points divides the k-order scale feature pixel image into four regions, denoted as L-U region, U-R region, R-B region, and B-L region.
[0012] Step S6: searching for key points in the four regions based on the spatial characteristics of the feature pixel image, and searching for key points in the k-order cell corresponding to the edge point in the four regions in the feature pixel image in the corresponding clockwise direction.
[0013] Taking the L-U region as an example: first, starting from the lower left corner of the k-order cell, search from bottom to top and from left to right in the clockwise direction, if an edge point is found at P(i, j), the search range is reduced, and the edge point is searched from the point P(i-1, j+1) until the k-order cell boundary is found. The edge point searched by the flow is the key point.
[0014] Step S7: extracting feature pixel image fitting points, smoothing the k-order cell, dividing each k-order cell into four subgrids, and then taking the barycentric coordinates of the key points in each subgrid as the final fitting points.
[0015] Step S8: fitting an ellipse, fitting the feature pixel image fitting points extracted in step S7 using the least square method, and finally obtaining the parameter representation of the ellipse.
[0016] Step S9: feature pixel image output, based on different input image data in different use cases, coloring the input image using the feature pixel image, and outputting the fitting result.
[0017] Advantages of the present application:
[0018] The core technical feature of the present application is to optimize the selection of fitting points to achieve high fitting accuracy and high robustness. The present application combines a multi-scale operator and a key point search algorithm, uses a multi-scale operator to reduce the original image to remove most outliers, searches for key points through a key point search algorithm, and then smooths the key points to improve the fitting accuracy of the algorithm. Experiments show that, compared with other traditional ellipse fitting methods, the method has more accurate and powerful performance, and at the same time, the method can also be used as a preprocessing method for other fitting algorithms to improve their fitting accuracy. BRIEF DESCRIPTION OF DRAWINGS
[0019] Figure 1 Flowchart of the ellipse fitting method based on multi-scale smoothing and key point search;
[0020] Figure 2 Image processing of image data collected by an unmanned vehicle
[0021] Figure 3 Circular arc segment type diagram
[0022] Figure 4 Feature pixel marking of image data collected by an unmanned vehicle
[0023] Figure 5 Isolated point schematic diagram
[0024] Figure 6 Feature pixel image division principle schematic diagram
[0025] Figure 7A flowchart for L-U region key point search.
[0026] Figure 8 A key point search result for unmanned vehicle image data collection.
[0027] Figure 9 A fitting result map for unmanned vehicle image data collection. DETAILED DESCRIPTION
[0028] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to embodiments. The application background of the embodiments of the present application is ellipse detection for unmanned vehicle image collection. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.
[0029] Reference Figure 1 , which shows a flowchart of an embodiment of an ellipse fitting method based on multi-scale smoothing and key point search, and is described in detail as follows:
[0030] In step S1, a road image in a vehicle driving scene is obtained, which can be obtained by an image storage module or an image sensor of the vehicle in the application scenario. The camera image can be compatible with grayscale images and RGB, HSV, CMYK and other multi-color images.
[0031] In step S2, the obtained road image is processed to extract the arc segments of the input image. Referring to Figure 2 , by performing edge detection, arc segmentation and ellipse combination operations on the input image, the grouped ellipse arc segment group is finally obtained.
[0032] In step S21, the Canny edge detection algorithm is used to extract the arc segments of the input image.
[0033] In step S22, starting from a point in the edge image, each arc is traversed clockwise, only adjacent inclined point pairs are retained to form an edge point sequence, the coordinates of each edge point are recorded and the direction information thereof is calculated.
[0034] The definition of the inclined point pair is that two points p i (x i ,y i ), p i+1 (x i+1 ,y i+1 ) in an inclined point pair satisfy x i ! = x i+1 and y i ! = y i+1 .
[0035] In step S23, according to the coordinate transformation rule of the inclined point pair, any elliptic arc segment can be divided into four categories: lower convex, upper convex, lower concave and upper concave (for a longer elliptic arc segment, it can contain more than two kinds of elliptic arc categories). Let p i (x i ,y i ), p i+1 (x i+1 ,y i+1 ) be two adjacent edge points, and Δx = x i+1 -x i , Δy = y i+1 -y i , then the coordinate transformation of point p i is defined as:
[0036]
[0037] As Figure 3 known, if p i belongs to the upper convex arc segment, then p i+1 must belong to the upper convex arc segment or the lower convex arc segment, otherwise p i is a split point.
[0038] In step S24, according to step S23, each arc segment in the edge map is segmented to obtain an elliptic arc segment.
[0039] In step S25, the segmented elliptic arc is combined according to the elliptic combination algorithm provided in the edge connection method-based elliptic detection algorithm AAMED.
[0040] In step S3, the multi-scale feature pixel of the elliptic arc segment of the road image is marked. k*k pixels are taken as a k-order cell. The k-order cell in the elliptic arc combination image obtained in step S2 is mapped into a point in the k-order scale feature pixel image, and the image with a size of m*n is reduced to k*k k-order scale feature pixel images are obtained. If the multi-scale feature value of the k-order cell is greater than a given threshold θ, the pixel point corresponding to the k-order cell in the k-order scale feature pixel image is marked as an edge point; otherwise, it is marked as a non-edge point. Referring to Figure 4 part (a) of FIG. 6 is a 6-order feature pixel image after reduction by k = 6. Each feature pixel point in the image corresponds to a k*k grid in the original image.
[0041] In step S4, the feature pixel image is smoothed, the k-order scale feature pixel image is calculated by Zigzag scanning using a smoothing operator, and isolated points in the small image are deleted. That is, an 8-neighborhood operator is used to calculate the 3*3 grid centered on each edge point g0 in the small image Smooth if g0 is isolated point, refer to Figure 5 is isolated point principle diagram. Refer to Figure 4 (b) part of the feature pixel smoothing, all isolated points in the feature pixel map are deleted.
[0042] In step S5, the feature pixel image is divided, as Figure 6 shown, by finding four extreme points of the elliptical arc segment, so that the elliptical arc segment can be divided into four regions. A two-dimensional Cartesian coordinate system is established, and the pixel point position of the feature pixel image is represented as P(i,j), i represents the column number of the pixel in the image, j represents the row number of the pixel in the image, i∈(a,b), j∈(c,d), and the four extreme points of the upper extreme point P(i,d), the lower extreme point P(i,c), the left extreme point P(a,j), and the right extreme point P(b,j) in the k-order scale feature pixel image are used as the division boundary points. Connecting the upper and lower extreme points and the left and right extreme points divides the k-order scale feature pixel image into four regions, denoted as L-U region, U-R region, R-B region, and B-L region.
[0043] In step S6, the feature pixel image key point search based on spatial characteristics, refer to Figure 7 shows a flowchart of the feature pixel image key point search based on spatial characteristics, and the details are as follows. For the edge points in the four regions divided in step 5, the corresponding k-order unit cells in the feature pixel image are searched in the corresponding direction. Taking the L-U region as an example, first, from the lower left corner of the k-order unit cell, search from bottom to top and from left to right. If an edge point is found at P(i,j), reduce the search range and continue to search for the edge point from point P(i-1,j+1), until the k-order unit cell boundary is found. The edge points searched by the key point search process are the key points, as Figure 8 shown, the key points searched by the key point search process can better fit the elliptical arc segment, thereby greatly improving the fitting accuracy of the method.
[0044] In step S7, the feature pixel image fitting point extraction, the k-order unit cell is smoothed, and each k-order unit cell is divided into four subgrids. Then, the key points in each subgrid are taken as the center coordinates of the points used for fitting.
[0045] In step S8, the feature pixel image fitting point fitting ellipse, the feature pixel image fitting points extracted in step S7 are fitted using the least squares method, and finally the objects of all elliptical contours in the road map are obtained.
[0046] In step S9, the feature pixel image is output. Based on the different input image data under different usage scenarios, the feature pixel image transformation module is used to perform the final fitting. (Reference) Figure 9 Color-mark the ellipses in the road image.
[0047] The following table, which combines the high fitting accuracy and high robustness of the ellipse fitting of the present invention with a test comparison table, provides a detailed description of the application effects of the present invention.
[0048] Table 1: Comparison of MSE values for different ellipse fitting methods
[0049] OURS DLSF RANSAC Wu Taubin ellipse1 1.0825 1.5993 1.2010 1.7115 1.7134 ellipse2 0.0102 0.0111 0.4017 1.0643 1.0582 ellipse3 1.0431 1.0589 9.2869 1.8416 1.8881 ellipse4 0.0179 0.5770 0.5347 1.1256 1.0579
[0050] In Table 1, for Figure 9 The four ellipses shown are ellipses 1 and ellipse 3, which are elliptical arc segments with missing edges, and ellipses 2 and ellipses 4, which are relatively complete elliptical edges. The method proposed in this invention is compared with DLSF (Direct leastsquares fitting), RANSAC (Random Sample Consensus Algorithm), the geometry-based ellipse fitting algorithm proposed by Wu et al., and the robustness-based ellipse fitting algorithm proposed by Taubin et al. MSE (Mean Squared Error) is used for evaluation; a smaller MSE value indicates higher fitting accuracy.
[0051] The test results show that in the application of ellipse detection in autonomous driving image acquisition, compared with other algorithms, the fitting accuracy of the method proposed in this invention can be improved by an average of 72.6%, with a maximum improvement of 80.7%. The MSE of ellipses ellipse1 and ellipse3 is higher than that of ellipse2 and ellipse4, but the MSE of this method is still lower than that of other algorithms. This indicates that the ellipse fitted by this method better matches the actual ellipse edge, and the method has a lower MSE, thus exhibiting higher ellipse fitting accuracy and better robustness. In summary, this method, after testing, shows improvements in both the accuracy and robustness of ellipse fitting. The above description is only a preferred embodiment of the present invention and is not intended to limit the invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A high-precision ellipse fitting method based on multi-scale smoothing and key point search, characterized in that The method comprises the following steps: Step S1: obtaining an image to be fitted with an ellipse, wherein the image to be fitted with an ellipse is obtained by using an image storage module or an image sensor of a device in an application scenario; Step S2: image processing, wherein an elliptic arc segment of the input image to be fitted with an ellipse is extracted by using a Canny edge detection algorithm, then each arc segment is segmented according to the concave-convex property of the elliptic arc segment to obtain an elliptic arc, and finally an elliptic arc combination algorithm provided in an ellipse detection algorithm AAMED based on an edge connection method is used to combine the segmented elliptic arcs to obtain an elliptic arc combination image; Step S3: Multi-scale feature pixel marking, taking k*k pixels as a k-order cell, mapping the k-order cell in the elliptical arc combination image obtained in step S2 into a pixel point in the k-order scale feature pixel image, and reducing the size of the elliptical arc combination image from m*n to If the multi-scale feature value of the k-order cell is greater than a given threshold θ, the corresponding pixel point in the k-order scale feature pixel image is marked as an edge point. Otherwise, the point is marked as a non-edge point. The multiscale feature value of the k-order cell is defined as follows: the multiscale feature value of the k-order cell is defined as the number of edge points in the k-order cell. Step S4: smoothing the k-order scale feature pixel image, using a smoothing operator to calculate the Zigzag scanning of the k-order scale feature pixel image, using an 8-neighborhood operator smoothing the k-order scale feature pixel image, deleting isolated points in the k-order scale feature pixel image; Step S5: k-order scale feature pixel image division, wherein the pixel position of the k-order scale feature pixel image is represented as P(i,j), i represents the column number of the pixel in the image, j represents the row number of the pixel in the image, i∈(a,b), j∈(c,d), four extreme points of the k-order scale feature pixel image are taken as division boundary points, the upper and lower extreme points and the left and right extreme points are connected to divide the k-order scale feature pixel image into four regions, which are denoted as an L-U region, a U-R region, a R-B region and a B-L region; Step S6: based on the spatial characteristics, key points are searched in the four regions respectively, and the edge points in the four regions are searched for key points in the k-order cell corresponding to the feature pixel image in the corresponding clockwise direction; The key point search of the L-U region is implemented as follows: Firstly, the edge points are searched in the clockwise direction from the lower left corner of the k-order cell, if an edge point is found at P(i,j), the search range is reduced, and the edge point is searched from the point P(i-1,j+1), until the k-order cell boundary is found, and the edge point searched by the process is the key point; Step S7: extracting feature pixel image fitting points, smoothing the k-order cell, dividing each k-order cell into four subgrids, and then taking the barycentric coordinates of the key points in each subgrid as the final fitting points; Step S8: fitting an ellipse, fitting the feature pixel image fitting points extracted in step S7 by using a least square method, and finally obtaining the parameter representation of the ellipse; Step S9: feature pixel image output, based on different input image data in different use cases, the input image is colored by using the feature pixel image, and the fitting result is output.
2. The high-precision ellipse fitting method based on multi-scale smoothing and key point searching according to claim 1, characterized in that The isolated point is defined as follows: the isolated point is defined as an edge point with a degree of 0, that is, there is no edge point in the 8-neighborhood of the point.