An ellipse detection method, system, and point cloud reconstruction method based on arc segment union

Abstract

This invention belongs to the field of three-dimensional measurement technology, and specifically discloses an ellipse detection method, system, and point cloud reconstruction method based on arc segment joint analysis. The method includes: acquiring elliptical arc segments and edge points in a target image based on an edge detection algorithm; for elliptical arc segments with an angle span not less than a first span threshold, each elliptical arc segment is used as a candidate ellipse; for elliptical arc segments with an angle span less than the first span threshold, the elliptical arc segments are paired based on convexity constraints and relative position constraints; if the span of the paired elliptical arc segments is between a second span threshold and 2π, they are used as candidate ellipses; ellipse fitting is performed on the edge points of the candidate ellipses to obtain fitted ellipses; for each candidate ellipse, the extraction quality is judged based on the continuity and gradient of its edge points and the shape of the fitted ellipse, retaining ellipses with higher quality. This invention can achieve high-precision ellipse detection, thereby stably and accurately extracting circular marker points in an image that are interfered with by light stripes.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional measurement technology, and more specifically, relates to an ellipse detection method, system, and point cloud reconstruction method based on arc segment combination. Background Technology

[0002] Optical 3D measurement technology is a measurement method that uses optical principles to acquire the three-dimensional spatial shape, position, and surface features of an object. It can generally be divided into active optical 3D measurement and passive optical 3D measurement. Active optical 3D measurement projects a light pattern onto the surface of the object, captures the modulated light pattern on the object's surface using a stereo camera, and then demodulates and reconstructs a 3D point cloud. Compared to passive optical 3D measurement, active optical 3D measurement can acquire more complete 3D topographic data and has a wider range of applications.

[0003] However, limited by the measurement perspective, a single 3D measurement still cannot acquire all the 3D data of an object's surface. In practice, marker points are often pasted onto the surface of the object to be measured. While acquiring the 3D point cloud of the surface from a single perspective, the 3D point cloud of the center of the marker points is extracted. During multiple measurements, based on the relatively fixed 3D coordinates of the marker points on the object's surface, the 3D point clouds of the surface from each perspective are stitched together in the same coordinate system. The multi-view measurement point cloud stitching method based on marker points has high measurement accuracy and fast stitching speed. However, during active ray 3D measurement, the projected light fringes can interfere with the 3D extraction of marker points, resulting in challenges such as a decrease in the number of extracted marker points and unstable stitching.

[0004] Therefore, there is an urgent need for a method that can stably and accurately extract circular marker points in images affected by light stripe interference, so as to provide support for the stable stitching of multi-view 3D point clouds in active optical 3D measurement processes. Summary of the Invention

[0005] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides an ellipse detection method, system and point cloud reconstruction method based on arc segment combination. Its purpose is to achieve high-precision ellipse detection and extraction, so as to stably and accurately extract circular marker points in the image affected by light stripe interference.

[0006] To achieve the above objectives, according to a first aspect of the present invention, an ellipse detection method based on arc segment combination is proposed, comprising the following steps:

[0007] S1. Obtain all elliptical arc segments and edge points contained in the elliptical arc segments in the target image based on the edge detection algorithm;

[0008] S2. Calculate the angular span of each elliptical arc segment;

[0009] For elliptical arc segments with an angle span not less than the first span threshold, each elliptical arc segment is treated as a candidate ellipse.

[0010] For elliptical arc segments with an angular span less than the first span threshold, the elliptical arc segments are paired based on convexity constraints and relative position constraints; if the span of the paired elliptical arc segment is between the second span threshold and 2π, it is used as a candidate ellipse.

[0011] S3. Perform ellipse fitting on the edge points of each candidate ellipse to obtain the fitted ellipse expression;

[0012] S4. For each candidate ellipse, the extraction quality of the ellipse is judged based on the continuity and gradient of its edge points and the shape of the fitted ellipse. Ellipses with higher quality are retained to complete the ellipse detection.

[0013] As a further preferred embodiment, the pairing of elliptical arc segments based on convexity constraints and relative position constraints specifically involves:

[0014] When two elliptical arc segments satisfy the following formula, the two elliptical arc segments are paired;

[0015]

[0016] Where A1, B1, and M1 are the two endpoints and midpoint of the elliptical arc A1M1B1, and A2, B2, and M2 are the two endpoints and midpoint of the elliptical arc A2M2B2; l1 and l2 are the equations of the lines containing chords A1B1 and A2B2, respectively.

[0017] When an elliptical arc segment is successfully paired with multiple other elliptical arc segments, only the one with the largest span among the other elliptical arc segments is paired with the elliptical arc segment.

[0018] As a further preferred embodiment, the first span threshold is 3π / 2, and the second span threshold is 4π / 3.

[0019] As a further preferred method, for each candidate ellipse, the extraction quality is judged based on the continuity and gradient of its edge points, as well as the shape of the fitted ellipse, including:

[0020] The edge points of the candidate ellipses are sorted in a counterclockwise direction to obtain an ordered set of points. For each candidate ellipse, its corresponding ellipse quality evaluation function value is calculated. If the ellipse quality evaluation function value is greater than a preset evaluation threshold, the corresponding ellipse is retained. The ellipse quality evaluation function P... s for:

[0021]

[0022] Where SI is the shape index, LI i GI i WI iThese are the location index, gradient index, and weighted distance index of the i-th edge point, respectively, and n is the total number of edge points in the ordered point set.

[0023] As a further preferred embodiment, the shape index SI is expressed as:

[0024]

[0025] Where a and b are the lengths of the semi-major and semi-minor axes of the fitted ellipse, respectively, and θ arc This is the threshold value for the arc segment bending angle.

[0026] As a further preferred option, the location index LI i The determination method is as follows: for the i-th edge point V in the ordered point set i If an adjacent edge point V appears in its 8 neighboring pixels i-1 or V i+1 , then LI i =1; otherwise LI i =0.

[0027] As a further preferred option, the gradient index GI i , is represented as:

[0028]

[0029]

[0030] Among them, g i I is the theoretical gradient at the i-th edge point determined according to the fitted ellipse equation. i Let be the actual gradient calculated based on the image grayscale at the i-th edge point, and ∥·∥2 represent the L2 norm.

[0031] As a further preferred option, the weighted distance index WI i Represented as:

[0032]

[0033] Where, r i Let θ be the distance from the i-th edge point to the center of the fitted ellipse, R be the length of the semi-major axis of the ellipse, and θ be the distance from the i-th edge point to the center of the fitted ellipse. i Let be the rotation angle of the i-th edge point.

[0034] According to a second aspect of the present invention, an ellipse detection system based on arc segment combination is provided, including a processor for executing the above-described ellipse detection method based on arc segment combination.

[0035] According to a third aspect of the present invention, a method for reconstructing a three-dimensional point cloud of marker points is provided, comprising the following steps:

[0036] Circular markers are pre-attached to the target object, and two corresponding target images containing the markers are acquired using a binocular camera.

[0037] The ellipse detection method based on arc segment union described above was used to detect ellipses, i.e., marker points, on the two target images respectively.

[0038] For the ellipses on the two target images, an initial matching is first performed using the epipolar matching method, and then an accurate matching is performed using the random sampling consensus method to obtain the circular outline of the marker point; then, based on the actual size of the marker point, the matched marker points that do not conform to the size are removed to obtain the three-dimensional point cloud of the marker point.

[0039] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages:

[0040] 1. After obtaining elliptical arc segments through edge detection algorithm, this invention groups and matches them based on the span, convexity, and relative positional relationship of the arc segments. Then, through ellipse fitting and screening, it achieves highly robust and high-precision ellipse detection and extraction, and can stably and accurately extract circular marker points in the image affected by light stripe interference.

[0041] 2. Based on the ellipse detection method of the present invention, the marker point circle affected by the laser line can be accurately extracted under the interference of light stripes, increasing the number of marker points in each frame of image, realizing higher precision three-dimensional point cloud stitching of marker points, and providing support for stable stitching of multi-view three-dimensional point clouds in active optical three-dimensional measurement process.

[0042] 3. This invention creates an ellipse quality evaluation function based on the continuity and gradient of edge points and the shape of the ellipse, thereby verifying whether the fitted ellipse is true, eliminating the interference of poor quality marker points, and improving detection accuracy. Attached Figure Description

[0043] Figure 1 (a) and (b) are schematic diagrams of the three-dimensional point cloud reconstruction device and the marker points according to an embodiment of the present invention;

[0044] Figure 2 This is a flowchart of a point cloud reconstruction method based on ellipse detection using arc segment union, according to an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0046] This invention provides an ellipse detection method based on arc segment combination, such as... Figure 2 As shown, it includes the following steps:

[0047] S1. Detect all elliptical arc segments and their contained edge points in the target image based on the edge detection algorithm.

[0048] In this embodiment: circular markers are pre-attached to the surface of the object to be measured. During online laser or other active optical 3D scanning, an industrial camera is used to acquire an image of the markers on the object surface, including the projected light stripes, i.e., the target image. Due to interference from the light stripes, the circular markers appear as incomplete ellipses in the image, i.e., elliptical arc segments. Based on an edge detection operator (such as EDPF), the elliptical arc segments in the marker image are extracted.

[0049] Furthermore, based on the edge detection algorithm, all elliptical arc segments and their contained edge points in the target image are detected, specifically including the following steps:

[0050] S11. Perform Gaussian filtering preprocessing on the target image to reduce the influence of random noise and smooth the image.

[0051] S12. Calculate the gray-level gradient of each pixel in the image, and use the absolute value of the gray-level gradient as its gradient magnitude. Set a minimum threshold for the gradient magnitude and remove pixels whose gradient magnitude is lower than the minimum threshold.

[0052] S13. Obtain local maxima of gradient magnitude as edge points: Obtain gradient maxima: For any pixel, obtain two adjacent pixels A and B in the gradient direction (forward and backward). If the gradient magnitude of the pixel is greater than the gradient magnitudes of pixels A and B, it is a local maximum of gradient, and the pixel is used as an edge point; connect the edge points to obtain line segments.

[0053] S14. An elliptical arc segment can be considered as a combination of multiple continuous line segments rotating in the same direction with an included angle within a certain threshold range. The line segment obtained by connecting candidate edge points includes several smaller line segments (the connecting line between adjacent edge points is one smaller line segment). A suitable threshold for the included angle of continuous line segments is set (it can be set to 2π / 3 to 8π / 9). When the included angle of adjacent smaller line segments is within the threshold, they are considered to belong to the same arc segment; otherwise, they do not belong to the same arc segment and are broken at that point. Thus, the elliptical arc segment is extracted from the line segment.

[0054] S2. Based on the convexity, relative positional relationship, and angular span of the elliptical arc segments, elliptical arc segments that may belong to the same ellipse are grouped together as candidate ellipses.

[0055] Specifically, the steps include the following:

[0056] S21. Adjust the polarity of the elliptical arc segments to be the same, that is, sort the edge points in the elliptical arc segments according to the same rotation direction, and each elliptical arc segment forms a corresponding ordered set of points. In this embodiment, the polarity of the elliptical arc segments is adjusted to be positive, that is, the edge points in each elliptical arc segment are sorted in a counterclockwise direction.

[0057] S22. Calculate the angular span of each elliptical arc segment. The angular span is the sum of the angles between the smaller line segments within the elliptical arc segment. For elliptical arc segments with a span not less than the first span threshold, they can be directly used as candidate ellipses. The first span threshold is set to 3π / 2.

[0058] S23. For elliptical arc segments with a span smaller than the first span threshold, a method based on convexity constraints and relative position constraints is used to pair up two arc segments that may belong to the same ellipse. The relative positions of the two segments must satisfy the following:

[0059]

[0060] Where A1, B1, and M1 are the two endpoints and midpoint of the elliptical arc A1M1B1, and A2, B2, and M2 are the two endpoints and midpoint of the elliptical arc A2M2B2. l1 and l2 are the equations of the lines containing chords A1B1 and A2B2, respectively, and their forms are ax + by = 0.

[0061] It should be noted that the above pairwise relative position constraint judgment is performed on all elliptical arc segments with a span smaller than the preset first span threshold. When the same elliptical arc segment is successfully paired with multiple other elliptical arc segments, the elliptical arc segment with the largest span is selected to be paired with it, and it is ensured that the sum of the spans of the two arc segments is between the second span threshold and 2π, where the second span threshold can be set to 4π / 3.

[0062] After completing the above pairing, the paired elliptical arc segments will also be considered as candidate ellipses.

[0063] S3. For the candidate ellipse, a continuous optimization algorithm is used to fit the edge points of the candidate ellipse (elliptical arc segment) to obtain the ellipse equation and ellipse parameters.

[0064] Specifically, the steps include the following:

[0065] S31. Set the edge point set {(x} corresponding to each group of elliptical arc segments. i ,y i Substituting |i=1,2,…,n} into the expression for the ellipse ax 2 +bxy+cy 2 In +dx+ey+f=0, the distance to be optimized term is obtained:

[0066]

[0067] Among them, Xi for A is a matrix of parameters to be determined [abcdef] T .

[0068] S32. Solving the above equation using the least squares method yields the ellipse parameter matrix A and the fitted ellipse equation.

[0069] S4. Create an ellipse quality evaluation function. Calculate the ellipse quality evaluation function value for each ellipse based on the extracted candidate edge point set and the fitted ellipse expression. If the ellipse quality evaluation function value is greater than the preset evaluation threshold, the corresponding fitted ellipse is determined to be a true marker point; otherwise, it is discarded.

[0070] Furthermore, the elliptic quality evaluation function P s It consists of four parts: the ellipse quality evaluation function P is defined based on the shape index SI of the fitted ellipse, and the position index LI, gradient index GI, and weighted distance index WI based on the edge point set. s , represented as:

[0071]

[0072] For a given fitted ellipse, all the edge points in the elliptical arc segments it contains constitute the corresponding set of edge points, where n is the total number of edge points in the set.

[0073] Specifically:

[0074] (1) The shape index SI constrains the shape of the ellipse, expressed as:

[0075]

[0076] Where a is the length of the major semi-axis of the ellipse, and b is the length of the minor semi-axis of the ellipse.

[0077] (2) The position index LI constrains the continuity of edge points. For the i-th point V in the ordered point set corresponding to the ellipse... i If there are adjacent points V in its 8 neighboring pixels i-1 or V i+1 , then LI i =1, otherwise LI i =0.

[0078] (3) The gradient index GI constrains the error between the actual gradient and the theoretical gradient at the edge points, expressed as:

[0079]

[0080]

[0081] Among them, g iI is the theoretical gradient calculated at the i-th edge point based on the fitted ellipse equation. i The gradient at the i-th edge point is calculated based on the image grayscale.

[0082] (4) The weighted distance index WI constrains the weighting coefficient of each edge point, expressed as:

[0083]

[0084] Where, r i Let θ be the distance from the i-th edge point to the center of the fitted ellipse, R be the length of the semi-major axis of the ellipse, and θ be the distance from the i-th edge point to the center of the fitted ellipse. i Let be the rotation angle of the i-th edge point.

[0085] This invention also provides a method for reconstructing a 3D point cloud of marker points based on the above-described ellipse detection method, comprising the following steps:

[0086] S5. Use the methods in steps S1 to S4 to obtain the marker points in the stereo images respectively; then extract the center coordinates of the marker point ellipse in the stereo images respectively, obtain the marker point matching relationship in the stereo images based on epipolar matching and random sampling consensus (RANSAC), and reconstruct the marker point cloud in three-dimensional space based on the stereo system parameters.

[0087] Specifically, the steps include the following:

[0088] S51. Based on the principle of epipolar matching, obtain the initial set of matching point pairs P of the marker points.

[0089] S52. From the set of matching point pairs, randomly select 4 sets of matching point pairs and calculate the corresponding affine transformation matrix H. i According to matrix H i Calculate the left image point (x) in other matching groups in P. l ,y l The corresponding pixel (x) under this transformation l ′,y l ′).

[0090] S53. Based on the above theoretical matching point (x) l ′,y l ′), calculate its true matching point (x′) with the right image found based on adjacency relationships. r ,y r If the distance to a point is less than one pixel, then the point is denoted as H by the affine transformation. i The interior point of .

[0091] S54. Select different initial matching point pairs to calculate multiple affine transformation matrices, calculate the total number of interior points of the affine transformation, and select the affine transformation matrix with the largest number of interior points as the final marker point matching result.

[0092] S55. Based on the binocular matching results of the marker points, and based on the intrinsic and extrinsic parameters of the binocular system, reconstruct the three-dimensional point cloud of the marker points. Based on the binocular depth of field and the ellipse quality evaluation results of the marker points, screen and evaluate the extracted three-dimensional marker points.

[0093] This invention also provides a device for reconstructing a 3D point cloud of marker points based on the above-described ellipse detection method, such as... Figure 1 As shown, it includes:

[0094] The projection and measurement module is used to project light stripes onto the surface of an object, and can reconstruct a three-dimensional point cloud of the object's surface based on the light stripes captured by the camera.

[0095] Binocular camera module: used to synchronously acquire images of marker points on the surface of an object.

[0096] Binocular calibration module: used to calibrate the intrinsic and extrinsic parameters of the stereo camera.

[0097] Landmark Image Detection Module: Used to extract and fit an ellipse based on two images acquired at the same time.

[0098] The marker point 3D reconstruction module is used to reconstruct the 3D point cloud of the marker points based on the ellipses extracted by the left and right cameras and the dual-target positioning parameters.

[0099] When the above-mentioned device is tested, if Figure 1 As shown, the projection unit is used to project stripe light onto the surface of the object to reconstruct a three-dimensional point cloud of the object's surface; the left and right cameras are used to synchronously acquire images of the object's surface to reconstruct a three-dimensional point cloud of the marker points.

[0100] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An ellipse detection method based on arc segment combination, characterized in that, Includes the following steps: S1. Obtain all elliptical arc segments and edge points contained in the elliptical arc segments in the target image based on the edge detection algorithm; S2. Calculate the angular span of each elliptical arc segment; For elliptical arc segments with an angle span not less than the first span threshold, each elliptical arc segment is treated as a candidate ellipse. For elliptical arc segments with an angular span smaller than the first span threshold, the elliptical arc segments are paired based on convexity constraints and relative position constraints. If the span of the paired elliptical arc segment is at the second span threshold and If it falls between these points, then consider it as an alternative ellipse; S3. Perform ellipse fitting on the edge points of each candidate ellipse to obtain the fitted ellipse expression; S4. For each candidate ellipse, the extraction quality is judged based on the continuity and gradient of its edge points, as well as the shape of the fitted ellipse. Ellipses with higher quality are retained to complete ellipse detection; this includes: The edge points of the candidate ellipses are sorted in a counter-clockwise direction to obtain an ordered set of points. For each candidate ellipse, its corresponding ellipse quality evaluation function value is calculated. If the ellipse quality evaluation function value is greater than a preset evaluation threshold, the corresponding ellipse is retained. for: in, SI For shape indicators, LI i , GI i , WI i The first i Location metrics, gradient metrics, and weighted distance metrics for each edge point. n The total number of edge points in the ordered point set; Shape index SI Represented as: in, a , b These represent the lengths of the semi-major and semi-minor axes of the fitted ellipse, respectively. The threshold for the arc segment bending angle; Location indicators LI i The method for determining is as follows: for the first point in the ordered set... i edge points If adjacent edge points appear in its 8 neighboring pixels or ,but LI i ;otherwise LI i ; Gradient index GI i Represented as: in, For the first i The theoretical gradient at each edge point, determined based on the fitted ellipse equation. For the first i The actual gradient at each edge point is calculated based on the image grayscale. Represents the L2 norm; Weighted distance index WI i Represented as: in, For the first i The distance from each edge point to the center of the fitted ellipse R The length of the semi-major axis of the ellipse. For the first i The rotation angle of each edge point.

2. The ellipse detection method based on arc segment combination as described in claim 1, characterized in that, The pairing of elliptical arc segments based on convexity constraints and relative position constraints is specifically as follows: When two elliptical arc segments satisfy the following formula, the two elliptical arc segments are paired; in, Elliptical arc The two endpoints and the midpoint, Elliptical arc The two endpoints and the midpoint; and Each is a string Chords The equation of the line in question; When an elliptical arc segment is successfully paired with multiple other elliptical arc segments, only the one with the largest span among the other elliptical arc segments is paired with the elliptical arc segment.

3. The ellipse detection method based on arc segment combination as described in claim 1, characterized in that, The first span threshold is The second span threshold is .

4. An ellipse detection system based on arc segment combination, characterized in that, Includes a processor for executing the ellipse detection method based on arc segment union as described in any one of claims 1-3.

5. A method for reconstructing a three-dimensional point cloud of marker points, characterized in that, Includes the following steps: Circular markers are pre-attached to the target object, and two corresponding target images containing the markers are acquired using a binocular camera. Ellipses, i.e. marker points, on the two target images are detected by the ellipse detection method based on arc segment combination as described in any one of claims 1-3. For the ellipses on the two target images, an initial matching is first performed using the epipolar matching method, and then an accurate matching is performed using the random sampling consensus method to obtain the circular outline of the marker point; then, based on the actual size of the marker point, the matched marker points that do not conform to the size are removed to obtain the three-dimensional point cloud of the marker point.

Images