Ellipse detection method and system based on segmental arc combination and point cloud reconstruction method

Abstract

The invention belongs to the technical field of three-dimensional measurement, and particularly discloses an ellipse detection method and system based on segmental arc combination and a point cloud reconstruction method, and the method comprises the steps: obtaining ellipse segmental arcs and edge points in a target image based on an edge detection algorithm; for the elliptic arc sections with the angle spans not smaller than a first span threshold value, taking each elliptic arc section as an alternative ellipse; for the elliptic arc segments with the angle span smaller than a first span threshold value, pairing the elliptic arc segments based on convexity constraint and relative position constraint; if the span of the paired elliptic arc section is between a second span threshold and 2pi, taking the elliptic arc section as an alternative ellipse; performing ellipse fitting on edge points in the alternative ellipse to obtain a fitted ellipse; and for each alternative ellipse, the extraction quality of the ellipse is judged according to the continuity and gradient of the edge points in the alternative ellipse and the shape of the fitting ellipse, and the ellipse with relatively high quality is reserved. According to the invention, high-precision ellipse detection can be realized, so that the circular mark points interfered by light stripes in the image can be stably and accurately extracted.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional measurement technology, and more specifically, relates to an ellipse detection method, system and point cloud reconstruction method based on arc segment union. Background Art

[0002] Optical 3D measurement technology is a measurement method that uses optical principles to obtain the three-dimensional spatial shape, position and surface features of an object. It can generally be divided into active optical 3D measurement and passive optical 3D measurement. Active optical 3D measurement projects a light pattern onto the surface of the object to be measured, collects the modulated light pattern on the surface of the object through a stereo camera, and then demodulates and stereoscopically reconstructs a 3D point cloud. Compared with passive optical 3D measurement, active optical 3D measurement can obtain more complete 3D shape data and is more widely used.

[0003] However, due to the limitation of the measurement perspective, a single 3D measurement still cannot obtain all the 3D data of the object surface. In practice, it is often necessary to paste marker points on the surface of the object to be measured, obtain the single-view surface 3D point cloud, and extract the 3D point cloud at the center of the marker points. In the process of multiple measurements, the 3D point clouds of the surface from each perspective are spliced ​​into the same coordinate system based on the relatively fixed 3D coordinates of the marker points on the object surface. The multi-view measurement point cloud splicing method based on marker points has high measurement accuracy and fast splicing speed, but the light stripes projected during the active light 3D measurement process will interfere with the 3D extraction of the marker points, resulting in challenges such as a decrease in the number of marker points extracted and unstable splicing.

[0004] Therefore, there is an urgent need for a method that can stably and accurately extract circular landmark points in images that are affected by light fringes, so as to provide support for the stable stitching of multi-view 3D point clouds in the active optical 3D measurement process. Summary of the invention

[0005] In view of the above defects or improvement needs of the prior art, the present invention provides an ellipse detection method, system and point cloud reconstruction method based on arc segment combination, whose purpose is to achieve high-precision ellipse detection and extraction, so as to stably and accurately extract circular landmark points in the image affected by light streak interference.

[0006] To achieve the above object, according to a first aspect of the present invention, an ellipse detection method based on arc segment union is proposed, comprising the following steps:

[0007] S1, obtaining all elliptical arc segments and edge points contained in the target image based on an edge detection algorithm;

[0008] S2, calculating the angle span of each elliptical arc segment;

[0009] For ellipse arc segments whose angle span is not less than a first span threshold, each ellipse arc segment is used as a candidate ellipse;

[0010] For elliptical arc segments whose angle span is less than the first span threshold, the elliptical arc segments are paired based on the convexity constraint and the relative position constraint; if the span of the paired elliptical arc segment is between the second span threshold and 2π, it is used as a candidate ellipse;

[0011] S3, performing ellipse fitting on the edge points in each candidate ellipse to obtain a fitted ellipse expression;

[0012] S4. For each candidate ellipse, the extraction quality of the ellipse is judged according to the continuity and gradient of the edge points therein and the shape of the ellipse obtained by fitting, and the ellipse with higher quality is retained to complete the ellipse detection.

[0013] As a further preferred embodiment, the pairing of elliptical arc segments based on convexity constraints and relative position constraints is specifically as follows:

[0014] When two elliptical arc segments satisfy the following formula, the two elliptical arc segments are paired;

[0015]

[0016] Among them, A1, B1, M1 are the two endpoints and the midpoint of the elliptical arc A1M1B1, A2, B2, M2 are the two endpoints and the midpoint of the elliptical arc A2M2B2; l1 and l2 are the equations of the straight lines where the chords A1B1 and A2B2 are located respectively;

[0017] When an elliptical arc segment is successfully paired with multiple other elliptical arc segments, only the one with the largest span among the other elliptical arc segments is paired with the elliptical arc segment.

[0018] As further preferred, the first span threshold is 3π / 2, and the second span threshold is 4π / 3.

[0019] As a further preferred embodiment, for each candidate ellipse, judging the extraction quality of the ellipse according to the continuity and gradient of the edge points therein and the shape of the ellipse obtained by fitting, includes:

[0020] The edge points in the candidate ellipse are sorted in a counterclockwise direction to obtain an ordered point set; for each candidate ellipse, the corresponding ellipse quality evaluation function value is calculated, and if the ellipse quality evaluation function value is greater than a preset evaluation threshold, the corresponding ellipse is determined to be retained; the ellipse quality evaluation function P s for:

[0021]

[0022] Among them, SI is the shape index, LI i GI i , WI iare the position index, gradient index, and weighted distance index of the i-th edge point respectively, and n is the total number of edge points in the ordered point set.

[0023] As a further preferred embodiment, the shape index SI is expressed as:

[0024]

[0025] Among them, a and b are the lengths of the semi-major axis and semi-minor axis of the fitted ellipse, respectively, and θ arc The arc segment bending angle threshold.

[0026] As a further preferred embodiment, the position indicator LI i The determination method is as follows: for the i-th edge point V in the ordered point set i , if there are adjacent edge points V on its 8 neighboring pixels i-1 or V i+1 , then LI i =1; otherwise LI i =0.

[0027] As a further preferred embodiment, the gradient index GI i , expressed as:

[0028]

[0029]

[0030] Among them, g i is the theoretical gradient determined by the fitted ellipse equation at the i-th edge point, I i is the actual gradient calculated according to the image grayscale at the i-th edge point, and ∥·∥2 represents the binary norm.

[0031] As a further preferred embodiment, the weighted distance index WI i It is expressed as:

[0032]

[0033] Among them, r i is the distance from the i-th edge point to the center of the fitted ellipse, R is the length of the major semi-axis of the ellipse, θ i is the rotation angle of the i-th edge point.

[0034] According to a second aspect of the present invention, there is provided an ellipse detection system based on arc-segment union, comprising a processor, wherein the processor is used to execute the above-mentioned ellipse detection method based on arc-segment union.

[0035] According to a third aspect of the present invention, a method for reconstructing a three-dimensional point cloud of a marker point is provided, comprising the following steps:

[0036] A circular marker is pre-pasted on the target object, and two target images containing the marker are obtained through a binocular camera.

[0037] The ellipses on the two target images, i.e., the landmark points, are detected respectively by the above-mentioned ellipse detection method based on arc segment union;

[0038] For the ellipses on the two target images, the initial matching is first performed by the epipolar matching method, and then the accurate matching is performed by the random sampling consistency method to obtain the circular contour of the marker points; then, according to the actual size of the marker points, the matched marker points that do not match the size are eliminated to obtain the three-dimensional point cloud of the marker points.

[0039] In general, the above technical solution conceived by the present invention has the following technical advantages compared with the prior art:

[0040] 1. After obtaining the elliptical arc segments through the edge detection algorithm, the present invention groups and matches the arc segments based on their span, convexity and relative position relationship, and then realizes highly robust and high-precision ellipse detection and extraction through ellipse fitting and screening, and can stably and accurately extract circular landmark points in the image affected by light streak interference.

[0041] 2. Based on the ellipse detection method of the present invention, the circle of marker points affected by the laser line can be accurately extracted under the interference of light stripes, the number of marker points in each frame of the image can be increased, and higher-precision three-dimensional point cloud stitching of marker points can be achieved, providing support for the stable stitching of multi-perspective three-dimensional point clouds in the active optical three-dimensional measurement process.

[0042] 3. The present invention creates an ellipse quality evaluation function based on the continuity and gradient of edge points and the shape of the ellipse, thereby verifying whether the fitted ellipse is true, eliminating the interference of poor quality landmarks and improving detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 (a) and (b) are schematic diagrams of a marker point 3D point cloud reconstruction device and marker points according to an embodiment of the present invention;

[0044] Figure 2 This is a flow chart of a point cloud reconstruction method based on arc-segment-joint ellipse detection according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0046] An embodiment of the present invention provides an ellipse detection method based on arc segment combination, such as Figure 2 As shown, the following steps are included:

[0047] S1. Detect all elliptical arc segments and the edge points they contain in the target image based on the edge detection algorithm.

[0048] In this embodiment: circular marking points are pre-pasted on the surface of the object to be measured, and during the online laser or other active optical three-dimensional scanning process, an industrial camera is used to obtain an image of the marking points on the object surface including the projected light stripes, i.e., the target image; due to the interference of the light stripes, the circular marking points appear as incomplete ellipses in the image, i.e., elliptical arc segments; based on the edge detection operator (such as EDPF), the elliptical arc segments in the marking point image are extracted.

[0049] Furthermore, all elliptical arc segments and edge points contained in the target image are detected based on an edge detection algorithm, which specifically includes the following steps:

[0050] S11. Perform Gaussian filtering preprocessing on the target image to reduce the influence of random noise and make the image smooth.

[0051] S12, calculating the grayscale gradient of each pixel in the image, and taking the absolute value of the grayscale gradient as its gradient amplitude, setting the gradient amplitude minimum threshold, and eliminating pixels whose gradient amplitude is lower than the minimum threshold.

[0052] S13. Obtain the local maximum point of the gradient amplitude as the edge point: Obtain the gradient maximum point: For any pixel point, obtain two pixel points A and B adjacent to the pixel point in the gradient direction (forward and reverse). If the gradient amplitude of the pixel point is greater than the gradient amplitude of pixel points A and B, it is the local maximum point of the gradient and the pixel point is taken as the edge point; the edge points are connected to obtain a line segment.

[0053] S14. Elliptical arc segments can be regarded as being composed of multiple continuous line segments rotating in the same direction and with angles within a certain threshold range. The line segments obtained by connecting the edge candidate points include several small line segments (the connecting line of adjacent edge points is a small line segment); set a suitable continuous line segment angle threshold (which can be set to 2π / 3~8π / 9). When the angle between adjacent small line segments is within the continuous line segment angle threshold, it is considered to belong to an arc segment, otherwise it does not belong to an arc segment and is disconnected from that point; thus, the elliptical arc segment is extracted from the line segment.

[0054] S2. According to the convexity, relative position relationship and angle span of the elliptical arc segments, the elliptical arc segments that may belong to the same ellipse are grouped into the same group as candidate ellipses.

[0055] The specific steps include:

[0056] S21, adjusting the polarity of the elliptical arc segments to be the same, that is, the edge points in the elliptical arc segments are all sorted in the same rotation direction, and each elliptical arc segment forms a corresponding ordered point set. In this embodiment, the polarity of the elliptical arc segments is adjusted to be positive, that is, the edge points in each elliptical arc segment are sorted in a counterclockwise direction.

[0057] S22, respectively calculate the angle span of each ellipse arc segment, the angle span being the sum of the angles between the small line segments in the ellipse arc segment. For an ellipse arc segment whose span is not less than a first span threshold, it can be directly used as a candidate ellipse. The first span threshold is set to 3π / 2.

[0058] S23. For ellipse arc segments whose span is less than the first span threshold, a method based on convexity constraint and relative position constraint is used to pair two arc segments that may belong to the same ellipse. The relative positions of the two arc segments must satisfy:

[0059]

[0060] Among them, A1, B1, and M1 are the two endpoints and the midpoint of the elliptical arc A1M1B1, A2, B2, and M2 are the two endpoints and the midpoint of the elliptical arc A2M2B2, l1 and l2 are the equations of the straight lines where the chords A1B1 and A2B2 are located, respectively, and their form is ax+by=0.

[0061] It should be noted that the above-mentioned pairwise relative position constraint judgment is performed on all elliptical arc segments with a span less than the preset first span threshold. When the same elliptical arc segment is successfully paired with multiple other elliptical arc segments, the elliptical arc segment with the largest span is paired with it, and it is ensured that the sum of the spans of the two arc segments is between the second span threshold and 2π, where the second span threshold can be set to 4π / 3.

[0062] After completing the above pairing, the paired ellipse arc segment pair is also used as a candidate ellipse.

[0063] S3. For the candidate ellipse, a continuity optimization algorithm is used to perform ellipse fitting on the edge points in the candidate ellipse (ellipse arc segment) to obtain the ellipse equation and ellipse parameters.

[0064] The specific steps include:

[0065] S31, the edge point set {(x i ,y i )|i=1,2,…,n} into the expression ax of the ellipse 2 +bxy+cy 2 +dx+ey+f=0, the distance to be optimized is obtained:

[0066]

[0067] Among them, Xi for A parameter matrix to be determined [abcdef] T .

[0068] S32. Solving the above equation using the least squares method can obtain the ellipse parameter matrix A and the fitted ellipse equation.

[0069] S4. Create an ellipse quality evaluation function, and calculate the ellipse quality evaluation function value corresponding to each ellipse according to the extracted candidate edge point set and the fitted ellipse expression. If the ellipse quality evaluation function value is greater than the preset evaluation threshold, the corresponding fitted ellipse is judged to be a true landmark point, otherwise it is discarded.

[0070] Furthermore, the ellipse quality evaluation function P s It consists of four parts: the ellipse quality evaluation function P is defined according to the shape index SI of the fitted ellipse, the position index LI based on the edge point set, the gradient index GI and the weighted distance index WI s , expressed as:

[0071]

[0072] For a certain fitted ellipse, all edge points in the elliptical arc segments it contains are the corresponding edge point set, and n is the total number of edge points in the edge point set.

[0073] Specific:

[0074] (1) The shape index SI constrains the shape of the ellipse and is expressed as:

[0075]

[0076] Among them, a is the length of the major axis of the ellipse, and b is the length of the minor axis of the ellipse.

[0077] (2) The position index LI constrains the continuity of edge points. For the i-th point V in the ordered point set corresponding to the ellipse i , if the adjacent point V appears on its 8 neighboring pixels i-1 or V i+1 , then LI i =1, otherwise LI i =0.

[0078] (3) The gradient index GI constrains the error between the actual gradient of the edge point and the theoretical gradient, which can be expressed as:

[0079]

[0080]

[0081] Among them, g iis the theoretical gradient calculated according to the fitted ellipse equation at the i-th edge point, I i is the actual gradient calculated based on the image grayscale at the i-th edge point.

[0082] (4) The weighted distance index WI constrains the weight coefficient of each edge point and is expressed as:

[0083]

[0084] Among them, r i is the distance from the i-th edge point to the center of the fitted ellipse, R is the length of the major semi-axis of the ellipse, θ i is the rotation angle of the i-th edge point.

[0085] The embodiment of the present invention further provides a method for reconstructing a three-dimensional point cloud of a landmark point based on the above-mentioned ellipse detection method, comprising the following steps:

[0086] S5. Use the methods in steps S1 to S4 to respectively obtain the landmark points in the binocular stereo image; then extract the center coordinates of the landmark point ellipse for the binocular stereo image, obtain the binocular image landmark point matching relationship based on epipolar matching and random sampling consensus method (RANSAC), and reconstruct the landmark point cloud in three-dimensional space based on the binocular system parameters.

[0087] The specific steps include:

[0088] S51. Based on the epipolar matching principle, obtain an initial matching point pair set P of the landmark points.

[0089] S52. Randomly select 4 sets of matching point pairs from the set of matching point pairs and calculate the corresponding affine transformation matrix H i According to the matrix H i , calculate the left image point (x l ,y l ) corresponds to the pixel point (x) under this transformation l ′,y l ′).

[0090] S53. According to the above theoretical matching point (x l ′,y l ′), calculate the real matching point (x r ,y r ), if the distance is less than one pixel, the point is recorded as affine transformation H i The inner point of .

[0091] S54. Select different initial matching point pairs to calculate multiple affine transformation matrices, calculate the total number of inner points of the affine transformation, and select the affine transformation matrix with the largest number of inner points as the final landmark point matching result.

[0092] S55. According to the binocular matching results of the marker points, based on the internal and external parameters of the binocular system, the three-dimensional point cloud of the marker points is reconstructed, and according to the binocular depth of field and the ellipse quality evaluation results of the marker points, the extracted three-dimensional marker points are screened and evaluated.

[0093] The embodiment of the present invention also provides a three-dimensional point cloud reconstruction device of a landmark point based on the above ellipse detection method, such as Figure 1 As shown, it includes:

[0094] The projection and measurement module is used to project light stripes onto the surface of an object and reconstruct a partial three-dimensional point cloud of the object's surface based on the light stripes captured by the camera.

[0095] Binocular camera module: used to synchronously collect images of landmark points on the surface of objects.

[0096] Binocular calibration module: used to calibrate the internal and external parameters of the binocular camera.

[0097] Landmark point image detection module: used to extract and fit the ellipse based on the two images collected at the same time.

[0098] Marker point 3D reconstruction module: used to reconstruct the marker point 3D point cloud based on the binocular positioning parameters according to the ellipses extracted by the left and right cameras.

[0099] When the above device is tested, if Figure 1 As shown, the projection unit is used to project stripe light onto the surface of an object to reconstruct a three-dimensional point cloud of the object surface; the left and right cameras are used to synchronously capture images of the object surface to reconstruct a three-dimensional point cloud of the marker points.

[0100] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An ellipse detection method based on arc segment union, characterized in that: The steps include: S1, obtaining all elliptical arc segments and edge points contained in the target image based on an edge detection algorithm; S2, calculating the angle span of each elliptical arc segment; For ellipse arc segments whose angle span is not less than a first span threshold, each ellipse arc segment is used as a candidate ellipse; For elliptical arc segments whose angle span is less than a first span threshold, the elliptical arc segments are paired based on a convexity constraint and a relative position constraint; If the span of the paired ellipse arc segment is between the second span threshold and 2π, it is used as a candidate ellipse; S3, performing ellipse fitting on the edge points in each candidate ellipse to obtain a fitted ellipse expression; S4. For each candidate ellipse, the extraction quality of the ellipse is judged according to the continuity and gradient of the edge points therein and the shape of the ellipse obtained by fitting, and the ellipse with higher quality is retained to complete the ellipse detection.

2. The ellipse detection method based on arc segment combination as claimed in claim 1, characterized in that: The pairing of elliptical arc segments based on convexity constraints and relative position constraints is specifically as follows: When two elliptical arc segments satisfy the following formula, the two elliptical arc segments are paired; Among them, A1, B1, M1 are the two endpoints and the midpoint of the elliptical arc A1M1B1, A2, B2, M2 are the two endpoints and the midpoint of the elliptical arc A2M2B2; l1 and l2 are the equations of the straight lines where the chords A1B1 and A2B2 are located respectively; When an elliptical arc segment is successfully paired with multiple other elliptical arc segments, only the one with the largest span among the other elliptical arc segments is paired with the elliptical arc segment.

3. The ellipse detection method based on arc segment combination as claimed in claim 1, characterized in that: The first span threshold is 3π / 2, and the second span threshold is 4π / 3.

4. The ellipse detection method based on arc segment combination according to any one of claims 1 to 3, characterized in that: For each candidate ellipse, the quality of ellipse extraction is judged based on the continuity and gradient of the edge points and the shape of the fitted ellipse, including: The edge points in the candidate ellipse are sorted in a counterclockwise direction to obtain an ordered point set; for each candidate ellipse, the corresponding ellipse quality evaluation function value is calculated, and if the ellipse quality evaluation function value is greater than a preset evaluation threshold, the corresponding ellipse is retained; the ellipse quality evaluation function P s for: Among them, SI is the shape index, LI i , GI i , WI i are the position index, gradient index, and weighted distance index of the i-th edge point respectively, and n is the total number of edge points in the ordered point set.

5. The ellipse detection method based on arc segment combination as claimed in claim 4, characterized in that: The shape index SI is expressed as: Among them, a and b are the lengths of the semi-major axis and semi-minor axis of the fitted ellipse, respectively, and θ arc The arc segment bending angle threshold.

6. The ellipse detection method based on arc segment combination as claimed in claim 4, characterized in that: Position indicator LI i The determination method is as follows: for the i-th edge point V in the ordered point set i , if there are adjacent edge points V on its 8 neighboring pixels i-1 or V i+1 , then LI i =1; otherwise LI i =0.

7. The ellipse detection method based on arc segment combination as claimed in claim 4, characterized in that: Gradient Index GI i , expressed as: Among them, g i is the theoretical gradient determined by the fitted ellipse equation at the i-th edge point, I i is the actual gradient calculated according to the image grayscale at the i-th edge point, and ∥·∥2 represents the binary norm.

8. The ellipse detection method based on arc segment combination as claimed in claim 4, characterized in that: Weighted distance index WI i It is expressed as: Among them, r i is the distance from the i-th edge point to the center of the fitted ellipse, R is the length of the major semi-axis of the ellipse, θ i is the rotation angle of the i-th edge point.

9. An ellipse detection system based on arc segment combination, characterized in that: It includes a processor, and the processor is used to execute the ellipse detection method based on arc segment union as described in any one of claims 1-8.

10. A method for reconstructing a three-dimensional point cloud of a landmark point, characterized in that: The steps include: A circular marker is pre-pasted on the target object, and two target images containing the marker are obtained through a binocular camera. By using the ellipse detection method based on arc segment combination as described in any one of claims 1 to 8, ellipses, i.e., marker points, on the two target images are respectively detected; For the ellipses on the two target images, the initial matching is first performed by the epipolar matching method, and then the accurate matching is performed by the random sampling consistency method to obtain the circular contour of the marker points; then, according to the actual size of the marker points, the matched marker points that do not match the size are eliminated to obtain the three-dimensional point cloud of the marker points.

Images