A handheld 3D scanning method and system for correcting circular markers

Abstract

This invention proposes a handheld 3D scanning method and system for correcting circular marker points, comprising the following steps: S1, acquiring a calibration image of a circular array calibration board using a handheld scanning device, and calculating a calibration result based on the calibration image; S2, optimizing the calibration result using an optimization method based on point-to-epidial distance constraints to obtain an optimized intrinsic and extrinsic parameter matrix for the camera; S3, acquiring marker point images based on the calibration result, and extracting the center coordinates of the marker points using the marker point images; wherein, the marker point images include a first marker point image and a second marker point image; S4, finding the correspondence between circular marker points in the first and second marker point images; S5, performing 3D reconstruction of the circular marker points based on the correspondence between the marker points to obtain a 3D coordinate set of the circular marker points; S6, using the depth information of the marker points to remove outliers from the 3D coordinate set of the circular marker points.

Description

Technical Field

[0001] This invention relates to the field of 3D scanning technology, and in particular to a method and system for correcting errors in circular marker points using a handheld 3D scanner. Background Technology

[0002] In 3D scanning measurement, a single scan can often only obtain local point cloud data of the object under test from one viewpoint. To obtain complete point cloud data of the object under test, it is generally necessary to scan the object from multiple angles and then stitch together the local point cloud data from each viewpoint to obtain the shape information of the entire surface of the object.

[0003] Common stitching methods include: 1. Using sophisticated hardware to calculate the point cloud transformation matrix from two viewpoints, such as robotic arms, robots, and rotating platforms. This method requires high hardware precision, and the measurement range is limited by the working range of the hardware. Since sophisticated hardware is very expensive, this method is rarely chosen in practice due to cost considerations. 2. Matching by calculating the two-dimensional and three-dimensional invariants of the point clouds from two viewpoints. This method has the advantage of not requiring any processing of the scanned object and works well for objects with complex shapes and obvious features. However, it cannot stitch objects with unclear texture information, such as planes and spheres. 3. Placing auxiliary spherical objects within the field of view. These artificially placed auxiliary spherical objects have obvious features compared to the object being measured, allowing for quick and accurate location of corresponding spheres from various viewpoints, thus enabling the calculation of the transformation matrix from different viewpoints. 4. Pasting artificial markers on the surface of the object being measured and using these markers to calculate the transformation matrix. This method relies on inexpensive hardware and is suitable for objects of any curvature and shape, except in a few cases where marking is not allowed. It is the most widely used stitching method.

[0004] Chinese patent CN107230233A discloses a calibration method and apparatus for a telecentric lens three-dimensional imaging system based on beam adjustment. The three-dimensional imaging system uses a camera device with a telecentric lens and a projection device with a telecentric lens. The camera device and the projection device are nonlinearly calibrated by beam adjustment, and the calibration parameters of the calibrated camera device and the projection device are used to calibrate the three-dimensional imaging system.

[0005] These methods all require repeated iterations to find the correct correspondence during the calculation process, which takes a long time. They are not suitable for handheld devices that have high requirements for real-time computing. A fast solution method needs to be found. Summary of the Invention

[0006] In view of this, the present invention proposes a method and system for correcting circular markers in handheld 3D scanning. The method sets up a circular marker correction method on the handheld device, which reduces the error rate in the scanning results. At the same time, it eliminates abnormal points in the 3D coordinate set of the circular markers by using depth information and slope, thus ensuring the correctness of the scanning results.

[0007] The technical solution of this invention is implemented as follows:

[0008] In a first aspect, the present invention provides a method for correcting errors in circular marker points using a handheld 3D scanning device, comprising the following steps:

[0009] S1. Use a handheld scanning device to acquire images of the dot array calibration board to obtain calibration images, and calculate the calibration results based on the calibration images;

[0010] S2. The calibration results are optimized using an optimization method based on the point-to-epidial distance constraint to obtain the optimized intrinsic and extrinsic parameter matrix of the camera.

[0011] S3. Acquire marker point images based on the calibration results, and extract the center coordinates of the marker points using the marker point images; wherein, the marker point images include a first marker point image and a second marker point image;

[0012] S4. Find the correspondence between circular marker points in the first marker point image and the second marker point image;

[0013] S5. Based on the correspondence of the marker points, perform three-dimensional reconstruction of the circular marker points to obtain the three-dimensional coordinate set {Q} of the circular marker points. i |i = 1, 2, 3, ..., N};

[0014] S6. Use the depth information of the marker points to eliminate the three-dimensional coordinate set {Q} of the circular marker points. i Outliers in the group |i=1,2,3,……,N}.

[0015] Based on the above technical solutions, preferably, the handheld scanning device is equipped with a left camera and a right camera, and the calibration image includes a first calibration plate image and a second calibration plate image. Step S2 specifically includes:

[0016] S21. Based on the first calibration board image and the second calibration board image, obtain the intrinsic and extrinsic parameter matrices and distortion matrices of the left and right cameras, respectively;

[0017] S22. Calculate the fundamental matrix between the left and right cameras based on the intrinsic and extrinsic parameter matrices and distortion matrix of the left and right cameras;

[0018] S23. Based on the correspondence between the pixel coordinates of the dots in the dot array calibration board images acquired by the left and right cameras, the first calibration board image, the second calibration board image, and the basic matrix are used as initial values. The distance between corresponding points in the first and second calibration board images is used as the optimization objective function to optimize the matrix, thereby obtaining the optimized intrinsic and extrinsic parameter matrix.

[0019] Based on the above technical solutions, preferably, the calculation formula for the basic matrix between the left camera and the right camera in step S22 is as follows:

[0020]

[0021]

[0022] Among them, K -1 L K represents the intrinsic parameter matrix of the left camera. L The inverse matrix, K -T R K represents the intrinsic parameter matrix of the right camera. R The transpose of the inverse matrix, R represents the rotation matrix between the left and right cameras, T(t) x , t y , t z ) represents the three components of the translation matrix between the left and right cameras, and S is the middle matrix.

[0023] Based on the above technical solutions, the preferred calculation formula for the objective function in step S23 is as follows:

[0024] m i '×F×m i =0;

[0025] Where, m i and m i ' represents the coordinates of the corresponding point in the dot array calibration board image acquired by the left and right cameras, respectively, and i represents the corresponding point number.

[0026] Based on the above technical solutions, preferably, step S6 specifically includes:

[0027] S61, Based on the three-dimensional coordinate set {Q} of the circular marker point i Calculate the slope of the nearest neighbor point of the circular marker in the depth direction for the circular marker point |i=1,2,3,……,N};

[0028] S62. Determine whether the slope of the nearest point is greater than a first set threshold. If so, mark the circular marker point as an abnormal point.

[0029] S63. Delete the abnormal points;

[0030] S64. Repeat steps S61-S63 until the three-dimensional coordinate set of the circular marker point {Q} is reached. i There are no outliers in the group |i=1,2,3,……,N}.

[0031] Based on the above technical solutions, preferably, step S3 specifically includes:

[0032] S31. Perform edge extraction on the marker point image to obtain the pixel-level contour of the marker point;

[0033] S32. Establish a window centered on any pixel of the pixel outline, and use surface fitting to fit the pixels within the window to obtain the sub-pixel coordinates of the pixel-level outline.

[0034] S33. Use the least squares method to perform ellipse fitting on the sub-pixel coordinates to obtain the center coordinates of the marker point.

[0035] Based on the above technical solutions, preferably, step S4 specifically includes:

[0036] S41. Calculate the center coordinates of each circular marker in the first marker image, and calculate the corresponding point of the center coordinates in the second marker image;

[0037] S42. For the set of center coordinates of the circles in the image of the first marker point {A i |i=1,2,3,……,N}, traverse the coordinate set {B} in the image of the second marker point. i For each |i=1,2,3,……,N}, calculate {B}. i The distance d from each point in |i=1,2,3,……,N} to the epipolar line of the first marker point is d i ;

[0038] S43, Determine the {B} i If the distance from each point in |i=1,2,3,……,N} to the epipolar line of the first marker point image exceeds the second set threshold, then the pixel is marked as a point to be matched.

[0039] S44. Find d at the point to be matched. i Pixel C in the image of the second marker point corresponding to the minimum value f The pixel C f For A in the first marker image i The corresponding point.

[0040] Secondly, the present invention provides a handheld three-dimensional scanning circular marker error correction system, employing the handheld three-dimensional scanning circular marker error correction method as described in any one of the above descriptions, including:

[0041] The calibration module is used to acquire images of the dot array calibration board using a handheld scanning device to obtain calibration images, and calculate calibration results based on the calibration images;

[0042] The optimization module is used to optimize the calibration results using an optimization method based on the point-to-epidial distance constraint, so as to obtain the optimized intrinsic and extrinsic parameter matrix of the camera.

[0043] An extraction module is used to acquire marker point images based on the calibration results and extract the center coordinates of the marker points using the marker point images; wherein, the marker point images include a first marker point image and a second marker point image;

[0044] The corresponding module is used to find the correspondence between circular marker points in the first marker point image and the second marker point image;

[0045] The 3D reconstruction module is used to perform 3D reconstruction of the circular marker points based on the correspondence between the marker points, and to obtain the 3D coordinate set {Q} of the circular marker points. i |i = 1, 2, 3, ..., N};

[0046] The elimination module is used to eliminate the 3D coordinate set {Q} of circular marker points using the depth information of the marker points. i Outliers in the group |i=1,2,3,……,N}.

[0047] Thirdly, the present invention provides an electronic device, comprising: at least one processor, at least one memory, a communication interface, and a bus; wherein,

[0048] The processor, memory, and communication interface communicate with each other through the bus;

[0049] The memory stores program instructions that can be executed by the processor, which invokes the program instructions to perform error correction on the handheld 3D scanning circular markers as described above.

[0050] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions that cause the computer to implement the error correction method for handheld three-dimensional scanning circular markers as described in any of the preceding claims.

[0051] The handheld 3D scanning circular marker correction method of the present invention has the following advantages over the prior art:

[0052] Beneficial effects:

[0053] (1) By applying the error correction method of circular markers to handheld scanning devices, the accuracy of scanning circular markers was improved. Based on the results of camera calibration of intrinsic and extrinsic parameters, optimization was carried out using the distance from the corresponding point in the image to the epipolar line as a constraint, which improved the accuracy of intrinsic and extrinsic parameters and ensured the accuracy of finding the corresponding point using the camera's basic matrix.

[0054] (2) By utilizing depth information and slope judgment methods, abnormal points are accurately identified and eliminated, thereby eliminating the interference of abnormal data on the scanning results, further ensuring the correctness of the scanning data throughout the scanning process, and improving the robustness of the handheld scanning device. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart of the handheld three-dimensional scanning circular marker correction method of the present invention;

[0057] Figure 2 This is a schematic diagram of epipolar geometric constraints in the handheld three-dimensional scanning circular marker correction method of the present invention. Detailed Implementation

[0058] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0059] like Figure 1 As shown, the present invention provides a method for correcting errors of circular marker points using a handheld 3D scanning device, comprising the following steps:

[0060] S1. Use a handheld scanning device to acquire images of the dot array calibration board to obtain calibration images, and calculate the calibration results based on the calibration images.

[0061] In this embodiment, the handheld scanning device is equipped with a left camera and a right camera. A dot array calibration plate is selected as the calibration target to ensure that both cameras can simultaneously acquire images of the calibration plate. The dot array calibration plate is positioned facing the device and near the focusing positions of the left and right cameras. Images of the dot array calibration plate in different placement positions are acquired by the left and right cameras respectively, avoiding image distortion and motion blur. For each pose of the calibration plate image, its image coordinates are mapped to world coordinates, and the reprojection error is calculated to evaluate the accuracy of the calibration results.

[0062] Specifically, in this embodiment, a left camera and a right camera are used to acquire images of the dot array calibration board. The dot array calibration board is placed facing the device and near the focusing position of the left and right cameras. Images of the dot array calibration board at different placement positions are acquired by the left and right cameras, and the image coordinates of the calibration board at each placement pose are calculated. The world coordinates of each point on the dot array calibration board used in the calibration have been measured in advance using high-precision equipment and are known quantities. The image coordinates and world coordinates of the calibration board at each pose are substituted into the camera model, and the intrinsic parameters, distortion, and rotation and translation matrix between the left and right cameras are calculated using a nonlinear optimization method. The calculation formula is as follows:

[0063]

[0064] Where, m c The coordinates of the dot array image under each placement posture, m k Let X represent the image coordinates after the world coordinates of the dot array calibration plate are projected onto the camera model, K represent the camera intrinsic parameter matrix, k represent the lens distortion coefficient, r represent the rotation matrix from the world coordinate system of the dot array calibration plate to the camera coordinate system, and t represent the translation matrix from the world coordinate system of the dot array calibration plate to the camera coordinate system. W This represents the world coordinates of the dot array calibration plate.

[0065] S2. The calibration results are optimized using an optimization method based on the point-to-epidial distance constraint to obtain the optimized intrinsic and extrinsic parameter matrix of the camera.

[0066] like Figure 2As shown, calibration is performed using a left and a right camera. I represents the image from the left camera, I' represents the image from the right camera, and M is any point on the object under test. The projections of M onto the left camera image I and the right camera image I' are m and m', respectively. L and R represent the centers of the left and right cameras, respectively. The line connecting the centers of the left and right cameras intersects the left camera image I and the right camera image I' at poles e and e', respectively. The intersection lines l and l' of the MLR plane with the left and right camera images I and I' are the epipolar lines of point M on planes I and I', respectively. The line l' is the epipolar line of point m on the right camera, and the line l is the epipolar line of m' on the left camera. According to the epipolar geometric constraint, the matching point m' of point m in another image must lie on the epipolar line of that point. The epipolar geometric constraint is calculated by determining the fundamental matrix F between the left and right cameras. The fundamental matrix F is a non-invertible 3×3 matrix, obtained from the calibration results between the left and right cameras.

[0067] Specifically, the handheld scanning device is equipped with a left camera and a right camera, and the calibration image includes a first calibration plate image and a second calibration plate image. Step S2 specifically includes:

[0068] S21. Based on the first calibration board image and the second calibration board image, obtain the intrinsic and extrinsic parameter matrices and distortion matrices of the left and right cameras, respectively. The intrinsic parameter matrix includes the camera's focal length, principal point coordinates, and pixel scaling factor; the distortion matrix includes radial distortion and tangential distortion parameters.

[0069] S22. Based on the intrinsic and extrinsic parameter matrices and distortion matrices of the left and right cameras, calculate the fundamental matrix between the left and right cameras. The fundamental matrix describes the geometric relationship between the cameras and can be used for feature point matching and 3D reconstruction in stereo vision.

[0070] Furthermore, the formula for calculating the basic matrix between the left and right cameras mentioned in step S22 is as follows:

[0071]

[0072]

[0073] Among them, K -1 L K represents the intrinsic parameter matrix of the left camera. L The inverse matrix, K -T R K represents the intrinsic parameter matrix of the right camera. R The transpose of the inverse matrix, R represents the rotation matrix between the left and right cameras, T(t) x , t y , t z ) represents the three components of the translation matrix between the left and right cameras, and S is the middle matrix.

[0074] S23. Based on the correspondence between the pixel coordinates of the dots in the dot array calibration board images acquired by the left and right cameras, the first calibration board image, the second calibration board image, and the basic matrix are used as initial values. The distance between corresponding points in the first and second calibration board images is used as the optimization objective function to optimize the matrix, thereby obtaining the optimized intrinsic and extrinsic parameter matrix.

[0075] Specifically, the formula for calculating the objective function in step S23 is as follows:

[0076] m i '×F×m i =0

[0077] Where, m i and m i ' represents the coordinates of the corresponding point in the dot array calibration board image acquired by the left and right cameras, respectively, and i represents the corresponding point number.

[0078] Understandably, in this embodiment, camera calibration tools (such as OpenCV) are used for camera calibration. By capturing a series of known calibration board images, the intrinsic parameter matrix and distortion matrix of the camera are calculated using a calibration algorithm. By accurately calculating the intrinsic parameter matrix and distortion matrix of the camera, as well as the fundamental matrices between cameras, the accuracy of the stereo vision calibration results can be improved.

[0079] S3. Acquire marker point images based on the calibration results, and extract the center coordinates of the marker points using the marker point images; wherein, the marker point images include a first marker point image and a second marker point image.

[0080] Understandably, the first calibration board image is the calibration image captured by the left camera, and the second calibration board image is the calibration image captured by the right camera. Correspondingly, the first marker point image is captured by the left camera, and the second marker point image is captured by the right camera.

[0081] In this embodiment, based on the calibration results, a calibrated camera is used to capture an image of the first marker point. Image processing, such as noise reduction and filtering, is then performed on the first marker point image to improve edge detection. An edge detection algorithm (such as Canny edge detection) is used to extract the edge pixel coordinates of the marker point in the first marker point image. The pixel-level edge coordinates are traversed, and a surface fitting method is used within a selected window to calculate the sub-pixel coordinates, improving edge localization accuracy. The sub-pixel coordinates are then fitted with an ellipse to obtain the center coordinates of the marker point. The aforementioned steps are repeated to capture an image of the second marker point using the calibrated camera, and the center coordinates of the marker point in the second marker point image are extracted.

[0082] Specifically, step S3 includes:

[0083] S31. Perform edge extraction on the marker image to obtain the pixel-level contours of the markers. The Canny edge detection algorithm or other edge detection algorithms can be used to extract the edges of the markers.

[0084] S32. Establish a window centered on any pixel of the pixel outline. Use surface fitting to fit the pixels within the window to obtain the sub-pixel coordinates of the pixel-level outline. Fitting algorithms, such as least squares or Gaussian fitting, can be used to perform surface fitting on the pixels within the window.

[0085] S33. Use the least squares method to perform ellipse fitting on the sub-pixel coordinates to obtain the center coordinates of the marker point. The least squares method or other ellipse fitting algorithms can be used to fit the sub-pixel coordinates into an ellipse, and the center coordinates of the ellipse can be extracted as the center coordinates of the marker point.

[0086] This application embodiment performs edge extraction based on the marker point image. By utilizing sub-pixel coordinates and ellipse fitting methods, the positioning accuracy of the marker point center coordinates can be improved, and the error caused by the discreteness of pixel-level contours can be reduced. Using surface fitting and ellipse fitting methods can improve the stability of the marker point center coordinates and reduce the error caused by noise and deformation.

[0087] S4. Find the correspondence between circular markers in the first marker image and the second marker image.

[0088] In this embodiment of the application, by finding the correspondence between circular marker points in the first marker point image and the second marker point image, matching point pairs of marker points are found, providing a basis for further calculation of the spatial three-dimensional coordinates of the marker points.

[0089] Specifically, step S4 includes:

[0090] S41. Calculate the center coordinates of each circular marker in the first marker image, and calculate the corresponding point of the center coordinates in the second marker image.

[0091] In this embodiment, the circular markers in the first marker image are detected and located. Image processing algorithms (such as edge detection methods) can be used to extract the edge image coordinates of the circular markers. Further, sub-pixel edge solving and ellipse fitting methods are used to determine the center coordinates of the circular markers. This process is repeated for the second marker image to obtain the center coordinates of the circular points in the second marker image.

[0092] S42. For the set of center coordinates of the circles in the image of the first marker point {A i|i=1,2,3,……,N}, traverse the coordinate set {B} in the image of the second marker point. i For each |i=1,2,3,……,N}, calculate {B}. i The distance d from each point in |i=1,2,3,……,N} to the epipolar line of the first marker point is d i Among them, A i B represents the coordinates of the i-th circle center in the first marker point image. i This represents the coordinates of the i-th circle center in the image of the second marker point.

[0093] S43, Determine the {B} i If the distance from each point in |i=1,2,3,……,N} to the epipolar line of the first marker point image exceeds the second set threshold, then the pixel is marked as a point to be matched.

[0094] Understandably, the second threshold can be set according to actual needs, when d i If the value exceeds the second set threshold, it is considered a mismatch, and d is discarded. i The corresponding point in the second marker image.

[0095] S44. Find d at the point to be matched. i Pixel C in the image of the second marker point corresponding to the minimum value f The pixel C f For A in the first marker image i The corresponding point.

[0096] This application improves the accuracy of image matching by calculating the center coordinates of each circular marker in the first marker image and finding the corresponding pixel in the second marker image.

[0097] S5. Based on the correspondence of the marker points, perform three-dimensional reconstruction of the circular marker points to obtain the three-dimensional coordinate set {Q} of the circular marker points. i |i=1,2,3,……,N}.

[0098] In this embodiment, the three-dimensional coordinates of the circular markers under the current viewpoint can be calculated using the calibration parameters of the left and right cameras, the center coordinates of the corresponding circular markers in the first and second marker images, and the triangular relationship of binocular imaging.

[0099] Specifically, step S5 includes:

[0100] S51. Based on the calibration parameters of the left and right cameras, and the center coordinates of the corresponding circular markers in the first and second marker images, the three-dimensional coordinates of the circular markers under the current view can be calculated. The calibration parameters include the intrinsic parameter matrix, distortion coefficients, and extrinsic parameter matrix.

[0101] S52. Based on the intrinsic parameter matrices and distortion coefficients of the left and right cameras, perform distortion correction processing on the center coordinates of the circles in the first and second marker point images to obtain the distortion-corrected coordinates.

[0102] S53. Based on the extrinsic parameter matrices of the left and right cameras, transform the distorted coordinates in the first and second marker point images to the camera coordinate system.

[0103] S54. By utilizing the triangular relationship of binocular imaging and the light convergence method of the left and right cameras, the three-dimensional coordinates of the circular marker point under the current viewpoint can be calculated.

[0104] S55. Repeat the above steps to perform 3D reconstruction on each circular marker point, obtaining the 3D coordinate set {Q} of the circular marker points. i |i=1,2,3,……,N}.

[0105] This application utilizes the calibration parameters of the left and right cameras and the triangulation relationship of binocular imaging to calculate the three-dimensional coordinates of each circular marker point in the current viewpoint, avoiding the depth estimation errors and uncertainties existing in traditional monocular image-based methods, and improving the accuracy of three-dimensional reconstruction.

[0106] S6. Use the depth information of the marker points to eliminate the three-dimensional coordinate set {Q} of the circular marker points. i Outliers in the group |i=1,2,3,……,N}.

[0107] In this embodiment, the method for optimizing the acquisition of corresponding points using point-to-epidial distance constraints may still encounter situations where two very close adjacent points show incorrect correspondences from certain individual viewpoints. Therefore, erroneous points are removed based on the depth information of the reconstructed center marker. In a handheld device scanning a local viewpoint, the scanning field of view is limited, and the marker points change continuously and gradually in the depth direction. When a point is reconstructed due to an incorrect correspondence, its 3D coordinates will exhibit a clear "protrusion" or "concavity" in the depth direction compared to surrounding points. By calculating the slope of the nearest neighbor point along the depth direction for the point in the current viewpoint, points with larger slopes are considered abnormal and deleted to improve the accuracy and reliability of the data.

[0108] Specifically, step S6 includes:

[0109] S61, Based on the three-dimensional coordinate set {Q} of the circular marker point i Calculate the slope of the nearest neighbor point of the circular marker in the depth direction, given the range |i=1,2,3,……,N}.

[0110] Furthermore, select a reference point Q. ref, can be a circular marker point in the three-dimensional coordinate set of circular marker points, serving as the reference point for calculating the slope; for each circular marker point Q i Calculate its relationship with Q in the depth direction. ref The slope is given by the following formula: The slope k is calculated using the following formula:

[0111]

[0112] Among them, Q i_z Represents the circular marker point Q i The depth value, Q ref_z Representing reference point Q ref The depth value, ||Q i_z -Q ref_d || represents Q i With Q ref The distance between two points.

[0113] S62. Determine whether the slope of the nearest point is greater than a first set threshold. If so, mark the circular marker point as an abnormal point.

[0114] Understandably, the first threshold can be set according to the actual situation. When the slope of the nearest neighbor point is greater than the first threshold, the circular marker point corresponding to the slope of the nearest neighbor point is marked as an anomaly.

[0115] S63. Delete the abnormal points. That is, remove the coordinates of all abnormal points from the three-dimensional coordinate set {Q}. i Remove from |i=1,2,3,……,N} to improve the accuracy and reliability of the data.

[0116] S64. Repeat steps S61-S63 until the three-dimensional coordinate set of the circular marker point {Q} is reached. i We proceed until there are no outliers in the set |i = 1, 2, 3, ..., N}. Through multiple iterations, we can ensure that all outliers are removed, thus obtaining a set of three-dimensional coordinates of circular marker points after outlier removal.

[0117] In this embodiment, by utilizing depth information and slope judgment, we can more accurately identify and remove outliers, thereby improving data accuracy. Furthermore, through multiple iterative operations, outliers can be gradually removed, thus improving algorithm robustness. This ensures that the 3D coordinates of the circular markers obtained by the handheld scanning device are more precise.

[0118] This application also provides a handheld 3D scanning circular marker error correction system, employing the handheld 3D scanning circular marker error correction method as described in any of the above descriptions, including:

[0119] The calibration module is used to acquire images of the dot array calibration board using a handheld scanning device to obtain calibration images, and calculate calibration results based on the calibration images;

[0120] The optimization module is used to optimize the calibration results using an optimization method based on the point-to-epidial distance constraint, so as to obtain the optimized intrinsic and extrinsic parameter matrix of the camera.

[0121] An extraction module is used to acquire marker point images based on the calibration results and extract the center coordinates of the marker points using the marker point images; wherein, the marker point images include a first marker point image and a second marker point image;

[0122] The corresponding module is used to find the correspondence between circular marker points in the first marker point image and the second marker point image;

[0123] The 3D reconstruction module is used to perform 3D reconstruction of the circular marker points based on the correspondence between the marker points, and to obtain the 3D coordinate set {Q} of the circular marker points. i |i = 1, 2, 3, ..., N};

[0124] The elimination module is used to eliminate the 3D coordinate set {Q} of circular marker points using the depth information of the marker points. i Outliers in the group |i=1,2,3,……,N}.

[0125] This application uses the calibration module of a handheld scanning device to acquire images of a circular array calibration board for camera calibration. An optimization module is used to optimize the calibration results, improving the camera's calibration accuracy. An extraction module processes the first and second marker point images to extract the center coordinates of the marker points. A correspondence module finds the corresponding circular marker points in the first and second marker point images. A 3D reconstruction module performs 3D reconstruction of the circular marker points based on the correspondence, obtaining a set of 3D coordinates for the circular marker points. An outlier module removes outliers from the set of 3D coordinates of the circular marker points, making the 3D reconstruction results of the circular marker points more accurate, precise, reliable, and efficient.

[0126] This application also provides an electronic device, including: at least one processor, at least one memory, a communication interface, and a bus; wherein the processor, memory, and communication interface communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement error correction of handheld three-dimensional scanning circular markers as described in any of the above claims.

[0127] This application also provides a computer-readable storage medium storing computer instructions that cause the computer to implement the error correction method for handheld 3D scanning circular markers as described in any of the preceding claims.

[0128] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for correcting errors in circular marker points using a handheld 3D scanning device, characterized in that, Includes the following steps: S1. Use a handheld scanning device to acquire images of the dot array calibration board to obtain calibration images, and calculate the calibration results based on the calibration images; S2. The calibration results are optimized using an optimization method based on the point-to-epidial distance constraint to obtain the optimized intrinsic and extrinsic parameter matrix of the camera. S3. Acquire marker point images based on the calibration results, and extract the center coordinates of the marker points using the marker point images; wherein, the marker point images include a first marker point image and a second marker point image; S4. Find the correspondence between circular marker points in the first marker point image and the second marker point image; S5. Based on the correspondence of the marker points, perform three-dimensional reconstruction of the circular marker points to obtain the three-dimensional coordinate set {Q} of the circular marker points. i |i = 1, 2, 3, ..., N}; S6. Use the depth information of the marker points to eliminate the three-dimensional coordinate set {Q} of the circular marker points. i Outliers in the group |i = 1, 2, 3, ..., N; Step S6 specifically includes: S61, Based on the three-dimensional coordinate set {Q} of the circular marker point i Calculate the slope of the nearest neighbor point of the circular marker in the depth direction for the circular marker point |i=1,2,3,……,N}; S62. Determine whether the slope of the nearest point is greater than a first set threshold. If so, mark the circular marker point as an abnormal point. S63. Delete the abnormal points; S64. Repeat steps S61-S63 until the three-dimensional coordinate set of the circular marker point {Q} is reached. i There are no outliers in the group |i=1,2,3,……,N}.

2. The error correction method for circular marker points using a handheld 3D scanning device as described in claim 1, characterized in that, The handheld scanning device is equipped with a left camera and a right camera. The calibration image includes a first calibration plate image and a second calibration plate image. Step S2 specifically includes: S21. Based on the first calibration board image and the second calibration board image, obtain the intrinsic and extrinsic parameter matrices and distortion matrices of the left and right cameras, respectively; S22. Calculate the fundamental matrix between the left and right cameras based on the intrinsic and extrinsic parameter matrices and distortion matrix of the left and right cameras; S23. Based on the correspondence between the pixel coordinates of the dots in the dot array calibration board images acquired by the left and right cameras, the first calibration board image, the second calibration board image, and the basic matrix are used as initial values. The distance between corresponding points in the first and second calibration board images is used as the optimization objective function to optimize the matrix, thereby obtaining the optimized intrinsic and extrinsic parameter matrix.

3. The error correction method for circular marker points using a handheld 3D scanning device as described in claim 2, characterized in that, The formula for calculating the basic matrix between the left and right cameras in step S22 is as follows: Among them, K -1 L K represents the intrinsic parameter matrix of the left camera. L The inverse matrix, K -T R K represents the intrinsic parameter matrix of the right camera. R The transpose of the inverse matrix, R represents the rotation matrix between the left and right cameras, T(t) x , t y , t z ) represents the three components of the translation matrix between the left and right cameras, and S represents the middle matrix.

4. The error correction method for circular marker points using a handheld 3D scanning device as described in claim 3, characterized in that, The formula for calculating the objective function in step S23 is as follows: m i ' ×F×m i =0; Where, m i and m i ' represents the coordinates of the corresponding point in the dot array calibration board image acquired by the left and right cameras, respectively, and i represents the corresponding point number.

5. The error correction method for circular marker points using a handheld 3D scanning device as described in claim 1, characterized in that, Step S3 specifically includes: S31. Perform edge extraction on the marker point image to obtain the pixel-level contour of the marker point; S32. Establish a window centered on any pixel of the pixel outline, and use surface fitting to fit the pixels within the window to obtain the sub-pixel coordinates of the pixel-level outline. S33. Use the least squares method to perform ellipse fitting on the sub-pixel coordinates to obtain the center coordinates of the marker point.

6. The error correction method for circular marker points using a handheld 3D scanning device as described in claim 2, characterized in that, Step S4 specifically includes: S41. Calculate the center coordinates of each circular marker in the first marker image, and calculate the corresponding point of the center coordinates in the second marker image; S42. For the set of center coordinates of the circles in the image of the first marker point {A i |i=1,2,3,……,N}, traverse the coordinate set {B} in the image of the second marker point. i For each |i=1,2,3,……,N}, calculate {B}. i The distance d from each point in |i=1,2,3,……,N} to the epipolar line of the first marker point is d i ; S43, Determine the {B} i If the distance from each point in |i=1,2,3,……,N} to the epipolar line of the first marker point image exceeds the second set threshold, then the pixel is marked as a point to be matched. S44. Find d at the point to be matched. i Pixel C in the image of the second marker point corresponding to the minimum value f The pixel C f For A in the first marker image i The corresponding point.

7. A handheld 3D scanning circular marker error correction system, characterized in that, The error correction method for circular markers using a handheld 3D scanning device as described in any one of claims 1 to 6 includes: The calibration module is used to acquire images of the dot array calibration board using a handheld scanning device to obtain calibration images, and calculate calibration results based on the calibration images; The optimization module is used to optimize the calibration results using an optimization method based on the point-to-epidial distance constraint, so as to obtain the optimized intrinsic and extrinsic parameter matrix of the camera. An extraction module is used to acquire marker point images based on the calibration results and extract the center coordinates of the marker points using the marker point images; wherein, the marker point images include a first marker point image and a second marker point image; The corresponding module is used to find the correspondence between circular marker points in the first marker point image and the second marker point image; The 3D reconstruction module is used to perform 3D reconstruction of the circular marker points based on the correspondence between the marker points, and to obtain the 3D coordinate set {Q} of the circular marker points. i |i = 1, 2, 3, ..., N}; The elimination module is used to eliminate the 3D coordinate set {Q} of circular marker points using the depth information of the marker points. i Outliers in the group |i=1,2,3,……,N}.

8. An electronic device, characterized in that, include: At least one processor, at least one memory, a communication interface, and a bus; wherein, The processor, memory, and communication interface communicate with each other through the bus; The memory stores program instructions that can be executed by the processor, which calls the program instructions to implement error correction of the handheld three-dimensional scanning circular markers as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that enable the computer to implement the error correction method for handheld three-dimensional scanning circular markers as described in any one of claims 1 to 6.

Images