A method and system for non-overlapping 3D point cloud stitching based on a calibration board

Abstract

This invention provides a method and system for stitching non-overlapping 3D point clouds based on a calibration plate. The product includes M non-overlapping borders. The calibration plate designed in this invention includes M calibration borders, each with at least three non-collinear calibration spheres. The positions of the M calibration borders and the M non-overlapping borders are identical. Non-overlapping borders refer to the absence of overlapping areas between the 3D point cloud data of the borders acquired by 3D sensors from multiple fields of view. Based on the calibration plate, this invention achieves coordinate system unification for the multi-sensor measurement system through a coordinate unification algorithm, solving the problem of non-overlapping areas preventing high-precision stitching when multiple high-precision small-field-of-view 3D sensors measure the product borders at different orientations. Furthermore, the affine transformation matrix for point cloud stitching is further optimized through auxiliary holes and an exhaustive iterative algorithm, ultimately achieving high-precision stitching of 3D point clouds.

Description

Technical Field

[0001] This invention belongs to the field of 3D point cloud stitching technology, and more specifically, relates to a method and system for non-overlapping 3D point cloud stitching based on a calibration plate. Background Technology

[0002] With the booming development of my country's manufacturing industry, product quality inspection has become increasingly important. Automated product quality inspection systems are crucial for enterprise development, helping companies eliminate defective products, provide timely feedback on processing results, and improve production efficiency. Currently, 3D modeling technology is widely used in industrial automated inspection. By using 3D measuring equipment to acquire 3D surface data of products and comparing it with the actual CAD model of the product being tested, the production and processing quality of the product can be inspected. More importantly, it provides a reference and evaluation standard for enterprises to innovate their production processes.

[0003] When acquiring 3D point cloud data for high-precision industrial products, a multi-sensor vision system is required due to the limited field of view of 3D sensors. Since different 3D sensors have their own coordinate systems, existing coordinate unification algorithms require different 3D sensors to measure the same control point, obtaining the coordinate representation of the control point in different coordinate systems, and then unifying the coordinate systems based on the coordinate transformation of the control point.

[0004] However, when existing products collect 3D point cloud data, there are non-overlapping borders on the product. Since there is no overlapping area between the non-overlapping borders, different 3D sensors cannot directly use the point cloud data to perform high-precision 3D point cloud stitching of the product after collecting 3D point cloud data from different borders. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for stitching non-overlapping 3D point cloud data based on a calibration board, thereby solving the problem that existing non-overlapping 3D point cloud data cannot be directly stitched together.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a method for non-overlapping 3D point cloud stitching based on a calibration plate, comprising the following steps:

[0007] The calibration plate is determined based on the border distribution of the surface point cloud data product to be collected; the product includes M non-overlapping borders, the calibration plate includes M calibration borders, and at least 3 non-collinear calibration spheres are placed on each calibration border. The positions of the M calibration borders and the M non-overlapping borders are identical, and M is an integer greater than 1; the non-overlapping borders refer to the absence of overlapping areas between the border 3D point cloud data collected by 3D sensors under multiple fields of view.

[0008] The point cloud data of the spherical surface of all calibration spheres is obtained by using a three-dimensional coordinate measuring instrument when the calibration plate is in a preset position, and the first center coordinates of each calibration sphere are obtained by fitting.

[0009] Based on a three-dimensional sensor, the spherical point cloud data of all calibration spheres on a calibration frame when the calibration plate is at the preset position is obtained, and the second sphere center coordinates of all calibration spheres on the calibration frame are obtained by fitting.

[0010] Based on the first and second center coordinates of all calibration spheres on a certain calibration frame, the affine transformation matrix between the three-dimensional sensor coordinate system for measuring the calibration frame and the three-dimensional coordinate measuring instrument coordinate system is obtained;

[0011] Determine the multiple 3D sensors that collect point cloud data for each calibration bounding box and the corresponding affine transformation matrix;

[0012] Based on the multiple 3D sensors, the 3D point cloud data of each border of the product at the preset position is collected, and the collected 3D point cloud data is transformed to the coordinate system of the 3D coordinate measuring instrument based on the corresponding affine transformation matrix so as to stitch together the 3D point cloud data of the product border field of view that do not overlap.

[0013] It should be noted that the calibration sphere is a high-standard sphere, and its precision and material will affect the splicing accuracy. Low-expansion, hard materials, such as ceramics, are preferred. The machining precision of this sphere must also match the splicing accuracy requirements.

[0014] In one possible implementation, when there are parallel borders under the M non-overlapping borders, a through hole is added to each of the parallel calibration borders corresponding to the M calibration borders, and the line connecting the centers of all the through holes on the parallel calibration borders is perpendicular to the border where the through hole is located.

[0015] After determining the affine transformation matrix based on the coordinates of the first and second centers of all calibration spheres on a certain calibration border, the following steps are also included:

[0016] The point cloud data of all the penetrating circular holes on each set of parallel calibration frames are obtained based on the three-dimensional sensor when the calibration plate is in the preset position. The coordinates of the center of the penetrating circular hole are obtained by performing circular fitting based on the point cloud data of the circular hole. The alignment error of the center is determined according to the center coordinates fitted inside and outside the penetrating circular hole.

[0017] Based on the alignment error, the affine transformation matrix of the corresponding parallel calibration border is compensated for the first time to obtain the affine transformation matrix after the first compensation, which is then used as the affine transformation matrix of the corresponding calibration border after the first compensation.

[0018] It is understandable that the high-precision machining of the aforementioned circular holes will affect the splicing accuracy. This requires ensuring that the diameter error, roundness, perpendicularity, and cylindricity of the circular holes match the splicing accuracy requirements.

[0019] In one possible implementation, the affine transformation matrix is ​​determined based on the coordinates of the first and second centers of all calibration spheres on a certain calibration border, specifically as follows:

[0020] First, let A, B, and C represent the coordinates of the centers of the three calibration spheres in the coordinate system of the 3D coordinate measuring machine. Let a, b, and c represent the coordinates of the centers of the three calibration spheres in the coordinate system of the 3D sensor. Let H be the affine transformation matrix from the 3D sensor coordinate system to the 3D coordinate measuring machine coordinate system. Then we can obtain:

[0021] H = P' × P -1

[0022] in: Let D be a point on the extension of the center A of the sphere, and let d be a point on the extension of the center a of the sphere, where:

[0023] In one possible implementation, after the three-dimensional coordinate measuring instrument and the three-dimensional sensor acquire the spherical point cloud data of the calibration sphere, the spherical point cloud data is denoised, and then the sphere center coordinates are obtained by fitting based on the denoised spherical point cloud data. The denoising of the spherical point cloud data employs a statistical filtering-based outlier removal algorithm, and the specific steps are as follows:

[0024] Calculate the average μ and standard deviation σ of the distance between each point and any other point. Let the standard deviation factor be std. If the distance between the point cloud data and all its neighboring points is within (μ-σ·std, μ+σ·std), retain the point; otherwise, consider it an outlier and remove it.

[0025] In one possible implementation, the coordinates of the center of the calibration sphere are obtained by fitting the data of multiple points in the spherical point cloud data to the sphere using the least squares method.

[0026] In one possible implementation, when determining the center coordinates of the calibrated sphere based on the spherical point cloud data, an exhaustive iterative algorithm is used to fine-tune the top point cloud data, reducing the stitching error of the 3D point cloud data under different views, and determining the corresponding stitching error compensation value. Based on the stitching error compensation value, a second compensation is performed on the affine transformation matrix after the first compensation, resulting in a second-compensated affine transformation matrix, which is used as the final affine transformation matrix. The stitching error refers to the Euclidean distance between the center coordinates of the sphere fitted from the 3D point cloud data of the sphere under different views and the center coordinates of the sphere measured by the 3D coordinate measuring instrument.

[0027] The exhaustive iterative algorithm is as follows: the top spherical point cloud data is finely adjusted along the z-axis with a fixed step size. After each fine adjustment, the error of fitting the sphere center coordinates of the spherical point cloud data is recalculated. If the error value decreases, the iteration continues; if the error value increases, the iteration terminates and the sum of the iteration steps is returned. The sum of the iteration steps is used as the stitching error compensation value.

[0028] In one possible implementation, the calibration plate is machined with an inclined surface, and the point cloud data on the inclined surface can be scanned by at least two three-dimensional sensors with different fields of view;

[0029] The method further includes the following steps:

[0030] For the 3D point cloud data collected on the calibration board, using the data from the inclined plane collected by one 3D sensor as the reference and its normal vector as the reference vector, and the normal vectors of the data from the inclined plane collected by other 3D sensors as the non-reference vectors, the angle θ between the reference vector and the non-reference vector is calculated based on the 3D point cloud data on the calibration board; and the reference vector is set as... Non-reference vector is pass Cross product Obtain the rotation axis vector

[0031] For the collected 3D point cloud data of the product, let the product bounding box point cloud dataset corresponding to the non-reference vector of the product be U={u1,u2,...,u i The point cloud dataset U is the result set after transformation by the affine transformation matrix;

[0032] Rotate all points in the point cloud dataset U around the axis vector. After rotating by an angle θ, we obtain a further optimized point cloud dataset V = {v1, v2, ..., v}. i According to Rodriguez's rotation formula, we have:

[0033]

[0034] Secondly, the present invention provides a non-overlapping 3D point cloud stitching system based on a calibration board, comprising:

[0035] A calibration plate determination unit is used to determine a calibration plate based on the border distribution of the surface point cloud data product to be collected. The product includes M non-overlapping borders, and the calibration plate includes M calibration borders. At least 3 non-collinear calibration spheres are placed on each calibration border. The positions of the M calibration borders and the M non-overlapping borders coincide one-to-one, where M is an integer greater than 1. The non-overlapping borders refer to the absence of overlapping areas between the border 3D point cloud data collected by 3D sensors under multiple fields of view.

[0036] The sphere center coordinate determination unit is used to acquire spherical point cloud data of all calibration spheres when the calibration plate is in a preset position based on a three-dimensional coordinate measuring instrument, so as to fit and obtain the first sphere center coordinate of each calibration sphere; and to acquire spherical point cloud data of all calibration spheres on a calibration frame when the calibration plate is in the preset position based on a three-dimensional sensor, so as to fit and obtain the second sphere center coordinate of all calibration spheres on the calibration frame.

[0037] The transformation matrix determination unit is used to obtain the affine transformation matrix between the three-dimensional sensor coordinate system for measuring the calibration frame and the three-dimensional coordinate measuring instrument coordinate system based on the first and second center coordinates of all calibration spheres on a calibration frame; and to determine multiple three-dimensional sensors that collect point cloud data for each calibration frame and their corresponding affine transformation matrices.

[0038] The three-dimensional point cloud data stitching unit is used to collect three-dimensional point cloud data of each border of the product at the preset position based on the multiple three-dimensional sensors, and transform the collected three-dimensional point cloud data to the coordinate system of the three-dimensional coordinate measuring instrument based on the corresponding affine transformation matrix so as to stitch together the three-dimensional point cloud data of the product border field of view without overlap.

[0039] In one possible implementation, when there are parallel borders under the M non-overlapping borders, the calibration plate determining unit adds a penetrating circular hole to each of the parallel calibration borders corresponding to the M calibration borders, and the line connecting the centers of all the penetrating circular holes on the parallel calibration borders is perpendicular to the border where the penetrating circular hole is located.

[0040] The system also includes:

[0041] The error compensation unit is used to acquire point cloud data of all penetrating circular holes on each set of parallel calibration frames when the calibration plate is in a preset position based on a three-dimensional sensor, and to obtain the coordinates of the center of the penetrating circular hole by performing circular fitting based on the point cloud data of the circular hole, and to determine the alignment error of the center based on the center coordinates fitted inside and outside the penetrating circular hole; and to perform the first compensation on the affine transformation matrix of the corresponding parallel calibration frame based on the alignment error, to obtain the affine transformation matrix after the first compensation, and to use it as the affine transformation matrix after the first compensation of the corresponding calibration frame.

[0042] In one possible implementation, the transformation matrix determination unit determines the affine transformation matrix based on the first and second center coordinates of all calibration spheres on a certain calibration border. Specifically: First, let A, B, and C represent the center coordinates of the three calibration spheres in the 3D coordinate measuring instrument coordinate system; let a, b, and c represent the center coordinates of the three calibration spheres in the 3D sensor coordinate system; and let H be the affine transformation matrix from the 3D sensor coordinate system to the 3D coordinate measuring instrument coordinate system. Then:

[0043] H = P' × P-1

[0044] in: Let D be a point on the extension of the center A of the sphere, and let d be a point on the extension of the center a of the sphere, where:

[0045] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any possible implementation thereof.

[0046] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0047] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0048] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0049] In summary, the technical solutions conceived by this invention have the following beneficial effects compared with the prior art:

[0050] This invention provides a method and system for non-overlapping 3D point cloud stitching based on a calibration plate. The invention designs a corresponding calibration plate and uses a coordinate unification algorithm to unify the coordinate system of a multi-sensor measurement system. This solves the problem that high-precision stitching is impossible when multiple high-precision small-field-of-view 3D sensors measure the product frame in different orientations because there is no overlapping area. Furthermore, the affine transformation matrix for point cloud stitching is further optimized through auxiliary holes and an exhaustive iterative algorithm. The final stitching error is below 0.04mm, meeting the requirements for high-precision stitching. Attached Figure Description

[0051] Figure 1 A flowchart of a non-overlapping 3D point cloud stitching method based on a calibration board provided in an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of the non-overlapping 3D point cloud stitching process provided in an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of the operation of the multi-sensor measurement system provided in an embodiment of the present invention;

[0054] Figure 4 A schematic diagram illustrating the principle of coordinate unification provided in this embodiment of the invention;

[0055] Figure 5 This is a schematic diagram of the ball calibration technology provided in an embodiment of the present invention;

[0056] Figure 6 This is a schematic diagram of a calibration plate provided in an embodiment of the present invention;

[0057] Figure 7 This is a point cloud segmentation effect diagram based on model fitting provided in an embodiment of the present invention;

[0058] Figure 8 The denoising effect diagram based on statistical filtering provided in the embodiment of the present invention;

[0059] Figure 9 This is a stitched image of non-overlapping 3D point cloud data provided in an embodiment of the present invention.

[0060] Figure 10 This is a schematic diagram showing the stitching of spherical point cloud data from the outer, inner, and top views of the three calibration spheres on the calibration board provided in an embodiment of the present invention.

[0061] Figure 11 This is a schematic diagram of the auxiliary holes on the calibration plate provided in an embodiment of the present invention;

[0062] Figure 12 This is a schematic diagram of the auxiliary inclined plane of the calibration plate provided in an embodiment of the present invention;

[0063] Figure 13 This is a diagram illustrating the architecture of a calibration board-based non-overlapping 3D point cloud stitching system provided in an embodiment of the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0065] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0066] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0067] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0068] Next, the technical solutions provided in the embodiments of this application will be described.

[0069] The purpose of this invention is to overcome the above-mentioned shortcomings and provide a design for a non-overlapping 3D point cloud stitching system based on a calibration plate. Using the calibration plate as an intermediate medium, it unifies the coordinate systems of 3D sensors in different orientations under one- to three-coordinate measuring machine coordinate systems. The same measurement system is used to acquire non-overlapping 3D point cloud data of the product to be stitched. Since the 3D sensor measurement group is in a fixed position and the motion path of the stage is also fixed, the transformation matrix between the coordinate systems calculated above can be applied to the corresponding 3D point cloud data of the product to be stitched, thus achieving the stitching of the non-overlapping 3D point cloud data of that product.

[0070] Figure 1 The flowchart of the non-overlapping 3D point cloud stitching method based on a calibration board provided in the embodiments of the present invention is as follows: Figure 1 As shown, it includes the following steps:

[0071] S101, determine the calibration plate based on the border distribution of the surface point cloud data product to be collected; the product includes M non-overlapping borders, the calibration plate includes M calibration borders, each calibration border has at least 3 non-collinear calibration spheres placed on it, the positions of the M calibration borders and the M non-overlapping borders coincide one-to-one, M is an integer greater than 1; the non-overlapping borders refer to the border 3D point cloud data collected by 3D sensors under multiple fields of view that do not have overlapping areas.

[0072] S102, Based on the three-dimensional coordinate measuring instrument, obtain the spherical point cloud data of all calibration spheres when the calibration plate is in the preset position, and then fit the first center coordinates of each calibration sphere to obtain the first center coordinates of each calibration sphere;

[0073] S103, Based on a three-dimensional sensor, acquire the spherical point cloud data of all calibration spheres on a calibration frame when the calibration plate is at the preset position, and fit it to obtain the second sphere center coordinates of all calibration spheres on the calibration frame;

[0074] S104, based on the first and second center coordinates of all calibration spheres on a certain calibration frame, obtain the affine transformation matrix between the three-dimensional sensor coordinate system for measuring the calibration frame and the three-dimensional coordinate measuring instrument coordinate system;

[0075] S105, determine the multiple 3D sensors that collect point cloud data for each calibration bounding box and the corresponding affine transformation matrix;

[0076] S106, based on the multiple three-dimensional sensors, collect the three-dimensional point cloud data of each border of the product at the preset position, and transform the collected three-dimensional point cloud data to the coordinate system of the three-dimensional coordinate measuring instrument based on the corresponding affine transformation matrix so as to stitch together the three-dimensional point cloud data of the product border field of view that do not overlap.

[0077] This invention provides an example of non-overlapping 3D point cloud stitching based on a calibration board, the implementation flowchart of which is shown below. Figure 2 As shown, the specific implementation includes the following steps:

[0078] Step one: Fix the spatial position of the 3D sensor measurement group so that the movement of the stage can acquire point cloud data of the outer, inner, and top sides of the product to be stitched. The principle is as follows: Figure 3 As shown.

[0079] Step two: Since there is no overlap between the 3D point cloud data of the product to be stitched obtained by the measurement system in step one, this invention uses a calibration plate to achieve coordinate unification of the multi-sensor vision system. For example... Figure 4 As shown, the measurement result of a point P in three-dimensional space in the coordinate system of the three-dimensional sensor module is P. i =(x i ,y i ,z i ) T The measurement result in the coordinate system of the coordinate measuring machine is P. o =(x o ,y o ,z o ) T Let the transformation formula from the 3D sensor coordinate system to the coordinate system of the coordinate measuring machine be as follows:

[0080] P O =R·P i +T

[0081] in Let T be a rotation matrix, where T = (t1, t2, t3). T Let be the translation vector. The above equation can be expressed in coordinates as:

[0082]

[0083] make This is the transformation matrix from the 3D sensor coordinate system to the coordinate system of the coordinate measuring machine. Based on the above, the designed calibration board needs to be able to measure its control points using the coordinate measuring machine, and also be able to extract the corresponding control points from the data measured by the 3D sensor.

[0084] Specifically, this invention uses a standard calibration ball as the target and the center of the standard ball as the control point. The basic principle of the ball calibration technology is as follows: Figure 5 As shown, when the 3D sensor scans and measures the calibration sphere, it can obtain the measurement results in the 3D sensor coordinate system. These results characterize the local coordinate information of the calibration sphere's surface. Based on the local coordinate data, the coordinates of the sphere's center can be calculated in the 3D sensor coordinate system. Simultaneously, by measuring the surface of the calibration sphere with a coordinate measuring machine, the coordinates of the sphere's center in the 3D coordinate measuring machine coordinate system can be calculated. Therefore, the correspondence between the sphere's center coordinates in the 3D sensor coordinate system and the 3D coordinate measuring machine coordinate system can be obtained. Since the transformation matrix H contains 12 unknown parameters, at least 3 sets of corresponding point relationships are required to calculate H. Therefore, 3 non-collinear standard calibration spheres are placed on the designed calibration plate, specifically as follows... Figure 6 As shown, the point cloud data of the calibration board from three perspectives were obtained using the above measurement system.

[0085] Step 3: Using algorithms such as point cloud segmentation and point cloud denoising, spherical point cloud data is extracted from the calibration board point cloud data. The least squares method is then used to fit the coordinates of the sphere's center. These fitted sphere center coordinates represent the coordinates of the product to be stitched together in the 3D sensor coordinate system from a certain perspective.

[0086] Specifically, the point cloud segmentation algorithm used in this invention is based on the RANSAC model fitting algorithm. The RANSAC-based algorithm has two main stages: the first stage is hypothesis generation; the second stage is model validation of the data. Since the calibration board point cloud data collected in step two mainly contains planar and spherical models, this invention separates the planar point cloud model and the spherical point cloud model based on model fitting, thereby extracting the spherical point cloud data, such as... Figure 7 This is the segmentation result.

[0087] Specifically, the point cloud denoising algorithm used in this invention is a statistical filtering-based outlier removal algorithm. Outlier noise points are far from the main point cloud and are mainly distributed around it. Unlike outliers, the main point cloud has a concentrated distribution and high density. Based on this, the statistical filtering algorithm analyzes and removes outliers by statistically analyzing the neighborhood of each point cloud data point. Let P be the coordinates of the nth point in the point cloud. n (x n y n , z n ), from this point to any point P m (x m y m , z m The distance is:

[0088]

[0089] The formula for calculating the average distance from each point to any other point is:

[0090]

[0091] The standard deviation is:

[0092]

[0093] Let std be the standard deviation factor. A point is retained if the distance between the point cloud data and all its neighboring points is within (μ-σ·std, μ+σ·std); otherwise, it is considered an outlier and removed. This invention uses this algorithm for denoising outlier noise points in spherical point clouds, such as... Figure 8 This is a comparison of the results before and after noise reduction.

[0094] Specifically, this invention employs the least squares method for spherical fitting. Let the radius of the sphere be R, and the approximate coordinates of the sphere's center be (x0, y0, z0). The coordinates of the i-th point on the sphere, measured by a three-dimensional sensor, are (x0, y0, z0). i ,y i ,z i If the coordinates of each point are related to the radius of the sphere by the following formula:

[0095] (x0-x i ) 2 +(y0-y i ) 2 +(z0-z i ) 2 =R 2

[0096] Expanded to:

[0097]

[0098] Let A = 2x0, B = 2y0, C = 2z0, The above equation can then be simplified to:

[0099]

[0100] Let E be the sum of squared errors to be minimized. Least squares fitting must ensure that E is minimized:

[0101]

[0102] Using the matrix method, we get:

[0103]

[0104] Multiply both sides of the above equation by the left side. get:

[0105]

[0106] get:

[0107]

[0108] The values ​​of A, B, C, and D can be obtained from the above formula. Then, the coordinates of the sphere's center and the sphere's radius can be calculated using the following relationship.

[0109]

[0110] Step four: Use a coordinate measuring machine (CMM) to measure the coordinates of the three standard calibration spheres on the calibration plate in the CMM coordinate system. Based on the sphere center coordinates fitted from the point cloud data of the calibration plate in Step three and the measurement results from the CMM, calculate the transformation matrix from the 3D sensor coordinate system to the CMM coordinate system. The coordinate representations of the three standard calibration spheres on the calibration plate in the 3D sensor coordinate system and the CMM coordinate system at a certain viewpoint are presented in Table 1.

[0111] Table 1. Coordinates of the center of the calibrated ball on the calibration plate.

[0112]

[0113] Specifically, this invention uses the "three-point method" to solve for the transformation matrix from the three-dimensional sensor coordinate system to the coordinate system of the coordinate measuring machine. Taking the data in Table 1 as an example, firstly, let the coordinates of points A, B, and C represent the coordinates of the center of the calibration sphere in the coordinate system of the coordinate measuring machine, let the coordinates of points a, b, and c represent the coordinates of the center of the calibration sphere in the three-dimensional sensor coordinate system, and let H be the transformation matrix from the three-dimensional sensor coordinate system to the coordinate system of the coordinate measuring machine. We can then obtain:

[0114] P′=H×P

[0115] H = P' × P-1

[0116] in:

[0117] Based on the above model, the following transformation matrix H is obtained through calculation:

[0118]

[0119] Step 5: Calculate the transformation matrix from the 3D sensor coordinate system to the coordinate system of the coordinate measuring machine (CMM) from the outer, inner, and top viewpoints, using the method in Step 4. Apply this transformation matrix to the 3D point cloud data of the product to be stitched, collected by the 3D sensors from the corresponding viewpoints. This allows for the stitching of 3D point cloud data of the product with non-overlapping fields of view. The final result is as follows: Figure 9 As shown, it includes point cloud data of the outer side of the product, point cloud data of the inner side of the product, and point cloud data of the top of the product.

[0120] Step six: Using the above method for stitching non-overlapping 3D point cloud data of the product to be stitched, stitch the point cloud data of the calibration board. For example... Figure 10 The diagram shows the stitched point cloud data of the three calibration spheres on the calibration plate from three perspectives: the outer side, the inner side, and the top. The coordinates of the sphere center were fitted based on the stitched point cloud data, and the Euclidean distance between the fitted center coordinates and the coordinate measuring machine measurement results was used as the stitching error value to quantitatively evaluate the stitching accuracy of the 3D point cloud data. Experiments were conducted on four groups of products, and the results are shown in Table 2.

[0121] Table 2 shows the fitting results of the spliced ​​spherical surface data and the measurement results of the coordinate measuring machine.

[0122]

[0123] Step 7: Based on the results in Table 2 above, it can be concluded that the stitching error of Product 2 is relatively large, with an overall error exceeding 0.075mm. Therefore, in order to achieve high-precision stitching of the non-overlapping 3D point cloud data of the products to be stitched, further optimization is needed based on the above stitching. This invention uses Product 2 as the optimization object and employs two optimization methods to reduce the stitching error of the 3D point cloud of Product 2.

[0124] Specifically, this invention addresses the alignment error between the outer and inner point cloud data of the product to be stitched by adding auxiliary penetrating circular holes to the calibration plate to optimize this alignment error, such as... Figure 11The diagram shows an auxiliary through-hole. The basic principle of the auxiliary hole optimization scheme is that after the through-hole is ideally stitched together, the line connecting the centers of the two holes scanned from the outer and inner sides should be perpendicular to the outer and inner planes. Therefore, by extracting the inner and outer holes and calculating the center coordinates of the two holes, the alignment error of the two holes can be obtained. By compensating for the error, the alignment error of the inner and outer sides can be optimized.

[0125] Specifically, when optimizing the auxiliary circular hole, this invention uses the coordinates of the outer circular hole's center as a reference value, calculates the deviation of the inner circular hole's center coordinates from the reference value, and finally performs error compensation.

[0126] In a specific example, the center coordinates of the inner and outer auxiliary holes were fitted using a circle fitting algorithm, yielding the following results:

[0127] The coordinates of the outer center are: (0.0047, 101.3532, -4.0179)

[0128] Coordinates of the inner center: (4.1484, 101.3323, -4.0134)

[0129] Calculate the alignment error of the circle's center:

[0130] Δy=|101.3532-101.3323|=0.0209mm

[0131] Δz1=|-4.0179+4.0134|=0.0045mm

[0132] Let the alignment error compensation parameter be δ. T1 Its value is:

[0133]

[0134] Therefore, the mathematical model for the direct unification algorithm of inner point cloud data coordinates is revised as follows:

[0135] P o =R·P i +T+δ T1

[0136] In the formula, R is the rotation matrix and T is the translation vector.

[0137] Specifically, this invention addresses the stitching error of the top point cloud data of the product to be stitched by employing an exhaustive iterative algorithm. The exhaustive iterative algorithm optimizes the top point cloud data, and its basic principle is to iteratively fine-tune the top spherical point cloud data along the z-axis based on the optimization achieved by the auxiliary holes. After each fine-tuning, the error in fitting the center coordinates of the three parts of the spherical point cloud data is recalculated. If the error converges (i.e., the error value decreases), the iteration continues; if the error does not converge (i.e., the error value increases), the iteration terminates, and the final optimized result is returned. The final optimized result is shown in Table 3.

[0138] Let the error compensation parameter be δ. T2 Then we have:

[0139]

[0140] Therefore, the mathematical model for the algorithm that directly unifies the coordinates of the top point cloud data is revised as follows:

[0141] P o =R·P i +T+δ T2

[0142] In the formula, R is the rotation matrix, T is the translation vector, and Δz2 is the convergence value of the sphere center coordinate error.

[0143] Table 3. Error results after optimization

[0144]

[0145]

[0146] Table 4 Comparison results before and after optimization

[0147]

[0148] Conclusion: Based on the results in Table 4, the average error of the optimized splicing result of product 2 is 0.0194 mm, which is 0.067 mm higher than the average accuracy of the splicing result before optimization. Finally, the splicing error of all four products is less than 0.04 mm.

[0149] Specifically, this invention fixes the spatial position of the three-dimensional sensor measurement group, enabling the acquisition of three-dimensional point cloud data from the outer, inner, and top sides of the product to be assembled through the movement of the stage. The principle is as follows: Figure 3 As shown. The 3D point cloud data of the product to be stitched obtained by the above measurement system have no overlapping areas, making it impossible to stitch 3D point clouds using the point cloud data itself. Therefore, this invention uses a calibration plate to unify the coordinates of the multi-sensor vision system, ultimately achieving the stitching of 3D point cloud data with non-overlapping fields of view of the product to be stitched. For example... Figure 4As shown, the measurement result of a point P in three-dimensional space in the coordinate system of the three-dimensional sensor module is P. i =(x i ,y i ,z i ) T The measurement result in the coordinate system of the coordinate measuring machine is P. o =(x o ,y o ,z o ) T Let the transformation formula from the 3D sensor coordinate system to the coordinate system of the coordinate measuring machine be as follows:

[0150] P o =R·P i +T

[0151] Based on the above, the designed calibration board needs to be able to measure its control points using a coordinate measuring machine, and also be able to extract the corresponding control points from the data measured by a 3D sensor. Therefore, this invention uses a standard calibration sphere as the target and the center of the standard sphere as the control point. The basic principle of sphere calibration technology is as follows: Figure 5 As shown, when the 3D sensor scans and measures the calibration sphere, it can acquire the measurement results in the 3D sensor coordinate system. These results characterize the local coordinate information of the sphere's surface. Based on the local coordinate data, the coordinates of the sphere's center in the 3D sensor coordinate system can be calculated. Simultaneously, by measuring the surface of the calibration sphere with a coordinate measuring machine (CMM), the coordinates of the sphere's center in the CMM coordinate system can be calculated. Therefore, the correspondence between the sphere's center coordinates in the 3D sensor coordinate system and the CMM coordinate system can be obtained.

[0152] Since the transformation matrix H contains 12 unknown parameters, at least three sets of control point correspondences are required to calculate H. Therefore, the designed calibration plate is equipped with three non-collinear standard calibration spheres, as shown in the figure below. Figure 6 As shown, the point cloud data of the calibration board from three perspectives were obtained using the above measurement system.

[0153] Spherical point cloud data was extracted from the calibration board point cloud data using algorithms such as point cloud segmentation and point cloud denoising. The coordinates of the sphere center corresponding to the spherical point cloud data were then fitted using the least squares method. These fitted sphere center coordinates represent the coordinates in the three-dimensional sensor coordinate system at a certain viewpoint.

[0154] The coordinates of three standard calibration spheres on the calibration plate in the coordinate system of the coordinate measuring machine were measured using a coordinate measuring machine. Based on the sphere center coordinates fitted from the point cloud data of the calibration plate and the measurement results of the coordinate measuring machine, the transformation matrix from the 3D sensor coordinate system to the coordinate measuring machine coordinate system was calculated.

[0155] The aforementioned transformation matrix is ​​applied to the 3D point cloud data of the product to be stitched from the corresponding viewpoint, unifying the 3D point cloud data of the product to be stitched into the coordinate system of the coordinate measuring machine. The same method is used for the 3D point cloud data of the product to be stitched from other viewpoints, ultimately achieving the stitching of 3D point cloud data of the product to be stitched without overlapping views.

[0156] The above-described method for stitching 3D point cloud data is used to stitch together the 3D point cloud data of the calibration board. The coordinates of the sphere's center are fitted from the stitched spherical point cloud data, and the Euclidean distance between the fitted center coordinates and the coordinate measuring machine measurement results is used as the stitching error value to quantitatively evaluate the stitching accuracy of the 3D point cloud data.

[0157] To achieve high-precision stitching of 3D point cloud data with non-overlapping product views, further optimization is needed based on the above stitching process. Specific optimizations are as follows:

[0158] To address the alignment error between the outer and inner point cloud data of the product to be stitched, this invention adds auxiliary through-holes to the calibration plate to optimize this alignment error, such as... Figure 11 The diagram shows a schematic of the through-hole. The basic principle of the auxiliary hole optimization scheme is that after the through-hole is ideally stitched together, the line connecting the centers of the circular holes scanned on the outer and inner sides should be perpendicular to the outer and inner planes. Therefore, by extracting the inner and outer circular holes and calculating the coordinates of their centers, the alignment error of the two circular holes can be obtained. By compensating for this error, the alignment error between the inner and outer sides can be optimized.

[0159] To address the stitching error in the top point cloud data of the product to be stitched, this invention employs an exhaustive iterative algorithm for optimization. The exhaustive iterative algorithm optimizes the top point cloud data. Its basic principle is to iteratively fine-tune the top spherical point cloud data along the z-axis, based on the optimization achieved using the auxiliary holes. After each fine-tuning, the error in fitting the center coordinates of the three parts of the spherical point cloud data is recalculated. If the error converges (i.e., the error value decreases), the iteration continues; if the error does not converge (i.e., the error value increases), the iteration terminates, and the final optimized result is returned.

[0160] It should be noted that the through hole is an auxiliary hole on the calibration plate, which penetrates the inner and outer sides of the calibration plate frame. Therefore, after ideal splicing, the line connecting the centers of the inner and outer holes should be perpendicular to the plane where the hole is located. Furthermore, the center is obtained by performing circular fitting based on the 3D point cloud data of the hole.

[0161] Furthermore, the coordinates of the center of the penetrating hole are not measured by a 3D coordinate measuring machine, but are directly obtained from the point cloud data of the calibration plate acquired by a 3D sensor. The point cloud data of the hole on the calibration plate is then segmented, and the center coordinates are obtained by fitting the points to the hole. After fitting the centers of both the outer and inner sides of the hole, the alignment error is calculated using these two center coordinates. Finally, this alignment error is used as a compensation value to compensate for the translation vector in the corresponding affine transformation matrix.

[0162] Understandably, after optimizing the initially obtained affine transformation matrix by measuring the alignment error of the inner and outer circular holes on the calibration board, the exhaustive iterative algorithm further refines the matrix by only targeting the 3D point cloud data at the top of the bounding box. This refinement is performed along the z-axis, perpendicular to the top plane, with a fixed step size for each adjustment. If the error decreases after refinement, the process is considered convergent, and refinement can continue until the error no longer decreases, at which point iteration stops. The total number of adjustments made along the z-axis is recorded, and this adjustment is used as a second error compensation, applied to the translation vector of the corresponding affine transformation matrix to obtain the final affine transformation matrix.

[0163] It should be noted that the exhaustive iterative algorithm is used to reduce the final stitching error. The stitching error is defined as follows: extract 3D point cloud data of the sphere from three perspectives, then fit the coordinates of the sphere center using these three parts of the sphere center cloud data, and then compare them with the sphere center coordinates measured by the 3D coordinate measuring machine at the beginning. In other words, calculate the Euclidean distance between the two sphere centers. Ideally, after stitching, the two sphere centers should coincide, but because there is a stitching error, the magnitude of the error is judged by the Euclidean distance between the coordinates of the two sphere centers.

[0164] In another specific embodiment, the present invention precisely machines a bevel on the edge of the calibration plate, such as... Figure 12 As shown. When the top 3D sensor and the outer 3D sensor scan the calibration plate with the inclined plane, the point cloud data of the inclined plane can be scanned. Therefore, after calculating the affine transformation matrix (the matrix optimized by the auxiliary aperture optimization and exhaustive iterative algorithm) and achieving preliminary stitching, the inclined plane point cloud data of the two different 3D sensors should ideally overlap, that is, the normal vectors of the inclined plane point cloud data of the two 3D sensors should be parallel. However, due to the existence of stitching error, there is a certain angle between the two normal vectors. This invention takes the inclined plane data collected by a certain 3D sensor as the reference and its normal vector as the reference vector, and calculates the angle θ between the normal vector (also known as the non-reference vector) of the inclined plane point cloud data collected by another 3D sensor and the reference vector. Let the reference vector be... Non-reference vector is So through Cross product To obtain a new vector that is perpendicular to both vectors. Also known as the rotation axis vector. Therefore, all point cloud data not located at the reference vector are rotated around the rotation vector. By rotating by an angle θ, the non-reference vector and the reference vector become parallel, thus further improving the splicing accuracy.

[0165] Calculate vectors sum vector The included angle θ is given by the following formula:

[0166]

[0167]

[0168]

[0169] Calculate the reference vector Non-benchmark vector After the included angle θ, the non-reference vector The corresponding point cloud data is based on the rotation axis vector. Rotate θ as the axis of rotation. The specific process is as follows:

[0170] Let non-reference vector The corresponding product bounding point cloud dataset is U = {u1, u2, ..., u}. i The point cloud dataset U is the result set after affine transformation matrix transformation, where all points in U are rotated around the vector. The point cloud dataset obtained after rotating by an angle θ is V={v1,v2,…,v i}, then according to Rodriguez's rotation formula, we have:

[0171]

[0172] Based on the above formula, we get V = {v1, v2, ..., v} i This represents the final processed result of the product border point cloud data corresponding to the non-reference vector. At this point, the product border point cloud data and the product border point cloud data corresponding to the reference vector are stitched together more accurately.

[0173] Figure 13 The system architecture diagram of a non-overlapping 3D point cloud stitching system based on a calibration board provided in the embodiments of the present invention is as follows: Figure 13 As shown, it includes:

[0174] The calibration plate determination unit 1310 is used to determine the calibration plate based on the border distribution of the surface point cloud data product to be collected; the product includes M non-overlapping borders, the calibration plate includes M calibration borders, each calibration border has at least 3 non-collinear calibration spheres placed on it, the positions of the M calibration borders and the M non-overlapping borders are one-to-one, and M is an integer greater than 1; the non-overlapping borders refer to the absence of overlapping areas between the border 3D point cloud data collected by 3D sensors under multiple fields of view.

[0175] The sphere center coordinate determination unit 1320 is used to acquire spherical point cloud data of all calibration spheres when the calibration plate is in a preset position based on a three-dimensional coordinate measuring instrument, so as to fit and obtain the first sphere center coordinates of each calibration sphere; and to acquire spherical point cloud data of all calibration spheres on a calibration frame when the calibration plate is in the preset position based on a three-dimensional sensor, so as to fit and obtain the second sphere center coordinates of all calibration spheres on the calibration frame.

[0176] The transformation matrix determination unit 1330 is used to obtain the affine transformation matrix between the three-dimensional sensor coordinate system for measuring the calibration frame and the three-dimensional coordinate measuring instrument coordinate system based on the first and second center coordinates of all calibration spheres on a calibration frame; and to determine multiple three-dimensional sensors that collect point cloud data for each calibration frame and their corresponding affine transformation matrices.

[0177] The three-dimensional point cloud data stitching unit 1340 is used to collect three-dimensional point cloud data of each border of the product at the preset position based on the multiple three-dimensional sensors, and transform the collected three-dimensional point cloud data to the coordinate system of the three-dimensional coordinate measuring instrument based on the corresponding affine transformation matrix so as to stitch together the three-dimensional point cloud data of the product border field of view that do not overlap.

[0178] When there are parallel borders under the M non-overlapping borders, the calibration plate determining unit adds a penetrating circular hole to each of the parallel calibration borders corresponding to the M calibration borders, and the line connecting the centers of all the penetrating circular holes on the parallel calibration border is perpendicular to the border where the penetrating circular hole is located.

[0179] The error compensation unit 1350 is used to acquire point cloud data of all penetrating circular holes on each set of parallel calibration frames when the calibration plate is in a preset position based on a three-dimensional sensor, and to obtain the coordinates of the center of the penetrating circular hole by performing circular fitting based on the point cloud data of the circular hole, and to determine the alignment error of the center based on the center coordinates fitted inside and outside the penetrating circular hole; and to perform the first compensation on the affine transformation matrix of the corresponding parallel calibration frame based on the alignment error, to obtain the affine transformation matrix after the first compensation, and to use it as the affine transformation matrix after the first compensation of the corresponding calibration frame.

[0180] It should be understood that the above system is used to execute the methods in the above embodiments. The corresponding program modules in the system are similar in implementation principle and technical effect to those described in the above methods. The working process of the system can be referred to the corresponding process in the above methods, and will not be repeated here.

[0181] Based on the methods described in the above embodiments, this application provides an electronic device. The device may include at least one memory for storing a program and at least one processor for executing the program stored in the memory. When the program stored in the memory is executed, the processor performs the methods described in the above embodiments.

[0182] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0183] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0184] It is understood that the processor in the embodiments of this application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0185] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0186] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0187] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0188] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for stitching non-overlapping 3D point clouds based on a calibration board, characterized in that, Includes the following steps: The calibration plate is determined based on the border distribution of the surface point cloud data product to be collected; the product includes M non-overlapping borders, the calibration plate includes M calibration borders, and at least 3 non-collinear calibration spheres are placed on each calibration border. The positions of the M calibration borders and the M non-overlapping borders are identical, and M is an integer greater than 1; the non-overlapping borders refer to the absence of overlapping areas between the border 3D point cloud data collected by 3D sensors under multiple fields of view. The point cloud data of the spherical surface of all calibration spheres is obtained by using a three-dimensional coordinate measuring instrument when the calibration plate is in a preset position, and the first center coordinates of each calibration sphere are obtained by fitting. Based on a three-dimensional sensor, the spherical point cloud data of all calibration spheres on a calibration frame when the calibration plate is at the preset position is obtained, and the second sphere center coordinates of all calibration spheres on the calibration frame are obtained by fitting. Based on the first and second center coordinates of all calibration spheres on a certain calibration frame, the affine transformation matrix between the three-dimensional sensor coordinate system for measuring the calibration frame and the three-dimensional coordinate measuring instrument coordinate system is obtained; Determine the multiple 3D sensors that collect point cloud data for each calibration bounding box and the corresponding affine transformation matrix; Based on the multiple three-dimensional sensors, the three-dimensional point cloud data of each border of the product at the preset position are collected, and the collected three-dimensional point cloud data are transformed to the coordinate system of the three-dimensional coordinate measuring instrument based on the corresponding affine transformation matrix so as to stitch together the three-dimensional point cloud data of the product border field of view that do not overlap. When there are parallel borders under the M non-overlapping borders, a through hole is added to each of the parallel calibration borders corresponding to the M calibration borders. The line connecting the centers of all the through holes on the parallel calibration border is perpendicular to the border where the through hole is located. After determining the affine transformation matrix based on the coordinates of the first and second centers of all calibration spheres on a certain calibration border, the following steps are also included: The point cloud data of all the penetrating circular holes on each set of parallel calibration frames are obtained based on the three-dimensional sensor when the calibration plate is in the preset position. The coordinates of the center of the penetrating circular hole are obtained by performing circular fitting based on the point cloud data of the circular hole. The alignment error of the center is determined according to the center coordinates fitted inside and outside the penetrating circular hole. Based on the alignment error, the affine transformation matrix of the corresponding parallel calibration border is compensated for the first time to obtain the affine transformation matrix after the first compensation, which is used as the affine transformation matrix of the corresponding calibration border after the first compensation. The affine transformation matrix is ​​determined based on the coordinates of the first and second centers of all calibration spheres on a certain calibration border, specifically as follows: First, let A, B, and C represent the coordinates of the centers of the three calibration spheres in the coordinate system of the 3D coordinate measuring machine. Let a, b, and c represent the coordinates of the centers of the three calibration spheres in the coordinate system of the 3D sensor. Let H be the affine transformation matrix from the 3D sensor coordinate system to the 3D coordinate measuring machine coordinate system. Then we can obtain: in: Let D be a point on the extension of the center A of the sphere, and d be a point on the extension of the center a of the sphere, where: .

2. The method according to claim 1, characterized in that, The calibration plate is machined with an inclined surface, and the point cloud data on the inclined surface can be obtained by scanning at least two three-dimensional sensors with different fields of view; The method further includes the following steps: For the 3D point cloud data collected on the calibration board, the data collected by one 3D sensor on the inclined plane is used as the reference, and its normal vector is used as the reference vector. The normal vectors of the data collected by other 3D sensors on the inclined plane are used as non-reference vectors. The angle between the reference vector and the non-reference vector is calculated based on the 3D point cloud data on the calibration board. ; And let the reference vector be The non-reference vector is ,pass Cross product To obtain the rotation axis vector ; For the collected 3D point cloud data of the product, let the product bounding box point cloud dataset corresponding to the non-reference vector of the product be denoted as . The point cloud dataset This is the result set after the affine transformation matrix has been applied. Point cloud dataset All points about the axis of rotation vector Rotation After adjusting the angle, we obtain the point cloud dataset V, which is further optimized from the point cloud dataset U. According to Rodriguez's rotation formula, we have: 。 3. The method according to claim 1, characterized in that, After the three-dimensional coordinate measuring instrument and the three-dimensional sensor acquire the spherical point cloud data of the calibration sphere, the spherical point cloud data is denoised, and then the sphere center coordinates are obtained by fitting based on the denoised spherical point cloud data. The denoising of the spherical point cloud data adopts an outlier removal algorithm based on statistical filtering, and the specific steps are as follows: Calculate the average distance from each point to any other point during the iteration. and standard deviation Let the standard deviation multiple be . When the distance between point cloud data and all its neighboring points is If the point is valid, retain it; otherwise, treat it as an outlier and remove it.

4. The method according to any one of claims 1 to 3, characterized in that, Based on the data of multiple points in the spherical point cloud data, the least squares method is used to fit the sphere to obtain the coordinates of the center of the calibration sphere.

5. The method according to claim 1, characterized in that, When determining the center coordinates of the calibrated sphere based on the spherical point cloud data, an exhaustive iterative algorithm is used to fine-tune the top point cloud data, reduce the stitching error of the 3D point cloud data under different views, and determine the corresponding stitching error compensation value. Based on the stitching error compensation value, the affine transformation matrix after the first compensation is compensated a second time to obtain the affine transformation matrix after the second compensation, which is used as the final affine transformation matrix. The splicing error refers to the Euclidean distance between the center coordinates of the sphere fitted from the three-dimensional point cloud data of the sphere under different viewpoints and the center coordinates of the sphere measured by the three-dimensional coordinate measuring instrument. The exhaustive iterative algorithm is as follows: the top spherical point cloud data is finely adjusted along the z-axis with a fixed step size. After each fine adjustment, the error of fitting the sphere center coordinates of the spherical point cloud data is recalculated. If the error value decreases, the iteration continues; if the error value increases, the iteration terminates and the sum of the iteration steps is returned. The sum of the iteration steps is used as the stitching error compensation value.

6. A non-overlapping 3D point cloud stitching system based on a calibration board, characterized in that, include: A calibration plate determination unit is used to determine a calibration plate based on the border distribution of the surface point cloud data product to be collected. The product includes M non-overlapping borders, and the calibration plate includes M calibration borders. At least 3 non-collinear calibration spheres are placed on each calibration border. The positions of the M calibration borders and the M non-overlapping borders coincide one-to-one, where M is an integer greater than 1. The non-overlapping borders refer to the absence of overlapping areas between the border 3D point cloud data collected by 3D sensors under multiple fields of view. The sphere center coordinate determination unit is used to acquire spherical point cloud data of all calibration spheres when the calibration plate is in a preset position based on a three-dimensional coordinate measuring instrument, so as to fit and obtain the first sphere center coordinate of each calibration sphere; and to acquire spherical point cloud data of all calibration spheres on a calibration frame when the calibration plate is in the preset position based on a three-dimensional sensor, so as to fit and obtain the second sphere center coordinate of all calibration spheres on the calibration frame. The transformation matrix determination unit is used to obtain the affine transformation matrix between the three-dimensional sensor coordinate system for measuring the calibration frame and the three-dimensional coordinate measuring instrument coordinate system based on the first and second center coordinates of all calibration spheres on a certain calibration frame. And determine the multiple 3D sensors that collect point cloud data for each calibration bounding box and the corresponding affine transformation matrix; The three-dimensional point cloud data stitching unit is used to collect three-dimensional point cloud data of each border of the product at the preset position based on the multiple three-dimensional sensors, and transform the collected three-dimensional point cloud data to the coordinate system of the three-dimensional coordinate measuring instrument based on the corresponding affine transformation matrix so as to stitch together the three-dimensional point cloud data of the product border field of view without overlap. When there are parallel borders under the M non-overlapping borders, the calibration plate determining unit adds a penetrating circular hole to each of the parallel calibration borders corresponding to the M calibration borders, and the line connecting the centers of all the penetrating circular holes on the parallel calibration border is perpendicular to the border where the penetrating circular hole is located. The system also includes: The error compensation unit is used to acquire point cloud data of all penetrating circular holes on each set of parallel calibration frames when the calibration plate is in a preset position based on a three-dimensional sensor, and to obtain the coordinates of the center of the penetrating circular hole by performing circular fitting based on the point cloud data of the circular hole, and to determine the alignment error of the center based on the center coordinates fitted inside and outside the penetrating circular hole; and to perform the first compensation on the affine transformation matrix of the corresponding parallel calibration frame based on the alignment error, to obtain the affine transformation matrix after the first compensation, and to use it as the affine transformation matrix after the first compensation of the corresponding calibration frame. The transformation matrix determination unit determines the affine transformation matrix based on the first and second center coordinates of all calibration spheres on a certain calibration border. Specifically, it first lets A, B, and C represent the center coordinates of the three calibration spheres in the 3D coordinate measuring instrument coordinate system, lets a, b, and c represent the center coordinates of the three calibration spheres in the 3D sensor coordinate system, and lets H be the affine transformation matrix from the 3D sensor coordinate system to the 3D coordinate measuring instrument coordinate system. Then, we can obtain: in: Let D be a point on the extension of the center A of the sphere, and d be a point on the extension of the center a of the sphere, where: .

Images