A 3D scanning method and system based on reverse positioning and 3D camera fusion

Abstract

This application discloses a 3D scanning method and system based on reverse positioning and 3D camera fusion. The method includes: establishing a reference coordinate system for the measurement scene and setting multiple target points on the stable reference body; scanning the target points with a positioning camera to obtain a reverse positioning matrix; determining the relative relationship matrix between the 3D camera and the positioning camera using a preset calibration method; wherein the 3D camera and the positioning camera are rigidly connected; scanning the object to be measured placed in the measurement scene with the 3D camera to obtain point cloud data of the object to be measured; and obtaining the coordinates of the object to be measured in the reference coordinate system based on the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera. This invention uses a high-resolution 3D camera to scan the object and a positioning camera for positioning, which improves the surface scanning effect, eliminates the need for additional positioning components, increases flexibility, and meets the requirements of high-precision dimensional measurement.

Description

Technical Field

[0001] This invention relates to the field of 3D scanning technology, and specifically to a 3D scanning method and system based on reverse positioning and 3D camera fusion. Background Technology

[0002] 3D scanning is a high-tech process integrating optics, mechanics, electronics, and computer technology. It is mainly used to scan the spatial shape, structure, and color of objects to obtain the spatial coordinates of the object's surface. With technological advancements, customer demand for non-stick scanning of products is increasing. However, due to the limitations of laser scanning resolution, the non-stick scanning effect for features such as sheet metal boundaries and holes is generally not ideal. To improve the accuracy of non-stick scanning for specific locations, scanning systems incorporating 3D cameras have emerged on the market.

[0003] Currently, the application of 3D cameras in scanning systems on the market can be broadly categorized into three types: 1. Using a 3D camera alone for local scanning and point cloud image analysis; 2. Combining a 3D camera with high-precision equipment for positioning and scanning fusion; 3. Using a tracker for positioning and scanning fusion. All three application scenarios have significant limitations: 1. Using a 3D camera alone can only measure local data, suitable for small products or specific locations, and cannot achieve data fusion analysis; 2. Combining a 3D camera with high-precision equipment is difficult to control in terms of cost, resulting in low market competitiveness; 3. Positioning via a tracker requires a positioning device, and the device must remain within the tracker's field of view, limiting its orientation.

[0004] Therefore, it is necessary to propose a 3D camera scanning system based on reverse positioning to realize data fusion measurement and analysis of small products or specific locations, and to be able to be used with fixed platforms or automated scanning platforms to perform flexible and high-precision measurement of products. Summary of the Invention

[0005] In view of this, it is necessary to provide a 3D scanning method and system based on reverse positioning and 3D camera fusion to solve the problems in the existing technology that when scanning small products or specific locations without attaching to the points, data fusion analysis cannot be performed, and positioning components are required, resulting in low measurement accuracy and limited flexibility.

[0006] To address the aforementioned problems, this invention provides a 3D scanning method based on reverse positioning and 3D camera fusion, comprising:

[0007] Using a stable reference body as the measurement scene, a reference coordinate system for the measurement scene is established, and multiple target points are set on the stable reference body.

[0008] The target point is scanned by a positioning camera to obtain the reverse positioning matrix of the positioning camera in the reference coordinate system; and the relative relationship matrix between the 3D camera and the positioning camera is determined using a preset calibration method; wherein the 3D camera and the positioning camera are rigidly connected.

[0009] The 3D camera scans the object to be measured placed in the measurement scene to obtain point cloud data of the object to be measured. Based on the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera, the point cloud data of the object to be measured is transformed into the reference coordinate system to obtain the coordinates of the object to be measured in the reference coordinate system.

[0010] Furthermore, using a preset calibration method, the relative relationship matrix between the 3D camera and the positioning camera is determined, including:

[0011] The 3D camera is used to scan the preset geometric objects placed in the measurement scene under different poses to obtain a set of calibration point coordinates of multiple preset geometric objects in the 3D camera coordinate system.

[0012] While the 3D camera is scanning, the positioning camera scans the target points in the test scene to determine the calibration matrix of the positioning camera under different poses of the 3D camera.

[0013] Initialize a preset number of initial relationship matrices between the 3D cameras and the positioning cameras. Based on the initial relationship matrix and the calibration matrix, transform the coordinate set of the calibration points to the reference coordinate system to obtain the coordinate set of the calibration points in the reference coordinate system.

[0014] The initial relationship matrix is ​​optimized based on the coordinate overlap in the calibration point reference coordinate system, and the relative relationship matrix between the 3D camera and the positioning camera is obtained based on the optimized initial relationship matrix.

[0015] Furthermore, the initial relationship matrix is ​​optimized based on the coordinate overlap in the calibration point reference coordinate system. The relative relationship matrix between the 3D camera and the positioning camera is obtained from the optimized initial relationship matrix, including:

[0016] The initial relation matrix with the highest coordinate overlap of the calibration point coordinate set in the reference coordinate system is taken as the relation matrix to be optimized.

[0017] The relationship matrix to be optimized is optimized using the confidence region method. The coordinates of the calibration point in the reference coordinate system are calculated based on the optimized relationship matrix, and the degree of overlap of the coordinates is calculated.

[0018] When the overlap converges, the optimal relation matrix is ​​obtained, and the optimal relation matrix is ​​used as the relative relation matrix between the 3D camera and the positioning camera.

[0019] Furthermore, the target point is scanned by the positioning camera to obtain the reverse positioning matrix of the positioning camera in the reference coordinate system, including:

[0020] The positioning camera scans at least three target points, and determines the reverse positioning matrix of the positioning camera in the reference coordinate system based on the focal center of the positioning camera, the coordinates of the target points in the reference coordinate system, and the coordinates of the target points in the positioning camera coordinate system.

[0021] Furthermore, based on the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera, the point cloud data of the object is transformed into the reference coordinate system to obtain the coordinates of the object in the reference coordinate system, including:

[0022] Based on the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera, the point cloud transformation matrix between the 3D camera coordinate system and the reference coordinate system is obtained;

[0023] Based on the point cloud transformation matrix, the point cloud data is transformed into the reference coordinate system to obtain the coordinates of the item in the reference coordinate system.

[0024] Furthermore, after scanning the objects placed in the measurement scene using the 3D camera to obtain the point cloud data of the objects, the process further includes:

[0025] Based on the point cloud data, a preset shape extraction method is used to determine whether the item contains a preset shape.

[0026] Furthermore, based on the point cloud data, a preset shape extraction method is used to determine whether the item contains a preset shape, including:

[0027] Calculate the curvature of each point in the point cloud data, and designate points with curvature greater than a preset curvature threshold as edge points;

[0028] The edge points are connected using a connected component search method to obtain the edge lines;

[0029] The edge line is fitted using a preset fitting method. When the fitting result meets the preset shape judgment criteria, it is determined that the item contains a preset shape.

[0030] Furthermore, when the 3D camera scans the object to be measured placed in the measurement scene for a time exceeding a preset time threshold, the method further includes:

[0031] While the 3D camera scans the object under test, the positioning camera acquires the coordinates of multiple target points, and the anti-shake reverse positioning matrix of the positioning camera is obtained based on the coordinates of the multiple target points.

[0032] Based on the anti-shake reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera, the point cloud data of the object under test is transformed into the reference coordinate system to obtain the coordinates of the object under test in the reference coordinate system.

[0033] The present invention also provides a three-dimensional scanning device based on reverse positioning and 3D camera fusion, comprising:

[0034] The scene establishment module is used to establish a reference coordinate system for the measurement scene using a stable reference body as the measurement scene, and to set multiple target points on the stable reference body.

[0035] The calibration module is used to scan the target point with the positioning camera to obtain the reverse positioning matrix of the positioning camera in the reference coordinate system; and to determine the relative relationship matrix between the 3D camera and the positioning camera using a preset calibration method; wherein the 3D camera and the positioning camera are rigidly connected.

[0036] The conversion module is used to scan the object placed in the measurement scene using the 3D camera to obtain the point cloud data of the object, and convert the point cloud data of the object to the reference coordinate system according to the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera to obtain the coordinates of the object in the reference coordinate system.

[0037] The present invention also provides a three-dimensional scanning system based on reverse positioning and 3D camera fusion, comprising: a stable reference body, a measuring device, and a fusion computing device;

[0038] The stable reference body serves as a measurement scenario, providing a target point for the measurement device.

[0039] The measuring device includes a 3D camera and a positioning camera, wherein the 3D camera and the positioning camera are rigidly connected; the measuring device is used to scan the target point and scan the object placed in the measuring scene to obtain the point cloud data of the object;

[0040] The fusion computing device is used to establish a reference coordinate system for the measurement scene, obtain the reverse positioning matrix of the positioning camera in the reference coordinate system based on the scanning results of the target point by the measuring device, and determine the relative relationship matrix between the 3D camera and the positioning camera using a preset calibration method; and convert the point cloud data of the object to the reference coordinate system based on the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera to obtain the coordinates of the object in the reference coordinate system.

[0041] Compared with existing technologies, the beneficial effects of this invention include: First, establishing a measurement scene with a stable reference body and establishing a reference coordinate system, and setting target points for positioning on the reference body; second, determining the reverse positioning matrix of the positioning camera in the reference coordinate system, and the relative relationship matrix between the 3D camera and the positioning camera; finally, scanning the object placed in the measurement scene with the 3D camera to obtain point cloud data, and transforming the point cloud data of the object into the reference coordinate system according to the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera to obtain the coordinates of the object in the reference coordinate system. The method of this invention combines a positioning camera and a 3D camera, using a high-resolution 3D camera to scan the object and the positioning camera for positioning, eliminating the need for additional positioning components, thus improving the flexibility of the scanning system. It also eliminates the need to attach positioning markers to the object being measured, meeting the requirements for high-precision dimensional measurements, and improving the surface scanning effect compared to laser line extraction methods. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating an embodiment of a three-dimensional scanning method based on reverse positioning and 3D camera fusion provided by the present invention.

[0043] Figure 2 This is a schematic diagram illustrating an embodiment of the connection relationship between a positioning camera and a 3D camera provided by the present invention.

[0044] Figure 3 This is a schematic diagram illustrating the principle of an embodiment of determining the reverse positioning matrix provided by the present invention;

[0045] Figure 4 A schematic diagram illustrating a scenario of measuring an item to be tested according to an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of an embodiment of a three-dimensional scanning device based on reverse positioning and 3D camera fusion provided by the present invention. Detailed Implementation

[0047] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0048] Before describing the specific implementation plan, we will first explain the existing 3D scanning technology.

[0049] Point-based scanning: In 3D scanning, the workpiece to be scanned is usually pre-processed by point-based scanning. That is, according to the characteristics of the workpiece, marker points are pasted on the surface of the workpiece. After point-based scanning, because the position of the marker points relative to the workpiece remains unchanged, the scanner and the workpiece can move relative to each other, ensuring stitching accuracy and enabling omnidirectional stitching scanning of the workpiece.

[0050] However, for workpieces that are small in size or have complex structural features, the point-based scanning method is too complicated. Therefore, a non-point-based scanning method has emerged that uses the background or fixture to set marker points to complete the positioning.

[0051] Currently, unlabeled 3D scanning systems on the market incorporate 3D cameras, but they still suffer from limitations such as the inability to perform data fusion analysis when scanning small products or specific locations, the need for positioning devices, low measurement accuracy, and limited flexibility.

[0052] This invention designs a three-dimensional scanning method and system based on reverse positioning and 3D camera fusion. Combining the principle of reverse positioning, a positioning camera and a 3D camera are used to scan objects using a high-resolution 3D camera, and the scanning system is positioned by the positioning camera, thereby improving the accuracy of local scanning. Compared with laser line extraction, it improves the surface scanning effect and does not require additional positioning components, thus increasing the flexibility of the scanning system.

[0053] This invention provides a 3D scanning method based on reverse positioning and 3D camera fusion, such as... Figure 1 As shown, Figure 1 This is a flowchart illustrating the 3D scanning method based on reverse positioning and 3D camera fusion, including:

[0054] Step S101: Using a stable reference body as the measurement scene, establish a reference coordinate system for the measurement scene, and set multiple target points on the stable reference body;

[0055] Step S102: The target point is scanned by the positioning camera to obtain the reverse positioning matrix of the positioning camera in the reference coordinate system; and the relative positional relationship between the 3D camera and the positioning camera is determined by a preset calibration method; wherein the 3D camera and the positioning camera are rigidly connected.

[0056] Step S103: The 3D camera scans the object to be measured placed in the measurement scene to obtain the point cloud data of the object to be measured. Based on the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera, the point cloud data of the object to be measured is transformed into the reference coordinate system to obtain the coordinates of the object to be measured in the reference coordinate system.

[0057] This embodiment provides a 3D scanning method and system based on reverse positioning and 3D camera fusion. First, a measurement scene is established using a stable reference body, and a reference coordinate system is established. Target points for positioning are set on the reference body. Second, the reverse positioning matrix of the positioning camera in the reference coordinate system and the relative relationship matrix between the 3D camera and the positioning camera are determined. Finally, the object placed in the measurement scene is scanned by the 3D camera to obtain point cloud data. Based on the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera, the point cloud data of the object is transformed into the reference coordinate system to obtain the coordinates of the object in the reference coordinate system. This embodiment combines a positioning camera and a 3D camera, using a high-resolution 3D camera to scan the object and the positioning camera for positioning. No additional positioning components are needed, improving the flexibility of the scanning system. Positioning marks do not need to be affixed to the object being measured, meeting the requirements for high-precision dimensional measurements. Compared to laser line extraction methods, it improves the surface scanning effect.

[0058] As a specific embodiment, in step S101, the stable reference body can be a wall. A control rod and a target point are set on the reference body. The control rod and the target point have nominal values. An image is captured by a specific device, the control rod and the target point are extracted, a reference coordinate system is established through the control rod, and the coordinate value of the target point in the reference coordinate system is recorded.

[0059] As a specific embodiment, in step S102, such as Figure 2 As shown, Figure 2 The method of rigid connection between the 3D camera and the positioning camera is shown.

[0060] It should be noted that there may be multiple positioning cameras, and they are not limited to acquiring target points on a stable reference body in one direction.

[0061] In a preferred embodiment, the target point is scanned by a positioning camera to obtain the reverse positioning matrix of the positioning camera in the reference coordinate system, including:

[0062] The positioning camera scans at least three target points, and determines the reverse positioning matrix of the positioning camera in the reference coordinate system based on the focal center of the positioning camera, the coordinates of the target points in the reference coordinate system, and the coordinates of the target points in the positioning camera coordinate system.

[0063] The following is combined with Figure 3 The above process will be described in detail.

[0064] As a specific example, such as Figure 3 As shown, a positioning camera scans three target points A, B, and C within the field of view, with the camera's focal center located at point P. Given |BC| = a', |AC| = b', |AB| = c'; let X = |PA|, Y = |PB|, Z = |PC|, X = xZ, Y = yZ; given α = ...<PB,PC> ,β=<PA,PB> γ=<PA,PC> ;

[0065] Let a' 2 =ac' 2 b' 2 =bc' 2 c' 2 =vZ 2 Given p = 2cosα, q = 2cosβ, and r = 2cosγ, then a' 2 =ac' 2 =avZ 2 ,b' 2 =bc' 2 =bvZ 2 The constraint equation is derived from the fact that P, A, B, and C are not coplanar: p 2 +q 2 +r 2 -pqr-1≠0.

[0066] According to the Law of Cosines for triangles, the following formula can be derived for the three triangles formed by points P, A, B, and C:

[0067] Y 2 +Z 2 -2YZcosα=a' 2 (1)

[0068] X 2 +Z 2 -2XZcosβ=b' 2 (2)

[0069] Y 2 +X 2 -2YXcosγ=c' 2 (3)

[0070] Based on the defined parameter relationships, the above three formulas (1), (2), and (3) are transformed into the following equations:

[0071] (1-a)y 2 -ax 2+axyr-yp+1=0 (4)

[0072] (1-b)x 2 -by 2 +bxyr-xq+1=0 (5)

[0073] Based on equations (4) and (5), x, y, and v are solved using the Wu zero-point decomposition method; combined with the parameter relationship bc' 2 =bvZ 2 Given X = xZ and Y = yZ, find X, Y, and Z.

[0074] After determining the distances X, Y, and Z from points A, B, and C to the focal point of the positioning camera, the coordinates of points A, B, and C in the positioning camera coordinate system are calculated according to the principle of triangle similarity. Combined with the pre-measured coordinates of A, B, and C in the reference coordinate system, the pose (X, Y, Z, α, β, γ) of the monocular camera in the reference coordinate system is obtained.

[0075] The angle values ​​are converted into an angle matrix, and the positioning camera is expressed in the reference coordinate system using a reverse positioning matrix. The reverse positioning matrix includes a rotation matrix and a translation vector. Therefore, we have: Rotation matrix Translation vector

[0076] a 00 =cos(β)*cos(γ)-sin(α)*sin(β)*sin(γ);

[0077] a 01 =-cos(β)*sin(γ)-sin(α)*sin(β)*cos(γ);

[0078] a 02 = -sin(β)*cos(α);

[0079] a 10 =cos(α)*sin(γ)

[0080] a 11 =cos(α)*cos(γ);

[0081] a 12 = -sin(α);

[0082] a 20 =sin(β)*cos(γ)+cos(β)*sin(α)*sin(γ);

[0083] a 21 =-sin(α)*sin(γ)+cos(β)*sin(α)*cos(γ);

[0084] a 22 =cos(α)*cos(γ).

[0085] In a preferred embodiment, a preset calibration method is used to determine the relative relationship matrix between the 3D camera and the positioning camera, including:

[0086] The 3D camera is used to scan the preset geometric objects placed in the measurement scene under different poses to obtain a set of calibration point coordinates of multiple preset geometric objects in the 3D camera coordinate system.

[0087] While the 3D camera is scanning, the positioning camera scans the target points in the test scene to determine the calibration matrix of the positioning camera under different poses of the 3D camera.

[0088] Initialize a preset number of initial relationship matrices between the 3D cameras and the positioning cameras, and transform the calibration point coordinate set to the reference coordinate system based on the initial relationship matrix and the calibration matrix;

[0089] The initial relationship matrix is ​​optimized based on the overlap of the calibration point coordinates in the reference coordinate system, and the relative relationship matrix between the 3D camera and the positioning camera is obtained based on the optimized initial relationship matrix.

[0090] In a preferred embodiment, the initial relationship matrix is ​​optimized based on the overlap of the calibration point coordinates in the reference coordinate system. The relative relationship matrix between the 3D camera and the positioning camera is then obtained based on the optimized initial relationship matrix, including:

[0091] The initial relation matrix with the highest coordinate overlap of the calibration point coordinate set in the reference coordinate system is taken as the relation matrix to be optimized.

[0092] The relationship matrix to be optimized is optimized using the confidence region method. The coordinates of the calibration point in the reference coordinate system are calculated based on the optimized relationship matrix, and the degree of overlap of the coordinates is calculated.

[0093] When the overlap converges, the optimal relation matrix is ​​obtained, and the optimal relation matrix is ​​used as the relative relation matrix between the 3D camera and the positioning camera.

[0094] As a specific embodiment, the following describes the method for determining the relative positional relationship between the 3D camera and the positioning camera, taking the preset geometry as a single standard sphere and the calibration point as the center of the sphere.

[0095] The 3D camera scans a standard sphere in N poses to obtain the coordinate set P1…Pn of the center of the N standard spheres in the 3D camera coordinate system.

[0096] While the 3D camera scans the standard sphere, the positioning camera captures the target points in the scene and calculates the calibration matrix of the positioning camera in the reference coordinate system under the N poses of the 3D camera: RTc, (Rc1+Tc1)…(Rcn+Tcn).

[0097] The software pre-sets more than a preset number (here, 10^8) of initial relation matrices RTc_m to represent the relative positional relationship between the 3D camera and the positioning camera. The initial relation matrix RTc_m is then used to perform matrix calculations with the positioning camera's calibration matrix RTc: R m =R c *R c_m ,T m =R c *T c_m +T c_m The transformation matrix RT between the 3D camera and the reference coordinate system is obtained.

[0098] The coordinates of the sphere's center obtained by the 3D camera in N poses are respectively The coordinate transformation from the 3D camera coordinate system to the reference coordinate system is performed using the transformation matrix RT: P′ n =P n *R m +T m By analyzing and calculating the overlap degree of N points, the optimal matrix is ​​selected from the initial relation matrix as the matrix to be optimized, and the confidence region method is used to calculate:

[0099] Ω k ={x∈R n |‖xx k ‖≦Δ k},

[0100] x is the relation matrix, Δ k Let be the confidence region radius.

[0101] Within this neighborhood, the relation matrix that maximizes the overlap is calculated. The initial relation matrix is ​​then replaced, and the confidence region algorithm is used iteratively. When the overlap of the sphere centers gradually converges and approaches the extreme value, the optimal relative relation matrix RT between the positioning camera and the 3D camera is obtained. c_m .

[0102] Through the above process, the calibration of the 3D camera and the positioning camera is completed, that is, the reverse positioning matrix of the positioning camera in the reference coordinate system and the relative relationship matrix between the 3D camera and the positioning camera are obtained. At this time, the 3D camera can be used to scan the object to be tested, and the coordinates of the scanned object can be uniformly expressed using the reference coordinate system.

[0103] In a preferred embodiment, in step S103, the point cloud data of the object is converted to the reference coordinate system according to the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera, to obtain the coordinates of the object in the reference coordinate system, including:

[0104] Based on the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera, the point cloud transformation matrix between the 3D camera coordinate system and the reference coordinate system is obtained;

[0105] Based on the point cloud transformation matrix, the point cloud data is transformed into the reference coordinate system to obtain the coordinates of the item in the reference coordinate system.

[0106] In actual measurements, 3D cameras typically need to scan the object under test for a long time to obtain more comprehensive feature points. During the scanning process, in order to further determine the positioning accuracy of the 3D camera and the positioning camera, the reverse positioning matrix needs to be image stabilized. As a preferred embodiment, when the 3D camera scans the object under test placed in the measurement scene for a time exceeding a preset time threshold, the method further includes:

[0107] While the 3D camera scans the object under test, the positioning camera acquires the coordinates of multiple target points, and the anti-shake reverse positioning matrix of the positioning camera is obtained based on the coordinates of the multiple target points.

[0108] Based on the anti-shake reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera, the point cloud data of the object under test is transformed into the reference coordinate system to obtain the coordinates of the object under test in the reference coordinate system.

[0109] like Figure 4 As shown, Figure 4 A schematic diagram showing the measurement of the object to be tested is presented. Figure 4 In this system, the positioning camera and the 3D camera are rigidly connected. The object to be tested is located within the field of view of the 3D camera. Target points are set on the stable reference body. The positioning camera scans the target points to determine the anti-shake reverse positioning matrix.

[0110] As a specific embodiment, while the 3D camera is measuring the object to be tested, a positioning camera is capturing images of target points for reverse positioning. This allows the acquisition of N reverse positioning matrices RT at the same location of the positioning camera during the 3D camera's scanning time. c–1 …RT c–n The final anti-shake reverse positioning matrix RT is obtained by averaging the T values ​​of the reverse positioning matrix, setting a T value fluctuation threshold, and averaging the positioning coordinates within the threshold. c .

[0111] To quickly determine whether the object under test contains preset shape features, in a preferred embodiment, after scanning the object placed in the measurement scene with the 3D camera to obtain the point cloud data of the object, the method further includes:

[0112] Based on the point cloud data, a preset shape extraction method is used to determine whether the item contains a preset shape.

[0113] In a preferred embodiment, determining whether the item contains a preset shape based on the point cloud data using a preset shape extraction method includes:

[0114] Calculate the curvature of each point in the point cloud data, and designate points with curvature greater than a preset curvature threshold as edge points;

[0115] The edge points are connected using a connected component search method to obtain the edge lines;

[0116] The edge line is fitted using a preset fitting method. When the fitting result meets the preset shape judgment criteria, it is determined that the item contains a preset shape.

[0117] As a specific embodiment, the above-mentioned preset shape extraction method will be described below with the preset shape being a circle (i.e., determining whether there is a circular hole on the object to be tested) as an example.

[0118] The first step is to calculate the curvature of each point in the point cloud data. Points with curvature greater than a preset curvature threshold are considered edge points. In addition, if there are invalid values ​​around a point, that point is also considered an edge point.

[0119] The second step is to calculate the gradient direction of an edge point. If there is a point with greater curvature along the gradient direction, delete the edge point.

[0120] The third step is to use the connected component search method to connect nearby edge points to form edge lines.

[0121] The fourth step is to perform planar fitting using each edge line. If the error is less than the fitting error threshold, then continue with circular fitting; otherwise, determine that the item does not contain a circle.

[0122] The plane fitting uses least squares, and the formula is as follows:

[0123] ax + by + cz + 1 = 0

[0124] In the formula, a, b, c are plane parameters, and x, y, z are points on the plane.

[0125] Let the points on the edge be {(x1,y1,z1),(x2,y2,z2),…,(x n ,y n ,z n The least squares fitting plane formula is:

[0126]

[0127] abbreviated as

[0128] AP = L

[0129] in

[0130]

[0131]

[0132]

[0133] Its least squares solution

[0134] P=(A T A) -1 A T L

[0135] Circular fitting also uses least squares, and the circle formula is:

[0136] x 2 +y 2 +z 2 +dx+ey+fz+g=0

[0137] Where d, e, and f satisfy the constraints

[0138] -0.5ad-0.5be-0.5cf+1=0

[0139] In the formula, a, b, and c are plane parameters, meaning that the center of the circle lies on the plane represented by a, b, and c. Under this constraint, f is represented by d and e. There are three unknowns in the circle fitting: d, e, and g. The circle expression is denoted as:

[0140] x 2 +y 2 +z 2+dx+ey+[(2-ad-be) / c]z+g=0

[0141] Right now:

[0142]

[0143] Least-squares fit of a circle using points on the edge:

[0144]

[0145] Abbreviated as:

[0146] BQ = R

[0147] in

[0148]

[0149]

[0150]

[0151] Its least squares solution

[0152] Q = (B T B) -1 B T R

[0153] Through the above process, it is possible to determine whether the object being measured contains a preset shape, automatically extract features, and improve the efficiency of 3D scanning.

[0154] This invention also provides a 3D scanning device 500 based on reverse positioning and 3D camera fusion, comprising:

[0155] The scene establishment module 501 is used to establish a reference coordinate system for the measurement scene using a stable reference body as the measurement scene, and to set multiple target points on the stable reference body.

[0156] The calibration module 502 is used to scan the target point with the positioning camera to obtain the reverse positioning matrix of the positioning camera in the reference coordinate system; and to determine the relative relationship matrix between the 3D camera and the positioning camera using a preset calibration method; wherein the 3D camera and the positioning camera are rigidly connected.

[0157] The conversion module 503 is used to scan the object placed in the measurement scene using the 3D camera to obtain the point cloud data of the object, and convert the point cloud data of the object to the reference coordinate system according to the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera to obtain the coordinates of the object in the reference coordinate system.

[0158] This invention also provides a three-dimensional scanning system based on reverse positioning and 3D camera fusion, including a stable reference body, a measuring device, and a fusion computing device;

[0159] The stable reference body serves as a measurement scenario, providing a target point for the measurement device.

[0160] The measuring device includes a 3D camera and a positioning camera, wherein the 3D camera and the positioning camera are rigidly connected; the measuring device is used to scan the target point and scan the object placed in the measuring scene to obtain the point cloud data of the object;

[0161] The fusion computing device is used to establish a reference coordinate system for the measurement scene, obtain the reverse positioning matrix of the positioning camera in the reference coordinate system based on the scanning results of the target point by the measuring device, and determine the relative relationship matrix between the 3D camera and the positioning camera using a preset calibration method; and convert the point cloud data of the object to the reference coordinate system based on the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera to obtain the coordinates of the object in the reference coordinate system.

[0162] This invention provides a 3D scanning method and system based on reverse positioning and 3D camera fusion. First, a measurement scene is established using a stable reference body, and a reference coordinate system is established. Target points for positioning are set on the reference body. Second, the reverse positioning matrix of the positioning camera in the reference coordinate system and the relative relationship matrix between the 3D camera and the positioning camera are determined. Finally, the object placed in the measurement scene is scanned by the 3D camera to obtain point cloud data. Based on the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera, the point cloud data of the object is transformed into the reference coordinate system to obtain the coordinates of the object in the reference coordinate system.

[0163] The method of this invention combines a positioning camera and a 3D camera. The high-resolution 3D camera scans the object, and the positioning camera is used for positioning. No additional positioning components are needed, which improves the flexibility of the scanning system. There is no need to attach positioning marks to the object being measured. It can meet the requirements of high-precision size measurement and improves the surface scanning effect compared with the laser line extraction method.

[0164] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A three-dimensional scanning method based on reverse positioning and 3D camera fusion, characterized in that, include: Using a stable reference body as the measurement scene, a reference coordinate system for the measurement scene is established, and multiple target points are set on the stable reference body. The target point is scanned by a positioning camera to obtain the reverse positioning matrix of the positioning camera in the reference coordinate system; and the relative relationship matrix between the 3D camera and the positioning camera is determined using a preset calibration method; wherein there are multiple positioning cameras, and the 3D camera is rigidly connected to the positioning camera. The 3D camera scans the object to be measured placed in the measurement scene to obtain the point cloud data of the object to be measured. Based on the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera, the point cloud data of the object to be measured is transformed into the reference coordinate system to obtain the coordinates of the object to be measured in the reference coordinate system. The step of scanning the target point with a positioning camera to obtain the inverse positioning matrix of the positioning camera in the reference coordinate system includes: The positioning camera scans at least three target points, and determines the reverse positioning matrix of the positioning camera in the reference coordinate system based on the focal center of the positioning camera, the coordinates of the target points in the reference coordinate system, and the coordinates of the target points in the positioning camera coordinate system.

2. The three-dimensional scanning method based on reverse positioning and 3D camera fusion according to claim 1, characterized in that, Using a preset calibration method, the relative relationship matrix between the 3D camera and the positioning camera is determined, including: The 3D camera is used to scan the preset geometric objects placed in the measurement scene under different poses to obtain a set of calibration point coordinates of multiple preset geometric objects in the 3D camera coordinate system. While the 3D camera is scanning, the positioning camera scans the target points in the measurement scene to determine the calibration matrix of the positioning camera under different poses of the 3D camera. Initialize a preset number of initial relationship matrices between the 3D cameras and the positioning cameras. Based on the initial relationship matrix and the calibration matrix, transform the coordinate set of the calibration points to the reference coordinate system to obtain the coordinate set of the calibration points in the reference coordinate system. The initial relationship matrix is ​​optimized based on the coordinate overlap in the calibration point reference coordinate system, and the relative relationship matrix between the 3D camera and the positioning camera is obtained based on the optimized initial relationship matrix.

3. The three-dimensional scanning method based on reverse positioning and 3D camera fusion according to claim 2, characterized in that, The initial relationship matrix is ​​optimized based on the coordinate overlap in the calibration point reference coordinate system. The relative relationship matrix between the 3D camera and the positioning camera is then obtained from the optimized initial relationship matrix, including: The initial relation matrix with the highest coordinate overlap of the calibration point coordinate set in the reference coordinate system is taken as the relation matrix to be optimized. The relationship matrix to be optimized is optimized using the confidence region method. The coordinates of the calibration point in the reference coordinate system are calculated based on the optimized relationship matrix, and the degree of overlap of the coordinates is calculated. When the overlap converges, the optimal relation matrix is ​​obtained, and the optimal relation matrix is ​​used as the relative relation matrix between the 3D camera and the positioning camera.

4. The three-dimensional scanning method based on reverse positioning and 3D camera fusion according to claim 1, characterized in that, Based on the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera, the point cloud data of the object is transformed into the reference coordinate system to obtain the coordinates of the object in the reference coordinate system, including: Based on the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera, the point cloud transformation matrix between the 3D camera coordinate system and the reference coordinate system is obtained; Based on the point cloud transformation matrix, the point cloud data is transformed into the reference coordinate system to obtain the coordinates of the item in the reference coordinate system.

5. A three-dimensional scanning method based on reverse positioning and 3D camera fusion according to claim 1, characterized in that, After scanning the object placed in the measurement scene using the 3D camera to obtain the point cloud data of the object, the method further includes: Based on the point cloud data, a preset shape extraction method is used to determine whether the item contains a preset shape.

6. A three-dimensional scanning method based on reverse positioning and 3D camera fusion according to claim 5, characterized in that, Based on the point cloud data, a preset shape extraction method is used to determine whether the item contains a preset shape, including: Calculate the curvature of each point in the point cloud data, and designate points with curvature greater than a preset curvature threshold as edge points; The edge points are connected using a connected component search method to obtain the edge lines; The edge line is fitted using a preset fitting method. When the fitting result meets the preset shape judgment criteria, it is determined that the item contains a preset shape.

7. The three-dimensional scanning method based on reverse positioning and 3D camera fusion according to claim 1, characterized in that, When the time taken for the 3D camera to scan the object to be measured placed in the measurement scene exceeds a preset time threshold, the method further includes: While the 3D camera scans the object under test, the positioning camera acquires the coordinates of multiple target points, and the anti-shake reverse positioning matrix of the positioning camera is obtained based on the coordinates of the multiple target points. Based on the anti-shake reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera, the point cloud data of the object under test is transformed into the reference coordinate system to obtain the coordinates of the object under test in the reference coordinate system.

8. A three-dimensional scanning device based on reverse positioning and 3D camera fusion, characterized in that, include: The scene establishment module is used to establish a reference coordinate system for the measurement scene using a stable reference body as the measurement scene, and to set multiple target points on the stable reference body. The calibration module is used to scan the target point with the positioning camera to obtain the reverse positioning matrix of the positioning camera in the reference coordinate system; and to determine the relative relationship matrix between the 3D camera and the positioning camera using a preset calibration method; wherein there are multiple positioning cameras, and the 3D camera is rigidly connected to the positioning camera. The conversion module is used to scan the object placed in the measurement scene using the 3D camera to obtain the point cloud data of the object, and convert the point cloud data of the object to the reference coordinate system according to the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera to obtain the coordinates of the object in the reference coordinate system. The step of scanning the target point with a positioning camera to obtain the inverse positioning matrix of the positioning camera in the reference coordinate system includes: The positioning camera scans at least three target points, and determines the reverse positioning matrix of the positioning camera in the reference coordinate system based on the focal center of the positioning camera, the coordinates of the target points in the reference coordinate system, and the coordinates of the target points in the positioning camera coordinate system.

9. A three-dimensional scanning system based on reverse positioning and 3D camera fusion, characterized in that, include: Stable reference body, measuring equipment, and fusion computing device; The stable reference body serves as a measurement scenario, providing a target point for the measurement device. The measuring device includes a 3D camera and a positioning camera, wherein the 3D camera and the positioning camera are rigidly connected; the measuring device is used to scan the target point and scan the object placed in the measuring scene to obtain the point cloud data of the object; The fusion computing device is used to establish a reference coordinate system for the measurement scene, obtain the reverse positioning matrix of the positioning camera in the reference coordinate system based on the scanning results of the target point by the measuring device, and determine the relative relationship matrix between the 3D camera and the positioning camera using a preset calibration method; and convert the point cloud data of the object to the reference coordinate system based on the reverse positioning matrix and the relative relationship matrix between the 3D camera and the positioning camera to obtain the coordinates of the object in the reference coordinate system.

Images