Point cloud stitching method, system and readable storage medium of 3D line scanning camera

Abstract

The present invention discloses a point cloud stitching method, system and readable storage medium for a 3D line scan camera, wherein the method comprises: after setting a calibration part on a mobile platform, scanning the calibration part with two 3D line scan cameras, respectively obtaining a first point cloud image and a second point cloud image of the calibration part and fitting three planes in the image that are not coplanar; calculating the rigid transformation matrix between the point cloud images of the two 3D line scan cameras based on the coplanarity of the same plane; transforming a point cloud image of the object to be measured by the rigid transformation matrix to obtain a transformed image; splicing the transformed image with another point cloud image to obtain an overall image. The positional relationship between the two 3D line scan cameras is calibrated by the three-plane method, and the rigid transformation matrix between the point clouds of the two 3D line scan cameras is calculated by plane fitting and matrix calculation, without the need for iteration and with fast calculation speed.

Description

Technical Field

[0001] The present invention relates to the field of industrial cameras, and more specifically, to a point cloud stitching method, system and readable storage medium of a 3D line scanning camera. Background Art

[0002] 3D line scan cameras are widely used in industry and have the following advantages: (1) Line scan cameras can have multiple K unit pixels, so the resolution characteristics are very high, which is very suitable for high-precision detection and measurement applications. At the same time, they can also be used for applications that require continuous high-resolution imaging and continuous imaging of moving objects. The measurement can be accurate to microns. (2) Line scan cameras have a very large dynamic range and high sensitivity, which is very suitable for industrial inspection projects that require a large field of view.

[0003] In actual use, line scan cameras have high requirements for coordination with industrial mechanisms (motion platforms). Generally, 3D line scan cameras are fixedly installed, and the object being measured is fixed on a high-precision motion platform and moves at a constant speed, and the movement speed is consistent with the frequency of the laser emitted by the camera. The imaging result of a 3D line scan camera is a point cloud under an orthogonal coordinate system XYZ, which requires that after the camera is fixed, its laser plane XOZ is perpendicular to the motion direction Y of the motion platform, otherwise it will cause distortion of the imaging point cloud.

[0004] In industrial applications, a single 3D line scan camera cannot meet the needs in some complex application scenarios, such as: Figure 2 , the stable moving distance of the motion platform in a single time is L1, and the length of the object to be measured is L2. When L2>L1, a single 3D line scan camera cannot capture the entire object to be measured. At this time, two 3D line scan cameras are needed to capture the object to be measured, and then the two point clouds are registered and spliced. The general point cloud registration method is the corresponding point coarse registration + IC fine registration. The ICP registration method is based on the principle of minimizing the distance between points in point cloud 1 and the nearest point in point cloud 2. It performs multiple iterations to find the overall minimum distance between each point in point cloud 1 and the nearest point in point cloud 2. This method has the following disadvantages: 1) The two point clouds must have part of the same actual space area, and the registration is based on this area, that is, there is overlap after splicing; 2) The coarse registration needs to be relatively accurate, and then the rotation and translation matrix obtained by the coarse registration is fine-tuned, otherwise it is easy to fall into the local optimum; 3) Before ICP, a point cloud is generally downsampled to reduce the number of points to be calculated, which is easy to lose features; 4) Although a point cloud is downsampled, it still has at least thousands of points, and multiple iterations are required until convergence, which is slow.

[0005] Therefore, how to design a point cloud stitching method for a 3D line scan camera to achieve simple operation, complete features, and fast calculation is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0006] In view of the above problems, the purpose of the present invention is to provide a point cloud stitching method, system and readable storage medium for a 3D line scan camera to solve the problems of complex operation, easy loss of features and slow calculation speed in the point cloud image stitching of the prior art.

[0007] A first aspect of the present invention provides a point cloud stitching method of a 3D line scanning camera applied to a 3D line scanning device, wherein the 3D line scanning device comprises two 3D line scanning cameras, a mobile platform and a calibration piece, wherein the calibration piece is a standard block composed of at least three non-coplanar structures, and the method comprises the following steps:

[0008] After the calibration object is placed on the mobile platform, the calibration object is scanned by the two 3D line scanning cameras to obtain a first point cloud image and a second point cloud image of the calibration object respectively;

[0009] Fitting three planes in the first point cloud image and the second point cloud image that are not coplanar in normal phase respectively, and the plane equations of the same plane in the two point cloud images are referred to as a set of plane equations;

[0010] Calculating a rigid transformation matrix between the point cloud images of the two 3D line scan cameras according to the coplanarity of the three sets of plane equations;

[0011] A point cloud image of the object to be measured is transformed by the rigid transformation matrix to obtain a transformed image;

[0012] The converted image is spliced ​​with another point cloud image to obtain an overall image.

[0013] In this solution, before fitting the three planes in the first point cloud image and the second point cloud image that are not coplanar in normal phase, and calling the plane equations of the same plane in the two point cloud images as a set of plane equations, it also includes:

[0014] Correct the distortion of the first point cloud image and the second point cloud image caused by camera installation error.

[0015] In this solution, the correction of the distortion of the first point cloud image and the second point cloud image caused by the camera installation error is specifically:

[0016] According to the consistency of distances between fixed point pairs in the first point cloud image and the second point cloud image in different postures, distortion of the first point cloud image and the second point cloud image caused by camera installation errors is corrected.

[0017] In this solution, the calculation of the rigid transformation matrix between the point cloud images of the two 3D line scan cameras according to the coplanarity of the three sets of plane equations specifically includes:

[0018] According to the coplanarity of the three sets of plane equations, a transformation matrix composed of coefficients of the three sets of plane equations is calculated;

[0019] The elements in the transformation matrix are restricted according to the rigid transformation properties between the point cloud images of the two 3D line scanning cameras to obtain a rigid transformation matrix.

[0020] In this solution, the fitting of three planes in the first point cloud image and the second point cloud image that are not coplanar in normal phase is specifically as follows:

[0021] The least square method is used to perform plane fitting on three non-coplanar planes in the first point cloud image and the second point cloud image respectively.

[0022] The second aspect of the present invention further provides a point cloud stitching system for a 3D line scanning camera, comprising a 3D line scanning device, a memory and a processor, wherein the 3D line scanning device comprises two 3D line scanning cameras, a mobile platform and a calibration piece, wherein the calibration piece is a standard block composed of at least three non-coplanar structures, and the memory comprises a point cloud stitching method program for the 3D line scanning camera, and when the point cloud stitching method program for the 3D line scanning camera is executed by the processor, the following steps are implemented:

[0023] After the calibration object is placed on the mobile platform, the calibration object is scanned by the two 3D line scanning cameras to obtain a first point cloud image and a second point cloud image of the calibration object respectively;

[0024] Fitting three planes in the first point cloud image and the second point cloud image that are not coplanar in normal phase respectively, and the plane equations of the same plane in the two point cloud images are referred to as a set of plane equations;

[0025] Calculating a rigid transformation matrix between the point cloud images of the two 3D line scan cameras according to the coplanarity of the three sets of plane equations;

[0026] A point cloud image of the object to be measured is transformed by the rigid transformation matrix to obtain a transformed image;

[0027] The converted image is spliced ​​with another point cloud image to obtain an overall image.

[0028] In this solution, before fitting the three planes in the first point cloud image and the second point cloud image that are not coplanar in normal phase, and calling the plane equations of the same plane in the two point cloud images as a set of plane equations, it also includes:

[0029] Correct the distortion of the first point cloud image and the second point cloud image caused by camera installation error.

[0030] In this solution, the correction of the distortion of the first point cloud image and the second point cloud image caused by the camera installation error is specifically:

[0031] According to the consistency of distances between fixed point pairs in the first point cloud image and the second point cloud image in different postures, distortion of the first point cloud image and the second point cloud image caused by camera installation errors is corrected.

[0032] In this solution, the calculation of the rigid transformation matrix between the point cloud images of the two 3D line scan cameras according to the coplanarity of the three sets of plane equations specifically includes:

[0033] According to the coplanarity of the three sets of plane equations, a transformation matrix composed of coefficients of the three sets of plane equations is calculated;

[0034] The elements in the transformation matrix are restricted according to the rigid transformation properties between the point cloud images of the two 3D line scanning cameras to obtain a rigid transformation matrix.

[0035] A third aspect of the present invention provides a computer-readable storage medium, which includes a point cloud stitching method program for a 3D line scanning camera of a machine. When the point cloud stitching method program for a 3D line scanning camera is executed by a processor, the steps of the point cloud stitching method for a 3D line scanning camera as described in any one of the above items are implemented.

[0036] The present invention discloses a point cloud stitching method for a 3D line scanning camera, which is applied to a 3D line scanning device. The 3D line scanning device includes two 3D line scanning cameras, a mobile platform and a calibration piece, wherein the calibration piece is a standard block composed of at least three structures with non-coplanar normals. The method includes the following steps: after the calibration piece is set on the mobile platform, the calibration piece is scanned by two 3D line scanning cameras to obtain a first point cloud image and a second point cloud image of the calibration piece respectively; three planes with non-coplanar normals in the first point cloud image and the second point cloud image are fitted respectively, and the plane equations of the same plane in the two point cloud images are called a set of plane equation groups; according to the coplanarity of the three sets of plane equation groups, a rigid transformation matrix between the point cloud images of the two 3D line scanning cameras is calculated; a point cloud image of the object to be measured is transformed by the rigid transformation matrix to obtain a transformed image; and the transformed image is spliced ​​with another point cloud image to obtain an overall image.

[0037] This method uses two 3D line scan cameras, which are respectively installed at the starting point and the end point of the motion path of the motion platform, to scan and shoot the front half and the back half of the object to be measured respectively. The positional relationship between the two 3D line scan cameras is calibrated by the three-plane method. The rigid transformation matrix between the point clouds of the two 3D line scan cameras is calculated by plane fitting and matrix calculation, so that the two point clouds taken by the two 3D line scan cameras can be spliced ​​into a point cloud to fully represent the object to be measured. No iteration is required and the calculation speed is fast. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A flow chart of a point cloud stitching method of a 3D line scan camera of the present application is shown;

[0039] Figure 2 shows a schematic structural diagram of a 3D line scanning device;

[0040] Figure 3 A schematic diagram of the structure of the calibration component in an embodiment of the present application is shown;

[0041] Figure 4 A schematic diagram showing the distortion of a scanned image due to camera installation errors;

[0042] Figure 5 A block diagram of a point cloud stitching system of a 3D line scanning camera of the present invention is shown. DETAILED DESCRIPTION

[0043] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0044] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0045] Figure 1 A flow chart of a point cloud stitching method of a 3D line scanning camera of the present application is shown.

[0046] like Figure 1 As shown, the present application discloses a point cloud stitching method of a 3D line scanning camera, which is applied to a 3D line scanning device, wherein the 3D line scanning device includes two 3D line scanning cameras, a mobile platform and a calibration piece, wherein the calibration piece is a standard block composed of at least three non-coplanar structures, and includes the following steps:

[0047] S102, after placing the calibration object on the mobile platform, scanning the calibration object with the two 3D line scanning cameras to obtain a first point cloud image and a second point cloud image of the calibration object respectively;

[0048] S104, respectively fitting three planes in the first point cloud image and the second point cloud image that are not coplanar in normal direction, and referring the plane equations of the same plane in the two point cloud images as a set of plane equations;

[0049] S106, calculating a rigid transformation matrix between the point cloud images of the two 3D line scan cameras according to the coplanarity of the three sets of plane equations;

[0050] S108, transforming a point cloud image of the object to be measured by using the rigid transformation matrix to obtain a transformed image;

[0051] S110, stitching the converted image with another point cloud image to obtain an overall image.

[0052] It should be noted that the present invention uses two 3D line scanning cameras, which are respectively installed at the starting point and the end point of the motion path of the motion platform, to scan and photograph the front half and the second half of the object to be measured, and calibrate the positional relationship between the two 3D line scanning cameras through the three-plane method, so that the two point clouds taken by the two 3D line scanning cameras can be spliced ​​into one point cloud to fully represent the object to be measured.

[0053] It should be noted that the calibration piece is any rigid material block that is easy to obtain characteristic points, and does not need to be specially made. Figure 3 , Figure 3 The schematic diagram of the structure of the calibration part in the embodiment of the present application is shown. To calibrate the positional relationship between two 3D line scan cameras and perform point cloud stitching, only three flatness standard blocks are needed to build a structure in which three extended planes can intersect at one point. The construction can be more casual, without precise restrictions, and the operation is convenient. The two 3D line scan cameras can both capture these three planes, and the fields of view do not need to overlap. By utilizing the coplanarity of the same plane and the common plane position relationship, the rigid transformation matrix between the point clouds of the two 3D line scan cameras is calculated through plane fitting. No iteration is required, and the calculation speed is fast.

[0054] According to an embodiment of the present invention, before respectively fitting three planes whose normals are not coplanar in the first point cloud image and the second point cloud image and calling the plane equations of the same plane in the two point cloud images as a set of plane equations, the method further includes:

[0055] Correct the distortion of the first point cloud image and the second point cloud image caused by camera installation error.

[0056] It should be noted that due to possible installation errors, the light plane of the scanning camera may not be perpendicular to the movement direction of the mobile platform, which will cause distortion between the imaged point cloud image and the three-dimensional structure of the actual object. Therefore, before fitting the plane in the point cloud image, it is necessary to eliminate the distortion of the first point cloud image and the second point cloud image caused by the camera installation error.

[0057] According to an embodiment of the present invention, the correction of the distortion of the first point cloud image and the second point cloud image caused by the camera installation error is specifically:

[0058] According to the consistency of distances between fixed point pairs in the first point cloud image and the second point cloud image in different postures, distortion of the first point cloud image and the second point cloud image caused by camera installation errors is corrected.

[0059] Please note that Figure 4 , the light plane XOZ of the 3D line scan camera rotates around the X axis by an angle of θ, that is, the distorted coordinate system is the Z axis in the actual coordinate system rotated around the X axis by an angle of θ, and the X axis and Y axis remain unchanged. The distortion correction matrix is ​​in the form of:

[0060] Select the easily graspable feature points A and B of the calibration part, and place them in any posture greater than 2. Use a 3D line scan camera to shoot the calibration part in each posture, and obtain the positions of points A and B in the point cloud. Record [X A Y A Z A ]、[X B Y B Z B ] and calculate [ΔX ΔY ΔZ] of points A and B in each posture.

[0061] Let the position in the actual coordinate system be expressed as [X′ Y′ Z′], because the distance between points A and B in the actual coordinate system is constant in different postures, that is: Constant.

[0062] and Therefore, any two different postures i and j satisfy:

[0063]

[0064] Right now:

[0065]

[0066] make:

[0067]

[0068] Each posture combination can be listed as a set of equations Yes ij b ij ] to perform SVD decomposition and solve for sinθ. The sin function is Monotonically increasing, the Rx installation deflection angle of -90° to 90° can be uniquely determined to obtain the correction matrix:

[0069]

[0070] The first point cloud image and the second point cloud image are corrected using a correction matrix.

[0071] According to an embodiment of the present invention, calculating the rigid transformation matrix between the point cloud images of the two 3D line scan cameras according to the coplanarity of the three sets of plane equations specifically includes:

[0072] According to the coplanarity of the three sets of plane equations, a transformation matrix composed of coefficients of the three sets of plane equations is calculated;

[0073] The elements in the transformation matrix are restricted according to the rigid transformation properties between the point cloud images of the two 3D line scanning cameras to obtain a rigid transformation matrix.

[0074] It should be noted that if Figure 3 As shown in the figure, three standard blocks of flatness are used to build a structure with three non-coplanar normals. Since the rigid transformation is a translation or rotation along the x, y, and z axes, there are 6 degrees of freedom transformations. Since the three planes with non-coplanar normals will intersect at one point, the translation amount can be constrained; at the same time, since the normals of the three planes can constrain the rotation amount, a unique three-dimensional space can be constrained.

[0075] Two 3D line scan cameras are used to respectively shoot the three planes. After the point cloud images shot by the two cameras are respectively corrected using the correction matrices of the distortion of the two cameras obtained by the above calibration, there is only a rigid transformation between the coordinate systems of the two 3D line scan cameras.

[0076] Fit three non-coplanar planes in two coordinate systems respectively, the plane equation is: aX+bY+cZ+d=0, for the two plane equations of the same plane in two point cloud images, then:

[0077]

[0078] where a 1 、b 1 、c 1 is the coordinate system X in the first point cloud image 1 Y 1 Z 1 The coefficient of the plane equation, a 2 、b 2、c 2 is the coordinate system X in the point cloud image 2 Y 2 Z 2 The coefficients of the plane equation in . The plane equations of the same plane in two point cloud images are called a set of plane equations.

[0079] The transformation matrix of two 3D line scan cameras is calculated based on three sets of plane equations. Let the transformation matrix be H, then: [X 1 Y 1 Z 1 1] T =H[X 2 Y 2 Z 2 1] T . Then we have:

[0080]

[0081] Since there are three sets of corresponding planes, then:

[0082]

[0083] Understandably, a ij It represents the jth plane in the i-th point cloud image. The annotations of other coefficients are similar and will not be described here.

[0084] Because the point cloud is not distorted and the two cameras shoot the same three planes, the transformation matrix H is ideally a rigid transformation matrix. However, due to possible data errors, it is impossible to guarantee that the transformation matrix H is a standard rigid transformation matrix. To obtain a rigid transformation matrix, the data errors in the transformation matrix H must be eliminated. The transformation matrix H can be expressed as the product of a matrix A and a rigid transformation matrix. A can be calculated first, and then combined with the transformation matrix H to finally obtain the rigid transformation matrix.

[0085] First, we need to decompose the rigid transformation matrix from the transformation matrix H. The first three rows and three columns of the rigid transformation matrix are a rotation matrix: R = [R 1 R 2 R 3 ], each column should be a unit vector and two-to-two orthogonal, and the unit orthogonal constraint equation is as follows:

[0086]

[0087] Then calculate the error matrix A:

[0088] Assuming that H=A[R1 R2 R3 T] in the previous step, we have:

[0089] R1=A -1H1, R2 = A -1 H2, R3=A -1 H3;

[0090]

[0091] a) Note: B=A -T A -1 , then B is a symmetric matrix:

[0092]

[0093] have:

[0094]

[0095] make:

[0096] v ij =[H 1i H 1j H 1i H 2j +H 2i H 1j H 2i H 2j H 1i H 3j +H 3i H 1j H 2i H 3j +H 3i H 2j H 3i H 3j ];

[0097] The unit orthogonal constraint equation can be transformed into:

[0098]

[0099] Thus, B is solved.

[0100] b) Next, B calculates A -1 .make: but:

[0101]

[0102]

[0103] c) Solved A -1 , H is known, the rigid transformation matrix is:

[0104] [R1 R2 R3 T]=A -1 H.

[0105] According to an embodiment of the present invention, the fitting of three planes in the first point cloud image and the second point cloud image that are not coplanar in normal phase is specifically as follows:

[0106] The least square method is used to perform plane fitting on three non-coplanar planes in the first point cloud image and the second point cloud image respectively.

[0107] It should be noted that, when fitting a plane in a point cloud image, the least square method may be used for plane fitting.

[0108] Figure 5 A block diagram of a point cloud stitching system of a 3D line scanning camera of the present invention is shown.

[0109] like Figure 5 As shown, the present invention discloses a point cloud stitching system 5 of a 3D line scanning camera, comprising a 3D line scanning device 51, a memory 52 and a processor 53, wherein the 3D line scanning device 51 comprises two 3D line scanning cameras, a mobile platform and a calibration component, wherein the calibration component is a standard block composed of at least three normal non-coplanar structures, and the memory 53 comprises a point cloud stitching method program of the 3D line scanning camera, and when the point cloud stitching method program of the 3D line scanning camera is executed by the processor 53, the following steps are implemented:

[0110] After the calibration object is placed on the mobile platform, the calibration object is scanned by the two 3D line scanning cameras to obtain a first point cloud image and a second point cloud image of the calibration object respectively;

[0111] Fitting three planes in the first point cloud image and the second point cloud image that are not coplanar in normal phase respectively, and the plane equations of the same plane in the two point cloud images are referred to as a set of plane equations;

[0112] Calculating a rigid transformation matrix between the point cloud images of the two 3D line scan cameras according to the coplanarity of the three sets of plane equations;

[0113] A point cloud image of the object to be measured is transformed by the rigid transformation matrix to obtain a transformed image;

[0114] The converted image is spliced ​​with another point cloud image to obtain an overall image.

[0115] It should be noted that the present invention uses two 3D line scanning cameras, which are respectively installed at the starting point and the end point of the motion path of the motion platform, to scan and photograph the front half and the second half of the object to be measured, and calibrate the positional relationship between the two 3D line scanning cameras through the three-plane method, so that the two point clouds taken by the two 3D line scanning cameras can be spliced ​​into one point cloud to fully represent the object to be measured.

[0116] It should be noted that the calibration piece is any rigid material block that is easy to obtain characteristic points, and does not need to be specially made. Figure 3 , Figure 3 The schematic diagram of the structure of the calibration part in the embodiment of the present application is shown. To calibrate the positional relationship between two 3D line scan cameras and perform point cloud stitching, only three flatness standard blocks are needed to build a structure in which three extended planes can intersect at one point. The construction can be more casual, without precise restrictions, and the operation is convenient. The two 3D line scan cameras can both capture these three planes, and the fields of view do not need to overlap. By utilizing the coplanarity of the same plane and the common plane position relationship, the rigid transformation matrix between the point clouds of the two 3D line scan cameras is calculated through plane fitting. No iteration is required, and the calculation speed is fast.

[0117] According to an embodiment of the present invention, before respectively fitting three planes whose normals are not coplanar in the first point cloud image and the second point cloud image and calling the plane equations of the same plane in the two point cloud images as a set of plane equations, the method further includes:

[0118] Correct the distortion of the first point cloud image and the second point cloud image caused by camera installation error.

[0119] It should be noted that due to possible installation errors, the light plane of the scanning camera may not be perpendicular to the movement direction of the mobile platform, which will cause distortion between the imaged point cloud image and the three-dimensional structure of the actual object. Therefore, before fitting the plane in the point cloud image, it is necessary to eliminate the distortion of the first point cloud image and the second point cloud image caused by the camera installation error.

[0120] According to an embodiment of the present invention, the correction of the distortion of the first point cloud image and the second point cloud image caused by the camera installation error is specifically:

[0121] According to the consistency of distances between fixed point pairs in the first point cloud image and the second point cloud image in different postures, distortion of the first point cloud image and the second point cloud image caused by camera installation errors is corrected.

[0122] Please note that Figure 4 , the light plane XOZ of the 3D line scan camera rotates around the X axis by an angle of θ, that is, the distorted coordinate system is the Z axis in the actual coordinate system rotated around the X axis by an angle of θ, and the X axis and Y axis remain unchanged. The distortion correction matrix is ​​in the form of:

[0123] Select the easily graspable feature points A and B of the calibration part, and place them in any posture greater than 2. Use a 3D line scan camera to shoot the calibration part in each posture, and obtain the positions of points A and B in the point cloud. Record [X A Y A ZA ]、[X B Y B Z B ] and calculate [ΔX ΔY ΔZ] of points A and B in each posture.

[0124] Let the position in the actual coordinate system be expressed as [X′ Y′ Z′], because the distance between points A and B in the actual coordinate system is constant in different postures, that is: Constant.

[0125] and Therefore, any two different postures i and j satisfy:

[0126]

[0127] Right now:

[0128]

[0129] make:

[0130]

[0131] Each posture combination can be listed as a set of equations Yes ij b ij ] to perform SVD decomposition and solve for sinθ. The sin function is Monotonically increasing, the Rx installation deflection angle of -90° to 90° can be uniquely determined to obtain the correction matrix:

[0132]

[0133] The first point cloud image and the second point cloud image are corrected using a correction matrix.

[0134] According to an embodiment of the present invention, calculating the rigid transformation matrix between the point cloud images of the two 3D line scan cameras according to the coplanarity of the three sets of plane equations specifically includes:

[0135] According to the coplanarity of the three sets of plane equations, a transformation matrix composed of coefficients of the three sets of plane equations is calculated;

[0136] The elements in the transformation matrix are restricted according to the rigid transformation properties between the point cloud images of the two 3D line scanning cameras to obtain a rigid transformation matrix.

[0137] It should be noted that if Figure 3As shown in the figure, three standard blocks of flatness are used to build a structure with three non-coplanar normals. Since the rigid transformation is a translation or rotation along the x, y, and z axes, there are 6 degrees of freedom transformations. Since the three planes with non-coplanar normals will intersect at one point, the translation amount can be constrained; at the same time, since the normals of the three planes can constrain the rotation amount, a unique three-dimensional space can be constrained.

[0138] Two 3D line scan cameras are used to respectively shoot the three planes. After the point cloud images shot by the two cameras are respectively corrected using the correction matrices of the distortion of the two cameras obtained by the above calibration, there is only a rigid transformation between the coordinate systems of the two 3D line scan cameras.

[0139] Fit three non-coplanar planes in two coordinate systems respectively, the plane equation is: aX+bY+cZ+d=0, for the two plane equations of the same plane in two point cloud images, then:

[0140]

[0141] where a 1 、b 1 、c 1 is the coordinate system X in the first point cloud image 1 Y 1 Z 1 The coefficient of the plane equation, a 2 、b 2 、c 2 is the coordinate system X in the point cloud image 2 Y 2 Z 2 The coefficients of the plane equation in . The plane equations of the same plane in two point cloud images are called a set of plane equations.

[0142] The transformation matrix of two 3D line scan cameras is calculated based on three sets of plane equations. Let the transformation matrix be H, then: [X 1 Y 1 Z 1 1] T =H[X 2 Y 2 Z 2 1] T . Then we have:

[0143]

[0144] Since there are three sets of corresponding planes, then:

[0145]

[0146] Understandably, a ijIt represents the jth plane in the i-th point cloud image. The annotations of other coefficients are similar and will not be described here.

[0147] Because the point cloud is not distorted and the two cameras shoot the same three planes, the transformation matrix H is ideally a rigid transformation matrix. However, due to possible data errors, it is impossible to guarantee that the transformation matrix H is a standard rigid transformation matrix. To obtain a rigid transformation matrix, the data errors in the transformation matrix H must be eliminated. The transformation matrix H can be expressed as the product of a matrix A and a rigid transformation matrix. A can be calculated first, and then combined with the transformation matrix H to finally obtain the rigid transformation matrix.

[0148] First, the rigid transformation matrix needs to be decomposed from the transformation matrix H. The first three rows and three columns of the rigid transformation matrix are a rotation matrix: R = [R1 R2 R3], which should satisfy that each column is a unit vector and is orthogonal to each other, and has a unit orthogonal constraint equation as follows:

[0149]

[0150] Assuming that H=A[R1 R2 R3 T] in the previous step, we have:

[0151] R1=A -1 H1, R2 = A -1 H2, R3=A -1 H3;

[0152]

[0153] a) Note: B=A -T A -1 , then B is a symmetric matrix:

[0154]

[0155] have:

[0156]

[0157] make:

[0158] v ij =[H 1i H 1j H 1i H 2j +H 2i H 1j H 2i H 2j H 1i H 3j +H 3i H 1j H 2i H 3j +H3i H 2j H 3i H 3j ];

[0159] The unit orthogonal constraint equation can be transformed into:

[0160]

[0161] Thus, B is solved.

[0162] b) Next, B calculates A -1 .make: but:

[0163]

[0164]

[0165] c) Solved A -1 , H is known, the rigid transformation matrix is:

[0166] [R1 R2 R3 T]=A -1 H.

[0167] A third aspect of the present invention provides a computer-readable storage medium, which includes a point cloud stitching method program for a 3D line scanning camera of a machine. When the point cloud stitching method program for a 3D line scanning camera is executed by a processor, the steps of the point cloud stitching method for a 3D line scanning camera as described in any one of the above items are implemented.

[0168] The present invention discloses a point cloud stitching method and system of a 3D line scanning camera and a readable storage medium. Two 3D line scanning cameras are respectively installed at the starting point and the end point of the motion path of a motion platform to respectively scan and shoot the front half and the back half of the object to be measured. The positional relationship between the two 3D line scanning cameras is calibrated by a three-plane method. The rigid transformation matrix between the point clouds of the two 3D line scanning cameras is calculated by plane fitting, primary matrix calculation, and rigid matrix decomposition. The two point clouds taken by the two 3D line scanning cameras can be stitched into one point cloud to fully represent the object to be measured. No iteration is required and the calculation speed is fast.

[0169] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0170] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0171] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0172] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0173] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

Claims

1. A point cloud stitching method for a 3D line scan camera, characterized in that: Applied to a 3D line scanning device, the 3D line scanning device includes two 3D line scanning cameras, a mobile platform and a calibration piece, the calibration piece is a standard block composed of at least three normal non-coplanar structures, the method includes the following steps: After the calibration object is placed on the mobile platform, the calibration object is scanned by the two 3D line scanning cameras to obtain a first point cloud image and a second point cloud image of the calibration object respectively; Fitting three planes whose normals are not coplanar in the first point cloud image and the second point cloud image respectively, and calling the plane equations of the same plane in the two point cloud images a set of plane equations; Calculating a rigid transformation matrix between the point cloud images of the two 3D line scan cameras according to the coplanarity of the three sets of plane equations; A point cloud image of the object to be measured is transformed by the rigid transformation matrix to obtain a transformed image; splicing the converted image with another point cloud image to obtain an overall image; The step of calculating the rigid transformation matrix between the point cloud images of the two 3D line scan cameras according to the coplanarity of the three sets of plane equations specifically includes: According to the coplanarity of the three sets of plane equations, a transformation matrix composed of coefficients of the three sets of plane equations is calculated; Restricting the elements in the transformation matrix according to the rigid transformation properties between the point cloud images of the two 3D line scanning cameras to obtain a rigid transformation matrix; The fitting of three planes in which the normals of the first point cloud image and the second point cloud image are not coplanar is specifically as follows: Plane fitting is performed on three planes whose normal directions are not coplanar in the first point cloud image and the second point cloud image respectively using the least square method.

2. The point cloud stitching method of a 3D line scan camera according to claim 1, characterized in that: Before respectively fitting three planes in the first point cloud image and the second point cloud image that are not coplanar in normal direction and calling the plane equations of the same plane in the two point cloud images as a set of plane equations, the method further includes: Correct the distortion of the first point cloud image and the second point cloud image caused by camera installation error.

3. The point cloud stitching method of a 3D line scan camera according to claim 2, characterized in that: The correction of the distortion of the first point cloud image and the second point cloud image caused by the camera installation error is specifically as follows: According to the consistency of distances between fixed point pairs in the first point cloud image and the second point cloud image in different postures, distortion of the first point cloud image and the second point cloud image caused by camera installation errors is corrected.

4. A point cloud stitching system for a 3D line scan camera, characterized in that: The invention comprises a 3D line scanning device, a memory and a processor, wherein the 3D line scanning device comprises two 3D line scanning cameras, a mobile platform and a calibration piece, wherein the calibration piece is a standard block composed of at least three normal non-coplanar structures, and the memory comprises a point cloud stitching method program of the 3D line scanning camera, and when the point cloud stitching method program of the 3D line scanning camera is executed by the processor, the following steps are implemented: After the calibration object is placed on the mobile platform, the calibration object is scanned by the two 3D line scanning cameras to obtain a first point cloud image and a second point cloud image of the calibration object respectively; Fitting three planes whose normals are not coplanar in the first point cloud image and the second point cloud image respectively, and calling the plane equations of the same plane in the two point cloud images a set of plane equations; Calculating a rigid transformation matrix between the point cloud images of the two 3D line scan cameras according to the coplanarity of the three sets of plane equations; A point cloud image of the object to be measured is transformed by the rigid transformation matrix to obtain a transformed image; splicing the converted image with another point cloud image to obtain an overall image; The step of calculating the rigid transformation matrix between the point cloud images of the two 3D line scan cameras according to the coplanarity of the three sets of plane equations specifically includes: According to the coplanarity of the three sets of plane equations, a transformation matrix composed of coefficients of the three sets of plane equations is calculated; Restricting the elements in the transformation matrix according to the rigid transformation properties between the point cloud images of the two 3D line scanning cameras to obtain a rigid transformation matrix; The fitting of three planes in which the normals of the first point cloud image and the second point cloud image are not coplanar is specifically as follows: Plane fitting is performed on three planes whose normal directions are not coplanar in the first point cloud image and the second point cloud image respectively using the least square method.

5. The point cloud stitching system of the 3D line scanning camera according to claim 4, characterized in that: Before respectively fitting three planes in the first point cloud image and the second point cloud image that are not coplanar in normal direction and calling the plane equations of the same plane in the two point cloud images as a set of plane equations, the method further includes: Correct the distortion of the first point cloud image and the second point cloud image caused by camera installation error.

6. The point cloud stitching system of the 3D line scanning camera according to claim 5, characterized in that: The correction of the distortion of the first point cloud image and the second point cloud image caused by the camera installation error is specifically as follows: According to the consistency of distances between fixed point pairs in the first point cloud image and the second point cloud image in different postures, distortion of the first point cloud image and the second point cloud image caused by camera installation errors is corrected.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a point cloud stitching method program for a 3D line scan camera. When the point cloud stitching method program for a 3D line scan camera is executed by a processor, the steps of the point cloud stitching method for a 3D line scan camera as described in any one of claims 1 to 3 are implemented.

Images