A three-dimensional topography method for large-scale cuboids in a mobile state

Abstract

The application discloses a kind of three-dimensional topography methods for large-scale cuboid in mobile state, in point cloud segmentation and denoising, direct filtering algorithm, radius filtering method and RANSAC plane segmentation method are established, and the object point cloud dataset of effective area is obtained, and the interference of irrelevant area data and noise is reduced.In the aspect of point cloud downsampling, the improved VoxelGrid filtering algorithm is established, so as to appropriately reduce the density of point cloud data without affecting the surface details and characteristics of the target object.According to the acquisition mode and the geometric characteristics of cuboid object shape, the characteristic vector is calculated using PCA method, so that the first and second principal axes are reversed to obtain the coarse rotation matrix, and the centroid is used to calculate the translation vector method for coarse registration.Then, the point-plane ICP algorithm is used for fine registration, and the registration point cloud set of large-scale cubic object is obtained.Finally, the greedy projection is used for surface reconstruction of the registration point cloud, and the three-dimensional topography model of large-scale cubic object is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of point cloud three-dimensional reconstruction, and particularly relates to a three-dimensional topography method for large-scale cuboids in a moving state. BACKGROUND

[0002] With the proposal of China's high-quality economic development strategy, the national economy is booming, and the demand for products of basic industries such as metals, chemicals and new materials in various fields is increasing. Many products have cuboid shape structures, such as metal slabs represented by billets as important production raw materials, known as "industrial food", which have been widely used in various national defense industries, basic equipment, construction and other fields; wooden boards, as the only degradable, renewable and green environmental protection material among the four modern basic materials, have long played an important role in furniture, decoration, construction and other fields; carbon fiber composite boards have played an indispensable role in high-end equipment manufacturing, aerospace and other fields due to their high strength, high modulus and high temperature resistance.

[0003] Due to factors such as raw materials, production processes, manufacturing equipment and production environment, these cuboid objects have shape quality defects, which need to be monitored in real time, and in addition, due to the need of process flow, accurate shape size measurement is also required. SUMMARY

[0004] 1. Technical problem to be solved

[0005] The purpose of the present application is to solve the problem that there is no method for three-dimensional topography reconstruction of large-scale cuboid objects in the prior art in order to monitor the shape of the cuboid object, and a three-dimensional topography method for large-scale cuboids in a moving state is proposed.

[0006] 2. Technical solution

[0007] In order to achieve the above purpose, the present application adopts the following technical solution:

[0008] A three-dimensional topography method for large-scale cuboids in a moving state, comprising the following steps:

[0009] Step 1: build a double-view linear laser point cloud automatic acquisition system;

[0010] Step 2: calibrate the three-dimensional laser linear array camera using Zhang Zhengyou calibration method;

[0011] Step 3: move the large-scale cuboid object using a roller, before the cuboid object reaches the gantry position, trigger the sensor to start and send a collection signal to the host computer, after receiving the signal, the host computer triggers two three-dimensional laser linear array cameras at the same time through software, and the three-dimensional laser linear array cameras start to collect, obtaining two point cloud sets M and N.

[0012] Step 4: Obtain the effective area point cloud: pass-through filtering and radius filtering are performed on the point cloud sets M and N respectively, then RANSAC fitting roller plane is used for segmentation, and single-view point cloud data of the cuboid is obtained, and finally radius filtering algorithm is used for noise reduction on the point cloud data set;

[0013] Step 5: Use the improved VoxelGrid filtering algorithm to downsample the point cloud, and the whole point cloud space is divided into several voxel grids;

[0014] Step 6: Calculate the principal axis by using the PCA method: use the PCA method to calculate the characteristic vector of the two point cloud sets respectively, according to the geometric feature relationship of the cuboid object point cloud, the characteristic vector corresponding to the maximum eigenvalue is approximately parallel to the Y axis; the characteristic vector corresponding to the second maximum eigenvalue is approximately parallel to the X axis; the characteristic vector corresponding to the third maximum eigenvalue is approximately parallel to the Z axis;

[0015] Step 7: Calculate the coarse registration transformation matrix, and place the two point clouds in the same coordinate system, taking A as the green point cloud and B as the red point cloud, taking A as the target point cloud and B as the source point cloud; the feature matrix of point cloud A is U A , where l A , w A , h A are the characteristic vectors; the feature matrix of point cloud B is U B , where l B , w B , h B are the characteristic vectors, because the cuboid object is arranged horizontally in the Y direction under the above two-view angle, and the shapes of A and B point clouds are approximately mirror-symmetric, so the first principal axis and the second principal axis of the source point cloud B are reversed, that is, l B =-l B , w B =-w B ; a new characteristic vector matrix U B can be constructed, and a rotation matrix R1 can be obtained, which can make the source point cloud B obtain the correct attitude;

[0016]

[0017] Step 8: Calculate the centroids of the two point clouds respectively,

[0018]

[0019]

[0020] Calculate the translation vector of point cloud B:

[0021] T1=X A -XB R1

[0022] The point cloud C after rough registration of the point cloud B is obtained by rotating the vector and the translation vector:

[0023] C = BR + T

[0024] Step 9: Point cloud fine matching: using point-plane ICP algorithm to match two point clouds C and A;

[0025] Step 10: applying the optimal rigid transformation matrix to the source point cloud C to perform rigid transformation, obtaining the transformed point cloud D, and then completing the registration;

[0026] Step 11: using greedy triangle projection algorithm to perform surface reconstruction on the point cloud D, obtaining a three-dimensional topographic model.

[0027] Preferably, in the step 1, the automatic acquisition system relies on a conveyor belt to convey large-scale cuboid objects to realize dynamic and real-time acquisition of point clouds.

[0028] Preferably, in the step 2, the parameter scanning line frequency of the three-dimensional laser line array camera is set to make the transverse scanning accuracy equal to the longitudinal scanning accuracy.

[0029] Preferably, in the step 3, after the cuboid object passes through the gantry completely, a sensor sends a stop acquisition signal to the host computer, and the host computer controls the three-dimensional laser line array camera to stop acquisition at the same time.

[0030] Preferably, in the step 4, for the point cloud set A and the point cloud set B, the point cloud data of the effective object to be obtained is the point cloud data of the cuboid object above the roller plane, and straight-through filtering is performed from the X, Y and Z directions to obtain the point cloud data of the cuboid and the roller plane.

[0031] Preferably, in the step 5, if the voxel grid contains n points, the theoretical coordinates of the center of mass representing the entire voxel grid can be obtained, and then the nearest neighbor search algorithm of KDtree is used for each center of mass coordinate of the voxel grid, so as to search for the nearest point as the final down-sampling point.

[0032] Preferably, in the step 9, the specific steps of the point-plane ICP algorithm are as follows: taking C as the source point cloud and A as the target point cloud, finding the corresponding nearest point A i in A through the nearest neighbor search, using the ICP algorithm of the point-plane distance to calculate the objective function E and the optimal rigid transformation matrix Z that minimizes the error; i

[0033]

[0034] ​

[0035] where A i is the normal vector of the plane i (n ix , n iy , n ix ), R is a 3x3 rotation matrix, and T is a 1x3 translation vector; each time the matching point pair point plane distance is calculated to calculate the transformation matrix process, in essence, it is to solve the rotation matrix F opt that makes the objective function E minimum, which can be expressed as follows:

[0036]

[0037] In the fine registration stage, the angle of each rotation is very small, when the rotation angle is close to 0, sinθ≈θ and cosθ≈1 can be considered, so at this time the above problem can be approximately expressed as:

[0038]

[0039] The above formula is a classic linear least squares problem, and the best transformation can be solved;

[0040] In summary, the point set C obtains the point D after the rigid transformation F, and the rigid transformation can be expressed as:

[0041] D=R2xC+T2.

[0042] 3. Beneficial effects

[0043] Compared with the prior art, the advantages of the present application are:

[0044] (1) In the present application, the point cloud of the cuboid object under the double visual angle is automatically collected, and a large-scale cuboid three-dimensional topographic model is reconstructed, the three-dimensional reconstruction model has accurate shape parameters, and the problem that contact measurement cannot measure the size and shape of a large-scale cuboid in a moving state is solved.

[0045] (2) In the present application, the straight-through filtering algorithm, RANSAC plane segmentation, and radius filtering method are used to obtain the point cloud set of the effective area of the cuboid object, thereby reducing the interference of irrelevant area data and noise. In the aspect of point cloud downsampling, an improved VoxelGrid filtering algorithm is established, so that the density of the point cloud data is appropriately reduced without affecting the details and characteristics of the target object, so as to meet the requirement of improving the processing rate of the algorithm.

[0046] (3) In the present application, the PCA method is used to calibrate the principal axis of the point cloud collected in the scene. The problem of easy misoverlap of two point clouds in the registration process in the scene is solved. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1The flowchart of the three-dimensional topography method for large-scale cuboid in mobile state provided by the present application is shown in the figure.

[0048] Figure 2 The schematic diagram of the collection system of the three-dimensional topography method for large-scale cuboid in mobile state provided by the present application is shown in the figure.

[0049] Figure 3 The collection example diagram of the three-dimensional topography method for large-scale cuboid in mobile state provided by the present application is shown in the figure.

[0050] Figure 4 The original point cloud diagram in the present application is shown in the figure.

[0051] Figure 5 The point cloud diagram after straight-through filtering, RANSAC plane segmentation, radius filtering and point cloud simplification in the present application is shown in the figure.

[0052] Figure 6 The cuboid object axial diagram in the present application is shown in the figure.

[0053] Figure 7 The double-viewpoint point cloud diagram of the cuboid object in the same coordinate system in the present application is shown in the figure.

[0054] Figure 8 The coarse registration effect diagram in the present application is shown in the figure.

[0055] Figure 9 The fine registration effect diagram in the present application is shown in the figure.

[0056] Figure 10 The cuboid object topography reconstruction effect in the present application is shown in the figure.

[0057] In the figure: 1 gantry, 2 roller, 3 three-dimensional laser line array camera, 4 trigger sensor, 5 host computer. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application.

[0059] Embodiment 1

[0060] A three-dimensional topography method for large-scale cuboid in mobile state, comprising the following steps:

[0061] Step 1: A double-viewpoint line laser point cloud automatic collection system is built, which relies on a conveyor belt to convey large-scale cuboid objects to realize dynamic and real-time collection of point clouds. The system comprises a gantry 1, a roller 2, two three-dimensional laser line array cameras 3, a trigger sensor 4, a host computer 5 and a data transmission line.

[0062] AsFigure 2 As shown in the figure, the roller 2 is a conveying device for conveying large-scale cuboid objects. The gantry 1 is across the roller 2 device, and two three-dimensional laser line array cameras 3 are installed on the beam of the gantry 1; the trigger sensor 4 is installed on one side of the roller 2, 0.5 meters in front of the gantry 1; the host computer 5 is located in the control room; the three-dimensional laser line array camera 3 and the trigger sensor 4 are connected with the host computer 5 through data transmission lines.

[0063] Step 2: Use Zhang Zhengyou calibration method to calibrate the three-dimensional laser line array camera 3. And set the parameter scanning line frequency to make the horizontal scanning accuracy equal to the vertical scanning accuracy;

[0064] Step 3: Move the large-scale cuboid object using the roller 2. Before the cuboid object reaches the position of the gantry 1, the trigger sensor 4 starts and sends a collection signal to the host computer 5, and the host computer 5 triggers two three-dimensional laser line array cameras 3 at the same time through software after receiving the signal, and the three-dimensional laser line array camera 3 starts to collect. After the cuboid object completely passes through the gantry 1, the trigger sensor 4 sends a stop collecting signal to the host computer 5, and the host computer 5 controls the three-dimensional laser line array camera 3 to stop collecting at the same time. Two point cloud sets are obtained, which are M and N;

[0065] Step 4: Obtain the effective area point cloud: straight-through filtering and radius filtering are performed on the point cloud sets M and N respectively to obtain point cloud set A and point cloud set B. The M point cloud set before filtering is as shown in the figure Figure 4 As shown in the figure, the yellow part in M is the point cloud of the cuboid object. For the two point cloud data sets, the point cloud data of the effective object to be obtained is the point cloud data of the cuboid object above the roller plane. Straight-through filtering is performed from the X, Y and Z directions to obtain the point cloud data of the cuboid and the roller plane, and then the roller plane is fitted using RANSAC to obtain the single-view point cloud data of the cuboid. Finally, the radius filtering algorithm is used to denoise the point cloud data set.

[0066] Step 5: Use the improved VoxelGrid filtering algorithm to downsample the point cloud. The entire point cloud space is divided into several voxel grids. Assuming that the voxel grid contains n points, the theoretical coordinates of the centroid representing the entire voxel grid can be obtained. Then, the nearest neighbor search algorithm of KDtree is used for the centroid coordinates of each voxel grid, so that the nearest point to the centroid is searched as the final downsampled point. The cuboid point cloud set A obtained by filtering is as shown in the figure Figure 5 .

[0067] Step 6: Calculate the principal axis using the PCA method: Use the PCA method to calculate the eigenvectors of the two point cloud sets respectively. According to the geometric feature relationship of the cuboid object point cloud, the eigenvector corresponding to the maximum eigenvalue is approximately parallel to the Y axis. The eigenvector corresponding to the second maximum eigenvalue is approximately parallel to the X axis. The eigenvector corresponding to the third maximum eigenvalue is approximately parallel to the Z axis. The coordinate system is shown in Figure 6 .

[0068] Step 7: Calculate the coarse registration transformation matrix. Place the two point clouds in the same coordinate system, as shown in 7. A is the green point cloud and B is the red point cloud. Take A as the target point cloud and B as the source point cloud. The feature matrix of point cloud A is U A , where l A , w A , h A are the eigenvectors. The feature matrix of point cloud B is U B , where l B , w B , h B are the eigenvectors. Because the cuboid object is horizontally arranged in the Y direction under the above two-view angle, the shapes of A and B point clouds are approximately mirror-symmetric, as shown in Figure 8 . Reverse the first and second principal axes of the source point cloud B, i.e. l B =-l B , w B =-w B . A new eigenvector matrix U B can be constructed to obtain the rotation matrix R1, which can make the source point cloud B obtain the correct attitude.

[0069]

[0070] Step 8: Calculate the centroids of the two point clouds respectively,

[0071]

[0072]

[0073] Calculate the translation vector of point cloud B:

[0074] T1=X A -X B R1

[0075] Get the point cloud C after coarse registration of point cloud B through the rotation vector and the translation vector:

[0076] C=BR+T

[0077] The registration effect is shown in Figure 9 , where the red point cloud is the target point cloud and the green point cloud is the source point cloud.

[0078] Step 9: Point cloud fine matching: taking C as the source point cloud and A as the target point cloud, the point C i in the point cloud set C, find the corresponding nearest point A i in A by nearest neighbor search, and use the point-to-plane distance ICP algorithm to calculate the optimal rigid transformation matrix Z that minimizes the error.

[0079]

[0080]

[0081] where the normal vector of A i is n i (n ix , n iy , n ix ), R is a 3x3 rotation matrix, and T is a 1x3 translation vector. Each time the point-to-plane distance of the matched point pair is calculated to obtain the transformation matrix, it is essentially to solve the rotation matrix F opt that minimizes the objective function E, which can be expressed as follows.

[0082]

[0083] In the fine registration stage, the angle of rotation is very small each time, and when the rotation angle approaches 0, sinθ≈θ and cosθ≈1 can be considered. Therefore, the problem at this time can be approximately expressed as:

[0084]

[0085] The above formula is a classic linear least squares problem, and the best transformation can be solved.

[0086] In summary, the point set C is obtained after rigid transformation F, and the rigid transformation can be expressed as:

[0087] D = R2xC + T2

[0088] Step 10: Apply the optimal rigid transformation matrix to the source point cloud C to perform rigid transformation, and obtain the transformed point cloud D; the final point cloud registration effect is shown in Figure 10 , where the red point cloud is the target point cloud A, the blue point cloud is the point cloud C before fine registration, and the green point cloud is the point cloud D after fine registration.

[0089] Step 11: Use the greedy triangle projection algorithm to perform surface reconstruction on the point cloud D to obtain a three-dimensional topographic model.

[0090] In the application, by automatically collecting the point cloud of the cuboid object under the double visual angle, the large-scale cuboid three-dimensional appearance model is reconstructed, the three-dimensional reconstruction model shape parameter is accurate, and the problem that the contact type measurement cannot measure the size and appearance of the large-scale cuboid under the moving state is solved.

[0091] In the application, by the straight-through filtering algorithm, the RANSAC plane segmentation and the radius filtering method, the point cloud set of the effective area of the cuboid object is obtained, and the interference of irrelevant area data and noise is reduced. In the point cloud downsampling aspect, the improved VoxelGrid filtering algorithm is established, so that the density of the point cloud data is appropriately reduced without affecting the details and characteristics of the target object, so as to meet the requirement of improving the processing rate of the algorithm.

[0092] In the application, the principal axis calibration of the point cloud in the collection scene is carried out by the PCA method. The problem that the two point clouds are easily misoverlapped in the registration process in the scene is solved.

[0093] The above is only the preferred specific embodiment of the application, but the protection scope of the application is not limited to this, any person skilled in the art can make equivalent replacement or change according to the technical scheme and the inventive concept of the application within the technical range disclosed by the application, which should be covered in the protection scope of the application.

Claims

1. A method for determining the three-dimensional shape of a large-scale cuboid in a moving state, characterized in that, Includes the following steps: Step 1: Constructing a dual-viewline laser point cloud automatic acquisition system; Step 2: Calibrate the three-dimensional laser linear array camera (3) using the Zhang Zhengyou calibration method; Step 3: Use the roller conveyor (2) to move the large-scale cuboid object. Before the cuboid object reaches the position of the gantry (1), trigger the sensor (4) to start and send a collection signal to the host (5). After receiving the signal, the host (5) triggers two three-dimensional laser line array cameras (3) simultaneously through software. The three-dimensional laser line array cameras (3) start to collect data and obtain two point clouds, M and N respectively. Step 4: Obtain the effective region point cloud: The point cloud sets M and N are obtained by passing through filtering and radius filtering respectively. Then, RANSAC is used to fit the roller (2) plane for segmentation to obtain the single-view point cloud data of the cuboid. Finally, the point cloud dataset is denoised by the radius filtering algorithm. Step 5: Downsample the point cloud using the improved VoxelGrid filtering algorithm, and the entire point cloud space is divided into several voxel grids; Step 6: Calculate the principal axes using PCA: Calculate the eigenvectors for the two point clouds using PCA. Based on the geometric features of the cuboid point cloud, the eigenvector corresponding to the largest eigenvalue is approximately parallel to the Y-axis; the eigenvector corresponding to the second largest eigenvalue is the second principal direction, approximately parallel to the X-axis; and the eigenvector corresponding to the third largest eigenvalue is the third principal direction, approximately parallel to the Z-axis. Step 7: Calculate the coarse registration transformation matrix, placing the two point clouds in the same coordinate system. Let A be the green point cloud and B be the red point cloud, with A as the target point cloud and B as the source point cloud; the characteristic matrix of point cloud A is U. A , where l A w A h A The feature vector is U; the feature matrix of point cloud B is U. B , where l B w B h B The feature vector is defined because, under the aforementioned dual-viewpoint arrangement, the cuboid objects are horizontally aligned in the Y direction, and the point clouds A and B are mirror-symmetric, causing the first and second principal axes of the source point cloud B to be reversed. B =-l B w B =-w B ; to form a new eigenvector matrix U B We obtain the rotation matrix R1 so that the source point cloud B can obtain the correct pose; Step 8: Calculate the centroids of the two point clouds separately. Calculate the translation vector of point cloud B: T1=X A -X B R1 Point cloud C, coarsely registered with point cloud B, is obtained using rotation and translation vectors: C = BR + T Step 9: Point cloud fine matching: Use the point-to-surface ICP algorithm to match the two point clouds C and A; Step 10: Apply the optimal rigid body transformation matrix to the source point cloud C to perform a rigid transformation, obtain the transformed point cloud D, and then complete the registration. Step 11: Use the greedy triangular projection algorithm to reconstruct the surface of the point cloud D to obtain a three-dimensional shape model.

2. The method for determining the three-dimensional shape of a large-scale cuboid in a moving state according to claim 1, characterized in that, In step 1, the automatic acquisition system relies on a conveyor belt to transport a large-scale cuboid object to achieve dynamic real-time acquisition of point clouds.

3. The method for determining the three-dimensional shape of a large-scale cuboid in a moving state according to claim 1, characterized in that, In step 2, the scanning line frequency of the three-dimensional laser linear array camera (3) is set so that the horizontal scanning accuracy is equal to the vertical scanning accuracy.

4. The method for determining the three-dimensional shape of a large-scale cuboid in a moving state according to claim 1, characterized in that, In step 3, after the cuboid object has completely passed through the gantry (1), the sensor (4) is triggered to send a stop acquisition signal to the host (5), and the host (5) controls the three-dimensional laser linear array camera (3) to stop acquisition at the same time.

5. The method for determining the three-dimensional shape of a large-scale cuboid in a moving state according to claim 1, characterized in that, In step 4, for point cloud set A and point cloud set B, the point cloud data of the effective objects to be obtained are the point cloud data of the cuboid objects above the plane of roller conveyor (2). The point cloud data of the cuboid and the plane of roller conveyor 2 are obtained by passing through filtering in the X, Y and Z directions respectively.

6. The method for determining the three-dimensional shape of a large-scale cuboid in a moving state according to claim 1, characterized in that, In step 5, if the voxel grid contains n points, the theoretical coordinates of the centroid representing the entire voxel grid are obtained. Then, the nearest neighbor search algorithm of KDtree is used for the centroid coordinates of each voxel grid to search for the point closest to the centroid as the final downsampling point.

7. The method for determining the three-dimensional shape of a large-scale cuboid in a moving state according to claim 1, characterized in that, The specific steps of the point-to-surface ICP algorithm in step 9 are as follows: Taking C as the source point cloud and A as the target point cloud, perform point C in the point cloud set C. i Find the nearest point A in A using the nearest neighbor search. i The objective function of the ICP algorithm using point-to-surface distance is E, and the optimal rigid body transformation matrix Z that minimizes the error is calculated. In the formula, A i The normal vector is n i (n ix n iy n ix R is a 3x3 rotation matrix, and T is a 1x3 translation vector; Each step involves calculating the transformation matrix by matching the point-to-surface distance between point pairs, and then solving for the rotation matrix F that minimizes the objective function E. opt It can be expressed as the following formula: In the fine registration stage, the rotation angle is very small each time. When the rotation angle is close to 0, we assume sinθ≈θ and cosθ≈1. Therefore, the above problem can be approximately expressed as: The above equation is a classic linear least squares problem, and the optimal transformation is obtained by solving it; In summary, point set C is transformed into point D by rigid transformation F, and the rigid transformation can be expressed as follows: D = R² × C + T².

Images