A method for quickly generating a 3D mesh from 3D point clouds

Abstract

The present invention relates to the field of three-dimensional scanning technology, and discloses a method for quickly generating a three-dimensional mesh from three-dimensional point clouds. The point cloud P after three-dimensional reconstruction is read in. This frame of point cloud P is the three-dimensional point cloud corresponding to the two-dimensional fringe pattern in the current camera coordinate system, which is mapped to the UV plane, and linear interpolation is used for interpolation in the U and V directions respectively. When using this method for quickly generating a three-dimensional mesh from three-dimensional point clouds, by setting the characteristics of the three-dimensional point cloud generated based on the fringe pattern, a method for generating a three-dimensional mesh based on the UV plane is proposed. The triangulation of the point cloud of the current frame is quickly completed, and both the interpolation and triangulation steps can be performed in parallel, with extremely high speed, which can meet the requirements of three-dimensional scanning devices with relatively high speed requirements. The memory requirement is much lower than that of the voxel method, and it can meet the scanning environment with low memory requirements. It can generate faster and occupy less memory than the voxel method.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional scanning, and particularly to a method for quickly generating a three-dimensional mesh from three-dimensional point clouds. Background Art

[0002] In the field of three-dimensional scanning devices, the working process of a scanning device is as follows: 2D image information is acquired to reconstruct 3D point clouds, and the 3D point clouds are stitched together according to their features to form a complete point cloud under a unified coordinate system. Finally, a 3D mesh is generated from the 3D point clouds to complete the digitization of real objects. In three-dimensional real-time scanning applications, real-time rendering where what you see is what you get is an important link. In real-time scanning, there are two ways of user interaction: point cloud rendering and mesh rendering. Point cloud rendering is faster than mesh rendering and can meet the requirements of real-time rendering. However, in many scenarios, 3D meshes are used for real-time rendering and final output in the field of three-dimensional scanning devices. The main reasons are as follows: (1) 3D meshes are the basis for 3D printing; (2) Through rendering, 3D meshes can effectively reflect the features of real objects; (3) 3D meshes have geometric topology characteristics and can effectively express connectivity; (4) 3D meshes can be effectively texture mapped to truly reflect the object surface. Real-time rendering of 3D meshes can improve the scanning quality and enhance the user experience; real-time rendering of 3D meshes during three-dimensional scanning, presenting the characteristics of the scanned data immediately, can effectively improve scanning navigation; its geometric topology characteristics can help algorithms remove noise points in a timely and effective manner, avoiding the accumulation of noise data. Therefore, realizing mesh real-time during scanning can not only improve the scanning accuracy but also greatly enhance the user experience, and it is an essential function for many three-dimensional scanning device products.

[0003] When the common methods for generating a three-dimensional mesh from three-dimensional point clouds are used, the voxel method is mainly utilized, and finding implicit function values to realize the generation of a three-dimensional mesh from three-dimensional point clouds is widely used. This method can control the mesh accuracy by adjusting the voxel accuracy, but it has a large amount of calculation and a slow speed, and cannot meet the working requirements for generating a three-dimensional mesh from three-dimensional point clouds. Therefore, a method for quickly generating a three-dimensional mesh from three-dimensional point clouds is proposed. Summary of the Invention

[0004] (I) Technical Problems to be Solved

[0005] Aiming at the deficiencies of the prior art, the present invention provides a method for quickly generating a three-dimensional mesh from three-dimensional point clouds, solving the technical problems raised in the above background art.

[0006] (II) Technical Solutions

[0007] To achieve the above object, the present invention provides the following technical solutions: A method for quickly generating a three-dimensional mesh from three-dimensional point clouds, including the following steps:

[0008] Read in the point cloud P after three-dimensional reconstruction. This point cloud P is the three-dimensional point cloud corresponding to the two-dimensional fringe pattern in the current camera coordinate system;

[0009] Map it to the UV plane. When performing linear interpolation in the U and V directions respectively, according to the characteristics of the fringe pattern, interpolation is performed in the U direction, or the V direction, or both the U and V directions;

[0010] Traverse UV, take vertices according to the quadrilateral rule and triangulate the mesh. Traverse the r-th row and the (r + 1)-th data of the UV coordinates, and take four points p(u, v), p(u, v + 1), p(u + 1, v + 1), and p(u + 1, v) respectively to generate a triangle;

[0011] Combine all the generated triangles to form mesh0, and transform it to the global coordinate system through the transformation matrix;

[0012] According to the provided index set set(indexes), find the bounding box corresponding to each index, take the vertices of the overall mesh corresponding to each bounding box and the vertices of the current mesh0 for neighboring vertex merging, and finally generate the overall mesh.

[0013] Preferably, the attributes of the point cloud P include the three-dimensional spatial positions x, y, z in the camera coordinate system, the three-dimensional spatial normal vectors xn, yn, zn in the camera coordinate system, the RGB color value of each point in the point cloud P, the weight q of each point in the point cloud P. The greater the weight, the higher the credibility, and the smaller the weight, the lower the credibility. The coordinates u, v of each point in the point cloud P in the UV plane, the transformation matrix M of the point cloud P from the camera coordinate system to the global coordinate system, and the index set set(indexes) of the neighboring point cloud of the point cloud P.

[0014] Preferably, if the fringe pattern is uniform in the UV direction and there is no sampling in the point cloud generated after three-dimensional reconstruction of the point cloud, then no interpolation is required or linear interpolation is performed in the U and V directions respectively according to whether the resolution meets the requirements. If the fringe pattern is not uniform in the UV direction, or there is sampling in the point cloud after three-dimensional reconstruction of the point cloud, then linear interpolation in the U or V direction is required to ensure the same point cloud density in the UV direction.

[0015] Preferably, the linear interpolation in the V direction is applicable to a Range Image with a resolution of 400 pixels by 600 pixels or its equivalent. Since the data is severely missing in the cols direction due to sampling in the three-dimensional point cloud reconstruction, for each row of data r, where r represents the r-th row of data, find v0 and v1, and both the three-dimensional data corresponding to v0 and v1 exist, and all the three-dimensional data corresponding to the interval between v0 and v1 in the r-th row do not exist. Then linearly interpolate (v1 - v0 - 1) data between v0 and v1.

[0016] Preferably, considering linear interpolation in the V direction, noise points will be generated at places where the change is drastic in the Z direction. Two thresholds, f3DThreshold and FVThreshold, need to be added to the algorithm for judgment. Only when dist(p(r, v0), p(r, v1)) < f3DThreshold and v1 - v0 < FVThread can interpolation be performed, where dist represents the distance between the three-dimensional points corresponding to p(r, v0) and p(r, v1), such as the Euclidean distance.

[0017] Preferably, in actual calculation, f3DThreshold is generally related to the distance between fringes, and FVThreshold is the pixel value on the UV plane, which is used to prevent error point clouds in holes and needs to be verified and set according to the specific nature of the device.

[0018] Preferably, if the three-dimensional point clouds corresponding to the four points all exist, the quadrilateral is divided into two triangles. If three corresponding three-dimensional point clouds exist, a triangle is constructed. In other cases, no triangle is generated.

[0019] (III) Beneficial effects

[0020] Compared with the prior art, the present invention provides a method for generating a three-dimensional mesh from a fast three-dimensional point cloud, having the following beneficial effects:

[0021] This method for generating a three-dimensional mesh from a fast three-dimensional point cloud, by setting the characteristics of the three-dimensional point cloud generated based on the fringe pattern, proposes a method for generating a three-dimensional mesh based on the UV plane, quickly completes the triangulation of the point cloud of the current frame, and both the interpolation and triangulation steps can be operated in parallel, with extremely high speed, which can meet the requirements of three-dimensional scanning devices with high speed requirements. The memory requirement is much lower than the voxel method, and it can meet the scanning environment with low memory requirements, and can generate faster and occupy less memory than the voxel method. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic flow diagram of the present invention;

[0023] Figure 2 It is the UV coordinate map before interpolation in the present invention;

[0024] Figure 3 It is the UV coordinate map after interpolation in the present invention;

[0025] Figure 4 It is the schematic diagram of triangulation in the present invention;

[0026] Figure 5 It is the display diagram of the triangulation result in the present invention. Detailed implementation manners

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts belong to the protection scope of the present invention.

[0028] The present invention provides a technical solution, a method for quickly generating a three-dimensional mesh from a three-dimensional point cloud, including the following steps:

[0029] Please refer to Figure 1

[0030] S1. Read in the point cloud P after three-dimensional reconstruction. The point cloud P is the three-dimensional point cloud corresponding to the two-dimensional fringe pattern in the current camera coordinate system. The attributes of the point cloud P include the three-dimensional spatial position x, y, z in the camera coordinate system, the three-dimensional spatial normal vectors xn, yn, zn in the camera coordinate system, the RGB color value of each point in the point cloud P, the weight q of each point in the point cloud P. The greater the weight, the higher the credibility, and the smaller the weight, the lower the credibility. The coordinates u, v of each point in the point cloud P in the UV plane, the transformation matrix M of the point cloud P from the camera coordinate system to the global coordinate system, and the index set set (indexes) of the neighborhood point cloud of the point cloud P.

[0031] S2. Map to the UV plane, and use linear interpolation when interpolating in the UV direction. According to the characteristics of the fringe pattern, interpolation is used in the U direction, or in the V direction, or in both UV directions. If the fringe pattern is uniform in the UV direction, and the point cloud generated after the 3D reconstruction of the point cloud has no sampling, no interpolation is required, or linear interpolation is performed in the UV direction according to whether the resolution meets the requirements. If the fringe pattern is not uniform in the UV direction, or the point cloud is sampled after the 3D reconstruction of the point cloud, linear interpolation is required in the U or V direction to ensure that the density of the point cloud in the UV direction is the same. The linear interpolation in the V direction is suitable for RangeImage with a resolution of 400 pixels by 600 pixels or equivalent. Since the data is missing in the cols direction due to the sampling of the 3D point cloud reconstruction, The loss is serious, so for each row of data r, r represents the rth row of data, find v0 and v1, and the 3D data corresponding to v0 and v1 exist, and all the 3D data corresponding to r rows v0 and v1 do not exist, then linearly interpolate (v1-v0-1) data between v0 and v1. Considering the linear interpolation in the V direction, noise will be generated in places where the Z direction changes drastically. It is necessary to add two thresholds f3DThreshold and FVThreshold to the algorithm. In the actual calculation of the threshold, f3DThreshold is generally related to the distance between stripes. FVThreshold is the pixel value on the UV plane, which is used to prevent error point clouds in holes. It needs to be verified and set according to the specific properties of the device, where dist(p(r, v0), p(r, v1)) is satisfied. <f3DThreshold且v1-v0<FVThread条件才能进行插值,其中dist表示p(r,v0)和p(r,v1)对应的三维点的距离,如欧式距离,请参阅 Figure 2 and Figure 3 ;

[0032] S3. Traverse UV and take vertices according to the quadrilateral rule and triangulate the mesh. Traverse the rth row and r+1th data of the UV coordinates, and take four points p(u, v), p(u, v+1), p(u+1, v+1), and p(u+1, v) to generate triangles. If the three-dimensional point clouds corresponding to the four points all exist, the quadrilateral is divided into two triangles. If there are three corresponding three-dimensional point clouds, a triangle is constructed. In other cases, no triangle is generated. Please refer to Figure 4 ;

[0033] S4, all the generated triangles are formed into mesh0, and transformed into the global coordinate system through the transformation matrix;

[0034] S5. According to the provided index set set (indexes), find the enclosing box corresponding to each index, merge the vertices of the overall mesh corresponding to each enclosing box with the vertices of the current mesh0 for neighboring vertices, and finally generate the overall mesh. Please refer to Figure 5 .

[0035] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device.

[0036] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for generating a three-dimensional mesh from a fast three-dimensional point cloud, characterized in that, The steps include: S1, read in the point cloud P after 3D reconstruction, which is the 3D point cloud corresponding to the 2D fringe pattern in the current camera coordinate system; S2, mapped to the UV plane, linear interpolation is used when interpolating in the UV direction. According to the characteristics of the fringe pattern, interpolation is used in the U direction, or in the V direction, or in both the UV directions; S3, traverse UV and take vertices according to the quadrilateral rule and triangulate the mesh, traverse the rth and r+1th rows of UV coordinates, and take four points p(u, v), p(u, v+1), p(u+1, v+1), and p(u+1, v) to generate a triangle; S4, all the generated triangles are formed into mesh0, and transformed into the global coordinate system through the transformation matrix; S5. According to the provided index set (indexes), find the outer box corresponding to each index, take the vertices of the overall mesh corresponding to each outer box and the vertices of the current mesh0 to merge the adjacent vertices, and finally generate the overall mesh; If the fringe pattern is uniform in the UV direction and the point cloud generated after the three-dimensional reconstruction of the point cloud is not sampled, there is no need to perform interpolation or linear interpolation is performed in the UV direction according to whether the resolution meets the requirements. If the fringe pattern is not uniform in the UV direction, or the point cloud is sampled after the three-dimensional reconstruction of the point cloud, linear interpolation is required in the U or V direction to ensure that the density of the point cloud in the UV direction is the same.

2. A method for generating a three-dimensional mesh from a fast three-dimensional point cloud according to claim 1, characterized in that: The attributes of the point cloud P include the three-dimensional spatial position x, y, z in the camera coordinate system, the three-dimensional spatial normal xn, yn, zn in the camera coordinate system, the RGB color value of each point in the point cloud P, the weight q of each point in the point cloud P, the larger the weight, the higher the credibility, and the smaller the weight, the lower the credibility, the coordinates u, v of each point in the point cloud P in the UV plane, the transformation matrix M of the point cloud P from the camera coordinate system to the global coordinate system, and the index set (indexes) of the neighborhood point cloud of the point cloud P.

3. A method for generating a three-dimensional mesh from a fast three-dimensional point cloud according to claim 1, characterized in that: The linear interpolation in the V direction is applicable to the Range Image with a resolution of 400 pixels by 600 pixels. Since the data in the cols direction is seriously missing due to the sampling of the 3D point cloud reconstruction, for each row of data r, r represents the rth row of data, find v0 and v1, and the 3D coordinates corresponding to v0 and v1 exist, and the corresponding 3D data between r rows of v0 and v1 do not exist, then linearly interpolate (v1-v0-1) data between v0 and v1.

4. A method for generating a three-dimensional mesh from a fast three-dimensional point cloud according to claim 3, characterized in that: Taking into account the linear interpolation in the V direction, noise will be generated in places where the Z direction changes dramatically. It is necessary to add two thresholds f3DThreshold and FVThreshold to the algorithm, where dist(p(r, v0), p(r, v1)) is satisfied. <f3DThreshold且v1-v0<FVThread条件才能进行插值,其中dist表示p(r,v0)和p(r,v1)对应的三维点的距离。 5. A method for generating a three-dimensional mesh from a fast three-dimensional point cloud according to claim 4, characterized in that: In the actual calculation of the threshold, f3DThreshold is related to the distance between stripes, and FVThreshold is the pixel value on the UV plane, which is used to prevent error point clouds in holes.

6. A method for generating a three-dimensional mesh from a fast three-dimensional point cloud according to claim 1, characterized in that: If the three-dimensional point clouds corresponding to the four points all exist, the quadrilateral is divided into two triangles. If there are three corresponding three-dimensional point clouds, a triangle is constructed. In other cases, no triangles are generated.

Images