Normal vector-based laser point cloud three-dimensional grid reconstruction method and system

Abstract

The invention discloses a laser point cloud three-dimensional grid reconstruction method and system based on a normal vector, and belongs to the field of on-satellite intelligent recognition. The method comprises the following steps: denoising and down-sampling laser point cloud data of a single-photon laser radar detector in sequence; performing grid connection on every four adjacent points from the most boundary of the processed point cloud data to generate a voxel cube, and connecting a diagonal line in each voxel cube according to normal vectors of the four points to form two triangular patches; and all the triangular patches form a three-dimensional grid of the laser point cloud and are used for target detection. According to the method, the point cloud is fully utilized to perform three-dimensional connection and establish the grid, the calculation is fast, and the three-dimensional reconstruction precision is high.

Description

Technical Field

[0001] The invention relates to a normal vector-based laser point cloud three-dimensional grid reconstruction method and system, belonging to the field of on-board intelligent recognition. Background Art

[0002] In recent years, the frequency of use of laser point cloud 3D reconstruction technology has increased, and its scope of use has become increasingly wider. At the same time, the efficiency and real-time performance of on-board intelligent processing are also key research topics. Point cloud 3D reconstruction technology can be used in a variety of fields, and this application research has broad prospects.

[0003] By summarizing the existing research, we can find that the current reconstruction technologies all pursue high precision, and the algorithms are relatively complex and cumbersome, which is not conducive to real-time and fast processing by FPGA. At the same time, the amount of point cloud data is large, while the amount of single-photon lidar data is small, which is not suitable for most of the existing complex reconstruction algorithms. The existing surface point cloud reconstruction method performs point cloud recovery on the global image, which lacks directionality, easily causes resource waste, and reduces computing efficiency. There are many algorithms for 3D mesh reconstruction. The more mature and widely used ones are the surface reconstruction algorithm and the Poisson reconstruction algorithm. Both the surface reconstruction and the Poisson reconstruction algorithms need to calculate the normal vector of the point cloud.

[0004] The principle of the surface reconstruction algorithm is to perform the process of vector fitting to the plane based on the normal vector of each point cloud, and try to keep the normal vector of each point perpendicular to the surface passing through this point. This algorithm has high complexity and is effective for reconstructing three-dimensional objects. However, for extraterrestrial bodies, the terrain information of extraterrestrial bodies is still unclear, and the algorithm is not suitable for reconstructing uncertain terrain grids.

[0005] The principle of the Poisson reconstruction algorithm is to subdivide the point cloud data set into cubic grids, calculate the Poisson equation based on the points and normal vectors in each grid, use the Laplace operator to reconstruct the surface, extract the isosurfaces in the unit grid and stitch them together, and finally generate a three-dimensional grid. This algorithm was improved and experimented in the early stage of extraterrestrial research, but the algorithm complexity is high and the running speed is slow, which cannot meet the requirements of fast processing and reconstruction. This algorithm occupies too many resources, which will result in few computing resources in the future, so it is not used. Summary of the invention

[0006] The technical problem solved by the present invention is: to overcome the deficiencies of the prior art, and to propose a laser point cloud 3D mesh reconstruction method and system based on normal vectors, which makes full use of point clouds for 3D connection and grid establishment, has fast calculation and high 3D reconstruction accuracy.

[0007] The technical solution of the present invention is:

[0008] A laser point cloud three-dimensional mesh reconstruction method based on normal vectors, comprising:

[0009] S1: De-noising and down-sampling the laser point cloud data of the single-photon laser radar detector in sequence;

[0010] S2: Calculate the normal vector of each point in the processed point cloud data;

[0011] S3: Starting from the outermost edge of the processed point cloud data, every four adjacent points are meshed to generate a voxel cube; in each voxel cube, two triangular facets are formed by connecting a diagonal line according to the normal vectors of the four points; all the triangular facets constitute a three-dimensional mesh of the laser point cloud for target detection.

[0012] Furthermore, in step S3, in each voxel cube, according to the normal vectors of the four points, a diagonal line is connected to form two triangular facets, and the method for selecting the diagonal line is:

[0013] Within each voxel cube, the four points form two diagonal lines;

[0014] Calculate the angle between the normal vectors of the two diagonal points corresponding to each diagonal line;

[0015] The two normal vector angles are compared, and the diagonal corresponding to the smaller normal vector angle is selected, and the diagonal is used as a common edge to form two triangular facets.

[0016] Furthermore, if there is missing data, or the four points cannot be gathered together, resulting in the inability to generate a voxel cube, the outermost contour method is used for three-dimensional patch reconstruction, that is, crossing the gap and taking the next nearest valid point as the vertex of the triangular patch reconstruction, that is, generating a spatial hexahedron, and reconstructing the triangular mesh based on the four valid points on the four vertical edges.

[0017] Furthermore, in step S2, the normal vector of each point in the processed point cloud data is calculated by:

[0018] The following operation is performed for each point in the point cloud data: the current point and its adjacent points are fitted into a plane, and the unit normal vector corresponding to the plane is the normal vector of the current point. The method for obtaining the fitted plane is:

[0019] Establish the equation of the fitting plane:

[0020] z=ax+by+c

[0021] Where (x, y, z) is the coordinate of the point in the plane, a, b, c are coefficients;

[0022] The current point and its neighboring points are regarded as a group of points. The goal is to find the values ​​of a, b, and c so that the sum of the squares of the distances from all points in the group to the plane is minimized. Specifically, the following objective function is minimized:

[0023]

[0024] Where n is the number of points.

[0025] Furthermore, in step S2, in order to solve a, b, c, the partial derivatives of E(a, b, c) with respect to a, b, c are calculated and set equal to 0, thereby obtaining a linear equation system. The values ​​of a, b, c are obtained by solving the linear equation system; the linear equation system is as follows:

[0026]

[0027] By solving the equations, we get a, b, and c, and the equation of the fitting plane.

[0028] Furthermore, in step S1, the method for denoising the original laser point cloud data is: voxel-by-voxel differentiation of the array pixel coordinates, noise removal based on the single-photon histogram statistical results, and retention of valid point cloud positions.

[0029] Furthermore, in step S1, the method of down-sampling the denoised point cloud data is as follows: only one valid point data is retained at each pixel position.

[0030] A 3D mesh reconstruction system using a normal vector-based laser point cloud 3D mesh reconstruction method, including a SPAD array laser detector, a 3D mesh real-time reconstruction module and an FPGA computing board;

[0031] The SPAD array laser detector is used for acquiring and aligning point cloud data;

[0032] The 3D mesh real-time reconstruction module includes a point cloud simplification and denoising module and a 3D mesh reconstruction module; the point cloud simplification and denoising module performs point cloud simplification and denoising to generate point cloud data for real-time fast mesh reconstruction; the 3D mesh reconstruction module calculates the point cloud normal vector and performs fast real-time reconstruction of the 3D mesh;

[0033] The FPGA computing board is used to set up the 3D mesh real-time reconstruction module and output the results in real time.

[0034] Furthermore, multiple three-dimensional grid real-time reconstruction modules are provided, and the input end of each three-dimensional grid real-time reconstruction module is connected to the output port of the SPAD array laser detector. At the same time, the output port of each three-dimensional grid real-time reconstruction module is bidirectionally connected to the input and output end of the FPGA computing board.

[0035] The advantages of the present invention compared with the prior art are:

[0036] (1) The present invention improves the processing speed of a large amount of point cloud data while ensuring the integrity and continuity of the mesh by simplifying the core triangle mesh construction algorithm, and can be used to implement subsequent algorithms such as target detection and extraction. When hardware resources are insufficient, such as real-time processing on a satellite or other similar situations, the present invention solves the problems that are difficult to implement with existing algorithms. The algorithm can be used to achieve a lightweight model with a wider range of applications.

[0037] (2) Compared with the existing laser point cloud three-dimensional grid reconstruction method, the algorithm complexity of the present invention is lower, and the accuracy does not decrease much. The algorithm processing time for the same set of data is less than the existing complex and high-precision calculation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0039] Figure 1 This is a flow chart of a method for reconstructing a three-dimensional mesh of a laser point cloud based on a normal vector according to an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of mesh reconstruction between four adjacent points according to an embodiment of the present invention;

[0041] Figure 3 A point cloud data map of the lunar surface model actually scanned for the present invention embodiment;

[0042] Figure 4 This is an experimental point cloud data diagram captured from the lunar surface point cloud data in an embodiment of the present invention;

[0043] Figure 5 This is a rendering of the lunar surface model reconstruction according to an embodiment of the present invention;

[0044] Figure 6 A three-dimensional mesh data graph generated from the cropped point cloud according to an embodiment of the present invention;

[0045] Figure 7 This is an analysis diagram of point cloud data actually collected by an embodiment of the present invention;

[0046] Figure 8 This is the actual histogram statistical result of the embodiment of the present invention. DETAILED DESCRIPTION

[0047] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0048] The present invention proposes a laser point cloud 3D mesh reconstruction method based on normal vectors, which collects 3D point cloud data generated by a single photon detector for ground detection and reconstructs the 3D mesh. Figure 1 As shown, including:

[0049] (1) Point cloud denoising adopts a fast point cloud denoising algorithm based on histogram statistics. The denoising method is to distinguish voxels according to the coordinates of the array pixels, remove noise points according to the single-photon histogram statistics, and retain the valid point cloud positions.

[0050] The specific method is: make histogram statistics based on multiple frames of data of the same pixel, obtain the denoised value, and generate a complete set of point cloud data, retaining only one valid point data at each pixel position. Compared with other filtering algorithms, this algorithm has a shorter processing time and meets the requirements, that is, it forms a set of usable point cloud data, which can achieve a good denoising effect for single-photon laser detectors. At the same time, if a smaller amount of point cloud data is required, traditional filtering algorithms such as radius filtering can be used to remove noise and downsample the point cloud.

[0051] In this embodiment, the actual collected data is as follows: Figure 7 As shown in the figure, the statistical results of the single photon return time data histogram at a certain pixel position are as follows Figure 8 As shown, the single photon return time of the actual target can be known, and the target position can be determined, that is, this value is extracted as the effective point of the point cloud.

[0052] (2) After the point cloud is downsampled, the point cloud normal vector is calculated based on the least square method to simplify the current complex plane equation fitting algorithm. The specific algorithm formula of the plane equation is:

[0053] A x +B y +C z -D=0

[0054] Among them, A, B, C, and D are the four parameters that need to be solved. The actual meaning of the vector (A, B, C) whose elements are not all 0 is the normal vector n of the plane equation, and D is the distance from the origin to the plane.

[0055] To simplify this problem, rewrite the equation as:

[0056] z=ax+by+c

[0057] Computing the fitted plane becomes a linear regression problem with respect to x and y.

[0058] Given a set of points (x i ,y i ,z i ), the goal is to find the values ​​of a, b, c so that the sum of the squares of the distances from all points to the plane is minimized, specifically minimizing the following objective function:

[0059]

[0060] Where n is the number of points. In order to solve a, b, c, we need to find the partial derivatives of E(a, b, c) with respect to a, b, c and set them equal to 0, thus obtaining a linear system of equations. By solving this system of equations, we can get the values ​​of a, b, c. The linear system of equations is as follows:

[0061]

[0062] By solving this set of equations, we can obtain a, b, c and get the specific plane equation.

[0063] For each point k in the point cloud data i , we can get a plane fitted according to the number of n nearest points, and the unit normal vector corresponding to this plane is point k i The point cloud normal vector.

[0064] (3) After determining the unit normal vector corresponding to each point, a grid is connected according to every four neighboring points, and a voxel cube is established based on the four neighboring points to reconstruct the triangular face.

[0065] The specific algorithm is:

[0066] 1) Since the restriction condition of the point cloud data after filtering corresponding to this algorithm is: there is only one z under the same (x, y) coordinate, that is, the point cloud will not have two upper and lower points under the same pixel coordinate, and the mesh data reconstructed from the point cloud will only be a plane;

[0067] 2) Start from the very edge, such as Figure 2 As shown, every four points can determine a cube, and in the voxel formed by these four points, the four points are connected in sequence to form two triangular facets;

[0068] 3) There are two situations when constructing two triangles from four points. If the four points are A, B, C, and D, two groups of situations will appear when two different diagonals are used as the adjacent hypotenuses of the two triangular facets. That is, if AC is connected, ABC and ACD are two triangles, or if BD is connected, ABD and BCD are two triangles. Here, the normal vector angle between the two diagonal points corresponding to each diagonal is calculated for comparison:

[0069]

[0070] By calculating from the above formula, we can get the two angles θ1 and θ2. We can judge the size of these two angles. If the angle is small, it means that the normal vectors of the two points are roughly in the same direction. The diagonal line connecting the two points is taken as the common edge of the two triangular patches.

[0071] 4) Two triangular facets will be formed in each generated voxel. If there is missing data or the voxel cannot be generated due to the inability to gather four points, the outermost contour method is used to reconstruct the three-dimensional facet, that is, crossing the gap and taking the next nearest valid point as the vertex of the triangular facet reconstruction, that is, generating a spatial hexahedron and reconstructing the triangular mesh based on the four valid points on the four vertical edges.

[0072] In this embodiment, Figure 3 as well as Figure 4 The point cloud data used in the implementation is shown, using the lunar surface point cloud data obtained by scanning the sand table. Figure 5 and Figure 6 In order to test the reconstruction results, the use of the present invention during the scanning and reconstruction of an actual extraterrestrial object is simulated.

[0073] When in use, the module input parameters are the original point cloud data, the normal vector calculation coefficient n, and the final 3D mesh reconstruction voxel size k, that is, the cube or other hexahedron voxel size taken in the final reconstruction. When in use, by monitoring: the number of point clouds, the number of point clouds after sampling, and the content of the generated triangular mesh Mesh file (including the triangle sequence number and the corresponding three vertex numbers), the appropriate coefficients are judged and used in the actual situation, and the most efficient content that meets the requirements of different projects is used for modification.

[0074] Based on the proposed laser point cloud 3D network reconstruction method, a laser point cloud 3D network reconstruction system is established. The system includes a SPAD array laser detector, a 3D grid real-time reconstruction module and an FPGA computing board.

[0075] SPAD array laser detector is used for acquisition and registration of point cloud data.

[0076] The 3D mesh real-time reconstruction module includes a point cloud simplification and denoising module and a 3D mesh reconstruction module. The point cloud simplification and denoising module is used to perform point cloud simplification and denoising to generate point cloud data that can be used for real-time and fast mesh reconstruction. The 3D mesh reconstruction module is used to calculate the point cloud normal vector and perform fast real-time reconstruction of the 3D mesh.

[0077] The FPGA computing board is used to set up the three-dimensional mesh real-time reconstruction module to output hardware real-time results.

[0078] A plurality of three-dimensional grid real-time reconstruction modules can be provided, and the input ends of the plurality of three-dimensional grid real-time reconstruction modules are connected to the output ports of the SPAD array laser detector. Meanwhile, the output ports of the plurality of three-dimensional grid real-time reconstruction modules are bidirectionally connected to the input and output ends of the FPGA computing board.

[0079] The present invention meets the requirements for real-time reconstruction of single-photon lidar detectors under different circumstances. Currently, it can read 50-100 frames of point cloud data of small array detectors at a frame rate of 40Hz and perform real-time 3D reconstruction algorithms including denoising, normal vector calculation and 3D mesh generation. It can be applied to different intelligent processing environments and ensure reconstruction accuracy.

[0080] For the existing algorithms with high computational complexity, such as greedy projection or Poisson reconstruction, taking 1024*1024 point cloud data as an example, the average single-frame reconstruction time of this algorithm is about 13.75ms, which is 3.6 times faster than the average processing time of 50ms to 70ms of conventional algorithms. In addition, the algorithm complexity is relatively low and can be used to realize real-time reconstruction processing on board or in other fields.

[0081] The above-described embodiments are only preferred specific implementations of the present invention. Common changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.

Claims

1. A laser point cloud 3D mesh reconstruction method based on normal vectors, characterized in that: include: S1: De-noising and down-sampling the laser point cloud data of the single-photon laser radar detector in sequence; S2: Calculate the normal vector of each point in the processed point cloud data; S3: Starting from the outermost edge of the processed point cloud data, every four adjacent points are meshed to generate a voxel cube; in each voxel cube, two triangular facets are formed by connecting a diagonal line according to the normal vectors of the four points; all the triangular facets constitute a three-dimensional mesh of the laser point cloud for target detection.

2. The method for reconstructing three-dimensional mesh from laser point cloud based on normal vector according to claim 1, characterized in that: In step S3, in each voxel cube, according to the normal vectors of the four points, a diagonal line is connected to form two triangular facets. The method for selecting the diagonal line is: Within each voxel cube, the four points form two diagonal lines; Calculate the angle between the normal vectors of the two diagonal points corresponding to each diagonal line; The two normal vector angles are compared, and the diagonal corresponding to the smaller normal vector angle is selected, and the diagonal is used as a common edge to form two triangular facets.

3. The method for reconstructing three-dimensional mesh from laser point cloud based on normal vector according to claim 1 or 2, characterized in that: If there is missing data, or the four points cannot be gathered together to generate a voxel cube, the outermost contour method is used for three-dimensional patch reconstruction, that is, crossing the gap and taking the next nearest valid point as the vertex of the triangular patch reconstruction, that is, generating a spatial hexahedron and reconstructing the triangular mesh based on the four valid points on the four vertical edges.

4. The method for reconstructing three-dimensional mesh from laser point cloud based on normal vector according to claim 1, characterized in that: In step S2, the normal vector of each point in the processed point cloud data is calculated by: The following operation is performed for each point in the point cloud data: the current point and its adjacent points are fitted into a plane, and the unit normal vector corresponding to the plane is the normal vector of the current point. The method for obtaining the fitted plane is: Establish the equation of the fitting plane: z=ax+by+c Where (x, y, z) is the coordinate of the point in the plane, a, b, c are coefficients; The current point and its neighboring points are regarded as a group of points. The goal is to find the values ​​of a, b, and c so that the sum of the squares of the distances from all points in the group to the plane is minimized. Specifically, the following objective function is minimized: Among them, n is the number of points in the group.

5. The method for reconstructing three-dimensional mesh from laser point cloud based on normal vector according to claim 4, characterized in that: In step S2, in order to solve a, b, c, the partial derivatives of E(a, b, c) with respect to a, b, c are calculated and set equal to 0, thereby obtaining a linear equation system. The values ​​of a, b, c are obtained by solving the linear equation system; the linear equation system is as follows: By solving the equations, we get a, b, and c, and the equation of the fitting plane.

6. The method for reconstructing three-dimensional mesh from laser point cloud based on normal vector according to claim 1, characterized in that: In step S1, the method for denoising the original laser point cloud data is: voxel-by-voxel differentiation of the array pixel coordinates, noise elimination based on the single-photon histogram statistics, and retention of valid point cloud positions.

7. The method for reconstructing three-dimensional mesh from laser point cloud based on normal vector according to claim 1, characterized in that: In step S1, the method of downsampling the denoised point cloud data is as follows: only one valid point data is retained at each pixel position.

8. A 3D mesh reconstruction system using the normal vector-based laser point cloud 3D mesh reconstruction method according to claim 1, characterized in that: It includes SPAD array laser detector, 3D grid real-time reconstruction module and FPGA computing board; The SPAD array laser detector is used for acquiring and aligning point cloud data; The 3D mesh real-time reconstruction module includes a point cloud simplification and denoising module and a 3D mesh reconstruction module; the point cloud simplification and denoising module performs point cloud simplification and denoising to generate point cloud data for real-time fast mesh reconstruction; the 3D mesh reconstruction module calculates the point cloud normal vector and performs fast real-time reconstruction of the 3D mesh; The FPGA computing board is used to set up the 3D mesh real-time reconstruction module and output the results in real time.

9. The three-dimensional mesh reconstruction system according to claim 8, characterized in that: A plurality of three-dimensional grid real-time reconstruction modules are provided, and the input end of each three-dimensional grid real-time reconstruction module is connected to the output port of the SPAD array laser detector. Meanwhile, the output port of each three-dimensional grid real-time reconstruction module is bidirectionally connected to the input and output end of the FPGA computing board.

Images