A point cloud denoising method, device and application based on Mesh

Abstract

The present invention relates to a Mesh-based point cloud denoising method, comprising: constructing a Mesh structure composed of a plurality of triangular meshes based on a point cloud set to be denoised; respectively calculating and correcting the normal vectors of all triangular meshes so that the directions of all normal vectors are the same; for each triangular mesh, calculating the average azimuth change of the corrected normal vector of the triangular mesh and the corrected normal vectors of its K nearest triangular meshes, and constructing a first texture feature vector; normalizing the size of each triangular mesh, performing an anti-folding translation, and constructing a second texture feature vector; fusing two texture feature vectors of each triangular mesh to obtain a unified feature vector; obtaining a fitting function based on the eigenvalues ​​of all the unified feature vectors; using the eigenvalue corresponding to the first slope mutation in the fitting function as a denoising threshold; selecting triangular meshes whose eigenvalues ​​are greater than the denoising threshold, and forming a denoised point cloud set with the vertices of all the selected triangular meshes.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud denoising, and in particular to a Mesh-based point cloud denoising method, device and application. Background Art

[0002] With the popularity of three-dimensional sensing devices such as lidar, binocular cameras, and depth cameras, the process of obtaining point cloud data from real scenes has become more convenient. Due to the rich information of point clouds, point clouds are widely used in fields such as autonomous driving, robotics, and mapping. However, the point clouds actually collected by three-dimensional sensing devices usually contain noise. Noise reduces the accuracy of point cloud data and sometimes interferes with downstream tasks. Therefore, point cloud denoising is a basic task. This task plays an important role in many applications. For example, in the upgrading and transformation of three-dimensional sensing equipment, point cloud is a common format for three-dimensional data, and point cloud denoising can be used as the last step before the data output of the acquisition device; in multimodal data fusion, the features of point cloud and other types of data (such as image data) complement each other to improve the accuracy of the system; in reverse engineering of 3D modeling and scene reconstruction, denoised point cloud data can generate more accurate 3D models; in model development, in order to solve various challenges in point cloud processing, researchers are constantly developing new algorithms and models, and point cloud denoising improves the quality of the model and the accuracy of analysis. At the same time, point cloud denoising technology is being applied to more and more fields, including topographic mapping, environmental monitoring, medical image analysis, structural monitoring, etc.

[0003] At present, from the perspective of methods, point cloud denoising is divided into deep learning-based methods and model optimization-based methods. Deep learning-based methods extract underlying features through neural networks. Supervised learning usually requires point cloud data without noise as the true value to participate in training; however, in actual scenarios, it is impossible to obtain point cloud data without noise as the true value. To solve this problem, unsupervised learning methods have been widely and deeply studied in point cloud denoising tasks. When the number of samples is sufficient, unsupervised learning methods can directly learn underlying features from noisy point clouds to complete denoising; however, when the noise level of point cloud data is too high, abnormal shrinkage may form. From the perspective of results, point cloud denoising methods can be divided into removing noise points and obtaining smooth surfaces. The smoothing method estimates the surface morphology of the object and moves the position of the point to obtain a smooth surface that approximates the true shape of the object; however, this method is prone to over-smoothing and losing key gradient information; and the displacement of the point is a resampling of the original data. The resampling process may cause new precision loss, which may affect downstream tasks, especially in the task of multimodal data fusion. The change in the coordinates of the point causes the external parameter matrix calibrated by the camera and lidar to no longer be applicable.

[0004] Therefore, in order to solve the aforementioned problems of difficulty in obtaining samples and loss of key gradient information caused by smoothing, point cloud denoising methods based on removing noise points are now mostly used; the noise removal method is a method with relatively small loss of accuracy; statistical filtering, radius filtering and other filtering methods are commonly used to remove noise points, but the performance of such methods usually depends on the selection of point cloud denoising thresholds, and fixed parameters are difficult to adapt to point clouds of different forms, and the constructed features cannot describe the point cloud features from a geometric level. The operator selects the threshold based on experience, resulting in denoising performance related to the operator's level. Different denoising thresholds lead to different point cloud denoising results. There are cases where effective points are removed and noise points are retained, and accurate denoising cannot be achieved. Summary of the invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem of inaccurate denoising caused by the point cloud denoising method based on removing noise points in the prior art because the denoising threshold needs to be manually selected.

[0006] In order to solve the above technical problems, the present invention provides a point cloud denoising method based on Mesh, comprising:

[0007] Obtain a set of point clouds to be denoised, and construct a Mesh structure consisting of multiple triangular meshes based on the topological relationship of all point clouds in the set of point clouds to be denoised;

[0008] Calculate the normal vectors of all triangular meshes respectively, and correct the normal vectors of all triangular meshes so that the normal vectors of all triangular meshes have the same direction;

[0009] For each triangular mesh, calculate the average azimuth change of the corrected normal vector of the triangular mesh and the corrected normal vectors of its K nearest triangular meshes, and construct the first texture feature vector of each triangular mesh;

[0010] Normalizing the size of each triangular mesh, deconvoluting and translating the normalized triangular mesh, and constructing a second texture feature vector for each triangular mesh;

[0011] Merging the first texture feature vector of each triangular mesh with the second texture feature vector to obtain a unified feature vector of each triangular mesh;

[0012] Based on the eigenvalues ​​of the unified eigenvectors of all triangular meshes, high-order fitting is performed to obtain a fitting function; the eigenvalue corresponding to the first slope mutation in the fitting function is used as the denoising threshold;

[0013] Select the triangular meshes whose eigenvalues ​​are greater than the denoising threshold, and use the vertices of all the selected triangular meshes to form a denoised point cloud set.

[0014] Preferably, an improved greedy projection triangulation method is used to construct a Mesh structure composed of multiple triangular meshes based on the topological relationship of all point clouds in the point cloud set to be denoised, including:

[0015] Based on the topological relationship between all point clouds in the point cloud set to be denoised, the connection relationship between different point clouds is obtained;

[0016] Project all point clouds in the point cloud set to be denoised onto a plane, triangulate them according to the connection relationship between different point clouds, generate multiple triangular meshes, and form a Mesh structure;

[0017] If there is a point cloud that is not connected to the Mesh structure, search for two neighboring point clouds of the point cloud among the vertices of all triangular meshes in the Mesh structure; form a new triangular mesh with the point cloud and its two neighboring point clouds and add it to the Mesh structure until all point clouds are added to the Mesh structure to complete the construction of the Mesh structure.

[0018] Preferably, the normal vector of the triangle mesh is expressed as:

[0019]

[0020] Among them, f n ' represents the normal vector of the nth triangle mesh; A, B, C are the three vertices that make up the triangle mesh, and the coordinate of A is (x 1 ,y 1 ,z 1 ), the coordinates of B are (x 2 ,y 2 ,z 2 ), the coordinates of C are (x 3 ,y 3 ,z 3 ).

[0021] Preferably, the first texture feature vector of the triangular mesh is expressed as:

[0022]

[0023] in, represents the first texture feature vector, K represents the total number of K nearest neighbor triangle meshes; f i represents the corrected normal vector of the i-th triangle mesh, ||f i || represents the corrected normal vector f of the i-th triangle mesh i The modulus length; f ik represents the corrected normal vector of the kth neighboring triangle mesh of the i-th triangle mesh, ||f ik || represents the vector f ik The mold length.

[0024] Preferably, the second texture feature vector of the triangular mesh is expressed as:

[0025]

[0026] in, is the second texture feature vector, Represents the normalized mesh size of the i-th triangle mesh, expressed as p is represented by a, b, and c represent the lengths of the three sides of the triangular mesh respectively; D represents the anti-fold translation operation.

[0027] Preferably, the eigenvalues ​​of the unified eigenvectors of all triangular meshes are used to perform high-order fitting to obtain a fitting function, which is expressed as:

[0028] y=a k x k +a k-1 x k-1 +…+a 1 x+a 0 ;

[0029] Among them, y represents the fitted eigenvalue, k represents the highest degree of the polynomial, and a k represents the coefficient of the k-th term after fitting, x k represents the kth power of the eigenvalue.

[0030] Preferably, based on the relationship between the minimum circumscribed rectangular frame of the Mesh structure and the end point of the normal vector of each triangular mesh, the normal vectors of all triangular meshes are corrected so that the normal vectors of all triangular meshes have the same direction, including:

[0031] Compare the minimum circumscribed rectangle of the Mesh structure with the end point of the normal vector of each triangular mesh until the normal vectors of all triangular meshes in the Mesh structure are determined and the directions of the normal vectors of all triangular meshes are corrected, including:

[0032] If the end point of the normal vector is outside the minimum circumscribed rectangle, the normal vector is retained;

[0033] If the end point of the normal vector is within the minimum circumscribed rectangle, the minimum distance from the end point of the normal vector to the minimum circumscribed rectangle is calculated as the first distance; the minimum distance from the end point of the normal vector in the opposite direction to the minimum circumscribed rectangle is calculated as the second distance; the first distance is compared with the second distance, and the normal vector corresponding to the smaller distance between the first distance and the second distance is retained.

[0034] Preferably, based on the direction consistency relationship between the normal vector of each triangular mesh and the unit vector in the depth direction, the normal vectors of all triangular meshes are corrected so that the normal vectors of all triangular meshes have the same direction, including:

[0035] Based on the radar coordinate system, with the x-axis as the depth direction, establish the depth direction vector of the normal vector of each triangular mesh;

[0036] Get the depth direction unit vector, check the direction consistency with the depth direction vector of the normal vector of each triangular mesh, retain the normal vector represented by the depth direction vector that is consistent with the depth direction unit vector, and complete the correction of the direction of the normal vectors of all triangular meshes.

[0037] The embodiment of the present invention further provides a Mesh-based point cloud denoising device, comprising:

[0038] Mesh structure building module, used to obtain the point cloud set to be denoised, and build a Mesh structure composed of multiple triangular meshes based on the topological relationship of all point clouds in the point cloud set to be denoised;

[0039] A normal vector correction module is used to calculate the normal vectors of all triangular meshes respectively and correct the normal vectors of all triangular meshes so that the normal vectors of all triangular meshes have the same direction;

[0040] The feature vector construction module is used to calculate, for each triangular mesh, the average azimuth change of the corrected normal vector of the triangular mesh and the corrected normal vectors of its K nearest triangular meshes, and construct the first texture feature vector of each triangular mesh; normalize the size of each triangular mesh, deconvolute and translate the normalized triangular mesh, and construct the second texture feature vector of each triangular mesh; merge the first texture feature vector of each triangular mesh with the second texture feature vector, and obtain a unified feature vector of each triangular mesh;

[0041] The fitting denoising module is used to perform high-order fitting based on the eigenvalues ​​of the unified eigenvectors of all triangular meshes to obtain the fitting function; the eigenvalue corresponding to the first slope mutation in the fitting function is used as the denoising threshold; the triangular meshes with eigenvalues ​​greater than the denoising threshold are selected, and the vertices of all the selected triangular meshes are used to form a denoised point cloud set.

[0042] An embodiment of the present invention further provides an application of the Mesh-based point cloud denoising method as described in any one of the above items in the field of three-dimensional modeling.

[0043] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0044] The mesh-based point cloud denoising method described in the present invention converts a point cloud set to be denoised into a mesh structure. Noise is a point that causes a sudden change in the local curvature of an object surface. In the absence of noise, the azimuth angles of normal vectors in the neighborhood are similar; the azimuth angles of the normal vector containing noise are significantly different from the direction angles of the normal vectors of its neighboring normal vectors without noise; the present invention constructs a first texture feature vector based on the average azimuth change of the normal vector of a triangular mesh and the normal vectors of its K nearest triangular meshes, accurately characterizing the geometric details in the mesh structure; the value of the normalized triangular mesh size is reversed and translated to construct a second texture feature vector, accurately characterizing the degree of discreteness of the point cloud in the point cloud set to be denoised; the two texture feature vectors are fused to obtain a unified feature vector; based on the eigenvalues ​​of all unified feature vectors, function fitting is performed to obtain a fitting function, and the eigenvalue corresponding to the sudden change in the slope of the fitting function is used as the denoising threshold, the point cloud set to be denoised is denoised, and the three vertices of all triangular meshes whose eigenvalues ​​are greater than the denoising threshold are retained to obtain a denoising result. The denoising method of the present invention is based on the mutation slope as the denoising threshold, which realizes the automatic selection of the denoising threshold, can simply and robustly realize the identification of noise points, and can retain the valid point cloud as much as possible during denoising, thereby reducing the impact on the correct point cloud and improving the point cloud denoising accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

[0046] Figure 1 It is a flowchart of the steps of the Mesh-based point cloud denoising method provided by the present invention;

[0047] Figure 2 It is a schematic diagram of the distance-based normal vector direction correction provided by the present invention;

[0048] Figure 3 It is a schematic diagram of the direction correction of the normal vector based on the direction provided by the present invention;

[0049] Figure 4 It is a schematic diagram of the surface normal vector azimuth angles of objects of different shapes provided by the present invention;

[0050] Figure 5 It is a schematic diagram of different types of normal vectors provided by the present invention;

[0051] Figure 6 (a) is the effect diagram of the aircraft point cloud containing noise. Figure 6 (b) is the denoised aircraft point cloud effect;

[0052] Figure 7 (a) is the point cloud effect of the vase containing noise. Figure 7 (b) is the denoised vase point cloud effect;

[0053] Figure 8 (a) is the first human point cloud denoising effect diagram. Figure 8 (b) is the fitting function curve of the first humanoid point cloud. Figure 8 (c) is a scatter plot of the feature values ​​of the first human point cloud;

[0054] Fig. 9 (a) is the denoising effect of the second human point cloud. Fig. 9 (b) is the fitting function curve of the second humanoid point cloud. Fig. 9 (c) is a scatter plot of the feature values ​​of the second humanoid point cloud;

[0055] Fig.10 (a) is the denoising effect of the third human point cloud. Fig.10 (b) is the fitting function curve of the third humanoid point cloud. Fig.10 (c) is a scatter plot of the feature values ​​of the third human point cloud. DETAILED DESCRIPTION

[0056] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0057] Reference Figure 1 As shown, the step flow chart of the Mesh-based point cloud denoising method of the present invention specifically includes:

[0058] S101: Obtain a point cloud set to be denoised, and construct a Mesh structure consisting of multiple triangular meshes based on the topological relationship of all point clouds in the point cloud set to be denoised;

[0059] S102: Calculate the normal vectors of all triangular meshes respectively, and correct the normal vectors of all triangular meshes so that the normal vectors of all triangular meshes have the same direction;

[0060] S103: For each triangular mesh, calculate the average azimuth change of the corrected normal vector of the triangular mesh and the corrected normal vectors of its K nearest triangular meshes, and construct a first texture feature vector for each triangular mesh;

[0061] S104: normalizing the size of each triangular mesh, deconvoluting and translating the normalized triangular mesh, and constructing a second texture feature vector for each triangular mesh;

[0062] S105: fusing the first texture feature vector and the second texture feature vector of each triangular mesh to obtain a unified feature vector of each triangular mesh;

[0063] S106: performing high-order fitting based on the eigenvalues ​​of the unified eigenvectors of all triangular meshes to obtain a fitting function; taking the eigenvalue corresponding to the first slope mutation in the fitting function as a denoising threshold;

[0064] S107: Selecting triangular meshes whose eigenvalues ​​are greater than a denoising threshold, and composing vertices of all selected triangular meshes into a denoised point cloud set.

[0065] Specifically, in an embodiment of the present invention, an improved greedy projection triangulation method is used to construct a Mesh structure composed of multiple triangular meshes based on the topological relationship of all point clouds in the point cloud set to be denoised, including: obtaining the connection relationship between different point clouds based on the topological relationship between all point clouds in the point cloud set to be denoised; projecting all point clouds in the point cloud set to be denoised onto a plane, triangulating them according to the connection relationship between different point clouds, generating multiple triangular meshes to form a Mesh structure; if there is a point cloud that is not connected to the Mesh structure, searching for two neighboring point clouds of the point cloud among the vertices of all triangular meshes in the Mesh structure; forming a new triangular mesh of the point cloud and its two neighboring point clouds, and adding the mesh structure until all point clouds are added to the Mesh structure to complete the construction of the Mesh structure.

[0066] Specifically, in the embodiment of the present invention, the normal vector of the triangular mesh is expressed as:

[0067]

[0068] Among them, f n ' represents the normal vector of the nth triangle mesh; A, B, C are the three vertices that make up the triangle mesh, and the coordinate of A is (x 1 ,y 1 ,z 1 ), the coordinates of B are (x 2 ,y 2 ,z 2 ), the coordinates of C are (x 3 ,y 3 ,z 3 ).

[0069] From a local perspective, noise is a point where the local curvature of the object surface mutates. In the absence of noise, the normal vectors in the neighborhood point to the same very small area, that is, the normal vector azimuth points to a similar direction. The normal vector azimuth of the noisy Mesh is significantly different from the normal vector direction angle of its neighboring non-noisy Mesh. Therefore, the change in the normal vector azimuth can reflect the geometric details in a certain area. Therefore, in an embodiment of the present invention, the first texture feature vector of the triangular mesh is constructed based on the change in the normal vector azimuth, expressed as:

[0070]

[0071] in, represents the first texture feature vector, K represents the total number of K nearest neighbor triangle meshes; f i represents the corrected normal vector of the i-th triangle mesh, ||f i || represents the corrected normal vector f of the i-th triangle mesh i The modulus length; f ik represents the corrected normal vector of the kth neighboring triangle mesh of the i-th triangle mesh, ||f ik || represents vector f ik The mold length.

[0072] From a global perspective, noise is points that are discretely distributed around an object. Since the triangular mesh is associated with the relative positions of the three vertices, the normalized triangular mesh size can reflect the degree of discreteness of the point cloud. Then, after deconvolution and translation of the normalized triangular mesh size value, the second texture feature vector of the triangular mesh is constructed, which is expressed as:

[0073]

[0074] in, is the second texture feature vector, Represents the normalized mesh size of the i-th triangle mesh, expressed as p is represented by a, b, and c represent the lengths of the three sides of the triangular mesh respectively; D represents the anti-fold translation operation.

[0075] Specifically, in the embodiment of the present invention, using The area of ​​the triangular mesh is calculated; all Constitute an array, min is recorded as the minimum value in this array, and max is recorded as the maximum value in this array. The value of does not necessarily belong to (0,1). Assuming it belongs to (min,max), we need to normalize this value to make it belong to (0,1). The method is for After the fold, the value is (-1,0). Shift the value back to (0,1); this process makes the change trends of the two types of texture features consistent, making it easier to construct a unified feature vector.

[0076] Based on the first texture feature vector and the second texture feature vector, a unified feature vector is constructed to fuse the two feature vectors. Based on the unified feature vector, a global distribution approximation function of the triangular mesh texture feature is established. By using the simple and robust method of derivation, the denoising threshold is automatically selected to complete the point cloud denoising work. Specifically, the fitting function is expressed as:

[0077] y=a k x k +a k-1 x k-1 +…+a 1 x+a 0 ;

[0078] Among them, y represents the fitted eigenvalue, k represents the highest degree of the polynomial, and a k represents the coefficient of the k-th term after fitting, x k represents the kth power of the eigenvalue.

[0079] In the embodiment of the present invention, the basis for correcting the normal vectors of all triangular meshes so that the normal vectors of all triangular meshes have the same direction includes:

[0080] ① Based on the relationship between the minimum bounding rectangle of the Mesh structure and the end point of each triangle mesh normal vector, refer to Figure 2 As shown, the specific correction methods include:

[0081] Compare the minimum circumscribed rectangle of the Mesh structure with the end point of the normal vector of each triangular mesh until the normal vectors of all triangular meshes in the Mesh structure are determined and the directions of the normal vectors of all triangular meshes are corrected, including:

[0082] If the end point of the normal vector is outside the minimum circumscribed rectangle, the normal vector is retained;

[0083] If the end point of the normal vector is within the minimum circumscribed rectangle, the minimum distance from the end point of the normal vector to the minimum circumscribed rectangle is calculated as the first distance; the minimum distance from the end point of the normal vector in the opposite direction to the minimum circumscribed rectangle is calculated as the second distance; the first distance is compared with the second distance, and the normal vector corresponding to the smaller distance between the first distance and the second distance is retained.

[0084] Specifically, the modulus of each normal vector is equal to 1 / 2 of the minimum side length of the minimum circumscribed rectangle; first, define the starting point of the normal vector at the center of each triangular mesh. If the end point of the normal vector is outside the minimum circumscribed rectangle, the direction is correct; if the end point is inside the minimum circumscribed rectangle, it is pending. Secondly, perform the next round of judgment for all pending arrows. Calculate the minimum distance from the end points of two normal vectors in opposite directions to the minimum rectangular box, recorded as dis1 and dis2. If dis1 is less than dis2, the direction of the normal vector represented by dis1 is correct, otherwise the direction is wrong. At this time, correct f i =-f i .

[0085] ② Based on the directional consistency relationship between the normal vector of each triangle mesh and the unit vector in the depth direction, refer to Figure 3 As shown, the specific correction methods include:

[0086] Based on the radar coordinate system, with the x-axis as the depth direction, establish the depth direction vector of the normal vector of each triangular mesh;

[0087] Get the depth direction unit vector, check the direction consistency with the depth direction vector of the normal vector of each triangular mesh, retain the normal vector represented by the depth direction vector that is consistent with the depth direction unit vector, and complete the correction of the direction of the normal vectors of all triangular meshes.

[0088] Specifically, based on the above embodiment, the embodiment of the present invention performs point cloud denoising based on the features of Mesh, including two parts, namely, point cloud texture feature construction and parameter adaptation, specifically including:

[0089] S201: Point cloud texture feature construction: The invention converts the point cloud into a triangular mesh by establishing the topological relationship between the points in the point cloud. The change in the azimuth angle of the normal vector of the mesh can represent the change in the local curvature of the object surface.

[0090] Reference Figure 4 The figure shows the surface normal vector azimuth angles of objects of different shapes. Taking a sphere and a cube as examples, the surface normal vector azimuth angle of a sphere changes evenly; the surface normal vector azimuth angle of a single rectangular surface of a cube hardly changes, and the surface normal vector azimuth angle near the edge changes suddenly. Figure 5 As shown in the figure, it is a schematic diagram of different types of normal vectors. Based on this feature, noise can generally be divided into the following types: Figure 5The two categories of "Sample 1" and "Sample 2" shown are points that cause a sudden change in the curvature of the object surface and points that are particularly far away from the main body of the point cloud, respectively. For "Sample 1", the change in the normal vector azimuth can effectively distinguish between noise points and valid points. However, when the noise points are particularly discrete, it is difficult to form an effective curvature feature with its neighboring points, and sometimes it happens to appear as a small change in the normal vector azimuth, as shown in "Sample 2". Therefore, the present invention proposes a normalized Mesh size as the second type of point cloud texture feature. The more discrete the points are, the larger the value of the normalized Mesh size.

[0091] The qualitative relationship between the two types of features is as follows: the azimuth change of the normal vector containing noise points is larger than that of the normal vector not containing noise points. When the cosine value is used to represent the azimuth change of the normal vector, the cosine value containing noise points is smaller than that of the normal vector not containing noise points. The normalized Mesh size containing noise points is larger than that of the normal vector not containing noise points. In order to fuse the two types of features, the qualitative relationship must be unified. To this end, the present invention deconvolutes and translates the normalized Mesh size so that the eigenvalue containing noise points is larger than that containing no noise points. Therefore, the processed normalized triangular mesh size value is used as the modulus of the normal vector to construct a unified eigenvector.

[0092] The quantitative relationship between the two types of features is as follows: First, the texture feature corresponding to "sample 1" is defined as The texture feature corresponding to "sample 2" is defined as Assume that any triangular mesh m i The nearest neighbor without noise is m j , normal vector f i The azimuth change is f i With f j If the azimuth angle changes slightly, then m i With m j are all triangle meshes without noise; if the azimuth angle changes greatly, then m i is a triangular mesh containing noise. However, in the actual point cloud collected, it is impossible to determine whether m j To solve this problem, the embodiment of the present invention uses f i The average azimuth change of its K nearest neighbors' normal vectors replaces f i With f j The azimuth deviation is calculated as follows:

[0093]

[0094] in, represents the first texture feature vector, K represents the total number of K nearest neighbor triangle meshes; f i represents the corrected normal vector of the i-th triangle mesh, ||f i|| represents the corrected normal vector f of the i-th triangle mesh i The modulus length; f ik represents the corrected normal vector of the kth neighboring triangle mesh of the i-th triangle mesh, ||f ik || represents the vector f ik The mold length.

[0095] The size of the Mesh is used to measure the discreteness of the three vertices, which is normalized as the second type of texture feature. The calculation formula is as follows:

[0096]

[0097] in, is the second texture feature vector, Represents the normalized mesh size of the i-th triangle mesh, expressed as p is represented by a, b, and c represent the lengths of the three sides of the triangular mesh respectively; D represents the anti-fold translation operation.

[0098] S202: Parameter adaptation: Sort the eigenvalues, and the curvature of the approximate function changes suddenly near the threshold. Based on this, the embodiment of the present invention automatically selects the threshold by studying the slope of the approximate function. The eigenvalues ​​are fitted with a high degree, and near the threshold, the polynomial fitting fluctuates due to the rapid change of the slope. The embodiment of the present invention calculates the position where the first peak appears, and uses the eigenvalue corresponding to the position as the denoising threshold.

[0099] In the embodiment of the present invention, the effect of the present invention is verified by using an actually collected point cloud data set. The specific denoising process includes:

[0100] Establish topological relationships between the collected point clouds and construct triangular meshes;

[0101] Calculate the normal vector of the triangle mesh and correct it to point outwards;

[0102] Two types of texture features are calculated based on the triangle mesh and the triangle mesh normal vector respectively; a unified feature vector is constructed that integrates the two types of texture features;

[0103] According to the eigenvalue, fit the characteristic function;

[0104] According to the change characteristics of the slope of the characteristic function, the parameters are adaptively controlled;

[0105] Determine whether the eigenvalue corresponding to a single triangular mesh is greater than a threshold;

[0106] The three vertices of the triangular mesh whose eigenvalues ​​are greater than the threshold are retained, and the set of all retained vertices is the denoising result.

[0107] Reference Figure 6 As shown, Figure 6 (a) is the effect diagram of the aircraft point cloud containing noise. Figure 6 (b) is the denoised aircraft point cloud effect diagram; refer to Figure 7 As shown, Figure 7 (a) is the point cloud effect of the vase containing noise. Figure 7 (b) is the denoised vase point cloud effect. Figure 6 and Figure 7 It can be seen that the present invention creates point cloud texture features based on Mesh, finds the slope mutation position through a simple and robust polynomial fitting function, and obtains a suitable denoising threshold. Compared with previous methods, the Mesh-based point cloud denoising method provided by the present invention can retain valid points as much as possible while denoising, thereby maintaining data accuracy.

[0108] Reference Figure 8 As shown, Figure 8 (a) is the first human point cloud denoising effect diagram. Figure 8 (b) is the fitting function curve of the first humanoid point cloud. Figure 8 (c) is a scatter plot of the feature values ​​of the first human point cloud; Fig. 9 As shown, Fig. 9 (a) is the denoising effect of the second human point cloud. Fig. 9 (b) is the fitting function curve of the second humanoid point cloud. Fig. 9 (c) is a scatter plot of the feature values ​​of the second humanoid point cloud; Fig.10 As shown, Fig.10 (a) is the denoising effect of the third human point cloud. Fig.10 (b) is the fitting function curve of the third humanoid point cloud. Fig.10 (c) is a scatter plot of the feature values ​​of the third human point cloud.

[0109] Based on the above embodiment, in an embodiment of the present invention, a point cloud denoising device based on Mesh is also provided, including:

[0110] The Mesh structure construction module 100 is used to obtain a point cloud set to be de-noised, and to construct a Mesh structure composed of a plurality of triangular meshes based on the topological relationship of all point clouds in the point cloud set to be de-noised;

[0111] A normal vector correction module 200 is used to calculate the normal vectors of all triangular meshes respectively, and correct the normal vectors of all triangular meshes so that the normal vectors of all triangular meshes have the same direction;

[0112] The feature vector construction module 300 is used to calculate, for each triangular mesh, the average azimuth change of the corrected normal vector of the triangular mesh and the corrected normal vectors of its K nearest triangular meshes, and construct a first texture feature vector for each triangular mesh; normalize the size of each triangular mesh, deconvolute and translate the normalized triangular mesh, and construct a second texture feature vector for each triangular mesh; merge the first texture feature vector and the second texture feature vector of each triangular mesh to obtain a unified feature vector for each triangular mesh;

[0113] The fitting denoising module 400 is used to perform high-order fitting based on the eigenvalues ​​of the unified eigenvectors of all triangular meshes to obtain a fitting function; the eigenvalue corresponding to the first slope mutation in the fitting function is used as a denoising threshold; and triangular meshes whose eigenvalues ​​are greater than the denoising threshold are selected, and the vertices of all selected triangular meshes are combined into a denoised point cloud set.

[0114] The mesh-based point cloud denoising device of the present embodiment is used to implement the aforementioned mesh-based point cloud denoising method. Therefore, the specific implementation method of the mesh-based point cloud denoising device can be seen in the embodiment part of the mesh-based point cloud denoising method in the previous text, for example, the mesh structure construction module 100 and the normal vector correction module 200 are used to implement steps S101 and S102 in the aforementioned mesh-based point cloud denoising method; the feature vector construction module 300 is used to implement steps S103, S104 and S105 in the aforementioned mesh-based point cloud denoising method; the fitting denoising module 400 is used to implement steps S106 and S107 in the aforementioned mesh-based point cloud denoising method. Therefore, its specific implementation method can refer to the description of the corresponding embodiments of each part, which will not be repeated here.

[0115] In summary, the Mesh-based point cloud denoising method and device described in the present invention converts the point cloud set to be denoised into a Mesh structure. Noise is a point that causes a sudden change in the local curvature of the object surface. In the absence of noise, the azimuth angles of the normal vectors in the neighborhood are similar; the azimuth angle of the normal vector containing noise is significantly different from the direction angle of the normal vector of its neighboring normal vector without noise; the present invention constructs a first texture feature vector based on the average azimuth change of the normal vector of the triangular mesh and the normal vectors of its K nearest triangular meshes, accurately characterizing the geometric details in the Mesh structure; the value of the normalized triangular mesh size is reversed and translated to construct a second texture feature vector, accurately characterizing the degree of discreteness of the point cloud in the point cloud set to be denoised; the two texture feature vectors are fused to obtain a unified feature vector; based on the eigenvalues ​​of all unified feature vectors, function fitting is performed to obtain a fitting function, and the eigenvalue corresponding to the sudden change in the slope of the fitting function is used as the denoising threshold, the point cloud set to be denoised is denoised, and the three vertices of all triangular meshes whose eigenvalues ​​are greater than the denoising threshold are retained to obtain a denoising result. The denoising method of the present invention is based on the mutation slope as the denoising threshold, which realizes the automatic selection of the denoising threshold, can simply and robustly realize the identification of noise points, and can retain the valid point cloud as much as possible during denoising, thereby reducing the impact on the correct point cloud and improving the point cloud denoising accuracy.

[0116] Based on the above embodiments, an embodiment of the present invention further provides an application of the Mesh-based point cloud denoising method as described above in the field of three-dimensional modeling, and uses the precise point cloud denoised by the method of the present invention to construct an accurate three-dimensional model.

[0117] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0118] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0121] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A point cloud denoising method based on Mesh, characterized in that: include: Obtain a set of point clouds to be denoised, and construct a Mesh structure consisting of multiple triangular meshes based on the topological relationship of all point clouds in the set of point clouds to be denoised; Calculate the normal vectors of all triangular meshes separately and correct the normal vectors of all triangular meshes so that the normal vectors of all triangular meshes have the same direction, including: Based on the relationship between the minimum circumscribed rectangular box of the Mesh structure and the end point of the normal vector of each triangular mesh, the normal vectors of all triangular meshes are corrected so that the normal vectors of all triangular meshes have the same direction, including: comparing the minimum circumscribed rectangular box of the Mesh structure with the end point of the normal vector of each triangular mesh until the normal vectors of all triangular meshes in the Mesh structure are judged, and the direction of the normal vectors of all triangular meshes are corrected, including: if the end point of the normal vector is outside the minimum circumscribed rectangular box, the normal vector is retained; if the end point of the normal vector is within the minimum circumscribed rectangle, the minimum distance from the end point of the normal vector to the minimum circumscribed rectangular box is calculated as the first distance; the minimum distance from the end point of the normal vector in the opposite direction to the normal vector to the minimum circumscribed rectangular box is calculated as the second distance; comparing the first distance with the second distance, and retaining the normal vector corresponding to the smaller distance value between the first distance and the second distance; Or based on the direction consistency relationship between the normal vector of each triangular mesh and the unit vector in the depth direction, correct the normal vectors of all triangular meshes so that the directions of the normal vectors of all triangular meshes are the same, including: based on the radar coordinate system, with the x-axis as the depth direction, establishing the depth direction vector of the normal vector of each triangular mesh; obtaining the depth direction unit vector, performing direction consistency judgment with the depth direction vector of the normal vector of each triangular mesh, retaining the normal vector represented by the depth direction vector that is consistent with the depth direction unit vector, and completing the correction of the direction of the normal vectors of all triangular meshes; For each triangular mesh, calculate the average azimuth change of the corrected normal vector of the triangular mesh and the corrected normal vectors of its K nearest triangular meshes, and construct the first texture feature vector of each triangular mesh; Normalizing the size of each triangular mesh, deconvoluting and translating the normalized triangular mesh, and constructing a second texture feature vector for each triangular mesh; Merging the first texture feature vector of each triangular mesh with the second texture feature vector to obtain a unified feature vector of each triangular mesh; Based on the eigenvalues ​​of the unified eigenvectors of all triangular meshes, high-order fitting is performed to obtain a fitting function; the eigenvalue corresponding to the first slope mutation in the fitting function is used as the denoising threshold; Select the triangular meshes whose eigenvalues ​​are greater than the denoising threshold, and use the vertices of all the selected triangular meshes to form a denoised point cloud set.

2. The point cloud denoising method based on Mesh according to claim 1, characterized in that: Using the improved greedy projection triangulation method, based on the topological relationship of all point clouds in the point cloud set to be denoised, a Mesh structure composed of multiple triangular meshes is constructed, including: Based on the topological relationship between all point clouds in the point cloud set to be denoised, the connection relationship between different point clouds is obtained; Project all point clouds in the point cloud set to be denoised onto a plane, triangulate them according to the connection relationship between different point clouds, generate multiple triangular meshes, and form a Mesh structure; If there is a point cloud that is not connected to the Mesh structure, search for two neighboring point clouds of the point cloud among the vertices of all triangular meshes in the Mesh structure; form a new triangular mesh with the point cloud and its two neighboring point clouds and add it to the Mesh structure until all point clouds are added to the Mesh structure to complete the construction of the Mesh structure.

3. The point cloud denoising method based on Mesh according to claim 1, characterized in that: The normal vector of a triangle mesh is expressed as: Among them, f n ' represents the normal vector of the nth triangle mesh; A, B, and C are the three vertices that make up the triangle mesh, the coordinates of A are (x1, y1, z1), the coordinates of B are (x2, y2, z2), and the coordinates of C are (x3, y3, z3).

4. The point cloud denoising method based on Mesh according to claim 1, characterized in that: The first texture feature vector of a triangle mesh is expressed as: in, represents the first texture feature vector, K represents the total number of K nearest neighbor triangle meshes; f i represents the corrected normal vector of the i-th triangle mesh, ||f i || represents the corrected normal vector f of the i-th triangle mesh i The modulus length; f ik represents the corrected normal vector of the kth neighboring triangle mesh of the i-th triangle mesh, ||f ik || represents vector f ik The mold length.

5. The point cloud denoising method based on Mesh according to claim 1, characterized in that: The second texture feature vector of the triangle mesh is expressed as: in, is the second texture feature vector, Represents the normalized mesh size of the i-th triangle mesh, expressed as p is represented by a, b, and c represent the lengths of the three sides of the triangular mesh respectively; D represents the anti-fold translation operation.

6. The point cloud denoising method based on Mesh according to claim 1, characterized in that: The eigenvalues ​​of the unified eigenvectors of all triangular meshes are used to perform high-order fitting to obtain a fitting function, which is expressed as: y=a k x k +a k-1 x k-1 +…+a1x+a0; Among them, y represents the fitted eigenvalue, k represents the highest degree of the polynomial, and a k represents the coefficient of the k-th term after fitting, x k represents the kth power of the eigenvalue.

7. A point cloud denoising device based on Mesh, characterized in that: include: Mesh structure building module, used to obtain the point cloud set to be denoised, and build a Mesh structure composed of multiple triangular meshes based on the topological relationship of all point clouds in the point cloud set to be denoised; The normal vector correction module is used to calculate the normal vectors of all triangular meshes respectively and correct the normal vectors of all triangular meshes so that the normal vectors of all triangular meshes have the same direction, including: Based on the relationship between the minimum circumscribed rectangular box of the Mesh structure and the end point of the normal vector of each triangular mesh, the normal vectors of all triangular meshes are corrected so that the normal vectors of all triangular meshes have the same direction, including: comparing the minimum circumscribed rectangular box of the Mesh structure with the end point of the normal vector of each triangular mesh until the normal vectors of all triangular meshes in the Mesh structure are judged, and the direction of the normal vectors of all triangular meshes are corrected, including: if the end point of the normal vector is outside the minimum circumscribed rectangular box, the normal vector is retained; if the end point of the normal vector is within the minimum circumscribed rectangle, the minimum distance from the end point of the normal vector to the minimum circumscribed rectangular box is calculated as the first distance; the minimum distance from the end point of the normal vector in the opposite direction to the normal vector to the minimum circumscribed rectangular box is calculated as the second distance; comparing the first distance with the second distance, and retaining the normal vector corresponding to the smaller distance value between the first distance and the second distance; Or based on the direction consistency relationship between the normal vector of each triangular mesh and the unit vector in the depth direction, correct the normal vectors of all triangular meshes so that the directions of the normal vectors of all triangular meshes are the same, including: based on the radar coordinate system, with the x-axis as the depth direction, establishing the depth direction vector of the normal vector of each triangular mesh; obtaining the depth direction unit vector, performing direction consistency judgment with the depth direction vector of the normal vector of each triangular mesh, retaining the normal vector represented by the depth direction vector that is consistent with the depth direction unit vector, and completing the correction of the direction of the normal vectors of all triangular meshes; The feature vector construction module is used to calculate, for each triangular mesh, the average azimuth change of the corrected normal vector of the triangular mesh and the corrected normal vectors of its K nearest triangular meshes, and construct the first texture feature vector of each triangular mesh; normalize the size of each triangular mesh, deconvolute and translate the normalized triangular mesh, and construct the second texture feature vector of each triangular mesh; merge the first texture feature vector of each triangular mesh with the second texture feature vector, and obtain a unified feature vector of each triangular mesh; The fitting denoising module is used to perform high-order fitting based on the eigenvalues ​​of the unified eigenvectors of all triangular meshes to obtain the fitting function; the eigenvalue corresponding to the first slope mutation in the fitting function is used as the denoising threshold; the triangular meshes with eigenvalues ​​greater than the denoising threshold are selected, and the vertices of all the selected triangular meshes are used to form a denoised point cloud set.

Images