Directional normal estimation method, device and equipment for three-dimensional point cloud and medium

Abstract

The invention discloses a three-dimensional point cloud orientation normal estimation method and device, equipment and a medium, and the method comprises the steps: obtaining point cloud data, and carrying out the sampling of an original point cloud P, and obtaining a query point set Q; constructing a mobile operation network based on Neure-Pull, inputting a query point q in the query point set Q into the mobile operation network, and moving the query point q to a bottom level q '; training the mobile operation network according to the point q'after the mobile operation and the loss function so as to adjust parameters of the mobile operation network; and estimating the orientation normal of the three-dimensional point cloud by adopting the trained mobile operation network to obtain an evaluation result. According to the method, local constraints are designed, and a new loss function is introduced to train the neural network, so that the implicit field can faithfully describe the bottom surface described by the point cloud; in addition, no training needs to be carried out under a training set, and the operation process is simple and convenient. The method can be widely applied to the fields of computer vision and computer graphics.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and computer graphics, and particularly to a method, apparatus, device, and medium for estimating oriented normals of three-dimensional point clouds. Background Art

[0002] In computer vision and graphics, normal estimation of point clouds is a prerequisite for many techniques. As an important geometric property of point clouds, the normal vector is one of the local descriptors of point clouds and can provide additional geometric information for many downstream applications, such as denoising, segmentation, and registration. In addition, some computer vision tasks require the normals to have a consistent direction, that is, oriented normals. Oriented normals can clearly reveal the geometric structure and play an important role in downstream applications such as rendering and surface reconstruction. Therefore, point cloud normal estimation has long been an important research topic.

[0003] Generally speaking, estimating oriented normals requires two stages: (1) estimating non-oriented normals from the local neighborhood of the query point; (2) performing normal orientation to make the normal directions globally consistent, such as towards the outside of the surface. Although non-oriented normals can be estimated by fitting a plane or surface to the local neighborhood, the direction of the normal, whether it is inwards or outwards, cannot be determined. Therefore, it is still a challenge to estimate oriented normals from point clouds with noise, varying point densities, and complex geometries in an end-to-end manner.

[0004] Normal orientation methods can be divided into two main categories, namely propagation-based and volume-based methods. The most popular and simplest normal orientation method is based on local propagation, that is, spreading the direction of the seed point to adjacent points through a minimum spanning tree (MST). Most existing normal orientation methods are based on this strategy, such as the pioneering work and its improved methods. These methods are limited by error accumulation, and incorrect positioning may degrade all subsequent propagation steps. In addition, these methods rely to a large extent on the assumptions of smoothness and cleanliness, which makes them easily fail in the presence of sharp corners, density variations, and noise. At the same time, their accuracy is sensitive to the size of the propagated neighborhood. For example, large sizes are usually used to eliminate outliers and noise, but they may also incorrectly include nearby surfaces.

[0005] Since local consistency is usually not sufficient to achieve a robust orientation between different inputs, normal directions are determined by applying various volumetric representation techniques, such as signed distance functions and variational formulations. These methods aim to partition the space into inside / outside and determine whether the point normals are inward or outward. Correctly oriented normals can be obtained in the solution representations of these methods, but their normals are not precise in the perpendicular direction. Although these methods have improved in terms of accuracy and robustness, they cannot be scaled to large point clouds due to their computational complexity. Generally speaking, propagation-based methods are difficult to handle sharp features, while volumetric-based methods are difficult to handle open surfaces. In addition, the above methods are usually complex and require two-stage operations, and their performance depends to a large extent on the parameter adjustment of each individual stage.

[0006] Recently, several learning-based methods have been proposed to provide oriented normals from point clouds, such as PCPNet, SHS-Net, etc., and have shown good performance. Since they focus on learning accurate local feature descriptors without fully exploring the relationship between surface normal directions and the underlying surface, their performance on different noise levels and geometric structures cannot be guaranteed. Summary of the Invention

[0007] To at least to some extent solve one of the technical problems existing in the prior art, an object of the present invention is to provide a method, device, equipment and medium for estimating oriented normals of three-dimensional point clouds.

[0008] The first technical solution adopted by the present invention is:

[0009] Obtain point cloud data, and sample a query point set Q from the original point cloud P;

[0010] Based on Neural-Pull, construct a movement operation network, input the query point q in the query point set Q into the movement operation network, and move the query point q to the underlying surface q';

[0011] Train the movement operation network according to the moved point q' and the loss function to adjust the parameters of the movement operation network;

[0012] Use the trained movement operation network to estimate the oriented normals of the three-dimensional point cloud and obtain an evaluation result.

[0013] Further, the obtaining point cloud data and sampling a query point set Q from the original point cloud P includes:

[0014] Uniformly extract query points q from P to form a query point set where N Q is the number of query points;

[0015] For each point qj , establish an isotropic Gaussian function distribution According to this distribution randomly sample multiple query positions.

[0016] Furthermore, the mobile operation network constructed based on Neural-Pull includes:

[0017] Use a neural network to learn an implicit field representing a three-dimensional shape. Train a neural network f with parameter θ according to the query point q, and at the same time predict the signed distance value s and gradient g of the query three-dimensional object position q = [x, y, z] to represent the three-dimensional shape;

[0018] By learning, pull the randomly sampled query position q i to its nearest neighbor t on the surface i , where the query positions form a set Q, along or against the gradient g i at q i in the direction of, with the signed distance s i as the step size to pull the query position q i , move q i to the nearest neighborhood t on the surface i .

[0019] Furthermore, inputting the query point q in the query point set Q into the mobile operation network and moving the query point q to the bottom layer q', includes:

[0020] Adopt the mobile operation in Neural-Pull to project the query point q to q'. When the function f is continuous and differentiable, the normal vector perpendicular to the surface at point p where ||·|| represents the vector norm, to obtain q ′ = q - f(q; θ)·n q ;

[0021] For the point q ∈ Q, it is expected that the function f can provide the accurate signed distance f(q; θ) and gradient to move q to the nearest point q' on the surface and minimize the error ‖q ′ - q‖.

[0022] Furthermore, the expression of the mobile operation is:

[0023]

[0024] In the formula, θ is the parameter of the mobile operation network;

[0025] During normal vector estimation, the entire point cloud P is input into the network to derive the gradient embedded in the learning function; since the gradient is perpendicular to the surface, along the gradient direction, the distance from the surface increases fastest, and the deviation from P to L is measured by finding the optimal gradient.

[0026] Furthermore, the training process of the movement operation network includes:

[0027] The original point cloud contains noise and outliers, which will seriously reduce the accuracy of surface approximation. At the same time, in the initial stage of training, the global network f θ will project the query point to a position far from the surface, which makes optimization difficult. Therefore, surface point constraints are introduced to encourage q′ to be closer to its nearest point projection q on the sparse point cloud s where q s is the neighborhood of the query point q;

[0028] The deviation from the point set P to Q is measured by finding the optimal gradient. Generally speaking, the original point cloud may contain noise and outliers, which will seriously reduce the accuracy of surface approximation; for the local neighborhood of each point p Given the neighborhood size K, represents the set of central coordinates of the points in this neighborhood;

[0029] To reduce the influence of inaccuracies in P, multi-scale neighborhoods are introduced based on a statistical method for to obtain the statistical expression of the neighborhood q where K s is the size of the multi-scale neighborhood, q is the size of the multi-scale neighborhood, q s is the size of the multi-scale neighborhood, q k is the point within the neighborhood q s and represents the K s nearest neighbor points of q ∈ Q in P;

[0030] The acquired point set is input into the neural network f θ and a local loss function L surf (θ) is added to describe the local details, so that the query point q can better approach the bottom surface described by the original three-dimensional point cloud.

[0031] Furthermore, the loss function in the training of the movement operation network is:

[0032] L = L v (θ) + L d (θ) + L surf (θ) + L con (θ) + L reg (θ) + L dis (θ)

[0033] The content of the six loss functions is as follows:

[0034] 1) L v : Minimize the loss function to reduce the deviation from P to Q; where i is the number of iterations, Q is the point set of the query point q, and f i Q is the expression (signed distance value) output by the neural network function f for the point q in the point set Q.

[0035] 2) L d : Introduce the surface point constraint L d =‖q′ - q s ‖, encourage q′ to be closer to its nearest point projection q on the sparse point cloud s , where q s is the neighborhood of the query point q;

[0036] 3) L surf : Introduce the local constraint L surf =|f θ (q s )|, input the neighborhood point set v s (Q) of the query point q into the global network f θ ; where q s is the neighborhood of the query point q;

[0037] 4) L con : Introduce the confidence weighted cosine distance to evaluate the gradient consistency of the point set Q, and the expression is where <·> represents the cosine distance of the vector, and ω is an adaptive weight;

[0038] 5) L reg : Use the regression loss L reg to calculate the predicted distance f i Q ,

[0039] 6) L dis : Introduce the moving distance loss function where d gt and d pred are the distances from the query point q and the moving point q′ to the bottom surface respectively.

[0040] The second technical solution adopted by the present invention is:

[0041] A three-dimensional point cloud orientation normal estimation device, comprising:

[0042] A point cloud sampling module, configured to obtain point cloud data and sample a query point set Q from the original point cloud P;

[0043] A model construction module, configured to construct a mobile operation network based on Neural-Pull, input a query point q in the query point set Q into the mobile operation network, and move the query point q to the bottom layer q';

[0044] A model training module, configured to train the mobile operation network according to the moved point q' and a loss function to adjust the parameters of the mobile operation network;

[0045] A model testing module, configured to estimate the directional normal of the three-dimensional point cloud by using the trained mobile operation network to obtain an evaluation result.

[0046] The third technical solution adopted by the present invention is:

[0047] An electronic device, the electronic device includes a processor and a memory, and at least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement a method for estimating the directional normal of a three-dimensional point cloud as described above.

[0048] The fourth technical solution adopted by the present invention is:

[0049] A computer-readable storage medium, and at least one instruction, at least one program, a code set or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement a method for estimating the directional normal of a three-dimensional point cloud as described above.

[0050] The fifth technical solution adopted by the present invention is:

[0051] A computer program product or a computer program, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device executes the above method.

[0052] The beneficial effects of the present invention are: By designing local constraints and introducing a new loss function to train a neural network, the implicit field more faithfully describes the underlying surface described by the point cloud; in addition, no training is required under the training set, and the operation process is simple and convenient. Description of the Drawings

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following introduces the accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings in the following introduction are only for conveniently and clearly presenting some embodiments of the technical solutions in the present invention. For those skilled in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0054] Figure 1 is the flowchart of the steps of a method for estimating the oriented normal of a three-dimensional point cloud in an embodiment of the present invention;

[0055] Figure 2 is the overall flowchart of the method for estimating the oriented normal of a three-dimensional point cloud in an embodiment of the present invention;

[0056] Figure 3 is the flowchart of the movement operation in an embodiment of the present invention;

[0057] Figure 4 is a comparison schematic diagram between the method proposed in an embodiment of the present invention and the existing implicit representation method for surface reconstruction. Detailed Embodiment

[0058] The following details the embodiments of the present invention. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as a limitation of the present invention. For the step numbers in the following embodiments, they are only set for the convenience of explanation and illustration, and no limitation is imposed on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0059] In the description of the present invention, it should be understood that for the orientation description, such as the orientation or positional relationship indicated by up, down, front, back, left, right, etc., is based on the orientation or positional relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0060] In the description of the present invention, the meaning of several is one or more, the meaning of multiple is two or more, greater than, less than, exceeding, etc. are understood as not including the present number, and above, below, within, etc. are understood as including the present number. If there is a description of first and second, it is only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features or the sequence of the indicated technical features.

[0061] In the description of the present invention, unless otherwise clearly defined, terms such as "set", "install", "connect" should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific content of the technical solution.

[0062] Term Explanation:

[0063] Neural-Pull: An abbreviation for "Learning signed distance functions from point clouds by learning to pull space onto surfaces", which is used to learn signed distance functions from point cloud data.

[0064] In view of the existing technical problems, the present invention provides an end-to-end directed normal estimation scheme for 3D point clouds. By introducing a neural network for learning neural gradient functions and two new functions to guide the implicit field to more faithfully describe the underlying surface described by the sparse point cloud, and without any training on the training set, the directed normal estimation effect on the PCPNet dataset is better than previous works.

[0065] Based on Neural-Pull, the present invention introduces a neural network for learning neural gradient functions to learn a direction-consistent neural gradient field from point clouds. Specifically, a neural network is trained to obtain the signed distance value and the gradient of the query position, both of which are calculated by the network itself. According to the distance and gradient predicted by the network, the query point can be pulled back to the nearest point on the surface, which is a differentiable operation that allows us to update the signed distance value and the gradient simultaneously during training. However, due to the lack of local detail description, the derived gradient usually deviates from the ground truth direction normal. Therefore, we introduce local constraints, which promote the query point to iteratively reach the moving target and aggregate onto the approximate surface, so that the implicit field more faithfully describes the underlying surface described by the point cloud. Through ablation experiments, it is verified that the newly designed functions can effectively improve the directed normal estimation effect and enhance the robustness to noise, outliers, and density changes.

[0066] Embodiment 1

[0067] As Figure 1 and Figure 2 shown, this embodiment provides a method for estimating the directed normal of a 3D point cloud, including the following steps:

[0068] S1. Obtain point cloud data, and sample a query point set Q from the original point cloud P.

[0069] Sample the query point set from the original point cloud P N Q is the number of point clouds.

[0070] Exemplarily, for each point q j ∈P, similar to Neural-Pull, an isotropic Gaussian function distribution is established, and 25 query positions are randomly sampled according to this distribution, where σ 2 is a parameter that controls how far we are from the surface, and the query positions can be sampled. 5000 query positions are randomly selected from Q as a batch for training. Specifically, let the distribution be concentrated in the neighborhood of P in three-dimensional space, query points q are uniformly drawn from P, and an isotropic Gaussian with its standard deviation parameter σ 2 is adaptively set to the distance to the l-th point closest to q. In this embodiment, l of the PCPNet dataset is set to 25.

[0071] In addition, for the query point q, we do not use the unsigned distance or its closest sampled point to obtain the query point neighborhood q s , but consider the average vector of its neighborhood and we set its k to 64.

[0072] S2. Build a movement operation network based on Neural-Pull, input the query point q in the query point set Q into the movement operation network, and move the query point q to the bottom surface q'.

[0073] As an implementation, to learn the neural gradient function, a simple neural network similar to Neural-Pull is used, which consists of eight linear layers and a skip connection from the input to the middle layer. Except for the last layer, each linear layer contains 512 hidden units and uses the ReLU activation function. The last layer outputs the signed distance of each point, realizing the construction of the implicit field, and the parameters of the linear layer are initialized using geometric initialization.

[0074] After optimization, the point cloud dataset Q processed in step S1 is input into the movement operation network, and then the movement distance d with dimension 1 and the movement direction n with dimension 3 can be obtained. After the movement operation the query point q can be moved to the bottom surface q' through two iterations.

[0075] S3. Train the movement operation network according to the moved point q' after the movement operation and the loss function to adjust the parameters of the movement operation network.

[0076] As an implementation, the moved point q′, the moving distance d, and the moving direction n are obtained. According to the loss function, the loss of each part is calculated. Finally, the parameters θ of the neural network f are continuously adjusted through loss backpropagation. The optimizer used in this embodiment is Adam, the batch size is 1, and the initial learning rate is 1e-3 and is decreased to 0 through a cosine decay scheduler.

[0077] S4. Use the trained movement operation network to estimate the orientation normal of the 3D point cloud and obtain the evaluation result.

[0078] This embodiment does not require any training on the training set, so only the test set of the PCPNet dataset needs to be processed. Our test set contains 19 shapes, a mixture of statues and man-made objects. It also includes 3 shapes that are constructed and analyzed by sampling from differentiable surfaces, and these surfaces all have well-defined normals and curvatures. For these shapes, the normal and curvature of each sampled point are calculated in an exact manner, rather than approximated by the faces and vertices of the mesh. Specifically, the root mean square error (RMSE) is used to evaluate the estimated normal, and the smaller the metric, the better the effect.

[0079] The method of this embodiment realizes end-to-end orientation normal vector estimation. By introducing local constraints, the implicit field can better describe the underlying surface. At the same time, it does not require any training on the training set, the environment configuration is simple, and it achieves the best performance on the PCPNet dataset.

[0080] The above method will be explained in detail below in combination with the accompanying drawings and specific embodiments.

[0081] The method of this embodiment aims to use a neural network to learn an implicit field representing a 3D shape. Train a neural network f with parameters θ from the input point q, and at the same time predict the signed distance value and the gradient g of the query 3D object position q = [x, y, z] to represent the 3D shape.

[0082] This method is a deep learning method for learning direction-consistent gradient vectors from 3D point clouds for normal estimation. It has excellent gradient approximation characteristics for the underlying geometry of the data. We use a simple neural network to parameterize the objective function, so as to reproduce the gradient generated at the point using the global implicit field. However, due to the lack of local detail description, the obtained gradient usually deviates from the ground truth orientation normal. Therefore, we design a local constraint to make the implicit field more faithfully describe the underlying surface described by the point cloud.

[0083] This embodiment attempts to learn to pull the query position q i randomly sampled around the surface to its nearest neighbor t i , where the query positions form the set Q = {qi , where \(i\in[1, I]\). It is possible to move along or against the gradient \(g\) obtained within the network i at \(q\) i in the direction of the signed distance \(s\) i with a step size to pull the query position \(q\) i , and move \(q\) i to the nearest neighborhood \(t\) on the surface i . More specifically, the gradient indicates the direction in which the signed distance from the surface in three-dimensional space increases fastest. Therefore, moving a point along the gradient will find its shortest path to the surface, obtaining where

[0084] For the above task, this embodiment proposes a method for estimating the oriented normal of a three-dimensional point cloud based on the Neural-Pull network. The method specifically includes three parts: (1) learning an implicit global surface; (2) learning an oriented gradient; (3) a loss function. These three parts will be introduced separately below.

[0085] (1) Learning an implicit global surface through a signed distance field

[0086] Generally, the gradient of a real-valued function \(f(x, y, z)\) in a 3D Cartesian coordinate system (also known as a gradient field) is given by a vector whose components are the first partial derivatives of \(f\). For example, where \(i\), \(j\), and \(k\) are the standard unit vectors in the \(x\), \(y\), and \(z\) coordinate directions, respectively. If the function \(f\) is differentiable at a point \(p\), and then the gradient field has the following two properties: 1) The maximum rate of change of the function \(f\) is determined by the magnitude of the gradient and the direction is the same as . 2) The gradient vector is perpendicular to the horizontal plane \(f(p)=0\).

[0087] Recently, deep neural networks have been widely used to reconstruct surfaces from three-dimensional point cloud data by learning implicit functions. These methods represent the surface as the zero-level surface of an implicit function, and mostly use a signed distance function as the shape representation. For example, where the function is a neural network with parameters \(\theta\), such as a multi-layer perceptron.

[0088] See Figure 3 , because the direction of the gradient is the direction in which the signed distance from the surface in three-dimensional space increases fastest. Therefore, moving a point along the gradient will find its shortest path to the surface. According to this property, we use the moving operation in Neural-Pull to project the query point \(q\) to \(q'\). When the function \(f\) is continuous and differentiable, the normal vector perpendicular to the surface at point \(p\), where \(\|\cdot\|\) represents the vector norm, obtaining \(q\)′ = q - f(q; θ)·n q For a point q ∈ Q, we expect the function f to provide an accurate signed distance f(q; θ) and gradient to move q to the closest point q′ on the surface, and we minimize the error ‖q ′ - q′‖. In this way, the function f can serve as an implicit representation of the underlying surface, and the zero-level set of f is an effective manifold describing the point cloud.

[0089] (2) Learning the Oriented Gradient

[0090] Sample a set of query points from the original point cloud P Specifically, for each point q j ∈ P, similar to Neural-Pull, an isotropic Gaussian function distribution is established, according to which we randomly sample 25 query positions, where σ 2 is a parameter controlling how far we are from the surface and can sample the query positions.

[0091] To better find the closest points of the query point set Q from the original point cloud P as the target positions in practice. We introduce a multi-step movement strategy to optimize the point set Q′ to cover the surface, rather than a single-step movement, referring to NeuralGF. If some points are distributed far from the surface or are disturbed by noise, it is difficult to directly reach the target point in one step. At this time, multi-step movement operations can better move the points to the underlying surface. More specifically, in the present invention, two iterations of the query point set Q are performed, and we directly estimate the oriented normal by incorporating the neural gradient into the implicit function learning.

[0092] During normal estimation, the entire point cloud P is input into the network to derive the gradient embedded in the learned function. Specifically, the gradient is perpendicular to the surface, and along the gradient direction, the distance from the surface increases fastest. Therefore, we measure the deviation from P to Q by finding the optimal gradient. The query point q can be projected onto the surface point along the gradient

[0093] Generally speaking, the original point cloud may contain noise and outliers, which will seriously reduce the accuracy of surface approximation. At the same time, in the initial stage of training, the global network f θ will project the query points to positions far from the surface, which makes optimization difficult. Therefore, we introduce a surface point constraint to encourage q′ to project closer to its closest point q on the sparse point cloud s , where q s is the neighborhood of the query point q.

[0094] For the local neighborhood of each point p Given the neighborhood size K, represents the set of central coordinates of the points in the neighborhood. We refer to NeuralGF. To reduce the impact of inaccuracies in P, we introduce a multi-scale neighborhood for v(x) based on a statistical approach The neighborhood q can be obtained s The expression of the statistical approach: where represents the K s nearest neighbor points of q ∈ Q in P.

[0095] To make up for the lack of local detail description in the global implicit field, we input the above-acquired point set into the neural network f θ and add a local loss function L surf (θ) to describe the local details, so that the query point q can better approach the bottom surface described by the original 3D point cloud.

[0096] (3) Loss function

[0097] The expression of the loss function of the present invention is:

[0098] L = L v (θ) + L d (θ) + L surf (θ) + L con (θ) + L reg (θ) + L dis (θ)

[0099] Next, these six loss functions will be introduced separately.

[0100] 1) L v : In this work, we directly estimate the direction of the normal by incorporating the neural gradient into implicit function learning. However, generally speaking, the original point cloud contains noise and outliers, which will seriously reduce the accuracy of surface approximation. To reduce the impact of inaccuracies in P, based on a statistical approach for introduce a multi-scale neighborhood The neighborhood q can be obtained s The expression of the statistical approach where represents the K s nearest neighbor points of q ∈ Q in P. By minimizing the loss function to reduce the deviation from P to Q. Where is the number of steps.

[0101] 2) L d: In actual operation, we find the nearest point of Q from the original point cloud P as the target position. We refer to NeuralGF and introduce a multi-step movement strategy to optimize the point set Q' obtained after movement to better cover the surface. If some points are distributed far from the surface or are disturbed by noise, it is difficult to directly reach the target point in one step. At this time, the optimization of the movement operation may experience many twists and turns before converging. In the initial stage of training, the global network f θ will project the query point to a position far from the surface, causing difficulties for optimization. Therefore, we introduce the surface point constraint L d =‖q′ - q s ‖, which encourages q′ to be closer to its nearest point projection q s on the sparse point cloud, where q s is the neighborhood of the query point q.

[0102] 3) L surf : To make the implicit field faithfully describe the lower surface described by the sparse point cloud, we add a local constraint and input the neighborhood point set of the query point q into the global network f θ . This makes up for the shortcoming that the gradient derived from neural gradient learning usually deviates from the ground truth direction normal due to the lack of local detail description. We introduce the local constraint L surf =|f θ (q s )|, where q s is the neighborhood of the query point q.

[0103] 4) L con : To pursue the consistency of the gradient in the iteration, that is, to keep the gradient directions obtained in each iteration parallel, align the gradient in the second step with the gradient in the first step. At the same time, we also need to consider the confidence of each point relative to the underlying surface. Therefore, we introduce a confidence-weighted cosine distance to evaluate the gradient consistency of point Q, and its form is where <·> represents the cosine distance of the vector. ω is an adaptive weight that represents the importance of each input query point according to the predicted distance, making the model pay more attention to the points with greater weights closer to the surface. This makes up for the deficiencies that the input point cloud has noise and some points are not on the underlying surface, and at the same time unifies the gradients of the query point q and its neighborhood q s .

[0104] 5) L reg : Due to the approximation error of the point-to-plane distance, the gradient of the implicit field is not the accurate normal direction, resulting in an error when q′ is very close to the lower bottom surface. To understand the more accurate distance and direction of the query point q, we introduce the shortest path constraint L regFor the moved point set Q′, we hope that they are as close to the bottom surface as possible. Therefore, we utilize the regression loss L reg to calculate the predicted distance f of the point set Q i Q ,

[0105] 6)L dis : To better enable the implicit field to describe the bottom surface layer, the iterated point q′ is closer to q s . We introduce the movement distance loss function where d gt and d pred are the distances from the query point q and the moved point q′ to the bottom surface layer respectively.

[0106] (4) Advantages and positive effects

[0107] To sum up, the main advantages of the present invention include the following aspects: 1) Design local constraints and introduce a new loss function to train the neural network, enabling the implicit field to more faithfully describe the underlying surface described by the point cloud, making up for the lack of local detail description in neural gradient learning; 2) Implement direction-consistent oriented normal estimation in an end-to-end manner, and have state-of-the-art performance in the oriented normal estimation of 3D point clouds on the PCPNet dataset; 3) Do not require any training under the training set, and the operation process is simple and convenient.

[0108] As can be seen from the data in Table 1, our experimental results are better compared with previous work. We not only compared with single-stage baseline methods such as PCPNet and SHS-Net, but also compared with two-stage baseline methods combined with representative algorithms based on different design concepts. Specifically, we used various combinations of undirected normal estimation methods (PCA, AdaFit, and HSurf-Net) and normal orientation methods (SNO and ODP). We reported the quantitative evaluation results of the PCPNet dataset in Table 1. Under the vast majority of data categories (noise level and density change), our method has better performance and achieves the best average result.

[0109] Table 1 Comparison of PCPNet Dataset Results

[0110]

[0111] As Figure 4 shown, Figure 4 shows some intuitive comparisons of the reconstructed surfaces of implicit representation methods. Figure 4 The figures in Figure 4It can be seen that the reconstructed graphics we obtained are more complete and have more precise details. Therefore, our method has stronger robustness and better effects under high noise.

[0112] The reason why the method of the present invention can achieve double optimization of performance and efficiency is as follows: an end-to-end directional normal vector estimation is adopted, without a training stage, which greatly reduces the training time. Local constraints are introduced, which well make up for the deficiency of only using the neural gradient algorithm and lacking local detail description. At the same time, the local detail description enables the movement operation to move the query point to the bottom layer surface, enabling the implicit field to more faithfully describe the underlying surface.

[0113] Embodiment 2

[0114] This embodiment provides a device for directional normal vector estimation of three-dimensional point clouds, including:

[0115] A point cloud sampling module, configured to obtain point cloud data and sample a query point set Q from the original point cloud P;

[0116] A model construction module, configured to construct a movement operation network based on Neural-Pull, input the query point q in the query point set Q into the movement operation network, and move the query point q to the bottom layer surface q';

[0117] A model training module, configured to train the movement operation network according to the moved point q' and the loss function to adjust the parameters of the movement operation network;

[0118] A model testing module, configured to estimate the directional normal vector of the three-dimensional point cloud by using the trained movement operation network to obtain an evaluation result.

[0119] Since this device is a device for directional normal vector estimation of three-dimensional point clouds in an embodiment of the present invention, and the principle of solving problems by this device is similar to that of this method, the implementation of this device can refer to the implementation process of the above method embodiment, and the repeated parts will not be described again.

[0120] Embodiment 3

[0121] An embodiment of the present invention further provides an electronic device, where the electronic device includes a processor and a memory, and at least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement Figure 1 a method for directional normal vector estimation of three-dimensional point clouds as shown.

[0122] It can be understood that the memory may include a Random Access Memory (RAM) and may also include a Read-Only Memory. Optionally, the memory includes a non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a program storage area and a data storage area. Among them, the program storage area can store instructions for implementing the operating system, instructions for at least one function, instructions for implementing the above various method embodiments, etc.; the data storage area can store data created according to the use of the server, etc.

[0123] The processor may include one or more processing cores. The processor uses various interfaces and circuits to connect various parts within the entire server. By running or executing instructions, programs, code sets, or instruction sets stored in the memory, and by calling data stored in the memory, it executes various functions of the server and processes data. Optionally, the processor may be implemented in at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor may integrate a combination of one or several of a Central Processing Unit (CPU) and a modem, etc. Among them, the CPU mainly processes the operating system and application programs, etc.; the modem is used to process wireless communication. It can be understood that the above modem may not be integrated into the processor and may be implemented separately by a single chip.

[0124] Since this electronic device is an electronic device corresponding to a method for estimating the oriented normal of a three-dimensional point cloud according to an embodiment of the present invention, and the principle by which this electronic device solves problems is similar to that of this method, the implementation of this electronic device can refer to the implementation process of the above method embodiment, and the repeated parts will not be elaborated here.

[0125] Embodiment 4

[0126] An embodiment of the present invention further provides a computer-readable storage medium, in which at least one instruction, at least one segment of program, code set, or instruction set is stored, and the at least one instruction, the at least one segment of program, the code set, or the instruction set is loaded and executed by a processor to implement Figure 1 a method for estimating the oriented normal of a three-dimensional point cloud as shown.

[0127] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium, which includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disc memories, tape memories, or any other computer-readable medium capable of carrying or storing data.

[0128] Since this storage medium is the storage medium corresponding to a method for estimating the directional normal of a three-dimensional point cloud in an embodiment of the present invention, and the principle of solving problems by this storage medium is similar to that of this method, the implementation of this storage medium can refer to the implementation process of the above method embodiment, and the repeated parts will not be elaborated.

[0129] Embodiment 5

[0130] In some possible implementation manners, various aspects of the method in an embodiment of the present invention can also be implemented in the form of a program product, which includes program code. When the program product runs on a computer device, the program code is used to cause the computer device to execute the steps of a method for estimating the directional normal of a three-dimensional point cloud according to various exemplary implementation manners described above in this specification. Among them, the executable computer program code or "code" for executing each embodiment can be written in a high-level programming language such as Python, C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, structured query language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0131] It should be understood that each part of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application specific integrated circuits with appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0132] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0133] The above embodiments are only for illustrating the technical concept and features of the present invention, and the purpose is to enable those of ordinary skill in the art to understand the content of the present invention and implement it accordingly, and cannot be used to limit the protection scope of the present invention. All equivalent changes or modifications made according to the essence of the content of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for estimating oriented normals of a three-dimensional point cloud, characterized in that: The following steps are involved: Obtain point cloud data and sample the query point set Q from the original point cloud P; Construct a mobile operation network based on Neural-Pull, input the query point q in the query point set Q into the mobile operation network, and move the query point q to the bottom layer q′; The mobile operation network is trained according to the point q′ after the mobile operation and the loss function to adjust the parameters of the mobile operation network; The trained mobile manipulation network is used to estimate the oriented normals of the 3D point cloud and obtain the evaluation results.

2. A method for estimating oriented normals of a three-dimensional point cloud according to claim 1, characterized in that: The step of acquiring point cloud data and sampling the original point cloud P to obtain a query point set Q includes: Uniformly extract query points q from P to form a query point set Where N Q is the number of query points; For each point q j , establish an isotropic Gaussian function distributed According to this distribution Randomly sample multiple query locations.

3. The method for estimating oriented normals of a three-dimensional point cloud according to claim 1, characterized in that: The mobile operation network is constructed based on Neural-Pull, including: Use a neural network to learn an implicit field representing a 3D shape. Train a neural network f with a parameter θ based on the query point q. Simultaneously predict the signed distance value s and gradient g of the query 3D object position q = [x, y, z] to represent the 3D shape. By learning to randomly sample query positions q around the surface i Pull to the nearest neighbor t on its surface i , where the query positions form a set Q, along or against the q obtained within the network i The gradient g i direction, with a signed distance s i The step length pulls the query position q i , q i Move to the nearest neighbor t on the surface i .

4. The method for estimating oriented normals of a three-dimensional point cloud according to claim 1, characterized in that: The step of inputting a query point q in the query point set Q into the mobile operation network and moving the query point q to the bottom layer q′ comprises: The query point q is projected to q′ using the move operation in Neural-Pull. When the function f is continuous and differentiable, the normal vector perpendicular to the surface at point p is where ||·|| represents the vector norm, and we get q′=qf(q;θ)·n q ; For a point q∈Q, we expect the function f to provide the exact signed distance f(q;θ) and gradient Move q to the nearest point q′ on the surface and minimize the error ‖q′-q‖.

5. The method for estimating oriented normals of a three-dimensional point cloud according to claim 1, characterized in that: The expression for the move operation is: Where θ is the parameter of the mobile operation network; During normal estimation, the entire point cloud P is input into the network to derive the gradients embedded in the learned function; Because the gradient is perpendicular to the surface, the distance from the surface increases fastest along the gradient direction, and the deviation from P to Q is measured by finding the optimal gradient.

6. The method for estimating oriented normals of a three-dimensional point cloud according to claim 1, characterized in that: The training process of the mobile operation network includes: The original point cloud contains noise and outliers, which seriously degrades the accuracy of surface approximation. At the same time, in the early stage of training, the global network f θ The query point will be projected to a position far away from the surface, which makes optimization difficult, so the surface point constraint L is introduced np =‖q′-q s ‖, encourages q′ to be closer to its nearest point projection q on the sparse point cloud s , where q s is the neighborhood of query point q; For each point p in the local neighborhood Given a neighborhood size K, Represents the central coordinate set of the points in the neighborhood; in order to reduce the impact of inaccuracy in P, based on the statistical method Introducing multi-scale neighborhood Get the neighborhood q s The statistical expression of Among them, K s is the size of the multi-scale neighborhood, q k is the neighborhood q s points within, K represents q∈Q in P s nearest neighbor points; Collect point set Input to the neural network f θ And add a local loss function L surf (θ) is used to describe local details, so that the query point q can be better close to the underlying surface described by the original 3D point cloud.

7. The method for estimating oriented normals of a three-dimensional point cloud according to claim 1, characterized in that: The loss function in the mobile operation network training is: L=L v (θ)+L d (θ)+L surf (θ)+L con (θ)+L reg (θ)+L dis (i) The contents of the six loss functions are as follows: 1) L v :By minimizing the loss function To reduce the deviation from P to Q; where f i Q It is the set of expressions of the point q in the query point set Q output by the neural network function f; 2) L d :Introduce surface point constraint L d =‖q′-q s ‖, encourages q′ to be closer to its nearest point projection q on the sparse point cloud s , where q s is the neighborhood of query point q; 3) L surf :Introduce local constraint L surf =|f θ (q s )|, the neighborhood point set of the query point q Input to the global network f θ Among them, q s is the neighborhood of the query point q; 4) L con : The confidence-weighted cosine distance is introduced to evaluate the gradient consistency of the point set Q, and the expression is Where <·> represents the cosine distance of the vector, and ω is an adaptive weight; 5) L reg : Using regression loss L reg To calculate the predicted distance f of the point set Q i Q , 6) L dis : Introduced the moving distance loss function where d gt and d pred are the distances from the query point q and the moving point q′ to the bottom layer respectively.

8. A device for estimating oriented normals of a three-dimensional point cloud, characterized in that: include: Point cloud sampling module, used to obtain point cloud data and sample the query point set Q from the original point cloud P; A model building module is used to build a mobile operation network based on Neural-Pull, input a query point q in the query point set Q into the mobile operation network, and move the query point q to the bottom layer q′; A model training module, used for training the mobile operation network according to the point q′ after the mobile operation and the loss function, so as to adjust the parameters of the mobile operation network; The model testing module is used to estimate the oriented normal of the three-dimensional point cloud using the trained mobile operation network to obtain evaluation results.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

Images