A Mesh Filtering Method Based on Joint Perception of Noise and Geometric Characteristics

Abstract

The present invention discloses a mesh filtering method for joint perception of noise and geometric characteristics, the method comprising the following steps: step 1, designing a filter coefficient learner; step 2, calculating an average filter angle; step 3, estimating noise intensity; step 4, extracting geometric features; step 5, generating coefficients and updating vertices. The present invention cleverly generates coefficients to filter surface normals, dynamically learns coefficients based on noise intensity and geometric features, it does not rely on any specific assumptions, and achieves improved performance and more robust denoising results; designs a noise intensity module, derives the noise perception component of the filter coefficient by predicting the angle between pairs of noise clean normals; develops a geometric description module to capture comprehensive geometric features to generate the geometric perception component of the filter coefficient.

Description

Technical Field

[0001] The invention belongs to the technical field of Mesh denoising, and relates to a mesh filtering method, and in particular to a method for performing mesh denoising by using a filtering coefficient that takes both noise and geometric features into consideration. Background Art

[0002] In recent years, it has become increasingly easier to obtain meshes from real-world objects due to advances in 3D scanning equipment and reconstruction algorithms. However, even with advanced techniques, meshes obtained from the real world are inevitably contaminated by noise, which can lead to inefficiency or even failure of downstream geometric tasks. Therefore, mesh denoising has become a fundamental research topic in geometry processing. The goal of mesh denoising is to smooth the noisy surface while maintaining the underlying geometric features.

[0003] Filter-based methods are widely used for feature-preserving mesh denoising due to their effectiveness and simplicity. However, a common limitation is that the coefficients are derived based on assumptions that are difficult to generalize across various meshes and noises. Recently, deep learning-based methods have been proposed to predict noise-free surface normals for mesh denoising. Due to the irregularity of meshes, general convolutional networks are not directly applicable to meshes. Previous works have addressed this issue through clever mesh representations. However, although deep learning-based methods have achieved remarkable performance without relying on specific assumptions, none of these networks have explicitly designed noise and geometry, which are essential factors for mesh denoising. If the coefficients can be learned dynamically based on the noise intensity and geometric features, so that they do not rely on any specific assumptions; and if the network is designed by comprehensively considering the noise intensity and geometric features, better denoising results can be achieved. Summary of the invention

[0004] In order to improve the denoising performance on mesh data, the present invention provides a Mesh filtering method that combines perceived noise and geometric characteristics.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A mesh filtering method combining perception of noise and geometric characteristics comprises the following steps:

[0007] Step 1: Design a filtering coefficient learner (FCL):

[0008] The FCL includes a noise intensity module M n and geometric description module M g ,in:

[0009] The noise intensity module M nResponsible for generating the noise-aware filter coefficient W i n , geometric description module M g Responsible for generating the geometric perception filter coefficient W i g ;

[0010] The noise intensity module M n It consists of a three-layer graph convolutional network (GCN) and a multi-layer perceptron (MLP). GCN captures the neighborhood P i The features of each face in the neighborhood P are extracted by maximum pooling and average pooling. i The global features in the image are used to regress the noise-perceived filter coefficient W using MLP. i n ;

[0011] The geometric description module M g It includes a category feature extraction branch and a geometric feature extraction branch. The category feature extraction branch includes a three-layer graph convolutional network (GCN) and two fully connected layers. The GCN captures the neighborhood P i The features of each face in the neighborhood P are extracted by maximum pooling and average pooling. i The global features in the image are extracted through two fully connected layers to extract the category features of the center face; the geometric feature extraction branch includes a three-layer graph convolutional network (GCN), a multi-layer perceptron and a fully connected layer, and the GCN captures the neighborhood P i The features of each face in the center are concatenated with the category features of the center face and then fused together using a multi-layer perceptron to generate a fused feature map. The first row of the fused feature map is the center feature corresponding to the center face. i The fusion features of each face in the center are multiplied by the central feature to obtain the similarity between each face and the central face, and then the geometric perception filter coefficient W is obtained through the fully connected layer. i g ;

[0012] Step 2: Calculate the average filter angle

[0013] Perform a simple mean filter operation on the grid M to obtain a new grid M M , the new grid M M The face normal of is the filter normal; for each face f i , is its normal n i and the angle between the filtered normal;

[0014] Step 3: Noise intensity estimation:

[0015] Neighborhood P iNormals of all faces in i , centroid information C i and the average filter angle As the noise intensity module M n Input, get the neighborhood P i The noise perception filter coefficient W of all faces in i n ;

[0016] Step 4: Geometric feature extraction:

[0017] Neighborhood P i Normals of all faces in i and centroid information C i As a geometric description module M g Input, get the neighborhood P i The geometric perception filter coefficient W of all faces in i g ;

[0018] Step 5: Coefficient generation and vertex update:

[0019] Combined with the noise intensity module M n and geometric description module M g The noise perception filter coefficient W is obtained i n and geometric perception filter coefficient W i g To generate the final filter coefficient W i , and denoising is accomplished by updating the vertex positions by considering the contribution of neighboring faces.

[0020] Compared with the prior art, the present invention has the following advantages:

[0021] 1. The present invention cleverly generates coefficients to filter surface normals, and dynamically learns the coefficients based on noise intensity and geometric features. It does not rely on any specific assumptions, achieving improved performance and more robust denoising results.

[0022] 2. The present invention designs a noise intensity module to derive the noise perception component of the filter coefficient by predicting the angle between pairs of noise clean normals.

[0023] 3. The present invention develops a geometric description module to capture comprehensive geometric features to generate geometrically perceptual components of filter coefficients. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is the structure of FCL;

[0025] Figure 2 Results on the Kinect series datasets. DETAILED DESCRIPTION

[0026] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0027] The present invention provides a mesh filtering method combining perception noise and geometric characteristics, and the specific steps are as follows:

[0028] Step 1: Design FCL:

[0029] By jointly considering the noise intensity and geometric characteristics, the FCL is designed. The structure of FCL is as follows Figure 1 As shown, the noise intensity module M n and geometric description module M g Composition, of which:

[0030] The noise intensity module M n Responsible for generating the noise-aware filter coefficient W i n , geometric description module M g Responsible for generating the geometric perception filter coefficient W i g , the two perceptual filter coefficients are combined to obtain the complete filter coefficient W i , use this filter coefficient to filter the neighborhood P i Filter the normals of the faces in to get the denoised normals;

[0031] The noise intensity module M n It consists of a three-layer graph convolutional network (GCN) and a multi-layer perceptron (MLP). GCN captures the neighborhood P i The features of each face in the neighborhood P are extracted by maximum pooling and average pooling. i The global features in the image are used to regress the noise-perceived filter coefficient W using MLP. i n ;

[0032] The geometric description module M g It includes a category feature extraction branch and a geometric feature extraction branch. The category feature extraction branch includes a three-layer graph convolutional network (GCN) and two fully connected layers ([128, 128, 128]). GCN captures the neighborhood P i The features of each face in the neighborhood P are extracted by maximum pooling and average pooling. iThe global features in the image are extracted through two fully connected layers to extract the category features of the center face; the geometric feature extraction branch includes a three-layer graph convolutional network (GCN), a multi-layer perceptron [512,512] and a fully connected layer ([128,128,128]), and GCN is used to capture the neighborhood P i The features of each face in the center are concatenated with the category features of the center face and then fused together using a multi-layer perceptron to generate a fused feature map. The first row of the fused feature map is the center feature corresponding to the center face. i The fusion features of each face in the center are multiplied by the central feature to obtain the similarity between each face and the central face, and then the geometric perception filter coefficient W is obtained through the fully connected layer. i g .

[0033] Step 2: Calculate the average filter angle

[0034] Perform a simple mean filter operation on the grid M to obtain a new grid M M , the new grid M M The face normal of is the filter normal; for each face f i , is its normal n i The angle between the filter normal and the filter normal.

[0035] Step 3: Noise intensity estimation:

[0036] Neighborhood P i Normals of all faces in i , centroid information C i and the average filter angle As the noise intensity module M n Input, get the neighborhood P i The noise perception filter coefficient W of all faces in i n .

[0037] Step 4: Geometric feature extraction:

[0038] Neighborhood P i Normals of all faces in i and centroid information C i As a geometric description module M g Input, get the neighborhood P i The geometric perception filter coefficient W of all faces in i g

[0039] Step 5: Coefficient generation and vertex update:

[0040] Combined with the noise intensity module Mn and geometric description module M g The noise perception filter coefficient W is obtained i n and geometric perception filter coefficient W i g To generate the final filter coefficient W i , and denoising is accomplished by updating the vertex positions by considering the contribution of neighboring faces.

[0041] Embodiment 1:

[0042] Step 1: Input data preparation:

[0043] Divide the grid M into r-ring patches / r-ring neighborhood blocks, denoted as P i For the surface i , its patch / neighborhood P i is initialized to {f i}, and by iteratively adding all i The faces that share at least one vertex are generated r times. In this embodiment, r=3. In order to remove unnecessary degrees of freedom from the input patch, P i Translate to the origin, scale it to the unit bounding box, and rotate it to P i The average normal of is in the direction of [0,0,1].

[0044] Step 2: Use FCL to process each face:

[0045] Use the noise intensity module M in FCL n and geometric description module M g Evaluate the intensity of the noise and capture the comprehensive geometric features to generate filtering coefficients and optimize the filtering process. The structure of FCL is as follows Figure 1 As shown, the specific steps are as follows:

[0046] Step 21: Filter noise and estimate filter normals. Perform a simple mean filter operation on the mesh M to obtain a new mesh M. M , the new grid M M The face normal of M is the filter normal. M is generated by iterative normal filtering and vertex updating, and is obtained by iterating 20 times in this embodiment. i , is its normal n i The angle between the filter normal and the filter normal. i f is included with f i All faces that share an edge, combined with P i fAfter k iterations, the filtered normal is calculated. The average filter angle is calculated by combining the filtered normal and the normal.

[0047] Step 2: Generate noise perception components and calculate noise perception filter coefficients. i Normals of all faces in i , centroid information C i and the average filter angle As the noise intensity module M n Input, get the neighborhood P i The noise perception filter coefficient W of all faces in i n .

[0048] Step 2: Design a category feature extraction branch to extract the global description of the mesh. i , according to the maximum angle difference in the 2-ring patch / 2 ring patch, it is divided into four categories, the maximum angle difference is called A, and the four categories are:

[0049] Type 1: 0°<A≤20°, smooth region

[0050] Type 2: 20°<A≤50°, curved region

[0051] Type 3: 50°<A≤80°, small edge region

[0052] Type 4: 80°<A≤180°, large edge region

[0053] Step 24: Generate geometric perception filter coefficients. i Normals of all faces in i and centroid information C i As a geometric description module M g Input, get the neighborhood P i The geometric perception filter coefficient W of all faces in i g .

[0054] Step 3: Vertex update:

[0055] Step 3.1: Combine the noise perception filter coefficient W in step 2 i n and geometric perception filter coefficient W i g Get the final filter coefficient W i , which helps preserve the original geometric features during vertex updating while reducing the effects of noise.

[0056] Step 32: Filter the neighborhood P i The normal in is used to generate the normal n of the center surface after denoising i ′.

[0057] Step 3. For each vertex v in the Mesh mesh k , update the vertex v by calculating the contribution of its neighboring faces k The coordinates of v′ are obtained k . Then the sum of the contributions is calculated by v k The number of faces in the neighborhood is averaged and the position is updated to achieve the denoising function. The contribution is composed of the denoised normal n′ of each face in the neighborhood j With vector (c j -v k ), c j Represents the denoised normal n′ j The corresponding face centroid.

[0058] In Example 1, FCL is evaluated on a synthetic dataset (SynData) and a real scan dataset (Kinect series dataset and meshes on the Internet). The synthetic dataset includes 14 training meshes and 10 test meshes, which are divided into simple geometry meshes and complex object meshes. The embodiment is qualitatively and quantitatively compared with the most advanced mesh denoising methods, including bilateral mesh filtering (BMF), bilateral normal filtering (BNF), guided normal filtering (GNF), mesh total generalized variation (TGV) and GCN denoiser (GCN). To ensure a fair comparison, the embodiment carefully selects the best results obtained using BMF, BNF, GNF and TGV fine-tuning parameters as competitors of FCL. For GCN, we train it using the same training data as FCL. The average normal angle difference E is used. a and the normalized average Hausdorff distance E v The two data are used as indicators for evaluation, and the smaller the indicator, the better the performance. Figure 2 As shown. Figure 2 It can be seen that the method of the present invention is superior to the most advanced methods in noise removal and feature preservation.

[0059] Embodiment 2:

[0060] This embodiment is different from Embodiment 1 in that the denoised normal is directly output instead of the noise-aware filter coefficient and the geometry-aware filter coefficient. This variant is implemented by adding two fully connected layers at the end of the FCL, namely: the FCL also includes two fully connected layers, the noise-aware filter coefficient W i n and geometric description module M gAfter passing through two fully connected layers, the denoised normal is obtained. This example aims to verify the superiority of the filtering mechanism.

[0061] Embodiment 3:

[0062] The difference between this embodiment and embodiment 1 is that: the noise intensity module M is removed n This embodiment aims to study the noise intensity module M n The importance of.

[0063] Embodiment 4:

[0064] The difference between this embodiment and embodiment 1 is that: the category feature extraction branch is deleted, which means that the geometric description module M g No category features are used in this embodiment. This embodiment aims to study the importance of category features.

[0065] The comparative experimental results of Examples 1-4 are shown in Table 1:

[0066] Table 1

[0067]

Claims

1. A mesh filtering method that combines perception of noise and geometric characteristics, characterized in that The method inputs data obtained by 3D scanning, and includes the following steps: Step 1: Design the filter coefficient learner FCL: The FCL includes a noise intensity module M n and geometric description module M g ,in: The noise intensity module M n Responsible for generating the noise-aware filter coefficient W i n , geometric description module M g Responsible for generating the geometric perception filter coefficient W i g ; The noise intensity module M n It includes a three-layer graph convolutional network GCN and a multi-layer perceptron MLP. GCN captures the neighborhood P i The features of each face in the neighborhood P are extracted by maximum pooling and average pooling. i The global features in the image are used to regress the noise-perceived filter coefficient W using MLP. i n ; The geometric description module M g It includes a category feature extraction branch and a geometric feature extraction branch. The category feature extraction branch includes a three-layer graph convolutional network GCN and two fully connected layers. GCN captures the neighborhood P i The features of each face in the neighborhood P are extracted by maximum pooling and average pooling. i The global features in the image are extracted through two fully connected layers to extract the category features of the center face; the geometric feature extraction branch includes a three-layer graph convolutional network GCN, a multi-layer perceptron and a fully connected layer, and GCN captures the neighborhood P i The features of each face in the center are concatenated with the category features of the center face and then fused together using a multi-layer perceptron to generate a fused feature map. The first row of the fused feature map is the center feature corresponding to the center face. i The fusion features of each face in the center are multiplied by the central feature to obtain the similarity between each face and the central face, and then the geometric perception filter coefficient W is obtained through the fully connected layer. i g ; Step 2: Calculate the average filter angle Perform a simple mean filter operation on the grid M to obtain a new grid M M , the new grid M M The face normal of is the filter normal; for each face f i , is its normal n i and the angle between the filtered normal; Step 3: Noise intensity estimation: Neighborhood P i Normals of all faces in i , centroid information C i and the average filter angle As the noise intensity module M n Input, get the neighborhood P i The noise perception filter coefficient W of all faces in i n ; Step 4: Geometric feature extraction: Neighborhood P i Normals of all faces in i and centroid information C i As a geometric description module M g Input, get the neighborhood P i The geometric perception filter coefficient W of all faces in i g ; Step 5: Coefficient generation and vertex update: Combined with the noise intensity module M n and geometric description module M g The noise perception filter coefficient W is obtained i n and geometric perception filter coefficient W i g To generate the final filter coefficient W i , and denoising is accomplished by updating the vertex positions by considering the contribution of neighboring faces.

2. The mesh filtering method for joint perception of noise and geometric characteristics according to claim 1 is characterized in that The average filter angle The calculation method is as follows: Perform a simple mean filter operation on the grid M to obtain a new grid M M , the new grid M M The face normal of is the filter normal; for each face f i , is its normal n i and the angle between the filtered normal, P i f is included with f i All faces that share an edge, combined with P i f After k iterations, the filtered normal is calculated, and the average filter angle is calculated by combining the filtered normal and the normal.

3. The mesh filtering method for joint perception of noise and geometric characteristics according to claim 1 is characterized in that In the category feature extraction branch, for each face f i , according to the maximum angle difference A in the 2-ring patch / 2 ring patch, it can be divided into four categories. These four categories are: Type 1: 0°<A≤20°, smoothregion; Type 2: 20°<A≤50°, curved region; Type 3: 50°<A≤80°, small edge region; Type 4: 80°<A≤180°, large edge region.

4. The mesh filtering method for joint perception of noise and geometric characteristics according to claim 1 is characterized in that The specific method of vertex updating is as follows: For each vertex v in the Mesh k , update the vertex v by calculating the contribution of its neighboring faces k The coordinates of v′ are obtained k , and then sum the contributions by v k The number of domain faces is averaged and the position is updated to achieve the denoising function.

5. The mesh filtering method for joint perception of noise and geometric characteristics according to claim 4 is characterized in that The contribution is composed of the denoised normal n′ of each face in the neighborhood j With vector (c j -v k ), c j Represents the denoised normal n′ j The corresponding face centroid.

6. The mesh filtering method for joint perception of noise and geometric characteristics according to claim 1 is characterized in that The FCL also includes two fully connected layers, the noise-aware filter coefficient W i n and geometric description module M g The denoised normal is obtained after passing through two fully connected layers.

Images