Method for optimizing 3D Gaussian model construction based on threshold value and statistical filtering

Abstract

The invention discloses a method for constructing a 3D Gaussian model based on threshold and statistical filtering optimization, which comprises the following steps: firstly, carrying out batch background removal on a reconstructed picture, cutting the size of the picture into half of the original size, and carrying out 3DGS reconstruction by taking the image as input data; and removing black Gaussian ellipsoids and discrete points by adopting a threshold and statistical filtering method, finally optimizing a 3D Gaussian point cloud file format, removing useless parameters in the 3D Gaussian point cloud file, and exporting the optimized point cloud file. According to the method, the background of the picture is firstly removed, the sparse point cloud reconstruction speed is accelerated, only the Gaussian model of the target object is generated, in addition, the black Gaussian ellipsoid and discrete points are removed by adopting a threshold and statistical filtering method, and the accuracy and the stability of the model are improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to a method for constructing a 3D Gaussian model based on threshold and statistical filtering optimization. Background Art

[0002] 3DGS, or 3D Gaussian Splatting, is a 3D modeling technology whose main purpose is to achieve efficient and accurate representation of 3D scenes. This technology uses Gaussian functions to represent points in 3D space. Each point is described by a Gaussian function that defines the spatial position of the point and the degree of diffusion along each axis. In 3DGS, the probability of the existence of a point is determined by the value of the Gaussian function, which allows the fuzzy boundaries of the surface to be modeled instead of strict geometric boundaries. In addition, 3DGS can also represent 3D shapes at multiple scales, thereby capturing different levels of details from micro to macro. In terms of reconstruction effect, 3DGS is able to retain detailed information in 3D scenes, including geometric shapes, structural details, etc. At the application level, 3DGS has shown broad application potential in many fields, such as construction, autonomous driving, and e-commerce. In the construction industry, it can help architects and engineers create accurate building models; in autonomous driving, it can enhance the environmental perception of autonomous driving systems and improve safety and accuracy; in e-commerce, it can provide realistic product displays and enhance users' shopping experience.

[0003] However, 3DGS still has defects in single object reconstruction and cannot directly reconstruct the target object and maintain high quality. Specifically, 3DGS reconstructs the sparse point cloud of the object from SFM (Structure-from-Motion). In addition to reconstructing the target, the image contains a lot of environmental information, and the information for extracting features is large. The amount of data processed is large, resulting in very slow reconstruction speed. In addition, there are many black Gaussian noise points when reconstructing the 3D Gaussian model after removing the background. Improvements are needed to solve the above problems. Summary of the invention

[0004] To solve the above problems, the present invention discloses a method for constructing a 3D Gaussian model based on threshold and statistical filtering optimization, which removes the background of the picture, speeds up the reconstruction of sparse point cloud, and only generates a Gaussian model of the target object. In addition, a threshold and statistical filtering method is used to remove black Gaussian ellipsoids and discrete points, thereby improving the accuracy and stability of the model.

[0005] To achieve the above object, the technical solution of the present invention is as follows: A method for constructing a 3D Gaussian model based on threshold and statistical filtering optimization comprises the following steps: Step 1: Build a data set, perform batch background removal on the images to be reconstructed, filter the images obtained after processing, and remove images with incomplete background removal and images with incomplete target reconstruction objects; Step 2: Crop the image in step 1 to half its original size; Step 3: Use the image processed in step 2 as input data to perform 3DGS reconstruction; Step 4: According to the point cloud file output by 3DGS in step 3, extract the spherical harmonic primary color data in the point cloud file, set the RGB (an additive color model) channel threshold condition threshold, and when the RGB three colors are less than or equal to the threshold at the same time or the opacity is less than another threshold, the black Gaussian ellipsoid condition is met and removed. That is, remove the Gaussian ellipsoid close to black and almost transparent; Step 5: Design a statistical filter to remove discrete points; Step 6: Optimize the 3D Gaussian point cloud file format processed by steps 4 and 5, remove useless parameters (normals, high-order spherical harmonic coefficients), and export the optimized point cloud file.

[0006] Furthermore, in step 1 and step 2, the image background is removed and the size is adjusted, and the expressions are as follows: (1) (2) In formula (1), represents the original image, R ( ) function is used to remove the image background. Represents a picture with the background removed. N represents the number of input images; in formula (2), S ( ) is the judgment function value is 0 or 1, 0 means Remove, 1 means keep , size is the factor for adjusting the image size, take 2, Represents the resized image. M Indicates the total number of images after removal.

[0007] Furthermore, the 3DGS reconstruction described in step 3 optimizes the generation of a 3D Gaussian model. The algorithm first generates a sparse point cloud and adjusts the number of 3DGS iterations to 7000, so that the 3D Gaussian model is generated quickly and with high quality.

[0008] Furthermore, in step 4, a method for optimizing the 3D Gaussian model based on a threshold is proposed. According to a large number of experiments, when the color threshold is set to 0.001, the effect of removing the black Gaussian ellipsoid is good, and when the opacity is set to 0.05, the effect of removing the Gaussian ellipsoid with a transparency of almost 0 is good. The removal formula expression is as follows: (3) (4) (5) (6) Among them, formula (3) calculates RGB, where, is a constant with a value of 0.28209479177387814. represents the DC component of spherical harmonics; in formula (4), Indicates red, Indicates green, Indicates blue, represents the color threshold, and its value is set to 0.001; in formula (5), The value after activation is used to obtain the true transparency through the formula , in formula (6), Indicates the transparency threshold, the value is set to 0.05, It is a removal function. When the value in {} is 0, it is removed, and when it is 1, it is retained.

[0009] Furthermore, in step 5, the present invention designs a statistical filter: based on the average distance from each Gaussian ellipsoid to its k nearest neighboring points, a distance threshold is set to determine whether the average distance of each Gaussian ellipsoid is less than the threshold. If it is less, it is a discrete point and should be removed. The present invention uses the KNN algorithm based on the KD-Tree data structure to calculate the nearest k points of each point, thereby calculating the average distance of each point, and uses the Gaussian distribution to calculate the standard deviation of the sample, and uses it as the distance threshold to determine whether it is a discrete point. The discrete point threshold setting formula is as follows: (7) (8) (9) (10) (11) In formula (7), represents the distance from the i-th Gaussian ellipsoid to the j-th Gaussian ellipsoid, represents the center point of the i-th Gaussian ellipsoid, represents the center point of the jth Gaussian ellipsoid. In formula (8), Indicates i The average distance from a Gaussian ellipsoid to its k nearest neighbors, The function is used to calculate the k nearest neighbor points, k is 50, in formulas (9) and (10), represents the mean of the sample distances, Represents the distance threshold, M=2.5.

[0010] Furthermore, in step 6, the normals and high-order spherical harmonic coefficients are removed, and the resulting point cloud file has fewer points and parameters, reducing the pressure on file storage, and its file size is 4-5 times smaller than the original file.

[0011] The beneficial effects of the present invention are: 1. The present invention removes the background of the image, which not only speeds up the reconstruction of the sparse point cloud, but also generates only the Gaussian model of the target object. First, the object is separated from the background to obtain a background-removed image, which lays the foundation for the subsequent generation of a three-dimensional model of the target object; secondly, the size of the background-removed image is adjusted to half of the original image, which speeds up the reconstruction of the sparse point cloud without losing accuracy.

[0012] 2. The present invention uses a threshold-based and statistical filtering method to remove black Gaussian ellipsoids and discrete points, thereby improving the quality of the model. First, a threshold is set based on color and opacity to remove black and almost transparent Gaussian ellipsoids, thereby reducing the number of irrelevant Gaussian ellipsoids while ensuring a high model quality; secondly, a KNN algorithm based on KD-Tree is implemented to quickly calculate the K nearest neighbor points of each Gaussian ellipsoid, and a statistical filter is designed to remove discrete points based on the standard deviation of the average distance of the points, thereby ensuring a high model quality.

[0013] 3. The present invention generates a high-quality 3D Gaussian model with less noise, which is easier to convert to mesh, so that it can be applied to mainstream 3D modeling software and AR / VR devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a diagram of the 3D Gaussian model generation and optimization process described in the present invention.

[0015] Figure 2 A diagram of the steps for adjusting the 3D Gaussian model generation according to the present invention.

[0016] Figure 3 This is a diagram of the 3D Gaussian model optimization process based on threshold and statistical filtering described in the present invention. DETAILED DESCRIPTION

[0017] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0018] like Figure 1 As shown, the method for optimizing a 3D Gaussian model based on threshold and statistical filtering according to the present invention comprises the following steps: Step 1: Build a data set, perform batch background removal on the images to be reconstructed, filter the images obtained after processing, and remove images with incomplete background removal and images with incomplete target reconstruction objects; Step 2: Crop the image in step 1 to half its original size; Step 3: Use the image processed in step 2 as input data to perform 3DGS reconstruction; Step 4: According to the point cloud file output by 3DGS in step 3, extract the spherical harmonic base color data in the point cloud file, set the RGB channel threshold condition threshold, and when the RGB three colors are less than or equal to the threshold at the same time or the opacity is less than another threshold, the black Gaussian ellipsoid condition is met and removed. That is, remove the Gaussian ellipsoid close to black and almost transparent; Step 5: Design a statistical filter to remove discrete points; Step 6: Optimize the 3D Gaussian point cloud file format processed by steps 4 and 5, remove useless parameters (normals, high-order spherical harmonic coefficients), and export the optimized point cloud file.

[0019] In step 1 and step 2, the present invention removes the image background and adjusts the size, and the expression is as follows: (1) (2) In formula (1), represents the original image, R ( ) function is used to remove the image background. Represents a picture with the background removed. N represents the number of input images; in formula (2), S ( ) is the judgment function value is 0 or 1, 0 means Remove, 1 means keep , size is the factor for adjusting the image size, take 2, Represents the resized image. M Indicates the total number of images after removal.

[0020] In step 3, the step of generating a 3D Gaussian model is optimized, such as Figure 2The algorithm first generates a sparse point cloud and adjusts the number of 3DGS iterations to 7000, which generates a 3D Gaussian model quickly and with high quality.

[0021] In step 4, a method for optimizing the 3D Gaussian model based on a threshold is proposed. According to a large number of experiments, when the color threshold is set to 0.001, the effect of removing the black Gaussian ellipsoid is good, and when the opacity is set to 0.05, the effect of removing the Gaussian ellipsoid with a transparency of almost 0 is good. The specific process is as follows Figure 3 As shown, the removal formula expression is as follows: (3) (4) (5) (6) Among them, formula (3) calculates RGB, where, is a constant with a value of 0.28209479177387814. represents the DC component of spherical harmonics; in formula (4), Indicates red, Indicates green, Indicates blue, represents the color threshold, and its value is set to 0.001; in formula (5), The value after activation is used to obtain the true transparency through the formula , in formula (6), Indicates the transparency threshold, the value is set to 0.05, It is a removal function. When the value in {} is 0, it is removed, and when it is 1, it is retained.

[0022] In step 5, the present invention designs a statistical filter: based on the average distance from each Gaussian ellipsoid to its k nearest neighboring points, a distance threshold is set to determine whether the average distance of each Gaussian ellipsoid is less than the threshold. If it is less, it is a discrete point and should be removed. The present invention uses the KNN algorithm based on the KD-Tree data structure to calculate the nearest k points of each point, thereby calculating the average distance of each point, and uses the Gaussian distribution to calculate the standard deviation of the sample, and uses it as the distance threshold to determine whether it is a discrete point. The specific process is as follows Figure 3 As shown, the discrete point threshold setting formula is as follows: (7) (8) (9) (10) (11) In formula (7), represents the distance from the i-th Gaussian ellipsoid to the j-th Gaussian ellipsoid, represents the center point of the i-th Gaussian ellipsoid, represents the center point of the jth Gaussian ellipsoid. In formula (8), Indicates i The average distance from a Gaussian ellipsoid to its k nearest neighbors, The function is used to calculate the k nearest neighbor points, k is 50, in formulas (9) and (10), represents the mean of the sample distances, Represents the distance threshold, M=2.5.

[0023] In step 6, the present invention removes useless parameters, mainly normals and high-order spherical harmonic coefficients, and the resulting point cloud file has fewer points and parameters, reducing the pressure on file storage, and its file size is 4.5 times smaller than the original file.

[0024] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. A method for constructing a 3D Gaussian model based on threshold and statistical filtering optimization, characterized in that: The following steps are involved: Step 1: Build a data set, perform batch background removal on the images to be reconstructed, filter the images obtained after processing, and remove images with incomplete background removal and images with incomplete target reconstruction objects; Step 2: Crop the image in step 1 to half its original size; Step 3: Use the image processed in step 2 as input data to perform 3DGS reconstruction; Step 4: According to the point cloud file output by 3DGS in step 3, extract the spherical harmonic primary color data in the point cloud file, set the RGB channel threshold condition threshold, and when the RGB three colors are less than or equal to the threshold at the same time or the opacity is less than another threshold, the black Gaussian ellipsoid condition is met and removed; that is, remove the Gaussian ellipsoid close to black and almost transparent; Step 5: Design a statistical filter to remove discrete points; Step 6: Optimize the 3D Gaussian point cloud file format processed by steps 4 and 5, remove useless parameters, and export the optimized point cloud file.

2. The method for constructing a 3D Gaussian model based on threshold and statistical filtering optimization according to claim 1, characterized in that: In step 1 and step 2, the image background is removed and the size is adjusted, and the expressions are as follows: (1) (2) In formula (1), represents the original image, R ( ) function is used to remove the image background. Represents a picture with the background removed. N represents the number of input images; in formula (2), S ( ) is the judgment function value is 0 or 1, 0 means Remove, 1 means keep , size is the factor for adjusting the image size, take 2, Represents the resized image. M Indicates the total number of images after removal.

3. The method for constructing a 3D Gaussian model based on threshold and statistical filtering optimization according to claim 1, characterized in that: The 3DGS reconstruction described in step 3 optimizes the generation of a 3D Gaussian model. The algorithm first generates a sparse point cloud and adjusts the number of 3DGS iterations to 7000. The 3D Gaussian model is generated quickly and with high quality.

4. The method for constructing a 3D Gaussian model based on threshold and statistical filtering optimization according to claim 1, characterized in that: In step 4, a method for optimizing a 3D Gaussian model based on a threshold is proposed. Based on the following principles: when the color threshold is set to 0.001, the effect of removing the black Gaussian ellipsoid is good, and when the opacity is set to 0.05, the effect of removing the Gaussian ellipsoid with almost 0 transparency is good; The removal formula expression is as follows: (3) (4) (5) (6) Among them, formula (3) calculates RGB, where, is a constant with a value of 0.28209479177387814. represents the DC component of spherical harmonics; in formula (4), Indicates red, Indicates green, Indicates blue, represents the color threshold, and its value is set to 0.001; in formula (5), The value after activation is used to obtain the true transparency through the formula , in formula (6), Indicates the transparency threshold, the value is set to 0.05, It is a removal function. When the value in {} is 0, it is removed, and when it is 1, it is retained.

5. The method for constructing a 3D Gaussian model based on threshold and statistical filtering optimization according to claim 1, characterized in that: In step 5, a statistical filter is designed: based on the average distance from each Gaussian ellipsoid to its k nearest neighboring points, a distance threshold is set to determine whether the average distance of each Gaussian ellipsoid is less than the threshold. If it is small, it is a discrete point and should be removed; The KNN algorithm based on the KD-Tree data structure is used to calculate the nearest k points of each point, thereby calculating the average distance of each point. The standard deviation of the sample is calculated using Gaussian distribution and used as the distance threshold to determine whether it is a discrete point. The discrete point threshold setting formula is as follows: (7) (8) (9) (10) (11) In formula (7), represents the distance from the i-th Gaussian ellipsoid to the j-th Gaussian ellipsoid, represents the center point of the i-th Gaussian ellipsoid, represents the center point of the jth Gaussian ellipsoid; in formula (8), Indicates i The average distance from a Gaussian ellipsoid to its k nearest neighbors, The function is used to calculate the k nearest neighbor points, k is 50, in formulas (9) and (10), represents the mean of the sample distances, Represents the distance threshold, M=2.

5.

6. The method for constructing a 3D Gaussian model based on threshold and statistical filtering optimization according to claim 1, characterized in that: In step 6, the normals and high-order spherical harmonic coefficients are removed, and the resulting point cloud file has fewer points and parameters, reducing the pressure on file storage. The file size is 4-5 times smaller than the original file.

Images