A 3D scene arbitrary angle model segmentation method based on 3D Gaussian Splatting

Abstract

The invention provides a 3D scene arbitrary angle model segmentation method based on 3D Gaussian Splatting, which belongs to the field of machine perception technology and includes point cloud data extraction, 3D model training, model category feature extraction, model category feature training, model segmentation, and noise elimination; the system uses Colmap to reconstruct dense 3D point cloud information and camera trajectory from an image sequence; uses a 3DGS method to perform 3D reconstruction on the point cloud information; uses a DEVA model to perform image segmentation and image category preprocessing; trains a multilayer perceptron composed of multiple fully connected layers as a discriminator for segmenting the 3D model, which greatly reduces the discriminator training overhead; uses a KNN algorithm to calculate the average distance between each Gaussian kernel and several adjacent Gaussian kernels to judge the noise, so as to solve the detail problem that occurs during model segmentation, and can also solve the object boundary roughness problem; finally, the 3D model arbitrary angle segmentation method based on 3DGS is realized.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision technology, and in particular to the technical fields of computer vision, computer graphics, and three-dimensional reconstruction, and specifically to a method for segmenting a three-dimensional scene arbitrary angle model based on 3D Gaussian Splatting. Background Art

[0002] In the field of 3D reconstruction, model segmentation can extract more important 3D models from complex 3D scenes, helping to understand and reconstruct 3D scenes. How to accurately extract specified 3D models from complex scenes has also been a hot topic in the field of 3D reconstruction this year.

[0003] The recently emerged 3DGS technology has made significant progress compared to previous 3D representation methods such as point cloud, mesh and Nerf, especially in terms of training time and scene reconstruction quality. Based on the recent 3D reconstruction technology, many researchers have proposed novel model segmentation methods, but they all have certain limitations. For example, the method of using convolutional neural network as discriminator and using 3D regularization loss and cross entropy loss as loss function requires a large amount of video memory during the training process, and non-professional graphics cards cannot train the model; some segmentation methods are very complex to implement, and it is difficult to segment multiple objects at the same time. It will take up more overhead during the storage process, and it is difficult to map 2D semantic features to 3D space. Summary of the invention

[0004] In order to solve the problem of accurately extracting designated model information from a complex three-dimensional scene, the present invention provides a three-dimensional scene arbitrary angle model segmentation method based on 3D Gaussian Splatting. The present invention performs three-dimensional reconstruction on a series of pictures based on 3DGS to obtain a reconstructed three-dimensional scene; the model is segmented and the model category is assigned through the DEVA model, and 32-bit Gaussian category features and a multi-layer perceptron are trained for model segmentation; the segmentation object is determined in combination with the SAM prompt feature to realize an interactive model segmentation method; and finally the designated three-dimensional model is extracted from the reconstructed three-dimensional scene.

[0005] The technical solution adopted by the 3D scene arbitrary angle model segmentation method based on 3D Gaussian Splatting of the present invention is:

[0006] A 3D scene arbitrary angle model segmentation method based on 3D Gaussian Splatting, comprising the following steps:

[0007] S1, using the Colmap open source software package to reconstruct dense 3D point cloud data and camera position information from the image sequence;

[0008] S2, using the generated 3D point cloud data and camera position information, using 3DGS to train a series of Gaussian kernels from the 3D point cloud data to represent the real 3D scene;

[0009] S3. Use the DEVA model to segment the original image sequence and generate a grayscale image of each image. Use different grayscale values ​​to represent different objects in the image. The grayscale of the same object in different images is the same. Use the segmented image grayscale to represent the category of the object in the scene.

[0010] S4. On the trained Gaussian kernel, use the grayscale image to train a 32-bit category feature; at the same time, use the neural network to train a discriminator, the input of the discriminator is the 32-bit category feature, and the output value range is 0-255, indicating the object category;

[0011] S5. Use the trained discriminator to input the 32-bit feature of each Gaussian kernel to obtain the category of each Gaussian kernel; extract the Gaussian kernel of the specified category according to the category code or directly select the object from the image at any angle;

[0012] S6. Use the KNN algorithm to remove outlier Gaussian kernels from the extracted Gaussian kernels to reduce the noise of the generated image.

[0013] Furthermore, each of the Gaussian kernels contains position μ, covariance matrix ∑, transparency α and spherical harmonic function information SH, and the color information is represented by spherical harmonic functions; under a specific camera posture, 3DGS can project the 3D Gaussian kernel to the 2D imaging plane, and then superimpose the colors of the N ordered Gaussian kernels mapped to the pixel by the α blending method, thereby calculating the color of the pixel. The calculation formula is:

[0014]

[0015] Where c represents the calculated pixel color, c i Represents the color information carried by each Gaussian kernel, α i Indicates the transparency of each Gaussian kernel.

[0016] Further, step S3 includes: for the input image I, the corresponding 2D segmentation mask M can be expressed as:

[0017] M p =DEVA(I) (1.2)

[0018] Among them, p represents all pixels of the image, M p Represents the category code for each pixel in the image.

[0019] Further, the step S4 includes the following steps:

[0020] S4.1. An additional category feature code is trained on the trained Gaussian kernel. The category feature code is a tensor with a length of 32 bits. This category feature code can effectively distinguish different objects in the scene and has high computational efficiency.

[0021] S4.2, map the Gaussian kernel category feature encoding to the image, and then calculate the loss of the image prediction category and the true category;

[0022] S4.3. Train a multi-layer perceptron as a discriminator to restore the 32-bit feature encoding to the original 256-bit information.

[0023] Furthermore, step S4.2 includes: using the 3D Gaussian renderer in 3DGS, the category feature encoding is mapped to each pixel of the image in a differentiable manner, combined with the α hybrid rendering method in Gaussian, the influence weight of each Gaussian kernel on the image pixel is calculated, and then the N ordered Gaussian kernel feature encodings mapped to the pixel are accumulated to obtain the feature encoding of each pixel. Similar to the calculation process of color features, the calculation method of category feature encoding is:

[0024]

[0025] Among them, F represents the 32-bit category feature code finally rendered on each pixel of the image, and f i represents the category feature encoding on each Gaussian kernel, α i Indicates the transparency of each Gaussian kernel.

[0026] Furthermore, the step S4.3 includes: assuming that the number of pixels of image I is P, I p Represents each pixel of the image, h(I p ) represents the 256-bit feature obtained by mapping the 32-bit feature by the discriminator, g(h(I p )) means that the 256-bit feature is converted into a probability distribution through the softmax function, so the category of each pixel in the image can be expressed as:

[0027]

[0028] Among them, g(h(I p )) c represents the probability of the cth category in the probability distribution, Represents the predicted category of each pixel in the image. After obtaining the predicted category, the cross entropy loss between the real category and the predicted category can be calculated using the real image category for back propagation. The calculation method is:

[0029]

[0030] Among them, L i Represents the cross entropy loss of the i-th image in the classification task.

[0031] Furthermore, when the step S5 uses the method of directly selecting an object from an image at any angle, since the three-dimensional scene is displayed on the screen in the form of an image, the SAM model can be used to segment the image I displayed on the screen first, and the click position is used as the input prompt β to obtain the mask of the clicked object in the two-dimensional space, which can be expressed as:

[0032] M=SAM(I,β) (1.6)

[0033] Among them, M represents the segmentation mask obtained by combining the input prompt;

[0034] Since the displayed image is obtained by Gaussian kernel mapping using the 3DGS model, the 32-bit category feature encoding trained on the Gaussian kernel can be stored on each pixel through mapping. Using the previously trained discriminator and the segmentation mask, the category of each segmented pixel can be obtained.

[0035] F p =h(I(M)) (1.7)

[0036] Among them, F p Represents the category set of each pixel in the segmented image, I(M) represents the mask part of the displayed image, and the category of the segmented object is determined by counting the frequency of each category appearing in the mask part, thereby determining the category code of the current segmented object. The calculation method of the category code can be expressed as:

[0037]

[0038] Among them, δ(x, y) is the Kronecker delta function, which returns 1 when x=y, otherwise it returns 0. Then, the Gaussian kernel belonging to the category is found according to the category number to achieve three-dimensional model segmentation.

[0039] Further, the step S6 includes: selecting a suitable K value, calculating the average distance between each segmented Gaussian kernel and the K Gaussian kernels closest to it, and selecting a suitable K value and distance measurement method to effectively identify and delete noise points that do not belong to the segmentation model;

[0040] Assume that the number of Gaussian kernels in the scene is N, and use P i represents the coordinates of the i-th Gaussian kernel, K represents the number of nearest neighbor Gaussian kernels, and D represents the average distance threshold; first, calculate the average distance D of the nearest K neighbor points of each point i :

[0041]

[0042] Among them, d ij represents the i-th Gaussian kernel P i The distance between the jth nearest neighbor point; Next, the noise points are deleted by the distance threshold D. The new Gaussian kernel obtained after deleting the early points can be expressed as:

[0043] {P i |D i ≥D}(1.10).

[0044] The beneficial effects of the present invention are:

[0045] 1. In the 3D scene arbitrary angle model segmentation method based on 3D Gaussian Splatting, a new Gaussian kernel category discriminator structure is proposed. A 32-bit category feature is trained on the trained Gaussian kernel. The trained discriminator can discriminate the category code of the Gaussian kernel through the 32-bit category feature. The discriminator network structure proposed by the present invention can greatly reduce memory overhead and can run successfully with only 8GB of video memory.

[0046] 2. The 3D scene arbitrary angle model segmentation method based on 3D Gaussian Splatting not only supports the use of category coding to segment the 3D model, but also supports the segmentation of the 3D model from any angle, thereby improving the analysis and extraction capabilities of complex scenes and improving the accuracy and robustness of model segmentation.

[0047] 3. The 3D scene arbitrary angle model segmentation method based on 3D Gaussian Splatting can effectively solve the noise problem caused by the high complexity of the scene and the fuzzy model boundary problem, and improve the quality and accuracy of the segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of a method for segmenting a three-dimensional scene arbitrary angle model based on 3D Gaussian Splatting provided by an embodiment of the present invention;

[0049] Figure 2 It is a network structure diagram of a category discriminator trained in a three-dimensional scene arbitrary angle model segmentation method based on 3D Gaussian Splatting provided in an embodiment of the present invention;

[0050] Figure 3 It is a piece of data sampled from an original data set in a three-dimensional scene arbitrary angle model segmentation method based on 3D Gaussian Splatting provided in an embodiment of the present invention;

[0051] Figure 4 yes Figure 3 The category feature grayscale image obtained after processing the original data set;

[0052] Figure 5 It is a schematic diagram of a three-dimensional reconstruction result, a model segmentation result at different viewing angles and a denoising result in a three-dimensional scene arbitrary angle model segmentation method based on 3D Gaussian Splatting provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The system of the present invention includes point cloud file generation, three-dimensional scene training, model category preprocessing, training category features and discriminators, model segmentation, noise removal, etc.

[0054] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below in combination with specific implementation methods and with reference to the accompanying drawings. In the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0055] like Figure 1 As shown, the present invention specifically includes:

[0056] 1. Point cloud file generation:

[0057] The Colmap open source software package can be used to reconstruct dense 3D point cloud data and camera position information based on the input image sequence. This information will serve as the basic training data for the 3DGS model. Through training, the model can learn more accurate and robust feature representations, thereby improving the quality of the model and the training effect.

[0058] 2. Training 3D scenes:

[0059] 3DGS is an emerging method for real-time radiation field rendering, with high-precision rendering quality and real-time rendering speed. 3DGS can train a large number of Gaussian kernels to represent three-dimensional scene information. Each 3D Gaussian kernel contains position μ, covariance matrix ∑, transparency α and spherical harmonic function information SH, and color information is represented by spherical harmonic functions. Under a specific camera pose, 3DGS can project the 3D Gaussian kernel to the 2D imaging plane, and then superimpose the colors of the N ordered Gaussian kernels mapped to the pixel through the α blending method to calculate the color of the pixel. The calculation formula is:

[0060]

[0061] Where c represents the calculated pixel color, c i Represents the color information carried by each Gaussian kernel, α i Indicates the transparency of each Gaussian kernel.

[0062] 3. Model category preprocessing

[0063] The DEVA model is used for image segmentation. The DEVA model can segment the same objects in a continuous image sequence. The grayscale image mask predicted by the DEVA model represents the categories of different objects in the scene. In different images, the grayscale value of the same object is the same. For the input image I, the corresponding 2D segmentation mask M can be expressed as:

[0064] M p =DEVA(I) (1.2)

[0065] Among them, p represents all pixels of the image, M p Represents the category code of each pixel in the image. The specific results are as follows Figure 3 , Figure 4 shown.

[0066] 4. Training category features and discriminators

[0067] In order to generate 3D consistent category features in the scene view, an additional category feature encoding is trained on the trained Gaussian kernel according to Figure 2 It can be seen that the category feature encoding is a tensor with a length of 32 bits. This category feature encoding can effectively distinguish different objects in the scene and has high computational efficiency.

[0068] In order to train effective category feature encoding on the Gaussian kernel, it is necessary to map the category feature encoding of the Gaussian kernel to the image, and then find the loss of the image prediction category and the true category. With the help of the 3D Gaussian renderer in 3DGS, the category feature encoding is mapped to each pixel of the image in a differentiable way. Combined with the α-mixed rendering method in Gaussian, the influence weight of each Gaussian kernel on the image pixel is calculated, and then the N ordered Gaussian kernel feature encodings mapped to the pixel are accumulated to obtain the feature encoding of each pixel. Similar to the calculation process of color features, the calculation method of category feature encoding is:

[0069]

[0070] Among them, F represents the 32-bit category feature code finally rendered on each pixel of the image, and f i represents the category feature encoding on each Gaussian kernel, α i Represents the transparency of each Gaussian kernel. Through the trained feature encoding, the category characteristics of different objects can be accurately represented in the 3D scene, thus providing more accurate and efficient input for subsequent segmentation tasks.

[0071] Since the image grayscale value ranging from 0 to 255 is used to represent the category of the object, and the encoding dimension is 32, after obtaining the image category map feature encoding, it is necessary to train a multi-layer perceptron as a discriminator to restore the 32-bit feature to the original 256-bit information. The network structure of the multi-layer perceptron is shown in the figure. Assume that the number of pixels of image I is P, I p Represents each pixel of the image, h(I p ) represents the 256-bit feature obtained by mapping the 32-bit feature by the discriminator, g(h(I p )) means that the 256-bit feature is converted into a probability distribution through the softmax function, so the category of each pixel in the image can be expressed as:

[0072]

[0073] Among them, g(h(I p )) c represents the probability of the cth category in the probability distribution. Represents the predicted category of each pixel in the image. After obtaining the predicted category, the cross entropy loss between the real category and the predicted category can be calculated using the real image category for back propagation. The calculation method is:

[0074]

[0075] Among them, L i Represents the cross entropy loss of the i-th image in the classification task.

[0076] 5. Model Segmentation

[0077] The present invention can not only directly extract the specified object by category number, but also supports directly clicking the three-dimensional scene to extract the specified object, providing a more flexible and intuitive model extraction method, improving operation efficiency and accuracy. When using the segmentation method of directly clicking the scene to extract the specified object, since the three-dimensional scene is displayed on the screen in the form of an image, the SAM model can be used to segment the image I displayed on the screen first, and the click position is used as the input prompt β to obtain the mask of the clicked object in the 2D space. It can be expressed as:

[0078] M=SAM(I,β) (1.6)

[0079] Where M represents the segmentation mask obtained by combining the input prompt. Since the displayed image is obtained by Gaussian kernel mapping using the 3DGS model, the 32-bit category feature encoding trained on the Gaussian kernel can be stored on each pixel through mapping. Using the previously trained discriminator and the segmentation mask, the category of each segmented pixel can be obtained.

[0080] Fp =h(I(M)) (1.7)

[0081] Among them, F p Represents the category set of each pixel in the segmented image, and I(M) represents the mask part of the displayed image. The category of the segmented object is determined by counting the frequency of each category appearing in the mask part, thereby determining the category code of the current segmented object. The calculation method of the category code can be expressed as:

[0082]

[0083] Among them, δ(x, y) is the Kronecker delta function, which returns 1 when x=y, otherwise it returns 0. Then, by finding the Gaussian kernel belonging to the category according to the category number, the 3D model segmentation can be achieved. This pixel-level statistical method can effectively handle complex scenes and model occlusion, and improve the accuracy and robustness of model segmentation.

[0084] 6. Remove noise

[0085] Due to the fuzzy boundary factors of the model and the high complexity of the scene, the discriminator cannot perfectly segment the specified object, and the extracted Gaussian kernel will have certain noise points. Therefore, it is necessary to remove the redundant noise points segmented. The present invention uses the K nearest neighbor algorithm (KNN) to remove noise points. The KNN algorithm is a classic machine learning algorithm that completes classification or regression tasks by measuring the distance between data points. In the present invention, the KNN algorithm is used for noise point removal, and a suitable K value is selected to calculate the average distance between each segmented Gaussian kernel and the nearest K Gaussian kernels. By selecting a suitable K value and distance measurement method, it is possible to effectively identify and delete noise points that do not belong to the segmentation model.

[0086] Assume that the number of Gaussian kernels in the scene is N, and use P i represents the coordinates of the i-th Gaussian kernel, K represents the number of nearest neighbor Gaussian kernels, and D represents the average distance threshold. First, calculate the average distance D of the nearest K neighbor points of each point i :

[0087]

[0088] Among them, d ij represents the i-th Gaussian kernel P i The distance between the jth nearest neighbor point. Next, the noise points are removed by the distance threshold D. The new Gaussian kernel obtained after removing the early points can be expressed as:

[0089] {P i |D i ≥D} (1.10)

[0090] By introducing the KNN algorithm to remove noise, the present invention improves the quality and accuracy of the segmentation results, making the segmented image clearer and more reliable. Figure 5 As shown in the figure, the first row is the 3D reconstruction result, the point pointed by the arrow in the picture represents the click position, the second row is the segmentation result without deleting the noise, and the third row is the segmentation result after deleting the noise.

[0091] In the above embodiments, the present invention provides a method for segmenting a three-dimensional scene model at any angle based on 3D Gaussian Splatting, and the present invention realizes accurate segmentation of a 3DGS reconstructed scene; after three-dimensional modeling, 32-bit category features are trained on the trained Gaussian kernel, and the discriminator is trained at the same time, solving the classification problem of the Gaussian kernel; the network structure and loss function of the training discriminator are modified, which greatly reduces the memory occupied by the training process; after clicking an image from any perspective, the image segmentation model is used to segment the clicked object from the image, and the discriminator is used to predict the category of the clicked object, thereby realizing a method for segmenting a three-dimensional model at any angle; the KNN algorithm is used to delete the noise points of the extracted Gaussian kernel and enhance the model details; finally, a method for segmenting a three-dimensional scene model at any angle based on 3DGS is realized, and the specified object is extracted from the reconstructed scene quickly and efficiently.

[0092] The above-described embodiments are merely descriptions of preferred implementations of the present invention, and are not intended to limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various modifications and improvements made by ordinary persons in the art to the technical solution of the present invention should fall within the protection scope of the present invention, and the technical contents for which protection is sought in the present invention have been fully recorded in the claims.

Claims

1. A 3D scene arbitrary angle model segmentation method based on 3D Gaussian Splatting, characterized by: The following steps are involved: S1. Use the Colmap open source software package to reconstruct 3D point cloud data and camera position information based on the image sequence; S2, using the generated 3D point cloud data and camera position information, using 3DGS to train a Gaussian kernel from the 3D point cloud data to represent the real 3D scene; S3. Use the DEVA model to segment the original image sequence and generate a grayscale image of each image. Use different grayscale values ​​to represent different objects in the image. The grayscale of the same object in different images is the same. Use the segmented image grayscale to represent the category of the object in the scene. S4. Based on the trained Gaussian kernel, use the grayscale image to train a 32-bit category feature; At the same time, a neural network is used to train a discriminator. The input of the discriminator is a 32-bit category feature, and the output value range is 0-255, indicating the object category; S5. Use the trained discriminator to input the 32-bit feature of each Gaussian kernel and obtain the category of each Gaussian kernel. Extract the Gaussian kernel of the specified category based on the category code or by directly selecting objects from images at any angle; S6. Use the KNN algorithm to remove outlier Gaussian kernels from the extracted Gaussian kernels to reduce the noise of the generated image.

2. The method for segmenting a 3D scene model at any angle based on 3D Gaussian Splatting according to claim 1, characterized in that: Each of the Gaussian kernels contains position μ, covariance matrix Σ, transparency α and spherical harmonic function information SH, and the color information is represented by spherical harmonic functions. Under a specific camera posture, 3DGS can project the 3D Gaussian kernel to the 2D imaging plane, and then superimpose the colors of the N ordered Gaussian kernels mapped to the pixel by the α blending method to calculate the color of the pixel. The calculation formula is: Where c represents the calculated pixel color, c i Represents the color information carried by each Gaussian kernel, α i Indicates the transparency of each Gaussian kernel.

3. The method for segmenting a three-dimensional scene model at any angle based on 3D Gaussian Splatting according to claim 2, characterized in that: The step S3 includes: for the input image I, the corresponding 2D segmentation mask M can be expressed as: M p =DEVA(I)(1.2) Among them, p represents all pixels of the image, M p Represents the category code for each pixel in the image.

4. The method for segmenting a 3D scene model at any angle based on 3D Gaussian Splatting according to claim 3, characterized in that: The step S4 comprises the following steps: S4.

1. An additional category feature code is trained on the trained Gaussian kernel. The category feature code is a tensor with a length of 32 bits. This category feature code can distinguish different objects in the scene. S4.2, map the Gaussian kernel category feature encoding to the image, and then calculate the loss of the image prediction category and the true category; S4.

3. Train a multi-layer perceptron as a discriminator to restore the 32-bit feature encoding to the original 256-bit information.

5. The method for segmenting a 3D scene model at any angle based on 3D Gaussian Splatting according to claim 4, characterized in that: The step S4.2 includes: using the 3D Gaussian renderer in 3DGS, mapping the category feature code to each pixel of the image in a differentiable manner, combining the α hybrid rendering method in Gaussian, calculating the influence weight of each Gaussian kernel on the image pixel, and then accumulating the N ordered Gaussian kernel feature codes mapped to the pixel to obtain the feature code of each pixel. Similar to the calculation process of color features, the calculation method of category feature coding is: Among them, F represents the 32-bit category feature code finally rendered on each pixel of the image, and f i represents the category feature encoding on each Gaussian kernel, α i Indicates the transparency of each Gaussian kernel.

6. The method for segmenting a 3D scene model at any angle based on 3D Gaussian Splatting according to claim 5, characterized in that: The step S4.3 comprises: assuming that the number of pixels of image I is P, I p Represents each pixel of the image, h(I p ) represents the 256-bit feature obtained by mapping the 32-bit feature by the discriminator, g(h(I p )) means that the 256-bit feature is converted into a probability distribution through the softmax function, so the category of each pixel in the image can be expressed as: Among them, g(h(I p )) c represents the probability of the cth category in the probability distribution, Represents the predicted category of each pixel in the image. After obtaining the predicted category, the cross entropy loss between the real category and the predicted category can be calculated using the real image category for back propagation. The calculation method is: Among them, L i Represents the cross entropy loss of the i-th image in the classification task.

7. The method for segmenting a 3D scene model at any angle based on 3D Gaussian Splatting according to claim 6, characterized in that: When the step S5 uses the method of directly selecting an object from an image at any angle, since the three-dimensional scene is displayed on the screen in the form of an image, the SAM model can be used to segment the image I displayed on the screen first, and the click position is used as the input prompt β to obtain the mask of the clicked object in the two-dimensional space, which can be expressed as: M = SAM(I,β)(1.6) where M represents the segmentation mask obtained by combining the input prompts; Since the displayed image is obtained by Gaussian kernel mapping using the 3DGS model, the 32-bit category feature encoding trained on the Gaussian kernel can be stored on each pixel through mapping. Using the previously trained discriminator and the segmentation mask, the category of each segmented pixel can be obtained. F p =h(I(M))(1.7) Among them, F p Represents the category set of each pixel in the segmented image, I(M) represents the mask part of the displayed image, and the category of the segmented object is determined by counting the frequency of each category appearing in the mask part, thereby determining the category code of the current segmented object. The calculation method of the category code can be expressed as: Among them, δ(x, y) is the Kronecker delta function, which returns 1 when x=y, otherwise it returns 0. Then, the Gaussian kernel belonging to the category is found according to the category number to achieve three-dimensional model segmentation.

8. The method for segmenting a 3D scene model at any angle based on 3D Gaussian Splatting according to claim 7, characterized in that: The step S6 includes: selecting a suitable K value, calculating the average distance between each segmented Gaussian kernel and the K Gaussian kernels closest to it, and selecting a suitable K value and distance measurement method to identify and delete noise points that do not belong to the segmentation model; Assume that the number of Gaussian kernels in the scene is N, and use P i represents the coordinates of the i-th Gaussian kernel, K represents the number of nearest neighbor Gaussian kernels, and D represents the average distance threshold; first, calculate the average distance D of the nearest K neighbor points of each point i : Among them, d ij represents the i-th Gaussian kernel P i The distance between the jth nearest neighbor point; Next, the noise points are deleted by the distance threshold D. The new Gaussian kernel obtained after deleting the noise points can be expressed as: {P i |D i ≥D}(1.10)。

Images