Training method, device, equipment and medium for three-dimensional Gaussian splash algorithm

Abstract

The present invention provides a training method, device, equipment and medium for a three-dimensional Gaussian splash algorithm, which belongs to the field of image processing technology. The method includes acquiring an image data set, downsampling images in the image data set based on at least one sampling rate, and determining a target image data set; determining a three-dimensional Gaussian distribution model according to the image data set; performing multiple rounds of iterative training on the three-dimensional Gaussian distribution model according to the target image data set and a two-dimensional filter; determining an anti-aliasing filter according to the target image data set and a preset two-dimensional filter in each round of iterative training; filtering the three-dimensional Gaussian distribution model according to the anti-aliasing filter, and training the filtered model. The present invention uses low-resolution images instead of original images for iterative training, reduces computational complexity, realizes training acceleration, filters the three-dimensional Gaussian distribution model through an anti-aliasing filter, overcomes the aliasing effect, and ensures rendering quality while reducing training time.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a training method, device, equipment and medium for a three-dimensional Gaussian splash algorithm. Background Art

[0002] In recent years, Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have become mainstream methods for Novel View Synthesis (NVS) and have been widely used. Specifically, the NeRF algorithm encodes the geometry and appearance of the scene into implicit representations through multi-layer perceptrons (MLP) to perform novel view synthesis through volume rendering. The 3DGS algorithm uses an explicit representation based on 3D Gaussian distribution, which not only improves the training efficiency of high-fidelity scene representation, but also allows direct manipulation of elements in the scene.

[0003] As a pioneering method for new view synthesis, the NeRF algorithm surpasses many traditional algorithms in generating high-quality photorealistic images, but its training time is long and its requirements for computing resources are high, especially when dealing with complex scenes and dynamic objects. In addition, the implicit representation of the NeRF algorithm often has difficulty maintaining rendering quality when dealing with high-frequency details and complex lighting conditions, resulting in blurry or distorted results in some scenes.

[0004] Compared with the NeRF algorithm, the 3DGS algorithm has some improvements in training time and rendering quality, reducing the original training time of several hours to dozens of minutes. However, it still takes a long time to train outdoors or in some larger scenes, which limits the applicability of the 3DGS algorithm in real-time applications, such as robot navigation and positioning. Most of the existing methods for training acceleration of the 3DGS algorithm cannot guarantee the rendering quality of the 3DGS algorithm. Therefore, it is necessary to propose a technical solution for training acceleration of the 3DGS algorithm while ensuring the rendering quality. Summary of the invention

[0005] The present invention provides a training method, device, equipment and medium for a three-dimensional Gaussian splatting algorithm, which can accelerate the training of a 3DGS algorithm and ensure the rendering quality of the 3DGS algorithm.

[0006] The present invention provides a training method for a three-dimensional Gaussian splashing algorithm, comprising:

[0007] Acquire an image data set, downsample images in the image data set based on at least one sampling rate, and determine a target image data set;

[0008] Determining a three-dimensional Gaussian distribution model according to the image data set;

[0009] Performing multiple rounds of iterative training on the three-dimensional Gaussian distribution model according to the target image data set and a preset two-dimensional filter;

[0010] Each round of iterative training includes:

[0011] Determine an anti-aliasing filter for this round of iterative training according to the target image data set and a preset two-dimensional filter;

[0012] The three-dimensional Gaussian distribution model is filtered according to the anti-aliasing filter to obtain a filtered model, and the filtered model is trained.

[0013] As an embodiment, determining the anti-aliasing filter for this round of iterative training according to the target image data set and a preset two-dimensional filter includes:

[0014] Determine the target sampling rate corresponding to this round of iterative training according to the rules of coarse-to-fine layered training;

[0015] Selecting a target image corresponding to this round of iterative training from the target image data set according to the target sampling rate;

[0016] Constructing a three-dimensional filter corresponding to this round of iterative training according to the resolution of the target image;

[0017] The anti-aliasing filter is determined based on the three-dimensional filter and the two-dimensional filter.

[0018] As an embodiment, the target sampling rate is calculated as follows:

[0019] ;

[0020] in, f is the target sampling rate corresponding to the current round of iterative training, N is the number of iterations, n is the number of iterations of this round of iterative training, .

[0021] As an embodiment, filtering the three-dimensional Gaussian distribution model according to the anti-aliasing filter to obtain a filtered model, and training the filtered model includes:

[0022] Filtering the three-dimensional Gaussian distribution model according to the three-dimensional filter, and performing two-dimensional projection on the filtered three-dimensional Gaussian distribution model to obtain a two-dimensional Gaussian distribution model;

[0023] Filtering the two-dimensional Gaussian distribution model according to the two-dimensional filter, and using the filtered two-dimensional Gaussian distribution model as the filtered model;

[0024] Inputting the filtered model into a renderer and an optimizer to determine the degree of parameter optimization of the three-dimensional Gaussian distribution model;

[0025] The parameters of the three-dimensional Gaussian distribution model are updated according to the degree of parameter optimization to complete this round of iterative training.

[0026] As an embodiment, the three-dimensional filter is a low-pass Gaussian filter, and the two-dimensional filter is a two-dimensional Gaussian low-pass filter.

[0027] As an embodiment, the expression of the three-dimensional Gaussian distribution model after filtering is as follows:

[0028] ;

[0029] ;

[0030] in, is the three-dimensional Gaussian distribution model before filtering, is the three-dimensional Gaussian distribution model after filtering, x represents the spatial position in the three-dimensional Gaussian distribution model, represents convolution, is the covariance matrix of the three-dimensional Gaussian distribution model, is the covariance matrix of the three-dimensional Gaussian distribution model after filtering, is a three-dimensional filter, is the covariance matrix of the three-dimensional filter, is the identity matrix, is a scalar that adjusts the filter size, is the maximum sampling rate.

[0031] As an embodiment, the expression of the filtered two-dimensional Gaussian distribution model is as follows:

[0032] ;

[0033] ;

[0034] in, is the two-dimensional Gaussian distribution model before filtering, is the two-dimensional Gaussian distribution model after filtering, represents the plane position in the two-dimensional Gaussian distribution model, is the covariance matrix of the two-dimensional Gaussian distribution model, is the covariance matrix of the two-dimensional Gaussian distribution model after filtering, is a two-dimensional filter; is the covariance matrix of the two-dimensional filter.

[0035] The present invention also provides a training device for a three-dimensional Gaussian splashing algorithm, comprising:

[0036] A first determination module is used to obtain an image data set, downsample images in the image data set based on at least one sampling rate, and determine a target image data set;

[0037] A second determination module, used to determine an initialized three-dimensional Gaussian distribution model according to the image data set;

[0038] A training module, used for performing multiple rounds of iterative training on the three-dimensional Gaussian distribution model according to the target image data set and a preset two-dimensional filter;

[0039] Each round of iterative training includes:

[0040] Determine an anti-aliasing filter for this round of iterative training according to the target image data set and a preset two-dimensional filter;

[0041] The three-dimensional Gaussian distribution model is filtered according to the anti-aliasing filter to obtain a filtered model, and the filtered model is trained.

[0042] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a training method for the three-dimensional Gaussian splashing algorithm as described above is implemented.

[0043] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training method for a three-dimensional Gaussian splashing algorithm as described in any one of the above.

[0044] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the training method of the three-dimensional Gaussian splashing algorithm as described above is implemented.

[0045] The training method, device, equipment and medium of the three-dimensional Gaussian splash algorithm provided by the present invention obtain an image data set, downsample the images in the image data set based on at least one sampling rate, and determine the target image data set; determine the three-dimensional Gaussian distribution model according to the image data set; perform multiple rounds of iterative training on the three-dimensional Gaussian distribution model according to the target image data set and a preset two-dimensional filter; wherein each round of iterative training includes: determining the anti-aliasing filter of this round of iterative training according to the target image data set and the preset two-dimensional filter; filtering the three-dimensional Gaussian distribution model according to the anti-aliasing filter to obtain the filtered model, and training the filtered model. The present invention downsamples the images in the image data set based on different sampling rates to reduce the resolution, and uses low-resolution images instead of original images for iterative training, thereby reducing the computational complexity, accelerating the training of the model, and thus improving its feasibility in real-time applications. In addition, during the iterative training process, the present invention filters the three-dimensional Gaussian distribution model according to the anti-aliasing filter, overcomes the aliasing effect caused by multi-resolution training and cross-resolution rendering, and reduces the training time while ensuring the rendering quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 It is one of the flow charts of the training method of the three-dimensional Gaussian splashing algorithm provided by the present invention.

[0048] Figure 2 This is the second flow chart of the training method of the three-dimensional Gaussian splashing algorithm provided by the present invention.

[0049] Figure 3 It is a structural schematic diagram of a training device for a three-dimensional Gaussian splashing algorithm provided by the present invention.

[0050] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] It should be noted that all actions of acquiring signals, information or data in the present invention are performed in compliance with the corresponding data protection laws and policies of the location and with the authorization given by the owner of the corresponding device.

[0053] The 3DGS algorithm uses anisotropic 3D Gaussian to represent 3D scenes, where the k A Gaussian can be expressed as:

[0054] ;

[0055] in, represents the mean value (position), is the covariance matrix, and x represents the spatial position in the model. For convenience, the subscripts are omitted k and the average That is to say, the above formula can be simplified to:

[0056] ;

[0057] In addition, the 3DGS algorithm uses spherical harmonics and opacity to represent the properties of 3D Gaussians to achieve high-fidelity rendering. The training of the 3DGS algorithm is completed through gradient-based optimization, which is determined by the difference between the rendered image and the corresponding true value image. The gradient-based optimization consists of two main steps, namely densification and back-propagation. The densification strategy supports adaptive control of the Gaussian, so that the 3D Gaussian expands in under-reconstructed areas and splits in over-reconstructed areas. The gradient information is used in the back-propagation process to update the parameters, which are adjusted according to the error between the rendered image and the true value image. During the rendering process, each 3D Gaussian is projected onto the view plane and forms a 2D Gaussian distribution, which is represented as:

[0058] ;

[0059] in, is a 2D covariance matrix, and x' represents the plane position in the model. The 3DGS algorithm uses a 2D dilation filter To ensure that each Gaussian occupies at least one pixel in the rendered image, x represents the plane position in the model. Therefore, the filtered 2D Gaussian follows the following formula:

[0060] ;

[0061] Currently, algorithms for accelerating 3DGS algorithm training can be divided into two categories: operation-level acceleration and algorithm-level acceleration. In theory, these two methods can be combined to achieve better acceleration. Operation-level acceleration methods refer to improving the basic operation process of gradient calculation to reduce calculation time. For example, the Fast Differentiable Rendering on Raster-based Rendering Pipelines (DISTWAR) algorithm is an operation-level acceleration algorithm that accelerates the training process of the 3DGS algorithm by increasing the amount of atomic operations. The DISTWAR algorithm optimizes the calculation of gradients during the rendering process, making each step of the calculation more accurate and efficient, thereby significantly reducing training time. The algorithm-level acceleration method focuses on optimizing the overall training strategy of the 3DGS algorithm. Multi-resolution training is usually used to reduce computational overhead. Representative algorithms include Efficient Accelerated 3D Gaussians with Lightweight EncodingS (EAGLES) and Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular, Stereo, and RGB-D Cameras. By introducing a multi-resolution training strategy, the details of the scene are gradually increased during the training of the model, which is similar to the progressive training method of NeRF (such as InstantNGP algorithm, NSVF algorithm, DVGO algorithm). By gradually refining the scene details, a rough scene representation is trained in the early stage, and then more details are gradually added to improve the overall training efficiency and quality.

[0062] As the resolution of the input image increases, the computational complexity of training the 3DGS algorithm increases dramatically. Comparing the time consumption of training the 3DGS algorithm using images of different resolutions for the same scene, under the same experimental conditions, the experimental results show that when the image resolution is reduced, the time consumption of the training process will decrease.

[0063] The training process of the 3DGS algorithm is mainly divided into two parts, namely densification and back propagation. Both are accelerated for low-resolution images. For low-resolution images, the requirements for reconstructing details (such as leaves on a tree) are reduced. Therefore, the problems of under-reconstruction and over-reconstruction are alleviated. The size of the gradient information in the back-propagation process is determined by the image resolution, because the distortion is calculated by the rendered image and the true value image. The computational complexity of rasterization rendering during the back-propagation process is also proportional to the image resolution. Therefore, as the image resolution decreases, the time consumption of back-propagation decreases.

[0064] However, algorithm-level acceleration methods do not consider the aliasing effect caused by using multi-scale images for training, resulting in a suboptimal training process. The resolution of the image during rendering determines the sampling frequency of continuous scenes, and the scene is represented by many 3D Gaussians, so the image resolution also determines the frequency of sampling different 3D Gaussians. The training quality is not taken into account for training acceleration.

[0065] To this end, the present invention uses a low-resolution hierarchical training method, which trains the 3DGS algorithm in layers by downsampling the training images to different degrees, and does not use the original resolution images as training sets, thereby significantly reducing the computational complexity of training the 3DGS algorithm, achieving accelerated training of the model, and thus improving the feasibility of the 3DGS algorithm in real-time applications. At the same time, the present invention overcomes the aliasing effect caused by multi-resolution training and cross-resolution rendering through filtering, reducing training time while ensuring rendering quality. Experimental results show that the technology proposed in the present invention shortens the training time of the original 3DGS algorithm by 57.81% without significantly affecting the reconstruction quality. The training method, device, equipment and medium of the three-dimensional Gaussian splash algorithm provided by the present invention are described in detail below in conjunction with the accompanying drawings.

[0066] Figure 1 is one of the flow charts of the training method of the three-dimensional Gaussian splashing algorithm provided by the present invention, such as Figure 1 As shown, the present invention provides an accelerated training method for a three-dimensional Gaussian splash algorithm, including steps S100 to S300.

[0067] Step S100, obtain an image data set, downsample the images in the image data set based on at least one sampling rate, and determine a target image data set. The image data set is an existing image set used to train the 3DGS algorithm. Downsample the same image in the image data set based on different sampling rates to obtain at least one low-resolution sampling image corresponding to the image. All low-resolution sampling images are collected together to obtain the target image data set. In the implementation process, each image in the image data set is downsampled to different degrees to obtain a new training data set.

[0068] Step S200: determining a three-dimensional Gaussian distribution model according to the image data set.

[0069] The images in the original resolution image data set are input into the 3D reconstruction software to generate a scene point cloud, and then the scene point cloud is used to obtain an initialized 3D Gaussian distribution model. In an embodiment of the present invention, the images in the image data set are processed using COLMAP software, and high-quality scene point clouds can be generated through steps such as image matching, feature extraction, and camera posture estimation.

[0070] Step S300: performing multiple rounds of iterative training on the three-dimensional Gaussian distribution model according to the target image data set and a preset two-dimensional filter.

[0071] Each round of iterative training includes:

[0072] Determine an anti-aliasing filter for this round of iterative training according to the target image data set and a preset two-dimensional filter;

[0073] The three-dimensional Gaussian distribution model is filtered according to the anti-aliasing filter to obtain a filtered model, and the filtered model is trained.

[0074] The target image data set includes multi-scale images obtained by downsampling. The present invention accelerates the training process by using the down-sampled multi-scale images instead of using the original images for training.

[0075] It can be understood that the present invention downsamples images in an image data set based on different sampling rates to reduce the resolution, and uses low-resolution images instead of original images for iterative training, thereby reducing computational complexity and accelerating the training of the model, thereby improving its feasibility in real-time applications. In addition, during the iterative training process, the present invention filters the three-dimensional Gaussian distribution model according to an anti-aliasing filter, thereby overcoming the aliasing effect caused by multi-resolution training and cross-resolution rendering, and reducing training time while ensuring rendering quality.

[0076] Based on the above embodiment, as an optional embodiment, determining the anti-aliasing filter for this round of iterative training according to the target image data set and a preset two-dimensional filter includes:

[0077] Determine the target sampling rate corresponding to this round of iterative training according to the rules of coarse-to-fine layered training;

[0078] Selecting a target image corresponding to this round of iterative training from the target image data set according to the target sampling rate;

[0079] Constructing a three-dimensional filter corresponding to this round of iterative training according to the resolution of the target image;

[0080] The anti-aliasing filter is determined based on the three-dimensional filter and the two-dimensional filter.

[0081] In the early stages of training, a low-resolution image is used for fast refinement, resulting in a coarse 3D representation of the scene, with the reconstruction focusing on the overall structure and main features of the scene rather than finer details. As training progresses, the number of 3D Gaussians increases to capture more detailed features in the image, at which point the 3D Gaussians fit the fine structure of the scene on top of the coarse 3D description, thereby reducing the time consumed by the slower refinement using high-resolution images.

[0082] It can be understood that the present invention makes the anti-aliasing filter related to the target image data set of this round of iterative training, realizes low-resolution hierarchical training from coarse to fine, and accelerates high-resolution rendering, which significantly speeds up the training speed while ensuring high-fidelity scene reconstruction.

[0083] Based on the above embodiment, as an optional embodiment, the calculation formula of the target sampling rate is as follows:

[0084] ;

[0085] in, f is the target sampling rate corresponding to the current round of iterative training, N is the number of iterations, n is the number of iterations of this round of iterative training, .

[0086] For the entire training process, in this round of iterative training (stage n), use The 3D Gaussian is optimized with the target image downsampled by a factor of 2, the finest target image is downsampled by a factor of 2, and no full-resolution images are used during the entire training process.

[0087] No. n The expression of the loss function of the stage is as follows:

[0088] ;

[0089] in, represents the mathematical expectation, are the rendered image and the downsampled true image, Obtained by Gaussian rendering, is the pixel of the original resolution image, is the attribute parameter of Gauss.

[0090] It can be understood that, according to the Nyquist theorem, the anti-aliasing filter of the present invention can ensure alias-free sampling of 3D Gaussians by discrete pixels during rendering, limit the finest target image to 2 times downsampling, and do not use full-resolution images during the entire training process, thereby significantly accelerating the training speed while ensuring high-fidelity scene reconstruction.

[0091] Figure 2 This is a second flow chart of the training method of the three-dimensional Gaussian splashing algorithm provided by the present invention, such as Figure 2 As shown, based on the above embodiment, as an optional embodiment, filtering the three-dimensional Gaussian distribution model according to the anti-aliasing filter to obtain a filtered model and training the filtered model includes the following steps:

[0092] Step S310, filtering the three-dimensional Gaussian distribution model according to the three-dimensional filter, and performing two-dimensional projection on the filtered three-dimensional Gaussian distribution model to obtain a two-dimensional Gaussian distribution model.

[0093] Step S320, filtering the two-dimensional Gaussian distribution model according to the two-dimensional filter, and using the filtered two-dimensional Gaussian distribution model as the filtered model.

[0094] Step S330, input the filtered model to a renderer and an optimizer to determine the degree of parameter optimization of the three-dimensional Gaussian distribution model. The renderer is used to generate a rendered image, calculate the difference based on the rendered image, and input the difference to the optimizer, which determines the degree of parameter optimization of the three-dimensional Gaussian distribution model based on the difference.

[0095] Step S340: update the parameters of the three-dimensional Gaussian distribution model based on the parameter optimization degree to complete this round of iterative training. The parameters of the three-dimensional Gaussian distribution model include mean, covariance matrix, spherical harmonic function coefficients and opacity.

[0096] It is understandable that hierarchical training brings about aliasing problems due to the use of multi-scale training images, and simple hierarchical training will affect reconstruction. The present invention introduces three-dimensional filters and two-dimensional filters to achieve smooth and robust training.

[0097] Based on the above embodiment, as an optional embodiment, the three-dimensional filter is a low-pass Gaussian filter, the two-dimensional filter is a two-dimensional Gaussian low-pass filter, and the two-dimensional Gaussian low-pass filter is a 2D Mip filter. Replacing the original 2D expansion filter with a 2D Mip filter can effectively reduce the impact of high-frequency details that may cause aliasing.

[0098] Optionally, the three-dimensional Gaussian distribution model and the three-dimensional filter are both Gaussian distributions, and the convolution is still Gaussian distribution. The expression of the three-dimensional Gaussian distribution model after filtering is as follows:

[0099] ;

[0100] ;

[0101] in, is the three-dimensional Gaussian distribution model before filtering, is the three-dimensional Gaussian distribution model after filtering, x represents the spatial position in the three-dimensional Gaussian distribution model, represents convolution, is the covariance matrix of the three-dimensional Gaussian distribution model, is the covariance matrix of the three-dimensional Gaussian distribution model after filtering, is a three-dimensional filter, is the covariance matrix of the three-dimensional filter, is the identity matrix, is a scalar that adjusts the filter size, is the maximum sampling rate.

[0102] The two-dimensional Gaussian distribution model and the two-dimensional filter are both Gaussian distributions, and they are still Gaussian distributions after convolution.

[0103] The expression of the two-dimensional Gaussian distribution model after filtering is as follows:

[0104] ;

[0105] ;

[0106] in, is the two-dimensional Gaussian distribution model before filtering, is the two-dimensional Gaussian distribution model after filtering, represents the plane position in the two-dimensional Gaussian distribution model, is the covariance matrix of the two-dimensional Gaussian distribution model, is the covariance matrix of the two-dimensional Gaussian distribution model after filtering, is a two-dimensional filter; is the covariance matrix of the two-dimensional filter.

[0107] It can be understood that the present invention uses filters to perform filtering processing on the basis of low-resolution layered training to filter out image aliasing across resolutions. During cross-resolution training, high-frequency artifacts of rendered images can be effectively reduced while retaining their details and quality. This solves the rendering distortion caused by resolution changes during low-resolution layered training and high-resolution rendering. The training of multi-scale images can be smoother and more robust, and the quality of rendered images can be maintained while accelerating training.

[0108] The technical effects of the present invention are illustrated below with reference to experimental data.

[0109] Table 1 Performance comparison between the present invention and other methods based on 3DGS evolution.

[0110]

[0111] Experimental results show that the present invention significantly speeds up the training process on all datasets. The present invention (N = 3) reduces the time consumption to 42.19% of 3DGS, 47.21% of EAGLES, 29.23% of the original Mip-Splatting algorithm, and 50.6% of Mip-Splatting (using 2x downsampled images for training and original resolution images for inference). In addition, the proposed method has no significant impact on the reconstruction quality.

[0112] When the number of low-resolution layers N changes, the present invention achieves a better balance between training speed and rendering quality. The experiment was conducted on the Mip-NeRF360 dataset, and the experimental results also show that by increasing the number of stages N, the present invention can further accelerate the training process.

[0113] Table 2 Performance comparison of the present invention and no filter.

[0114]

[0115] Table 2 shows the numerical results of the proposed method with and without anti-aliasing filter on Mip-NeRF360 dataset. The results show that the reconstruction performance can be significantly improved by using anti-aliasing filter. Using anti-aliasing filter helps to improve the reconstruction performance by more than 1 dB. All experiments shown in Table 2 are implemented with N = 3.

[0116] The training device of the three-dimensional Gaussian splash algorithm provided by the present invention is described below. The training device of the three-dimensional Gaussian splash algorithm described below and the training method of the three-dimensional Gaussian splash algorithm described above can refer to each other.

[0117] Figure 3is a schematic diagram of the structure of the training device of the three-dimensional Gaussian splashing algorithm provided by the present invention, such as Figure 3 As shown, the present invention also provides a training device for a three-dimensional Gaussian splashing algorithm, comprising the following modules.

[0118] A first determination module 310 is configured to obtain an image data set, downsample images in the image data set based on at least one sampling rate, and determine a target image data set;

[0119] A second determination module 320, configured to determine an initialized three-dimensional Gaussian distribution model according to the image data set;

[0120] A training module 330, configured to perform multiple rounds of iterative training on the three-dimensional Gaussian distribution model according to the target image data set and a preset two-dimensional filter;

[0121] Each round of iterative training includes:

[0122] Determine an anti-aliasing filter for this round of iterative training according to the target image data set and a preset two-dimensional filter;

[0123] The three-dimensional Gaussian distribution model is filtered according to the anti-aliasing filter to obtain a filtered model, and the filtered model is trained.

[0124] As an embodiment, determining the anti-aliasing filter for this round of iterative training according to the target image data set and a preset two-dimensional filter includes:

[0125] Determine the target sampling rate corresponding to this round of iterative training according to the rules of coarse-to-fine layered training;

[0126] Selecting a target image corresponding to this round of iterative training from the target image data set according to the target sampling rate;

[0127] Constructing a three-dimensional filter corresponding to this round of iterative training according to the resolution of the target image;

[0128] The anti-aliasing filter is determined based on the three-dimensional filter and the two-dimensional filter.

[0129] As an embodiment, the target sampling rate is calculated as follows:

[0130] ;

[0131] in, f is the target sampling rate corresponding to the current round of iterative training, N is the number of iterations, n is the number of iterations of this round of iterative training, .

[0132] As an embodiment, filtering the three-dimensional Gaussian distribution model according to the anti-aliasing filter to obtain a filtered model, and training the filtered model includes:

[0133] Filtering the three-dimensional Gaussian distribution model according to the three-dimensional filter, and performing two-dimensional projection on the filtered three-dimensional Gaussian distribution model to obtain a two-dimensional Gaussian distribution model;

[0134] Filtering the two-dimensional Gaussian distribution model according to the two-dimensional filter, and using the filtered two-dimensional Gaussian distribution model as the filtered model;

[0135] Inputting the filtered model into a renderer and an optimizer to determine the degree of parameter optimization of the three-dimensional Gaussian distribution model;

[0136] The parameters of the three-dimensional Gaussian distribution model are updated according to the degree of parameter optimization to complete this round of iterative training.

[0137] As an embodiment, the three-dimensional filter is a low-pass Gaussian filter, and the two-dimensional filter is a two-dimensional Gaussian low-pass filter.

[0138] As an embodiment, the expression of the three-dimensional Gaussian distribution model after filtering is as follows:

[0139] ;

[0140] ;

[0141] in, is the three-dimensional Gaussian distribution model before filtering, is the three-dimensional Gaussian distribution model after filtering, x represents the spatial position in the three-dimensional Gaussian distribution model, represents convolution, is the covariance matrix of the three-dimensional Gaussian distribution model, is the covariance matrix of the three-dimensional Gaussian distribution model after filtering, is a three-dimensional filter, is the covariance matrix of the three-dimensional filter, is the identity matrix, is a scalar that adjusts the filter size, is the maximum sampling rate.

[0142] As an embodiment, the expression of the filtered two-dimensional Gaussian distribution model is as follows:

[0143] ;

[0144] ;

[0145] in, is the two-dimensional Gaussian distribution model before filtering, is the two-dimensional Gaussian distribution model after filtering, represents the plane position in the two-dimensional Gaussian distribution model, is the covariance matrix of the two-dimensional Gaussian distribution model, is the covariance matrix of the two-dimensional Gaussian distribution model after filtering, is a two-dimensional filter; is the covariance matrix of the two-dimensional filter.

[0146] It should be noted that the training device for the three-dimensional Gaussian splash algorithm provided by the present invention can execute the training method for the three-dimensional Gaussian splash algorithm described in any of the above embodiments during specific operation, and has the technical effect corresponding to the method, which will not be elaborated in this embodiment.

[0147] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the training method of the three-dimensional Gaussian splash algorithm, the method comprising: acquiring an image data set, downsampling the images in the image data set based on at least one sampling rate, and determining a target image data set; determining a three-dimensional Gaussian distribution model according to the image data set; performing multiple rounds of iterative training on the three-dimensional Gaussian distribution model according to the target image data set and a preset two-dimensional filter; wherein each round of iterative training comprises: determining an anti-aliasing filter for this round of iterative training according to the target image data set and a preset two-dimensional filter; filtering the three-dimensional Gaussian distribution model according to the anti-aliasing filter to obtain a filtered model, and training the filtered model.

[0148] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0149] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the training method of the three-dimensional Gaussian splattering algorithm provided by the above-mentioned methods, and the method includes: acquiring an image data set, downsampling the images in the image data set based on at least one sampling rate, and determining a target image data set; determining a three-dimensional Gaussian distribution model based on the image data set; performing multiple rounds of iterative training on the three-dimensional Gaussian distribution model based on the target image data set and a preset two-dimensional filter; wherein each round of iterative training includes: determining an anti-aliasing filter for this round of iterative training based on the target image data set and the preset two-dimensional filter; filtering the three-dimensional Gaussian distribution model according to the anti-aliasing filter to obtain a filtered model, and training the filtered model.

[0150] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training method for a three-dimensional Gaussian splattering algorithm provided by the above-mentioned methods, the method comprising: acquiring an image data set, downsampling images in the image data set based on at least one sampling rate, and determining a target image data set; determining a three-dimensional Gaussian distribution model based on the image data set; performing multiple rounds of iterative training on the three-dimensional Gaussian distribution model based on the target image data set and a preset two-dimensional filter; wherein each round of iterative training comprises: determining an anti-aliasing filter for this round of iterative training based on the target image data set and a preset two-dimensional filter; filtering the three-dimensional Gaussian distribution model according to the anti-aliasing filter to obtain a filtered model, and training the filtered model.

[0151] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0152] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a three-dimensional Gaussian splash algorithm, characterized in that: include: Acquire an image data set, downsample images in the image data set based on at least one sampling rate, and determine a target image data set; Determining a three-dimensional Gaussian distribution model according to the image data set; Performing multiple rounds of iterative training on the three-dimensional Gaussian distribution model according to the target image data set and a preset two-dimensional filter; Each round of iterative training includes: Determine an anti-aliasing filter for this round of iterative training according to the target image data set and a preset two-dimensional filter; Filtering the three-dimensional Gaussian distribution model according to the anti-aliasing filter to obtain a filtered model, and training the filtered model; The step of filtering the three-dimensional Gaussian distribution model according to the anti-aliasing filter to obtain a filtered model and training the filtered model comprises: Filtering the three-dimensional Gaussian distribution model according to a three-dimensional filter, and performing two-dimensional projection on the filtered three-dimensional Gaussian distribution model to obtain a two-dimensional Gaussian distribution model; Filtering the two-dimensional Gaussian distribution model according to the two-dimensional filter, and using the filtered two-dimensional Gaussian distribution model as the filtered model; Inputting the filtered model into a renderer and an optimizer to determine the degree of parameter optimization of the three-dimensional Gaussian distribution model; Updating the parameters of the three-dimensional Gaussian distribution model according to the degree of parameter optimization to complete this round of iterative training; The expression of the three-dimensional Gaussian distribution model after filtering is as follows: ； ； in, is the three-dimensional Gaussian distribution model before filtering, is the three-dimensional Gaussian distribution model after filtering, x represents the spatial position in the three-dimensional Gaussian distribution model, represents convolution, is the covariance matrix of the three-dimensional Gaussian distribution model, is the covariance matrix of the three-dimensional Gaussian distribution model after filtering, is a three-dimensional filter, is the covariance matrix of the three-dimensional filter, is the identity matrix, is a scalar that adjusts the filter size, is the maximum sampling rate; The expression of the two-dimensional Gaussian distribution model after filtering is as follows: ； ； in, is the two-dimensional Gaussian distribution model before filtering, is the two-dimensional Gaussian distribution model after filtering, represents the plane position in the two-dimensional Gaussian distribution model, is the covariance matrix of the two-dimensional Gaussian distribution model, is the covariance matrix of the two-dimensional Gaussian distribution model after filtering, is a two-dimensional filter; is the covariance matrix of the two-dimensional filter.

2. The training method of the three-dimensional Gaussian splash algorithm according to claim 1, characterized in that: Determining the anti-aliasing filter for this round of iterative training according to the target image data set and a preset two-dimensional filter includes: Determine the target sampling rate corresponding to this round of iterative training according to the rules of coarse-to-fine layered training; Selecting a target image corresponding to this round of iterative training from the target image data set according to the target sampling rate; Constructing a three-dimensional filter corresponding to this round of iterative training according to the resolution of the target image; The anti-aliasing filter is determined based on the three-dimensional filter and the two-dimensional filter.

3. The training method of the three-dimensional Gaussian splash algorithm according to claim 2, characterized in that: The target sampling rate is calculated as follows: ； in, f is the target sampling rate corresponding to the current round of iterative training, N is the number of iterations, n is the number of iterations of this round of iterative training, .

4. The training method of the three-dimensional Gaussian splash algorithm according to claim 1, characterized in that: The three-dimensional filter is a low-pass Gaussian filter, and the two-dimensional filter is a two-dimensional Gaussian low-pass filter.

5. A training device for a three-dimensional Gaussian splash algorithm, characterized in that: A training method for implementing the three-dimensional Gaussian splashing algorithm according to any one of claims 1 to 4, the device comprising: A first determination module is used to obtain an image data set, downsample images in the image data set based on at least one sampling rate, and determine a target image data set; A second determination module, used to determine an initialized three-dimensional Gaussian distribution model according to the image data set; A training module, used for performing multiple rounds of iterative training on the three-dimensional Gaussian distribution model according to the target image data set and a preset two-dimensional filter; Each round of iterative training includes: Determine an anti-aliasing filter for this round of iterative training according to the target image data set and a preset two-dimensional filter; The three-dimensional Gaussian distribution model is filtered according to the anti-aliasing filter to obtain a filtered model, and the filtered model is trained.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the training method of the three-dimensional Gaussian splashing algorithm according to any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the training method of the three-dimensional Gaussian splattering algorithm according to any one of claims 1 to 4 is implemented.

Images