Scene fidelity parameter calculation method and system of three-dimensional Gaussian primitive

Abstract

The invention provides a scene fidelity parameter calculation method and system of a three-dimensional Gaussian primitive, and relates to the technical field of video processing, and the method comprises the steps: obtaining scene data after completing three-dimensional Gaussian splash reconstruction; for each three-dimensional Gaussian primitive in the scene data, searching a nearest neighbor according to the position information of the three-dimensional Gaussian primitive, and extracting a high-dimensional feature vector of the nearest neighbor; the high-dimensional feature vector comprises any one or a combination of any several of a distance feature, a color feature, a color specificity feature, a scale feature, a composite scale feature and a transparency feature; performing normalization processing on the high-dimensional feature vector to obtain a standard feature vector; and inputting the standard feature vector into a multi-layer perceptron regression model, and outputting to obtain a scene fidelity parameter of the three-dimensional Gaussian primitive. According to the method, the importance of each three-dimensional Gaussian primitive can be accurately reflected through the scene fidelity parameter without depending on rendering, so that the reconstruction, compression and transmission processes are optimized.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a method and system for calculating scene fidelity parameters of three-dimensional Gaussian elements. Background Technology

[0002] In existing technologies, the importance assessment of 3D Gaussian Splatting primitives typically employs a projection area measurement method based on manually set camera viewpoints. Specifically, this method involves pre-arranging a set of discrete viewpoints, sequentially projecting each 3D Gaussian primitive onto the imaging planes of these viewpoints, counting the number of pixels it occupies in the corresponding image, and using this pixel count as the basis for measuring the primitive's visual contribution. However, the above approach has the following inherent drawbacks:

[0003] The evaluation results are highly sensitive to viewpoints and unstable, heavily relying on the number, spatial distribution, and sampling density of preset viewpoints. When the camera pose is slightly adjusted or the number of samples changes, the number of pixels counted for the same primitive may fluctuate significantly, leading to a lack of consistency and robustness in importance ranking.

[0004] Secondly, it suffers from high computational complexity and low processing efficiency. To obtain relatively reliable statistics, it is often necessary to set up dozens or even hundreds of viewpoints and perform complete rendering and pixel counting operations on each viewpoint. As the number of 3D Gaussian primitives in the scene increases, the overall time complexity grows linearly or even superlinearly, significantly limiting the real-time or near-real-time processing efficiency in large-scale scenes. Summary of the Invention

[0005] The purpose of this application is to provide a method, system, computer-readable storage medium, and electronic device for calculating scene fidelity parameters of 3D Gaussian primitives, which can accurately reflect the importance of each 3D Gaussian primitive through scene fidelity parameters without relying on rendering, thereby optimizing the reconstruction, compression, and transmission process.

[0006] To address the aforementioned technical problems, this application provides a method for calculating scene fidelity parameters of three-dimensional Gaussian elements, the specific technical solution of which is as follows:

[0007] Acquire scene data that has been reconstructed using 3D Gaussian splashing;

[0008] For each 3D Gaussian element in the scene data, the nearest neighbor is searched based on the position information of the 3D Gaussian element, and the high-dimensional feature vector of the nearest neighbor is extracted; the high-dimensional feature vector contains any one or any combination of distance features, color features, color specificity features, scale features, composite scale features, and transparency features.

[0009] After normalizing the high-dimensional feature vector, a standard feature vector is obtained;

[0010] The standard feature vector is input into the multilayer perceptron regression model, and the scene fidelity parameter of the three-dimensional Gaussian unit is output; the scene fidelity parameter is used to indicate the three-dimensional Gaussian unit required for scene rendering.

[0011] Optionally, the distance feature extraction method includes:

[0012] Determine the nearest neighbor value N;

[0013] Calculate the logarithmic average distance to the N nearest neighbors based on the position information of the three-dimensional Gaussian elements;

[0014] The logarithmic mean distance of the N nearest neighbors is used as the overall distribution of all three-dimensional Gaussian elements, and the global mean and the first standard deviation are calculated based on the logarithmic mean distance.

[0015] Calculate the first distance feature based on the global mean and the first standard deviation;

[0016] Calculate the local mean and second standard deviation based on the logarithmic average distance;

[0017] The second distance feature is calculated based on the local mean and the second standard deviation; the distance feature includes the first distance feature and the second distance feature.

[0018] Optionally, after normalizing the high-dimensional feature vector, the standard feature vector obtained includes:

[0019] Each feature dimension is truncated according to a preset percentile threshold. Values ​​below the lower percentile threshold are set as the lower limit, and values ​​above the upper percentile threshold are set as the upper limit.

[0020] The truncated values ​​are mapped to the normalized interval to obtain the standard feature vector.

[0021] Optionally, before inputting the standard feature vector into the multilayer perceptron regression model, the method further includes:

[0022] The multilayer perceptron regression model is obtained through iterative training;

[0023] The training process of the multilayer perceptron regression model includes:

[0024] Each round randomly selects a 3D Gaussian splash scene and a rendering perspective within the 3D Gaussian splash scene;

[0025] For each 3D Gaussian primitive visible from the rendering perspective, the scene fidelity parameter is multiplied element by element by the three spatial scale factors and opacity of the 3D Gaussian primitive to simulate clipping.

[0026] A composite loss function is constructed based on rendering loss, score mean loss, and entropy loss.

[0027] The multilayer perceptron regression model is obtained by iterative training based on the composite loss function.

[0028] Optionally, after iteratively training the multilayer perceptron regression model, the method further includes:

[0029] The multilayer perceptron regression model and its parameters are saved to the server.

[0030] Optionally, after inputting the standard feature vector into the multilayer perceptron regression model and outputting the scene fidelity parameters of the three-dimensional Gaussian units, the method further includes:

[0031] Three-dimensional Gaussian primitives whose scene fidelity parameters are below a set threshold are used as redundant primitives.

[0032] The redundant primitives are removed, and scene rendering is performed based on the remaining three-dimensional Gaussian primitives.

[0033] Optionally, after inputting the standard feature vector into the multilayer perceptron regression model and outputting the scene fidelity parameters of the three-dimensional Gaussian units, the method further includes:

[0034] The corresponding quantization step size is determined based on the scene fidelity parameters of each 3D Gaussian element; wherein, the scene fidelity parameters of the 3D Gaussian element are negatively correlated with the quantization step size;

[0035] Scene reconstruction is performed based on the three-dimensional Gaussian elements and the corresponding quantization step size.

[0036] This application also provides a system for calculating scene fidelity parameters of three-dimensional Gaussian elements, including:

[0037] The data acquisition module is used to acquire scene data that has been reconstructed using 3D Gaussian splashing.

[0038] The feature extraction module is used to search for the nearest neighbor for each three-dimensional Gaussian element in the scene data based on the position information of the three-dimensional Gaussian element, and extract the high-dimensional feature vector of the nearest neighbor; the high-dimensional feature vector contains any one or any combination of distance features, color features, color specificity features, scale features, composite scale features and transparency features;

[0039] The data normalization module is used to normalize the high-dimensional feature vector to obtain a standard feature vector;

[0040] The parameter calculation module is used to input the standard feature vector into the multilayer perceptron regression model and output the scene fidelity parameter of the three-dimensional Gaussian unit; the scene fidelity parameter is used to indicate the three-dimensional Gaussian unit required for scene rendering.

[0041] This application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the scene fidelity parameter calculation method as described above.

[0042] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor, when calling the computer program in the memory, implements the steps of the scene fidelity parameter calculation method as described above.

[0043] This application also provides a computer program product, including a computer program that, when executed, implements the steps of the scene fidelity parameter calculation method as described above.

[0044] This application provides a method for calculating the scene fidelity parameter of a 3D Gaussian primitive, comprising: acquiring scene data that has been reconstructed using 3D Gaussian splashing; for each 3D Gaussian primitive in the scene data, searching for the nearest neighbor based on the position information of the 3D Gaussian primitive, and extracting the high-dimensional feature vector of the nearest neighbor; the high-dimensional feature vector includes any one or a combination of any of the following: distance feature, color feature, color specificity feature, scale feature, composite scale feature, and transparency feature; normalizing the high-dimensional feature vector to obtain a standard feature vector; inputting the standard feature vector into a multilayer perceptron regression model, and outputting the scene fidelity parameter of the 3D Gaussian primitive; the scene fidelity parameter is used to indicate the 3D Gaussian primitive required for scene rendering.

[0045] This application introduces high-dimensional feature vectors and a multilayer perceptron regression model to perform refined analysis on scene data reconstructed using 3D Gaussian splashes, accurately calculating the scene fidelity parameters of each 3D Gaussian primitive. By utilizing the positional information of the 3D Gaussian primitives to search for nearest neighbors, high-dimensional features including distance, color, color specificity, scale, composite scale, and transparency are extracted to comprehensively characterize the visual and geometric properties of the 3D Gaussian primitives in the scene. The normalized standard feature vectors eliminate dimensional differences, ensuring the consistency and comparability of the data input to the model. Based on these features, the multilayer perceptron regression model intelligently evaluates the contribution of each 3D Gaussian primitive to the scene rendering quality, outputting its scene fidelity parameters. These parameters effectively guide the selection and weight allocation of 3D Gaussian primitives in subsequent rendering processes, suppressing noise and artifacts caused by low-quality primitives, and improving overall visual consistency and detail reproduction. This application does not rely on rendering; it accurately reflects the importance of each 3D Gaussian primitive through scene fidelity parameters, thereby optimizing the reconstruction, compression, and transmission processes. It is suitable for 3D reconstruction and real-time rendering tasks with high visual fidelity requirements.

[0046] This application also provides a three-dimensional Gaussian primitive scene fidelity parameter calculation system, a computer-readable storage medium, and an electronic device, which have the above-mentioned beneficial effects, and will not be elaborated here. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0048] Figure 1 A flowchart illustrating a method for calculating scene fidelity parameters using three-dimensional Gaussian elements, provided in an embodiment of this application.

[0049] Figure 2 A schematic diagram of a scene fidelity parameter calculation system for three-dimensional Gaussian elements provided in this application embodiment;

[0050] Figure 3 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0052] See Figure 1 , Figure 1 A flowchart illustrating a method for calculating scene fidelity parameters using three-dimensional Gaussian elements, provided in this application embodiment, is included.

[0053] S101: Acquire scene data that has been reconstructed using 3D Gaussian splashing;

[0054] S102: For each three-dimensional Gaussian element in the scene data, search for the nearest neighbor based on the position information of the three-dimensional Gaussian element, and extract the high-dimensional feature vector of the nearest neighbor; the high-dimensional feature vector contains any one or any combination of distance features, color features, color specificity features, scale features, composite scale features and transparency features;

[0055] S103: After normalizing the high-dimensional feature vector, a standard feature vector is obtained;

[0056] S104: Input the standard feature vector into the multilayer perceptron regression model and output the scene fidelity parameter of the three-dimensional Gaussian unit; the scene fidelity parameter is used to indicate the three-dimensional Gaussian unit required for scene rendering.

[0057] The core objective of step S101 is to obtain a complete 3D scene representation. Specifically, based on the previously completed 3D Gaussian Splatting reconstruction process, a scene dataset containing all 3D Gaussian primitives and their attribute parameters needs to be exported. This dataset should cover the basic attributes of each 3D Gaussian primitive, such as its spatial coordinates, initial color value, and transparency weight, while preserving the overall topological structure and spatial distribution information of the scene. The technical effect of this step is to provide a complete and structured data foundation for subsequent feature extraction and analysis, ensuring that all 3D Gaussian primitives involved in the scene composition are included in the processing scope, and avoiding deviations in subsequent steps due to missing data.

[0058] In step S102, for each independent 3D Gaussian primitive in the scene dataset, other 3D Gaussian primitives can be retrieved as "nearest neighbors" within their neighborhood (e.g., using a fixed radius or K-nearest neighbor strategy) centered on the current 3D Gaussian primitive's location. This process relies on efficient spatial indexing structures (e.g., KD-trees or octrees) to accelerate the search, ensuring rapid location of associated primitives even in complex scenes. At least one of six key features is then extracted from the resulting nearest neighbor set:

[0059] Distance feature: Reflects the spatial distance distribution between the current primitive and its nearest neighbor primitive, used to measure local density;

[0060] Color characteristics: Statistically measure the mean, variance, or dominant hue of colors within the nearest neighbor region to characterize local color consistency.

[0061] Color specificity feature: quantifies the degree of difference in the current primitive color when viewed from different angles, highlighting visual uniqueness;

[0062] Scale features: Record the size parameters of the current primitive, reflecting its relative size in the scene;

[0063] Composite scale characteristics: Combining the scale ratio between the current primitive and its nearest neighbor primitives, reflecting the hierarchy of the local structure;

[0064] Transparency feature: Directly use the transparency value of the current primitive to help determine the occlusion relationship.

[0065] The technical effect of this step is that it comprehensively characterizes the local context of each three-dimensional Gaussian primitive by combining multi-dimensional features, laying the foundation for subsequent quantification of its contribution to the scene.

[0066] In one feasible implementation, the distance feature extraction method includes the following steps:

[0067] Step A1: Determine the nearest neighbor value N;

[0068] Step A2: Calculate the logarithmic average distance to the N nearest neighbors based on the position information of the three-dimensional Gaussian elements;

[0069] Step A3: Take the logarithmic mean distance of the N nearest neighbors as the overall distribution of all three-dimensional Gaussian elements, and calculate the global mean and the first standard deviation based on the logarithmic mean distance;

[0070] Step A4: Calculate the first distance feature based on the global mean and the first standard deviation;

[0071] Step A5: Calculate the local mean and second standard deviation based on the logarithmic mean distance;

[0072] Step A6: Calculate the second distance feature based on the local mean and the second standard deviation; the distance feature includes the first distance feature and the second distance feature.

[0073] First, determine the average distance from the current 3D Gaussian element to its K nearest neighbors, and then perform a logarithmic transformation. Next, calculate the first distance feature z-score (f0) of this value within the overall N-point range, and the second distance feature z-score (f1) within the neighborhood K-point range.

[0074] For color features, the three components of the basic color can be obtained through the coefficients of the zeroth-order spherical harmonic function, and then the z-scores of these three components in the neighborhood range can be calculated (represented as f2, f3, and f4 respectively).

[0075] For color-specific features, M directions (e.g., 60 directions) can be uniformly sampled on the bounding sphere, and the ratio of the variance of the directional color to the square of the mean can be calculated as the color specificity. Then, the z-score (denoted as f5) of this feature over the entire N-point range and the z-score (denoted as f6) over the neighborhood range can be calculated separately.

[0076] For scale features, we can first determine the three spatial scale factors of the three-dimensional Gaussian unit, sort them by size, and then take the logarithm. Then we can calculate the z-score of the three eigenvalues ​​in the global range (represented as f7, f8, and f9, respectively) and the z-score in the neighborhood range (represented as f10, f11, and f12, respectively).

[0077] For composite scale features, we can determine the product of the three spatial scale factors of the Gaussian element, take the logarithm, and then calculate the z-score (denoted as f13) of this value in the global range and the z-score (denoted as f14) in the neighborhood range.

[0078] For transparency features, the transparency of Gaussian elements can be determined, and then the z-score of transparency in the global range (denoted as f15) and the z-score in the neighborhood range (denoted as f16) can be calculated separately.

[0079] After the above processing, each three-dimensional Gaussian element can obtain a 17-dimensional (from f0 to f16) high-dimensional feature vector.

[0080] In step S103, the high-dimensional feature vector extracted in the previous step is standardized to map the feature values ​​of each dimension to a uniform numerical range (e.g., [0,1]). Specifically, the maximum and minimum values ​​of each feature dimension can be calculated individually, and dimensional differences can be eliminated through linear transformation. Outliers (such as extreme distances or abrupt colors) are truncated or smoothed to prevent a single feature from excessively affecting the overall representation.

[0081] Normalized feature vectors serve as standard feature vectors. By unifying the numerical scale among different features, the model's ability to learn the balance of importance of each feature can be enhanced, while improving the training stability and convergence speed of subsequent machine learning models.

[0082] In one feasible implementation, each feature dimension can be truncated according to a preset percentile threshold. Values ​​below the lower percentile threshold are set as the lower limit, and values ​​above the upper percentile threshold are set as the upper limit. Finally, the truncated values ​​are mapped to a normalized interval to obtain a standard feature vector.

[0083] First, based on preset percentile thresholds (e.g., lower percentile threshold P_min and upper percentile threshold P_max, which can be set according to actual application), each feature dimension is truncated within the overall range, with values ​​below P_min defined as the lower limit and values ​​above P_max defined as the upper limit. Then, the truncated values ​​are linearly mapped to the 0-1 interval to obtain the normalized feature vector.

[0084] In step S104, a multilayer perceptron regression model can be pre-constructed, receiving normalized standard feature vectors as input. After nonlinear transformation through several hidden layers, the scene fidelity parameter is output. The higher the scene fidelity parameter, the more critical the corresponding 3D Gaussian primitives are to scene rendering. In practical applications, a threshold can be set to directly filter low-scoring 3D Gaussian primitives, or the transparency can be adjusted according to fidelity weights to achieve adaptive simplification. This step transforms the "Gaussian importance" that originally required manual experience into a learnable regression task, significantly improving the automation level of scene compression and real-time rendering.

[0085] This application, through the introduction of high-dimensional feature vectors and a multilayer perceptron regression model, performs refined analysis on scene data reconstructed using 3D Gaussian splashing, accurately calculating the scene fidelity parameters of each 3D Gaussian primitive. It utilizes the positional information of the 3D Gaussian primitives to search for nearest neighbors, extracting high-dimensional features including distance, color, color specificity, scale, composite scale, and transparency, comprehensively characterizing the visual and geometric properties of the 3D Gaussian primitives in the scene. The normalized standard feature vectors eliminate dimensional differences, ensuring the consistency and comparability of the data input to the model. Based on these features, the multilayer perceptron regression model intelligently evaluates the contribution of each 3D Gaussian primitive to the scene rendering quality, outputting its scene fidelity parameters. These parameters effectively guide the selection and weight allocation of 3D Gaussian primitives in subsequent rendering processes, suppressing noise and artifacts caused by low-quality primitives, and improving overall visual consistency and detail reproduction. This application, without relying on rendering, accurately reflects the importance of each 3D Gaussian primitive through scene fidelity parameters, thereby optimizing the reconstruction, compression, and transmission processes. It is suitable for 3D reconstruction and real-time rendering tasks with high visual fidelity requirements.

[0086] The method for constructing the multilayer perceptron regression model is not limited here; the following is a feasible implementation method:

[0087] B1: In each round, a three-dimensional Gaussian splash scene and a rendering perspective under the three-dimensional Gaussian splash scene are randomly selected;

[0088] B2: For each 3D Gaussian primitive visible from the rendering perspective, the scene fidelity parameter is multiplied element by element by the three spatial scale factors and opacity of the 3D Gaussian primitive to simulate clipping;

[0089] B3: Construct a composite loss function based on rendering loss, average score loss, and entropy loss;

[0090] B4: Iterative training is performed based on the composite loss function to obtain the multilayer perceptron regression model.

[0091] When training a multilayer perceptron regression model, an end-to-end supervised learning framework can be used to drive model parameter updates with a large-scale labeled dataset. The initial stage loads a predefined network architecture (such as a fully connected network with several hidden layers), followed by a multi-round training process. Each round of training includes three core steps: forward propagation to calculate predicted values, backpropagation to calculate gradients, and parameter optimization, until the model performance stabilizes or reaches a preset termination condition (such as the maximum number of training rounds or the validation set error no longer decreasing).

[0092] A 3D Gaussian splash scene instance is randomly sampled from a pre-built scene library. This scene consists of multiple discrete Gaussian primitives, each with independent spatial location, shape, and attribute parameters. Next, a specific viewpoint is randomly selected from all possible observation views of this scene as the input viewpoint for the current training sample. This process completely covers all potential scene-viewpoint combinations without replacement, ensuring comprehensive data sampling.

[0093] For the visible area under the current rendering viewpoint, traverse all overlapping Gaussian primitives. For each primitive, extract its spatial scale factor (corresponding to the size of the X / Y / Z dimensions) and opacity value along its three axes, and perform element-wise multiplication with pre-defined scene fidelity parameters (a set of scalar weight coefficients). The result directly affects the display state of the primitive, effectively adjusting its visibility and size ratio under the current viewpoint.

[0094] Simultaneous calculation of three independent loss components:

[0095] 1. Rendering loss: quantifies the pixel-level differences between the model-generated image and the real reference image to ensure the quality of the reconstructed image;

[0096] 2. Mean score loss: The average value of the constraint model's primitive importance scores approximates the ideal distribution of manually labeled data. For example, the mean of the constraint importance scores is close to a certain preset value (such as 0.25).

[0097] 3. Entropy loss is used to penalize the phenomenon that the probability distribution of the model output is too concentrated or too dispersed. It is used to limit the distribution of scores and prevent all scores from being over-binarized to 0 or 1.

[0098] Finally, these three factors are weighted and summed according to preset weights to form a comprehensive evaluation index.

[0099] In B4, the composite loss function constructed in B3 is used as the optimization objective, and efficient optimizers such as Adaptive Moment Estimation (Adam) are employed to drive model parameter updates. During each backpropagation phase, the gradient of the loss function with respect to the weights of each layer is automatically calculated, and convergence is achieved quickly through momentum acceleration and adaptive learning rate adjustment strategies. The loss curve on the validation set is monitored in real time during training, and an early stopping mechanism is triggered as needed to prevent overfitting.

[0100] After training is complete, the multilayer perceptron regression model and its parameters are saved to the server for use in subsequent inference stages.

[0101] The following section explains several application scenarios for scene fidelity parameters:

[0102] The first application scenario is image rendering, specifically the server-side reconstruction process. In this scenario, 3D Gaussian primitives with scene fidelity parameters below a set threshold are considered redundant primitives. These redundant primitives are then removed, and scene rendering is performed based on the remaining 3D Gaussian primitives. The corresponding execution process is as follows:

[0103] Step 201: In the process of gradually reconstructing the 3D Gaussian splash scene using image data, when the reconstruction reaches a certain stage, feature extraction and feature normalization are performed on the currently generated 3D Gaussian primitives, and the saved multilayer perceptron regression model and its parameters are called.

[0104] The process of feature extraction and feature normalization for three-dimensional Gaussian elements will not be elaborated here.

[0105] Step 202: Input the normalized features into the trained multilayer perceptron regression model and output the scene fidelity parameter for each Gaussian unit.

[0106] Step 203: Sort the 3D Gaussian primitives according to the scene fidelity parameters and filter out a certain proportion of 3D Gaussian primitives with low scene fidelity parameters.

[0107] Step 204: After removing redundant primitives, continue with the subsequent reconstruction process to improve efficiency and reduce redundant points.

[0108] As can be seen, this embodiment can maintain high reconstruction quality while removing more redundant points when pruning three-dimensional Gaussian elements on a large scale, and has better quality preservation capabilities than existing methods.

[0109] The second application scenario is the server-side compression process. The corresponding quantization step size is determined based on the scene fidelity parameters of each 3D Gaussian unit. The scene fidelity parameters of the 3D Gaussian units are negatively correlated with the quantization step size. Finally, scene reconstruction is performed based on the 3D Gaussian units and the corresponding quantization step size. The specific implementation process is as follows:

[0110] Step 301: For the reconstructed 3D Gaussian splash scene, perform feature extraction and feature normalization on the 3D Gaussian splash scene corresponding to the 3D Gaussian splash scene, and call the saved multilayer perceptron regression model and its parameters.

[0111] Step 302: Input the normalized features into the importance prediction network to obtain the scene fidelity parameters of each 3D Gaussian unit.

[0112] Step 303: Sort the 3D Gaussian primitives according to the scene fidelity parameters, and assign corresponding quantization step sizes to their attributes according to different scene fidelity parameter levels: for Gaussian primitives with low scene fidelity parameters, use a larger quantization step size to reduce storage overhead; for Gaussian primitives with high scene fidelity parameters, use a smaller quantization step size to ensure reconstruction quality.

[0113] Through the above-mentioned differentiated quantization process, compression efficiency can be significantly improved with minimal impact on visual quality.

[0114] See Figure 2 , Figure 2 A schematic diagram of a three-dimensional Gaussian element scene fidelity parameter calculation system provided in this application embodiment is shown. The system includes:

[0115] The data acquisition module is used to acquire scene data that has been reconstructed using 3D Gaussian splashing.

[0116] The feature extraction module is used to search for the nearest neighbor for each three-dimensional Gaussian element in the scene data based on the position information of the three-dimensional Gaussian element, and extract the high-dimensional feature vector of the nearest neighbor; the high-dimensional feature vector contains any one or any combination of distance features, color features, color specificity features, scale features, composite scale features and transparency features;

[0117] The data normalization module is used to normalize the high-dimensional feature vector to obtain a standard feature vector;

[0118] The parameter calculation module is used to input the standard feature vector into the multilayer perceptron regression model and output the scene fidelity parameter of the three-dimensional Gaussian unit; the scene fidelity parameter is used to indicate the three-dimensional Gaussian unit required for scene rendering.

[0119] Based on the above embodiments, as a preferred embodiment, the system may further include a model training module for performing the following steps:

[0120] The multilayer perceptron regression model is obtained through iterative training;

[0121] The training process of the multilayer perceptron regression model includes:

[0122] Each round randomly selects a 3D Gaussian splash scene and a rendering perspective within the 3D Gaussian splash scene;

[0123] For each 3D Gaussian primitive visible from the rendering perspective, the scene fidelity parameter is multiplied element by element by the three spatial scale factors and opacity of the 3D Gaussian primitive to simulate clipping.

[0124] A composite loss function is constructed based on rendering loss, score mean loss, and entropy loss.

[0125] The multilayer perceptron regression model is obtained by iterative training based on the composite loss function.

[0126] This application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in the above method embodiments.

[0127] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0128] The computer-readable storage medium provided in this embodiment includes the method mentioned above, and has the same effect.

[0129] This application also provides an electronic device, see [link to document]. Figure 3 The present application provides a structural diagram of an electronic device, such as... Figure 3 As shown, it may include a processor 1410 and a memory 1420.

[0130] The processor 1410 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 1410 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1410 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 1410 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 1410 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0131] The memory 1420 may include one or more computer-readable storage media, which may be non-transitory. The memory 1420 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 1420 is used to store at least the following computer program 1421, which, after being loaded and executed by the processor 1410, is capable of implementing the relevant steps in the methods executed by the electronic device side as disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 1420 may also include an operating system 1422 and data 1423, etc., and the storage method may be temporary storage or permanent storage. The operating system 1422 may include Windows, Linux, Android, etc.

[0132] In some embodiments, the electronic device may further include a display screen 1430, an input / output interface 1440, a communication interface 1450, a sensor 1460, a power supply 1470, and a communication bus 1480.

[0133] certainly, Figure 3 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiments of this application. In practical applications, the electronic device may include more than [other components]. Figure 3 More or fewer components as shown, or combinations of certain components.

[0134] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. As the system provided in the embodiments corresponds to the method provided in the embodiments, the description is relatively simple; relevant parts can be found in the method section.

[0135] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

[0136] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for calculating scene fidelity parameters using three-dimensional Gaussian elements, characterized in that, include: Acquire scene data that has been reconstructed using 3D Gaussian splashing; For each 3D Gaussian element in the scene data, the nearest neighbor is searched based on the position information of the 3D Gaussian element, and the high-dimensional feature vector of the nearest neighbor is extracted; the high-dimensional feature vector contains any one or any combination of distance features, color features, color specificity features, scale features, composite scale features, and transparency features. After normalizing the high-dimensional feature vector, a standard feature vector is obtained; The standard feature vector is input into the multilayer perceptron regression model, and the scene fidelity parameter of the three-dimensional Gaussian unit is output; the scene fidelity parameter is used to indicate the three-dimensional Gaussian unit required for scene rendering.

2. The method for calculating scene fidelity parameters according to claim 1, characterized in that, The methods for extracting the distance features include: Determine the nearest neighbor value N; Calculate the logarithmic average distance to the N nearest neighbors based on the position information of the three-dimensional Gaussian elements; The logarithmic mean distance of the N nearest neighbors is used as the overall distribution of all three-dimensional Gaussian elements, and the global mean and the first standard deviation are calculated based on the logarithmic mean distance. Calculate the first distance feature based on the global mean and the first standard deviation; Calculate the local mean and second standard deviation based on the logarithmic average distance; The second distance feature is calculated based on the local mean and the second standard deviation; the distance feature includes the first distance feature and the second distance feature.

3. The method for calculating scene fidelity parameters according to claim 1, characterized in that, After normalizing the high-dimensional feature vector, the standard feature vector obtained includes: Each feature dimension is truncated according to a preset percentile threshold. Values ​​below the lower percentile threshold are set as the lower limit, and values ​​above the upper percentile threshold are set as the upper limit. The truncated values ​​are mapped to the normalized interval to obtain the standard feature vector.

4. The method for calculating scene fidelity parameters according to claim 1, characterized in that, Before inputting the standard feature vector into the multilayer perceptron regression model, the following steps are also included: The multilayer perceptron regression model is obtained through iterative training; The training process of the multilayer perceptron regression model includes: Each round randomly selects a 3D Gaussian splash scene and a rendering perspective within the 3D Gaussian splash scene; For each 3D Gaussian primitive visible from the rendering perspective, the scene fidelity parameter is multiplied element by element by the three spatial scale factors and opacity of the 3D Gaussian primitive to simulate clipping. A composite loss function is constructed based on rendering loss, score mean loss, and entropy loss. The multilayer perceptron regression model is obtained by iterative training based on the composite loss function.

5. The method for calculating scene fidelity parameters according to claim 4, characterized in that, After obtaining the multilayer perceptron regression model through iterative training, the process further includes: The multilayer perceptron regression model and its parameters are saved to the server.

6. The method for calculating scene fidelity parameters according to any one of claims 1-5, characterized in that, After inputting the standard feature vector into the multilayer perceptron regression model and outputting the scene fidelity parameters of the three-dimensional Gaussian elements, the model further includes: Three-dimensional Gaussian primitives whose scene fidelity parameters are below a set threshold are used as redundant primitives. The redundant primitives are removed, and scene rendering is performed based on the remaining three-dimensional Gaussian primitives.

7. The method for calculating scene fidelity parameters according to any one of claims 1-5, characterized in that, After inputting the standard feature vector into the multilayer perceptron regression model and outputting the scene fidelity parameters of the three-dimensional Gaussian elements, the model further includes: The corresponding quantization step size is determined based on the scene fidelity parameters of each 3D Gaussian element; wherein, the scene fidelity parameters of the 3D Gaussian element are negatively correlated with the quantization step size; Scene reconstruction is performed based on the three-dimensional Gaussian elements and the corresponding quantization step size.

8. A system for calculating scene fidelity parameters using three-dimensional Gaussian elements, characterized in that, include: The data acquisition module is used to acquire scene data that has been reconstructed using 3D Gaussian splashing. The feature extraction module is used to search for the nearest neighbor for each three-dimensional Gaussian element in the scene data based on the position information of the three-dimensional Gaussian element, and extract the high-dimensional feature vector of the nearest neighbor; the high-dimensional feature vector contains any one or any combination of distance features, color features, color specificity features, scale features, composite scale features and transparency features; The data normalization module is used to normalize the high-dimensional feature vector to obtain a standard feature vector; The parameter calculation module is used to input the standard feature vector into the multilayer perceptron regression model and output the scene fidelity parameter of the three-dimensional Gaussian unit; the scene fidelity parameter is used to indicate the three-dimensional Gaussian unit required for scene rendering.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the steps of the method as described in any one of claims 1 to 7.

Images