A Gaussian Radiance Field Rendering Optimization Method Based on Proxy Depth and Hierarchical Indexing
By constructing a Gaussian radiation field rendering optimization method based on hierarchical indexes and proxy meshes, the problems of memory overhead and over-drawing in large-scale scenes are solved, achieving high rendering efficiency and memory optimization across platforms.
Patent Information
- Application Number
- CN202610717378.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-22
- Publication Date
- 2026-07-03
AI Technical Summary
Existing Gaussian radiation field rendering technology suffers from high memory overhead, over-drawing, and cross-platform limitations in large-scale scenes, making it difficult to apply effectively on web and mobile devices.
A rendering optimization method based on proxy depth and hierarchical indexing is adopted. By constructing a hierarchical spatial index and proxy mesh of Gaussian radiation field, rendering is performed in combination with GPU hardware rasterization units. Morton coding and hierarchical bounding box trees are used for Gaussian cluster culling. A scene block organization and collaborative multi-level pipeline scheme is designed.
It significantly improves the rendering efficiency of Gaussian radiation field scenes, reduces video memory overhead, and achieves efficient rendering across platforms, especially with a significant increase in frame rate in large-scale scenes.
Smart Images
Figure CN122336100A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of Gaussian radiation field scene rendering technology, and in particular to a Gaussian radiation field rendering optimization method based on proxy depth and hierarchical indexing. Background Technology
[0002] Novel View Synthesis (NVS) is one of the core research topics in computer vision and computer graphics. In recent years, implicit scene representation methods, represented by Neural Radiation Fields (NeRF), have greatly improved the image quality of novel view synthesis. However, implicit scene representation requires dense sampling of light rays and frequent queries of a large neural network during rendering. This huge computational overhead severely restricts the real-time rendering capability of related algorithms. To address this, a series of Gaussian splatting methods, represented by 3D Gaussian Splatting (3DGS), have emerged. Using Gaussian radiation fields as scene primitives, it explicitly models the scene and combines a tile-based differentiable software rasterization pipeline to achieve real-time rendering of photorealistic scenes while providing extremely high visual fidelity, establishing a new paradigm for 3D scene representation and rendering in the NVS direction.
[0003] Despite Gaussian splashing's breakthrough in rendering efficiency, its current mainstream rendering solutions largely rely on software rasterization frameworks based on NVIDIA CUDA. This limitation severely restricts the technology's application in cross-platform scenarios such as web and mobile devices. To overcome this limitation, introducing a GPU hardware rasterization framework based on the graphics API standard library has become an inevitable trend. However, directly porting Gaussian splashing to the hardware rasterization pipeline faces a severe overdraw bottleneck. Furthermore, existing Gaussian splashing research often assumes that the entire scene can be loaded into video memory at once, presenting challenges in organizing and scheduling large-scale scenes. Scene organization and scheduling, along with limitations of hardware development platforms, hinder the further promotion of Gaussian splashing in practical applications. Summary of the Invention
[0004] To address the aforementioned shortcomings in existing technologies, this invention provides a Gaussian radiance field rendering optimization method based on proxy depth and hierarchical indexing. This method solves the problems of excessive video memory overhead, over-drawing, and cross-platform limitations of existing software rasterization frameworks when processing large-scale scenes.
[0005] To achieve the aforementioned objectives, the technical solution adopted by this invention is: a Gaussian radiation field rendering optimization method based on proxy depth and hierarchical indexing, comprising: Construct a Gaussian radiation field scene and proxy mesh from existing images; Construct a hierarchical spatial index for the Gaussian radiation field; The proxy mesh is rendered onto the depth buffer using the GPU hardware rasterization unit to obtain the proxy mesh depth. Apply a pixel-by-pixel offset to the depth of the proxy mesh, oriented behind the viewing direction; Offset depth is calculated based on proxy mesh depth; Combining the hierarchical spatial index of the Gaussian radiation field with cell-based Gaussian cluster removal based on offset depth; The Gaussian primitives in the visible Gaussian cluster are sorted and rendered. The beneficial effects of this invention are: 1. By using explicit quadrilateral geometry and hardware interpolation to replace traditional software rasterization, the limitations of cross-platform deployment are overcome and the basic rendering performance is improved; 2. A proxy depth conservation extraction method based on local depth gradient and a Gaussian cluster fast culling strategy are proposed, which effectively overcomes the over-rasterization bottleneck under the hardware rasterization pipeline. 3. A scene block organization and collaborative multi-level pipeline scheme based on Morton coding was designed, which realizes streaming data scheduling and multi-level visibility culling for large-scale Gaussian radiation field scenes, greatly improving the rendering efficiency of Gaussian radiation field scenes and reducing the runtime memory overhead of large-scale Gaussian radiation field scenes. Attached Figure Description
[0006] Figure 1 A flowchart of a Gaussian radiation field rendering optimization method based on proxy depth and hierarchical index is provided for an embodiment; Figure 2 This is a visual comparison diagram of the vertex processing method of the present invention and 2DGS in the rasterization process; Figure 3 This is a diagram illustrating the real-time rendering process. Figure 4 A comparison chart of the rendering effects of different methods in typical scenarios; Figure 5 The rendering results and cumulative transmittance visualization results before and after enabling early depth testing for the algorithm of this invention are shown in the figure. Detailed Implementation
[0007] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.
[0008] like Figure 1As shown, in one embodiment of the present invention, a Gaussian radiation field rendering optimization method based on proxy depth and hierarchical index includes the following steps: S1. Construct a Gaussian radiation field scene and proxy mesh using existing images.
[0009] To obtain depth information of the opaque surfaces of the scene, this embodiment uses the 2DGS method to construct a geometric proxy for the scene. 2DGS can generate a mesh model that maintains the overall structure and is roughly accurate, enabling efficient extraction of depth information from it. Although imperfections in the details of this proxy mesh may cause some errors in the extracted depth values, the subsequent occlusion culling used to filter Gaussian primitives is essentially a conservative process—allowing a small number of "false positives" but eliminating "false negatives." Therefore, as long as the extracted depth information can conservatively reflect the depth of the front surface of the scene, it is sufficient to effectively culle a large number of occluded primitives, and the imperfections in the mesh details have a limited impact on the reliability and security of the culling results.
[0010] S2. Construct a hierarchical spatial index for the Gaussian radiation field.
[0011] When faced with ultra-large-scale Gaussian radiation field scenes, where the data volume far exceeds the capacity of a single device's video memory, traditional solutions face a fundamental challenge: if all the data cannot be loaded at once, the culling process based on global space partitioning will be unsustainable. At this point, the core issue shifts from a simple "rendering efficiency" problem to an optimization problem subject to multiple constraints. Specifically, the rendering efficiency of large-scale Gaussian radiation field scenes is determined by the following three interdependent factors: Eliminating efficiency constraints Efficiency of real-time elimination algorithms Strongly dependent on the spatial compactness of its operating objects (Gaussian clusters / bounding boxes) Continuity of memory access ,Right now Loose data organization will lead to decreased culling accuracy and reduced cache hit rate.
[0012] Streaming loading constraints ( Scene data It should be classified as A static, uniformly sized block For streaming loading. The partitioning of data blocks must be strongly correlated with physical space to support location-based predictive loading.
[0013] Draw submission constraints ( To avoid the CPU becoming a bottleneck, the overhead of rendering instructions that submit visible primitives to the GPU should be kept as low as possible. This means aiming for the fewest possible draw calls and ensuring that the data submitted in each call is as contiguous as possible in memory.
[0014] Global BVH (Bounding Volume Hierarchy) based on the surface area heuristic (SAH) can theoretically generate the most compact bounding box for each node, thus optimizing... This provides optimal removal efficiency in isolated environments. However, it severely violates the constraints. and The leaf nodes of SAH-BVH are dynamically optimized in space, and their size and spatial range are irregular, making them unsuitable as stable streaming loading units. In addition, the leaf nodes of BVH are randomly laid out in memory. When it is necessary to terminate traversal early or submit the entire subtree of an intermediate node in batches, the Gaussian metadata contained therein is highly scattered in memory, resulting in a large number of independent drawing calls or high data collection costs, which significantly increases the submission overhead.
[0015] This invention proposes a hierarchical scene organization method based on a globally uniform grid and local BVH to solve the above problems, specifically including: The scene space is uniformly divided into a series of fixed-size cells; each cell is a fixed, uniformly sized physical space unit that can be directly used as the basic unit for streaming loading, ensuring the efficiency and predictability of the loading operation and satisfying the constraints. ; Based on the center position of each Gaussian element, the Gaussian elements are assigned to the corresponding cells, so that each cell corresponds to a Gaussian cluster, forming a series of Gaussian clusters; Construct an axis-aligned bounding box for each Gaussian cluster; the bounding box is defined by the minimum and maximum values of the center positions of all Gaussian elements it contains, and its expression is:
[0016] In the formula, Indicates the first i The minimum value of the center position of Gaussian elements in a Gaussian cluster; Indicates the first j The central position of each Gaussian element; Indicates the first i One cell; Indicates the first i The maximum value of the center position of Gaussian elements in a Gaussian cluster; For the first iAxis-aligned bounding boxes corresponding to each Gaussian cluster; Indicates the index of the Gaussian unit; Indicates the index of a cell or Gaussian cluster.
[0017] The organization of Gaussian elements within each cell is optimized, including: sorting the Gaussian elements in each cell using Morton coding to obtain an ordered Morton sequence; and constructing a hierarchical bounding box tree from the bottom up based on the ordered Morton sequence to optimize constraints. and The Gaussian metadata corresponding to each cell is packaged according to an ordered Morton sequence and stored as an independent cell data block, which serves as the basic unit for streaming loading. The index of each independent cell data block and the bounding box data corresponding to the cell are packaged together into an independent data block.
[0018] Finally, the original Gaussian radiation field scene was packaged into the following three parts of data: Cell data block: Each cell corresponds to an independent cell data block that can be loaded in a streaming manner. Each data block is a point cloud composed of all Gaussian elements in its corresponding cell. Proxy Mesh Data Block: The proxy mesh data block stores the complete proxy mesh data that provides prior depth for rendering Gaussian radiation field scenes; Bounding Box and Cell Index Data Block: The bounding box and cell index data block contains the bounding box information of all cells in the scene and the index of their corresponding cell data blocks, which can provide the necessary information for cell visibility calculation and streaming loading.
[0019] When streaming is enabled, before rendering the Gaussian radiance field, the bounding box, cell index data block, and proxy mesh data block need to be read. A complete occlusion culling and streaming process is then performed in the initial view to ensure that all visible Gaussian metadata is read before the first frame of the Gaussian radiance field scene is rendered.
[0020] Specifically, based on the ordered Morton sequence, a hierarchical bounding box tree is constructed in a bottom-up manner. This involves determining the quantization range based on the bounding box of each cell, and then quantizing the Gaussian elements within the cell, expressed as:
[0021] In the formula, Indicates the value of the Gaussian meta-attribute; Indicates the center coordinates of the Gaussian element; To quantize the bit width, take ; The Morton code for the current Gaussian cell is calculated using a bit-interleaving operation; its expression is:
[0022] In the formula, The Morton code for the current Gaussian unit; Indicates three Bit integer binary bits The order of the combination forms a A two-digit integer; Sort the Gaussian elements in each cell in ascending order according to their Morton coding values to obtain an ordered Morton sequence. This sequence has a key property: most Gaussian elements with similar coding values are also adjacent in three-dimensional space. This ordered sequence is formed within each cell. The internally constructed, compact hierarchical bounding box tree provides an ideal data foundation, enabling subsequent local hierarchical culling to be performed with extremely high efficiency and significantly optimizing constraints. .
[0023] Subsequently, based on the aforementioned ordered Merton sequence, a hierarchical bounding box tree is constructed in a bottom-up manner. This construction process follows the bottom-up aggregation principle, and its core is to utilize the spatial locality guaranteed by Merton coding to generate high-quality bounding box tree nodes through simple linear list block division.
[0024] Specifically: The ordered Morton sequence is divided into continuous, fixed-size subsequences, each subsequence forming a leaf Gaussian cluster, expressed as:
[0025] In the formula, For leaf Gaussian clusters; Indicates the first j A high-level unit; Indicates the index of the leaf node; The predefined capacity of the leaf cluster; Indicates the first i Gaussian clusters contained within each cell; The axis-aligned bounding box of each leaf Gaussian cluster is calculated using the following expression:
[0026] In the formula, For the axis-aligned bounding box of the leaf Gaussian cluster; The upper-level nodes of the bounding box tree are constructed through iterative merging: Order No. The set of nodes in the layer is Except for the leaf nodes, the nodes in the remaining layers are merged into the next layer that are consecutive on the Morton sequence. Each node is generated, and its expression is:
[0027] In the formula, Indicates the first Layer nodes, Indicates the total number of cells or Gaussian clusters; Indicates the first Layer nodes; Calculate the first l The bounding box corresponding to a node in a layer is expressed as:
[0028] In the formula, Indicates the first l The bounding box corresponding to the node of the layer; The number of consecutive nodes in the next layer on the Morton sequence is less than When merging all nodes at the highest level into the root node of the bounding box tree, a bounding box tree is obtained; where the bounding box corresponding to the root node of the bounding box tree is the first bounding box. i The bounding box corresponding to each cell.
[0029] Due to child nodes Derived from consecutive segments of the Merton sequence, their corresponding spatial regions are physically adjacent. Therefore, the parent node's bounding box... This allows for a compact wrapping of child node regions, making the hierarchical occlusion culling based on this BVH significantly more efficient than simple strategies based on spatially uniform partitioning. The entire construction process only requires a linear traversal of the sorted sequence and computation of bounding boxes, with a complexity of O(n log n). Furthermore, the generated data structure inherently possesses a high degree of memory access continuity. Based on this memory access continuity, for any BVH node, its corresponding Gaussian cluster can be quickly determined. This allows for the merging of draw calls for multiple leaf nodes that have passed the occlusion culling test, enabling the rendering of the entire Gaussian radiation field scene to be completed with only one draw call, thus maximizing the optimization of constraints. .
[0030] S3. Render the proxy mesh onto the depth buffer using the GPU hardware rasterization unit to obtain the proxy mesh depth.
[0031] S4. Apply a pixel-by-pixel offset to the depth of the proxy mesh, oriented behind the viewing direction.
[0032] To ensure that occlusion culling based on the proxy mesh depth is conservative—that is, never culling actually visible primitives—this invention applies a small, pixel-by-pixel offset towards the rear of the viewing direction to the pixel-by-pixel raw depth value obtained from the proxy mesh, dynamically compensating for the geometric approximation error of the proxy mesh. The core idea is that the offset should be proportional to the local uncertainty of the scene surface at that pixel.
[0033] S5. Calculate the offset depth based on the proxy mesh depth.
[0034] The specific expression is:
[0035] In the formula, Offset depth; This represents the pixel-by-pixel raw depth value obtained from the proxy mesh; Indicates depth offset; The minimum offset threshold ensures a safe, conservative margin even in flat regions. This is a scaling factor used to control the degree of influence of the gradient on the depth offset; The proxy mesh depth gradient for a pixel; A scale factor for transforming depth values from projection space to view space, used to ensure that the offset is meaningful at the actual physical scale of the scene; It is the base offset constant used to compensate for systematic errors that cannot be captured by gradients.
[0036] By calculating the offset depth, it is possible to ensure that larger offsets are allocated in regions with drastic changes in surrogate mesh geometry (high gradient) to accommodate higher reconstruction uncertainties; while smaller offsets are used in flat regions to maintain a high rejection rate.
[0037] Based on the offset scene depth, a hierarchical Z-Buffer (HZB) can be generated to enable subsequent fast culling calculations for Gaussian clusters.
[0038] S6. Perform Gaussian cluster removal based on offset depth and cell partitioning.
[0039] After obtaining the offset depth of the opaque surfaces in the scene, to avoid the huge bandwidth overhead caused by per-primitive depth testing, this invention proposes a batch culling strategy based on Gaussian clusters. The basic idea is to uniformly divide the scene space into a series of cells and organize Gaussian primitives into the corresponding cells, thus dividing the Gaussian radiation field scene into a series of Gaussian clusters. During subsequent rendering, occlusion testing is performed directly using Gaussian clusters as the basic unit. Only when a Gaussian cluster passes the occlusion test will all Gaussian primitives within it be submitted for rendering, thereby transforming the expensive per-primitive testing into an efficient batch operation, significantly reducing bandwidth consumption.
[0040] Transform the eight vertices of each Gaussian cluster bounding box to the normalized device coordinate space, and calculate the axis-aligned projection area of the bounding box in screen space. Calculate the minimum depth value corresponding to the bounding box within the obtained axis-aligned projection area. This value represents the surface depth closest to the camera that the cell may contribute. By querying the hierarchical depth buffer, the maximum value of the offset depth within the screen area covered by the bounding box is obtained. This value ensures that the depth of all real scene surfaces within this area is not less than this query value. The visibility of Gaussian clusters is determined based on the maximum offset depth within the screen area covered by the bounding box; the specific determination expression is as follows:
[0041] In the formula, Indicates the first Visibility of a Gaussian cluster; This is an indicator function; it takes the value 1 when the inequality within the parentheses is true, and 0 otherwise. This represents the minimum depth value corresponding to the bounding box; This represents the maximum offset depth within the screen area covered by the bounding box. This is true only if the above inequality holds (i.e., the nearest point of the cell lies in front of a conservative visible surface). The Gaussian cluster was determined to be visible.
[0042] Remove Gaussian clusters that do not meet the visibility preset conditions.
[0043] Ultimately, the system only collects all those that meet the requirements. The Gaussian clusters corresponding to the cells are identified, and all Gaussian primitives (including position, spherical harmonic coefficients, and other attributes) contained within these Gaussian clusters are batch-submitted to the GPU rendering pipeline. All cells deemed to be Gaussian are... The Gaussian cluster and its internal primitives are skipped entirely, without incurring any sorting, vertex processing, rasterization, or fragment shading overhead. Strict adherence to conservative culling principles is maintained, and computational overhead is concentrated on lightweight bounding box testing and efficient HZB lookups, thus achieving effective savings in memory bandwidth and GPU computing resources while ensuring correctness.
[0044] S7. Sort and render Gaussian primitives in the visible Gaussian cluster.
[0045] When rendering Gaussian primitives, the method for vertex processing of Gaussian primitives is as follows: each Gaussian primitive is explicitly represented as a quadrilateral geometry in its local coordinate system, and the quadrilateral geometry is directly transformed to screen space by constructing a homogeneous transformation matrix, and then rendered efficiently using the GPU hardware rasterization unit.
[0046] Specifically, it includes: For any Gaussian element, construct its enclosing quadrilateral in the local coordinate system, expressed as:
[0047] In the formula, Let Góski represent the coordinates of the vertices of the enclosing quadrilateral in its local coordinate system; Let G be the four vertices of the enclosing quadrilateral of the Gaussian element in the standard two-dimensional Gaussian local coordinate system. The transformation matrix from the standard two-dimensional Gaussian local coordinate system to the world coordinate system is constructed as follows:
[0048] In the formula, This represents the transformation matrix from a standard two-dimensional Gaussian local coordinate system to the world coordinate system; This indicates the center position of the Gaussian element in the scene space coordinate system; For the two-dimensional scale parameters of the Gaussian element; Let be the rotation matrix of the Gaussian element; The transformation matrix for transforming any position of each Gaussian element in the standard two-dimensional Gaussian local coordinate system to the projected coordinate system is expressed as follows:
[0049]
[0050] In the formula, To be Transform the position to the projected coordinate system; Let be the position of any coordinate in the Gaussian element's local spatial coordinate system; To be The position projected onto the standard two-dimensional Gaussian distribution local coordinate system; Transformation matrix from world coordinate system to projected coordinate system; superscript Represents the transpose of a matrix; Will Substituting into the above equation, we can transform the bounding box of any Gaussian element in its local spatial coordinate system to the projected coordinate system.
[0051] The bounding quadrilaterals of Gaussian primitives in the standard 2D Gaussian local coordinate system are transformed to the projective space, and then rendered efficiently using the GPU hardware rasterization unit. The vertex processing method described above is visually comparable to 2DGS in the rasterization process. Figure 2 As shown.
[0052] When calculating the local coordinates of the standard two-dimensional Gaussian distribution of Gaussian elements, the hardware rasterization interpolation function is used to... Interpolate the corresponding Gaussian local coordinates.
[0053] like Figure 3 As shown, this invention divides the real-time rendering process into three independently executable tasks: GPU data transfer, visible data reading, and the main rendering flow. These three tasks have only data dependencies and can be run on three separate threads, thereby maximizing the rendering efficiency of the Gaussian radiance field. The main rendering flow is entirely driven by the GPU, implementing the complete logic of Gaussian radiance field rendering, and can run independently of the streaming loading process.
[0054] GPU data transfer and visible data reading jointly undertake the task of streaming loading. To achieve efficient asynchronous scheduling, the results of reading new visible cell data from the previous frame are provided for GPU data transfer in the current frame. Specifically, GPU data transfer requires loading the results of reading new visible cell data from the previous frame into video memory, while unloading cell data that has been determined to be invisible for F consecutive frames. This decoupling mechanism ensures that the data iteration of streaming loading does not block the main rendering pipeline. To avoid data transfer waiting due to bandwidth limitations, this invention further divides GPU data transfer into two subtasks: bounding box tree and Gaussian meta-center position transfer, and Gaussian meta-other attribute transfer, thereby maximizing rendering efficiency. Visible data reading for each frame occurs after the visibility calculation phase. It reads back the visibility determination results of all cells from the GPU and, based on the cell index, reads the visible cell data blocks that have not been loaded in the current frame. Then, it extracts the bounding box tree structure for each newly loaded cell, providing complete memory-side data for GPU data transfer at the start of the next frame.
[0055] To verify the effectiveness of this invention, the rendering framework implemented in this invention was developed based on the Vulkan 1.4 graphics API. All experiments were conducted in an environment with an NVIDIA RTX A6000 GPU, an Intel Core i9-12900 CPU, and 128GB of memory. The parameter settings involved in this invention are as follows: the gradient influence coefficient used to calculate the proxy mesh offset depth. relative to the base offset All values are set to 0.0015, minimum offset threshold. Set the value to 0.002; when dividing the cells, move the scene along the feature vector. Divided into 16 16 A uniform 3D grid of 16; Morton code quantization bit width used for BVH generation. BVH leaf Gaussian cluster capacity The size of the child nodes of the intermediate node For streaming loading, the longest consecutive number of frames that non-visible cells reside in memory is taken. .
[0056] Experimental evaluations were conducted on the following four standard datasets: 1. NeRF Synthetic dataset: Contains 8 synthetic scenes; 2. Mip-NeRF 360 dataset: Contains 9 real-world 360-degree surround shooting scenes; 3. Tanks and Temples dataset: Contains two real-world outdoor scenes; 4. Deep Blending dataset: Contains two real-world indoor scenes.
[0057] The method proposed in this invention is compared with the following benchmark methods in an experimental manner: 1.3DGS: A standard CUDA implementation of 3D Gaussian splashing; 2. EAGLES: A Gaussian radiative field compression scheme proposed by Girish et al. This scheme introduces a progressive resolution rendering strategy and an influence-based pruning strategy during training, which can effectively reduce the number of Gaussian primitives in the scene, thereby improving the rendering efficiency of Gaussian radiative fields; 3. AdrGS: A Gaussian radiance field rendering acceleration scheme proposed by Wang et al. This scheme reduces the rendering range of the Gaussian volume by using an adaptive radius and a finer axis-aligned bounding box, and introduces a pixel thread load balancing algorithm to dynamically adjust the number of Gaussians to be rendered for different pixels to reduce GPU thread waiting time, thereby improving the rendering efficiency of Gaussian radiance fields; 4. SpeedyGS: A Gaussian radiation field rendering acceleration scheme proposed by Hanson et al. This scheme introduces efficient model pruning techniques during model training, significantly reducing model size, and reduces the amount of Gaussian radiation field sorting and rendering data through a more accurate bounding box-tile intersection algorithm. The results of the comparative experiment are shown in Table 1.
[0058] Table 1
[0059] Experimental results show that, on real-world scene datasets, the method provided in this invention achieves an average efficiency improvement of over 6 times compared to 2DGS. In terms of the magnitude of this improvement, the optimization level of this invention is significantly higher than that of the benchmark algorithm and other comparative algorithms. However, in terms of rendering efficiency, SpeedyGS only outperforms the algorithm of this invention in a few scenes. This is mainly due to the effective cropping of scene scale by SpeedyGS's model pruning technique: compared to 3DGS, SpeedyGS's improvement in FPS is far lower than its improvement in FPS / MP, indicating that its improved rendering frame rate is mainly due to optimization of scene scale, rather than optimization of Gaussian pixel rendering efficiency. Compared to the benchmark algorithm, the improvements in FPS and AdrGS in most scenes are similar to the improvements in FPS / MP, further proving that the improvement in rendering efficiency of this invention is mainly due to the improvement in Gaussian pixel rendering efficiency.
[0060] Table 2 presents the quantitative comparison results of each method on the synthetic dataset. Experimental data show that the present invention significantly outperforms other comparative algorithms in both scene rendering efficiency and Gaussian primitive rendering efficiency. Furthermore, compared to the results on the real dataset, the improvement of the present invention over the benchmark algorithm is only 3 times, far lower than the average improvement of the present invention on the real dataset. Subsequent ablation experiments indicate that the main reason for this result is the scene scale of the synthetic dataset. At this scale, the scene rendering efficiency of the present invention is higher than the submission efficiency of individual drawing commands and the CPU-related computational efficiency; that is, the overall performance bottleneck of the system has shifted from the GPU to the CPU.
[0061] Table 2
[0062] Figure 4 The presentation compares the rendering effects of different methods in typical scenarios. Visually, the rendering results of this invention are indistinguishable from the original 2DGS. This demonstrates that the present invention effectively maintains rendering quality while significantly improving rendering efficiency.
[0063] The specific contributions of each optimization scheme were analyzed through ablation experiments. The optimization schemes were gradually implemented in the experiments to quantify the contribution of each scheme to the overall performance improvement.
[0064] The following six configurations were set up for the ablation experiment: 1.2DGS: The original CUDA implementation of 2DGS; 2. Raster: This invention reproduces the 2DGS algorithm using a hard raster pipeline; 3. Raster + Culling: Based on Raster, enable the occlusion culling implemented in this invention, but disable streaming loading; 4. Ours with / o Culling: This invention's algorithm, but with occlusion culling disabled; 5. Ours w / o Streaming: The algorithm of this invention enables occlusion culling but disables streaming loading; 6. Ours: The complete implementation of this invention.
[0065] The experimental results are shown in Table 3.
[0066] Table 3
[0067] Table 3 presents the ablation experiment results on a real dataset. The following patterns can be observed from the results: the hard raster rendering pipeline, Gaussian meta-screen space OBB optimization, and interpolation optimization contribute the most significant improvements in rendering efficiency. The occlusion culling scheme based on proxy mesh depth only brings about a 10% frame rate improvement, while streaming loading brings about a 1% frame rate decrease, which can be considered a statistical error and is almost negligible. Furthermore, the experimental results show that enabling culling does not affect the image rendering quality, proving the correctness of the culling algorithm in this invention.
[0068] To verify the performance bottleneck hypothesis of this invention on synthetic datasets, the average GPU utilization (Occup.) during the ablation experiment on the synthetic dataset was additionally recorded, and the results are shown in Table 4. The results in the table further confirm the correctness of the algorithm proposed in this invention. Based on this, observing the statistical results of GPU utilization, it can be found that the GPU utilization drops sharply after enabling the bounding box calculation optimization in Section 2.4. On this basis, the scene rendering frame rate can still be effectively improved, confirming that the main performance bottleneck of this invention on synthetic datasets is on the CPU side, rather than the GPU performance bottleneck faced by existing Gaussian radiance field rendering algorithms.
[0069] Table 4
[0070] Although this invention shows a slight decrease in PSNR compared to 2DGS, this is not a substantial degradation in rendering quality. 2DGS employs a non-strict local and global Gaussian maximization strategy to fit benchmark data, while this invention, to achieve more generalizable cross-platform hardware rasterization, uses a stricter screen-space OBB geometric truncation and removes this overfitting strategy. This inevitably introduces pixel-level numerical fine-tuning. PSNR, as a... Traditional norm metrics excessively penalize subpixel-level geometric shifts and minute high-frequency edge variations, failing to accurately reflect the quality of human visual perception. In contrast, the depth feature-based perception metric LPIPS can more accurately assess structural fidelity. As shown in Tables 3 and 4, this invention significantly improves the rendering frame rate while maintaining a high degree of consistency with the original 2DGS metric (within 0.015 differences), and even achieving superior LPIPS performance in complex indoor scenes such as Deep Blending. This fully demonstrates that the hardware rasterization framework of this invention achieves efficient rendering while largely preserving photorealistic visual perception quality.
[0071] Due to the limited scene scale, there is no obvious occlusion relationship in most of the aforementioned small-scale scenes from any perspective. Therefore, to further test the performance improvement brought by the occlusion culling scheme in large-scale scenes, the rendering frame rate of each ablation experiment was statistically analyzed in the low-altitude perspective of the EntireCity dataset. The maximum video memory overhead (VRAM) and the number of Gaussian pixels actually involved in rendering (Mpts, in millions) were also recorded during the process. Table 5 shows the ablation experiment results in the low-altitude perspective of the EntireCity dataset. The results show that the present invention can bring about a 6-fold efficiency improvement compared to 2DGS, and can reduce runtime VRAM overhead by about two-thirds through streaming loading. The comparison results between different ablation methods show that, in the low-altitude perspective, an average of 81% of the Gaussian pixels in the scene are outside the view frustum and behind the actual scene depth. However, without occlusion culling enabled, nearly 60% of the Gaussian pixels in the scene participate in the actual rendering process, far exceeding the theoretical value of 19%. Occlusion culling, building upon frustum culling, removes over 40% of Gaussian primitives, significantly reducing the number of Gaussian primitives actually involved in the rendering process and thus improving rendering efficiency. Therefore, in large-scale scenes, occlusion culling delivers approximately a 3x improvement in rendering efficiency, becoming key to the frame rate improvement of this invention's algorithm in large-scale scenarios. Furthermore, the number of Gaussian primitives actually involved in rendering in this invention's algorithm is approximately 20% higher than that of the 2DGS algorithm. This result primarily stems from the conservative nature of the occlusion culling algorithm implemented in this invention: the occlusion culling implemented in this invention uses Gaussian clusters as the smallest unit for culling, and the proxy mesh depth used for occlusion culling is conservative. This results in some Gaussian clusters near opaque surfaces being deemed visible, and consequently, some Gaussian primitives within these clusters located behind opaque surfaces being deemed visible and participating in the rendering process. These actually invisible Gaussian primitives do not affect the final rendering result.
[0072] Table 5
[0073] Besides occlusion culling, early depth testing can theoretically improve rendering efficiency. However, experiments have shown that enabling early depth testing results in noticeable image holes. Because only a very small number of Gaussian pixels participate in rendering at these holes, the cumulative transmissivity on the corresponding pixels fails to reach the threshold. Therefore, these voids can be identified through visualization of cumulative transmittance. Figure 5The rendering results and cumulative transmittance visualizations before and after enabling early depth testing, along with depth maps from the corresponding viewpoints, are presented. The results show that most image holes after enabling early depth testing are located at the geometric boundaries of the proxy mesh. This is mainly because the proxy mesh struggles to perfectly fit the opaque surface of the scene, resulting in the depth values of some pixels in the final result being significantly ahead of the actual surface. Consequently, most segments on the corresponding pixels fail the early depth test, leading to noticeable holes in the final rendering result. Since these holes cause a sharp decline in rendering quality, early depth testing is not integrated into the final implementation of this invention.
[0074] This invention proposes a Gaussian radiance field rendering optimization method based on proxy depth and hierarchical spatial indexing, aiming to achieve real-time and efficient rendering of large-scale scenes under hardware rasterization pipelines. To address the overdraw problem, this invention designs a conservative occlusion culling scheme that integrates uniform spatial mesh and proxy mesh depth. This scheme utilizes 2D Gaussian splashing (2DGS) to construct an approximate geometric proxy for the scene to extract a conservative prior surface depth. After generating a hierarchical Z-Buffer (HZB), it performs batch early visibility testing on Gaussian clusters within the mesh cells. This mechanism transforms expensive per-primal testing into efficient batch culling, significantly alleviating the overdraw burden and memory bandwidth pressure under hardware rasterization. To address the data scheduling challenges of large-scale scenes, this invention further introduces a hierarchical local indexing structure based on uniform mesh partitioning. Specifically, within each fixed physical cell, Gaussian primitives are independently sorted using Morton coding, and a local hierarchical bounding volume (BVH) tree is constructed from bottom to top on this ordered sequence. The resulting two-level organizational structure of "global uniform grid + local BVH" not only meets the strict constraints of streaming loading on the predictability of physical space, but also utilizes spatial locality to achieve efficient occlusion culling from coarse to fine, and minimizes the number of draw calls.
[0075] In summary, this invention proposes a fast hardware rasterization framework for Gaussian radiance fields based on proxy depth and hierarchical spatial indexing. This framework significantly improves rendering efficiency for Gaussian radiance fields while effectively reducing runtime memory overhead in large-scale Gaussian radiance field scenes. Experimental results show that this invention achieves a frame rate improvement of more than 6 times in various Gaussian radiance field scenes and reduces runtime memory requirements for large-scale Gaussian radiance field scenes to one-third of the original requirements. It verifies the feasibility of occlusion culling in Gaussian radiance field scenes using proxy mesh depth. Future research can further improve the fitting ability of Gaussian radiance fields to complex scene geometric boundaries, thereby supporting more aggressive scene organization and occlusion culling schemes to further enhance the rendering performance of Gaussian radiance fields in large-scale scenes.
Claims
1. A Gaussian radiance field rendering optimization method based on proxy depth and hierarchical indexing, characterized in that, include: Construct a Gaussian radiation field scene and proxy mesh from existing images; Construct a hierarchical spatial index for the Gaussian radiation field; The proxy mesh is rendered onto the depth buffer using the GPU hardware rasterization unit to obtain the proxy mesh depth. Apply a pixel-by-pixel offset to the depth of the proxy mesh, oriented behind the viewing direction; Offset depth is calculated based on proxy mesh depth; Combining the hierarchical spatial index of the Gaussian radiation field with cell-based Gaussian cluster removal based on offset depth; Sort and render Gaussian primitives in a visible Gaussian cluster.
2. The method according to claim 1, characterized in that, The offset depth is calculated based on the proxy mesh depth, and the specific expression is as follows: In the formula, Offset depth; This represents the pixel-by-pixel raw depth value obtained from the proxy mesh; Indicates depth offset; The minimum offset threshold; This is a scaling factor used to control the degree of influence of the gradient on the depth offset; The proxy mesh depth gradient for a pixel; A scale factor for transforming depth values from projection space to view space, used to ensure that the offset is meaningful at the actual physical scale of the scene; It is the base offset constant used to compensate for systematic errors that cannot be captured by gradients.
3. The method according to claim 2, characterized in that, Gaussian cluster removal based on offset depth and cell partitioning includes: The hierarchical spatial index based on Gaussian radiation field uniformly divides the scene space into a series of cells of fixed size; Based on the center position of each Gaussian element, the Gaussian elements are assigned to the corresponding cells, so that each cell corresponds to a Gaussian cluster, forming a series of Gaussian clusters; Construct an axis-aligned bounding box for each Gaussian cluster; the bounding box is defined by the minimum and maximum values of the center positions of all Gaussian elements it contains, and its expression is: In the formula, Indicates the first i The minimum value of the center position of Gaussian elements in a Gaussian cluster; Indicates the first j The central position of each Gaussian element; Indicates the first i One cell; Indicates the first i The maximum value of the center position of Gaussian elements in a Gaussian cluster; For the first i Axis-aligned bounding boxes corresponding to each Gaussian cluster; Indicates the index of the Gaussian unit; Indicates the index of a cell or Gaussian cluster; Optimize the organization of Gaussian elements within each cell; Transform the eight vertices of each Gaussian cluster bounding box to the normalized device coordinate space, and calculate the axis-aligned projection area of the bounding box in screen space. Calculate the minimum depth value corresponding to the bounding box within the obtained axis-aligned projection area; Generate a hierarchical depth buffer based on the offset scene depth; By querying the hierarchical depth buffer, the maximum offset depth within the screen area covered by the bounding box is obtained; The visibility of Gaussian clusters is determined based on the maximum offset depth within the screen area covered by the bounding box. Remove Gaussian clusters that do not meet the visibility preset conditions.
4. The method according to claim 3, characterized in that, Optimize the organization of Gaussian elements within each cell, including: The Gaussian elements in each cell are sorted using Morton coding to obtain an ordered Morton sequence; Based on ordered Morton sequences, a hierarchical bounding box tree is constructed in a bottom-up manner; The Gaussian metadata corresponding to each cell is packaged according to an ordered Morton sequence and stored as an independent cell data block, which serves as the basic unit for streaming loading. The index of each individual cell data block and the corresponding bounding box data of the cell are packaged together into an independent data block.
5. The method according to claim 4, characterized in that, The Gaussian elements within each cell are sorted using Morton coding to obtain an ordered Morton sequence, including: The quantization range is determined based on the bounding box of each cell, and the Gaussian elements within the cell are quantized using the following expression: In the formula, Indicates the value of the Gaussian meta-attribute; Indicates the center coordinates of the Gaussian element; To quantize the bit width, take ; The Morton code for the current Gaussian cell is calculated using a bit-interleaving operation; its expression is: In the formula, The Morton code for the current Gaussian unit; Indicates three Bit integer binary bits The order of the combination forms a A two-digit integer; Sort the Gaussian elements in each cell in ascending order according to their Morton coding values to obtain an ordered Morton sequence.
6. The method according to claim 5, characterized in that, Based on ordered Morton sequences, a hierarchical bounding box tree is constructed in a bottom-up manner, including: The ordered Morton sequence is divided into continuous, fixed-size subsequences, each subsequence forming a leaf Gaussian cluster, expressed as: In the formula, For leaf Gaussian clusters; Indicates the first j A high-level unit; Indicates the index of the leaf node; The predefined capacity of the leaf cluster; Indicates the first i Gaussian clusters contained within each cell; The axis-aligned bounding box of each leaf Gaussian cluster is calculated using the following expression: In the formula, For the axis-aligned bounding box of the leaf Gaussian cluster; The upper-level nodes of the bounding box tree are constructed through iterative merging: Order No. The set of nodes in the layer is Except for the leaf nodes, the nodes in the remaining layers are merged into the next layer that are consecutive on the Morton sequence. Each node is generated, and its expression is: In the formula, Indicates the first Layer nodes, Indicates the total number of cells or Gaussian clusters; Indicates the first Layer nodes; Calculate the first l The bounding box corresponding to a node in a layer is expressed as: In the formula, Indicates the first l The bounding box corresponding to the node of the layer; The number of consecutive nodes in the next layer on the Morton sequence is less than When merging all nodes at the highest level into the root node of the bounding box tree, a bounding box tree is obtained; where the bounding box corresponding to the root node of the bounding box tree is the first bounding box. i The bounding box corresponding to each cell.
7. The method according to claim 3, characterized in that, The expression for determining the visibility of a Gaussian cluster is: In the formula, Indicates the first Visibility of a Gaussian cluster; This is an indicator function; it takes the value 1 when the inequality within the parentheses is true, and 0 otherwise. This represents the minimum depth value corresponding to the bounding box; This represents the maximum offset depth within the screen area covered by the bounding box.
8. The method according to claim 3, characterized in that, The Gaussian elements in the visible Gaussian cluster are sorted by: calculating the center depth of each Gaussian element in the visible Gaussian cluster; and sorting the Gaussian elements according to the obtained center depth.
9. The method according to claim 1, characterized in that, Methods for vertex processing of Gaussian elements during rendering include: For any Gaussian element, construct its enclosing quadrilateral in the standard two-dimensional Gaussian local coordinate system, its expression is: In the formula, The bounding quadrilateral of Gaussian elements in the standard two-dimensional Gaussian distribution local coordinate system; Let G be the four vertices of the enclosing quadrilateral of the Gaussian element in the standard two-dimensional Gaussian local coordinate system. The transformation matrix from the standard two-dimensional Gaussian local coordinate system to the world coordinate system is constructed as follows: In the formula, This represents the transformation matrix from a standard two-dimensional Gaussian local coordinate system to the world coordinate system; This indicates the center position of the Gaussian element in the scene space coordinate system; For the two-dimensional scale parameters of the Gaussian element; Let be the rotation matrix of the Gaussian element; The transformation matrix for transforming any position of each Gaussian element in the standard two-dimensional Gaussian local coordinate system to the projected coordinate system is expressed as follows: In the formula, To be Transform the position to the projected coordinate system; Let be the position of any coordinate in the Gaussian element's local spatial coordinate system; To be The position projected onto the standard two-dimensional Gaussian distribution local coordinate system; Transformation matrix from world coordinate system to projected coordinate system; superscript Represents the transpose of a matrix; Will Substitution The computational expression transforms the bounding box of any Gaussian element in its local spatial coordinate system to the projected coordinate system.
10. The method according to claim 9, characterized in that, When calculating the local coordinates of the standard two-dimensional Gaussian distribution of Gaussian elements, the hardware rasterization interpolation function is used to... Interpolate the corresponding Gaussian local coordinates.