Hybrid rendering method based on space coding

Abstract

The embodiment of the invention relates to the technical field of three-dimensional graphic rendering, and discloses a mixed rendering method based on voxel coding. According to the method, for a three-dimensional scene in which multiple rendering strategies coexist, voxel-level division is carried out through the scene, space coding of nodes is calculated based on finite element grids, and ordered arrangement and linear coding of the nodes in the three-dimensional space are achieved. And by using the codes, constructing a code-driven scene block scheduling mechanism, and binding corresponding rendering strategies (such as neural rendering, point cloud rendering and traditional graphic rendering) to different coding areas. In the rendering stage, a proper rendering mode can be selected according to observation positions, equipment capabilities or application requirements, and unified management and mixed scheduling of multiple rendering modes are realized by means of a unified spatial coding index. The method is suitable for meta-universe construction, digital twinborn visualization and scene organization in a complex three-dimensional graphic system, and the scheduling efficiency and space consistency in a large-scale environment are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer graphics and three-dimensional scene management technology, and in particular to a hybrid rendering method based on spatial coding, which is used to efficiently divide, schedule and manage scenes in a three-dimensional space where multiple rendering strategies (such as neural rendering, point cloud rendering and traditional graphics rendering) coexist. The method is suitable for rendering optimization of large-scale complex three-dimensional environments in applications such as digital twins, metaverses, and virtual reality. Background Art

[0002] In fields such as virtual reality, simulation, and digital twins, there is a growing demand for large-scale, highly realistic three-dimensional environments. Traditional rasterization and ray tracing rendering technologies have difficulty achieving photorealism when dealing with real-time rendering of complex scenes. With the emergence of new rendering technologies such as neural rendering (such as NeRF) and point cloud rendering (such as Gaussian Splatting), heterogeneous rendering architectures that support hybrid rendering technologies have emerged, that is, integrating neural rendering, point cloud rendering, and traditional rasterization or ray tracing rendering technologies in the same scene. However, most existing three-dimensional scene scheduling mechanisms are based on a unified rendering paradigm or static partitioning, and lack flexible, spatial structure-aware rendering strategy management methods.

[0003] Current spatial partitioning methods, such as octrees and Morton encoding, primarily serve data compression or accelerate structure construction. Their goal is to improve the efficiency of a single rendering pipeline, rather than supporting the parallelization and switching of multiple rendering mechanisms. Furthermore, neural network rendering technologies like NeRF are resource-intensive and have limited applicability, typically requiring only localized use. There remains a lack of effective mechanisms for flexibly integrating and scheduling these rendering technologies within large 3D scenes.

[0004] Therefore, there is an urgent need for a universal and efficient spatial organization and scheduling mechanism that can support the collaborative management, dynamic switching and spatial continuity maintenance of multiple rendering modes, thereby improving rendering efficiency and enhancing the scalability and intelligent scheduling capabilities of the system. Summary of the Invention

[0005] To address these issues, we propose a hybrid rendering method based on spatial coding. This method schedules different rendering engines (NeRF, Gaussian, and traditional rendering) based on the coding during the rendering process, enabling flexible multi-engine scheduling and hybrid rendering within a unified scene. This method supports efficient identification of different scene regions and binding them to adaptive rendering strategies, improving rendering efficiency and reducing redundant computation. It is particularly suitable for large-scale neural scene rendering, cross-platform graphics systems, and edge computing / lightweight terminal rendering systems.

[0006] The technical problem solved by this application is achieved by the following technical solutions:

[0007] 1. Voxel division and hierarchical construction: The input 3D scene is spatially voxelized and a hierarchical voxel tree (such as an octree or a custom hierarchical voxel grid) is constructed to provide a basis for subsequent spatial encoding and policy binding.

[0008] 2. Spatial encoding and topological calculation: Based on the voxel structure, each node (vertex) in the space is uniquely encoded, thereby establishing a mapping relationship between three-dimensional coordinates and one-dimensional codes. This process includes the following sub-steps:

[0009] (1) Obtaining the global location information of the node:

[0010] Read the position information of each voxel. Each voxel has a number in the X, Y, and Z directions. Combined with the local position of the node in the voxel, the position of the node in the global coordinate system is calculated. For example, if a node is located on the positive side of the voxel's Z axis, the global number of the node in the Z direction is the voxel number + 1. The position numbers in the X and Y directions are obtained in the same way.

[0011] (2) Calculation of node code:

[0012] According to the global position of the node, the one-dimensional code Cp of the node is obtained through the following mapping function, realizing the mapping from three-dimensional coordinates to one-dimensional code. The mapping rules are as follows:

[0013]

[0014] Among them C p is the node code, x, y, z are the node's order in the X, Y, and Z directions respectively, q = 2 n -1 is the number of nodes in each column, n is the number of voxel divisions, and b is the local number of the node in the voxel, ranging from [0,7].

[0015] 3. Intelligent Binding of Rendering Strategies: This pre-processes all 3D objects in the scene using object-specific rendering strategies. Rendering strategies are bound to 3D objects in the scene, including but not limited to Neural Radiance Field (NeRF) rendering, Gaussian Splatting rendering, ray tracing rendering, and rasterization rendering. For example, a Gaussian Splatting rendering strategy is assigned to static background objects, while a rasterization rendering strategy is assigned to dynamic geometry in the scene.

[0016] 4. View-Dependent Rendering Scheduling: Combining view frustum clipping with the data structure provided by the voxel encoding, the view frustum calculated based on the view angle is clipped. First, voxel cells that intersect the view frustum are selected, and the points in these voxel cells are pre-sorted by voxel-level depth based on the camera orientation. The object set within the voxel is then identified and, based on the voxel's rendering strategy, fed into different rendering modules for concurrent execution. Each rendering module outputs a standard screen-space RGBA image buffer.

[0017] 5. Multi-strategy image synthesis output: The intermediate image results output by multiple rendering modules are fused and synthesized to generate the final image frame. This process supports the processing of different resolutions, occlusion relationships, and boundary transitions, ensuring the consistency and continuity of the final image.

[0018] The embodiments of the present application have the following beneficial effects:

[0019] This application uses a regular grid in space, so adjacent nodes / units can directly calculate new codes using spatial coordinates (x±1, y±1, z±1), and then quickly access the neighborhood using linear indices. This access is equivalent to coordinate offset, and can be quickly located using integer addition and subtraction. There is no need to store explicit adjacency relationships, and it supports exchange with external memory, so it can support hybrid rendering of large-scale scenes. By intelligently binding multiple rendering strategies under a unified coding framework, the most suitable rendering method can be flexibly scheduled based on the local features of the scene. Since the voxel size is much larger than the point scale, voxel-based depth and sorting are used to avoid resource waste and redundant calculations caused by unified rendering of the entire scene, significantly improving the overall rendering efficiency and performance of the system.

[0020] The above description is only an overview of the technical solution of this application. In order to more clearly understand the technical means of this application, which can be implemented in accordance with the contents of the description, and to make the above and other purposes, features and advantages of this application more obvious and easy to understand, the following preferred embodiments are specifically described in detail with reference to the accompanying drawings. It should be understood that the above general description and the detailed description below are only exemplary and explanatory and do not limit this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 A schematic diagram of a hybrid scene rendering process provided by an embodiment.

[0023] Figure 2 A schematic diagram of the voxel and vertex encoding order in hybrid scene space encoding provided by one embodiment.

[0024] Figure 3 An example diagram of linear encoding of spatial voxel vertices provided by an embodiment.

[0025] Figure 4 A schematic diagram of voxel-level depth pre-sorting in a viewing cone provided by one embodiment.

[0026] Figure 5 A schematic diagram of the depth synthesis process of dual-channel rendering results provided by one embodiment. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0028] like Figure 1 As shown, the spatial coding-based hybrid rendering method provided in this embodiment includes the following steps: scene input and preprocessing, spatial voxelization and coding construction, rendering strategy binding, view-dependent rendering scheduling, and multi-strategy image synthesis output. This method aims to achieve efficient collaboration among multiple rendering engines through spatial coding technology, enabling complex 3D scenes to adopt the most appropriate rendering strategy based on the characteristics of different regions, thereby improving rendering efficiency and visual quality.

[0029] In the first scene input and preprocessing phase, the system receives 3D scene model data, including geometry, material information, and lighting information. This model is typically represented using various scene resources, including splat files. The input model is then cleaned and normalized to ensure it meets subsequent processing requirements. Furthermore, a preliminary scene analysis is performed to identify areas or objects with special rendering requirements, such as high-detail areas, dynamically changing areas, or areas of frequent interaction.

[0030] In the second step of spatial voxelization and encoding construction, such as Figure 2 and Figure 3 As shown, the system divides the 3D scene space into a regular voxel grid. It first determines the appropriate voxel size and division accuracy, typically adaptively adjusting based on scene complexity and rendering requirements. It then constructs a hierarchical voxel structure, typically using an octree or similar spatial partitioning data structure to support spatial representations at different scales.

[0031] Then, spatial coding and topological calculation are used to encode each vertex in the voxel space, and a mapping relationship between three-dimensional coordinates and one-dimensional coding is established. Figure 2 As shown in Figure 1, vertex encoding is performed in a specific order to ensure that adjacent vertices also maintain adjacent relationships in the encoding space. The specific encoding process is as follows:

[0032] First, the global position of the node is obtained. Each node is located at a specific position within a specific voxel (local number b ranges from 0 to 7). Combined with the voxel's position information, the node's position in the global coordinate system is calculated. If the node is located on one side of the voxel (for example, the positive Z axis), the global number of the node in that direction is the voxel number plus 1.

[0033] Then, according to the global position of the node, the mapping function proposed in this invention is applied to calculate the one-dimensional code Cp of the node:

[0034]

[0035] Among them, x v 、y v 、z v are the global position numbers of the node in the X, Y, and Z directions, respectively, q = 2 n -1 is the number of nodes in each column, n is the number of voxel divisions, and b is the local number of the node in the voxel (0-7). This encoding method ensures that adjacent vertices in space are also adjacent in their encoding values, which facilitates fast search and access to neighborhood information.

[0036] In the third step of rendering strategy intelligent binding stage, such as Figure 4 As shown, the system analyzes and classifies the 3D objects in the scene and binds the most suitable rendering strategy to each object. The selection criteria of the rendering strategy include but are not limited to: the semantic category of the object (such as buildings, vegetation, people, etc.), geometric complexity, material characteristics, interaction importance, visual priority, etc. The specific binding process is as follows:

[0037] For highly detailed, static areas with complex lighting effects (such as fine buildings and sculptures), the Neural Radiance Field (NeRF) rendering strategy is prioritized to present high-quality visual effects.

[0038] For objects with a certain degree of complexity that need to support dynamic deformation (such as vegetation and cloth), bind the Gaussian Splatting rendering strategy to balance rendering quality and performance.

[0039] For objects that require precise light interaction and have complex materials (such as mirrors, transparent objects, etc.), bind the ray tracing rendering strategy.

[0040] For objects with relatively simple geometry or at the edge of the field of view, bind the traditional rasterization rendering strategy to optimize performance.

[0041] In addition, the system supports dynamic adjustment of rendering strategy bindings based on runtime conditions (such as hardware performance and frame rate requirements) to achieve a balance between rendering quality and performance. Rendering strategy binding information is stored in object metadata for quick access during the rendering scheduling phase.

[0042] In the fourth step, the perspective-related rendering scheduling stage, such as Figure 4 As shown, the system performs frustum clipping based on the current viewing angle parameters to determine the set of objects that may appear in the field of view. The specific steps are as follows:

[0043] Get the current main viewing angle parameters V, including camera position, orientation, field of view, etc.

[0044] The viewing frustum is constructed based on the viewing angle parameters and tested for intersection with the spatial boundaries (usually bounding boxes) of objects in the scene.

[0045] For objects clipped by the view frustum, they are grouped according to their bound rendering strategies to form task queues for different rendering channels.

[0046] Based on the spatial coding system, the spatial relationship between adjacent objects can be quickly identified, and possible occlusion relationships and transition areas can be processed.

[0047] To improve performance, this method allocates and schedules policies on an object-by-object basis, avoiding voxel-by-voxel feature analysis or neural inference. Furthermore, it leverages the locality of spatial encoding to optimize the rendering consistency of adjacent objects within the same region.

[0048] In the fifth step, multi-strategy image synthesis output stage, such as Figure 5 As shown in the figure, the system fuses and synthesizes the intermediate image results produced by each rendering module (NeRF, Gaussian Splatting, ray tracing, rasterization, etc.) to generate the final output image. The specific synthesis process includes:

[0049] Each rendering module performs its own rendering tasks in parallel and outputs a screen space RGBA image buffer containing color and depth information.

[0050] Resolve occlusion relationships between different rendering results based on depth information and object priority.

[0051] A smooth transition algorithm is applied at the boundaries of different rendering areas to avoid obvious visual breaks or discontinuities.

[0052] Apply post-processing effects (such as tone mapping, anti-aliasing, depth of field, etc.) to improve the visual quality of the final image.

[0053] In addition, the system also supports time-correlation optimization in multi-frame rendering, such as reusing the rendering results of the previous frame or applying temporal anti-aliasing technology to further improve rendering efficiency and stability.

[0054] The core advantage of this invention lies in its efficient organization and management of complex 3D scenes through spatial encoding technology, which in turn builds an intelligent scheduling mechanism for multiple rendering strategies. Because the space adopts a regular grid structure, adjacent nodes can be quickly calculated using simple coordinate offsets (x±1, y±1, z±1), eliminating the need to store explicit adjacency relationships. This fast positioning mechanism based on integer arithmetic significantly improves the efficiency of spatial queries and traversal.

[0055] In practical applications, this method is particularly suitable for the following scenarios:

[0056] Large-scale digital twin systems, where different rendering strategies can be applied to different areas (such as factory equipment, building interiors and exteriors, landscapes, etc.).

[0057] Metaverse and virtual reality platforms need to balance high-quality visual experience and real-time interactive performance requirements.

[0058] Heterogeneous computing environments, such as systems that combine cloud rendering and local rendering, can dynamically allocate rendering tasks based on computing resources.

[0059] In mobile and edge computing scenarios, intelligent rendering strategy selection is used to achieve the best rendering effect under limited hardware resources.

[0060] In summary, the spatial coding-based hybrid rendering method provided by the present invention achieves an optimal balance between rendering efficiency and visual quality by spatially encoding complex three-dimensional scenes and intelligently binding multiple rendering strategies, providing a new technical path for the next generation of graphics rendering systems.

Claims

1. A large scene point cloud rendering method based on voxel coding, characterized in that: The following steps are involved: In the scene construction phase, the scene space is divided into voxels, the scene content is marked as static structure and dynamic structure, and stored in the corresponding voxel units according to the spatial position mapping, and each voxel unit is assigned a unique voxel code; During the rendering phase, the frustum range of the current frame is calculated based on the camera parameters; Based on the interaction between the view frustum and the voxel bounding box, the point cloud data in the relevant voxel units within the view frustum range is filtered and loaded into the memory; Perform Gaussian splash rendering on the static structures in the loaded scene data, and use rasterization rendering for the dynamic structures, outputting the corresponding color buffer and depth buffer respectively; The two types of rendering results are synthesized pixel by pixel based on the depth information to generate the final image output.

2. The method for rendering large scene point clouds based on voxel coding according to claim 1, characterized in that: The voxel division adopts a cube grid structure of uniform size, and the position of the voxel unit vertex in the three-dimensional space is mapped to a one-dimensional index value through encoding.

3. The method for rendering large scene point clouds based on voxel coding according to claim 2, characterized in that: The voxel encoding is calculated based on the global position of the node in the three-dimensional grid. Cp satisfies the following formula: Among them, x v 、y v 、z v They represent the numbers of node v in the X, Y, and Z axes, respectively, starting from 0. q is the number of nodes in each column, and q = 2n-1, where n is the number of partitions.

4. A large scene point cloud rendering method based on voxel coding according to claim 1, characterized in that: The step of screening voxels includes: using a camera frustum clipping algorithm to detect whether each voxel intersects with the frustum, and only screening the overlapping voxel units for loading.

5. The method for rendering large scene point clouds based on voxel coding according to claim 1, characterized in that: The Gaussian splash rendering includes: Dynamically generate screen-space Gaussian kernels for each overlapping point cloud in voxel units; Perform Gaussian weighted accumulation rendering in screen space; Outputs a buffer containing color and depth information.

6. The method according to claim 1, wherein The standard geometry rendering includes: Rendering is based on the standard lighting model for the triangular face expression of the geometric body; Generate independent color buffer and depth buffer.

7. The method according to claim 1, wherein The depth synthesis step includes: Compare the pixel-level depth values ​​of the color and depth buffers of Gaussian splat rendering and standard geometry rendering; The color value with the smaller depth value of each pixel is selected as the final output pixel.

Images