Methods and systems for node-based volumetric data generation and real-time rendering for DCC and Unreal Engine

Abstract

A method and system for node-based volumetric data generation and real-time rendering for DCC and Unreal Engine is disclosed. The method includes the following steps: creating and editing volumetric data based on the OpenVDB library in an external node-based editing tool; converting the edited OpenVDB volumetric data into a linearized NanoVDB buffer in real time; transferring the NanoVDB buffer from the external node-based editing tool to the UE5 plugin within the Unreal Engine via a memory-mapped file; receiving the NanoVDB buffer and uploading it to the structured buffer of the graphics processor; and performing real-time sampling and rendering of the volumetric data in the structured buffer within the Unreal Engine. The system includes a node-based volumetric editing module, a data format conversion module, a real-time data bridging module, and an engine-side rendering integration module. This invention directly utilizes UE5's native advanced rendering features such as Lumen and volumetric fog, solving the fundamental problem of poor compatibility in existing solutions.

Description

Technical Field

[0001] This invention relates to the fields of computer graphics and real-time rendering technology, specifically to a method and system for generating, processing, and rendering volumetric data (VDB) in real-time within the Unreal Engine game engine on an external platform. Background Technology

[0002] In film and television special effects and high-end game production, the Open Voxel Database format has become the standard for representing volumetric effects such as smoke, clouds, and flames due to its high efficiency and sparseness. However, integrating this format into a real-time rendering engine with Voxel Database as its core faces significant challenges.

[0003] Currently, there is an independent volumetric rendering pipeline implementation solution that bypasses the native sparse voxel texture system of UES, namely the ZibraVDB solution. This solution requires the implementation of a complete volumetric rendering engine from scratch, which is extremely difficult to develop and is difficult to deeply integrate and interact with the rendering pipeline characteristics of UES (such as Lumen global illumination and volumetric fog), resulting in poor compatibility and limited performance optimization.

[0004] Therefore, the industry needs a volumetric data processing solution that can be deeply integrated with the UES native rendering pipeline, has high development efficiency, and can make full use of hardware performance. Summary of the Invention

[0005] The technical problem to be solved by this invention is to overcome the difficulties in integrating volume rendering schemes with UES engines, the high development complexity, and the inability to fully utilize native rendering features in the existing technology.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: A node-based volumetric data generation and real-time rendering method for Unreal Engine, the core of which is: The NanoVDB format is used as the core data exchange medium at runtime, and a self-developed UE plugin, named "YWN VDB Importer" in this application, is used to realize real-time data synchronization between external tools and the engine.

[0007] The method includes the following steps: S1: In an external node-based editing tool, volume data is created and edited based on the OpenVDB library; S2: Converts the edited OpenVDB volume data into a linearized NanoVDB buffer for the GPU in real time; S3: Transfer the NanoVDB buffer from the external node-based editing tool to the UE5 plugin within the Unreal Engine via a memory-mapped file; S4: The UE5 plugin within the Unreal Engine receives the NanoVDB buffer and uploads it to the structured buffer of the graphics processor; S5: In the Unreal Engine, the volume data in the structured buffer is sampled and rendered in real time.

[0008] Furthermore: the UE5 plugin workflow is as follows: After receiving the data, the UE5 plugin directly uploads the NanoVDB data to the structured buffer of the graphics processor GPU through internal mechanisms (such as compute shaders and RDG). Real-time rendering: Finally, visual rendering is completed in the UE5 plugin.

[0009] Furthermore, to complete the visual rendering, the system provides two methods: Method 1: HLSL Ray Stepping: In the material editor, call the NanoVDB sampling function through a custom HLSL node to directly sample and render the data in the structured buffer in real time; Method 2: Native System Integration: Unpack the NanoVDB data on the GPU and inject it directly into UE5's native Volume Texture system for rendering.

[0010] Furthermore, the real-time sampling and rendering of volume data described in step S5 specifically includes: In the Unreal Engine material editor, the NanoVDB sampling function is called through a custom HLSL node to perform real-time sampling and ray step rendering of the data in the structured buffer.

[0011] Furthermore, the real-time sampling and rendering of volume data described in step S5 specifically includes: The NanoVDB data in the structured buffer is unpacked and injected into Unreal Engine's native volumetric texture system for rendering.

[0012] Furthermore, the conversion of OpenVDB data into a NanoVDB buffer in step S2 and the rendering sampling in step S5 are both processed based on Compute Shader.

[0013] Furthermore, step S2 also includes the step of introducing NeuralVDB to compress the data. The specific steps are as follows: Step 1: On the CPU side, OpenVDB is converted to NanoVDB for data dimensionality reduction. The system first needs to call the NanoVDB converter API. By passing the original OpenVDB grid (such as openvdb::FloatGrid) to the nanovdb::createNanoGrid() function, the system will traverse the original dynamic tree topology and flatten and freeze it. After the conversion is completed, a GridHandle object will be returned. This object manages a purely contiguous (Linearized) raw memory buffer containing all voxel and node data at the underlying level. The second step is high-speed cross-process IPC transfer. Since this large amount of data is generated in an external tool (such as VDB Maker), this continuous memory stream is written to the operating system-level memory-mapped file. The system uses Win32 APIs (such as CreateFileMapping and MapViewOfFile) to allocate a shared area in memory and seamlessly transfer this continuous byte stream to the Unreal Engine UE5 plugin process. Step 3: Allocating a Structured Buffer in UE5 using RDG (GPU Memory Allocation) In the UE5 rendering thread, after the plugin receives this memory pointer, it takes over the data and prepares to upload it to the graphics card's video memory (VRAM). To manage video memory safely and efficiently, developers need to use UE5's Render Dependency Graph (RDG) API. By calling GraphBuilder.CreateBuffer() in conjunction with an upload descriptor (such as FRDGBufferDesc::CreateUploadDesc), or by using the CreateStructuredBuffer helper function, a one-dimensional structured buffer of matching type is allocated on the GPU.

[0014] Step 4: Creating and Binding the Shader Resource View (SRV Binding) After the data is uploaded to the GPU buffer, the shader must be told how to read it. In C++, the system calls GraphBuilder.CreateSRV() to generate a Shader Resource View (SRV) for the buffer. The data format is typically specified as a 32-bit unsigned integer (e.g., PF_R32_UINT). Then, using the engine's macro definition SHADER_PARAMETER_RDG_BUFFER_SRV, this SRV is securely bound to the global shader parameters of the rendering pipeline.

[0015] Step 5: Direct Resolution of HLSL Material Nodes (GPU Real-Time Stepping) Finally, in UE5's Material Editor (Custom Node) or Custom Compute Shader, the HLSL code will include the pointer-less resolution library PNanoVDB.h for NanoVDB via the #include directive. The shader maps the underlying reader type to the structured buffer that was just bound (#define pnanovdb_buf_t StructuredBuffer). <uint>This allows thousands of concurrent GPU threads to directly use integer offsets to quickly query the density or temperature of voxels in contiguous memory blocks, thereby completing real-time volumetric ray stepping rendering.

[0016] The present invention also includes a system for implementing the above method, specifically comprising the following modules: Node-based volume editing module: used to generate and edit OpenVDB format volume data; Data format conversion module: used to convert OpenVDB data into NanoVDB buffers; Real-time data bridging module: a plugin developed based on the UE5 framework (named "YWN VDB Importer" in this application), used to receive and transmit the NanoVDB buffer to the Unreal Engine; Engine-side rendering integration module: Deployed within the UE5 engine, it is used to send the received NanoVDB data to the GPU memory and integrate it into the material system or native volumetric rendering pipeline for rendering.

[0017] Furthermore, the real-time data bridging module uses a memory-mapped file to transfer the NanoVDB buffer between the external tool process and the Unreal Engine process; the engine-side rendering integration module is specifically used to upload NanoVDB data to the structured buffer of the graphics processor.

[0018] Furthermore, the engine-side rendering integration module directly samples and renders the data in the structured buffer through HLSL custom nodes; the engine-side rendering integration module unpacks the NanoVDB data in the structured buffer and injects it into the Unreal Engine's native volumetric texture system for rendering.

[0019] The beneficial effects of this invention are at least one or more of the following: 1. Deep integration: Through self-developed plugins named "YWN VDB Importer" and NanoVDB, seamless and real-time integration between external volume tools and the UES rendering pipeline is achieved. It can directly utilize UE5's native advanced rendering features such as Lumen and volumetric fog, solving the fundamental problem of poor compatibility of existing solutions.

[0020] 2. Highly efficient development: Based on the open-source NanoVDB standard, there is no need to develop proprietary compression algorithms and rendering pipelines from scratch, which greatly reduces development difficulty and maintenance costs, in line with an efficient development strategy.

[0021] 3. Superior performance: NanoVDB's linear memory layout is well-suited for GPU random access, avoiding the complex pointer jumps of traditional VDB and providing extremely high sampling performance.

[0022] 4. High scalability: The architecture reserves interfaces for future integration of advanced compression technologies such as NeuralVDB, which is expected to enable the use of extremely small storage space to carry extremely detailed volume data.

[0023] Table 1 below is a comparison table of the prior art and the technology of this invention: Table 1. Evaluation Comparison Table Evaluation Dimensions Traditional VDB usage patterns (such as transferring 3D textures or third-party standalone pipelines) A new model based on NanoVDB and self-developed plugins Efficiency Improvement and Advantages Explanation Data Structures and GPU Affinity Dynamic tree structures containing a large number of memory pointers result in extremely high GPU dereferencing latency. Compacted, linearized, pointerless continuous read-only data stream Completely eliminates GPU addressing bottlenecks, significantly improving cache hit rate and read performance. Video RAM (VRAM) and memory overhead When large-scale textures are converted into dense volumetric textures, empty areas will also consume video memory resources. Extremely low memory usage, preserving the sparsity of data while significantly reducing memory consumption. Freeing up significant CPU and GPU resources for other computations allows for loading more complex effects in real-time environments. Data cross-process synchronization efficiency Low cost, typically relying on baking to export sequence frame assets or traditional network transmission. Highly efficient, data transfer is performed via operating system-level memory-mapped files. Achieving near-zero-copy high-bandwidth transmission ensures complete real-time synchronization between external tools and the engine. Engine pipeline integration Existing solutions (such as ZibraVDB) require the development of a separate renderer and are difficult to integrate with native lighting. Excellent, with deep integration into the UE5 native pipeline through structured buffers and HLSL custom nodes. It avoids the high difficulty of developing a rendering engine from scratch and can seamlessly interact with Lumen global illumination and volumetric fog to enhance the realism of the images. Attached Figure Description

[0024] Figure 1 This is a diagram of the overall system architecture of the present invention; Figure 2 This is a flowchart of the core data processing. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will describe the embodiments of the present invention in detail.

[0026] The core implementation process of this invention is as follows: Figure 1 As shown (the external node tool - platform name is "VDB Maker"), it serves as the creation end, and UE5 serves as the rendering and running end.

[0027] The creation and dimensionality reduction transformation stage (VDB Maker side) is the core of which is to remove heavy physical calculations from the rendering engine and complete the data format conversion to adapt to the modern graphics card architecture.

[0028] S1: Topology Creation and Simulation In the node-based tools of the externally developed platform "VDB Maker," the system first performs real-time physical simulation, creation, and editing of volumetric data such as smoke and flames based on the OpenVDB library. At this stage, the data maintains the OpenVDB's Sparse Dynamic Tree structure. The problem lies in the fact that traditional OpenVDB uses a hierarchical group of nodes with a root, internal, and leaf structure. Node data is scattered throughout memory, connected by numerous memory pointers. This discrete addressing leads to extremely high latency and cache misses on modern GPUs.

[0029] S2: Linearization Dimensionality Reduction and Reorganization To address the aforementioned pain points, the system freezes the dynamic topology of OpenVDB, converting it from a tree structure to a linearized, GPU-friendly NanoVDB buffer in real time. This process is based on Compute Shader to ensure the efficiency of GPU computation. Operating principle: The conversion engine compresses and quantizes the tree topology into a continuous byte stream. All dynamic memory pointers are erased and replaced with "integer offsets" relative to the base address. Performance benefits: The converted NanoVDB read-only buffer structure is extremely compact. Thousands of GPU threads can achieve O(1) level speed hits on target voxels by calculating one-dimensional offsets on a continuous memory bus. II. Zero-copy high-speed transmission stage (IPC communication) S3: Operating System Level Memory Mapping The system uses a memory-mapped file (MMF) to directly transfer the converted NanoVDB buffer from the tool to the UE5 plugin "YWN VDB Importer". How it works: This step completely bypasses the traditional file export / import process to disk. Writing the reduced, continuous byte stream to the MMF achieves "zero-copy" high-speed transfer between the external tool and the UE5 main process. The UE main process directly takes over this shared memory segment, effectively bypassing the performance limitations of traditional engine inter-process communication (IPC). III. Engine Reception and Memory Scheduling Stage (UE5 Plugin Side) S4: Non-blocking video memory allocation The "YWN VDB Importer" plugin within the UE5 engine, after capturing shared memory data, directly uploads it to the graphics processor's structured buffer. This step is also based on ComputeShader processing. How it works: The plugin uses UE5's internal Render Dependency Graph (RDG) mechanism to schedule resources. Since the input data is already prepared as "one-dimensional continuous read-only data," the plugin sends the data directly to the graphics card's structured buffer in a non-blocking manner, completely eliminating the overhead of secondary parsing and reassembly on the CPU side. IV. Real-time Rendering Stage (UE5 Materials and Pipeline) After entering the UE5 rendering pipeline, the system provides two parallel data processing paths to accommodate different art requirements and rendering scenes.

[0030] S5 Path A: HLSL Ultra-Fast Direct Sampling In the UE5 Material Editor, NanoVDB sampling functions are directly invoked via HLSL custom nodes to perform real-time sampling and rendering of volume data in the structured buffer. The operating principle is as follows: the PNanoVDB.h header file is called in the MaterialGraph, and the one-dimensional buffer is directly queried by calculating integer offsets. This method executes an efficient advanced raymarching algorithm and perfectly integrates the volumetric transmission and scattering results into UE5's Lumen global illumination system.

[0031] S5 Path B: Native Pipeline Unpacking and Injection NanoVDB data is unpacked and injected into the UE5 native volume texture system for rendering. How it works: The Compute Shader expands the encrypted / compressed data in the structured buffer and forcibly injects it into Unreal Engine's native dense volume texture. This mechanism is primarily used to enable physical interaction between the data and the scene's volumetric fog, or for backward compatibility with older rendering pipelines and the Niagara particle system.< / uint>

Claims

1. A node-based volumetric data generation and real-time rendering method for DCC and Unreal Engine, characterized in that, Includes the following steps: S1: In an external node-based editing tool, volume data is created and edited based on the OpenVDB library; S2: Converts the edited OpenVDB volume data into a linearized NanoVDB buffer for the GPU in real time; S3: Transfer the NanoVDB buffer from the external node-based editing tool to the UE5 plugin within the Unreal Engine via a memory-mapped file; S4: The UE5 plugin within the Unreal Engine receives the NanoVDB buffer and uploads it to the structured buffer of the graphics processor; S5: In the Unreal Engine, the volume data in the structured buffer is sampled and rendered in real time.

2. The UE5 plugin workflow according to claim 1 is as follows: GPU memory upload: After receiving the data, the UE5 plugin directly uploads the NanoVDB data to the structured buffer of the graphics processor GPU through an internal mechanism. Real-time rendering: Finally, visual rendering is completed in the UE5 plugin.

3. According to claim 2, the system provides two methods for completing visual rendering: Method 1: HLSL Ray Stepping: In the material editor, call the NanoVDB sampling function through a custom HLSL node to directly sample and render the data in the structured buffer in real time; Method 2: Native System Integration: Unpack the NanoVDB data on the GPU and inject it directly into UE5's native volumetric texture system for rendering.

4. The method according to claim 1, characterized in that, The real-time sampling and rendering of volume data described in step S5 specifically includes: In the Unreal Engine material editor, the NanoVDB sampling function is called through a custom HLSL node to perform real-time sampling and ray step rendering of the data in the structured buffer.

5. The method according to claim 1, characterized in that, The real-time sampling and rendering of volume data described in step S5 specifically includes: The NanoVDB data in the structured buffer is unpacked and injected into Unreal Engine's native volumetric texture system for rendering.

6. The method according to claim 1, characterized in that, The conversion of OpenVDB data into a NanoVDB buffer in step S2 and the rendering sampling in step S5 are both processed based on Compute Shader.

7. The method according to claim 1, characterized in that, Step S2 also includes the step of introducing NeuralVDB to compress the data. The specific steps are as follows: Step 1: On the CPU side, OpenVDB is converted to NanoVDB for data dimensionality reduction. The system first needs to call the NanoVDB converter API. By passing the original OpenVDB grid to the nanovdb::createNanoGrid() function, the system will traverse the original dynamic tree topology and flatten and freeze it. After the conversion is completed, a GridHandle object will be returned. This object manages a pure contiguous raw memory buffer containing all voxel and node data at the underlying level. Step 2: High-speed cross-process IPC transfer. Since this volume of data is generated in an external tool, this continuous memory stream will be written to the operating system-level memory-mapped file. The system uses the Win32 API to allocate a shared area in memory and seamlessly transfers this continuous stream of bytes to the Unreal Engine UE5 plugin process; Step 3: Allocate a structured buffer in UE5 using RDG—GPU memory allocation occurs within UE5's rendering thread. Once the plugin receives this memory pointer, it takes over the data and prepares to upload it to the GPU's VRAM. To manage VRAM securely and efficiently, developers need to use UE5's Render Dependency Graph (RDG) API. This can be done by calling GraphBuilder.CreateBuffer() in conjunction with an upload descriptor, or by using the CreateStructuredBuffer helper function, to allocate a one-dimensional structured buffer of the matching type on the GPU. Step 4: After creating the shader resource view and binding the parameter data to the GPU buffer, you must tell the shader how to read it. In C++, the system calls GraphBuilder.CreateSRV() to generate a shader resource view for the buffer, and the data format is usually specified as a 32-bit unsigned integer. Subsequently, using the engine's macro definition SHADER_PARAMETER_RDG_BUFFER_SRV, this SRV is safely bound to the global shader parameters of the rendering pipeline; Step 5: Direct Parsing of HLSL Material Nodes – GPU Real-Time Stepping Finally, in the UE5 material editor or custom compute shader, the HLSL code will include the pointer-free parsing library PNanoVDB.h of NanoVDB via the #include directive; the shader will then map the underlying reader type to the structured buffer that was just bound – #define pnanovdb_buf_t StructuredBuffer <uint> This allows thousands of concurrent GPU threads to directly use integer offsets to quickly query the density or temperature of voxels in consecutive memory blocks, thereby completing real-time volumetric ray stepping rendering.< / uint> 8. A system for implementing the method as described in any one of claims 1-7, characterized in that, include: Node-based volume editing module: used to generate and edit OpenVDB format volume data; Data format conversion module: used to convert OpenVDB data into NanoVDB buffers; Real-time data bridging module: A plugin developed based on the UE5 framework, used to receive and transmit the NanoVDB buffer to the Unreal Engine; Engine-side rendering integration module: Deployed within the UE5 engine, it is used to send the received NanoVDB data to the GPU memory and integrate it into the material system or native volumetric rendering pipeline for rendering.

9. The system according to claim 8, characterized in that, The real-time data bridging module uses a memory-mapped file to transfer the NanoVDB buffer between the external tool process and the Unreal Engine process; the engine-side rendering integration module is specifically used to upload NanoVDB data to the structured buffer of the graphics processor.

10. The system according to claim 8, characterized in that, The engine-side rendering integration module directly samples and renders the data in the structured buffer through HLSL custom nodes; the engine-side rendering integration module also renders the NanoVDB data in the structured buffer by unpacking it and injecting it into the Unreal Engine's native volumetric texture system.

Images