A general global neural rendering method and system based on neural voxel representation

Abstract

The application discloses a general global neural rendering method and system based on neural voxel representation, and belongs to the technical field of computer graphics, which comprises the following steps: voxelizing a three-dimensional scene to generate a three-dimensional voxel grid carrying physical properties, and then injecting initial light to form a three-dimensional voxel field carrying initial radiant energy; performing two-dimensional feature calculation along orthogonal three planes alternately in the three-dimensional voxel field to implicitly simulate indirect light transmission, and outputting an indirect light feature field; extracting a screen space geometry buffer to obtain three-dimensional world coordinates and high-frequency features of each pixel; sampling voxel-level features from the indirect light feature field according to the three-dimensional world coordinates, coupling the high-frequency features, and then outputting indirect radiant luminance through neural network decoding, and finally superimposing direct light to complete global neural rendering. The application can significantly improve the visual authenticity and physical accuracy of complex indirect light while ensuring real-time rendering efficiency, and realizes general global neural rendering across scenes and under different lighting conditions.

Description

Technical Field

[0001] This invention belongs to the field of computer graphics technology, specifically relating to a general global neural rendering method and system based on neurovoxel representation. Background Technology

[0002] In recent years, neural rendering has gradually evolved into a fundamental technology that promotes the deep integration of high-quality image synthesis and intelligent graphics expression. This technology, leveraging a deep learning architecture, enables rendering systems to autonomously uncover deep coupling mappings between lighting, materials, and geometric topology within a scene, thereby achieving image output that combines high realism with high execution efficiency. Conventional physically based graphics rendering pipelines heavily rely on analytical mathematical and physical modeling. While this ensures strong generalization capabilities and parameter controllability, it often suffers from computational bottlenecks and frame rate limitations when dealing with complex scenes involving intricate global illumination, fine micro-geometry, and high-order reflection properties. In contrast, neural rendering technology fully leverages the advantages of deep neural networks in representation extraction and nonlinear function approximation, reconstructing the originally complex image generation process into an end-to-end data-driven process. This not only improves visual continuity and image realism across viewpoints but also allows the system to present rendering results comparable to physically realistic levels while significantly reducing computational consumption.

[0003] In the mainstream graphics engine ecosystem, deep learning has been introduced into the core sub-stage of classic rendering pipelines to achieve a better balance between visual fidelity and computational efficiency. A typical example is neural network-based oversampling techniques (such as DLSS and FSR), which utilize network models supplemented by multi-dimensional features such as motion vectors, depth buffers, and surface normals to upscale and reconstruct low-resolution background images into ultra-high-definition images. This significantly reduces GPU rendering pressure while accurately restoring high-frequency information and ensuring temporal continuity. AI noise reduction algorithms are also being deployed on a large scale in low-sampling-rate scenes with global illumination and path tracing. By extracting and learning the inherent statistical features of scene radiance distribution, they efficiently filter out random noise generated by Monte Carlo integration, significantly reducing the convergence period required to generate high-quality unbiased images. Furthermore, neural material synthesis relies on network models to reverse-engineer real-world shooting data, automatically extracting physical properties such as surface roughness, diffuse reflectance, and normals, thus facilitating the automated generation and stylized reconstruction of materials. Meanwhile, view synthesis combines reprojection and information restoration mechanisms within the feature space to infer missing intermediate frames or interpolate transitions, enhancing the smoothness of dynamic scenes and the expressiveness of camera travel. Most of these deep learning solutions are integrated into existing rendering frameworks in a plug-in format, balancing performance and rendering quality without disrupting the core rendering logic.

[0004] Unlike the auxiliary deep learning methods mentioned earlier that aim to "enhance processing," the core of pure neural rendering mechanisms lies in solving the rendering integral equation end-to-end. That is, relying on neural networks, it directly predicts the final radiance of the target pixel based on given scene input information. This architecture abandons the cumbersome explicit construction process of light transmission paths and complex bidirectional reflectance distribution functions (BRDFs) in traditional computer graphics, instead relying on the strong nonlinear approximation characteristics of deep networks to directly fit the light energy transmission distribution. Compared to traditional analytical physical models, neural rendering can naturally encompass high-dimensional, nonlinear, and complex lighting interactions, and can adapt to varying surface material changes and indirect light diffusion phenomena, thus unleashing superior representation potential in specific scenes. Ultimately, as a learnable alternative to traditional rasterization or ray tracing pipelines, neural rendering has been widely applied to computationally expensive lighting approximation tasks such as global illumination and soft physical shadows, opening up a promising acceleration path to break through the performance limits of real-time rendering.

[0005] While current neural rendering techniques have demonstrated excellent visual quality in certain specific applications or limited scenarios, their inherent limitations remain significant. Specifically, these methods are highly dependent on the distribution characteristics of prior datasets. When faced with material properties, new perspectives, or dynamic lighting conditions not encountered during the network's training phase, the stability and accuracy of their outputs deteriorate considerably. Furthermore, most current mainstream network architectures are designed to fit single tasks or static input patterns, often revealing severe generalization bottlenecks when dealing with complex and varied lighting environments and cross-scene transitions. More critically, current neural rendering schemes generally lack a complete representation of the global structure and spatial matter of 3D scenes. This prevents the algorithms from effectively perceiving and simulating the long-distance transmission and multiple bounces of light in physical space, further limiting their generalization and extrapolation capabilities in tasks involving global indirect lighting prediction.

[0006] Therefore, there is an urgent need to build a new neural rendering architecture that combines high generalization potential with modular reusability, so as to realize the transformation of neural graphics technology from the traditional "task-specific" mode to a new stage of "general physical rendering system", and lay a solid theoretical and technical foundation for the underlying architecture of the next generation of intelligent light and shadow computing. Summary of the Invention

[0007] In view of the above, the purpose of this invention is to provide a general global neural rendering method and system based on neurovoxel representation. By performing voxelization processing on a complete 3D scene to aggregate structured spatial geometric features and simultaneously injecting the initial radiant energy of the light source, a unified initial 3D voxel field containing the scene's physical properties and energy distribution is constructed. On this basis, a neural network is used to perform deep information extraction and spatiotemporal correlation modeling to construct an implicit light transmission evolution process simulation mechanism. This enables accurate prediction of global indirect lighting effects under different scene layouts and light source configurations. Thus, while ensuring real-time rendering, the visual realism and universality of the neural rendering system in complex light and shadow interaction tasks are significantly improved. It is suitable for application scenarios that require efficient, universal, and realistic global illumination rendering, such as game development, virtual reality, film and television special effects, and real-time visualization.

[0008] To achieve the above-mentioned objectives, the present invention provides the following technical solution: In a first aspect, embodiments of the present invention provide a general global neural mapping method based on neurovoxel representation, comprising the following steps: The 3D scene is voxelized to generate a 3D voxel mesh carrying physical properties; Initial illumination is injected into the three-dimensional voxel mesh to form a three-dimensional voxel field carrying initial radiation energy; Two-dimensional feature calculations are performed alternately along three orthogonal planes within a three-dimensional voxel field, enabling the exchange and long-distance transmission of radiant energy, implicitly simulating indirect light transmission, and outputting an indirect illumination feature field. Extract the screen space geometry buffer to obtain the three-dimensional world coordinates and high-frequency features of each pixel; Based on three-dimensional world coordinates, voxel-level features are sampled from the indirect lighting feature field, coupled with high-frequency features, and then decoded by a neural network to output indirect radiance. This is then superimposed with direct lighting to complete global neural rendering.

[0009] Preferably, the voxelization process includes: The scene is rendered using orthogonal rasterization along three mutually orthogonal projection directions. The physical properties of the scene surface are extracted, including space occupancy, surface normal distribution, and basic material properties. The extracted information is then mapped onto a 3D voxel mesh.

[0010] Preferably, the injected initial illumination includes explicit physical radiation injection: Render the scene from the perspective of the light source to generate a reflection and shadow map; Traverse the three-dimensional voxel mesh, and perform projection and visibility depth tests on the reflection shadow map based on the three-dimensional coordinates of each voxel to determine whether the voxel is directly illuminated. For a voxel that is directly illuminated, the initial radiation energy is sampled from the reflection shadow map and fused with the material reflection properties of the voxel itself to obtain the voxel feature carrying the direct illumination energy.

[0011] Preferably, the injected initial illumination includes implicit feature radiation injection: The light source attributes in the scene are input into a pre-trained light source encoding network to generate an orthogonal three-plane feature field covering the scene bounding box; Traverse the three-dimensional voxel mesh, project the three-dimensional coordinates of each voxel onto the three planes of the orthogonal three-plane feature field and sample bilinear interpolation features, and aggregate the sampling results into illumination features; By fusing the lighting features with the geometric and material features of the voxel itself, voxel features carrying implicit lighting information are obtained.

[0012] Preferably, the step of alternately performing two-dimensional feature calculations along three orthogonal planes within a three-dimensional voxel field includes: Maintaining the complete grid tensor structure of the three-dimensional voxel field, two-dimensional neural network operators are independently applied to each plane slice along the XY plane, XZ plane, and YZ plane to perform feature interaction and aggregation, and the calculation is iteratively performed alternately along the three directions.

[0013] Preferably, the high-frequency features include pixel-level geometric information and physical material parameters. The geometric information includes at least depth information, world coordinates, and surface normals, and the physical material parameters include at least diffuse reflectance and roughness.

[0014] Preferably, voxel-level feature sampling is performed using trilinear interpolation, and the coupled features are decoded using a multilayer perceptron to output indirect radiance.

[0015] Secondly, embodiments of the present invention also provide a general global neural rendering system based on neurovoxel representation, which is implemented using the above-mentioned general global neural rendering method based on neurovoxel representation, including: a scene voxelization module, a lighting injection module, a scene encoding module, a screen space information extraction module, and a lighting decoding module; The scene voxelization module is used to perform voxelization processing on the three-dimensional scene to generate a three-dimensional voxel mesh carrying physical properties. The illumination injection module is used to inject initial illumination into the three-dimensional voxel mesh to form a three-dimensional voxel field carrying initial radiation energy. The scene encoding module is used to perform two-dimensional feature calculations alternately along three orthogonal planes in a three-dimensional voxel field, so that radiant energy can be exchanged and transmitted over long distances, implicitly simulating indirect light transmission and outputting an indirect illumination feature field. The screen space information extraction module is used to extract the screen space geometric buffer to obtain the three-dimensional world coordinates and high-frequency features of each pixel. The illumination decoding module is used to sample voxel-level features from the indirect illumination feature field according to the three-dimensional world coordinates, and after coupling with high-frequency features, it is decoded by a neural network to output indirect radiance, which is then superimposed with direct illumination to complete global neural rendering.

[0016] Thirdly, embodiments of the present invention also provide an electronic device, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-described general global neural mapping method based on neurovoxel representation when executing the computer program.

[0017] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-described general global neural mapping method based on neurovoxel representation.

[0018] This invention proposes a general-purpose global neural rendering framework that combines high generalization and high execution efficiency, significantly overcoming the bottlenecks of existing neural rendering technologies that heavily rely on overfitting within a single scene and the enormous computational overhead of traditional global optical rendering. Compared with existing technologies, the beneficial effects of this invention include at least the following: By voxelizing the 3D scene and injecting initial lighting, a 3D voxel field containing spatial geometry and lighting priors is constructed, thereby endowing the model with true global spatial perception and cross-scene generalization capabilities. At the same time, an innovative multi-dimensional orthogonal dimensional reduction implicit transmission simulation is adopted. By alternately performing 2D feature calculations along three orthogonal planes in the 3D voxel field, radiated energy is exchanged and transferred over long distances, implicitly simulating indirect light transmission. This allows for efficient completion of long-distance light energy evolution in the implicit space, avoiding expensive explicit ray intersection calculations and significantly reducing computational overhead. Finally, by precisely coupling the evolved low-frequency global voxel features with the microscopic geometric details of the high-resolution screen space across dimensions, a highly modular end-to-end indirect lighting solution is formed that is easy to seamlessly integrate into existing graphics pipelines while ensuring high-fidelity rendering and physical accuracy. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the general global neural mapping method based on neurovoxel representation provided in an embodiment of the present invention. Figure 2 This is a schematic diagram showing the internal details of the implicit feature radiation injection and output of the indirect illumination feature field provided in an embodiment of the present invention; Figure 3 This is a comparison chart of the drawing results of other existing advanced methods of this invention; Figure 4 This is a graph showing the generalization test results of the present invention under different scenarios and different light source conditions; Figure 5 This is a schematic diagram of the framework of a general global neural mapping system based on neurovoxel representation provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0022] The inventive concept of this invention is as follows: Addressing the difficulties in indirect lighting prediction and poor cross-scene adaptability caused by the lack of effective representation of global features in 3D scenes in existing technologies, this invention provides a universal global neural rendering method and system based on neural voxel representation. Building upon the powerful nonlinear mapping capabilities of deep neural networks, it embeds a rendering function representation with parameterized characteristics, thereby integrating the physical reflection laws of materials and the spatial lighting transmission mechanism within a globally generated theoretical framework. Through voxelization processing and initial lighting injection, a 3D voxel field containing spatial geometry and lighting priors is constructed. Two-dimensional feature calculations are then performed alternately along three orthogonal planes to achieve the exchange and long-range transmission of radiated energy, implicitly simulating indirect light transmission. Finally, by deeply analyzing high-dimensional light energy distribution and complex surface scattering behavior, the limitations of single-data fitting are overcome, thereby enabling adaptive evolution under diverse scene layouts and dynamic light source configurations, achieving universal neural rendering across scenes and lighting conditions.

[0023] like Figure 1 As shown, this embodiment provides a general global neural rendering method based on neurovoxel representation. Built on the Unreal Engine platform, it utilizes the engine's rendering pipeline and hardware acceleration capabilities, combined with deep learning inference, to achieve end-to-end global neural rendering. The specific implementation steps are as follows: S1 performs voxelization on the 3D scene to generate a 3D voxel mesh carrying physical properties.

[0024] In this embodiment, this step aims to transform complex 3D scene geometry data into a regular spatial structured representation. First, the 3D scene (i.e., the target rendering space) is voxelized. This is achieved by setting three mutually orthogonal orthogonal projection cameras outside the scene's bounding box, and performing orthogonal rasterization rendering of the scene geometry along the X, Y, and Z axes of the world coordinate system. During the orthogonal rasterization rendering stage, physical properties of the scene surface are extracted, such as space occupancy, surface normal distribution, and basic material properties. The 2D feature slices captured in these three orthogonal directions are then mapped and injected into pre-allocated 3D textures or structured buffers, thereby rapidly constructing a dense 3D voxel mesh with physical and material information.

[0025] By establishing this three-dimensional voxel mesh with clear topological relationships, a robust geometric framework is provided for subsequent deep feature evolution, enabling the perception of the global physical layout of the scene.

[0026] S2, inject initial illumination into the three-dimensional voxel grid to form a three-dimensional voxel field carrying initial radiation energy.

[0027] In this embodiment, this step aims to introduce the initial radiant energy distribution into the established three-dimensional voxel mesh, thereby providing the initial kinetic energy for simulating indirect light transmission. First, by comprehensively utilizing the spatial distribution, intensity attributes, and environmental occlusion relationships of scene light sources, the initial illumination prior received by each voxel unit is obtained. Subsequently, the obtained illumination information is deeply fused with the original geometric and material features of the voxels. This process constructs a three-dimensional voxel field carrying the initial energy state, enabling the voxels to not only possess static physical properties but also a radiance context for participating in dynamic light and shadow interactions, laying the energy foundation for subsequent simulations of the evolution of light in space.

[0028] To adapt to different rendering needs, hardware computing power conditions, and the complexity of different types of light sources, two implementation methods are supported: explicit physical radiative injection and implicit feature radiative injection. Method 1: Explicit Physical Radiation Injection. This method utilizes a reflection shadow map in computer graphics, using a 2D buffer rendered from the light source's perspective as the carrier for direct lighting. Specifically, the scene is first rendered from the light source's perspective, generating a 2D buffer texture that records depth, world space normals, and the light source's radiant flux. Then, a 3D voxel mesh is traversed, and the 3D coordinates of each voxel are projected onto the reflection shadow map. The depth of the occlusion relative to the light source in the reflection shadow map is extracted and compared with the voxel's depth relative to the light source. If the depths are similar, the voxel receives light; otherwise, it does not. For directly illuminated voxels, the corresponding initial radiant energy is sampled from the reflection shadow map, and this initial radiant energy is physically multiplied and added to the voxel's own diffuse reflectance and other material reflection properties, thus explicitly writing the precise direct lighting energy into the 3D voxel. Through these operations, the light source radiant energy stored in the reflection shadow map is accurately transferred to the 3D voxel field, providing a physically accurate initial radiation source for subsequent indirect light transmission simulations.

[0029] Method 2: Implicit Feature Radiation Injection. This method introduces a light source injection mechanism based on implicit neural representations to cope with complex and variable lighting environments. Specifically, such as... Figure 2 As shown, the light source attributes (such as position, direction, intensity distribution, etc.) within the scene are first input into a pre-trained light source encoding network (such as the Nelif model), which encodes them into a mutually orthogonal three-plane feature field covering the entire scene bounding box. This three-plane feature field stores highly compressed implicit lighting spatial distribution information on three orthogonal planes. Subsequently, in the injection phase, the three-dimensional voxel mesh is traversed, and the three-dimensional coordinates of each voxel are orthogonally projected onto the three planes of the three-plane feature field, and bilinear interpolation features are sampled. Figure 2 The symbol S represents sampling, and the features of the three sampled surfaces are aggregated to generate a lighting feature vector specific to the voxel. Finally, this implicit lighting feature is fused with the voxel's own geometric and material features through channels ( Figure 2 The symbol + is used to represent fusion, thereby completing the implicit injection of illumination priors at the latent space level.

[0030] S3 performs two-dimensional feature calculations alternately along three orthogonal planes within a three-dimensional voxel field, enabling the exchange and long-distance transmission of radiant energy, implicitly simulating indirect light transmission, and outputting an indirect illumination feature field.

[0031] In this embodiment, this step aims to implicitly simulate the 3D transmission of global illumination through a dimensionality reduction computation strategy. To avoid the huge computational overhead caused by standard 3D convolution or full-space attention mechanisms directly performing high-complexity 3D correlation calculations in the 3D full space, this invention adopts a dimensionality reduction decoupling strategy that maintains the integrity of the spatial structure: instead of compressing or projecting 3D features onto 2D patches, it maintains the complete 3D voxel mesh tensor structure and directly applies 2D neural network operators (such as 2D convolution kernels or window-based 2D multi-head attention mechanisms) alternately along three mutually orthogonal 2D plane dimensions (i.e., slice dimensions parallel to the YZ plane, XZ plane, and XY plane, respectively) within the 3D space to perform long-range feature modeling.

[0032] Specifically, such as Figure 2 As shown, the three-dimensional voxel mesh after radiation injection is first regarded as being composed of multiple continuous YZ plane slices, and the interaction and aggregation of two-dimensional spatial features are performed independently within each slice. Figure 2 In this context, a Yoz-plane transformer is used; subsequently, the features are modeled in two dimensions using the same mechanism along the XZ and XY planes. Figure 2 The transformation is represented by Xoz-plane transformers and Xoy-plane transformers, respectively, and is performed n times in a loop across three orthogonal two-dimensional planes. By performing this computational mode of alternating scans across orthogonal two-dimensional planes within three-dimensional space, features continuously intersect, flow, and merge on different axial slice planes. This method eliminates the need for explicit ray intersection calculations, significantly reducing the computational complexity of the core operations from O(N^2) to O(N^2). 6 ) sharply reduced to O(N 4 It more perfectly preserves the depth correlation and occlusion relationship of light transmission in three-dimensional physical space, so that light energy can efficiently complete the simulation of multi-level scattering and long-distance transmission in the latent space without losing any three-dimensional spatial depth and structural information, and finally outputs an indirect lighting feature field containing rich global context information.

[0033] S4 extracts the screen space geometry buffer to obtain the three-dimensional world coordinates and high-frequency features of each pixel.

[0034] In this embodiment, this step embeds the deferred rendering pipeline standard of Unreal Engine, aiming to capture high-resolution surface geometry and material details from the current camera's perspective. First, a geometry rendering stage is performed in the current viewport of the main camera, generating a high-resolution screen-space geometry buffer (G-Buffer) in real-time through rasterization. Then, this G-Buffer is directly read to precisely extract the world coordinates (reconstructed from depth information), high-precision microscopic normals, diffuse reflectance, and roughness, among other physical property parameters, for each visible pixel on the current frame's screen. Since some high-frequency structural information is inevitably lost during the voxelization discretization process of a 3D scene, the extracted high-frequency geometric and material features become a key data source to compensate for this deficiency, providing pixel-level alignment anchors for subsequent lighting decoding, ensuring that global illumination accurately matches the extremely complex microscopic surface undulations and material properties during final shading.

[0035] S5 samples voxel-level features from the indirect lighting feature field based on three-dimensional world coordinates, couples them with high-frequency features, decodes them through a neural network, and outputs indirect radiance, which is then superimposed with direct lighting to complete global neural rendering.

[0036] In this embodiment, this step aims to transform the deeply evolved indirect lighting feature field into the final pixel-level indirect lighting prediction result. First, based on the 3D world coordinates reconstructed in step S4 for the current pixel to be rendered, trilinear interpolation sampling is performed from the indirect lighting feature field (3D texture) generated in step S3 to obtain the voxel-level features of the pixel in macroscopic space. Then, the extracted voxel-level features are channel-concatenated with the pixel's high-frequency geometric and material features in the G-Buffer. The combined multidimensional feature vector is input into a lightweight multilayer perceptron (MLP) for decoding, ultimately predicting the pixel's physically accurate indirect radiance. Finally, the decoded indirect radiance is pixel-by-pixel superimposed with the scene's direct lighting result. Direct lighting can be achieved using a traditional deferred rendering lighting channel or reconstructed from the direct lighting information injected into the voxels in step S2 via screen space. Through this superposition operation, the final color value of each pixel is obtained, thus completing global neural rendering.

[0037] Figure 3 and Figure 4 The specific implementation results of the present invention are shown. For example... Figure 3 As shown, the rendering results of this invention are compared with other state-of-the-art methods (such as the ray tracing denoising method Oidn, the neural rendering methods Nelif and Lightformer). The scenes from top to bottom are living room 1, European-style bedroom, student bedroom, bathroom, living room 2, and kitchen. The results show that the method of this invention achieves the highest quality visual effect, with the smallest difference from the baseline image. Figure 4As shown, the generalization test results of the present invention in different scenarios and under different light source conditions are presented. The scenarios from top to bottom are fireplace, living room 3, European-style bedroom, and living room 1. The results show that the present invention can stably generate high-quality rendering results in different scenarios and under multiple light source conditions, demonstrating strong generalization ability and robustness.

[0038] In summary, the general global neural rendering method based on neurovoxel representation provided by this invention constructs a complete closed loop through the synergistic evolution of the above five steps, from spatial neurovoxel representation and initial radiative energy injection to implicit transport simulation based on multidimensional orthogonal correlation, and finally to pixel-level radiance reconstruction combined with high-frequency geometric details. This scheme not only effectively overcomes the bottleneck of huge computational overhead in traditional global illumination algorithms when dealing with complex indirect lighting, but also significantly compensates for the generalization defects caused by the lack of global spatial representation in existing neural rendering methods. With its ability to deeply mine three-dimensional latent feature fields and couple cross-dimensional features, this invention can achieve stable and high-fidelity indirect lighting prediction in dynamic environments and with diverse material configurations without performing expensive retraining processes for specific scenes. It provides a global lighting solution that balances operational efficiency and physical realism for fields such as real-time rendering, film and television post-production, virtual production, and high-precision digital twins.

[0039] Based on the same inventive concept, such as Figure 5 As shown, this embodiment of the invention also provides a general global neural rendering system based on neural voxel representation, including: a scene voxelization module 510, a lighting injection module 520, a scene encoding module 530, a screen space information extraction module 540, and a lighting decoding module 550.

[0040] The scene voxelization module 510 is used to perform voxelization processing on the 3D scene to generate a 3D voxel mesh carrying physical properties.

[0041] The illumination injection module 520 is used to inject initial illumination into the three-dimensional voxel mesh to form a three-dimensional voxel field carrying initial radiation energy.

[0042] The scene encoding module 530 is used to perform two-dimensional feature calculations alternately along three orthogonal planes in a three-dimensional voxel field, so that radiant energy can be exchanged and transmitted over long distances, implicitly simulating indirect light transmission, and outputting an indirect illumination feature field.

[0043] The screen space information extraction module 540 is used to extract the screen space geometric buffer and obtain the three-dimensional world coordinates and high-frequency features of each pixel.

[0044] The illumination decoding module 550 is used to sample voxel-level features from the indirect illumination feature field based on three-dimensional world coordinates. After coupling with high-frequency features, it is decoded by a neural network to output indirect radiance, which is then superimposed with direct illumination to complete global neural rendering.

[0045] Based on the same inventive concept, embodiments of the present invention also provide an electronic device, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-described general global neural mapping method based on neurovoxel representation when executing the computer program.

[0046] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-described general global neural mapping method based on neurovoxel representation.

[0047] It should be noted that the general global neural mapping system, electronic device, and computer-readable storage medium based on neurovoxel representation provided in the above embodiments all belong to the same inventive concept as the general global neural mapping method based on neurovoxel representation. For details of their specific implementation process, please refer to the embodiments of the general global neural mapping method based on neurovoxel representation, which will not be repeated here.

[0048] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A general global neural mapping method based on neurovoxel representation, characterized in that, Includes the following steps: The 3D scene is voxelized to generate a 3D voxel mesh carrying physical properties; Initial illumination is injected into the three-dimensional voxel mesh to form a three-dimensional voxel field carrying initial radiation energy; Two-dimensional feature calculations are performed alternately along three orthogonal planes within a three-dimensional voxel field, enabling the exchange and long-distance transmission of radiant energy, implicitly simulating indirect light transmission, and outputting an indirect illumination feature field. Extract the screen space geometry buffer to obtain the three-dimensional world coordinates and high-frequency features of each pixel; Based on three-dimensional world coordinates, voxel-level features are sampled from the indirect lighting feature field, coupled with high-frequency features, and then decoded by a neural network to output indirect radiance. This is then superimposed with direct lighting to complete global neural rendering.

2. The general global neural mapping method based on neurovoxel representation according to claim 1, characterized in that, The voxelization process includes: The scene is rendered using orthogonal rasterization along three mutually orthogonal projection directions. The physical properties of the scene surface are extracted, including space occupancy, surface normal distribution, and basic material properties. The extracted information is then mapped onto a 3D voxel mesh.

3. The general global neural mapping method based on neurovoxel representation according to claim 1, characterized in that, The initial illumination injection includes explicit physical radiation injection: Render the scene from the perspective of the light source to generate a reflection and shadow map; Traverse the three-dimensional voxel mesh, and perform projection and visibility depth tests on the reflection shadow map based on the three-dimensional coordinates of each voxel to determine whether the voxel is directly illuminated. For a voxel that is directly illuminated, the initial radiation energy is sampled from the reflection shadow map and fused with the material reflection properties of the voxel itself to obtain the voxel feature carrying the direct illumination energy.

4. The general global neural mapping method based on neurovoxel representation according to claim 1 or 3, characterized in that, The initial illumination injection includes implicit feature radiation injection: The light source attributes in the scene are input into a pre-trained light source encoding network to generate an orthogonal three-plane feature field covering the scene bounding box; Traverse the three-dimensional voxel mesh, project the three-dimensional coordinates of each voxel onto the three planes of the orthogonal three-plane feature field and sample bilinear interpolation features, and aggregate the sampling results into illumination features; By fusing the lighting features with the geometric and material features of the voxel itself, voxel features carrying implicit lighting information are obtained.

5. The general global neural mapping method based on neurovoxel representation according to claim 1, characterized in that, The method of alternately performing two-dimensional feature calculations along three orthogonal planes within a three-dimensional voxel field includes: Maintaining the complete grid tensor structure of the three-dimensional voxel field, two-dimensional neural network operators are independently applied to each plane slice along the XY plane, XZ plane, and YZ plane to perform feature interaction and aggregation, and the calculation is iteratively performed alternately along the three directions.

6. The general global neural mapping method based on neurovoxel representation according to claim 1, characterized in that, The high-frequency features include pixel-level geometric information and physical material parameters. The geometric information includes at least depth information, world coordinates, and surface normals. The physical material parameters include at least diffuse reflectance and roughness.

7. The general global neural mapping method based on neurovoxel representation according to claim 1, characterized in that, Voxel-level feature sampling is performed using trilinear interpolation, and the coupled features are decoded using a multilayer perceptron to output indirect radiance.

8. A universal global neural mapping system based on neurovoxel representation, implemented using the universal global neural mapping method based on neurovoxel representation as described in any one of claims 1 to 7, characterized in that, include: Scene voxelization module, lighting injection module, scene encoding module, screen space information extraction module, lighting decoding module; The scene voxelization module is used to perform voxelization processing on the three-dimensional scene to generate a three-dimensional voxel mesh carrying physical properties. The illumination injection module is used to inject initial illumination into the three-dimensional voxel mesh to form a three-dimensional voxel field carrying initial radiation energy. The scene encoding module is used to perform two-dimensional feature calculations alternately along three orthogonal planes in a three-dimensional voxel field, so that radiant energy can be exchanged and transmitted over long distances, implicitly simulating indirect light transmission and outputting an indirect illumination feature field. The screen space information extraction module is used to extract the screen space geometric buffer to obtain the three-dimensional world coordinates and high-frequency features of each pixel. The illumination decoding module is used to sample voxel-level features from the indirect illumination feature field according to the three-dimensional world coordinates, and after coupling with high-frequency features, it is decoded by a neural network to output indirect radiance, which is then superimposed with direct illumination to complete global neural rendering.

9. An electronic device comprising a memory and one or more processors, the memory for storing a computer program, characterized in that, The processor is used to implement the general global neural rendering method based on neurovoxel representation as described in any one of claims 1 to 7 when executing a computer program.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a computer, it implements the general global neural mapping method based on neurovoxel representation as described in any one of claims 1 to 7.

Images