Multi-primitive oriented separate lighting modeling global neural rendering method and system

Abstract

The application discloses a kind of multi-graph element-oriented separation type illumination modeling global neural rendering method and system, belong to image rendering technical field, including: the graph element of the object corresponding in scene is divided into general illumination set and special effect illumination set;All different types of graph elements in two sets are calculated with the color point information of target perspective using differentiated ray intersection and sampling strategy;According to the color point position, each type of graph element is inquired to the corresponding neural feature field to obtain the high-dimensional neural feature of each graph element at the color point;Using the dual synthesis mechanism based on depth and transmittance, at least one graph element color point information and high-dimensional neural feature of each pixel are synthesized to obtain the synthesis result;The synthesis results corresponding to two sets are respectively subjected to illumination calculation and then merged.

Description

Technical Field

[0001] This invention belongs to the field of computer graphics technology, specifically relating to a global neural rendering method and system for multi-primitive discrete lighting modeling. Background Technology

[0002] The rendering pipeline is a core infrastructure in computer graphics, tasked with converting 3D scene representations into 2D images. As applications such as games, virtual reality, digital twins, and intelligent simulation become increasingly complex, users' demands for rendering quality, real-time performance, and content production efficiency continue to rise. Existing rendering technologies are gradually revealing their limitations in areas such as unified processing of heterogeneous primitives, flexibility in lighting modeling, and balancing rendering efficiency with visual quality.

[0003] Traditional rendering techniques have evolved over a long period, resulting in two main approaches: rasterization and ray tracing. The rasterization pipeline uses triangular meshes as its primary primitives, achieving efficient rendering through vertex processing, primitive assembly, rasterization, and fragment shading, and boasts a mature hardware and software ecosystem. However, for non-mesh representations such as point clouds and voxels, indirect support is typically required via proxy geometry, volume rendering, or custom shading workflows. Furthermore, the rasterization pipeline's representation of global illumination, caustics, complex indirect reflections, and subsurface scattering often relies on pre-computation or approximation techniques, limiting its flexibility in dynamic scenes and complex lighting conditions. The ray tracing pipeline, by simulating the propagation of light, can effectively represent shadows, reflections, refractions, and some global illumination effects, but its computational cost is high, and it still faces significant performance challenges in complex geometry, complex materials, and high-resolution real-time rendering scenarios. In addition, existing ray tracing workflows are mainly geared towards the design of explicit geometric primitives. For novel neural representations such as Gaussian distributions, implicit fields, and neural radiation fields, it is usually necessary to build additional dedicated intersection, sampling, or volume rendering mechanisms, making it difficult to directly reuse existing standard rendering workflows.

[0004] With the development of neural network technology, neural rendering has gradually formed a technical path distinct from traditional graphics pipelines, demonstrating significant potential in tasks such as scene reconstruction and novel perspective synthesis. For example, methods represented by 3D Gaussian Splatting use anisotropic Gaussian distributions as the basic units of scene representation and achieve high-quality, real-time novel perspective synthesis through the splatting mechanism. Meanwhile, other neural rendering methods involve various schemes based on voxels, implicit fields, neural materials, or hybrid representations. Because different representations differ in data organization, visibility calculation mechanisms, shading interfaces, and lighting parameter semantics, existing methods typically design dedicated workflows for each single representation, lacking a unified primitive processing framework. Therefore, it is difficult to stably mix traditional primitives and neural representation primitives in the same scene, and it is also difficult to ensure consistency in occlusion relationships, lighting interactions, and material representations among multiple primitive types.

[0005] In terms of lighting modeling, existing neural rendering technologies still suffer from insufficient flexibility. Many solutions tightly couple geometry, materials, lighting, and viewpoint-related effects into a unified representation or network. While this can generate high-quality results for specific tasks, it limits the ability to hierarchically control general lighting components and specific complex lighting effects. It makes it difficult to enable, adjust, or replace specific lighting modules as needed based on rendering requirements, and it also hinders efficient collaboration with traditional lighting models or post-processing workflows. This leads to difficulties in engineering integration in real-world production environments and a challenge in balancing real-time performance with high-fidelity results.

[0006] Furthermore, in multi-primitive mixed scenes, how to maintain the energy consistency and visual consistency of lighting calculation under the premise of separate modeling, and how to reduce artifacts by realizing cross-primitive visibility judgment, shadow casting and lighting accumulation, remain challenges in existing technologies.

[0007] Therefore, existing rendering technologies still have shortcomings in terms of primitive support versatility, lighting modeling flexibility, and the balance between efficiency and quality. There is an urgent need for a unified rendering method and system for heterogeneous primitives to support multi-primitive fusion representation and separate neural lighting modeling, thereby improving the applicability, scalability, and overall performance of the rendering system. Summary of the Invention

[0008] In view of the above, the present invention aims to overcome the problems of fragmented primitive processing framework, high coupling of lighting modeling, and difficulty in balancing real-time performance and image quality in the prior art. It provides a global neural rendering method and system for multi-primitive discrete lighting modeling. Its core is to construct a rendering architecture for hierarchical scene lighting allocation and unified synthesis of heterogeneous primitives. The primitive representations supported include, but are not limited to, triangular mesh primitives, point cloud primitives, voxel primitives, parametric surface primitives, and Gaussian primitives.

[0009] To achieve the above-mentioned objectives, an embodiment provides a global neural rendering method for multi-primitive separable lighting modeling, comprising the following steps: Scene lighting grouping configuration: Based on scene complexity and visual requirements, the primitives corresponding to objects in the scene are divided into general lighting sets and special effects lighting sets; Heterogeneous primitive shading point calculation: Differentiated ray intersection and sampling strategies are used for all different types of primitives in two sets to calculate the shading point information of the target viewpoint. The shading point information includes the shading point position, depth map, auxiliary features, and mask map representing the screen space covered by the primitive. Neural feature field decoding: An additional neural feature field is introduced for each type of primitive. Based on the position of the colored point, the corresponding neural feature field of each type of primitive is queried to obtain the high-dimensional neural features of each primitive at the colored point. Multi-primitive feature synthesis: A dual synthesis mechanism based on depth and transmissivity is adopted to synthesize the color point information of at least one primitive and high-dimensional neural features for each pixel to obtain the synthesis result. Separate neural lighting estimation and merging: The synthesis results corresponding to the general lighting set and the special effects lighting set are input into the general lighting model and the special effects lighting model based on the neural network, respectively. The basic lighting effect and complex lighting effect of the scene are calculated separately and then merged to obtain the overall lighting result.

[0010] Preferably, for primitives assigned to the special effects lighting set, further primitive grouping and specification of the primitive shading point information calculation order are also supported.

[0011] Preferably, a differentiated ray intersection and sampling strategy is applied to all different types of primitives in the two sets to calculate the shading point information of the target viewpoint, including: For explicit geometric primitives, the position of the shading point is determined by calculating the geometric intersection of the camera ray and the scene, and the corresponding depth value of the shading point is calculated. At the same time, auxiliary features are obtained by sampling the texture coordinates of the shading point in the predefined texture data. The auxiliary features include normal, diffuse color, specular color, roughness, and metallicity. For non-explicit geometric primitives, a volume rendering cumulative transmittance strategy is adopted: calculate the transmittance of camera light passing through the primitive region, and calculate the cumulative depth and auxiliary features based on the cumulative transmittance, and then infer the position of the shading point. Simultaneously, mask maps of various primitives are also generated to identify the coverage area of ​​different primitives in screen space, providing information for subsequent lighting calculations and compositing.

[0012] Preferably, the neural feature field includes a three-plane feature grid, a hash feature grid, and a neural texture. For explicit geometric primitives, the neural texture specific to the primitive is queried by the texture coordinates of the shading points.

[0013] Preferably, a dual synthesis mechanism based on depth and transmittance is employed to synthesize the primitive shading point information and high-dimensional neural features for each pixel, resulting in a synthesis outcome including: Synthesis Method 1: Select the shallowest primitive for each pixel based on the depth map, and synthesize the shading point information of the primitive with high-dimensional neural features to obtain the synthesis result; Synthesis Method 2: Based on the depth sorting and transmittance accumulation formula, each pixel is weighted and synthesized from light to dark, using the color point information of different primitives and high-dimensional neural features to obtain the synthesis result.

[0014] Preferably, based on the depth sorting and transmittance accumulation formula, for each pixel, the color point information of different primitives and high-dimensional neural features are weighted and synthesized from light to dark to obtain the synthesis result, including: Based on the depth map, all primitives corresponding to each pixel are sorted in order from shallow to deep. Then, the primitives are traversed in depth order, and the chromatic point information and high-dimensional neural features of primitives at different depths are weighted and synthesized using the α-mixing method according to the transmittance. During synthesis, the chromatic point information and high-dimensional neural features of each primitive at different depths are weighted and synthesized as a whole to obtain the synthesized result.

[0015] Preferably, the general illumination model predicts the direct and indirect illumination results at each pixel based on the synthesis results and observation information corresponding to the general illumination set; The special effects lighting model takes the composite result corresponding to the special effects lighting set as input, or combines it with a mask image, to predict the complex lighting effects of a specific lighting area.

[0016] To achieve the above-mentioned objectives, embodiments of the present invention also provide a global neural rendering system for multi-primitive discrete lighting modeling, comprising: The scene lighting grouping configuration module is used to divide the primitives corresponding to objects in the scene into general lighting sets and special effects lighting sets according to the scene complexity and visual requirements. The heterogeneous primitive shading point calculation module is used to calculate the shading point information of the target viewpoint by applying differentiated ray intersection and sampling strategies to all different types of primitives in two sets. The shading point information includes shading point position, depth map, auxiliary features and mask map representing the screen space covered by the primitive. The neural feature field decoding module is used to introduce additional neural feature fields for each type of primitive. It queries the corresponding neural feature field of each type of primitive based on the position of the colored point to obtain the high-dimensional neural features of each primitive at the colored point. The multi-primitive feature synthesis module is used to synthesize the primitive color point information and high-dimensional neural features for each pixel using a dual synthesis mechanism based on depth and transmissivity, and obtain the synthesis result. The separate neural lighting estimation and merging module is used to input the synthesis results corresponding to the general lighting set and the special effects lighting set into the general lighting model and the special effects lighting model based on the neural network, respectively, and then merge them to obtain the overall lighting result after calculating the basic lighting effect and complex lighting effect of the scene.

[0017] To achieve the above-mentioned objectives, this invention also provides a computing device, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned global neural rendering method for multi-primitive discrete lighting modeling.

[0018] To achieve the above-mentioned objectives, embodiments of the present invention also provide a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the above-mentioned global neural rendering method for multi-primitive discrete lighting modeling.

[0019] Compared with the prior art, the beneficial effects of the present invention include at least the following: This invention can support triangular meshes, Gaussian primitives and other scalable primitive representations within a unified framework, and allows traditional primitives and neural primitives to be collaboratively rendered in the same scene. Therefore, it has significant advantages in terms of primitive support range and heterogeneous primitive mixing capabilities.

[0020] Since this invention maps the processing results of different primitives to a shared intermediate representation layer through unified neural features, it can effectively reduce the system complexity caused by the inconsistency of interfaces between different primitives and improve the reuse capability of subsequent lighting.

[0021] Furthermore, this invention employs a separate lighting modeling architecture, decoupling basic lighting from complex special effects. This allows for independent control of different lighting effects based on scene requirements, thereby enhancing both the flexibility of lighting modeling and the interpretability of the system. Since the general lighting model handles the frequently used basic calculations, while the special effects lighting model enables complex effects on demand, this invention is more conducive to achieving a better balance between performance and quality under real-time budget constraints.

[0022] In dynamic scenes, since primitive branches, lighting branches, and special effects modules can all be updated locally as needed, this approach is more conducive to reducing update overhead and improving system scalability and long-term evolution capabilities compared to highly tightly coupled end-to-end methods. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of the global neural rendering method for split lighting modeling oriented to multiple primitives provided in the embodiment; Figure 2 This is a schematic diagram of the structure of the global neural rendering system for multi-primitive discrete lighting modeling provided in the embodiment. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0026] The inventive concept of this invention is as follows: This invention provides a global neural rendering scheme for split lighting modeling oriented to multiple primitives, which aims to achieve at least the following objectives: (1) Provide a unified processing architecture for heterogeneous primitives, enabling the rendering system to support both traditional primitives and neural representation primitives, and to support the mixed use of multiple primitive types in the same scene; (2) Provide a split lighting modeling method that decouples general lighting calculation from specific complex optical effect calculation, so as to enable, adjust and optimize different lighting branches as needed; thereby improving the rendering quality, system flexibility and engineering integrability in complex scenes while ensuring real-time or near-real-time performance.

[0027] Based on the above inventive concept, the embodiment provides a global neural rendering method for multi-primitive split lighting modeling, comprising the following steps: S1, Scene lighting group configuration.

[0028] Before rendering begins, scene description data composed of different primitives is received. The key to step S1 is to allocate lighting strategies, that is, to configure scene lighting groups. Unlike traditional rendering pipelines that treat all objects the same, this invention allows users or systems to explicitly allocate lighting models to the primitives corresponding to objects in the scene according to scene complexity and visual requirements.

[0029] In this embodiment, all object instances in the scene are divided into two mutually exclusive logical sets based on a user-preset (or automatically analyzed) strategy: a general lighting set and a special effects lighting set. All primitives can be assigned to either the general lighting model for high efficiency or the special effects lighting model for high local fidelity. The general lighting set typically includes static backgrounds, objects dominated by diffuse reflection (such as walls and floors), and their primitives; these objects and primitives are marked for lighting calculations using the general lighting model. The special effects lighting set typically includes objects and primitives with complex materials (such as metal), dynamic light sources, or those requiring high-fidelity lighting reconstruction; these objects and primitives are marked for calculations using the special effects lighting model.

[0030] In this embodiment, for primitives assigned to the special effects lighting set, further primitive grouping and the specification of the calculation order of primitive shading point information are also supported, specifically by maintaining a manually specified calculation order queue. This mechanism allows priority calculation of objects that contribute significantly to global illumination or have complex optical properties (such as luminescent objects or highly reflective objects), thereby synthesizing high-quality lighting effects.

[0031] S2, Calculation of coloring points for heterogeneous primitives For each pixel in the screen space, its corresponding shading point needs to be determined. Therefore, for all different types of primitives in the two sets of general lighting and special effects lighting in step S1, a differentiated ray intersection and sampling strategy is used to calculate the shading point information of the target viewpoint. Specifically, this includes: (1) For primitives with visible surface geometry, i.e. explicit geometric primitives, such primitives include triangular meshes, parametric surfaces, etc. The intersection points of camera rays and scene geometric surfaces are calculated using traditional rasterization pipeline or ray-scene intersection methods to determine the position of the shading point and calculate the corresponding depth value accordingly; then, auxiliary features are obtained by sampling the predefined texture data based on the texture coordinates of the shading point in the model space. Specifically, the predefined G-Buffer channel is sampled to obtain normals, diffuse color, specular color, roughness, metallic properties, etc. as auxiliary features; finally, a binary mask is generated to identify the pixels covered by the corresponding primitive type.

[0032] (2) For primitives lacking explicit surface geometry, i.e., non-explicit geometric primitives, such primitives include point clouds, neural radiation fields, Gaussian splashes, voxels, etc. A cumulative transmittance strategy using screen space projection or ray-based volume rendering is adopted: the transmittance of camera rays passing through the primitive region is calculated, and the cumulative depth and auxiliary features are calculated based on the cumulative transmittance, thereby inferring the position of the shading point.

[0033] Specifically, firstly, the primitives of each pixel are sorted, and the cumulative transmittance is calculated from lightest to darkest; then, based on Alpha Blending (… (Mixed) Calculate additional features such as depth, diffuse color, specular color, roughness, and metallicity of the cumulative primitive; finally, calculate the corresponding shading point position based on the cumulative depth, and use the final transmittance as the mask image.

[0034] S3, Neural Feature Field Decoding After obtaining the location of the colored point, an additional neural feature field is queried based on the colored point location to obtain high-dimensional semantic features. In the embodiments, the neural feature field introduced for each type of primitive is flexible and diverse, including but not limited to a three-plane feature grid, a hash feature grid, or an implicit feature field based on a multilayer perceptron. The three-plane feature grid represents projecting the colored point location onto three orthogonal feature planes, with bilinear interpolation used to obtain the feature vector. For the hash feature grid: for large-scale scenes, a multi-resolution hash table is used to store features, and features are looked up through position hashing, significantly reducing memory usage. Based on these neural feature fields, the corresponding neural feature field for each type of primitive is queried based on the colored point location to obtain the high-dimensional neural features of each primitive at the colored point.

[0035] In this embodiment, for explicit geometric primitives, the feature field can be extended to a neural texture. For triangular meshes, instead of using traditional RGB mapping, a neural texture (parametric MLP or texture atlas) stored in UV space is used. Specifically, the primitive-specific neural texture is queried directly through the texture coordinates of the shading points, outputting a high-dimensional feature vector. This design enables explicit geometry to possess high-frequency detail representation capabilities similar to implicit fields, with extremely fast query speed, significantly reducing the feature extraction overhead of explicit geometric primitives while maintaining compatibility with implicit representations.

[0036] S4, Multi-Primary Feature Synthesis Since the same pixel in a scene may be covered by multiple primitives, meaning there may be a mixture of various primitive types in the scene, feature synthesis is required. A dual synthesis mechanism based on depth and transmissivity is adopted to synthesize the primitive shading point information and high-dimensional neural features for each pixel to obtain the synthesized result.

[0037] Specifically, this invention maintains two independent compositing buffers for the general lighting set and the special effects lighting set: a general feature buffer and a special effects feature buffer. One set contains the compositing results of general lighting primitives, and the other set contains the compositing results of special effects lighting primitives, used respectively for subsequent calculations of the general lighting model and the special effects lighting model. Different compositing methods can be used for the general feature buffer and the special effects feature buffer. The compositing process can have two options: Synthesis Method 1: For each pixel, the shallowest primitive is selected for each pixel based on the depth map, and the shading point information of the primitive is synthesized with the high-dimensional neural features to obtain the synthesis result. This synthesis method pursues synthesis efficiency and can obtain the synthesis effect efficiently. It is a choice that sacrifices some quality when performance is prioritized.

[0038] Method 2: Based on depth sorting and transmittance accumulation formulas, each pixel is weighted and synthesized from light to dark, combining the color point information of different primitives with high-dimensional neural features to handle semi-transparency or volumetric effects, resulting in a composite image. This method better considers the feature fusion relationships between semi-transparent primitives and pursues high quality, but its efficiency is relatively low due to the need for depth sorting.

[0039] The specific implementation process of the second synthesis method includes: sorting all primitives corresponding to each pixel in depth from shallow to deep according to the depth map; then traversing the primitives in depth order; calculating the transmittance similar to step S2; and using the transmittance... The color point information and high-dimensional neural features of primitives at different depths are weighted and synthesized in a hybrid manner. During synthesis, the color point information and high-dimensional neural features of each primitive at different depths are weighted and synthesized as a whole to obtain the synthesis result.

[0040] It is important to note that when a pixel ray does not intersect with a primitive, the shading point information of the primitive can retain its initial value, i.e., a null value, during compositing.

[0041] S5, Disjoint Neural Illumination Estimation and Combination In this embodiment, after obtaining the general feature buffer and the special effects feature buffer, the system uses these two sets of buffers to drive the general lighting model and the special effects lighting model for lighting estimation, respectively. Specifically, according to the grouping configuration in step S1, the synthesis results corresponding to the general lighting set and the special effects lighting set are input into the neural network-based general lighting model and the special effects lighting model, respectively, to calculate the basic lighting effect and complex lighting effect of the scene, and then merged to obtain the overall lighting result. The general lighting model is positioned to efficiently calculate the basic lighting effect of the scene, while the special effects lighting model is positioned to reproduce the complex lighting details of specific objects with high fidelity. Depending on the implementation method, a mask image may be used, and the two lighting results can be seamlessly fused.

[0042] In terms of specific implementation architecture, the implementation forms of general lighting calculation and special effects lighting calculation are not limited to specific algorithm models. They can be shaders based on traditional physical formulas or neural networks based on deep learning. This invention provides several preferred solutions: 1. The neural network corresponding to the general lighting model can adopt a global prediction architecture based on scene observation. Observational information such as the scene's geometry, normals, and materials is acquired at the light source. This observational information, along with the aforementioned synthetic results (i.e., the shading point positions, auxiliary features, and neural features in the general feature buffer under the target viewpoint), is input into the general lighting model to predict the direct and indirect lighting results at each pixel. This general lighting model is mainly used for unified and efficient basic lighting calculations on most primitives in the scene, thereby providing a stable overall lighting result for the final image. In a preferred embodiment, the neural network of the general lighting model can adopt a network framework similar to NeLiF (Neural Lighting Function Generation for Real-Time Indoor Rendering) to achieve fast estimation of basic lighting across the scene.

[0043] 2. In contrast to the general lighting model, the special effects lighting model focuses on fine-grained modeling of local light transmission, primarily used for high-fidelity local lighting modeling of primitives within the special effects lighting set. The composite results in the special effects feature buffer are input into the special effects lighting model. In practice, the special effects lighting model can also incorporate the generated mask map to limit the effective area of ​​the special effects lighting calculation.

[0044] In a preferred embodiment, a scene encoder similar to that in the paper *Neural Global Illumination via Superposed Deformable Feature Fields* can be used to obtain a high-precision implicit scene representation. This implicit scene representation is then combined with the synthesis results in the effects feature buffer and input into the decoder structure of NeLT (Object-Oriented Neural Light Transfer) to calculate the direct and indirect light transfer functions of a specific object (including the object's own direct illumination, the shadow cast by the object, the object's own indirect illumination, and the output radiance difference caused by the object's influence on the surrounding indirect illumination). Subsequently, according to the merging method proposed by NeLT, the system superimposes these direct and indirect light transfer functions onto the corresponding components of the output of the general illumination model, thereby obtaining the complex lighting representation of a specific object. Specifically, the visibility of a specific object is determined by combining the mask map and the depth map. Based on this visibility, the direct lighting of the object itself is replaced in the direct lighting output of the general lighting model, and the shadow cast by the object is applied to the surrounding objects. Similarly, the indirect lighting of the object itself is replaced in the indirect lighting output of the general lighting model using this visibility, and the resulting radiance effect is summed with the indirect lighting result of the surrounding environment output by the general lighting model.

[0045] In another implementation, the special effects lighting model can employ a convolutional neural network structure similar to that described in the paper *Deep Shading: Convolutional Neural Networks for Screen-Space Shading*. The system combines the scene encoder representation with the synthesized results in the special effects feature buffer to predict the lighting increment of the special effects primitives at the current viewpoint. This increment represents a supplementary portion relative to the output of the general lighting model, used to recover high reflectivity, complex indirect reflections, or other local special effects lighting details. Finally, the output of the general lighting model and the increment predicted by the special effects lighting model are linearly summed to generate the final high-fidelity rendered image.

[0046] Through the above method, this invention separates general lighting calculation from special effects lighting calculation, and achieves the collaborative merging of the two types of lighting results based on unified shading point information, auxiliary features, and neural features. Since the general lighting model is responsible for the frequently used basic illumination estimation, and the special effects lighting model only supplements complex lighting effects for specified primitives, the system can improve the lighting quality of local areas while ensuring overall rendering efficiency, and enhance its adaptability to different primitive types and different visual requirements.

[0047] To achieve the above-mentioned inventive concept, such as Figure 2 As shown, the embodiment also provides a global neural rendering system for split lighting modeling for multiple primitives, including a scene lighting grouping configuration module, a heterogeneous primitive shading point calculation module, a neural feature field decoding module, a multi-primitive feature synthesis module, and a split neural lighting estimation and merging module; The system comprises several modules: a scene lighting grouping configuration module, a heterogeneous primitive shading point calculation module, and a neural feature field module. The former uses differentiated ray intersection and sampling strategies to calculate shading point information for the target viewpoint, including shading point location, depth map, auxiliary features, and a mask representing the primitive's coverage of the screen space. The latter introduces additional neural feature fields for each primitive type, querying the corresponding neural feature field based on the shading point location to obtain high-dimensional neural features at the shading point. The latter employs a dual synthesis mechanism based on depth and transmittance to synthesize at least one primitive's shading point information and high-dimensional neural features for each pixel, yielding a synthesis result. The former separate neural lighting estimation and merging module inputs the synthesis results from the general lighting set and the special effects lighting set into a neural network-based general lighting model and a special effects lighting model, respectively, to calculate the basic and complex lighting effects of the scene before merging them to obtain the overall lighting result.

[0048] It should be noted that the multi-priority discrete lighting modeling global neural rendering system provided in the above embodiments should be illustrated using the above-described functional module division as an example when rendering images. The functions described above can be assigned to different functional modules as needed, i.e., the internal structure of the terminal or server can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the multi-priority discrete lighting modeling global neural rendering system and the multi-priority discrete lighting modeling global neural rendering method embodiments belong to the same concept. Their specific implementation process is detailed in the multi-priority discrete lighting modeling global neural rendering method embodiments, and will not be repeated here.

[0049] Based on the same inventive concept, the embodiment also provides a computing device, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the above-described multi-primitive-oriented split lighting modeling global neural rendering method, specifically including the following steps: S1, Scene lighting group configuration; S2, Calculation of coloring points for heterogeneous primitives; S3, Neural Feature Field Decoding; S4, Multi-primary feature synthesis; S5, Disjoint neural illumination estimation and synthesis.

[0050] The computing device provided in this embodiment, at the hardware level, includes not only a processor and memory, but also internal buses, network interfaces, memory, and other hardware required for business operations. The memory is non-volatile memory. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the multi-primitive-oriented discrete lighting modeling global neural rendering method described in S1-S5 above. Of course, besides software implementation, this invention does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution entity of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0051] Based on the same inventive concept, the embodiments also provide a computer-readable storage medium storing a program that, when executed by a processor, implements the above-described multi-primitive-oriented split lighting modeling global neural rendering method, specifically including the following steps: S1, Scene lighting group configuration; S2, Calculation of coloring points for heterogeneous primitives; S3, Neural Feature Field Decoding; S4, Multi-primary feature synthesis; S5, Disjoint neural illumination estimation and synthesis.

[0052] In this embodiment, the computer-readable medium includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data.

[0053] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A global neural rendering method for multi-primitive separable lighting modeling, characterized in that, include: Scene lighting grouping configuration: Based on scene complexity and visual requirements, the primitives corresponding to objects in the scene are divided into general lighting sets and special effects lighting sets; Heterogeneous primitive shading point calculation: Differentiated ray intersection and sampling strategies are used for all different types of primitives in two sets to calculate the shading point information of the target viewpoint; Neural feature field decoding: An additional neural feature field is introduced for each type of primitive. Based on the position of the colored point, the corresponding neural feature field of each type of primitive is queried to obtain the high-dimensional neural features of each primitive at the colored point. Multi-primitive feature synthesis: A dual synthesis mechanism based on depth and transmissivity is adopted to synthesize the color point information of at least one primitive and high-dimensional neural features for each pixel to obtain the synthesis result. Separate neural lighting estimation and merging: The synthesis results corresponding to the general lighting set and the special effects lighting set are input into the general lighting model and the special effects lighting model based on the neural network, respectively. The basic lighting effect and complex lighting effect of the scene are calculated separately and then merged to obtain the overall lighting result.

2. The global neural rendering method for multi-primitive-oriented discrete lighting modeling according to claim 1, characterized in that, For primitives assigned to special effects lighting sets, further primitive grouping and specification of the primitive shading point information calculation order are also supported.

3. The global neural rendering method for multi-primitive-oriented discrete lighting modeling according to claim 1, characterized in that, Differentiated ray intersection and sampling strategies are applied to all different types of primitives in the two sets to compute the shading point information of the target viewpoint, including: For explicit geometric primitives, the position of the shading point is determined by calculating the geometric intersection of the camera ray and the scene, and the corresponding depth value of the shading point is calculated. At the same time, auxiliary features are obtained by sampling the texture coordinates of the shading point in the predefined texture data. The auxiliary features include normal, diffuse color, specular color, roughness, and metallicity. For non-explicit geometric primitives, a volume rendering cumulative transmittance strategy is adopted: calculate the transmittance of camera light passing through the primitive region, and calculate the cumulative depth and auxiliary features based on the cumulative transmittance, and then infer the position of the shading point. Simultaneously, mask maps of various primitives are also generated to identify the coverage area of ​​different primitives in screen space, providing information for subsequent lighting calculations and compositing.

4. The global neural rendering method for multi-primitive separable lighting modeling according to claim 1, characterized in that, The neural feature field includes a three-plane feature grid, a hash feature grid, and a neural texture. For explicit geometric primitives, the neural texture specific to the primitive is queried by the texture coordinates of the shading points.

5. The global neural rendering method for multi-primitive separable lighting modeling according to claim 1, characterized in that, A dual synthesis mechanism based on depth and transmissivity is employed to synthesize primitive shading point information and high-dimensional neural features for each pixel, yielding the following synthesis results: Synthesis Method 1: Select the shallowest primitive for each pixel based on the depth map, and synthesize the shading point information of the primitive with high-dimensional neural features to obtain the synthesis result; Synthesis Method 2: Based on the depth sorting and transmittance accumulation formula, each pixel is weighted and synthesized from light to dark, using the color point information of different primitives and high-dimensional neural features to obtain the synthesis result.

6. The global neural rendering method for multi-primitive-oriented discrete lighting modeling according to claim 5, characterized in that, Based on the depth sorting and transmittance accumulation formula, for each pixel, the color point information of different primitives and high-dimensional neural features are weighted and synthesized from light to dark to obtain the synthesis result, including: For each pixel, all primitives are sorted by depth from lightest to darkest according to the depth map. Then, the primitives are traversed in depth order, and their transmittance is used... The color point information and high-dimensional neural features of primitives at different depths are weighted and synthesized in a hybrid manner. During synthesis, the color point information and high-dimensional neural features of each primitive at different depths are weighted and synthesized as a whole to obtain the synthesis result.

7. The global neural rendering method for multi-primitive separable lighting modeling according to claim 1, characterized in that, The general illumination model predicts the direct and indirect illumination results at each pixel based on the synthesis results and observation information corresponding to the general illumination set. The special effects lighting model takes the composite result corresponding to the special effects lighting set as input, or combines it with a mask image, to predict the complex lighting effects of a specific lighting area.

8. A global neural rendering system for multi-primitive discrete lighting modeling, characterized in that, include: The scene lighting grouping configuration module is used to divide the primitives corresponding to objects in the scene into general lighting sets and special effects lighting sets according to the scene complexity and visual requirements. The heterogeneous primitive shading point calculation module is used to calculate the shading point information of the target viewpoint by applying differentiated ray intersection and sampling strategies to all different types of primitives in two sets. The shading point information includes shading point position, depth map, auxiliary features and mask map representing the screen space covered by the primitive. The neural feature field decoding module is used to introduce additional neural feature fields for each type of primitive. It queries the corresponding neural feature field of each type of primitive based on the position of the colored point to obtain the high-dimensional neural features of each primitive at the colored point. The multi-primitive feature synthesis module is used to synthesize the primitive color point information and high-dimensional neural features for each pixel using a dual synthesis mechanism based on depth and transmissivity, and obtain the synthesis result. The separate neural lighting estimation and merging module is used to input the synthesis results corresponding to the general lighting set and the special effects lighting set into the general lighting model and the special effects lighting model based on the neural network, respectively, and then merge them to obtain the overall lighting result after calculating the basic lighting effect and complex lighting effect of the scene.

9. A computing device comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that, When the one or more processors execute the executable code, they are used to implement the global neural rendering method for multi-primitive-oriented discrete lighting modeling as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the global neural rendering method for multi-primitive-oriented discrete lighting modeling as described in any one of claims 1-7.

Images