Global neural rendering method and system based on sparsity adaptive sampling

Abstract

The application discloses a global neural rendering method and system based on sparsity adaptive sampling, and belongs to the technical field of image rendering. The method comprises the following steps: using a neural rendering method basic framework to perform frame-by-frame precalculation to obtain global scene expression information and a low-quality pre-rendering result; estimating rendering errors between the pre-rendering result and a reference true value result based on the low-quality pre-rendering result and in combination with the global scene expression information; statistically analyzing the distribution of the rendering errors, and recombining all pixels into same-group pixel data corresponding to each error level according to the statistical result; calling a pixel generator with a corresponding parameter quantity for each group of pixel data corresponding to each error level to complete rendering of the corresponding pixels, and putting the pixel-by-pixel rendering result back to the original pixel position to obtain a final rendering image. The application can reduce the overall calculation overhead required for complete resolution image rendering.

Description

Technical Field

[0001] This invention belongs to the field of real-time rendering technology, specifically relating to a global neural rendering method and system based on sparse adaptive sampling. Background Technology

[0002] Among the existing mainstream rendering algorithms, rasterization algorithms are efficient, but their use of a large number of approximations results in insufficient realism. Path tracing algorithms, while producing rendering effects that conform to physical laws, require a large number of light path samples for Monte Carlo integration to converge, thus incurring significant computational costs. In contrast, neural rendering methods that have emerged in recent years combine the advantages of both methods. By creating and training offline high-sample path tracing datasets, neural rendering models can achieve rendering effects that closely resemble physical optics while maintaining computational efficiency and achieving real-time interactive frame rates. As a result, they have gradually gained widespread application.

[0003] Existing representative neural rendering architectures typically consist of two computational phases. The first phase involves encoding the relationships between objects within the global scene, while the second phase is pixel generation, which predicts the color value for each pixel location in the image space. The encoding of global representation information in the first phase varies across different neural rendering methods, employing techniques such as positional encoding, object-centered view field encoding, triplane encoding, and deformable field encoding. However, the structure of the pixel generation phase is similar, using densely layered neural networks to decode the tightly packed global representation information into lighting information. Furthermore, while all methods maintain computational efficiency in the first phase (scene information collection), the second phase (information processing and decoding) is often computationally expensive, requiring large-parameter neural networks to perform intensive computations to achieve high-fidelity pixel generation. Using networks with small parameters makes it difficult to reconstruct complex global lighting effects, affecting image fidelity. The computational complexity of the pixel generation stage limits the computational efficiency of existing neural rendering methods. Although existing methods can achieve interactive frame rates at smaller screen resolutions, their performance often deteriorates as the resolution increases, let alone achieving a real-time, low-latency interactive rendering experience. This greatly limits the application scope of existing neural rendering methods.

[0004] In the rendering of 3D scenes, the rendering signal often exhibits sparsity in both the spatial and temporal domains. In the spatial domain, objects appearing within the camera's viewpoint have varying rendering complexities due to differences in geometry, material, and lighting complexity. This is also reflected in the performance of path tracing rendering algorithms. For example, regions with simple geometry and materials often converge with only a small number of Monte Carlo light path samples, while complex regions require a large number of samples to converge. In the temporal domain, many static regions do not change between consecutive frames, and rendering each frame involves repetitive calculations, wasting computational resources. Rendering acceleration methods address the sparsity of the rendering signal in both path tracing and rasterization rendering algorithms. Examples include adaptive path tracing sampling methods guided by the variance of pre-sampled light path samples, allowing for different numbers of light path samples to be sampled for pixels with different rendering complexities, and variable shading rate rendering methods that selectively render based on the dynamics of the scene. The shortcoming of existing neural rendering methods is that they ignore the sparsity in the rendering signal and use the same pixel decoding network with the same computational complexity for pixels of different frequencies, which leads to a waste of computing resources and affects the overall computational performance of the method.

[0005] A similar prior art to this application is the neural adaptive sampling method for path tracing rendering algorithms proposed by Kuznetsov et al. However, this method is geared towards offline path tracing applications and does not consider the special characteristics of sparsely sampled signals during the reconstruction stage, instead using uniform computational steps. Therefore, it only improves quality without improving computational efficiency. Unlike the sampling, rendering, and reconstruction steps in path tracing rendering, the computational pipeline of neural rendering methods merges the rendering and reconstruction steps. Acceleration methods for neural rendering are still lacking exploration, and there is an urgent need for more reasonable adaptive sampling methods and sparse pixel decoding networks to provide computational performance improvements, thereby meeting the computational budget for low-latency real-time rendering. Summary of the Invention

[0006] In view of the above, the purpose of this invention is to provide a global neural rendering method and system based on sparsity adaptive sampling. By analyzing the inherent sparsity of rendering signals in a 3D scene, the rendering complexity of pixels is classified. By using pixel generation neural networks with different parameter amounts for pixels of different complexities, the overall computational overhead required for rendering full-resolution images is reduced.

[0007] In a first aspect, to achieve the aforementioned objectives, an embodiment provides a global neural rendering method based on sparse adaptive sampling, comprising the following steps: The neural rendering infrastructure is used to perform frame-by-frame pre-computation to obtain global scene representation information and low-quality pre-rendered results. Based on the low-quality pre-drawn results and combined with global scene representation information, the drawing error between the pre-drawn results and the reference true value results is estimated. The distribution of drawing errors is statistically analyzed, and based on the statistical results, all pixels are reorganized into the same group of pixels corresponding to each error level according to the error level. For each set of pixel data corresponding to each error level, the corresponding pixel generator is called to complete the drawing of the corresponding pixel, and the pixel-by-pixel drawing result is put back into the original position of the pixel to obtain the final drawn image.

[0008] Preferably, the calculation process of the global scene representation information includes: Using a three-dimensional spatial location neural field as a complete global scene data representation, geometric buffer data is obtained through lightweight rasterization batch rendering. The spatial location coordinates in the geometric buffer data are used as keys to query the latent vector representation corresponding to each pixel position in the three-dimensional spatial location neural field. When there are multiple neural fields in the scene, the average of multiple latent vector representations is calculated to obtain a latent vector containing global scene representation information. The calculation process for the low-quality pre-drawn results includes: using a lightweight pixel generator with extremely low parameter count in the infrastructure to pre-generate low-quality pre-drawn results.

[0009] Preferably, based on the low-quality pre-drawn results and combined with global scene representation information, the drawing error between the pre-drawn results and the reference ground truth results is estimated, including: The input to the error estimation network is composed of low-quality pre-drawing results and global scene expression information. The error estimation network is used to analyze and predict the rendering computation frequency and complexity of pixel-by-pixel, and to estimate the rendering error between the pre-drawing results and the reference true value results. The error estimation networks were selected as multilayer perceptron neural network, convolutional neural network, and attention mechanism neural network.

[0010] Preferably, when using an error estimation network to estimate rendering errors, its input also includes additional scene geometry and material information contained in the geometry cache, and hidden features of the intermediate process generated by the low-quality rendering process.

[0011] Preferably, when the error estimation network is trained, the actual error between the pre-drawn result and the true reference result is used as a supervision signal to train the prediction error output by the error estimation network.

[0012] Preferably, when training the error estimation network, an additional pre-trained model is used to measure the pixel-wise average subjective score between the pre-drawn results and the ground truth reference results, which is then used as an additional weight to weight the loss function of the prediction error.

[0013] Preferably, the distribution of drawing errors is statistically analyzed, and based on the statistical results, all pixels are reorganized according to their error levels into the same group of pixels corresponding to each error level, including: Perform percentile statistics on the drawing error, and calculate the specific error value corresponding to the predefined percentile for the estimated drawing error values ​​of all pixels in parallel. Error levels are determined based on the specific error values ​​corresponding to percentiles. Each pair of adjacent percentiles with corresponding specific error values ​​forms an error level. Pixels are classified into corresponding error levels based on their drawing errors, and pixels with the same error level are rearranged and batched into the same group of pixel data in a parallel collection manner.

[0014] Secondly, to achieve the above-mentioned objectives, the embodiments also provide a global neural mapping system based on sparse adaptive sampling, comprising: The pre-computation module is used to perform frame-by-frame pre-computation using the neural rendering method infrastructure to obtain global scene representation information and low-quality pre-rendered results. The error estimation module is used to estimate the drawing error between the pre-drawn results and the reference true value results based on the low-quality pre-drawn results and combined with global scene expression information. The classification and rearrangement module is used to statistically analyze the distribution of drawing errors. Based on the statistical results, it reorganizes all pixels into the same group of pixels corresponding to each error level according to the error level. The multi-load drawing module is used to call the pixel generator with corresponding parameters for each set of pixel data corresponding to each error level, complete the drawing of the corresponding pixel, and put the pixel-by-pixel drawing result back to the original position of the pixel to obtain the final drawn image.

[0015] Thirdly, to achieve the above-mentioned objectives, the embodiments also provide a computing device, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the steps of the global neural rendering method based on sparse adaptive sampling described in the first aspect.

[0016] Fourthly, to achieve the above-mentioned objectives, the embodiments also provide a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the global neural rendering method based on sparse adaptive sampling described in the first aspect.

[0017] Compared with the prior art, the beneficial effects of the present invention include at least the following: Adaptive sampling effectively analyzes the sparsity of the rendering signal, estimating the rendering complexity of each pixel under geometry, material, and scene lighting conditions during the rendering process. This allows for the measurement of the error between the pre-rendered result and the ground truth result. The inconsistency of the error results provides valuable information to guide subsequent rendering calculations, thus playing a crucial role in reducing the computational load, throughput, memory usage, and bandwidth of the neural rendering process. Simultaneously, the estimated error classification results are used to generate pixel generators with small parameters for low-error pixels, avoiding wasted computation and achieving adaptive allocation of computing power. Furthermore, it can be easily deployed and integrated into existing neural rendering methods, offering simplicity and efficiency. It can improve the rendering efficiency of existing methods with high efficiency, low throughput, and low memory overhead while ensuring rendering quality. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a global neural rendering method based on sparse adaptive sampling provided in one embodiment; Figure 2 This is a flowchart of error estimation, prediction, and ranking statistics provided in one embodiment; Figure 3 This is a flowchart of a multi-load pixel decoding process consisting of pixel generators with different parameter values ​​provided in one embodiment; Figure 4 This is a schematic diagram of the structure of a global neural rendering system based on sparse adaptive sampling provided in one embodiment. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0021] The inventive concept of this invention is as follows: This invention primarily proposes a general method to accelerate existing neural rendering methods. Existing neural rendering methods use the same densely computed pixel generator for all pixels, meaning they compute pixel decoding with the same number of parameters for each pixel. This ignores the sparsity of the rendering signal itself, resulting in redundant computation in low-frequency rendering regions, wasting computational resources and affecting the overall computational performance of the method. Therefore, this invention provides a global neural rendering method and system based on sparse adaptive sampling. The goal is to solve the load distribution problem in the neural rendering process by improving the original balanced allocation strategy. Through adaptive sampling, more computational budget is allocated to complex regions, while simple regions use a smaller sample budget. This achieves a higher frame rate with lower computational load and the same image quality. This invention can be seamlessly integrated into existing mainstream neural rendering methods for computational acceleration, supporting higher frame rates and higher resolution real-time realistic rendering while maintaining image quality.

[0022] Figure 1 This is a flowchart of a global neural rendering method based on sparse adaptive sampling provided in one embodiment. For example... Figure 1 As shown in the embodiment, the global neural rendering method based on sparse adaptive sampling includes the following steps: S1 uses a neural rendering infrastructure to perform frame-by-frame pre-computation to obtain global scene representation information and low-quality pre-rendered results.

[0023] And use a pixel generator with extremely low parameter count and lightweight computation to pre-generate low-quality pre-drawn results.

[0024] In this embodiment, a three-dimensional spatial location neural field is used as a complete data representation of the global scene. Specifically, firstly, geometric buffer data is obtained by drawing batches of lightweight rasterization data. The spatial location coordinates in the data are used as keys to query the latent vector representation corresponding to each pixel position in the three-dimensional spatial location neural field. When there are multiple neural fields in the scene, the average of the multiple latent vector representations is calculated to obtain a latent vector containing global scene representation information, which is used as input for subsequent pixel error estimation.

[0025] To perform effective posterior estimation, this embodiment performs lightweight pre-rendering computation, using a lightweight pixel generator with extremely low parameter count to pre-generate low-quality pre-rendering results. Specifically, the pixel generator with a large number of learnable parameters in the original neural rendering method is replaced with a lightweight version with extremely low parameter count for low-quality pre-rendering, forming a coarse estimate of the complete rendering result. In this embodiment, a lightweight version with 1 / 16th the parameter count of the original pixel generator is used to construct the pre-rendering, achieving a trade-off between pre-computation efficiency and quality.

[0026] Through preliminary pre-computation processing, the latent vectors of global scene representation information corresponding to each pixel position and low-quality pre-drawing results are obtained. In this embodiment, these data are used as the main data features for subsequent calculations.

[0027] S2 estimates the rendering error between the pre-rendered results and the reference true value results based on the low-quality pre-rendered results and combined with global scene representation information.

[0028] In this embodiment, the low-quality pre-drawing results are combined with global scene representation information as input information for the error estimation network, thereby analyzing and predicting the pixel-by-pixel drawing calculation frequency and complexity, and estimating the drawing error between the pre-drawing results and the reference true value results.

[0029] In this embodiment, the latent vectors of the low-quality rendering results and global scene representation information still lack a complete representation of the rendering process. Therefore, additional auxiliary information is used to improve the completeness of the data representation. First, the additional scene geometric material information contained in the geometry buffer (G-buffer), including object surface normal vectors, object albedo texture maps, roughness attributes, metallic attributes, and camera viewpoint direction information, is used. Second, the intermediate hidden features of the lightweight pixel generator during the low-quality rendering process are used. These additional information are then concatenated with the latent vectors of the low-quality rendering results and global scene representation information along the channel dimension to form complete input features.

[0030] In this embodiment, complete input features are fed into an error estimation neural network for computation. This network predicts a single scalar value for each pixel, representing the prediction error between the pre-drawn result and the ground truth reference for that pixel. During the training phase of the entire pipeline, the true error between the pre-drawn result and the ground truth reference is first calculated, and then used as a supervisory signal to train the prediction error output by the error estimation network. To simulate the perceptual nature of the human visual system, only the pixel-by-pixel average subjective rating between the pre-drawn result and the ground truth reference is measured using an additional pre-trained model during the training of the error estimation network. This rating is used as an additional weight to weight the loss function of the prediction error, resulting in higher estimation accuracy for visually sensitive areas of the human eye.

[0031] In the embodiments, the error estimation network is preferably a multilayer perceptron neural network, a convolutional neural network, or an attention mechanism neural network.

[0032] S3, statistically analyze the distribution of drawing errors, and based on the statistical results, reorganize all pixels into the same group of pixels corresponding to each error level according to the error level.

[0033] In this embodiment, the distribution of drawing errors is statistically analyzed, such as by performing percentile statistics on the drawing errors. Based on the statistical results, each pixel is divided into different error levels, and pixels with the same error level are rearranged and grouped into the same group of pixel data.

[0034] like Figure 2 As shown, the specific error value corresponding to the predefined percentile is calculated in parallel for the estimated drawing error values ​​of all pixels: First, the extreme values ​​are counted for subsequent histogram statistical calculation. In this embodiment, the number of histogram units N=64 is used; then, the error range of the pixel estimation is divided into the corresponding histogram units to efficiently calculate the specific error value corresponding to the predefined percentile of all drawing errors in the screen space.

[0035] After obtaining the specific error value corresponding to the predefined percentile, the error level is determined based on the specific error value corresponding to the percentile. Each two adjacent percentiles correspond to specific error values ​​to form an error level. Each pixel is classified into the corresponding error level according to the drawing error, and pixels with the same error level are rearranged and batched into the same group of pixel data in a parallel collection manner.

[0036] S4 calls the pixel generator with corresponding parameters for each set of pixel data corresponding to each error level, completes the drawing of the corresponding pixel, and puts the pixel-by-pixel drawing result back to the original pixel position to obtain the final drawn image.

[0037] In this embodiment, several pixel generators with different parameter values ​​are predefined based on a neural network for subsequent rendering. Different parameter values ​​represent different rendering complexities; a higher parameter value indicates a larger and more powerful pixel generator, but also a higher computational budget. This embodiment selects M=4 pixel generators, and the number of neural network parameters is gradually increased in quadratic increments. Next, a total expected computational cost is predefined to allocate the proportion of pixels processed by each pixel generator. This embodiment selects allocation ratios of 5%, 15%, 30%, and 50% to achieve a trade-off between quality and efficiency.

[0038] In the embodiments, such as Figure 3 As shown, according to the error level classification results, the corresponding pixel data of the same group are sent to the corresponding pixel generator for neural rendering based on multiple load decoding, and the pixel-by-pixel rendering results are redistributed to the original pixel positions according to the sequence information of the batching process to obtain the final rendering result.

[0039] The global neural rendering method based on sparsity adaptive sampling provided in the above embodiments includes a process for efficiently estimating the error of the pre-rendering result. It can effectively learn the sparsity in a rendered image and adaptively render pixels of different frequencies with different levels of complexity, thereby significantly shortening the rendering time while ensuring the quality of the rendered image.

[0040] The global neural rendering method based on sparse adaptive sampling provided in the above embodiments includes a batching and divergent data rearrangement process, as well as parallel error estimation and adaptive pixel generation processes. The entire method only requires one forward computation during execution, and the percentile statistics and data rearrangement calculations are both parallel algorithms. Therefore, the entire acceleration method can be conveniently integrated into various existing neural rendering methods as an independent plug-in module. It can obtain high-quality neural rendering images in real time with only a small computational budget, which is simple and efficient.

[0041] like Figure 4 As shown, the embodiment also provides a global neural rendering system 400 based on sparse adaptive sampling, including: a pre-computation module 410, an error estimation module 420, a classification and rearrangement module 430, and a multi-load rendering module 440. The pre-computation module 410 performs frame-by-frame pre-computation using the neural rendering method infrastructure to obtain global scene representation information and low-quality pre-rendering results. The error estimation module 420 estimates the rendering error between the pre-rendering results and the reference ground truth results based on the low-quality pre-rendering results and the global scene representation information. The classification and rearrangement module 430 statistically analyzes the distribution of rendering errors and, based on the statistical results, reorganizes all pixels according to their error levels into groups of pixels corresponding to each error level. The multi-load rendering module 440 calls a pixel generator with corresponding parameter values ​​for each group of pixels corresponding to each error level to complete the rendering of the corresponding pixels and returns the pixel-by-pixel rendering results to their original positions to obtain the final rendered image.

[0042] It should be noted that the global neural rendering system based on sparse adaptive sampling provided in the above embodiments should be illustrated using the above-described functional module division as an example when performing global neural rendering. The functions described above can be assigned to different functional modules as needed, that is, the internal structure of the terminal or server can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the global neural rendering system based on sparse adaptive sampling provided in the above embodiments and the global neural rendering method based on sparse adaptive sampling belong to the same concept. For details of its specific implementation process, please refer to the global neural rendering method based on sparse adaptive sampling embodiment, which will not be repeated here.

[0043] Based on the same inventive concept, the embodiment also provides a computing device, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the above-described global neural rendering method based on sparse adaptive sampling, specifically including the following steps: S1 uses a neural rendering infrastructure to perform frame-by-frame pre-computation to obtain global scene representation information and low-quality pre-rendered results; S2, based on the low-quality pre-drawn results and combined with global scene representation information, estimates the drawing error between the pre-drawn results and the reference true value results; S3, statistically analyze the distribution of drawing errors, and based on the statistical results, reorganize all pixels into the same group of pixels corresponding to each error level according to the error level; S4 calls the pixel generator with corresponding parameters for each set of pixel data corresponding to each error level, completes the drawing of the corresponding pixel, and puts the pixel-by-pixel drawing result back to the original pixel position to obtain the final drawn image.

[0044] The computing device provided in this embodiment, at the hardware level, includes not only a processor and memory, but also internal buses, network interfaces, memory, and other hardware required for business operations. The memory is non-volatile memory. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the global neural rendering method based on sparse adaptive sampling described in S1-S4 above. Of course, besides software implementation, this invention does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution entity of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0045] Based on the same inventive concept, the embodiments also provide a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the above-described global neural rendering method based on sparse adaptive sampling, specifically including the following steps: S1 uses a neural rendering infrastructure to perform frame-by-frame pre-computation to obtain global scene representation information and low-quality pre-rendered results; S2, based on the low-quality pre-drawn results and combined with global scene representation information, estimates the drawing error between the pre-drawn results and the reference true value results; S3, statistically analyze the distribution of drawing errors, and based on the statistical results, reorganize all pixels into the same group of pixels corresponding to each error level according to the error level; S4 calls the pixel generator with corresponding parameters for each set of pixel data corresponding to each error level, completes the drawing of the corresponding pixel, and puts the pixel-by-pixel drawing result back to the original pixel position to obtain the final drawn image.

[0046] In this embodiment, the computer-readable medium includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data.

[0047] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A global neural network rendering method based on sparse adaptive sampling, characterized in that, Includes the following steps: The neural rendering infrastructure is used to perform frame-by-frame pre-computation to obtain global scene representation information and low-quality pre-rendered results. The input to the error estimation network is composed of low-quality pre-drawing results and global scene expression information. The error estimation network is used to analyze and predict the rendering calculation frequency and complexity of each pixel and estimate the rendering error between the pre-drawing results and the reference true value results. Percentile statistics are performed on the drawing error. The specific error value corresponding to the predefined percentile is calculated in parallel for the drawing error values ​​estimated for all pixels. The error level is determined based on the specific error value corresponding to the percentile. The specific error values ​​corresponding to every two adjacent percentiles form an error level. Each pixel is classified into the corresponding error level according to the drawing error. Pixels with the same error level are rearranged and batched into the same group of pixel data in parallel. For each set of pixel data corresponding to each error level, the corresponding pixel generator is called to complete the drawing of the corresponding pixel, and the pixel-by-pixel drawing result is put back into the original position of the pixel to obtain the final drawn image.

2. The global neural network rendering method based on sparse adaptive sampling according to claim 1, characterized in that, The calculation process of the global scene representation information includes: Using a three-dimensional spatial position neural field as a complete data representation of the global scene, geometric buffer data is obtained by batch rendering of lightweight rasterization data. The spatial position coordinates in the geometric buffer data are used as keys to query the latent vector representation corresponding to each pixel position in the three-dimensional spatial position neural field. When there are multiple neural fields in the scene, the average of the multiple latent vector representations is calculated to obtain a latent vector containing global scene representation information. The calculation process for the low-quality pre-drawn results includes: using a lightweight pixel generator with extremely low parameter count in the infrastructure to pre-generate low-quality pre-drawn results.

3. The global neural network rendering method based on sparse adaptive sampling according to claim 1, characterized in that, When using an error estimation network to estimate rendering errors, its input also includes additional scene geometry and material information contained in the geometry cache, as well as hidden features from intermediate processes generated during low-quality rendering.

4. The global neural network rendering method based on sparse adaptive sampling according to claim 1, characterized in that, When the error estimation network is trained, the actual error between the pre-drawn result and the true reference result is used as a supervision signal to train the prediction error output by the error estimation network.

5. The global neural network rendering method based on sparse adaptive sampling according to claim 4, characterized in that, When training the error estimation network, an additional pre-trained model is used to measure the pixel-wise average subjective score between the pre-drawn results and the ground truth reference results, which is used as an additional weight to weight the loss function of the prediction error.

6. A global neural rendering system based on sparse adaptive sampling, implemented using the global neural rendering method based on sparse adaptive sampling as described in any one of claims 1 to 5, characterized in that, include: The pre-computation module is used to perform frame-by-frame pre-computation using the neural rendering method infrastructure to obtain global scene representation information and low-quality pre-rendered results. The error estimation module is used to estimate the drawing error between the pre-drawn results and the reference true value results based on the low-quality pre-drawn results and combined with global scene expression information. The classification and rearrangement module is used to statistically analyze the distribution of drawing errors. Based on the statistical results, it reorganizes all pixels into the same group of pixels corresponding to each error level according to the error level. The multi-load drawing module is used to call the pixel generator with corresponding parameters for each set of pixel data corresponding to each error level, complete the drawing of the corresponding pixel, and put the pixel-by-pixel drawing result back to the original position of the pixel to obtain the final drawn image.

7. A computing device comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that, When the one or more processors execute the executable code, they are used to implement the global neural rendering method based on sparse adaptive sampling as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the global neural rendering method based on sparse adaptive sampling as described in any one of claims 1-5.

Images