Rendering controller for rendering light in three-dimensional scene and method therefor

Abstract

A rendering controller is provided for rendering light in a 3D scene that includes one or more objects, e.g., each object associated with a plurality of surface reservoirs. The rendering controller is also configured to provide optical transmission calculations in a texture space using the RIS or the WRS according to the surface cache. A rendering controller is used to efficiently compute shading (i.e., direct illumination and indirect illumination) of a 3D scene in a multi-viewer application involving complex illuminated scenes present in the cloud.

Description

Technical Field

[0001] This invention generally relates to the field of virtual environments, and more specifically, to a rendering controller for rendering light in a three-dimensional (3D) scene and a method for using the rendering controller. Background Technology

[0002] Typically, with the development of three-dimensional (3D) technology, 3D virtual environments have become widespread. 3D virtual environments offer new ways to virtually explore virtual spaces designed in 3D, such as 3D meeting rooms, 3D classrooms, and 3D museums. They are also used for playing video games or for virtual events, such as video conferencing. In a 3D virtual environment, a 3D scene is rendered through one or more viewpoints. For example, in a 3D video game running in a 3D virtual environment, numerous players or viewers view the same 3D scene from different locations with different views. In such a scene, the views of multiple viewers may partially or completely overlap. In a 3D virtual environment, rendering is performed to create the final visual appearance of the 3D scene as a two-dimensional (2D) image. The appearance of the surfaces of the 3D scene is created using one or more virtual light sources that illuminate different points on the 3D scene to generate realistic 2D images during rendering. Traditionally, complex calculations are required to determine the appearance of each surface of the 3D scene in a 3D virtual environment. Furthermore, global illumination is needed to determine the amount of light directed towards each point on the viewing surface of the 3D scene. Furthermore, complex material models are evaluated along with various effects that relate to the final appearance of each 3D scene in the 3D virtual environment. However, in complex lighting scenes with numerous moving light sources in a 3D virtual environment, calculating the effects of all moving light sources during rendering is costly.

[0003] Currently, some attempts have been made to address the problem of calculating multiple moving light sources during rendering in 3D virtual environments. One approach involves using separate instances of the application running in the cloud, configured alongside the 3D virtual environment. Each instance communicates with a central server in the cloud to receive updates from the 3D virtual environment or data needed for 2D image generation, such as geometry, lighting configurations, and physical updates. Each instance independently calculates the appearance of all visible screen fragments for each viewer and streams the resulting images onto the monitor. Rendering for each client is driven by the current view location and focuses on each viewer's screen space. In other words, rendering is output-driven, for example, tracing the light rays or paths entering the 3D scene from each individual client's current view and calculating special effects in screen space, such as screen space ambient occlusion. This traditional approach allows for rendering calculations of the same 3D scene separately and independently for each viewer. Furthermore, typical statistical techniques such as importance sampling are used to estimate shading in complexly lit scenes. Traditional importance sampling techniques primarily focus on selecting the best samples to estimate the integrand of the rendering equations (i.e., selecting samples from a probability distribution that approximates the distribution of the incident light and / or the material response of the shading points). Traditionally, certain statistical methods are used to preserve the light samples most meaningful for both direct and indirect illumination estimations. However, the problem with these statistical methods is that their screen-space nature presents a technical challenge: sharing and reusing computations across different viewers to use the best light samples for different viewers or purposes. Furthermore, these statistical methods rely on cost-intensive ad hoc reprojections (i.e., using motion vectors) to account for camera movement and on spatial adjacency filtering (e.g., checking if sampling points are on the same surface) to produce consistent and efficient spatiotemporal resampling. Therefore, the technical problem is how to efficiently compute the shading (i.e., direct and indirect illumination) of 3D scenes in multi-viewer applications involving complex lighting scenarios existing in the cloud.

[0004] Therefore, based on the above discussion, it is necessary to overcome the aforementioned shortcomings associated with traditional methods of calculating shading in complex lighting scenes. Summary of the Invention

[0005] This invention provides a rendering controller for rendering light in a three-dimensional (3D) scene and a method for using the rendering controller. The invention provides a solution to the existing problem of how to efficiently compute shading (i.e., direct and indirect lighting) of a 3D scene in a multi-viewer application involving complex lighting scenes existing in the cloud. The object of this invention is to provide a solution that at least partially overcomes the problems encountered in the prior art, and to provide a rendering controller and a method for using the rendering controller to render light in a three-dimensional (3D) scene, the method involving surface-weighted reservoir resampling for sharing lighting computation across time and space in a multi-viewer scene.

[0006] One or more objects of the present invention are achieved by the solutions provided in the appended independent claims. Advantageous embodiments of the invention are further defined in the dependent claims.

[0007] In one aspect, the present invention provides a rendering controller for rendering light in a 3D scene comprising one or more objects, wherein each object is associated with a plurality of surface reservoirs, wherein the rendering controller is further configured to provide light transport calculations in texture space based on surface caches using RIS or WRS.

[0008] The render controller is used to efficiently compute shading (i.e., direct and indirect lighting) of 3D scenes in multi-viewer applications with complex lighting scenarios existing in the cloud. The render controller renders light in a 3D scene comprising one or more objects, for example, by applying RIS or WRS to surface reservoirs to achieve efficient single and multi-viewer direct and indirect light sampling. The render controller provides surface-weighted reservoir sampling (i.e., “time-independent” resampling) for sharing lighting computations across time and space in multi-viewer scenes. In other words, the render controller fires rays for surface-weighted reservoir resampling without a specific frame time, because the surface spatial reservoir does not change position over time, and sampling can occur at any point. Furthermore, the render controller stores surface reservoirs in a surface cache, such as multiple surface reservoirs directly on the surface of an object. Advantageously, contiguous surface spatial reservoirs, compared to screen space, allow RIS to be performed using surrounding surface points that are not visible. The render controller can directly query adjacent reservoirs. Additionally, if multiple viewers or rays from other different sources hit the same surface point, reservoir sampling can be used to guide light sampling from that location. Therefore, there is no need to check whether the surrounding points meet any depth thresholds, etc.

[0009] In another aspect, the present invention provides a method for a rendering controller for rendering light in a 3D scene comprising one or more objects, wherein each object is associated with a plurality of surface reservoirs, wherein the method includes providing light transport calculations in texture space based on surface caches using RIS or WRS.

[0010] This method achieves all the advantages and technical effects of the rendering controller of the present invention.

[0011] It should be understood that all of the above implementations can be combined. It should be noted that all devices, elements, circuits, units, and modules described in this application can be implemented in software or hardware elements or any combination thereof. All steps performed by the various entities described in this application, and the functions to be performed by the various entities described, are intended to refer to the respective entities performing the respective steps and functions. Although in the following description of specific embodiments, the specific functions or steps performed by external entities are not reflected in the detailed description of the specific elements of the entities performing the specific steps or functions, it will be apparent to those skilled in the art that these methods and functions can be implemented by corresponding hardware or software elements or any combination thereof. It should be understood that the features of the invention are readily combined in various ways without departing from the scope of the invention as defined by the appended claims.

[0012] Additional aspects, advantages, features and objects of the invention will become apparent from the accompanying drawings and the detailed description of illustrative implementations as explained in conjunction with the following appended claims. Attached Figure Description

[0013] The above-described invention and the following detailed description of illustrative embodiments can be better understood when read in conjunction with the accompanying drawings. Exemplary structures of the invention are shown in the drawings to illustrate the invention. However, the invention is not limited to the specific methods and tools disclosed herein. Furthermore, those skilled in the art will understand that the drawings are not drawn to scale. Where possible, the same elements are represented by the same numbers.

[0014] The following figure is used as an example to illustrate an embodiment of the present invention.

[0015] Figure 1 This is a block diagram of a rendering controller according to an embodiment of the present invention.

[0016] Figure 2 This is a flowchart of a method for a rendering controller according to an embodiment of the present invention, wherein the rendering controller is used to render light in a three-dimensional (3D) scene.

[0017] Figure 3The figures illustrate exemplary scenarios of screen space reservoirs and surface space reservoirs in the on-surface weighted reservoir resampling (OSWRR) technology according to embodiments of the present invention.

[0018] Figure 4 This is a diagram of an exemplary scenario of a surface buffer reservoir in the on-surface weighted reservoir resampling (OSWRR) technique for performing direct illumination estimation according to embodiments of the present invention.

[0019] Figure 5 This is an exemplary diagram of an exemplary scenario in which the on-surface weighted reservoir resampling (OSWRR) technique is applied to global illumination estimation according to an embodiment of the present invention.

[0020] Figure 6 An exemplary diagram is shown illustrating an exemplary scenario in which on-surface weighted reservoir resampling (OSWRR) technology is applied to path guidance according to an embodiment of the present invention.

[0021] Figure 7 An exemplary diagram is shown illustrating an exemplary scenario of applying on-surface weighted reservoir resampling (OSWRR) to multiple ejection global illumination calculations according to an embodiment of the present invention.

[0022] In the accompanying diagrams, underlined numbers indicate the item in which the underlined number appears or the item adjacent to the underlined number. Ununderlined numbers relate to the item identified by the line that associates the ununderlined number with the item. When a number is ununderlined and has an associated arrow, the ununderlined number is used to identify the general item that the arrow points to. Detailed Implementation

[0023] The following detailed description illustrates embodiments of the present invention and ways in which these embodiments can be implemented. While some modes of implementing the invention have been disclosed, those skilled in the art will recognize that other embodiments for implementing or practicing the invention may also exist.

[0024] Figure 1 This is a block diagram of a rendering controller according to an embodiment of the present invention. (Reference) Figure 1The diagram shows a block diagram 100 of a rendering controller 102, which includes a communication interface 104 and a memory 106.

[0025] Render controller 102 is used to render light in a three-dimensional (3D) scene 108, which includes one or more objects 110. Render controller 102 may include appropriate logic, circuitry, interfaces, and / or code for providing shared access to the 3D scene 108 to multiple viewers. In one example, 3D scene 108 refers to a 3D virtual environment or virtual construct (e.g., a virtual model) designed using any appropriate 3D modeling techniques and computer-assisted drawing (CAD) methods that support exploration and communication between users through their corresponding virtual avatars. Examples of 3D scene 108 may include, but are not limited to, 3D roller coasters, 3D haunted houses in amusement parks, amusement parks, 3D video games, 3D museums, 3D cities, schools, factories, any venue, etc. Furthermore, one or more objects 110 in 3D scene 108 are virtual objects simulating real objects, which may include, but are not limited to, vehicles (e.g., cars), multiple vehicles, multiple buildings, etc. Additionally, when a viewer changes its position in 3D scene 108, or when the scene view changes, one or more objects 110 will affect each scene view. In one implementation, a change in the position of each viewer causes a change in the viewpoint of one or more objects 110, thereby affecting the scene view of the 3D scene 108. In another implementation, a change in the position of each viewer causes a new object to appear, affecting the scene view of the 3D scene 108.

[0026] Examples of implementations of the rendering controller 102 may include, but are not limited to, processors, digital signal processors (DSPs), microprocessors, microcontrollers, complex instruction set computing (CISC) processors, application-specific integrated circuit (ASIC) processors, reduced instruction set computing (RISC) processors, very long instruction word (VLIW) processors, state machines, data processing units, graphics processing units (GPUs), and other processors or control circuits. The communication interface 104 may include appropriate logic, circuitry, and / or interfaces for sending data processed by the rendering controller 102 to multiple viewers. Examples of implementations of the communication interface 104 may include, but are not limited to, network interfaces, computer ports, network sockets, network interface controllers (NICs), and any other network interface devices. The memory 106 may include appropriate logic, circuitry, and / or interfaces for storing data related to the 3D scene 108. Examples of implementations of memory 106 may include, but are not limited to, electrically erasable programmable read-only memory (EEPROM), dynamic random-access memory (DRAM), random access memory (RAM), read-only memory (ROM), hard disk drive (HDD), flash memory, secure digital (SD) card, solid-state drive (SSD), and / or CPU cache memory.

[0027] Furthermore, each of the one or more objects 110 is associated with multiple surface reservoirs. In one implementation, the multiple surface reservoirs correspond to reservoirs of surface caches. Additionally, the rendering controller 102 is used to utilize a view-independent dynamic caching structure, rather than individually computing all rendered views of the one or more objects 110, which allows sharing rendering computations with different viewers in the 3D scene 108. In one implementation, surface reservoirs store computations as caches. Surface reservoirs share all view-independent rendering computations with multiple viewers when views overlap, while sharing view-dependent computations is advantageous when views do not overlap. Surface reservoirs help reduce the computational cost of each view for each viewer (i.e., the processing time or power consumed by any system performing the rendering). Furthermore, the rendering controller 102 is also used to provide phototransmission computations in texture space based on the surface cache using resampled importance sampling (RIS) or weighted reservoir sampling (WRS). In one implementation, RIS is a statistical technique used in rendering to calculate lighting and shading effects in a 3D scene 108 with complex geometry and materials. In another implementation, WRS is a statistical technique that reduces the number of samples required during rendering. In WRS, lighting samples are collected at specific locations within the 3D scene 108, and the contribution of each sample is weighted according to its importance to form the final image.

[0028] In one implementation, the rendering controller 102 utilizes WRS and RIS in the texture space (i.e., surface space) of one or more objects 110 based on multiple surface reservoirs to overcome screen space constraints. Screen space constraints refer to limitations on what can be displayed on each viewer's screen. Examples of such limitations may include screen resolution, viewport size, available graphics processing power of the system configuring the screen, etc. Using WRS and RIS in texture space enables efficient single and multi-viewer direct and indirect light sampling. The advantages of applying WRS and RIS in texture space are the elimination of the need for ad hoc reprojection (e.g., in the case of conventional sampling techniques) and the simplification of spatial filtering by operating on the surfaces of one or more objects 110 in the 3D scene 108. Multiple surface reservoirs reside directly on the surfaces of one or more objects 110 and are used to store statistics. Furthermore, spatiotemporal reuse of samples in texture space allows multiple users with multiple viewpoints to access data and samples. In other words, reservoir sampling is used to direct light sampling to different light sources if multiple viewers or rays from other light sources hit the same surface point on one or more objects 110. In one implementation, the source of the light can be multiple viewers, stereo rendering, or light rays bouncing around in the 3D scene 108 (e.g., global illumination path tracing). In one implementation, RIS selects samples from a source distribution and generates a weighted subset of the samples using a target probability distribution function that approximates the integrand of the rendering equation. The rendering equation corresponds to a mathematical expression describing the interaction between light emitted from one or more light sources and the surfaces of one or more objects 110. In one implementation, the rendering controller 102 calculates the final radiation of points on the surfaces of one or more objects by integrating the radiation from all incoming directions. Utilizing RIS based on surface caching facilitates addressing the problem of one or more light sources by generating samples based on an approximate distribution of incident light from one or more light sources, or the product of all terms used for shading. In one implementation, RIS is executed in conjunction with WRS, which facilitates efficient operation of RIS with a low memory footprint. In WRS, light sample data from one of the multiple light sources is stored in a reservoir along with statistical parameters. New input samples generated using ray tracing update the reservoir for each frame. Furthermore, each new sample is selected or discarded based on a weighted probability.

[0029] According to one embodiment, a surface cache layer is located on the surface of a 3D object in a surface cache texture. In this embodiment, the surface cache layer includes a light sample point description, RIS weights, the number of seen samples, and the sum of their weights used for normalization. In one example, the light sample point description may correspond to triangle indices, UV coordinates, and outgoing radiation. In one implementation, the surface cache layer data is packaged into three half4 data structures, which are stored in the surface cache texture. For example, the total memory footprint of one cache layer is... Bytes = 24 bytes. In one example, the surface cache layer is stored directly on the surface of the object, making any reuse between different viewers impossible, such as... Figure 3 Further illustrations and descriptions are provided.

[0030] According to one embodiment, the surface cache layer enables the use of sampled information independently of the observer, view direction, or the direction of light rays departing from the surface location of the surface cache layer. Multiple surface cache layers are stored on the surface of one or more objects 110 (i.e., virtual objects) in the 3D scene 108. In one implementation, the surface cache layer is a view-independent data structure, which allows the use of sampled information independent of the observer and view direction. The advantage of using observer-independent sampled information is that it avoids the problem of view overlap from multiple viewers.

[0031] According to one embodiment, the rendering controller 102 is further configured to guide light sampling for direct illumination estimation and indirect illumination estimation based on a surface buffer. In one implementation, direct illumination refers to an illumination type in which multiple light sources in the 3D scene 108 directly illuminate one or more objects 110. Direct illumination is achieved by sampling the light sources based on buffer information to estimate the light received at a surface point from the multiple light sources, and then determining how the light interacts with the surfaces of one or more objects 110 in the 3D scene 108. In one implementation, indirect illumination refers to an illumination type in which simulated light bounces off the surface of one of the one or more objects 110 and illuminates another object in the one or more objects 110 in the 3D scene 108. In one implementation, the rendering controller 102 is configured to guide light sampling for direct illumination estimation and indirect illumination estimation, as well as path guidance. In this implementation, path guidance refers to tracing the light path from the camera toward the multiple light sources while considering bounces from one or more objects 110 in the 3D scene 108, and then using information collected along the corresponding paths to determine the color of each pixel in the final rendered 2D image. In one implementation, a surface reservoir is used to guide light sampling from surface points, which are the surfaces that light encounters as it bounces through the scene.

[0032] According to one embodiment, the rendering controller 102 is also used to reuse data from a single surface space reservoir to compute lighting effects for multiple viewpoints. Multiple surface reservoir (i.e., surface space reservoir) shading allows for the reuse of data from a single surface space reservoir to compute lighting effects for multiple viewpoints. Multiple surface reservoirs are generated for one view and reused for other views in the 3D scene 108. Furthermore, one or more sampling procedures can also be used for other views in the 3D scene 108, which occurs within the same virtual scene. In one implementation, the rendering controller 102 is used to reuse surface space reservoirs generated for one view or sampling procedure for other views or sampling procedures in the 3D scene 108. In another implementation, the frameless nature of surface space reservoirs facilitates sampling and updating of reservoirs independently of rendering frames or viewers. In one implementation, the rendering controller 102 is used to perform resampling on the surface of one or more objects without being affected by time (i.e., without a specific frame time). Furthermore, due to the frameless nature of the surface space reservoir, light rays from one or more light sources overlap on one or more objects 110 for resampling without requiring a specific frame time (i.e., unlike conventional rendering techniques that require frames with motion vectors). The rendering controller 102 prevents the position of the surface space reservoir on one or more objects 110 from changing over time, which facilitates sampling at any point on the surface of one or more objects 110.

[0033] According to one embodiment, the rendering controller 102 is also configured to access adjacent reservoirs on the surface of an object to facilitate resampling of the surface cache, thereby efficiently sharing candidate samples across multiple objects. The advantage of accessing adjacent reservoirs is that contiguous surface spaces can utilize RIS at points in the unseen surrounding area without checking whether the corresponding surrounding points satisfy any depth threshold. These surrounding points are checked by sending queries to adjacent reservoirs. In one implementation, the generation and sampling of the surface cache (which is invisible in all views but guided by the presence of other reservoirs) allows for more efficient resampling. In one implementation, the rendering controller 102 uses different resolutions when storing the surface cache. Furthermore, the rendering controller 102 is configured to resample the surface cache at various resolutions, which facilitates efficient coverage of different distances on the surface of one or more objects 110. According to one embodiment, the rendering controller 102 is also configured to receive visible texels and corresponding cache resolutions in all viewers. In one implementation, visible texels are the basic units of a texture map. In this implementation, the texture is represented by an array of visible texels (i.e., similar to an array of pixels in an image) representing the texture space. Furthermore, for each visible texel, the rendering controller 102 is also used to update the corresponding surface buffer layer using initial sampling, temporal resampling, and spatial resampling. Additionally, during initial sampling, the rendering controller 102 is used to generate new random light samples. According to one embodiment, during initial sampling, the rendering controller 102 is used to receive an initial sample count N, receive texel information and associated geometric data, and create a surface buffer layer. Thereafter, the rendering controller 102 is also used to generate N initial samples and update the buffer layer. Furthermore, the rendering controller 102 is also used to evaluate the visibility of the resulting samples in the buffer layer and calculate the weights of the resulting samples accordingly.

[0034] Furthermore, during temporal resampling, the rendering controller 102 updates the temporary reservoir at the current texel using new light samples. Additionally, the rendering controller 102 reads the temporary reservoir cache texture and samples the temporary cache texture at the same texel location to obtain the cached reservoir from the last frame. Subsequently, the rendering controller 102 combines the last frame cache layer and the newly generated reservoir in a new output cache layer. Furthermore, the rendering controller 102 recalculates the weights of the obtained samples in the output cache layer and writes them to the temporary reservoir cache texture. Subsequently, the rendering controller 102 performs spatial resampling, for example, selecting a randomly adjacent temporary reservoir and updating the current spatial reservoir using the selected randomly adjacent temporary reservoir. Furthermore, for each viewer, the rendering controller 102 samples the spatial reservoir texture at the correct cache resolution and uses the reservoir content for direct lighting shading. According to one embodiment, during spatial resampling, the rendering controller 102 receives a neighborhood radius R.n Count N with neighbors n The rendering controller 102 then samples the temporary cache texture to obtain the temporary cache of the newly generated cache. Furthermore, for N... n 102 is used to render adjacent samples in R n Random texel offsets are generated in the texel neighborhood, and a temporary reservoir cache texture is received. Subsequently, the rendering controller 102 is used for: for N n For N neighboring samples, the temporary reservoir cache texture at randomly adjacent texels is sampled. Furthermore, the rendering controller 102 is used to: for N n For each of the N neighboring samples, the current texel temporary storage is updated using the neighbor storage. Finally, the rendering controller 102 is used to: for N n The system evaluates the visibility of the resulting sample in the current cached reservoir by considering several neighboring samples. Furthermore, the rendering controller 102 calculates the weights of the resulting samples accordingly and writes them to the spatial reservoir cache texture.

[0035] According to one embodiment, during reservoir shading, the rendering controller 102 is further configured to receive spatial reservoir cache textures for each viewer and for each visible texel. Furthermore, the rendering controller 102 is also configured to receive cached spatial reservoirs to obtain associated light sample data for each viewer and for each visible texel. Subsequently, the rendering controller 102 is configured to calculate outgoing radiation by interpolating this data into the rendering equations for each viewer and each visible texel. Additionally, the rendering controller 102 is configured to determine weights based on the weights of the cached reservoirs and the output pixel color values ​​for each viewer and each visible texel, such as... Figure 4 Further illustrations and descriptions are provided below.

[0036] Render controller 102 is used to render light in a 3D scene 108 comprising one or more objects 110, for example by applying RIS or WRS to surface reservoirs to achieve efficient single and multi-viewer direct and indirect light sampling. Render controller 102 provides surface-weighted reservoir sampling (i.e., “time-independent” resampling) for sharing lighting computations across time and space in a multi-viewer scene. In other words, render controller 102 emits rays for surface-weighted reservoir resampling without a specific frame time, because surface-space reservoirs do not change position over time, and sampling can occur at any point. Furthermore, render controller 102 is used to store surface reservoirs in a surface cache, such as multiple surface reservoirs directly on the surface of an object. Advantageously, contiguous surface-space reservoirs, compared to screen space, allow RIS to be performed using surrounding surface points that are not visible. Render controller 102 can directly query adjacent reservoirs. Furthermore, if multiple viewers or rays from other different sources hit the same surface point, reservoir sampling can be used to guide light sampling from that location. Therefore, it is not necessary to check whether surrounding points meet any depth thresholds, etc.

[0037] Figure 2 This is a flowchart of a method for a rendering controller according to an embodiment of the present invention, wherein the rendering controller is used to render light in a three-dimensional (3D) scene. Combined with... Figure 1 describe Figure 2 . refer to Figure 2 This shows the rendering ( Figure 1 The flowchart of method 200 of controller 102, rendering controller 102 is used for rendering ( Figure 1 Light in 3D scene 108. Method 200 includes steps 202 to 206.

[0038] The method 200 for rendering controller 102 includes ( Figure 1One or more objects 110 in the 3D scene 108 are used by the rendering controller 102 to render light in the 3D scene. These objects 110 are virtual objects in a 3D virtual environment that mimic real objects in the real environment. Examples of the one or more objects 110 may include, but are not limited to, multiple vehicles, multiple buildings, multiple trees, etc. Furthermore, each of the one or more objects 110 is associated with multiple surface reservoirs. In one implementation, the multiple surface reservoirs are reservoirs of surface caches. In another implementation, the multiple surface reservoirs store computations in the form of caches. Multiple surface reservoirs share all view-independent rendering computations with multiple viewers when views overlap, and share view-dependent computations when views do not overlap. Multiple surface reservoirs help reduce the computational cost (i.e., the processing time or power consumed by any system performing rendering) of generating views for each of the multiple viewers.

[0039] In step 202, method 200 includes allocating surface / texture spatial reservoirs. In other words, method 200 allocates multiple surface reservoirs 306 on the surface of one or more objects 110. According to one embodiment, the surface reservoirs associated with one or more objects 110 in method 200 enable the use of sampling information to utilize the direction of light rays independent of the observer, the viewpoint, or the surface location away from the surface reservoir. In step 204, method 200 includes updating the multiple surface reservoirs 306 using resampled importance sampling (RIS) and weighted reservoir sampling (WRS). In one implementation, RIS is a statistical technique used in 3D rendering to calculate lighting and shading effects in a 3D scene 108 with complex geometry and materials. In another implementation, WRS is a statistical technique that reduces the number of samples required during rendering. In WRS, lighting samples are acquired at specific locations in the 3D scene 108, and the contribution of each sample is weighted according to the importance of each sample in the contribution to form the final image. In step 206, method 200 includes sampling a plurality of surface reservoirs 306. According to one embodiment, the surface cache includes a description of optical sample points, RIS weights, the number of seen samples, and a sum of their weights for normalization. Method 200 provides optical transport computation in texture space based on the surface cache using resampled importance sampling (RIS) or weighted reservoir sampling (WRS).

[0040] According to one embodiment, the rendering controller 102 is further configured to guide light sampling for direct illumination estimation and indirect illumination estimation based on a surface cache layer. In one implementation, direct illumination refers to an illumination type that simulates the way multiple light sources in a 3D scene 108 directly illuminate one or more objects 110. Direct illumination is achieved by calculating light emitted from multiple light sources and then tracing the path of the light to determine how the light interacts with the surfaces of one or more objects 110 in the 3D scene 108. In one implementation, indirect illumination refers to an illumination type that simulates light bouncing off the surface of one of the one or more objects 110 and illuminating another object in the one or more objects 110 in the 3D scene 108. In one implementation, the rendering controller 102 is configured to guide light sampling for direct illumination estimation and indirect illumination estimation, as well as path guidance. In this implementation, path guidance refers to tracing the light path from multiple light sources to the camera and then using information collected along the corresponding path to determine the color of each pixel in the final rendered 2D image.

[0041] According to one embodiment, the rendering controller 102 is further configured to reuse data from a single surface space reservoir to compute lighting effects for one or more viewpoints. In one implementation, one or more viewpoints refer to the location of each of a plurality of viewers viewing one or more objects 110. Each viewer has a corresponding viewpoint. According to one embodiment, the rendering controller 102 is further configured to access adjacent reservoirs on the surface of the object to facilitate resampling of the surface cache layer, thereby efficiently sharing candidate samples across multiple objects. Furthermore, according to one embodiment, the rendering controller 102 is further configured to receive visible texels and corresponding cache resolutions from all viewers among the plurality of viewers. Additionally, the rendering controller 102 is configured to update the corresponding surface cache layer for each visible texel. The rendering controller 102 is configured to generate new random light samples during initial sampling. According to one embodiment, the rendering controller 102 is configured to receive an initial sample count N and receive texel information and associated geometric data to create a surface cache layer during initial sampling. In such an embodiment, the rendering controller 102 is configured to generate N initial samples and update the cache layer. In addition, the rendering controller 102 is used to evaluate the visibility of the obtained samples in the cache layer and calculate the weight of the obtained samples accordingly.

[0042] Furthermore, method 200 includes: the rendering controller 102 updating the temporary reservoir at the current texel with new light samples during temporal resampling. According to one embodiment, method 200 includes: the rendering controller 102 reading the temporary reservoir cache texture during temporal resampling. Furthermore, the rendering controller 102 samples the temporary cache texture at the same texel location to obtain the cached reservoir from the last frame, and combines the last frame cache layer and the newly generated reservoir in a new output cache layer. Furthermore, the rendering controller 102 recalculates the weights of the obtained samples in the output cache layer and writes them to the temporary reservoir cache texture. Furthermore, the rendering controller 102 also selects randomly adjacent temporary reservoirs and uses the selected randomly adjacent temporary reservoirs to update the current spatial reservoir during spatial resampling.

[0043] According to one embodiment, the rendering controller 102 is used to receive the neighborhood radius R. n Count N with neighbors n This is used to read the temporary reservoir cache texture and sample the temporary cache texture to obtain the temporary reservoir of the newly generated cache during spatial resampling. In N n In the case of adjacent samples, the rendering controller 102 in method 200 is used to render in R n Random texel offsets are generated in the texel neighborhood, a temporary reservoir cache texture is received, and the temporary reservoir cache texture at random neighboring texels is sampled during spatial sampling. Furthermore, the rendering controller 102 is used to update the current texel temporary reservoir and neighboring reservoirs, evaluate the visibility of the resulting samples in the currently cached reservoir, and calculate the weights of the resulting samples accordingly. After calculating the weights, method 200 includes: the rendering controller 102 writing the spatial reservoir cache texture. According to one embodiment, the rendering controller 102 in method 200 is used to sample the spatial reservoir texture at the correct cache resolution and use reservoir content for direct lighting shading. According to one embodiment, the rendering controller 102 in method 200 is used to receive the spatial reservoir cache texture for each visible texel corresponding to each of a plurality of viewers, and receive the cached spatial reservoir to obtain associated light sample data. In this embodiment, outgoing radiation is calculated by inserting this data into the rendering equation, and weights are determined based on the weights of the cached reservoir and the output pixel color values.

[0044] A computer program product including program instructions is provided for performing method 200 when executed by one or more processors in a rendering controller system. The computer program product is implemented as an algorithm embedded in software stored in a non-transitory computer-readable storage medium containing the program instructions, which can be executed by one or more processors in the rendering controller system to perform method 200. The non-transitory computer-readable storage component may include, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. Examples of implementations of the computer-readable storage medium include, but are not limited to, electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), read-only memory (ROM), hard disk drive (HDD), flash memory, secure digital (SD) cards, solid-state drives (SSDs), computer-readable storage media, and / or CPU cache memory.

[0045] Figure 3 These figures illustrate exemplary scenarios of screen-space reservoirs and surface-space reservoirs in on-surface-weighted reservoir resampling (OSWRR) technology according to different embodiments of the present invention. Combined with... Figure 1 and Figure 2 Component description Figure 3 . refer to Figure 3 Figure 300 shows the surface space reservoir in the on-surface weighted reservoir resampling (OSWRR) technique.

[0046] In one implementation, in the case of OSWRR technology (such as...) Figure 3 As shown), two of the multiple viewers, such as the first viewer 302 and the second viewer 304, are viewing objects in one or more objects 110. In one implementation, in ( Figure 1In the OSWRR technology configured in the rendering controller 102, the surface cache reservoir 306 is directly stored on the surface of one or more objects 110 via a surface cache texture. Due to the position of the surface cache reservoir 306 on the object's surface, sampling information used for rendering calculations for each of the multiple viewers is utilized, independent of the viewer and the corresponding view. Furthermore, the surface cache reservoir 306 on the objects in one or more objects 110 (such as...) Figure 3 (As shown) Independent of the screen space of each viewer, this helps overcome screen space constraints and enables efficient light sampling for both direct and indirect illumination in multi-viewer applications. Therefore, the OSWRR technique configured in the rendering controller 102 helps reduce the computational cost of generating rendered views compared to conventional methods. Furthermore, the OSWRR technique allows sampling at different resolutions. In one example, MIP chains of the reservoir are generated, and samples can be easily distributed across scales with smooth transitions between MIP levels. Moreover, the OSWRR technique can be used by any source that wishes to sample surface locations, thus enabling its application in various applications such as multi-viewer or recursive ray / path tracing.

[0047] Figure 4 This is a diagram illustrating an exemplary scenario of a surface buffer reservoir in an on-surface weighted reservoir resampling (OSWRR) technique for performing direct illumination estimation according to embodiments of the present invention. Combined with... Figure 1 and Figure 3 describe Figure 4 . refer to Figure 4 This illustrates a technique for performing on-surface weighted reservoir resampling (OSWRR) for direct illumination estimation. Figure 3 An exemplary figure 400 illustrates an exemplary scenario of a surface cache reservoir 306.

[0048] In direct light, ( Figure 1 One or more objects in object 110 are directly illuminated by one or more virtual light sources to provide a realistic view of the objects in the final rendered image. In one example, light source 402 directly illuminates one or more objects in object 110. Light source 402 is a virtual light source, which is composed of ( Figure 1 The rendering controller 102 is used to simulate the behavior of objects during light interaction in order to calculate the rendering of the objects. In this example, ( Figure 3 The first viewer 302 and ( Figure 3The second viewer 304 views the object. During object rendering, light rays from the first viewer 302 and the second viewer 304 (e.g., virtual rays from the first viewer 302 and the second viewer 304) are projected onto the object, and the paths of the virtual rays are traced to form a rendered image for each of the first viewer 302 and the second viewer 304. In direct lighting estimation, the rendering controller 102 simulates the path of light from the light source 402 traveling towards the object. Furthermore, the rendering controller 102 combines the paths of the virtual rays and the light from the light source 402 to calculate the amount of light reflected from the object's surface. In OSWRR technology, the rendering controller 102 reads the spatial reservoir cache texture of each of the first viewer 302 and the second viewer 304 and retrieves the reservoir 306 of the surface cache to obtain associated light sample data corresponding to the light source 402. Furthermore, the rendering controller 102 calculates the reflected light (i.e., outgoing radiation) from the object by integrating the light sample data into the rendering equations. In one implementation, the rendering equation represents the amount of light leaving a point on the surface of an object in a specific direction by summing all incident light from all possible directions. Furthermore, the rendering controller 102 calculates the weights of the surface cache reservoir 306 and determines the color value of each pixel in the output rendered image. The OSWRR technique configured in the rendering controller 102 facilitates efficient rendering computation by storing light samples in the surface cache reservoir 306 and using these light samples for subsequent calculations.

[0049] Figure 5 This is an exemplary diagram illustrating an exemplary scenario in which on-surface weighted reservoir resampling (OSWRR) technology is applied to global illumination estimation according to an embodiment of the present invention. Combined with... Figure 1 , Figure 3 and Figure 4 describe Figure 5 . refer to Figure 5 Figure 500 illustrates an exemplary scenario in which on-surface weighted reservoir resampling (OSWRR) is applied to global illumination estimation.

[0050] In one implementation, global illumination includes indirect illumination. In another implementation, global illumination refers to the illumination of one or more objects 110 by means of light that has been ejected or reflected from surfaces other than the object's surface. Global illumination involves calculating and simulating how light interacts with the object's surface in a more realistic way to produce softer shading and more subtle lighting variations. Advantages compared to conventional methods include... Figure 1The rendering controller 102 applies the OSWRR technique to estimate global illumination by storing illumination samples in a reservoir 306 of the surface cache and using the illumination samples for continuous calculations, thereby reducing computational complexity, even in the case of multiple and dynamic light sources.

[0051] In an exemplary scenario (such as) Figure 5 As shown), a primary light source 502 (e.g., a virtual light source) indirectly illuminates one or more objects in object 110. The primary light source 502 illuminates surfaces other than the corresponding objects, and light reflected from the corresponding surfaces acts as a secondary light source 504, which illuminates the corresponding objects. In this example, the light sample associated with the object is the secondary light source 504. Furthermore, a first viewer 302 and a second viewer 304 view the objects. In operation, the rendering controller 102 projects virtual rays from the first viewer 302 and the second viewer 304, and the paths of the virtual rays are traced to form a rendered image of the object for each of the first viewer 302 and the second viewer 304. In OSWRR technology, the rendering controller 102 reads the spatial reservoir cache texture of each of the first viewer 302 and the second viewer 304 and retrieves ( Figure 3 The surface cache reservoir 306 is used to obtain associated light sample data corresponding to the secondary light source 504. The application of OSWRR to global illumination estimation differs from its application to direct illumination estimation regarding the properties of the light samples stored in the surface cache reservoir 306. In the case of direct illumination, the rendering controller 102 stores samples of the primary light source 502 in the surface cache reservoir 306, while in the case of global (i.e., indirect) illumination, the rendering controller 102 stores light samples associated with the secondary light source 504. Applying OSWRR to global illumination estimation allows for the use of the same statistical tools and the same data structure (i.e., the surface cache reservoir 306) to solve the direct illumination estimation problem, thereby efficiently calculating the effects of direct and indirect illumination on any of one or more objects.

[0052] Figure 6 An exemplary diagram is shown illustrating an exemplary scenario of applying on-surface weighted reservoir resampling (OSWRR) technology to path guidance according to an embodiment of the present invention. Combined with... Figures 1 to 5 Component description Figure 6 . refer to Figure 6 Figure 600 illustrates an exemplary scenario in which on-surface weighted reservoir resampling (OSWRR) technology is applied to path guidance.

[0053] In the exemplary scenario, ( Figure 4 The light source 402 directly illuminates one or more objects in the object 110. Furthermore, three viewers (e.g., first viewer 302, second viewer 304, and third viewer 602) view the corresponding objects. Additionally, ( Figure 1 The rendering controller 102 projects virtual rays from a first viewer 302, a second viewer 304, and a third viewer 602. Virtual rays from the first and second viewers 302 and 304 are reflected toward the light source 402, while virtual rays from the third viewer are reflected toward the light source 402 from surfaces other than the surface of the surface cache reservoir 306 before reaching it. Furthermore, the first and second viewers 302 and 304 view objects under the influence of direct lighting received from the light source 402, while the third viewer 602 views objects under the influence of indirect lighting received from the light source 402. The surface cache reservoir 306 stores light samples in a cached form. Due to the application of OSWRR technology to path guidance, the light samples from the surface cache reservoir 306 are shared among the first, second, and third viewers 302 for direct light estimation. Furthermore, the OSWRR technology enables the rendering controller 102 to use the surface cache reservoir 306 to terminate recursive light paths traversing the 3D scene when calculating indirect lighting. In the case of global (i.e., indirect) illumination, the rendering controller 102 is used to terminate the paths of virtual rays from the first viewer 302, the second viewer 304, and the third viewer 602 at the light source 402, so that each path can carry as much information about the object as possible. By combining path guidance technology with OSWRR technology, the rendering controller 102 is used to implement a dynamic structure for encoding light sampling data, which can be used to calculate direct lighting shading and indirect ray paths at the light source terminating the sampling for indirect lighting.

[0054] Figure 7 An exemplary diagram is shown illustrating an exemplary scenario of applying on-surface weighted reservoir resampling (OSWRR) to multiple ejection global illumination calculations according to an embodiment of the present invention. Combined with... Figures 1 to 6 Component description Figure 7 . refer to Figure 7 Figure 700 illustrates an exemplary scenario in which on-surface-weighted reservoir resampling (OSWRR) is applied to multiple ejection global illuminations.

[0055] In one implementation, multiple ejection global illumination refers to ( Figure 5 The secondary light source 504 indirectly illuminates one or more objects 110, wherein light from the secondary light source 504 bounces several times from surfaces outside the object's surface. During the estimation of multiple bounces for global illumination, the bounced light from multiple surfaces is recursively tracked. When a ray intersects a surface, it is reflected or refracted, generating new rays in the direction of reflection or refraction. Reflection or refraction is repeated until the maximum number of bounces is reached, or the light intensity of the secondary light source 504 drops below a certain threshold. In one implementation, due to ( Figure 5 The secondary light source 504 is generated by the illumination of the primary light source 502. In this case, the secondary light source 504 is generated by the rendering controller 102. Figure 3 The surface cache reservoir 306 generates multiple bounces for global illumination. Storing lighting samples corresponding to global illumination in the surface cache reservoir 306 facilitates the reconstruction of complex and extended optical paths by connecting the surface cache reservoirs 306 to each other. In a first case, the rendering controller 102 is used to reconnect to other resampled samples (e.g., those that are meaningful) to generate multiple bounce global illuminations. In the first case, the rendering controller 102 is used to reconstruct the optical path from the surface cache reservoir 306 for calculating multiple bounce global illumination estimates, which allows the rendering controller 102 to reuse occluded or out-of-frame lighting samples for path construction in one of the first and second viewer 304. In one example, the lighting samples used in the first viewer 302 are occluded for the second viewer 304 and can be reused to calculate the global illumination estimate. In the second case, when calculating a single bounce global illumination at a location on an object, the OSWRR technique configured in the rendering controller 102 increments the sample count. In one implementation, the rendering controller 102 is used to average the illumination samples by using a reservoir 306 of surface buffers in the case of multiple single-shot global illuminations.

[0056] Modifications to the embodiments of the invention described above may be made without departing from the scope of the invention as defined by the appended claims. Expressions such as “comprising,” “integrating,” “having,” and “is / are” used to describe and claim the invention are intended to be interpreted in a non-exclusive manner, supporting the presence of items, components, or elements not explicitly described. Singular references should also be interpreted to refer to the plural. The word “exemplary” as used herein means “as an example, instance, or illustration.” Any embodiment described as “exemplary” is not necessarily to be construed as more preferred or advantageous than other embodiments, nor does it necessarily exclude combinations of features from other embodiments. The word “optionally” as used herein means “provided in some embodiments and not in others.” It should be understood that certain features of the invention described in the context of a single embodiment for clarity may also be provided in a single embodiment by combining them. Conversely, various features of the invention described in the context of a single embodiment for brevity may also be provided individually, in any suitable combination, or suitably for any other described embodiment of the invention.

Claims

1. A rendering controller (102), characterized in that, The rendering controller (102) is used to render light in a 3D scene (108) comprising one or more objects (110), wherein each object is associated with multiple surface reservoirs (306), and wherein the rendering controller (102) is also used to provide light transport calculations in texture space using RIS or WRS based on the surface cache.

2. The rendering controller (102) according to claim 1, characterized in that, The surface buffer storage layer (306) can utilize sampling information independently of the direction of light from the observer, the direction of light looking towards or away from the surface position of the surface buffer storage layer (306).

3. The rendering controller (102) according to claim 1 or 2, characterized in that, The rendering controller (102) is also used to guide light sampling for direct illumination estimation and indirect illumination estimation based on the surface buffer layer (306).

4. The rendering controller (102) according to any one of the preceding claims, characterized in that, The rendering controller (102) is also used to reuse data from a single surface space reservoir for calculating lighting effects from multiple viewpoints.

5. The rendering controller (102) according to any one of the preceding claims, characterized in that, The rendering controller (102) is also used to access adjacent reservoirs on the surface of an object so that the surface cache layer (306) can be resampled to efficiently share candidate samples on multiple objects.

6. The rendering controller (102) according to any one of the preceding claims, characterized in that, The rendering controller (102) is also used for: Receive the visible texels and corresponding cache resolution of all viewers; For each visible texel, update the corresponding surface buffer layer: Initial sampling: Generate new random light samples Temporal resampling: Update the temporary reservoir of the current texel using the new light sample. Spatial resampling: Selecting randomly adjacent temporary reservoirs Update the current spatial reservoir using the selected random adjacent temporary reservoir; For each viewer, the spatial reservoir texture is sampled at the correct cache resolution, and the reservoir content is used for direct lighting shading.

7. The rendering controller (102) according to any one of the preceding claims, characterized in that, The rendering controller (102) is also configured to, during the initial sampling period: Receive the initial sample count N; Receive texel information and associated geometric data; Create the surface buffer storage layer; Generate N initial samples and update the cache layer; Assess the visibility of the resulting samples in the cache layer; The weights of the resulting samples are calculated accordingly.

8. The rendering controller (102) according to any one of the preceding claims, characterized in that, The rendering controller (102) is also configured to, during temporal resampling: Read the temporary storage cache texture; The temporary cache texture is sampled at the same texel position to obtain the cache reservoir from the last frame; The last frame cache layer and the newly generated cache layer are combined in a new output cache layer; Recalculate the weights of the obtained samples in the output cache layer; Write the texture to the temporary storage cache.

9. The rendering controller (102) according to any one of the preceding claims, characterized in that, The rendering controller (102) is also configured to, during spatial resampling: Receive the neighborhood radius Rn and the neighbor count Nn; Read the temporary storage cache texture; The temporary cache texture is sampled to obtain the temporary storage layer of the newly generated cache; For Nn neighboring samples, perform the following operation: Generate random texel offsets in the Rn texel neighborhood; Receive temporary storage cache textures; Sample the temporary reservoir cache texture of randomly adjacent texels; Update the current vertex temporary reservoir with the adjacent reservoir; Evaluate the visibility of the obtained samples in the current cache reservoir and calculate the weight of the obtained samples accordingly; Write the spatial storage cache texture.

10. The rendering controller (102) according to any one of the preceding claims, characterized in that, The rendering controller (102) is also configured to, during reservoir coloring, For each viewer: For each visible texel: Receive spatial storage cache texture; Receive the buffered spatial reservoir to obtain associated optical sample data; The emitted radiation is calculated by inserting the data into the rendering equation; The weight is determined based on the weight of the cache storage layer and the output pixel color value.

11. The rendering controller (102) according to any one of the preceding claims, characterized in that, The surface cache layer is located on the surface of 3D objects in the surface cache texture.

12. The rendering controller (102) according to any one of the preceding claims, characterized in that, The surface buffer layer includes a description of optical sample points, RIS weights, the number of seen samples, and the sum of their weights used for normalization.

13. A method (200) for a rendering controller (102), characterized in that, The rendering controller (102) is used to render light in a 3D scene (108) comprising one or more objects (110), wherein each object is associated with a plurality of surface reservoirs (306), wherein the method (200) includes providing light transport calculations in texture space based on surface caches using RIS or WRS.

14. A computer program product, characterized in that, Includes program instructions for performing the method (200) according to claim 13 when executed by one or more processors in a rendering controller system.

Images