Apparatus, method, program, and system for real-time volume rendering of dynamic particles

Abstract

A method is provided for real-time volumetric rendering of dynamic particles for a processing circuit. The method includes converting particle data representing each of the dynamic particles into a density volume representing a density distribution of the respective dynamic particles distributed in 3D space, pre-calculating a light distribution in the density volume representing a light value for each grid point in the density volume using ray marching from a light source, rendering the dynamic particles in real-time by ray marching toward a viewpoint position and calculating pixel color values ​​determined using the density volume and the light distribution, and outputting a representation of the dynamic particles based on the rendering. The method also includes generating particle data representing simulated particles composed of a simulated substance by a physically based simulation of a natural phenomenon, the generated particle data may include movement of the simulated particles in the 3D space.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Patent Application No. 18 / 070,326, filed November 28, 2022, the entire contents of which are incorporated herein by reference.

[0002] The present disclosure generally relates to processing systems that include one or more techniques for graphics processing. [Background technology]

[0003] Computing devices often utilize a graphics processing unit (GPU) or central processing unit (CPU) to render graphical data for display. Such computing devices may include, for example, computer workstations, mobile phones such as smartphones, embedded systems, personal computers, tablet computers, and video game consoles. A GPU processes instructions and / or data in a graphics processing pipeline that includes one or more processing stages that work together to execute graphics processing commands and output a frame. A CPU may control the operation of the GPU by issuing one or more graphics processing commands to the GPU. Modern CPUs can typically run multiple applications simultaneously, each of which may need to utilize a GPU during execution. Devices that provide content for visual presentation on a display generally include a GPU.

[0004] Typically, the GPU of a device is configured to execute processes in a graphics processing pipeline. However, as the complexity of the content to be rendered and the physical constraints of GPU memory increase, the need to improve computational or graphics processing increases. Summary of the Invention [Problem to be solved by the invention]

[0005] The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is not intended to identify key elements of all aspects or to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later. [Means for solving the problem]

[0006] The present disclosure relates to methods and apparatus for graphics processing. One aspect of the subject matter described in this disclosure is embodied in a method for real-time volumetric rendering of dynamic particles. The method includes converting particle data representing each of the dynamic particles into a density volume representing a density distribution of the dynamic particles distributed in a three-dimensional (3D) space. The method also includes pre-calculating a light distribution in the density volume representing light values ​​for each grid point in the density volume using ray marching from a light source. The grid points in the density volume correspond to fixed reference positions. The method further includes rendering the dynamic particles in real-time by calculating pixel color values ​​determined using (i) the ray marching toward a viewpoint position, (ii) the density volume, and (iii) the light distribution. The method includes outputting a representation of the dynamic particles based on the rendering.

[0007] Another further aspect of the subject matter described in this disclosure may be embodied in an apparatus for real-time volume rendering of dynamic particles. The apparatus includes a processing circuit configured to convert particle data representing each of the dynamic particles into a density volume representing a density distribution of the dynamic particles distributed in a three-dimensional (3D) space. The processing circuit is configured to pre-calculate a light distribution in the density volume representing a light value for each grid point in the density volume using ray marching from a light source. The grid points in the density volume correspond to fixed reference positions. The processing circuit is configured to render the dynamic particles in real-time by calculating pixel color values ​​determined using (i) the ray marching toward a viewpoint position, (ii) the density volume, and (iii) the light distribution, and generate an output of a representation of the dynamic particles based on the rendering.

[0008] Yet another further aspect of the subject matter described in this disclosure may be embodied in a system for real-time volume rendering of dynamic particles. The system includes a first processing circuit, a second processing circuit, and a control circuit. The control circuit is configured to control the first processing circuit to convert particle data representing each of the dynamic particles of a first frame into a density volume representing a density of the dynamic particles distributed in a three-dimensional (3D) space. The control circuit is also configured to control the first processing circuit to pre-calculate a light distribution in the density volume representing a light value for each grid point in the density volume in the first frame using ray marching toward a light source. The grid points in the density volume correspond to fixed reference positions. The control circuit is also configured to copy the density volume and light distribution of the first frame from the first processing circuit to the second processing circuit after completion of the pre-calculation in the first processing circuit for the first frame. The control circuit is further configured to control the first processing circuit to convert the particle data of the second frame into the density volume. The control circuitry is further configured to control the second processing circuitry to render in real time the dynamic particles in the first frame by calculating pixel color values ​​determined using (i) ray marching toward the viewpoint position, (ii) the density volume, and (iii) the light distribution copied from the first processing circuitry. The control circuitry is also configured to cause an output of a representation of the dynamic particles in the first frame based on the rendering.

[0009] Another further aspect of the subject matter described in the present disclosure may be embodied in a non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the processor to convert particle data representing each of the dynamic particles into a density volume representing a density distribution of the dynamic particles distributed in a three-dimensional (3D) space. The processor is also configured to pre-calculate a light distribution in the density volume representing light values ​​for each grid point in the density volume using ray marching from a light source. The processor is further configured to render the dynamic particles in real time by calculating pixel color values ​​determined using (i) the ray marching toward the viewpoint position, (ii) the density volume, and (iii) the light distribution. The processor is further configured to generate an output of a representation of the dynamic particles based on the rendering.

[0010] To the accomplishment of the foregoing and related ends, the one or more aspects comprise the features hereinafter fully described and particularly pointed out in the claims. The following description and the annexed drawings set forth in detail certain illustrative features of the one or more aspects. These features are indicative, however, of only a few of the various ways in which the principles of the various aspects may be employed and the description is intended to include all such aspects and their equivalents.

[0011] Details of one or more aspects of the subject matter described in this disclosure are set forth in the accompanying drawings and the following description. However, the accompanying drawings only illustrate some typical aspects of the disclosure and therefore should not be considered as limiting its scope. Other features, aspects, and advantages will be apparent from the description, drawings, and claims. [Brief description of the drawings]

[0012] [Figure 1A] FIG. 1 is a block diagram illustrating an example of a content generation system in accordance with one or more techniques of this disclosure. [Figure 1B]FIG. 1 is a block diagram illustrating an example of a content generation system in accordance with one or more techniques of this disclosure. [Diagram 2] 1 is an example diagram for a graphics processing pipeline in accordance with one or more techniques of this disclosure. [Figure 3A] 1 is an example diagram representing density and lighting distribution as axis-aligned bounding boxes in accordance with one or more techniques of this disclosure. [Figure 3B] 1 is an example diagram illustrating density and illumination distribution as a froxel volume in accordance with one or more techniques of the present disclosure. [Figure 4] FIG. 1 illustrates an example of per-slice light pre-computation in accordance with one or more techniques of this disclosure. [Diagram 5] 1 is a flowchart of an example method for graphics processing in accordance with one or more techniques of this disclosure. [Figure 6] 1 is a flowchart of an example method for graphics processing utilizing dual processing circuits in accordance with one or more techniques of the present disclosure. [Figure 7A] 1 is an example diagram for a graphics processing pipeline using a single processing circuit in accordance with one or more techniques of this disclosure. [Figure 7B] 1 is an example diagram for a graphics processing pipeline using dual processing circuits in accordance with one or more techniques of this disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Like reference numbers and designations in the various drawings indicate like elements.

[0014] The following description is directed to several exemplary embodiments for purposes of illustrating the innovative aspects of the present disclosure, however, those skilled in the art will readily recognize that the teachings herein can be applied in many different ways.

[0015] Related systems implement volume rendering methods in offline applications, where dense natural materials such as snow are typically rendered in volumetric representations. First, dense natural material particles are converted into a density volume and then rendered with various path tracing algorithms such as delta tracking and residual tracking. Although these methods can produce realistic and physically accurate results, they require many samples to be taken to reduce noise, thus consuming a significant amount of computation time. Therefore, these implementations are not suitable for real-time applications of rendering dynamic dense natural materials.

[0016] Related systems have also implemented snow rendering and volume rendering in real-time applications. There are several methods for rendering snow in real-time applications such as video games. For static scenes (e.g., snowy landscapes), one can utilize polygon meshes with subsurface scattering material that can be rendered quickly. However, this approach cannot address dynamic particles due to the difficulty of capturing the complex geometric shapes of the particles using polygons.

[0017] Volume rendering is also used in game engines such as the Unreal Engine. These real-time volume rendering techniques include volumetric clouds and volumetric fog (e.g., natural materials that are low-density media) that can be rendered efficiently with large ray marching steps. In contrast, semi-transparent materials such as snow, ash, or dust have high-density media that require smaller ray marching steps that increase computational cost. Thus, there is a need to optimize the ability to render dense materials with ray marching in real-time applications.

[0018] Aspects of the present disclosure can create more realistic simulations and real-time renderings of natural materials by implementing volume rendering of dynamic elements such as natural translucent (e.g., snow, dust, or ash) materials generated by physically based simulation. For example, aspects of the present disclosure provide a GPU pipeline that volume renders simulated natural particles at high density in real-time applications. By doing so, when aspects of the present disclosure simulate these natural particles, the data particles are converted into a volume texture and then ray marching is used to calculate the interaction between light and the medium. Thus, aspects of the present disclosure can create more accurate shading and more physically accurate rendering of dense natural particles than related rendering techniques. Additionally, some aspects of the present disclosure implement several graphics processing optimizations to improve performance by reducing computational costs and increasing frame rates.

[0019] Various aspects of the system, device, computer program product, and method are described more fully below with reference to the accompanying drawings. However, the present disclosure may be embodied in many different forms and should not be construed as limited to the specific structure or function presented throughout the present disclosure. Rather, these aspects are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. Based on the teachings herein, one skilled in the art should understand that the scope of the present disclosure is intended to cover any aspect of the system, device, computer program product, and method disclosed herein, whether implemented independently of other aspects of the present disclosure or combined with other aspects of the present disclosure. For example, an apparatus may be implemented or a method may be performed using any number of the aspects described herein. Furthermore, the scope of the present disclosure is intended to cover such an apparatus or method implemented with other structure, function, or structure and functionality in addition to or other than the various aspects of the present disclosure described herein. Any aspect disclosed herein may be embodied by one or more elements of a claim.

[0020] Although various aspects are described herein, many variations and permutations of these aspects are within the scope of the disclosure. Although some potential benefits and advantages of the aspects of the disclosure are mentioned, the scope of the disclosure is not intended to be limited to any particular benefit, use, or purpose. Rather, the aspects of the disclosure are intended to be broadly applicable to different wireless technologies, system configurations, networks, and transmission protocols, some of which are shown as examples in the drawings and the following description. The detailed description and drawings are merely illustrative of the disclosure rather than limiting, and the scope of the disclosure is defined by the appended claims and their equivalents.

[0021] Several aspects are presented with reference to various apparatus and methods that are described in the following detailed description and illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, etc. (collectively referred to as "elements"). These elements may be implemented using electronic hardware, computer software, or any combination thereof. Whether such elements are implemented as hardware or software depends on the particular application and design constraints imposed on the overall system.

[0022] As an example, an element, or any portion of an element, or any combination of elements, may be implemented as a "processing system" including one or more processors (which may also be referred to as processing circuits). One or more processors in the processing system may execute software. Software may be broadly construed to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software components, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, and the like, whether referred to as software, firmware, middleware, microcode, hardware description language, and the like. The term application may refer to software. As described herein, one or more techniques may refer to an application, i.e., software, configured to perform one or more functions. In such an example, the application may be stored in a memory, e.g., a processor's on-chip memory, a system memory, or any other memory. Hardware described herein, such as a processor, may be configured to execute the application. For example, an application may be described as including code that, when executed by the hardware, causes the hardware to execute one or more techniques described herein. As an example, the hardware may access code from a memory and execute the code accessed from the memory to execute one or more techniques described herein. In some examples, components are identified in this disclosure. In such examples, the components may be hardware, software, or a combination thereof. The components may be separate components or subcomponents of a single component.

[0023] Thus, in one or more examples described herein, the functions described may be implemented in hardware, software, or any combination thereof. If implemented in software, the functions may be stored or encoded as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer storage media. A storage medium may be any available medium that can be accessed by a computer. By way of example and not limitation, such computer-readable media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), optical disk storage, magnetic disk storage, other magnetic storage, combinations of the aforementioned types of computer-readable media, or any other medium that can be used to store computer-executable code in the form of instructions or data structures that can be accessed by a computer.

[0024] The present disclosure includes techniques for real-time volume rendering of elements such as dense materials generated by physics-based simulation. Aspects of the present disclosure enable realistic simulation of dynamic dense natural materials in real-time applications such as video games that include interactions with natural environments. Additionally, the disclosed real-time volume rendering techniques improve rendering of graphical content and / or reduce the load on any processing unit configured to execute one or more techniques described herein, such as a processing unit, i.e., a GPU (or processing circuit). Other exemplary advantages are described throughout the present disclosure.

[0025] As used herein, instances of the term "content" may refer to "graphical content," "images," and vice versa. This is true regardless of whether the terms are used as adjectives, nouns, or other parts of speech. In some examples, as used herein, the term "graphical content" may refer to content generated by one or more processes of a graphics processing pipeline. In some examples, as used herein, the term "graphical content" may refer to content generated by a processing unit configured to perform graphics processing. In some examples, as used herein, the term "graphical content" may refer to content generated by a graphics processing unit.

[0026] As used herein, the term "display content" may refer to content generated by a processing unit configured to perform display processing. In some examples, as used herein, the term "display content" may refer to content generated by a display processing unit. Graphical content may be processed to become display content. For example, a graphics processing unit may output graphical content, such as a frame, to a buffer (which may be referred to as a frame buffer). The display processing unit may read graphical content, such as one or more frames, from the buffer and perform one or more display processing techniques to generate the display content. For example, the display processing unit may be configured to perform compositing on one or more rendered layers to generate a frame. As another example, the display processing unit may be configured to composite, blend, or combine two or more layers together into a single frame. The display processing unit may be configured to perform scaling, such as upscaling or downscaling, on the frame. In some examples, a frame may refer to a layer. In other examples, a frame may refer to two or more layers that have already been blended together to form a frame, i.e., a frame includes two or more layers, and a frame including two or more layers may be subsequently blended.

[0027] In general, a GPU (or processing circuit) may be used to render a three-dimensional (3D) scene. Because such rendering of a 3D scene may be memory bandwidth intensive, a dedicated graphics memory ("GMEM") may be used. The GMEM may be located close to the graphics processing cores of the GPU so that it has high memory bandwidth (i.e., read and write access to the GMEM is fast). The scene may be rendered into the GMEM by the graphics processing cores of the GPU, and the scene may be resolved from the GMEM to a memory (e.g., a frame buffer) so that the scene can be displayed on a display device. Rendering an entire frame may be referred to as immediate mode rendering. However, the size of the GMEM is limited due to physical memory constraints, and as a result, the GMEM may not have sufficient memory capacity to encompass an entire three-dimensional scene (e.g., an entire frame).

[0028] Realistically simulating and rendering dense natural translucent materials (e.g., snow, ash, or dust) is challenging in real-time applications such as video games that involve interaction with natural environments, or other real-time applications that involve natural environments such as VR tourism or cultural sphere digitization due to the high density of natural materials. Accurate physically based algorithms are essential for realistic results, but have been too computationally prohibitively expensive to be applied in real-time applications. However, with the rapid development of GPU computing power, it has only recently become feasible to physically simulate and render these effects in real-time.

[0029] In the past decades, path tracing and its variants have become a standard method for offline rendering. Path tracing includes computer graphics algorithms for rendering 3D scenes that faithfully generate global illumination. The algorithm stochastically samples many paths of light transport, then calculates and accumulates the contributions of all paths to obtain a result that is rendered onto a screen pixel. Specifically, path tracing samples the path of light in a 3D scene. After light is emitted from a light source, the light ray travels in a straight line until it scatters on the surface of an object or in a medium, which causes the light direction to change and continues in the new direction until it hits another scattering point. Path tracing algorithms simulate such behavior of light and generate multiple light paths in a 3D scene to estimate the overall illumination in the scene.

[0030] Unlike surface rendering, where light continues to travel in a straight line until it hits a surface, in volume rendering, light can be scattered anywhere inside the medium volume, so path tracing can be used to render volumetric materials such as snow. To simulate this, path tracing randomly generates a scattering-free path length as the light travels through the medium, and scatters the light at the ends of the scattering-free length. There are also various methods for sampling the scattering-free path length, such as delta tracking and ratio tracking.

[0031] Path tracing can require a huge number of light path samples to evaluate the light value displayed at each pixel. Therefore, if the number of samples is small, the rendering result is noisy. Therefore, hundreds of samples are required for each pixel to obtain a noise-free result. However, this means that rendering a clean frame via path tracing can take minutes or even hours depending on the complexity of the scene, which is very expensive for real-time applications.

[0032] To render a static snow scene, a mesh with subsurface scattering material may be used. The mesh uses a collection of small triangles to represent the snow surface. Subsurface scattering is a modeling technique used to render semi-transparent materials. For example, subsurface scattering involves modeling light penetrating the material, scattering underneath it, and then bouncing off the surface. For real-time rendering, the GPU has a rasterization pipeline that can render triangles quickly, unlike path tracing in offline rendering. Thus, while a triangular mesh may be suitable for representing the surface of a semi-transparent material, it cannot efficiently capture the complex geometry of a semi-transparent material represented by dynamic particles (e.g., falling snow particles), because this requires a large number of triangles that are computationally very expensive.

[0033] To render dynamic translucent particles, sprites may be used. This technique is simple but cannot produce realistic results. A sprite may refer to a 2D bitmap that is drawn on the screen. When rendering translucent particles, a camera-facing sprite bitmap is placed and drawn at each particle's location. This technique is suitable for rendering moving particles, but 2D bitmaps cannot reproduce well the lighting and shadowing effects present in translucent particles, resulting in unrealistic repeating patterns.

[0034] As described herein, exemplary implementations render dynamic snow particles using volume rendering, which includes methods for rendering light that interacts with media within a volume, where it may be scattered and absorbed by materials before reaching the camera.

[0035] Volumetric rendering techniques may be used to render materials other than snow. The most common are volumetric clouds and volumetric fog. For example, volumetric clouds and / or fog may be rendered using ray marching techniques. Ray marching is a method of performing volumetric rendering. The ray marching technique simulates a ray passing through a volumetric medium. Samples are taken at regular intervals along the ray to calculate the interaction between the light and the medium, and the calculations are accumulated along the ray to obtain a result. However, ray marching techniques have only been applied to low-density media such as clouds and fog in game engines because low-density media are visually blurrier than high-density materials. This means that the density variations of the medium are smaller, thereby allowing larger steps to be used in ray marching to render them. Furthermore, while density volumes of fog and clouds can be easily generated using procedural materials, game engines have not traditionally supported volume generation from particles. This means that current plugins are specific to rendering low-density media, i.e., clouds and fog, and cannot be directly applied to render high-density dynamic particles.

[0036] Accordingly, embodiments of the present disclosure include a method for real-time volume rendering of dynamic translucent particles and an apparatus for reducing the computational overhead of rendering dynamic snow particles while ensuring that the simulated snow appears realistic. It should be noted that while snow particles are used as an example, aspects of the present disclosure may be applied to the simulation of dynamic particles of other dense translucent substances such as ash, dust, etc. A graphics processing pipeline may begin by converting the dynamic particles into a volume texture and then use ray marching to calculate the interaction between light and the medium. The subject matter described herein may be implemented to realize one or more benefits or advantages. For example, embodiments enable realistic volume rendering of dynamic snow particles in real-time applications such as video games. Additionally, embodiments may generate more realistic shading and render better volumes of dense natural particles as compared to related techniques.

[0037] 1A is a block diagram illustrating an example content generation system 100 configured to implement one or more techniques of the present disclosure. The content generation system 100 includes a processing unit 127, a GPU 120, and a system memory 124 configured to render a 3D scene according to an example embodiment. The processing unit 127 may execute software applications 111, an operating system (OS) 113, and a graphics driver 115. In addition, the system memory 124 may include an indirect buffer that stores command streams for rendering primitives, as well as secondary commands to be executed by the GPU 120. The GPU 120 may include a graphics memory (GMEM) 121, which may be "on-chip" with the GPU 120, coupled to a density volume processor 123 and a light distribution processor 125. As will be described in more detail in connection with FIG. 1B, the components of content generation system 100 may be part of devices including, but not limited to, video devices, media players, set-top boxes, wireless handsets such as mobile phones and so-called smart phones, personal digital assistants (PDAs), desktop computers, laptop computers, gaming consoles, video conferencing units, tablet computing devices, and the like.

[0038] Processing unit 127 may include one or more of a central processing unit (CPU). GPU 120 may include processing units configured to perform graphics-related functions, such as generating and outputting graphics data for presentation on a display, as well as non-graphics-related functions that utilize the massive processing parallelism provided by GPU 120. Because GPU 120 may provide general-purpose processing capabilities in addition to graphics processing capabilities, GPU 120 may be referred to as a general-purpose GPU (GP-GPU). Examples of processing unit 127 and GPU 120 include, but are not limited to, digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. In some examples, GPU 120 may be a microprocessor designed for a specific application, such as providing massive parallel processing for processing graphics as well as for running non-graphics related applications. Additionally, although processing unit 127 and GPU 120 are shown as separate components, aspects of the disclosure are not so limited and may, for example, be implemented on a common integrated circuit (IC).

[0039] Software applications 111 executing on processing unit 127 may include one or more graphics rendering instructions that instruct processing unit 127 to render graphics data to a display (not shown in FIG. 1A). In some examples, the graphics rendering instructions may include software instructions that may conform to a graphics application programming interface (API). To process the graphics rendering instructions, processing unit 127 may issue one or more graphics rendering commands to GPU 120 (e.g., via graphics driver 115) to cause GPU 120 to perform some or all of the rendering of the graphics data. In some examples, the graphics data to be rendered may include a list of graphics primitives, such as, for example, points, lines, triangles, quadrilaterals, triangle strips, etc.

[0040] GPU 120 may be configured to perform graphics processing for rendering one or more graphics primitives to a display. Thus, when one of the software applications executing on processing unit 127 requires graphics processing, processing unit 127 may provide graphics commands and graphics data to GPU 120 for rendering to a display. The graphics data may include, for example, drawing commands, state information, primitive information, texture information, etc. GPU 120 may be built with a highly parallel structure that, in some cases, provides more efficient processing of complex graphics-related operations than processing unit 127. For example, GPU 120 may include multiple processing elements configured to operate in parallel on multiple vertices or pixels.

[0041] GPU 120 may be directly coupled to GMEM 121. In other words, GPU 120 may process data locally using local storage instead of off-chip memory. This allows GPU 120 to operate in a more efficient manner by eliminating the need for GPU 120 to read and write data via, for example, a shared bus, which may experience a large amount of bus traffic. GMEM 121 may include one or more volatile or non-volatile memory or storage devices, such as, for example, a random access memory (RAM), a static RAM (SRAM), a dynamic RAM (DRAM), and one or more registers.

[0042] The GMEM 121 may also be directly coupled to at least the density volume processor 123 and a second processor (e.g., light distribution processor 125). The density volume processor 123 may be configured to convert the particle data representing each of the dynamic particles into a density volume representing the density distribution of the respective dynamic particles distributed in 3D space. The light distribution processor 125 may be configured to pre-calculate the light distribution in the density volume representing the light values ​​for each grid point in the density volume using ray marching from the light source. In some aspects, the grid points in the density volume may correspond to fixed reference positions. In some aspects, the processor performing the functions described above may be a general-purpose processor (e.g., a CPU).

[0043] The dense volume processor 123 may be a CPU, a GPU, a general purpose GPU (GPGPU), or any other processing device that may be configured to perform graphics processing. In some examples, the dense volume processor 123 may be integrated into the motherboard of the device 104. In some examples, the dense volume processor 123 may be on a graphics card installed in a port on the motherboard of the device 104, or may be embedded within a peripheral device configured to interoperate with the device 104. The dense volume processor 123 may include one or more processors, such as one or more microprocessors, GPUs, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), discrete logic, software, hardware, firmware, other equivalent integrated circuits or discrete logic circuits, or any combination thereof. If the techniques are implemented partially in software, the density volume processor 123 may store instructions for the software in a suitable non-transitory computer-readable storage medium, such as the internal memory 121, and execute the instructions in hardware using one or more processors to perform the techniques of the present disclosure. Any of the above, including hardware, software, combinations of hardware and software, etc., may be considered to be one or more processors.

[0044] The light distribution processor 125 may be a CPU, a GPU, a general-purpose GPU, or any other processing device that may be configured to perform graphics processing. In some examples, the light distribution processor 125 may be integrated into the motherboard of the device 104. In some examples, the light distribution processor 125 may be on a graphics card installed in a port on the motherboard of the device 104, or may be embedded within a peripheral device configured to interoperate with the device 104. The light distribution processor 125 may include one or more processors, such as one or more microprocessors, GPUs, ASICs, FPGAs, ALUs, DSPs, discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the techniques are implemented in part in software, the light distribution processor 125 may store instructions for the software in a suitable non-transitory computer-readable storage medium, such as the internal memory 121, and may execute the instructions in hardware using one or more processors to perform the techniques of the present disclosure. Any of the above, including hardware, software, a combination of hardware and software, etc., may be considered to be one or more processors.

[0045] Processing unit 127 and / or GPU 120 may store the rendered image data in frame buffer 128, which may be a separate memory or may be allocated within system memory 124. A display processor may retrieve the rendered image data from frame buffer 128 and display the rendered image data on a display.

[0046] System memory 124 may be memory within the device or may be external to processing unit 127 and GPU 120, i.e., off-chip to processing unit 127 and off-chip to GPU 120. System memory 124 may store applications executed by processing unit 127 and GPU 120. Additionally, system memory 124 may store data on which the executed applications operate, as well as data resulting from the applications.

[0047] System memory 124 may store program modules, instructions, or both accessible for execution by processing unit 127, data used by programs executing on processing unit 127, or a combination of these. For example, system memory 124 may store a window manager application used by processing unit 127 to present a graphical user interface (GUI) on a display. Additionally, system memory 124 may store user applications and application surface data associated with the applications. As described in more detail below, system memory 124 may act as device memory for GPU 120 and may store data to be operated on by GPU 120 as well as data resulting from operations performed by GPU 120. For example, system memory 124 may store any combination of texture buffers, depth buffers, stencil buffers, vertex buffers, frame buffers, and the like.

[0048] Examples of system memory 124 include, but are not limited to, random access memory (RAM), read only memory (ROM), or electrically erasable programmable read only memory (EEPROM), or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer or processor. As an example, system memory 124 may be removed from the device and moved to another device. As another example, a storage device substantially similar to system memory 124 may be inserted into the device.

[0049] FIG 1B is a more detailed block diagram illustrating an example content generation system 100 configured to implement one or more techniques of the present disclosure. It should be noted that the content generation system 100 illustrated in FIG 1B may correspond to the content generation system of FIG 1A. In this regard, the content generation system 100 of FIG 1B includes a processing unit 127, a GPU 120, and a system memory 124.

[0050] As further illustrated, the content generation system 100 includes a device 104 that may include one or more components configured to perform one or more techniques of the present disclosure. In the illustrated example, the device 104 may include a GPU 120, a content encoder / decoder 122, and a system memory 124. In some aspects, the device 104 may include several additional components, such as a communication interface 126, a transceiver 132, a receiver 133, and a transmitter 130, as well as one or more displays 131. References to the display 131 may refer to one or more displays 131. For example, the display 131 may include a single display or multiple displays. The display 131 may include a first display and a second display. In a further example, the results of the graphics processing may not be displayed on the device, for example, the display 131 may not receive frames for presentation. Also, the frames and / or graphics processing results may be forwarded to another device. In some aspects, this may be referred to as hybrid rendering.

[0051] The GPU 120 includes a GMEM 121. The GPU 120 may be configured to perform graphics processing, such as in a graphics processing pipeline 107. The graphics processing pipeline 107 may include at least generating dynamic particles, converting the dynamic particles to a density volume, pre-calculating a light volume, and then rendering the volume using ray marching, as described in more detail in FIG. 2 and FIG. 6. The GPU 120 may be configured to perform these steps within the graphics processing pipeline 107 using at least the GMEM 121, a density volume processor 123 coupled to the GMEM 121, and a second processor (e.g., a light distribution processor 125) coupled to the GMEM 121. The content encoder / decoder 122 may include an internal memory 129. In some examples, the device 104 may include a display processor, such as a processing unit 127, to perform one or more display processing techniques on one or more frames generated by the GPU 120 prior to presentation by one or more displays 131, as described above. The processing unit 127 may be configured to perform display processing. The one or more displays 131 may be configured to display or present frames processed by processing unit 127. In some examples, the one or more displays 131 may include one or more of a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, a projection display device, an augmented reality display device, a virtual reality display device, a head-mounted display, or any other type of display device.

[0052] Memory external to the GPU 120 and the content encoder / decoder 122, such as the system memory 124 as described above, may be accessible to the GPU 120 and the content encoder / decoder 122. For example, the GPU 120 and the content encoder / decoder 122 may be configured to read from and / or write to an external memory, such as the system memory 124. The GPU 120 and the content encoder / decoder 122 may be communicatively coupled to the system memory 124 via a bus. In some examples, the GPU 120 and the content encoder / decoder 122 may be communicatively coupled to each other via a bus or a different connection.

[0053] The content encoder / decoder 122 may be configured to receive graphical content from any source, such as the system memory 124 and / or the communication interface 126. The system memory 124 may be configured to store the received encoded or decoded graphical content. The content encoder / decoder 122 may be configured to receive encoded or decoded graphical content in the form of encoded pixel data, for example, from the system memory 124 and / or the communication interface 126. The content encoder / decoder 122 may be configured to encode or decode any graphical content.

[0054] According to some examples, GMEM 121 or system memory 124 may be a non-transitory computer-readable storage medium. The term "non-transitory" may indicate that the storage medium is not embodied in a carrier wave or propagating signal. However, the term "non-transitory" should not be interpreted to mean that GMEM 121 or system memory 124 is non-movable or that its contents are static. As an example, system memory 124 may be removed from device 104 and moved to another device. As another example, system memory 124 may not be removable from device 104.

[0055] The GPU (or processing circuitry) may be configured to perform graphics processing according to the exemplary techniques as described herein. In some examples, the GPU 120 may be integrated into the motherboard of the device 104. In some examples, the GPU 120 may be present on a graphics card installed in a port on the motherboard of the device 104 or may be embedded within a peripheral device configured to interoperate with the device 104. The GPU 120 may include one or more processors, such as one or more microprocessors, GPUs, ASICs, FPGAs, ALUs, DSPs, discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the techniques are implemented in part in software, the GPU 120 may store instructions for the software in a suitable non-transitory computer-readable storage medium and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Any of the above, including hardware, software, combinations of hardware and software, etc., may be considered to be one or more processors.

[0056] The content encoder / decoder 122 may be any processing unit configured to perform content encoding / decoding. In some examples, the content encoder / decoder 122 may be integrated into the motherboard of the device 104. The content encoder / decoder 122 may include one or more processors, such as one or more microprocessors, ASICs, FPGAs, ALUs, DSPs, video processors, discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuitry, or any combination thereof. If the techniques are implemented in part in software, the content encoder / decoder 122 may store instructions for the software in a suitable non-transitory computer-readable storage medium, such as the internal memory 129, and may execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Any of the above, including hardware, software, a combination of hardware and software, etc., may be considered to be one or more processors.

[0057] In some aspects, the content generation system 100 may include a communication interface 126. The communication interface 126 may include a receiver 133 and a transmitter 130. The receiver 133 may be configured to perform any receiving function described herein with respect to the device 104. Additionally, the receiver 133 may be configured to receive information from the other device, such as, for example, eye or head position information, rendering commands, or position information. The transmitter 130 may be configured to perform any transmitting function described herein with respect to the device 104. For example, the transmitter 130 may be configured to transmit information to the other device, which may include a request for content. The receiver 133 and the transmitter 130 may be combined into a transceiver 132. In such an example, the transceiver 132 may be configured to perform any receiving and / or transmitting functions described herein with respect to the device 104.

[0058] 1B , in certain embodiments, the processing unit 127 may include a control component 198 configured to control a processor (including a CPU or GPU) or a general-purpose processor to perform real-time volume rendering of the dense particles. Further, the control component 198 may be configured to convert particle data representing each of the dynamic particles into a density volume representing a density distribution of the dynamic particles distributed in a three-dimensional (3D) space, pre-calculate a light distribution in the density volume representing light values ​​for each grid point in the density volume using ray marching from a light source, render the dynamic particles in real-time by calculating pixel color values ​​determined using (i) the ray marching toward a viewpoint position, (ii) the density volume, and (iii) the light distribution, and output a representation of the dynamic particles based on the rendering.

[0059] As described herein, a device, such as device 104, may refer to any device, apparatus, or system configured to execute one or more techniques described herein. For example, a device may be a server, a base station, a user equipment, a client device, a station, an access point, a computer, such as a personal computer, a desktop computer, a laptop computer, a tablet computer, a computer workstation, or a mainframe computer, an end product, an apparatus, a phone, a smartphone, a server, a video game platform or console, a handheld device, such as a portable video game device or a personal digital assistant (PDA), a wearable computing device, such as a smart watch, an augmented device, or a virtual reality device, a non-wearable device, a display or display device, a television, a television set-top box, an intermediate network device, a digital media player, a video streaming device, a content streaming device, an in-vehicle computer, any mobile device, any device configured to generate graphical content, or any device configured to execute one or more techniques described herein. Although the processes herein may be described as being executed by a particular component, such as a GPU, in further embodiments, they may be executed using other components, such as a CPU, consistent with the disclosed embodiments.

[0060] 2 shows an example diagram for graphics processing according to one or more techniques of the present disclosure. Specifically, FIG. 2 shows an example diagram of a graphics processing pipeline 200 for real-time volume rendering of dynamic particles. As shown in FIG. 2, the graphics processing pipeline 200 includes a dynamic particle generation stage 201, a transformation stage 203, a pre-computation stage 205, and a rendering stage 207.

[0061] Initially, the graphics processing pipeline 200 begins by generating 201 dynamic snow particles from a physics-based simulation. Physics-based modeling and simulation systems are configured to map natural phenomena into computer simulation programs. Physics simulation is used in movies and video games to animate various phenomena such as explosions, car crashes, water, cloth, snow, etc.

[0062] Next, the graphics processing pipeline 200 includes converting 203 the dynamic particles into a density volume. The density volume represents the density distribution of each dynamic particle distributed in 3D space. For example, a simulated semi-transparent substance (e.g., snow, ash, dust, etc.) may be represented as a particle having a radius and a position in 3D space. Given the particle data, the particle may then be converted into a density value (or volume texture) that represents the density of the substance in a 3D grid. For each particle, the density contribution to each grid point within the radius of the particle may be calculated and accumulated to obtain the final density distribution in space.

[0063] The graphics processing pipeline 200 then includes precomputing 205 a light distribution within the density volume. The light distribution represents light values ​​for each grid point within the density volume using ray marching from the light source. The grid points within the density volume may correspond to fixed reference positions. To accelerate rendering, the light distribution may be precomputed within the volume and used later during rendering. In the context of graphics rendering within a video game, the light distribution may be precomputed since the position of the light source is known in advance. If the density volume or the light source changes, the light distribution may be recomputed.

[0064] In some embodiments, the light distribution may be considered as another volume texture. For each grid point, a light value is calculated using ray marching. Different methods of representing density volumes and light distributions are described below with reference to Figures 3A-3B. Additionally, several different methods of pre-calculating the lighting distribution are described in more detail below with reference to Figure 4.

[0065] As described herein, the graphics processing pipeline 200 may include pre-calculating the lighting distribution in several different ways. In a first embodiment, the most direct method is to pre-calculate the lighting distribution by creating a volume texture at the same resolution as the density volume and using ray marching to calculate the light at each voxel in parallel. In a second embodiment, the resolution of the light volume may be lower than the resolution of the density volume, in which case interpolation may be performed to obtain light values ​​at the queried locations. In a third embodiment, instead of calculating the light at each voxel in parallel, a slice-by-slice technique may be used to calculate the light volume, as shown in FIG. 4 below.

[0066] The graphics processing pipeline 200 then includes rendering 207 the dynamic particles in real-time using ray marching. The rendering is performed by calculating pixel color values ​​determined using ray marching towards the viewpoint position, density volume, and pre-computed light distribution. As described above, ray marching is a computer graphics technique that emits a ray that passes through a volume of a medium. Samples are then taken at regular intervals along the way to calculate the interaction between the light and the medium, and the calculations are accumulated along the ray to obtain the display parameters of the volume. Specifically, each pixel may emit a ray in the direction of the camera, and samples may be taken on the ray section that intersects with the density volume to determine the display parameters of that pixel. Illumination may then be obtained from the pre-computed light volume at each sample. Scattered luminance values ​​are then calculated and accumulated to generate the final pixel color values ​​for display.

[0067] Advantages of this graphics processing pipeline 200 include the application of real-time volume rendering to simulate dense semi-transparent natural materials such as snow, ash, or dust. Related real-time volume rendering techniques for low-density media such as volumetric clouds and volumetric fog have been used in game engines, but could not be directly applied to dense natural materials because dense natural materials could not be efficiently rendered with large ray marching steps.

[0068] Until recently, accurate physically-based algorithms for realistically simulating and rendering dense natural materials have been difficult because they are too computationally expensive to apply in real-time applications. As a result of the rapid development of computing power in GPUs, it is beginning to become feasible to physically simulate and render these effects in real-time. Furthermore, compared to related snow-rendering techniques, such as using sprites or polygon meshes with subsurface scattering materials, the graphics processing pipeline 200 produces more accurate shadows and realistic granular results for volumes of dense materials.

[0069] 3A shows an example diagram of representing density and lighting distributions as axis-aligned bounding boxes according to one or more techniques of the present disclosure. Example 300a represents density and lighting distributions in 3D using 3D boxes aligned with Cartesian axes (e.g., axis-aligned bounding boxes). This method is a quick and straightforward way to represent density and light, but has some drawbacks because objects in the grid near the camera may appear blurry because grid points near the camera may appear too far away on the screen due to perspective projection. This can result in blurred visual results for particles close to the camera.

[0070] FIG. 3B shows an example diagram of density and lighting distribution as a froxel volume according to one or more techniques of the present disclosure. Example 300b shows density and lighting distribution in 3D using a "froxel volume". Froxel is a combination of the words "frustum" and "voxel", which means that the voxel grid is arranged in a frustum shape that aligns with the camera frustum and moves with the camera. A frustum shape is a part of a solid body such as a cone or a truncated pyramid. Frustums are typically used in game engines to render objects that are located inside the camera frustum. The camera frustum represents the field of view of the camera. The advantage of froxels is that the grid is denser in areas near the camera, which produces a sharper visual result than a 3D box method with the same number of voxels (e.g., as shown in FIG. 3A).

[0071] Below are shown the results of a comparison between representing density and lighting distribution in 3D using 3D boxes and representing density and lighting distribution in 3D using froxels to render a snow scene.

[0072] [Table 1]

[0073] Compared to the 3D box method described in Figure 3A, the use of froxel volumes reduces the number of voxels required to reach the same visual quality compared to the 3D box method, thus reducing computational costs and increasing frame rates.

[0074] 4 shows an example of per-slice light pre-calculation according to one or more techniques of the present disclosure. Per-slice light pre-calculation can be an effective way to calculate light distribution. The general concept of per-slice techniques is that for each slice, the light calculation depends on the results of the previous slice.

[0075] As shown in example 400, a first step includes determining 401 a light pre-computation for an initial slice from multiple volume slices, propagating 403 the results of the light pre-computation from the initial slice when determining the light pre-computation for a second volume slice, propagating 405 the results of the light pre-computation from the second slice when determining the light pre-computation for a third volume slice, and so on until propagating 407 the results of the light pre-computation from the second slice to the final slice when determining the light pre-computation for the final volume slice. In this way, per-slice light pre-computation avoids redundant ray firing and provides faster computation compared to computing all voxels in parallel.

[0076] Based on a comparison between no light precomputation, per-voxel precomputation, and per-slice precomputation, per-slice precomputation can significantly outperform both the no light precomputation method and the per-voxel precomputation. Example results from a comparison between the three precomputation methods are shown below.

[0077] [Table 2]

[0078] 5 shows a flowchart of various exemplary methods for graphics processing according to one or more techniques of this disclosure. Method 500 may be performed by an apparatus such as control component 198, as described above. In some implementations, method 500 is performed by processing logic including hardware, firmware, software, or a combination thereof. In some implementations, method 500 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., memory). Method 500 includes volume rendering of dynamic particles in real-time applications.

[0079] At block 502, the method 500 includes generating particle data representing simulated particles comprised of a simulated substance via a physics-based simulation to simulate a natural phenomenon. The generated particles include movement of the simulated particles in 3D space. In some embodiments, the simulated substance is snow. In some embodiments, the simulated substance is ash. In some embodiments, the simulated substance is dust. In some embodiments, the simulated substance has translucent properties or is a translucent substance. For example, in the context of FIG. 2, the graphics processing pipeline 200 includes generating 201 dynamic particles using a physics-based simulation to simulate a natural phenomenon.

[0080] At block 504, the method 500 includes converting the particle data representing each of the respective dynamic particles into a density volume representing a density distribution of the respective dynamic particles distributed in three-dimensional (3D) space. In some aspects, the conversion includes determining a density value contribution of each grid point within a particle radius based on the particle data and summing the density value contributions to obtain a density distribution of the dynamic particles distributed in the 3D space. For example, in the context of FIG. 2, the graphics processing pipeline 200 includes converting 203 the particle data into a density volume.

[0081] In some embodiments, the density volume and light distribution are represented by 3D boxes aligned with Cartesian coordinate axes. For example, in the context of FIG. 3A, example 300a represents the density volume and light distribution using bounding boxes aligned with three-dimensional axes.

[0082] In some embodiments, the density volume and light distribution are represented using froxel volumes aligned to the camera position. For example, in the context of FIG. 3B, example 300b represents the density volume and light distribution using froxels.

[0083] At block 506, the method 500 includes pre-calculating a light distribution within the density volume representing light values ​​for each grid point within the density volume using ray marching from the light source. In some aspects, pre-calculating the light distribution includes generating a light distribution at the same resolution as the resolution of the density distribution of the dynamic particles. Light values ​​at each voxel of each grid point within the density volume are determined in parallel using ray marching. For example, in the context of FIG. 2, the graphics processing pipeline 200 includes pre-calculating 205 the light distribution.

[0084] In some aspects, the pre-computation includes generating a light distribution at a lower resolution than the resolution of the density distribution of the dynamic particles and interpolating the generated light distribution. This aspect is based on the idea that since the light distribution is typically smooth, the light volume can have a lower resolution than the density volume without significantly affecting the visual quality.

[0085] In some embodiments, the pre-computation includes generating multiple volume slices of the density volume, determining a light distribution result for an initial volume slice from the multiple volume slices, and for each additional volume slice after the initial volume slice, determining a light distribution result for each additional volume slice by performing ray marching based on the light distribution result from the previous volume slice. Furthermore, using a low-resolution light volume significantly reduces computational costs and increases frame rates. For example, in the context of FIG. 4, example 400 describes generating a light distribution using a slice-by-slice technique.

[0086] At block 508, the method 500 includes rendering the dynamic particles in real-time by calculating pixel color values ​​determined using ray marching toward the viewpoint position, the density volume, and the light distribution. In some aspects, the pixel color values ​​are calculated by generating rays for each pixel from the camera position, sampling the generated rays at sections that intersect with the density volume, obtaining light values ​​from the light distribution at each sample, calculating scattered intensity values ​​at each sample based on the obtained light values, and calculating pixel color values ​​based on the scattered intensity values. For example, in the context of FIG. 2, the graphics processing pipeline 200 includes rendering 207 the dynamic particles in real-time using ray marching.

[0087] At block 510, the method 500 includes outputting a representation of the dynamic particles based on the rendering.

[0088] 6 shows a flow chart illustrating an example method 600 for graphics processing utilizing dual processing circuits, e.g., GPUs, according to one or more techniques of the present disclosure. Method 600 may be performed by an apparatus, such as control component 198, as described above. In some implementations, method 600 is performed by processing logic including hardware, firmware, software, or a combination thereof. In some implementations, method 600 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., memory). Method 600 describes volume rendering of dynamic particles using dual processing circuits in a real-time application.

[0089] In some aspects, the method 600 includes generating particle data representing simulated particles comprised of a simulated substance by a physics-based simulation of a natural phenomenon. The generated particle data includes movement of the simulated particles in 3D space. In some aspects, the simulated substance is snow. In some aspects, the simulated substance is ash. In some aspects, the simulated substance is dust. In some aspects, the simulated substance has translucent properties or is a translucent substance. For example, in the context of FIG. 2, the graphics processing pipeline 200 includes generating 201 dynamic particles of a simulated substance by a physics-based simulation of a natural phenomenon.

[0090] At block 602, the method 600 includes controlling a first processing circuit to convert particle data representing each of the dynamic particles of the first frame into a density volume representing a density distribution of the respective dynamic particles distributed in three-dimensional (3D) space. For example, in the context of FIG. 2, the graphics processing pipeline 200 includes converting 203 the particle data into a density volume.

[0091] At block 604, the method 600 includes controlling the first processing circuit to pre-calculate a light distribution within the density volume representing light values ​​for each grid point within the density volume using ray marching from the light source. The grid points within the density volume correspond to fixed reference positions. For example, in the context of FIG. 2, the graphics processing pipeline 200 includes pre-calculating 205 the light distribution.

[0092] After block 604, blocks 606 and 608 are executed in parallel.

[0093] At block 606, method 600 includes copying the light distribution of the first frame from the first processing circuit to the second processing circuit. In some aspects, copying the light distribution of the first frame from the first processing circuit to the second processing circuit is performed in parallel with the first processing circuit beginning pre-processing of the second frame and before the second processing circuit performs volume rendering of the first frame in real time. In some aspects, block 606 includes copying both the light distribution and density volume of the first frame from the first processing circuit to the second processing circuit. In other aspects, block 606 includes copying the light distribution of the first frame to the second processing circuit and generating a density volume for the first frame in the second processing circuit, such that the density volume does not need to be copied from the first processing circuit.

[0094] At block 608, the method 600 includes controlling the first processing circuit to convert the particle data of the second frame to a density volume after completion of the pre-computation in the first processing circuit for the first frame. For example, in the context of FIG. 2, the graphics processing pipeline 200 includes converting 203 the particle data to a density volume. In other words, block 608 serves as the start of a new process 600, where the second frame is the first frame.

[0095] At block 610, the method 600 includes controlling the second processing circuit to render the dynamic particles in the first frame in real time by calculating pixel color values ​​determined using ray marching toward the viewpoint position, the density volume, and the light distribution copied from the first processing circuit. For example, in the context of FIG. 2, the graphics processing pipeline 200 includes rendering 207 the dynamic particles in real time using ray marching.

[0096] At block 612, the method 600 includes causing an output of a representation of the dynamic particles in the first frame based on the rendering.

[0097] The specific interactions between the steps performed by the first processing circuit and the second processing circuit in performing real-time volume rendering of dynamic particles are described in more detail below in FIG. 7B.

[0098] FIG. 7A illustrates an example diagram for a graphics processing pipeline using a single processing circuit, e.g., a GPU, according to one or more techniques of the present disclosure. Example 700a illustrates performing real-time volume rendering of dynamic particles using a single GPU 701. Here, splatting refers to converting particle data into a density volume representing the density distribution of dynamic particles distributed in 3D space. Illumination refers to pre-calculating a light distribution in the density volume representing the light value for each grid point in the density volume. Volume rendering refers to rendering dynamic particles in real-time by calculating pixel color values ​​determined using ray marching with respect to the camera position, density volume, and light distribution, and other rendering refers to other rendering processes such as object rendering with materials other than volume materials, shading, ray tracing, ray casting, refraction, texture mapping, etc. Here, the total time to render a single GPU frame may take a total of 25.16 milliseconds.

[0099] FIG. 7B shows an example diagram for a graphics processing pipeline using multiple processing circuits in accordance with one or more techniques of this disclosure.

[0100] To further accelerate the calculations, multiple processing circuits (e.g., GPUs) may be used in the rendering process. Example 700b illustrates using dual GPUs 703 and 705 to perform real-time volume rendering of dynamic particles. In example 700b, a dotted vertical line is aligned with the second GPU 705 to indicate the total time to render a single GPU frame by the second GPU. Compared to using a single GPU as shown in example 700a, example 700b illustrates that using dual GPUs to perform real-time volume rendering of dynamic particles significantly reduces the time to render a single GPU frame.

[0101] Example 700b shows that the first GPU 703 converts particle data of the first frame into a density volume representing the density of dynamic particles distributed in three-dimensional (3D) space, and pre-calculates a light distribution in the density volume representing light values ​​for each grid point in the density volume in the first frame using ray marching with respect to the light source. The light distribution of the first frame is then copied from the first GPU to the second GPU. The copying process is performed in parallel with other steps performed by the first GPU and the second GPU, such that by the time the second GPU performs a pre-rendering of the first frame, the first GPU has already moved onto the second frame. Additionally, the second GPU begins converting the particle data of the first frame into a density volume (shown as "splatting" in FIG. 7B) while the first GPU is in the process of pre-calculating the light distribution of the first frame. Thus, once the light distribution of the first frame is copied to the second GPU, the second GPU is prepared to perform rendering based on the light distribution and density volume of the first frame. In this way, different steps may be performed simultaneously on different GPUs, and the frame rate is increased by 30%.

[0102] Shown below are exemplary results from the performance between performing real-time volume rendering of dynamic particles with a single GPU and using dual GPUs to render a scene including snow.

[0103] [Table 3]

[0104] The subject matter described herein can be implemented to realize one or more benefits or advantages. For example, the techniques disclosed herein enable a method for real-time volume rendering of dynamic particles such as snow. As a result, realistic simulations of dense natural materials may be volume rendered in real-time applications such as video games. Furthermore, the techniques disclosed herein also produce more realistic shadows and render dense volumes more efficiently and with better visual quality than related real-time techniques.

[0105] The subject matter described herein can be implemented to realize one or more benefits or advantages. For example, the described graphics processing techniques can be used by a server, client, GPU, CPU, or any other processor capable of performing computer or graphics processing to implement the sharing techniques described herein. This can also be achieved at a low cost compared to other computer or graphics processing techniques. Furthermore, the computer or graphics processing techniques herein can improve or speed up data processing or execution. Furthermore, the computer or graphics processing techniques herein can improve resource or data utilization and / or resource efficiency.

[0106] In accordance with the present disclosure, the term "or" may be interpreted as "and / or" unless the context dictates otherwise. Furthermore, phrases such as "one or more" or "at least one" may be used with some features disclosed herein but not with other features, and features without such language may be interpreted as having such an implied meaning unless the context dictates otherwise.

[0107] In one or more examples, the functions described herein may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term "processing unit" is used throughout this disclosure, such a processing unit may be implemented in hardware (e.g., by a processing circuit), software, firmware, or any combination thereof. If any function, processing unit, technique described herein, or other module is implemented in software, the function, processing unit, technique described herein, or other module may be stored or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media may include computer data storage media or communication media, including any medium that facilitates transfer of a computer program from one place to another. In this manner, computer-readable media may generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium, such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to obtain instructions, code, and / or data structures for implementation of the techniques described in this disclosure. By way of example and not limitation, such computer readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks typically reproduce data magnetically, whereas discs reproduce data optically using a laser. Combinations of the above should also be included within the scope of computer readable media. A computer program product may include a computer readable medium.

[0108] The code may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), arithmetic logic units (ALUs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor" as used herein may refer to any of the foregoing structures, or any other structure suitable for implementing the techniques described herein. Also, the techniques may be implemented entirely in one or more circuits or logic elements.

[0109] The techniques of the present disclosure may be implemented in a wide variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or sets of ICs, such as chipsets. In this disclosure, various components, modules, or units are described to highlight functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined into any hardware unit, along with appropriate software and / or firmware, or may be provided by a collection of interoperating hardware units including one or more processors as described above. [Explanation of symbols]

[0110] 100 Content Generation System 104 Devices 107 Graphics Processing Pipeline 111 Software Applications 113 Operating System (OS) 115 Graphics Driver 120 GPU 121 Graphics Memory (GMEM) 122 Content Encoder / Decoder 123 Density Volume Processor 124 system memory 125 Light Distribution Processor 126 Communication Interface 127 Processing Unit 128 Frame Buffer 129 Internal Memory 130 Transmitter 131 Display 132 Transceiver 133 Receiver 198 Control Components 200 Graphics Processing Pipeline 201 Dynamic particle generation stage 203 Conversion Stage 205 Precomputation stage 207 Rendering Stage 701, 703, 705 GPUs

Claims

1. 1. A method for real-time volume rendering of dynamic particles, comprising: converting particle data representing each of the dynamic particles into a density volume representing a density distribution of the respective dynamic particles distributed in three-dimensional (3D) space; pre-calculating a light distribution within the density volume using ray marching from a light source, the light distribution representing a light value for each grid point within the density volume, the grid points within the density volume corresponding to fixed reference positions; Rendering the dynamic particles in real-time by (i) ray marching towards a viewpoint position, (ii) calculating pixel color values ​​determined using the density volume, and (iii) the light distribution; outputting a representation of the dynamic particles based on the rendering; A method comprising:

2. generating particle data representing simulated particles composed of a simulated substance by a physics-based simulation of a natural phenomenon, the generated particle data including movements of the simulated particles within the 3D space; The method of claim 1, further comprising:

3. The method of claim 2 , wherein the simulated material is snow.

4. The method of claim 2 , wherein the simulated material is ash.

5. The method of claim 2 , wherein the simulated substance is dust.

6. The method of claim 2 , wherein the simulated material is translucent.

7. The step of converting comprises: determining a density value contribution for each grid point within a particle radius based on the particle data; summing the density value contributions to obtain the density distribution of the dynamic particles distributed in the 3D space; 2. The method of claim 1, comprising:

8. The method of claim 1 , wherein the density volume and the light distribution are represented in a 3D box aligned with Cartesian coordinate axes.

9. The method of claim 1 , wherein the density volume and the light distribution are represented using a froxel volume aligned to the viewpoint position.

10. The step of pre-calculating the light distribution comprises: generating the light distribution at the same resolution as the density distribution of the dynamic particles, the light values ​​at each voxel of each grid point within the density volume being determined in parallel using the ray marching; 2. The method of claim 1, comprising:

11. The step of pre-calculating the light distribution comprises: generating the light distribution at a resolution lower than a resolution of the density distribution of the dynamic particles; Interpolating the generated light distribution.

2. The method of claim 1, comprising:

12. The step of pre-calculating the light distribution comprises: generating a number of volume slices of the density volume; determining a light distribution result for an initial volume slice from the plurality of volume slices; for each additional volume slice after the initial volume slice, determining a light distribution result for the respective additional volume slice by performing the ray marching based on a light distribution result from a previous volume slice; 2. The method of claim 1, comprising:

13. The pixel color value is generating a ray for each pixel from the viewpoint position; sampling the generated rays at sections that intersect with the density volume; obtaining light values ​​from the light distribution at each sample; calculating a diffuse luminance value for each sample based on the obtained light values; and calculating the pixel color value based on the diffuse luminance values. The method of claim 1, wherein the calculation is performed by

14. 1. An apparatus for real-time volume rendering of dynamic particles, the apparatus comprising: converting particle data representative of each of the dynamic particles into a density volume representative of a density distribution of each of the dynamic particles distributed in three-dimensional (3D) space; pre-calculating a light distribution within the density volume using ray marching from a light source, the light distribution representing a light value for each grid point within the density volume, the grid points within the density volume corresponding to fixed reference positions; Rendering the dynamic particles in real-time by calculating pixel color values ​​determined using (i) ray marching towards a viewpoint position, (ii) the density volume, and (iii) the light distribution; generating an output of a representation of the dynamic particles based on the rendering; A processing circuit configured as follows: An apparatus comprising:

15. The processing circuitry includes: generating said particle data representing simulated particles comprised of a simulated substance by a physics-based simulation of a natural phenomenon, said generated particle data including movements of said simulated particles within said 3D space; The apparatus of claim 14, further configured to:

16. The apparatus of claim 15 , wherein the simulated material is snow.

17. The apparatus of claim 14 , wherein the density volume and the light distribution are represented in a 3D box aligned with Cartesian coordinate axes.

18. 1. A system for real-time volume rendering of dynamic particles, comprising: A first processing circuit; A second processing circuit; controlling the first processing circuit to convert particle data representing each of the dynamic particles in a first frame into a density volume representing a density distribution of the respective dynamic particles distributed in three-dimensional (3D) space; controlling the first processing circuit to pre-calculate a light distribution within the density volume representing a light value for each grid point within the density volume using ray marching from a light source, the grid points within the density volume corresponding to fixed reference positions; copying the light distribution of the first frame from the first processing circuit to the second processing circuit; controlling the first processing circuit to convert particle data of a second frame into a density volume after completion of the pre-computation in the first processing circuit for the first frame; controlling the second processing circuit to render in real time the dynamic particles in the first frame by calculating pixel color values ​​determined using (i) ray marching towards a viewpoint position, (ii) the density volume, and (iii) the light distribution copied from the first processing circuit; generating an output of a representation of the dynamic particles in the first frame based on the rendering. A control circuit configured as follows: A system comprising:

19. 20. The system of claim 18, wherein the copying of the light distribution of the first frame from the first processing circuit to the second processing circuit is performed in parallel with the first processing circuit starting pre-processing of the second frame and before the second processing circuit performs volume rendering of the first frame in real time.

20. The system comprises: generating said particle data representing simulated particles comprised of a simulated substance by a physics-based simulation of a natural phenomenon, said generated particle data including movements of said simulated particles within said 3D space; 20. The system of claim 18, further configured to:

Images