Apparatus and method for real-time volume rendering of dynamic particles
By converting the particle data of dynamic particles into density volumes and pre-calculating the light distribution using light travel technology, the problem of high calculation cost and unrealistic effect of rendering dynamic high-density natural substances in the prior art is solved, and real-time applications are realized efficient and realistic rendering effect.
Patent Information
- Application Number
- CN202380027150.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-11-28
- Filing Date
- 2023-03-20
- Publication Date
- 2024-11-05
AI Technical Summary
The prior art is difficult to efficiently render dynamic high-density natural matter, such as snow, in real-time applications, resulting in high computational costs, unreal rendering effects, and difficulty in dealing with complex geometric shapes of dynamic particles.
By converting the particle data of dynamic particles into density volumes, and pre-calculating the light distribution using light travel technology, combining the light travel and density volume towards the viewpoint position, dynamic particles are rendered in real time to achieve real simulation and rendering of high-density natural matter.
Reduces the computational overhead of rendering dynamic snow particles, generates more realistic shadows and volume effects, and improves the rendering quality of graphics content and the performance of processing units.
Smart Images

Figure CN118901084A_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims priority to U.S. Patent Application No. 18 / 070,326, filed on November 28, 2022, the entire contents of which are incorporated herein by reference. Technical Field
[0003] The present disclosure generally relates to processing systems, including one or more techniques for graphics processing. Background Art
[0004] Computing devices typically utilize a graphics processing unit (GPU) or a central processing unit (CPU) to render graphics data to provide a display. Computing devices may include, for example, computer workstations, mobile phones such as smartphones, embedded systems, personal computers, tablet computers, and video game consoles. The GPU processes instructions and / or data in a graphics processing pipeline that includes one or more processing stages that can operate together to execute graphics processing commands and output frames. The CPU can control the operation of the GPU by issuing one or more graphics processing commands to the GPU. Current CPUs are typically capable of executing multiple applications simultaneously, each of which may require the use of a GPU during execution. Devices that provide content for visual presentation on a display typically include a GPU.
[0005] Generally, a GPU of a device is configured to perform processing in a graphics processing pipeline. However, as the complexity of rendered content and the physical limitations of GPU memory increase, the need for improved computer or graphics processing is also increasing. Summary of the invention
[0006] A simplified overview of one or more aspects is provided below to provide a basic understanding of these aspects. This overview is not an extensive summary of all anticipated aspects, and is neither intended to identify the key elements of all aspects nor to describe the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to a more detailed description presented later.
[0007] The present disclosure relates to methods and apparatus for graphics processing. One aspect of the subject matter described in the present disclosure is implemented in a method for real-time volume rendering of dynamic particles. The method includes converting particle data representing each dynamic particle into a density volume representing a density distribution of dynamic particles distributed in a three-dimensional (3D) space. The method also includes pre-calculating a light distribution within the density volume, the light distribution using light marches from a light source to represent the light value of each grid point within the density volume. The grid points within the density volume correspond to fixed reference positions. The method also includes rendering the dynamic particles in real time by calculating pixel color values. Wherein the pixel color value is determined using (i) light marches toward a viewpoint position, (ii) the density volume, and (iii) the light distribution. The method includes outputting a representation of the dynamic particles based on the rendering.
[0008] Another aspect of the subject matter described in the present disclosure can be implemented in a device for real-time volume rendering of dynamic particles. The device includes a processing circuit configured to convert particle data representing each dynamic particle into a density volume representing a density distribution of dynamic particles distributed in a three-dimensional (3D) space. The processing circuit is configured to pre-calculate a light distribution within the density volume, the light distribution using light marches from a light source to represent the light value of each grid point within the density volume. The grid points within the density volume correspond to fixed reference positions. The processing circuit is configured to render the dynamic particles in real time by calculating pixel color values. Wherein the pixel color value is determined using (i) light marches toward a viewpoint position, (ii) the density volume, and (iii) the light distribution. The processing circuit is configured to output a representation of the dynamic particles based on the rendering.
[0009] Another aspect of the subject matter described in the present disclosure can be implemented in a system for real-time volume rendering of dynamic particles. The system includes a first processing circuit, a second processing circuit, and a control circuit. The control circuit is configured to control the first processing circuit to convert particle data representing each dynamic particle of a first frame into a density volume representing the density of dynamic particles distributed in a three-dimensional (3D) space. The control circuit is also configured to control the first processing circuit to pre-calculate a light distribution within the density volume, which uses light rays traveling toward a light source to represent the light value of each grid point within the density volume of the first frame. The grid points within the density volume correspond to fixed reference positions. The control circuit is also configured to copy the density volume and the light distribution of the first frame from the first processing circuit to the second processing circuit when the pre-calculation of the first frame is completed in the first processing circuit. The control circuit is also configured to control the first processing circuit to convert the particle data of the second frame into a density volume. The control circuit is further configured to control the second processing circuit to render the dynamic particles in the first frame in real time by calculating pixel color values, wherein the pixel color values are determined using (i) ray marching toward a viewpoint position, (ii) the density volume, and (iii) the light distribution copied from the first processing circuit. The control circuit is further configured to output a representation of the dynamic particles in the first frame based on the rendering.
[0010] Another aspect of the subject matter described in the present disclosure can be implemented in a non-transitory computer-readable storage medium storing instructions. When executed by at least one processor, the instructions cause the processor to perform: converting particle data representing each dynamic particle into a density volume representing a density distribution of dynamic particles distributed in a three-dimensional (3D) space. The processor is also configured to pre-calculate a light distribution within the density volume, wherein the light distribution uses light rays from a light source to represent the light value of each grid point within the density volume. The processor is also configured to render the dynamic particles in real time by calculating pixel color values. Wherein, the pixel color value is determined using (i) light rays marching toward a viewpoint position, (ii) the density volume, and (iii) the light distribution. The processor is also configured to output a representation of the dynamic particles based on rendering.
[0011] To accomplish the above and related ends, one or more aspects include the features hereinafter fully described and particularly pointed out in the claims. The following description and the accompanying drawings set forth in detail some illustrative features of one or more aspects. However, these features are indicative of only a few of the various ways in which the principles of the various aspects may be employed, and this description is intended to include all such aspects and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The details of one or more aspects of the subject matter described in the present disclosure are set forth in the accompanying drawings and the description below. However, the accompanying drawings only illustrate some typical aspects of the present disclosure and therefore should not be considered as limiting the scope thereof. Other features, aspects and advantages will become apparent from the description, drawings and claims.
[0013] Figure 1A is a block diagram illustrating an example of a content generation system in accordance with one or more techniques of this disclosure.
[0014] Figure 1B is a block diagram illustrating an example of a content generation system in accordance with one or more techniques of this disclosure.
[0015] Figure 2 An example diagram of a graphics processing pipeline is shown, in accordance with one or more techniques of this disclosure.
[0016] Figure 3A An example diagram showing density and light distribution as axis-aligned bounding boxes is shown, in accordance with one or more techniques of this disclosure.
[0017] Figure 3B A schematic diagram showing density and light distribution as froxel volumes, in accordance with one or more techniques of this disclosure is shown.
[0018] Figure 4 An example of per-sheet light precomputation in accordance with one or more techniques of this disclosure is shown.
[0019] Figure 5 A flowchart of an exemplary method for graphics processing in accordance with one or more techniques of this disclosure is shown.
[0020] Figure 6 A flow chart illustrating an exemplary method of performing graphics processing using dual processing circuits, in accordance with one or more techniques of this disclosure.
[0021] Fig. 7A An example diagram of a graphics processing pipeline using a single processing circuit is shown, in accordance with one or more techniques of this disclosure.
[0022] Figure 7B An example diagram of a graphics processing pipeline using dual processing circuits is shown, in accordance with one or more techniques of this disclosure.
[0023] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION
[0024] The following description describes some exemplary aspects of the innovative aspects of the present disclosure. However, it will be readily appreciated by those skilled in the art that the teachings herein can be applied in many different ways.
[0025] Related systems have implemented volume rendering methods in offline applications. In offline applications, dense natural materials (such as snow) are usually rendered through volume representation. First, the dense natural material particles are converted into density volumes, and then rendered using various path tracking algorithms (such as delta tracking and residual tracking). These methods can produce realistic and physically accurate results, but require many samples to reduce noise, so a lot of computing time is consumed. Therefore, these implementations are not suitable for real-time applications to render dynamic dense natural materials.
[0026] Related systems have also implemented snow rendering and volume rendering in real-time applications. There are some methods to render snow in real-time applications such as video games. For static scenes (e.g., a snow-covered landscape), you can use polygonal meshes with subsurface scattering materials, which can be rendered quickly. However, this method cannot handle dynamic particles because it is difficult to capture the complex geometry of particles using polygons.
[0027] Volume rendering is also used in game engines such as Unreal Engine. These real-time volume rendering techniques include volumetric clouds and volumetric fog (e.g., natural materials with low-density media), which can be efficiently rendered with large ray marching steps. In contrast, translucent materials (such as snow, ash, or powder) have high-density media and require smaller ray marching steps, which will increase the computational cost. Therefore, optimization is needed to be able to render high-density materials using ray marching in real-time applications.
[0028] Aspects of the present disclosure can create more realistic simulations and real-time renderings of natural materials by implementing volume rendering of dynamic elements, such as natural translucent materials (e.g., snow, powder, or ash) generated by physically based simulations. For example, aspects of the present disclosure provide a GPU pipeline to render simulated natural particles in a volume at high density in real-time applications. By doing so, when aspects of the present disclosure simulate these natural particles, the data particles are converted into volume textures, and then light marching is used to calculate the interaction between light and the medium. Therefore, compared to related rendering techniques, various aspects of the present disclosure can create more accurate shadows for high-density particles, as well as more physically accurate depictions. In addition, some aspects of the present disclosure implement several graphics processing optimizations to improve performance by reducing computational costs and increasing frame rates.
[0029] The following describes various aspects of the system, device, computer program product, and method more fully with reference to the accompanying drawings. However, the present disclosure can be embodied in many different forms and should not be interpreted as being limited to any specific structure or function throughout the present disclosure. More specifically, these aspects are provided to make the present disclosure in-depth and complete, and fully convey the scope of the present disclosure to those skilled in the art. Based on the teachings of this article, it should be understood by those skilled in the art that the scope of the present disclosure is intended to cover any aspect of the system, device, computer program product, and method disclosed herein, whether implemented independently of other aspects of the present disclosure or combined with other aspects of the present disclosure. For example, any number of aspects set forth herein can be used to implement the device or implementation method. In addition, the scope of the present disclosure is intended to cover such a device or method, which is implemented using other structures, functions, or structures and functions other than or different from the various aspects of the present disclosure set forth herein. Any aspect disclosed herein may be embodied by one or more elements of the claims.
[0030] Although various aspects are described herein, many variations and permutations of these aspects are within the scope of the present disclosure. Although some potential benefits and advantages of various aspects of the present disclosure are mentioned, the scope of the present disclosure is not intended to be limited to specific benefits, uses or objectives. More specifically, various aspects of the present disclosure are intended to be widely applicable to different wireless technologies, system configurations, networks and transmission protocols, some of which are shown by way of example in the drawings and the following description. The detailed description and drawings are merely illustrative of the present disclosure and are not limiting, and the scope of the present disclosure is defined by the appended claims and their equivalents.
[0031] Several aspects are presented with reference to various apparatus and methods. These apparatus and methods are described in the detailed description below and illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, etc. (collectively referred to as "elements"). These elements may be implemented using electronic hardware, computer software, or any combination thereof. Whether these elements are implemented as hardware or as software depends on the specific application and design constraints imposed on the overall system.
[0032] For example, an element, any part of an element, or any combination of elements may be implemented as a "processing system" including one or more processors (also referred to as processing circuits). One or more processors in a processing system may execute software. Whether the software is referred to as software, firmware, middleware, microcode, hardware description language, or other, the software may be broadly interpreted as instructions, instruction sets, codes, code segments, program codes, programs, subroutines, software components, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, processes, functions, etc. The term application may refer to software. As described herein, one or more technologies may refer to applications configured to perform one or more functions, i.e., software. In these examples, applications may be stored on a memory, such as an on-chip memory, system memory, or any other memory of a processor. The hardware described herein, such as a processor, may be configured to execute an application. For example, the application may be described as including a code that causes the hardware to perform one or more technologies described herein when executed by the hardware. As an example, the hardware may access code from a memory and execute the code accessed from the memory to perform one or more technologies described herein. In some examples, components are identified in the present disclosure. In these examples, the components may be hardware, software, or a combination thereof. These components may be separate components or subcomponents of a single component.
[0033] Therefore, in one or more examples described herein, the functions described can be implemented with hardware, software, or any combination thereof. If implemented with software, these functions can be stored on one or more instructions or codes on a computer-readable medium or encoded as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media. Storage media can be any available media that a computer can access. As an example and not limitation, the computer-readable medium can include a random access memory (RAM), a read-only memory (ROM), an electrically erasable programmable ROM (EEPROM), an optical disk storage, a magnetic disk storage, other magnetic storage devices, a combination of computer-readable media of the above types, or any other medium that can be used to store computer executable code in the form of instructions or data structures that can be accessed by a computer.
[0034] The present disclosure includes real-time volume rendering techniques for elements (e.g., high-density substances generated by physically based simulations). Aspects of the present disclosure enable realistic simulation of dynamic high-density natural substances in real-time applications (e.g., video games that include interactions with natural environments). In addition, the disclosed real-time volume rendering techniques improve the rendering of graphical content and / or reduce the load on a processing unit, i.e., any processing unit, such as a GPU (or processing circuit), configured to perform one or more techniques described herein. Other exemplary benefits are described in the present disclosure.
[0035] As used herein, instances of the term "content" may refer to "graphic content," "images," and vice versa. This is true regardless of whether these terms are used as adjectives, nouns, or other parts of speech. In some examples, as used herein, the term "graphics content" may refer to content produced by one or more processes of a graphics processing pipeline. In some examples, as used herein, the term "graphics content" may refer to content produced by a processing unit configured to perform graphics processing. In some examples, as used herein, the term "graphics content" may refer to content produced by a graphics processing unit.
[0036] As used herein, the term "display content" may refer to content generated by a processing unit configured to perform display processing. In some examples, as used herein, the term "display content" may refer to content generated by a display processing unit. Graphics content may be processed into display content. For example, a graphics processing unit may output graphics content such as a frame to a buffer (which may be referred to as a frame buffer). The display processing unit may read graphics content, such as one or more frames, from the buffer and perform one or more display processing techniques on the graphics content to generate display content. For example, the display processing unit may be configured to perform synthesis on one or more rendering layers to generate a frame. As another example, the display processing unit may be configured to compose, mix or otherwise combine two or more layers into a single frame. The display processing unit may be configured to perform scaling on a frame, for example, zooming in or out. In some examples, a frame may refer to a layer. In other examples, a frame may refer to two or more layers that have been mixed together to form a frame, that is, the frame includes two or more layers, and the frame including two or more layers may be subsequently mixed.
[0037] In general, a GPU (or processing circuit) can be used to render a three-dimensional (3D) scene. Because such rendering of a 3D scene can be memory bandwidth intensive, a dedicated graphics memory (GMEM) can be used. The GMEM can be located near the graphics processing core of the GPU so that the GMEM has a high memory bandwidth (i.e., read and write access to the GMEM is very fast). The scene can be rendered from the graphics processing core of the GPU to the GMEM, and the scene can be parsed from the GMEM to a memory (e.g., a frame buffer) so that the scene can be displayed on a display device. The rendering of an entire frame can be referred to as immediate mode rendering. However, due to physical memory constraints, the size of the GMEM is limited so that the GMEM may not have enough memory capacity to contain the entire three-dimensional scene (e.g., the entire frame).
[0038] Due to the high density of natural materials, realistically simulating and rendering dense natural translucent materials (e.g., snow, ash, or powder) is difficult for real-time applications (e.g., video games that include interaction with natural environments) or any other real-time application involving natural environments (e.g., virtual reality (VR) tourism or cultural heritage digitization). Accurate physically based algorithms are essential for realistic results, but are computationally too expensive to be used in real-time applications. However, with the rapid development of GPU computing power, simulating and rendering these effects in a real-time physically based way has only recently become feasible.
[0039] Over the past few decades, path tracing and its variants have become the standard approach for offline rendering. Path tracing includes computer graphics algorithms for rendering 3D scenes that accurately produce global illumination. The algorithm randomly samples multiple light transport paths and then calculates and accumulates the contributions of all paths to obtain the result to be rendered on the screen pixels. Specifically, path tracing samples the paths of light in a 3D scene. After a light source emits light, the light travels in a straight line until it scatters on a surface or in a medium, causing the light to change direction and continue to travel in a new direction until it reaches another scattering point. Path tracing algorithms simulate this behavior of light and generate multiple light paths in a 3D scene to estimate the overall lighting in the scene.
[0040] Path tracing can be used to render volumetric materials, such as snow, because unlike surface rendering where light continues to travel in a straight line until it reaches a surface, in volume rendering, light may scatter anywhere within the volume of the medium. To simulate this, path tracing randomly generates a scatter-free path length as the light propagates through the medium, and lets the light scatter at the end of the scatter-free path length. There are also various methods to sample the scatter-free path length, such as delta tracing and ratio tracing.
[0041] Path tracing may require a large number of light path samples to evaluate the light value to be displayed at each pixel. Therefore, if the number of samples is small, the rendered result will be noisy. Therefore, in order to obtain noise-free results, hundreds of samples per pixel are required. However, this means that depending on the complexity of the scene, rendering a clean frame via path tracing may take minutes or even hours, which is very expensive for real-time applications.
[0042] Meshes with subsurface scattering materials can be used to render static snow scenes. The mesh uses a set of small triangles to represent the surface of the snow. Subsurface scattering is a modeling technique used to render semi-transparent materials. For example, subsurface scattering involves simulating light penetrating the material, scattering underneath, and then bouncing back from the surface. For real-time rendering, the GPU has a rasterization pipeline that can render triangles quickly, unlike path tracing in offline rendering. Therefore, triangle meshes can be suitable for representing the surface of semi-transparent materials, but cannot effectively capture the complex geometry of semi-transparent materials represented by dynamic particles (for example, falling snow particles) because this would require a large number of triangles and be computationally expensive.
[0043] Sprites can also be used to render dynamic translucent particles. This method is simple, but does not produce realistic results. Sprites can refer to 2D bitmaps that are drawn on the screen. When rendering translucent particles, a sprite bitmap facing the camera is placed at the location of each particle and drawn at each particle's location. This method works well for rendering moving particles, but 2D bitmaps do not reproduce the lighting and shadow effects present in translucent particles very well and can result in unrealistic repeating patterns.
[0044] As described herein, an exemplary implementation uses volume rendering to render dynamic snow particles. Volume rendering includes a method of rendering light that interacts with a medium within a volume, where the light may be scattered and absorbed by the material before reaching the camera.
[0045] Volume rendering techniques can be used to render materials other than snow. The most common are volumetric clouds and volumetric fog. For example, volumetric clouds and / or fog can be rendered using ray marching techniques. Ray marching is a method of performing volume rendering. Ray marching techniques simulate light traveling through a large volume of medium. Samples are taken at fixed intervals along the ray to calculate the interaction between the light and the medium, and the calculations are accumulated along the ray to obtain the result. However, ray marching techniques are only applied to low-density media, such as clouds and fog in game engines, because low-density media are visually more blurry than high-density substances. This means that the density of the medium varies less, making it possible to render them using larger step sizes in ray marching. In addition, density volumes for fog and clouds can be easily generated using procedural materials, but generally game engines do not support the generation of volumes from particles. This means that current plug-ins are dedicated to rendering low-density media, i.e., clouds and fog, and cannot be directly applied to rendering high-density dynamic particles.
[0046] Therefore, embodiments of the present disclosure include methods and apparatus for real-time volume rendering of dynamic translucent particles to reduce the computational overhead of rendering dynamic snow particles while ensuring that the simulated snow looks real. Although snow particles are used as an example, it should be noted that various aspects of the present disclosure can be applied to simulate dynamic particles of other high-density translucent materials (e.g., ash, powder, etc.). The graphics processing pipeline can start by converting dynamic particles into volume textures, and then use light marching to calculate the interaction between light and the medium. The subject matter described herein can be implemented to achieve one or more benefits or advantages. For example, this embodiment allows real volume rendering of dynamic snow particles in real-time applications such as video games. In addition, compared with related art, this embodiment can produce more realistic shadows and better depict the volume of high-density natural particles.
[0047] Figure 1A 1 is a block diagram illustrating an exemplary content generation system 100 configured to implement one or more techniques of the present disclosure. The content generation system 100 includes a processing unit 127, a GPU 120, and a system memory 124 configured to render a 3D scene according to exemplary aspects. The processing unit 127 can execute a software application 111, an operating system (OS) 113, and a graphics driver 115. In addition, the system memory 124 can include an indirect cache that stores a command stream for rendering primitives and auxiliary commands to be executed by the GPU 120. The GPU 120 may include a graphics memory (GMEM) 121, which may be interconnected "on-chip" with the GPU 120 coupled to a density volume processor 123 and a light distribution processor 125. As described with respect to Figure 1B As described in more detail in , the components of the content generation system 100 can be part of a device including, but not limited to, a video device, a media player, a set-top box, a wireless handset such as a mobile phone and so-called smartphone, a personal digital assistant (PDA), a desktop computer, a laptop computer, a game console, a video conferencing unit, a tablet computing device, etc.
[0048] The processing unit 127 may include one or more central processing units (CPUs). The GPU 120 may include a processing unit configured to perform graphics-related functions (e.g., generating and outputting graphics data for presentation on a display) and non-graphics-related functions that perform the massive processing parallelism provided by the GPU 120. Because the GPU 120 may provide general processing capabilities in addition to graphics processing capabilities, the GPU 120 may be referred to as a general-purpose GPU (GP-GPU). Examples of the processing unit 127 and the GPU 120 include, but are not limited to, a digital signal processor (DSP), a general-purpose microprocessor, an application-specific integrated circuit (ASIC), a field-programmable logic array (FPGA), or other equivalent integrated or discrete logic circuits. In some examples, the GPU 120 may be a microprocessor designed for a specific purpose (e.g., providing massive parallel processing for processing graphics, and executing applications unrelated to graphics). In addition, although the processing unit 127 and the GPU 120 are shown as separate components, aspects of the present disclosure are not limited thereto and may be implemented, for example, in a general-purpose integrated circuit (IC).
[0049] The software application 111 executed on the processing unit 127 may include one or more graphics rendering instructions that instruct the processing unit 127 to render graphics data to a display ( Figure 1A In some examples, the graphics rendering instructions may include software instructions that conform to a graphics application programming interface (API). To process the graphics rendering instructions, processing unit 127 may issue one or more graphics rendering commands to GPU 120 (e.g., via graphics driver 115) to cause GPU 120 to perform some or all of the rendering of the graphics data. In some examples, the graphics data to be rendered may include a list of graphics primitives, such as points, lines, triangles, quadrilaterals, triangle strips, etc.
[0050] GPU 120 may be configured to perform graphics operations to render one or more graphics primitives to a display. Thus, when one of the software applications executing on processing unit 127 requires graphics processing, processing unit 127 may provide graphics commands and graphics data to GPU 120 for rendering to a display. Graphics data may include, for example, drawing commands, state information, primitive information, texture information, etc. In some cases, GPU 120 may be constructed with a highly parallel structure that provides more efficient processing of complex graphics-related operations than processing unit 127. For example, GPU 120 may include multiple processing elements that are configured to operate on multiple vertices or pixels in a parallel manner.
[0051] GPU 120 may be directly coupled to GMEM 121. In other words, GPU 120 may process data locally using local memory rather than off-chip memory. This allows GPU 120 to operate in a more efficient manner by eliminating the need for GPU 120 to read and write data via, for example, a shared bus, which may experience a large amount of bus traffic. GMEM 121 may include one or more volatile or non-volatile memories or storage devices, such as random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), and one or more registers.
[0052] GMEM 121 may also be directly coupled to at least a density volume processor 123 and a second processor (e.g., a light distribution processor 125). The density volume processor 123 may be configured to convert particle data representing each dynamic particle into a density volume representing a density distribution of the individual dynamic particles distributed in 3D space. The light distribution processor 125 may be configured to pre-calculate a light distribution within the density volume that uses light rays from a light source to represent light values for each grid point within the density volume. In some aspects, the grid points within the density volume may correspond to fixed reference positions. In some aspects, the processor that performs the above functions may be a general purpose processor (e.g., a CPU).
[0053] The density volume processor 123 may be a CPU, a GPU, a general purpose GPU (GPGPU), or any other processing unit that may be configured to perform graphics processing. In some examples, the density volume processor 123 may be integrated into the motherboard of the device 104. In some examples, the density volume processor 123 may be present on a graphics card installed in a port of the motherboard of the device 104, or may be otherwise incorporated in a peripheral device configured to interoperate with the device 104. The density volume processor 123 may include one or more processors, such as one or more microprocessors, GPUs, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is partially implemented in software, the density volume processor 123 may store the instructions of the software in a suitable non-transitory computer-readable storage medium, such as an internal memory 121, and may use one or more processors to execute the instructions in hardware to perform the technology of the present disclosure. Any of the foregoing, including hardware, software, a combination of hardware and software, etc., may be considered to be one or more processors.
[0054] The light distribution processor 125 may be a CPU, a GPU, a general-purpose GPU, or any other processing unit that may be configured to perform graphics processing. In some examples, the light distribution processor 125 may be integrated into the motherboard of the device 104. In some examples, the light distribution processor 125 may be present on a graphics card installed in a port of the motherboard of the device 104, or may be otherwise incorporated in a peripheral device configured to interoperate with the device 104. The light distribution processor 125 may include one or more processors, such as one or more microprocessors, GPUs, ASICs, FPGAs, ALUs, DSPs, discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is partially implemented in software, the light distribution processor 125 may store the instructions of the software in a suitable non-transitory computer-readable storage medium, such as an internal memory 121, and may use one or more processors to execute the instructions in hardware to perform the technology of the present disclosure. Any of the foregoing, including hardware, software, a combination of hardware and software, etc., may be considered to be one or more processors.
[0055] Processing unit 127 and / or GPU 120 may store the rendered image data in frame buffer 128, which may be a separate memory or may be allocated in system memory 124. Display processor may retrieve the rendered image data from frame buffer 128 and display the rendered image data on a display.
[0056] The system memory 124 may be a memory in the device, and may be outside the processing unit 127 and the GPU 120, that is, an off-chip chip relative to the processing unit 127, and an off-chip chip relative to the GPU 120. The system memory 124 may store applications executed by the processing unit 127 and the GPU 120. In addition, the system memory 124 may store data on which the executed applications are based for operation, and data generated from the applications.
[0057] System memory 124 may store accessible program modules, instructions, or both that are available for execution by processing unit 127, data for programs executed on processing unit 127, or two or more of the above. For example, system memory 124 may store a window manager application that is used by processing unit 127 to present a graphical user interface (GUI) on a display. In addition, system memory 124 may store application surface data and user applications associated with the applications. As explained in detail below, system memory 124 may act as device memory for GPU 120 and may store data to be operated by GPU 120 and data generated by operations performed by GPU 120. For example, system memory 124 may store any combination of texture caches, depth caches, stencil caches, vertex caches, frame caches, etc.
[0058] Examples of system memory 124 include, but are not limited to, random access memory (RAM), read-only memory (ROM), or electrically erasable programmable read-only memory (EEPROM), or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer or processor. As an example, system memory 124 can be removed from a device and moved to another device. As another example, a storage device substantially similar to system memory 124 can be inserted into a device.
[0059] Figure 1B is a more detailed block diagram illustrating an exemplary content generation system 100 configured to implement one or more techniques of the present disclosure. It should be noted that Figure 1B The content generation system 100 shown may correspond to Figure 1A In this regard, Figure 1B The content generation system 100 includes a processing unit 127, a GPU 120, and a system memory 124.
[0060] As further shown, the content generation system 100 includes a device 104, which may include one or more components configured to perform one or more techniques of the present disclosure. In the example shown, the device 104 may include a GPU 120, a content encoder / decoder 122, and a system memory 124. In some aspects, the device 104 may include multiple additional components, such as a communication interface 126, a transceiver 132, a receiver 133, and a transmitter 130, and one or more displays 131. Reference to the display 131 may refer to one or more displays 131. For example, the display 131 may include one display or multiple displays. The display 131 may include a first display and a second display. In a further example, the result of the graphics processing may not be displayed on the device, for example, the display 131 may not receive any frame for presentation thereon. Instead, the frame or graphics processing result may be transmitted to another device. In some aspects, this may be referred to as hybrid rendering.
[0061] GPU 120 includes GMEM 121. GPU 120 may be configured to perform graphics processing, for example, in graphics processing pipeline 107. Figure 2 and Figure 6 As described in more detail in , the graphics processing pipeline 107 may include at least generating dynamic particles, converting the dynamic particles into density volumes, pre-calculating light volumes, and then rendering the volumes using ray marching. GPU 120 may be configured to perform these steps in the graphics processing pipeline 107 using at least GMEM 121, a density volume processor 123 coupled to GMEM 121, and a second processor (e.g., a light distribution processor 125) coupled to GMEM 121. Content encoder / decoder 122 may include internal memory 129. In some examples, device 104 may include a display processor, such as processing unit 127, to perform one or more display processing techniques on one or more frames generated by GPU 120 before being presented by one or more displays 131 as described above. Processing unit 127 may be configured to perform display processing. One or more displays 131 may be configured to display or otherwise present frames processed by processing unit 127. In some examples, one or more displays 131 may include one or more of the following: a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, a projection display device, an augmented reality display device, a virtual reality display device, a head-mounted display, or any other type of display device.
[0062] Memory external to GPU 120 and content encoder / decoder 122, such as system memory 124 as described above, may be accessible to GPU 120 and content encoder / decoder 122. For example, GPU 120 and content encoder / decoder 122 may be configured to read and / or write to external memory, such as system memory 124. GPU 120 and content encoder / decoder 122 may be communicatively coupled to system memory 124 via a bus. In some examples, GPU 120 and content encoder / decoder 122 may be communicatively coupled to each other via a bus or a different connection.
[0063] The content encoder / decoder 122 may be configured to receive graphics content from any source, such as the system memory 124 and / or the communication interface 126. The system memory 124 may be configured to store the received encoded or decoded graphics content. The content encoder / decoder 122 may be configured to receive the encoded or decoded graphics content in the form of encoded pixel data, such as from the system memory 124 and / or the communication interface 126. The content encoder / decoder 122 may be configured to encode or decode any graphics content.
[0064] According to some examples, GMEM 121 or system memory 124 may be a non-transitory computer-readable storage medium. The term "non-transitory" may mean that the storage medium is not embodied in a carrier wave or propagating signal. However, the term "non-transitory" should not be interpreted as GMEM 121 or system memory 124 being non-removable or that its contents are static. As an example, system memory 124 may be removed from device 104 and moved to another device. As another example, system memory 124 cannot be removed from device 104.
[0065] The GPU (or processing circuit) may be configured to perform graphics processing according to the exemplary techniques described herein. In some examples, the GPU 120 may be integrated into the motherboard of the device 104. In some examples, the GPU 120 may be present on a graphics card installed in a port of the motherboard of the device 104, or may be otherwise incorporated in a peripheral device configured to interoperate with the device 104. The GPU 120 may include one or more processors, such as one or more microprocessors, GPUs, ASICs, FPGAs, ALUs, DSPs, discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is partially implemented in software, the GPU 120 may store the instructions of the software in a suitable non-temporary computer-readable storage medium, and may use one or more processors to execute the instructions in hardware to perform the technology of the present disclosure. Any of the aforementioned hardware, software, a combination of hardware and software, etc. may be considered to be one or more processors.
[0066] The content encoder / decoder 122 may be any processing unit configured to perform content encoding / decoding. In some examples, the content encoder / decoder 122 may be integrated into the mainboard of the device 104. The content encoder / decoder 122 may include one or more processors, such as one or more microprocessors, ASICs, FPGAs, ALUs, DSPs, video processors, discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is partially implemented in software, the content encoder / decoder 122 may store the instructions of the software in a suitable non-temporary computer-readable storage medium, such as an internal memory 129, and may use one or more processors to execute the instructions in hardware to perform the technology of the present disclosure. Any of the foregoing including hardware, software, a combination of hardware and software, etc. may be considered to be one or more processors.
[0067] In some aspects, the content generation system 100 may include a communication interface 126. The communication interface 126 may include a receiver 133 and a transmitter 130. The receiver 133 may be configured to perform any receiving functions described herein with respect to the device 104. In addition, the receiver 133 may be configured to receive information from another device, such as eye or head position information, rendering commands, or position information. The transmitter 130 may be configured to perform any sending functions described herein with respect to the device 104. For example, the transmitter 130 may be configured to send information to another device, which may include a request for content. The receiver 133 and the transmitter 130 may be combined into a transceiver 132. In such an example, the transceiver 132 may be configured to perform any receiving functions and / or sending functions described herein with respect to the device 104.
[0068] Refer again Figure 1B In some aspects, the processing unit 127 may include a control component 198 configured to control a processor (including a CPU or a GPU) or a general-purpose processor to perform real-time volume rendering of high-density particles. In addition, the control component 198 may be configured to convert particle data representing each dynamic particle into a density volume representing a density distribution of dynamic particles distributed in a three-dimensional (3D) space, pre-calculate a light distribution within the density volume, the light distribution uses light marching from a light source to represent the light value of each grid point within the density volume, render the dynamic particles in real time by calculating pixel color values, and output a representation based on the rendered dynamic particles. The pixel color value is determined using (i) light marching toward the viewpoint position, (ii) the density volume, and (iii) the light distribution.
[0069] As described herein, a device (e.g., device 104) may refer to any device, apparatus, or system configured to perform one or more of the techniques described herein. For example, a device may be a server, a base station, a user device, a client device, a workstation, an access point, a computer (e.g., a personal computer, a desktop computer, a laptop computer, a tablet computer, a computer workstation, or a mainframe computer), a terminal product, a device, a phone, a smart phone, a server, a video game platform or console, a handheld device (e.g., a portable video game device or a personal digital assistant (PDA)), a wearable computing device (e.g., a smart watch, an augmented reality device, or a virtual reality device), a non-wearable device, a display or display device, a television, a television set-top box, an intermediate network device, a digital media player, a video streaming device, a content streaming device, an in-vehicle computer, any mobile device, any device configured to generate graphical content, or any device configured to perform one or more of the techniques described herein. The processing herein may be described as being performed by a specific component (e.g., a GPU), but, in further embodiments, other components (e.g., a CPU) may be used to perform, which is consistent with the disclosed embodiments.
[0070] Figure 2 An example diagram of graphics processing according to one or more techniques of the present disclosure is shown. Specifically, Figure 2 An example diagram of a graphics processing pipeline 200 for real-time volume rendering of dynamic particles is shown. Figure 2 As shown, the graphics processing pipeline 200 includes a dynamic particle generation stage 201 , a conversion stage 203 , a pre-calculation stage 205 , and a rendering stage 207 .
[0071] First, the graphics processing pipeline 200 begins to generate dynamic snow particles 201 according to a physics-based simulation 201. A physics-based modeling and simulation system is configured to map natural phenomena into a computer program. Physics simulations are used in movies and video games to animate various phenomena, such as explosions, car crashes, water, cloth, snow, and so on.
[0072] Next, the graphics processing pipeline 200 includes converting the dynamic particles into density volumes 203. The density volume represents the density distribution of the corresponding dynamic particles distributed in 3D space. For example, a simulated translucent substance (e.g., snow, ash, powder, etc.) can be represented as a particle with a radius and a position in 3D space. Given the particle data, the particles can then be converted into density values (or volume textures) representing the density of the substance in a 3D grid. For each particle, the density contribution to each grid point within the particle radius can be calculated and accumulated to obtain a final density distribution in space.
[0073] The graphics processing pipeline 200 then includes pre-computing a light distribution within the density volume 205. The light distribution uses light marches from the light source to represent the light value of each grid point within the density volume. The grid points within the density volume may correspond to fixed reference positions. To speed up rendering, the light distribution may be pre-computed within the volume and used later in rendering. In the context of graphics rendering in video games, the light distribution may be pre-computed because the position of the light source is known ahead of time. If the density volume or the light source changes, the light distribution may be re-computed.
[0074] In some ways, you can think of the light distribution as another volume texture. For each grid point, a light value is calculated using ray marching. Figure 3A and Figure 3B Different methods of representing density volumes and light distributions are described. In addition, reference will be made to Figure 4 Several different methods of pre-computing light distribution are described in more detail.
[0075] As described herein, the graphics processing pipeline 200 can include pre-computing the light distribution in several different ways. In a first aspect, the most straightforward approach is to pre-compute the light distribution by creating a volume texture with the same resolution as the density volume and using ray marching to compute the light at each voxel in parallel. In a second aspect, the resolution of the light volume can be lower than the resolution of the density volume and then interpolated to obtain the light value at the queried location. In a third aspect, as described below, Figure 4 As shown, we can compute the light volume using a slice-by-slice technique instead of computing the light at each voxel in parallel.
[0076] The graphics processing pipeline 200 then includes rendering the dynamic particles in real time using ray marching 207. Rendering is performed by calculating pixel color values, which are determined using ray marching towards the viewpoint position, the density volume, and the pre-computed light distribution. As described above, ray marching is a computer graphics technique that emits light through a certain volume of medium. Samples are then taken at fixed intervals along the path to calculate the interaction between the light and the medium, and the calculations are accumulated along the rays to obtain the display parameters of the volume. Specifically, each pixel can emit light in the direction of the camera, and samples can be taken on the ray segments that intersect the density volume to determine the display parameters of the pixel. The light can then be retrieved from the pre-computed light volume for each sample. The scattered radiation values are then calculated and accumulated to generate the final pixel color value for display.
[0077] Advantages of the graphics processing pipeline 200 include simulating high-density semi-transparent natural materials, such as snow, ash, or powder, using at least real-time volume rendering. Related real-time volume rendering techniques for low-density media (such as volumetric clouds and volumetric fog) are already used in game engines, but cannot be directly applied to high-density natural materials because they cannot be efficiently rendered by large ray marching steps.
[0078] Until now, accurate physically based algorithms for realistically simulating and rendering high density natural materials have been difficult to use for real-time applications because the computational cost was too high. Due to the rapid development of GPU computing power, real-time physical simulation and rendering of these effects are only now becoming feasible. In addition, the graphics processing pipeline 200 can create realistic particle results for high density materials that are more accurate in shadows and volumes compared to related snow rendering techniques such as using sprites or polygon meshes with subsurface scattering materials.
[0079] Figure 3A An example diagram showing density and light distribution represented as axis-aligned bounding boxes according to one or more techniques of the present disclosure is shown. Example 300a represents density and light distribution in 3D using a 3D box aligned with a Cartesian coordinate system (e.g., an axis-aligned bounding box). This method is a fast and direct way to represent density and light, but has some disadvantages in that objects in the mesh near the camera may appear blurry because mesh points near the camera may be too far apart on the screen due to perspective projection. This may result in blurry visual effects for particles close to the camera.
[0080] Figure 3B An example diagram showing density and light distribution represented as a froxel volume according to one or more techniques of the present disclosure is shown. Example 300b uses a "froxel volume" to represent density and light distribution in 3D. Froxel is a combination of the words "Frustum" and "voxel," which means that the voxel grid is arranged in a frustum shape that is aligned with the camera frustum and moves with the camera. A frustum shape is a portion of a solid, such as a cone or a truncated cone. Frustums are commonly used in game engines to render objects that are inside the camera frustum. The camera frustum represents the camera's field of view. The benefit of a froxel is that the grid is denser in areas near the camera, which produces a higher resolution than a 3D box approach with the same number of voxels (e.g., as in Figure 3A Sharper visual results.
[0081] The comparison between using 3D boxes to represent density and light distribution in 3D and using Froxel to represent density and light distribution in 3D to render a scene with snow is shown below.
[0082] 3D Frame Froxel Time per frame (milliseconds) 39.41 29.78 Frame rate (frames per second (FPS)) 25.37 33.58
[0083] Table 1 Performance comparison between Froxel and AABB
[0084] and Figure 3A Compared to the 3D box method explained in , using froxel volumes reduces the number of voxels required to achieve the same visual quality. Therefore, the froxel volume method reduces computational cost and increases frame rate.
[0085] Figure 4 An example of slice-by-slice light pre-calculation according to one or more techniques of the present disclosure is shown. Slice-by-slice light pre-calculation is an effective method for calculating light distribution. The general idea of the slice-by-slice technique is that for each slice, the light calculation depends on the result of the previous slice.
[0086] As shown in example 400, the first step includes determining light precalculation of an initial slice of a plurality of volume slices 401, propagating the result of light precalculation of the initial slice when determining light precalculation of a second volume slice 403, propagating the result of light precalculation of the second slice when determining light precalculation of a third volume slice 405, and so on, until propagating the result of light precalculation from the second slice to the last slice when determining light precalculation of the last volume slice (407). In this way, slice-by-slice light precalculation avoids redundant light emission and provides faster calculation compared to calculating all voxels in parallel.
[0087] Based on the comparison between unlit precomputation, per-voxel precomputation, and per-tile precomputation, per-tile precomputation can greatly outperform unlit precomputation and per-voxel precomputation. Example comparison results between the three precomputation methods are shown below:
[0088] Matte Precomputation Precompute each voxel Precompute slice by slice Light pre-calculation time (ms) 0 205.55 29.70 Total rendering time (ms) 149.49 208.92 33.07
[0089] Table 2 Computational cost of different methods for handling light calculations
[0090] Figure 5 Flowcharts of various example methods for graphics processing according to one or more techniques of the present disclosure are shown. As described above, method 500 can be performed by a device such as control component 198. In some embodiments, method 500 is performed by processing logic, which includes hardware, firmware, software, or a combination thereof. In some embodiments, method 500 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., a memory). Method 500 includes volume rendering of dynamic particles in real-time applications.
[0091] At block 502, method 500 includes generating particle data representing simulated particles composed of a simulated substance by a physics-based simulation that simulates a natural phenomenon. The generated particles include simulated particle motion in 3D space. In some aspects, the simulated substance is snow. In some aspects, the simulated substance is ash. In some aspects, the simulated substance is powder. In some aspects, the simulated substance has a translucent property or is a translucent substance. For example, in Figure 2 In the context of FIG. 2 , a graphics processing pipeline 200 includes generating dynamic particles 201 using a physics-based simulation for simulating natural phenomena.
[0092] At block 504, method 500 includes converting particle data representing each dynamic particle into a density volume representing a density distribution of the corresponding dynamic particle distributed in a three-dimensional (3D) space. In some aspects, the conversion includes: determining a density value contribution of each grid point within a particle radius based on the particle data, and adding the density value contributions to obtain a density distribution of the dynamic particles distributed in the 3D space. For example, in Figure 2 In the background, the graphics processing pipeline 200 includes converting particle data into density volumes 203.
[0093] In some aspects, density volumes and light distributions are represented in a 3D frame aligned with Cartesian coordinate axes. Figure 3A In the context of , example 300a uses 3D axis-aligned bounding boxes to represent density volumes and light distribution.
[0094] In some aspects, density volumes and light distribution are represented using froxel volumes aligned with the camera position. For example, in Figure 3B In the context of , Example 300b uses froxels to represent density volumes and light distribution.
[0095] At block 506, method 500 includes pre-computing a light distribution within the density volume, the light distribution using ray marching from a light source to represent a light value at each grid point within the density volume. In some aspects, pre-computing the light distribution includes generating a light distribution having the same resolution as the density distribution of the dynamic particles. Using ray marching, light values at each voxel of a corresponding grid point within the density volume are determined in parallel. For example, Figure 2 In the background, the graphics processing pipeline 200 includes a pre-computed light distribution 205.
[0096] In some aspects, precomputation includes generating a light distribution at a lower resolution than a density distribution of dynamic particles, and interpolating the generated light distribution. This aspect is based on the idea that because the light distribution is generally smooth, the light volume can have a lower resolution than the density volume without severely affecting visual quality.
[0097] In some aspects, the pre-calculation includes generating a plurality of volume slices of the density volume, determining a light distribution result for an initial volume slice of the plurality of volume slices, and for each additional volume slice after the initial volume slice, determining a light distribution result for the corresponding additional volume slice by performing ray marching based on the light distribution result from the previous volume slice. In addition, using a low-resolution light volume also greatly reduces computational cost and increases frame rate. For example, in Figure 4 In the context of , embodiment 400 describes generating a light distribution using a piece-by-piece technique.
[0098] At block 508, method 500 includes rendering dynamic particles in real time by using pixel color values determined by ray marching toward the viewpoint position, the density volume, and the light distribution. In some aspects, the pixel color values are calculated by generating a ray for each pixel based on the camera position, sampling the generated ray at a portion intersecting the density volume, retrieving a light value based on the light distribution at each sample, and calculating a diffuse radiation value at each sample based on the retrieved light value and calculating the pixel color value based on the diffuse radiation value. For example, Figure 2 In the background of FIG. 2 , the graphics processing pipeline 200 includes using ray marching 207 to render dynamic particles in real time.
[0099] At block 510 , method 500 includes outputting a rendering-based representation of dynamic particles.
[0100] Figure 6 A flow chart of an example method 600 for graphics processing using dual processing circuits (e.g., GPUs) in accordance with one or more techniques of the present disclosure is shown. The method 600 can be performed by a device such as the control component 198 as described above. In some embodiments, the method 600 is performed by processing logic that includes hardware, firmware, software, or a combination thereof. In some embodiments, the method 600 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., a memory). The method 600 describes volume rendering of dynamic particles using dual processing circuits in a real-time application.
[0101] In some aspects, method 600 includes generating particle data representing simulated particles composed of a simulated substance through a physics-based simulation of a natural phenomenon. The generated particle data includes the movement of the simulated particles in 3D space. In some aspects, the simulated substance is snow. In some aspects, the simulated substance is ash. In some aspects, the simulated substance is powder. In some aspects, the simulated substance has a translucent property or is a translucent substance. For example, in Figure 2 In the context of FIG. 1 , a graphics processing pipeline 200 includes dynamic particles 201 that generate simulated matter through a physically based simulation of natural phenomena.
[0102] At block 602, method 600 includes controlling a first processing circuit to convert particle data representing each dynamic particle of a first frame into a density volume representing a density distribution of the corresponding dynamic particles distributed in a three-dimensional (3D) space. Figure 2 In the background, the graphics processing pipeline 200 includes converting particle data into density volumes 203.
[0103] At block 604, method 600 includes controlling a first processing circuit to pre-compute a light distribution within a density volume, the light distribution using light marches from a light source to represent light values at each grid point within the density volume. The grid points within the density volume correspond to fixed reference locations. For example, at Figure 2 In the background, the graphics processing pipeline 200 includes a pre-computed light distribution 205.
[0104] After block 604, blocks 606 and 608 are performed in parallel.
[0105] At block 606, method 600 includes copying the light distribution of the first frame from the first processing circuit to the second processing circuit. In some aspects, copying the light distribution of the first frame from the first processing circuit to the second processing circuit is performed in parallel with the first processing circuit initiating pre-processing of the second frame, and is performed before the second processing circuit performs volume rendering of the first frame in real time. In some aspects, block 606 includes copying the light distribution and density volume of the first frame from the first processing circuit to the second processing circuit. In other aspects, block 606 includes copying the light distribution of the first frame to the second processing circuit and generating the density volume of the first frame in the second processing circuit so that the density volume is not copied from the first processing circuit.
[0106] At block 608, method 600 includes: upon completion of pre-calculation of the first frame in the first processing circuit, controlling the first processing circuit to convert particle data of the second frame into a density volume. Figure 2 In the background of FIG. 6 , the graphics processing pipeline 200 includes converting particle data into density volumes 203. In other words, block 608 serves as the beginning of a new process 600, wherein the second frame is now the first frame.
[0107] At block 610, method 600 includes controlling a second processing circuit to render dynamic particles in a first frame in real time by calculating pixel color values determined using ray marching toward a viewpoint position, density volumes, and light distribution copied from the first processing circuit. Figure 2 In the background, the graphics processing pipeline 200 includes using ray marching to render dynamic particles 207 in real time.
[0108] At block 612 , method 600 includes outputting a representation of dynamic particles in the rendered first frame.
[0109] The specific interactions between the steps performed by the first processing circuit and the second processing circuit when performing real-time volume rendering of dynamic particles are described below. Figure 7B Described in more detail in .
[0110] Fig. 7A An example diagram of a graphics processing pipeline using a single processing circuit (e.g., a GPU) according to one or more techniques of the present disclosure is shown. Example 700a shows the use of a single GPU 701 to perform real-time volume rendering of dynamic particles. As used herein, splashing refers to the step of converting particle data into a density volume representing the density distribution of dynamic particles distributed in 3D space. Rays refer to the step of pre-calculating the light distribution within the density volume, which represents the light value of each grid point within the density volume. Volume rendering refers to real-time rendering of dynamic particles by calculating pixel color values, which are determined using ray marching relative to the camera position, the density volume, and the light distribution, while other rendering refers to other rendering processes, such as object rendering, shading, ray tracing, ray casting, refraction, texture mapping, etc. using substances other than volume substances. As used herein, the total time to render a single GPU frame may require a total of 25.16 milliseconds.
[0111] Figure 7B A schematic diagram of a graphics processing pipeline using multiple processing circuits is shown, in accordance with one or more techniques of this disclosure.
[0112] To further speed up the computation, multiple processing circuits (e.g., GPUs) may be used in the rendering process. Example 700b illustrates the use of dual GPUs 703 and 705 to perform real-time volume rendering of dynamic particles. In example 700b, the vertical dashed line is aligned with the second GPU 705 to illustrate the total time for rendering a single GPU frame by the second GPU. Example 700b, which illustrates the use of two GPUs to perform real-time volume rendering of dynamic particles, will significantly reduce the time to render a single GPU frame compared to using a single GPU as shown in example 700a.
[0113] Example 700b shows that the first GPU 703 converts the particle data of the first frame into a density volume representing the density of dynamic particles distributed in a three-dimensional (3D) space, and pre-calculates the light distribution within the density volume, which uses light marches relative to the light source to represent the light value of each grid point within the density volume in the first frame. Next, the light distribution of the first frame is copied from the first GPU to the second GPU. The copying process is performed in parallel with other steps performed by the first GPU and the second GPU, so that when the second GPU performs preliminary rendering of the first frame, the first GPU has already moved to the second frame. In addition, the second GPU begins converting the particle data of the first frame into a density volume (such as Figure 7B), at this time the first GPU is pre-calculating the light distribution of the first frame. Therefore, once the light distribution of the first frame is copied to the second GPU, the second GPU is ready to perform rendering based on the light distribution and density volume of the first frame. In this way, different steps can be performed simultaneously on different GPUs, and the frame rate is increased by 30%.
[0114] Example results are shown below for performance between using a single GPU and dual GPUs to perform real-time volume rendering of dynamic particles to render a scene involving snow.
[0115] Single GPU Two GPUs Time per frame (milliseconds) 25.16 19.03 Frame rate (FPS) 39.75 52.55
[0116] Table 3 Performance comparison between single GPU and dual GPU
[0117] The subject matter described herein can be implemented to achieve one or more benefits or advantages. For example, the technology disclosed herein implements a method for real-time volume rendering of dynamic particles (e.g., snow). Therefore, realistic simulations of high-density natural materials can be volume rendered in real-time applications (e.g., video games). In addition, compared with related real-time technologies, the technology disclosed herein produces more realistic shadows and more effectively depicts high-density volumes with better visual quality.
[0118] The subject matter described herein can be implemented to achieve one or more benefits or advantages. For example, the described graphics processing techniques can be used by a server, client, GPU, CPU, or some other processor that can perform computer or graphics processing to implement the sharing techniques described herein. Compared with other computer or graphics processing techniques, the technology can also be implemented at a low cost. In addition, the computer or graphics processing techniques herein can improve or accelerate data processing or execution. In addition, the computer or graphics processing techniques herein can improve resource or data utilization and / or resource efficiency.
[0119] According to the present disclosure, the term "or" may be interrupted to "and / or" where the context does not dictate otherwise. Furthermore, while phrases such as "one or more" or "at least one" may have been used for some features disclosed herein but not for other features, features not using such language may be interpreted as having an implied meaning where the context does not dictate otherwise.
[0120] In one or more examples, the functions described herein may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term "processing unit" has been used in this disclosure, such a processing unit may be implemented in hardware (e.g., by a processing circuit), software, firmware, or any combination thereof. If any function, processing unit, technique, or other module described herein is implemented in software, then the function, processing unit, technique, or other module described herein may be stored on a computer-readable medium or transmitted on a computer-readable medium as one or more instructions or codes. A computer-readable medium may include a computer data storage medium or a communication medium, including any medium that facilitates the transfer of a computer program from one place to another. In this manner, a computer-readable medium may generally correspond to (1) a non-temporary tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in this disclosure. As an example and not limitation, such a computer-readable medium may include a RAM, a ROM, an EEPROM, a CD-ROM drive (CD-ROM) or other optical disk storage, a disk storage, or other magnetic storage device. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, wherein disks usually reproduce data magnetically, while discs reproduce data optically with lasers. The above combinations are also included in the scope of computer readable media. A computer program product may include a computer readable medium.
[0121] The code may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), arithmetic logic units (ALUs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the above structures or any other structure suitable for implementing the techniques described herein. Furthermore, these techniques may be fully implemented in one or more circuits or logic elements.
[0122] The technology of the present disclosure can be implemented in a variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs) or IC groups, such as chipsets. Various components, modules, or units are described in the present disclosure to emphasize the functional aspects of devices configured to perform the disclosed technology, but do not necessarily need to be implemented by different hardware units. Instead, as described above, the various units can be combined in any hardware unit, or provided by a collection of interoperable hardware units including one or more processors as described above in combination with appropriate software and / or firmware.
Claims
1. A method for real-time volume rendering of dynamic particles, characterized in that: The method comprises: converting particle data representing each of the dynamic particles into a density volume representing a density distribution of the corresponding dynamic particle distributed in a three-dimensional (3D) space; pre-calculating a light distribution within the density volume, the light distribution using light marches from a light source to represent light values at each grid point within the density volume, the grid points within the density volume corresponding to fixed reference positions; rendering the dynamic particles in real time by calculating pixel color values determined using (i) ray marching toward a viewpoint position, (ii) the density volume, and (iii) the light distribution; and A representation of the dynamic particles based on the rendering is output.
2. The method according to claim 1, characterized in that The method further comprises: The particle data representing simulated particles composed of a simulated substance is generated by physics-based natural phenomenon simulation, wherein the generated particle data includes movement of the simulated particles in the 3D space.
3. The method according to claim 2, characterized in that The simulated substance is snow.
4. The method according to claim 2, characterized in that: The simulated substance is ash.
5. The method according to claim 2, characterized in that: The simulation substance is a powder.
6. The method according to claim 2, characterized in that The simulated substance is translucent.
7. The method according to claim 1, characterized in that The conversion includes: Determining a density value contribution of each grid point within a particle radius based on the particle data, and The density value contributions are summed to obtain the density distribution of the dynamic particles distributed in the 3D space.
8. The method according to claim 1, characterized in that The density volume and the light distribution are represented in a 3D box aligned with Cartesian coordinate axes.
9. The method according to claim 1, characterized in that: The density volume and the light distribution are represented using a pyramidal voxel volume aligned with the viewpoint position.
10. The method according to claim 1, characterized in that The precalculating the light distribution comprises: The light distribution is generated at the same resolution as the density distribution of the dynamic particles, wherein the ray marching is used to determine in parallel a light value at each voxel of a corresponding grid point within the density volume.
11. The method according to claim 1, characterized in that: The precalculating the light distribution comprises: generating the light distribution having a resolution lower than a resolution of the density distribution of the dynamic particles; and The generated light distribution is interpolated.
12. The method according to claim 1, characterized in that The precalculating the light distribution comprises: generating a plurality of volume slices of the density volume, determining a light distribution result of an initial volume slice of the plurality of volume slices, and For each additional volume slice following the initial volume slice, a light distribution result for the respective additional volume slice is determined by performing the ray marching based on the light distribution result from the previous volume slice.
13. The method according to claim 1, characterized in that The pixel color value is calculated as follows: generating a ray for each pixel relative to the viewpoint position; sampling the generated rays at portions intersecting the density volume; retrieving a light value from said light distribution at each sample; as well as A scattered radiation value at each sample is calculated based on the retrieved light value, and the pixel color value is calculated based on the scattered radiation value.
14. A device for real-time volume rendering of dynamic particles, characterized in that: The device comprises: A processing circuit, the processing circuit being configured to: converting particle data representing each of the dynamic particles into a density volume representing a density distribution of the corresponding dynamic particle distributed in a three-dimensional (3D) space; pre-calculating a light distribution within the density volume, the light distribution using light marches from a light source to represent light values at each grid point within the density volume, the grid points within the density volume corresponding to fixed reference positions; rendering the dynamic particles in real time by calculating pixel color values determined using (i) ray marching toward a viewpoint position, (ii) the density volume, and (iii) the light distribution; and A representation of the dynamic particles based on the rendering is output.
15. The device according to claim 14, characterized in that The processing circuit is further configured to: The particle data representing simulated particles composed of a simulated substance is generated by physics-based natural phenomenon simulation, wherein the generated particle data includes movement of the simulated particles in the 3D space.
16. The device according to claim 15, characterized in that The simulated substance is snow.
17. The device according to claim 14, characterized in that The density volume and the light distribution are represented in a 3D box aligned with Cartesian coordinate axes.
18. A system for real-time volume rendering of dynamic particles, characterized in that: The system comprises: a first processing circuit, a second processing circuit, and A control circuit, the control circuit being configured to: controlling the first processing circuit to convert the particle data representing each of the dynamic particles into a density volume representing a density distribution of the corresponding dynamic particles distributed in a three-dimensional (3D) space; controlling the first processing circuit to precompute a light distribution within the density volume, the light distribution using light marches from a light source to represent light values at each grid point within the density volume, the grid points within the density volume corresponding to fixed reference locations; copying the light distribution of the first frame from the first processing circuit to the second processing circuit, When the pre-calculation with respect to the first frame is completed in the first processing circuit, controlling the first processing circuit to convert the particle data of the second frame into a density volume; controlling a second processing circuit to render the dynamic particles in the first frame in real time by calculating pixel color values determined using (i) ray marching toward a viewpoint position, (ii) the density volume, and (iii) the light distribution copied from the first processing circuit; and A representation of dynamic particles in the first frame based on the rendering is output.
19. The system according to claim 18, characterized in that The copying of the light distribution of the first frame from the first processing circuit to the second processing circuit is performed simultaneously with the first processing circuit initiating pre-processing of the second frame, and is performed before the second processing circuit performs volume rendering of the first frame in real time.
20. The system according to claim 18, characterized in that The system is also configured to: The particle data representing simulated particles composed of a simulated substance is generated by physics-based natural phenomenon simulation, wherein the generated particle data includes movement of the simulated particles in the 3D space.
