Synchronized free cross-pass binning via sub-path interleaving

Abstract

A method of tiled rendering is provided, the method including dividing a frame to be rendered into a number of tiles, receiving commands to execute a number of sub-passes of the tiles, and interleaving execution of the same sub-passes of the number of tiles of the frame. Interleaving the execution of the same sub-passes of the number of tiles includes executing a first sub-pass ordered ahead of a second tile between execution of a first sub-pass ordered ahead of the first tile and execution of a second sub-pass ordered after the first tile. Interleaving may be performed, for example, by executing the multiple sub-passes in a different order than the order in which the commands to execute the multiple sub-passes are stored and issued. Alternatively, interleaving may be performed by executing one or more sub-passes as skip operations such that the multiple sub-passes are executed in the same order.

Description

Technical Field

[0001] (Cross - reference to related applications) This application claims the benefit of U.S. Provisional Patent Application No. 63 / 166,956, filed Mar. 26, 2021, and U.S. Patent Application No. 17 / 565,394, filed Dec. 29, 2021, which are incorporated herein by reference in their entirety as if fully set forth herein.

Background Art

[0002] Graphics processing includes rendering a three - dimensional (3D) scene onto a two - dimensional (2D) screen. The 3D scene is rendered onto the display screen via a graphics pipeline that includes different stages (steps) of processing. Graphics processing commands in a command stream are received (e.g., from an application), and computational tasks are provided (e.g., to an acceleration processing device such as a GPU) for execution of the tasks.

[0003] Graphics are rendered onto the display screen using primitives (e.g., triangles, quadrilaterals, or other geometric shapes). Graphics processing commands include, for example, the number of primitives to be rendered on the display screen, the position of each primitive, and the attributes of each primitive.

[0004] A more detailed understanding can be obtained from the following description given by way of example with the accompanying drawings.

Brief Description of the Drawings

[0005] [Figure 1] It is a block diagram of an exemplary device in which one or more features of the present disclosure can be implemented. [Figure 2] It is a block diagram of the device of FIG. 1 showing additional details. [Figure 3] It is a block diagram showing a graphics processing pipeline according to an example. [Figure 4] This block diagram shows exemplary components of an accelerated processing device in which one or more features of the present disclosure may be implemented. [Figure 5] This is a diagram illustrating a portion of a graphics processing pipeline, showing the ordering of different rendering subpasses as an example. [Figure 6] This timing diagram illustrates the performance benefits of implementing an example of interleaving subpaths of consecutive bins. [Figure 7] This is a timing diagram showing an example for three bins, as shown in Figure 6. [Figure 8] This figure shows a first exemplary mode of interleaving subpaths of consecutive bins. [Figure 9] This figure shows a second exemplary mode in which subpaths of consecutive bins are interleaved. [Figure 10] Figure 9 is a timing diagram showing a second exemplary mode. [Figure 11] This flowchart illustrates an exemplary method of tiled image rendering based on the features of this disclosure. [Modes for carrying out the invention]

[0006] The terms “tile,” “bin,” “tiled,” “binned,” “tiling,” and “binning” are used interchangeably herein.

[0007] A graphics pipeline can be simplified to include a front-end geometry portion and a back-end portion. For example, the front-end geometry portion of the pipeline includes several shader stages (e.g., a vertex shader stage, a hull shader stage, a tesselator stage, a domain shader stage, and a geometry shader stage). During the shader stages, primitives are received as 3D objects and converted into 2D objects to be rendered on a 2D screen. The back-end portion includes a rasterizer stage and a pixel shader stage. During the rasterizer stage, the screen position of each primitive projected onto the 2D screen is determined. For example, during rasterization, an acceleration device (e.g., a GPU) determines which pixel (or subpixel sample) corresponds to each primitive to be rendered on the 2D screen. During the pixel shader stage, values ​​(e.g., brightness and color) are calculated for the pixels corresponding to the primitives.

[0008] Rendering a frame involves performing different tasks in a specific order. For example, frame rendering includes shadow rendering, rendering passes, and post-processing. A rendering pass includes various subpasses (e.g., visibility subpasses) that are performed in a specific order. A subpass is a sequence of draw commands that render to the same rendering target.

[0009] Commands (i.e., instructions) for subpaths are stored in a specific order (e.g., in a command buffer) and issued to the processor (e.g., the GPU scheduler) in a specific order within the buffer, as subsequent subpaths may depend on output data from preceding subpaths before execution. The data resulting from the execution of each subpath is typically stored in a cache (e.g., a render backend (RB) cache). However, the execution of subsequent tasks relies on reading data from another part of memory (i.e., not from the RB cache). Therefore, data from the RB cache is flushed and stored in a lower-level cache (e.g., a level 2 (L2) cache or other lower-level cache) or another part of memory, such as main memory, so that the data can be read to execute subsequent tasks.

[0010] A later-ordered subpath in a command buffer depends on data originating from a earlier-ordered subpath. Before a later-ordered subpath in the command buffer can begin execution (i.e., before it can begin any work), the earlier-ordered subpath in the command buffer must complete execution (i.e., the output data is flushed from the cache and stored in another part of memory). Therefore, in this case, if a later-ordered subpath has to wait for the earlier-ordered subpath to complete, it introduces overhead (e.g., latency) into the processing pipeline. In addition, during this waiting period, the processor is idle until the earlier-ordered subpath completes, resulting in pipeline inefficiencies and increased frame latency.

[0011] Tiling (or binning) is a technique that divides a frame into sections (e.g., tiles or bins) and renders one tile of the frame before rendering another tile of the frame. For example, if a frame (or image) is divided into four equal tiles (i.e., the top-left quadrant, top-right quadrant, bottom-left quadrant, and bottom-right quadrant), the first tile (e.g., the top-left quadrant) is rendered before proceeding to render any of the next tiles. Then, any of the other tiles (e.g., the top-right quadrant) is rendered before proceeding to render any of the last two tiles, and this continues until each tile of the frame has been rendered.

[0012] In traditional tiled (binning) rendering, each bin is rendered independently of the others. When rendering tiles (or bins), different subpasses of the same tile (or bin) must be executed in the order specified in the command buffer so that a subpass ordered later in the same tile can read the data resulting from the execution of a earlier subpass. Therefore, a subpass ordered later in the same tile must wait for an earlier subpass of the same tile to complete, introducing latency into the processing pipeline.

[0013] This application discloses a device and method that utilizes these attributes of tiled (binning) rendering to provide a more efficient processing pipeline by interleaving the same subpass of multiple bins (e.g., the first lighting subpass of bin 1 and the same first lighting subpass of bin 2). A feature of this application is that a later-ordered subpass of the same tile (different from a earlier-ordered subpass) interleaves the execution of the same subpass (i.e., the same subpass type) of a different tile during the period when a later-ordered subpass of the same tile is waiting for the earlier-ordered subpass of the same tile to complete. Thus, overall latency is reduced because these periods when the later-ordered subpass of the same tile is waiting are used to execute one or more subpasses of different tiles. In addition, because the processor executes one or more subpasses of different tiles during this period when the later-ordered subpass of the same tile is waiting, the processor does not idle during this period, resulting in more efficient use of resources. In addition, a feature of this invention is that, in order to further reduce latency, it utilizes the ability to process subpaths in parallel via an accelerated processing device (e.g., a GPU) by (temporarily) overlapping the execution of the same subpath in different bins.

[0014] In one exemplary mode, subpasses of consecutive bins are interleaved by executing the subpasses in a different order than the order in which the subpass commands are stored (e.g., in a command buffer) and issued to the processor (e.g., the GPU's scheduler). In another exemplary mode, subpasses of consecutive bins are interleaved by executing the subpasses as skipped actions so that the subpasses are executed in the same order in which the subpass commands are stored and issued to the processor. The interleaving mode to be implemented and the number of identical interleaved subpasses for multiple bins are determined frame by frame or per set of frames before rendering the frame.

[0015] A method for tiled image rendering is provided, which includes dividing the frame to be rendered into multiple tiles, receiving commands to execute multiple subpasses of the tiles, and interleaving the execution of the same subpasses of multiple tiles of the frame.

[0016] A processing device is provided for rendering tiled images, the processing device comprising memory and a processor. The processor is configured to divide the frame to be rendered into multiple tiles, receive commands to execute multiple subpasses of the tiles, and interleave the execution of the same subpasses of multiple tiles of the frame.

[0017] A non-temporary, computer-readable storage medium is provided, containing instructions for causing a computer to perform a tiled image rendering method. The instructions include dividing a frame to be rendered into multiple tiles, receiving commands to execute multiple subpasses of the tiles, and interleaving the execution of the same subpasses of multiple tiles of the frame.

[0018] Figure 1 is a block diagram of an exemplary device 100 in which one or more features of the present disclosure may be implemented. Device 100 may include, for example, a computer, a game device, a handheld device, a set-top box, a television, a mobile phone, or a tablet computer. Device 100 includes a processor 102, memory 104, storage device 106, one or more input devices 108, and one or more output devices 110. Device 100 may also optionally include an input driver 112 and an output driver 114. It should be understood that device 100 may include additional components not shown in Figure 1.

[0019] In various alternative examples, the processor 102 includes a central processing unit (CPU), a graphics processing unit (GPU), a CPU and a GPU located on the same die, or one or more processor cores, and each processor core can be a CPU or a GPU. In various alternative examples, the memory 104 is located on the same die as the processor 102 or is located separately from the processor 102. The memory 104 includes volatile or non-volatile memory (e.g., random access memory (RAM), dynamic RAM, cache).

[0020] The storage device 106 includes a fixed or removable storage device (e.g., hard disk drive, solid state drive, optical disk, flash drive). The input device 108 includes, but is not limited to, a keyboard, keypad, touch screen, touch pad, detector, microphone, accelerometer, gyroscope, biometric scanner, or a network connection (e.g., a wireless local area network card for transmitting and / or receiving wireless IEEE802 signals). The output device 110 includes, but is not limited to, a display, speaker, printer, tactile feedback device, one or more lights, antenna, or a network connection (e.g., a wireless local area network card for transmitting and / or receiving wireless IEEE802 signals).

[0021] The input driver 112 communicates with the processor 102 and the input device 108, enabling the processor 102 to receive input from the input device 108. The output driver 114 communicates with the processor 102 and the output device 110, enabling the processor 102 to send output to the output device 110. Note that the input driver 112 and the output driver 114 are optional components, and that device 100 operates in the same manner if the input driver 112 and the output driver 114 are not present. The output driver 116 includes an accelerated processing device (APD) 116 coupled to the display device 118. The APD receives computation commands and graphics rendering commands from the processor 102, processes those computation commands and graphics rendering commands, and provides pixel outputs to the display device 118 for display. As will be described in more detail below, the APD 116 includes one or more parallel processing units that perform computations according to the single-instruction-multiple-data (SIMD) paradigm. Thus, although various functions are described herein as being performed by or in conjunction with the APD116, in various alternative examples, functions described as being performed by the APD116 are additionally or alternatively performed by other computing devices having similar capabilities that are not driven by the host processor (e.g., processor 102) and provide graphic output to the display device 118. For example, any processing system that performs processing tasks according to the SIMD paradigm is intended to perform the functions described herein. Alternatively, a computing system that does not perform processing tasks according to the SIMD paradigm is intended to perform the functions described herein.

[0022] Figure 2 is a block diagram of device 100 and shows additional details regarding the execution of processing tasks on APD 116. Processor 102 maintains one or more control logic modules for execution by processor 102 within system memory 104. The control logic modules include operating system 120, kernel mode driver 122, and application 126. These control logic modules control various features of the operation of processor 102 and APD 116. For example, operating system 120 communicates directly with hardware and provides an interface to the hardware for other software executed on processor 102. Kernel mode driver 122 controls the operation of APD 116, for example, by providing an application programming interface (API) to software (such as application 126) executed on processor 102 to access various functions of APD 116. Also, kernel mode driver 122 includes a just-in-time compiler that compiles programs for execution by processing components of APD 116 (such as SIMD unit 138, which will be described in more detail below).

[0023] APD 116 executes commands and programs for selected functions such as graphic operations and non-graphic operations suitable for parallel processing. APD 116 can be used to execute graphics pipeline operations such as pixel operations, geometric calculations, and rendering of images to display device 118 based on commands received from processor 102. Also, APD 116 executes computational processing operations not directly related to graphics operations, such as operations related to video, physical simulation, computational fluid dynamics, or other tasks, based on commands received from processor 102.

[0024] The APD116 includes a compute unit 132 which includes one or more SIMD units 138 that operate in parallel according to the SIMD paradigm at the request of the processor 102. The SIMD paradigm is one in which multiple processing elements share a single program control flow unit and program counter and thus execute the same program, but can execute that program with different data. For example, each SIMD unit 138 includes 16 lanes, and each lane executes the same instruction simultaneously with other lanes in the SIMD unit 138, but can execute that instruction with different data. Lanes can be switched off by prediction if it is not necessary for all lanes to execute a given instruction. Prediction can also be used to execute programs with branched control flows. More specifically, for programs with conditional branches or other instructions in which the control flow is based on calculations performed by individual lanes, prediction of lanes corresponding to control flow paths that are not currently being executed and serial execution of different control flow paths enable arbitrary control flows.

[0025] The basic unit of execution within the computing unit 132 is a work item. Each work item represents a single instance of a program to be executed in parallel on a specific lane. Work items can be executed simultaneously as a "wavefront" on a single SIMD unit 138. One or more wavefronts constitute a "work group," which is a collection of work items designated to execute the same program. A work group can be executed by executing each of the wavefronts that make up the work group. In an alternative example, wavefronts are executed sequentially on a single SIMD unit 138, or partially or fully in parallel on different SIMD units 138. A wavefront can be thought of as the largest collection of work items that can be executed simultaneously on a single SIMD unit 138. Therefore, if a command received from processor 102 indicates that a particular program is parallelized to such an extent that it cannot be executed simultaneously on a single SIMD unit 138, the program is divided into wavefronts that are parallelized on two or more SIMD units 138, or serialized (or both parallelized and serialized as needed) on the same SIMD unit 138. The scheduler 136 performs operations related to scheduling various wavefronts on different computing units 132 and SIMD units 138.

[0026] The parallel processing provided by the computing unit 132 is suitable for graphics-related operations such as pixel value calculation, vertex transformation, and other graphics operations. Therefore, the graphics processing pipeline 134, which accepts graphics processing commands from the processor 102, may, in some cases, provide computing tasks to the computing unit 132 for parallel execution.

[0027] Furthermore, the computing unit 132 is used to perform computing tasks that are not related to graphics or that are not performed as part of the "normal" operation of the graphics processing pipeline 134 (for example, custom operations performed to supplement the operations performed on the graphics processing pipeline 134). An application 126 or other software running on the processor 102 sends a program that defines such computing tasks to the APD 116 for execution.

[0028] The APD116 is configured to implement the features of this disclosure by performing a number of functions. For example, the APD116 is configured to receive an image (i.e., a frame), divide the image into multiple tiles, receive subpath commands via a user-mode driver 200, interleave the same subpaths of multiple bins by executing subpaths in an order different from the order in which the subpath commands are stored and issued to the processor, and interleave the same subpaths of multiple bins by skipping the execution of subpaths so that the subpaths are executed in the same order in which the subpath commands are stored and issued to the processor, and determine the interleave mode and the number of interleaved identical subpaths of multiple bins for each frame or for multiple frames before rendering the frame.

[0029] Figure 3 is a block diagram showing additional details of the graphics processing pipeline 134 shown in Figure 2. The graphics processing pipeline 134 includes stages, each performing a specific function. The stages represent subdivisions of the functionality of the graphics processing pipeline 134. Each stage is implemented either partially or completely as a shader program executed within the programmable processing unit 202, or either partially or completely as fixed-function, non-programmable hardware outside the programmable processing unit 202.

[0030] Stages 302-313 represent the front-end geometry processing portion of the graphics processing pipeline 134. Stages 314-318 represent the back-end pixel processing portion of the graphics processing pipeline 134.

[0031] The input assembler stage 302 reads primitive data from a user-filled buffer (for example, a buffer filled with requests from software executed by processor 102, such as application 126) and assembles that data into primitives for use by the rest of the pipeline. The input assembler stage 302 can generate different types of primitives based on the primitive data contained in the user-filled buffer. The input assembler stage 302 formats the assembled primitives for use by the rest of the pipeline.

[0032] Vertex shader stage 304 processes the vertices of the primitive assembled by input assembler stage 302. Vertex shader stage 304 performs various per-vertex operations, such as transformations, skinning, morphing, and individual vertex illumination. Transformation operations include various operations for transforming the coordinates of vertices. These operations include one or more of modeling transformations, view transformations, projection transformations, perspective splitting, and viewport transformations. In this specification, such transformations are considered to change the coordinates or "position" of the vertex being transformed. Other operations of vertex shader stage 304 change attributes other than coordinates.

[0033] The vertex shader stage 304 is partially or completely implemented as a vertex shader program that runs on one or more computing units 132. The vertex shader program is provided by the processor 102 and is based on a program pre-written by a computer programmer. The driver 122 compiles such a computer program to generate a vertex shader program in a format suitable for execution within the computing units 132.

[0034] The hull shader stage 306, the mosaicizer stage 308, and the domain shader stage 310 work together to perform mosaication (tessellation), which transforms simple primitives into more complex primitives by subdividing them. The hull shader stage 306 generates patches for mosaication based on the input primitives. The mosaicizer stage 308 generates a sample set for the patches. The domain shader stage 310 calculates the vertex positions of the vertices corresponding to the samples in the patch. The hull shader stage 306 and the domain shader stage 310 can be implemented as shader programs executed on the programmable processing unit 202.

[0035] Geometry shader stage 312 performs vertex operations on a primitive basis. Various different types of operations can be performed by geometry shader stage 312, including point sprint expansion, dynamic particle system operations, fur-fin generation, shadow volume generation, single-pass render-to-cubemap, per-primitive material swapping, and per-primitive material setup. In some cases, a shader program running on programmable processing unit 202 performs the operations of geometry shader stage 312.

[0036] The binner stage 313 is the final stage of front-end processing. The binner stage performs a rough rasterization to determine whether the tiles (or bins) of the frame (or image) overlap with primitives. For example, the binner stage includes visibility checks (i.e., visibility passes) and tile walking for primitives determined to be present in each tile. The binner stage 313 is performed by fixed-function hardware.

[0037] As described above, stages 314-318 represent the backend processing of the graphics processing pipeline 134. The rasterizer stage 314 accepts a simple primitive, rasterizes it, and generates it upstream. Rasterization involves determining which screen pixels (or subpixel samples) are covered by a particular primitive. For example, the rasterizer stage 314 converts a primitive (e.g., a triangle) into screen space pixels and tests which pixels are covered by the primitive. Rasterization is performed by fixed-function hardware.

[0038] The pixel shader stage 316 calculates the output value of the screen pixels based on the primitives and rasterization results generated upstream. The pixel shader stage 316 can apply a texture from texture memory. The operation of the pixel shader stage 316 is performed by a shader program executed on the programmable processing unit 202.

[0039] The output merge stage 318 accepts the outputs from the pixel shader stage 316, merges those outputs, performs operations such as z-testing and alpha blending, and determines the final color of the screen pixels.

[0040] Texture data that defines the texture is stored and / or accessed by the texture unit 320. The texture is a bitmap image used at various points in the graphics processing pipeline 134. For example, in some cases, the pixel shader stage 316 applies the texture to pixels to improve the apparent rendering complexity (for example, to provide a more "photorealistic" appearance) without increasing the number of vertices being rendered.

[0041] In some cases, the vertex shader stage 304 modifies primitives to increase complexity by using texture data from the texture unit 320, for example, by generating or modifying vertices for improved aesthetics. For example, the vertex shader stage 304 modifies vertex displacement using a height map stored in the texture unit 320. This type of technique may be used, for example, to produce water with a more realistic appearance compared to a texture used only in the pixel shader stage 316 by changing the position and number of vertices used to render the water. In some cases, the geometry shader stage 312 accesses texture data from the texture unit 320.

[0042] Figure 4 is a block diagram showing exemplary components of the APD116 shown in Figure 2, in which one or more features of the present disclosure may be implemented. As shown in Figure 4, the APD116 includes compute units 132, each having an L1 cache. Each compute unit communicates with a rendering backend (RB) implemented via fixed-function hardware. Output data resulting from the execution of subpaths (e.g., subpath 0, subpath 1, and subpath 2 shown in Figure 5) is stored in the RB cache. Output data from earlier ordered subpaths is flushed from the RB cache and provided to another part of memory, such as the L2 cache 406 shown in Figure 4, and the data is read by later ordered subpaths for execution.

[0043] Figure 5 is a diagram of a portion of a graphics processing pipeline showing an example of ordering different rendering subpasses. Frame rendering includes shadow rendering 502, rendering pass 504, and post-processing 506. For simplicity of explanation, rendering pass 504 in the example shown in Figure 5 includes three subpasses: a geometry buffer (GBuffer) rendering subpass (i.e., subpass 0), a first lighting (light 1) subpass (i.e., subpass 1), and a second lighting subpass (i.e., subpass 2). The number and types of subpasses shown in Figure 5 are merely examples. Features of this disclosure can be implemented using a different number and different types of subpasses than those shown in Figure 5.

[0044] As described above, before a later-ordered subpath in the command buffer (e.g., subpath 1 in Figure 5) can begin execution (i.e., before any work can be started by the later-ordered subpath), a earlier-ordered subpath in the command buffer (e.g., subpath 0 in Figure 5) must finish execution (i.e., the output data is flushed from the cache and stored in another part of memory). Therefore, if subpath 1 has to wait for subpath 0 to complete, it introduces overhead (e.g., latency) into the processing pipeline. In addition, during this waiting period, the processor is idle until subpath 0 completes, resulting in pipeline inefficiency and increased frame latency. Figure 6 shows a timing diagram illustrating the performance benefits of implementing an example of interleaving subpaths in consecutive bins. In particular, Figure 6 shows the performance benefits (e.g., reduced latency) between executing two different subpaths in consecutive bins without interleaving and executing two different subpaths in consecutive bins with interleaving. Figure 6 shows two subpaths and two bins for the sake of simplicity. The features of this disclosure can be implemented using more than two subpaths and more than two bins.

[0045] The timing diagram in the upper part of Figure 6 shows the execution of two different subpasses (i.e., Path 1 and Path 2) of two consecutive bins (i.e., Bin 1 and Bin 2) without interleaving. The timing diagram in the lower part of Figure 6 shows the interleaving of two different subpasses (i.e., Path 1 and Path 2) of Bin 1 and 2, as is characteristic of the present disclosure. For simplicity of explanation, each of the exemplary subpasses shown in Figure 6 includes three stages of the processing pipeline (i.e., the vertex shader (VS) stage, the pixel shader (PS) stage, and the RB stage). However, the pipeline may include additional or other stages, such as the stages shown in pipeline 134 shown in Figure 3.

[0046] As shown in the upper part of Figure 6, the first subpass, path 1 (i.e., a first type of subpass such as geometry buffer rendering subpass 0 in Figure 5), is executed on bin 1, followed by the execution of the second subpass of bin 1, path 2 (i.e., a different second type of subpass such as the first lighting subpass 1 in Figure 5). Then, the first subpass, path 1, is executed on bin 2, followed by the execution of the second subpass, path 2, on bin 2.

[0047] In Figure 6, the curved arrow extending from the RB stage of Path 1 to the PS stage of Path 2 indicates that the execution of the PS stage of the later-ordered subpath depends on the RB stage in the earlier-ordered subpath and the data generated from the flushed (i.e., invalidated, INV) cache (e.g., the RB cache). That is, as shown in the upper part of Figure 6, the later-ordered subpath (Path 2) must wait for the earlier-ordered subpath (Path 1) to complete its execution, resulting in latency in the pipeline. In addition, the processor is idle during this waiting period, which is a waste of resources.

[0048] However, as shown in the lower part of Figure 6, the execution of the pre-ordered subpath in bin 2 (i.e., path 1) is interleaved between the execution of the same pre-ordered subpath in bin 1 (i.e., path 1) and the execution of the pre-ordered subpath in bin 1 (path 2). Therefore, the period in the upper part of Figure 6 when no subpath is executed and the processor is idle is utilized by the execution of the pre-ordered subpath in bin 2 (i.e., path 1), thereby reducing latency and improving overall performance. In addition, as shown in the lower part of Figure 6, latency is further reduced by overlapping (temporarily) portions of the execution of the pre-ordered subpath in bin 1 (i.e., path 1) with portions of the execution of the pre-ordered subpath in bin 2 (i.e., path 1).

[0049] Figure 7 is a timing diagram showing the same procedure as shown in the lower part of Figure 6, but implemented using three passes for three bins. The number of bins and subpasses (shown as passes in Figures 6 and 7) are merely examples for the sake of simplicity. The features of this invention can be implemented for any number of bins and any number of subpasses. In the example shown in Figure 7, the first subpass (pass 1), the second subpass (pass 2), and the third subpass (pass 3) are all distinct from one another. For example, pass 1 corresponds to the geometry buffer rendering subpass, pass 2 corresponds to the first lighting subpass, and pass 3 corresponds to the second lighting subpass. Although not shown for the sake of simplicity, it should be understood that the features of this disclosure, including those described with respect to Figure 7, can be implemented for additional subpasses (i.e., more than three subpasses).

[0050] As shown in both the lower part of the timing diagram in Figure 6 and the timing diagram in Figure 7, the execution of the first subpath ordered ahead of bin 2 (i.e., path 1) is interleaved between the execution of the same first subpath ordered ahead of bin 1 (i.e., path 1) and the execution of the second subpath ordered after bin 1 (i.e., path 2). In addition, as shown in the timing diagram in Figure 7, the execution of the second subpath ordered after bin 2 (i.e., path 2) is interleaved between the execution of the same second subpath ordered after bin 1 (i.e., path 2) and the execution of the third subpath ordered after bin 1 (path 3). Therefore, periods during which the processor would otherwise be idle are used to execute path 2 of bin 2, further reducing latency and improving overall performance. In addition, as shown in the lower part of Figure 6, latency is further reduced by overlapping (temporarily) portions of the execution of path 2 of bin 1 with portions of the execution of path 2 of bin 2. As can be understood, the additional latency is reduced for each additional pass of a frame and for multiple frames, resulting in a significant improvement in overall performance.

[0051] Figures 8 to 11 show examples of different interleaving modes that can be implemented according to the features of the present invention. The number of bins, subpaths (shown as paths in Figures 8 to 10 and as subpaths in Figure 11), and commands shown in Figures 8 to 11 are merely examples for the sake of simplicity. The features of the present invention can be implemented for any number of bins and any number of subpaths using any number of commands.

[0052] Figure 8 shows a first exemplary mode of interleaving subpaths of consecutive bins. Referring to Figure 8, commands issued to the scheduler 136 of the APD 115 (e.g., the GPU) via the user-mode driver 200 (i.e., a command for path 1, a command for path 2, and a command for path 3) are stored in the command buffer 802 in a specific order in which they are executed. When rendering a tile (or bin), different subpaths of the same tile (or bin) are executed in the order specified in the command buffer 802 so that a subpath ordered later in the same tile can read data resulting from the execution of a previously ordered subpath.

[0053] As shown in Figure 8, the execution of subpath 1 of bin 2 is interleaved between the execution of subpath 1 of bin 1 and the execution of subpath 2 of bin 1 while subpath 2 of bin 1 is waiting for subpath 1 of bin 1 to complete its execution. In addition, the execution of subpath 2 of bin 2 is interleaved between the execution of subpath 2 of bin 1 and the execution of subpath 3 of bin 1 while subpath 3 of bin 1 is waiting for subpath 2 of bin 1 to complete its execution. As can be understood, this process may continue for additional paths and bins (not shown in Figure 8).

[0054] In the exemplary interleaved mode shown in Figure 8, the execution order of subpaths differs from the order specified in the command buffer 802. That is, the order specified in the command buffer 802 is subpath 1 (path 1), subpath 2 (path 2), and then subpath 3 (path 3), but the interleaving described above causes the execution of subpaths 1 to 3 to differ from the order specified in the command buffer 802 (for example, subpath 1 for bin 1, followed by subpath 1 for bin 2, and then subpath 2 for bin 1).

[0055] These periods, during which ordered subpaths are waiting after the same bin, are used to execute one or more subpaths on different tiles, thus reducing overall latency. In addition, because the processor is executing one or more subpaths on different tiles during these periods when ordered subpaths are waiting after the same tile, the processor is not idle during these periods, resulting in more efficient use of resources.

[0056] Figures 9 and 10 illustrate a second exemplary mode of interleaving subpaths of consecutive bins. In the second exemplary mode, subpaths of consecutive bins are interleaved by executing the subpaths as skip operations so that the subpaths are executed in the same order specified in the command buffer 802.

[0057] In Figure 9, the execution order of each subpath (i.e., subpath 1, subpath 2, and subpath 3) is shown in matrix form (and vertically aligned with the commands in command buffer 802 at the top of Figure 9) to better illustrate that the subpaths are executed in the same order as specified in command buffer 802. The order in which subpaths and skips are executed is indicated by the corresponding numbers 1 to 12. For example, subpath 1 of bin 1 is executed, then subpaths 2 and 3 of bin 1 are executed as skipped executions, and then subpath 1 of bin 2 is executed.

[0058] In Figure 10, the execution order of each subpath (i.e., subpath 1, subpath 2, and subpath 3) is shown horizontally over time to better illustrate the subpath interleaving. In the second exemplary mode shown in Figures 9-11, there is some latency caused by skipping. However, as shown in Figures 9 and 10, the number of skips (skip operations) decreases over time until there are no more skips.

[0059] Figure 11 is a flowchart illustrating an exemplary method 1100 of tiled image rendering according to the features of the present disclosure. As shown in block 1102 of Figure 11, method 1100 includes dividing a frame to be rendered into a number of tiles. A frame to be rendered may be divided into any number of tiles. Each tile of the frame is rendered before another tile of the frame is rendered. For example, if the frame is divided into four equal tiles (i.e., top-left quadrant, top-right quadrant, bottom-left quadrant, and bottom-right quadrant), the first tile (e.g., top-left quadrant) is rendered before proceeding to render one of the next tiles. Then, one of the other tiles (e.g., top-right quadrant) is rendered before proceeding to render one of the last two tiles, and this continues until each tile of the frame has been rendered.

[0060] As shown in block 1104 of Figure 11, method 1100 includes receiving commands for executing multiple subpaths of a tile. For example, commands for executing multiple subpaths (i.e., paths 1, 2, and 3 shown in Figures 7 and 8) are received via command buffer 802 (shown in Figures 8 and 9) and issued via user-mode driver 200 to scheduler 136 of APD 115 (e.g., GPU), which receives them. The commands are stored in command buffer 802 in a specific order in which they are executed. When rendering a tile (or bin), different subpaths of the same tile (or bin) are executed in the order specified in command buffer 802 so that a subpath ordered later in the same tile can read data resulting from the execution of a previously ordered subpath.

[0061] As shown in block 1106 of Figure 11, method 1100 includes interleaving the execution of the same subpaths of multiple tiles in a frame (e.g., the subpath of the first tile and the same subpath of the second tile). For example, the first subpath ordered before the second tile (e.g., in the order in command buffer 802) is executed between the execution of the first subpath ordered before the first tile and the execution of the second subpath ordered after the first tile.

[0062] Execution of the same subpath across multiple tiles is interleaved, for example, by executing the subpaths in an order different from the order in which the commands for executing the subpaths are stored and issued. Alternatively, execution of the same subpath across multiple tiles is interleaved by skipping the execution of one or more subpaths so that the subpaths are executed in the same order as the commands for executing the subpaths are stored and issued.

[0063] It should be understood that many modifications are possible based on the disclosures herein. Although features and elements are described above in specific combinations, each feature or element can be used alone without other features and elements, or in various combinations with or without other features and elements.

[0064] Various functional units shown in each figure and / or described herein (including, but not limited to, the processor 102, input driver 112, input device 108, output driver 114, output device 110, APD 116, scheduler 136, graphics processing pipeline 134, compute unit 132, and SIMD unit 138) can be implemented as a general-purpose computer, processor, or processor core, or as a program, software, or firmware stored on a non-temporary computer-readable storage medium or another medium executable by a general-purpose computer, processor, or processor core. The methods provided can be implemented in a general-purpose computer, processor, or processor core. Suitable processors include, by example, general-purpose processors, dedicated processors, conventional processors, digital signal processors (DSPs), multiple microprocessors, one or more microprocessors associated with a DSP core, controllers, microcontrollers, application-specific integrated circuits (ASICs), field-programmable gate array (FPGA) circuits, any other type of integrated circuit (IC), and / or state machines. Such processors can be manufactured by configuring a manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediate data (instructions that can be stored in a computer-readable medium), including netlists. The results of such processing can be a mask work, which is used in a subsequent semiconductor manufacturing process to manufacture a processor that implements the features of the present disclosure.

[0065] The methods or flowcharts provided herein can be implemented in computer programs, software, or firmware embedded in non-temporary computer-readable storage media for execution by a general-purpose computer or processor. Examples of non-temporary computer-readable storage media include read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs).

Claims

1. A method for rendering tiled images, Dividing the rendered frame into multiple tiles, Receiving commands to execute multiple subpaths of the aforementioned multiple tiles, The method includes interleaving the execution of the same subpath of multiple tiles in the frame by executing the first subpath of a second tile, which is ordered before the first tile, and executing the second subpath of a second tile, which is ordered after the first tile and is different from the first subpath of the first tile, between these two executions. method.

2. The method further includes interleaving the execution of the same subpath of multiple tiles by executing the multiple subpaths in an order different from the order in which the commands for executing the multiple subpaths are stored and issued. The method according to claim 1.

3. The method further includes interleaving the execution of the same subpath on multiple tiles by executing one or more subpaths as skip operations, such that the multiple subpaths are executed in the same order as the commands for executing the multiple subpaths are stored and issued. The method according to claim 1.

4. Interleaving is determined frame by frame before rendering the frame. The method according to claim 1.

5. The execution of a portion of the first ordered subpath of the first tile overlaps in time with the execution of a portion of the second ordered subpath of the second tile. The method according to claim 1.

6. The portion of the first ordered subpass of the first tile is any of the vertex shader stage, the pixel shader stage and the rendering backend stage, and the portion of the first ordered subpass of the second tile is any other of the vertex shader stage, the pixel shader stage and the rendering backend stage. The method of claim 5.

7. A processing device used for rendering tiled images, Memory and Equipped with a processor, The aforementioned processor, Dividing the rendered frame into multiple tiles, Receiving commands to execute multiple subpaths of the aforementioned multiple tiles, Interleaving the execution of the same subpath for multiple tiles in the frame by executing the first subpath ordered before the first tile and the second subpath ordered after the first tile, which is different from the first subpath ordered before the first tile, between these two executions, by executing the first subpath ordered before the second tile for the second tile. It is configured to do, Processing device.

8. The system further includes a display device for displaying an image rendered by the aforementioned processor. The processing device according to claim 7.

9. The processor is configured to interleave the execution of the same subpath of multiple tiles by executing the multiple subpaths in an order different from the order in which the commands for executing the multiple subpaths are stored and issued. The processing device according to claim 7.

10. The processor is configured to interleave the execution of the same subpath of multiple tiles by executing one or more subpaths as skip operations, so that the multiple subpaths are executed in the same order as the commands for executing the multiple subpaths are stored and issued. The processing device according to claim 7.

11. The aforementioned commands are stored in the command buffer in the order in which they are executed. The processing device according to claim 7.

12. With an additional scheduler, The aforementioned commands are issued to the scheduler in the order in which they are stored in the command buffer. The processing device according to claim 11.

13. The command is issued to the command buffer via the user-mode driver. The processing device according to claim 11.

14. Interleaving is determined frame by frame before rendering the frame. The processing device according to claim 7.

15. The execution of a portion of the first ordered subpath of the first tile overlaps in time with the execution of a portion of the second ordered subpath of the second tile. The processing device according to claim 7.

16. The portion of the first ordered subpass of the first tile is any of the vertex shader stage, the pixel shader stage and the rendering backend stage, and the portion of the first ordered subpass of the second tile is any other of the vertex shader stage, the pixel shader stage and the rendering backend stage. The processing device according to claim 12.

17. A computer-readable storage medium containing instructions for causing a computer to perform a tiled image rendering method, The aforementioned instruction is, Dividing the rendered frame into multiple tiles, Receiving commands to execute multiple subpaths of the aforementioned multiple tiles, The method includes interleaving the execution of the same subpath of multiple tiles in the frame by executing the first subpath of a second tile, which is ordered before the first tile, and executing the second subpath of a second tile, which is ordered after the first tile and is different from the first subpath of the first tile, between these two executions. Computer-readable storage medium.

Images