Graphics Processing

Abstract

The present disclosure relates to graphics processing. When a series of rendering outputs is generated using a graphics processor, the completion status of rendering tasks for different rendering outputs is tracked so that processing tasks for later rendering outputs in the series of outputs can be processed simultaneously with processing tasks for earlier rendering outputs in the series of outputs, while ensuring that any dependencies between the rendering tasks are enforced.

Description

[0001] The present invention relates to performing graphics processing, and in particular to the operation of a graphics processor when performing a series of rendering jobs, eg, to generate one or more outputs, eg, frames, eg, for display.

[0002] Graphics processing is often performed by first breaking the graphics processing (rendering) output to be rendered (such as a frame to be displayed) into a number of similar geometrical basic components to allow graphics processing operations to be performed more easily. These geometrical basic components may often be referred to as graphics "primitives", and such "primitives" are often in the form of simple polygons (such as triangles), points, lines, etc. (or groups thereof).

[0003] Each primitive (e.g., polygon) is defined by and represented as a set of vertices at this stage. Each vertex of a primitive has associated with it a set of data representing that vertex (such as position, color, texture, and other attribute data). This "vertex data" is then used (e.g., when rasterizing and rendering the primitive to which the vertex refers) in order to generate the desired rendering output of the graphics processing system.

[0004] For a given output to be generated by a graphics processing system (e.g., a frame to be displayed), there will typically be a set of vertices defined for the output in question. The primitives to be processed for the output are then indicated as comprising a given vertex in that set of vertices for the graphics processing output being generated. Typically, the total output (e.g., a frame to be generated) will be divided into smaller processing units, called "draw calls." Each draw call will have a corresponding set of vertices defined for it and a corresponding primitive that uses those vertices. For a given frame, there may be, for example, on the order of several thousand draw calls and tens of thousands (or possibly millions) of primitives.

[0005] Once the primitives and their vertices have been generated and defined, they can be processed by a graphics processor in order to generate the desired graphics processing output (render target), such as a frame for display. This basically involves determining which sampling positions in an array of sampling positions associated with the render output area to be processed are covered by the primitives, and then determining the appearance that each sampling position should have (e.g., in terms of its color, etc.) to represent the primitive at that sampling position. These processes are often referred to as rasterization and rendering, respectively. (The term "rasterization" is sometimes used to refer to both the conversion of primitives to sampling positions and rendering. However, in this article, "rasterization" will be used only to refer to the conversion of primitive data into sampling position addresses.)

[0006] (It should be understood that rasterization-based rendering as described above is only one method of producing rendering output, and there are other types of rendering operations that can be implemented by a graphics processor, including, for example, ray tracing or hybrid ray tracing rendering operations.)

[0007] One form of graphics processing uses so-called "tile-based" rendering. In tile-based rendering, a two-dimensional rendering output (i.e., the output of a rendering process, which may be an output frame to be displayed, but may also be an "intermediate" output or other suitable rendering target, as will be explained further below) is rendered into multiple smaller areas, generally referred to as rendering "tiles", which are then (and may be) rendered separately. The rendered tiles are then reassembled to provide a complete rendering output, such as a frame, for display, for example.

[0008] In such arrangements, the rendering output (eg, a frame to be displayed) is typically divided (by area) into rendering tiles of regular size and shape (which are typically, for example, square or rectangular), but this is not necessary and other arrangements are possible.

[0009] (Other terms often used for "tiling" and "tile-based" rendering include "tiling" (rendered tiles are referred to as "chunks") and "bucket" rendering. The terms "tile" and "tiling" will be used hereinafter for convenience, but it should be understood that these terms are intended to encompass all alternative and equivalent terms and techniques in which the rendering output is rendered as multiple smaller areas.)

[0010] Modern graphics processors typically include one or more processing (shader) cores that execute programmable processing stages of a graphics processing pipeline implemented particularly by the graphics processor, often referred to as "shaders."

[0011] Thus, a graphics processor processing (shader) core is a processing unit that performs processing by running a (usually small) program for each "work item" in the output to be generated. In the case of generating graphics output (such as a render target, such as a frame to be displayed), a "work item" for this purpose may be a sample position (e.g. in the case of a fragment shader), but may also be a vertex or ray, for example, depending on the graphics processing (shading) operation in question. In the case of a compute shading operation, each "work item" in the output being generated will be a data instance (item) in the work "space" of the compute shading operation being performed.

[0012] To execute shader programs, the graphics processor will include one or more appropriate execution units (or circuits) for this purpose. An execution unit will include programmable processing circuitry for executing shader programs ("shaders" of the graphics processing pipeline).

[0013] The actual data processing operations performed by the execution unit when executing the shader program are typically performed by the corresponding functional units of the execution unit, which may include (but are not limited to) texture mapping units configured to perform certain texture operations. Thus, the functional units will respond to the instructions in the (shader) program being executed and perform appropriate data processing operations according to its requirements.

[0014] In addition to the programmable execution units (using their associated functional units) that execute shader programs, a graphics processor processing (shader) core may also typically include one or more essentially fixed-function (hardware) stages for implementing certain stages of the graphics processing (rendering) pipeline. These fixed-function stages may be used to handle certain fragment "front-end" processing operations to set up the fragment shader program (e.g., may include primitive list reading, resource allocation, vertex fetching, rasterization, early depth / stencil testing, but various arrangements will be possible, e.g., depending on the specific configuration of the graphics processing pipeline) as well as certain post-shader actions, such as late depth / stencil testing or tile writing out.

[0015] Thus, a graphics processor processing (shader) core is operable and configured to implement an instance of a graphics processing pipeline for processing a given rendering task, wherein the processing (shader) core is operable and configured to load the required data for setting up the shader program and then execute the required fragment shader program to perform the actual rendering operation. The output of the rendering task is then written out accordingly.

[0016] Typically, there may be many parallel processing (shader) cores within a graphics processor, enabling the graphics processor to simultaneously process multiple different rendering tasks in parallel. Therefore, in a tile-based rendering system, the corresponding tasks for processing different tiles may be issued to different processing (shader) cores, allowing the tiles to be rendered in parallel. Therefore, this may provide more efficient graphics processor operation.

[0017] Thus, when a graphics processor receives a command to generate a particular render target (i.e., by executing a rendering pass whose output is a desired render target) (and such a rendering pass may typically include one or more rendering jobs, each rendering job including a corresponding set of rendering tasks to be processed for the desired render target), the command is processed within an appropriate command processing unit (e.g., a command stream front end / job manager) of the graphics processor to identify the rendering job to be executed, and an appropriate (fragment) task iterator accordingly schedules processing of the corresponding set of rendering tasks to be performed for the rendering job of the rendering pass, where the rendering tasks are assigned to available processing (shader) cores for processing.

[0018] Although described above with respect to a single rendering pass, it should be understood that multiple rendering passes may generally be performed as part of generating a single output (eg, a single frame, such as for display).

[0019] For example, a given rendering job within a rendering pass may generate a "final" output (e.g., a frame), e.g., for display. However, in general, multiple rendering jobs and multiple rendering passes may be performed to generate a given final output (frame). Thus, a given rendering job (within a given rendering pass) may, for example, generate an "intermediate" output, such as a "render to texture" output (i.e., a texture), which is then used by a subsequent rendering job (e.g., in the next rendering pass, but it may also be used by another rendering job in the same rendering pass) when generating the final output (e.g., frame). This may provide more efficient graphics processor operation, e.g., compared to simply generating the final output (frame) in its entirety via a single rendering job (rendering pass).

[0020] Similarly, it should be appreciated that most graphics processing applications do not need to generate only a single output (eg, frame) alone, but rather typically need to generate a series of outputs (eg, frames), such as for a continuous display.

[0021] Thus, a graphics processor may typically be operable and configured to perform a series of rendering passes when generating one or more frames, wherein each rendering pass comprises one or more rendering jobs that generate a corresponding (overall) output for that rendering pass. In some cases, different rendering passes in a series of rendering passes being performed may be independent of one another (e.g., different rendering passes are associated with different frames, or specifically write to different (data buffers)). However, it is typically the case that at least some of the different rendering passes in a series of rendering passes being performed are related to one another. For example, this may occur when one rendering pass generates a "render to texture" or other suitable "intermediate" output that is then used by the next rendering pass. Dependencies may also exist between rendering jobs within a particular rendering pass, such as where a "compute" job performs computations based on the rendering output produced by a previous fragment job.

[0022] Therefore, in a given series of rendering jobs to be performed, there may be certain data (processing) dependencies between the rendering jobs, which, if not enforced, may result in certain artifacts in the final rendered output.

[0023] However, the present applicant believes that there is still room for improvement in the operation of a graphics processor when executing a series of rendering jobs, wherein certain data (processing) dependencies may exist between different rendering jobs in the series of rendering jobs being executed.

[0024] According to a first aspect of the present invention, there is provided a method for operating a graphics processor when executing a series of rendering jobs, the graphics processor comprising a set of one or more processing cores, the method comprising:

[0025] When executing a series of rendering jobs:

[0026] wherein each rendering job in the series of rendering jobs includes a respective set of one or more rendering tasks to be processed to obtain an overall output of the rendering job, and wherein at least some processing of the rendering tasks of a later rendering job in the series of rendering jobs may depend on processing of rendering tasks of an earlier rendering job in the series of rendering jobs:

[0027] issuing a rendering task of a first rendering job in the series of rendering jobs to a processing core of a graphics processor for processing; and

[0028] Tracking the completion status of the rendering tasks of the first rendering job;

[0029] The method further includes:

[0030] issuing a rendering task of a second, later rendering job in the series of rendering jobs to a processing core of a graphics processor for simultaneous processing with the rendering task of the first rendering job; and

[0031] Processing of rendering tasks of the second rendering job is controlled based on tracking of completion status of rendering tasks of the first rendering job to enforce a dependency between the first rendering job and the second rendering job.

[0032] According to a second aspect of the present invention, there is provided an image processor, the image processor comprising:

[0033] A collection of one or more processing cores;

[0034] a task issuing circuit operable to control issuing of rendering tasks to a plurality of processing cores for processing; and

[0035] A control circuit operable and configured to:

[0036] When the GPU is performing a series of rendering jobs:

[0037] wherein each rendering job in the series of rendering jobs comprises a respective set of one or more rendering tasks to be processed to obtain an overall output of the rendering job, and wherein processing of rendering tasks of a later rendering job in the series of rendering jobs may depend on processing of rendering tasks of an earlier rendering job in the series of rendering jobs:

[0038] Controls the operation of the graphics processor to:

[0039] The task issuing circuit is permitted to start issuing rendering tasks of a later rendering job in a series of rendering jobs being executed to a processing core of a graphics processor for processing without waiting for all rendering tasks of earlier rendering jobs in the series of rendering jobs being executed to have completed their processing, so that the rendering tasks of the later rendering jobs can be processed simultaneously with the rendering tasks of the earlier rendering jobs;

[0040] The control circuit is further configured as:

[0041] Completion status of rendering tasks of a rendering job is tracked, and processing of rendering tasks of a later rendering job in the series of rendering jobs is controlled using the tracking of the completion status of rendering tasks of an earlier rendering job in the series of rendering jobs being executed to enforce dependencies between different rendering jobs.

[0042] The present invention generally relates to the operation of a graphics processor when executing a series of rendering jobs, particularly where certain data (processing) dependencies may exist between different rendering jobs in the series of rendering jobs being executed.

[0043] For example, a graphics processor may and preferably is operable and configured to perform a series of rendering passes, wherein each rendering pass in the series of rendering passes generates and is associated with a corresponding output. Each rendering pass may then include one or more rendering jobs for processing (a portion of) the corresponding output of the rendering pass.

[0044] However, as described above, the output of a given rendering pass may be a "final" output, such as a frame for display, but may also be an "intermediate" output (e.g., a "render to texture" output) that is then used by a later rendering pass in the series of rendering passes to generate its output. Likewise, processing of a particular "final" output (e.g., a frame) may in some cases reuse (at least some) processing of an earlier "final" output (e.g., a frame). Thus, for different rendering passes, there may be certain data (processing) dependencies that should be enforced between rendering jobs. Likewise, there may be certain data (processing) dependencies between different rendering jobs within the same rendering pass that should be enforced, for example, where a "compute" job is related to output produced by a previous fragment rendering job, for example, so that the "compute" job uses data generated by the fragment rendering job. The present invention relates in particular to situations where there are possible data (processing) dependencies between rendering jobs that may need to be enforced.

[0045] Each rendering job in the series of rendering jobs being executed in the present invention (and may) be subdivided into a corresponding set of rendering tasks, which will be processed to obtain the overall output (render target) of the rendering channel in which the rendering job is being executed. For example, a rendering channel can and typically will include one or more fragment rendering jobs for producing the corresponding rendering output (e.g., frame) of the rendering channel, wherein the rendering tasks within the fragment rendering jobs generate corresponding parts (e.g., regions) of the overall output (frame). However, a particular rendering channel may also include various other types of rendering jobs, such as "compute" jobs for performing calculations on the (same) rendering output produced by the previous fragment rendering job within the same rendering channel (and wherein the rendering tasks of the "compute" jobs may therefore and preferably perform such calculations on certain parts (e.g., regions) of the overall output (render target)). That is, the rendering jobs within a given rendering channel can (and typically do) involve the same overall output (e.g., frame), and therefore the rendering tasks within each rendering job preferably involve rendering operations performed on the same overall output of the rendering channel (e.g., fragment shading in the case of a fragment shading job, or calculations in the case of a "compute" job).

[0046] The graphics processor includes a collection of one or more (preferably multiple) processing (shader) cores that are operable and configured to process these rendering tasks to obtain the overall output (render target) of the rendering job. In a preferred embodiment, at least some of the rendering jobs (and in some cases all) of the rendering jobs include multiple rendering tasks, wherein the multiple rendering tasks of the rendering jobs can be, for example and preferably, executed separately in parallel across multiple processing (shader) cores of the graphics processor. Therefore, each rendering task associated with the corresponding rendering job can and preferably does correspond to a corresponding portion of the overall output (render target) of the rendering job. Therefore, each rendering task preferably corresponds to a corresponding instance of a graphics processing workload that can be (and will be) executed by a corresponding processing (shader) core of the graphics processor.

[0047] Therefore, data (processing) dependencies between rendering tasks of different rendering jobs are typically (and preferably) related to only a portion of the overall output (e.g., the dependencies that can be handled by the present invention are typically data (processing) dependencies related to certain areas of the overall rendering output being generated). Therefore, when a rendering task within a particular rendering job performs processing related to a certain area of ​​the rendering output (e.g., the rendering task writes data to a specific location in the data buffer (render target) corresponding to that area), any rendering task of a later rendering job (if it needs data related to that (same) area of ​​the rendering output) can have data (processing) dependencies on earlier rendering tasks that need to be managed. (On the other hand, a later rendering task may not have any data (processing) dependencies on earlier rendering tasks that are strictly related to different areas of the rendering output.)

[0048] For example, and in particularly preferred embodiments, the output (render target) of each rendering job is subdivided into a plurality of rendering tasks, each rendering task corresponding to a respective "tile" or sub-region into which the overall rendering output (e.g., frame) associated with the rendering job has been subdivided for rendering purposes (i.e., for a tile-based rendering system), such that each rendering task to be processed for a particular rendering job generates a respective portion (e.g., sub-region) of the overall output (render target) associated with the rendering job. Thus, the rendering tasks resulting from the subdivision of a rendering job may, for example and preferably, correspond to the (separate) rendering tiles into which the output (render target) of the rendering job has been subdivided for rendering purposes (and any reference to a rendering "task" herein may be considered in various embodiments to refer to a rendering "tile"). However, the set of suitable rendering tasks to be performed in a given rendering job to generate the corresponding output of that rendering job may be defined by various arrangements.

[0049] Thus, the graphics processing workload within a given rendering job may (and preferably does) involve issuing multiple (separate) rendering tasks (e.g., tiles) for processing, each of which generates a respective rendering output (which is a respective portion of the overall rendering output associated with the rendering job), which may then be processed (e.g., in parallel) by respective processing (shader) cores of the graphics processor. The respective outputs of the respective rendering tasks may then, and preferably are, be stored in a suitable intermediate buffer (e.g., a tile buffer in a tile-based rendering system), and once processing of the rendering task is completed, the (final) output (or outputs) of the rendering task may be written out accordingly, for example, to a respective data buffer (render target) to which the rendering job is written, at which point processing of the rendering task is complete.

[0050] A rendering job is complete once all of its rendering tasks have completed their processing and the corresponding output has been written out appropriately.

[0051] The graphics processor then continues to execute the next rendering job in the series of rendering jobs being executed and additional rendering jobs in this manner until all rendering jobs in the series of rendering jobs are completed (or until an application requiring graphics processing stops the current series of rendering jobs, for example, to generate a different output sequence (e.g., frames)).

[0052] Thus, the graphics processing workload of each rendering pass in a series of rendering passes being executed is typically similar, wherein each rendering pass typically includes a similar set of rendering jobs, and each rendering job includes a similar set of rendering tasks to be executed (but different rendering passes produce different outputs (render targets)). The rendering tasks to be executed for the rendering jobs can and preferably are issued for processing in a set order (e.g., a predefined order), but the processing of the rendering tasks is (and can be) appropriately allocated to the available processing (shader) cores of the graphics processor so that the processing of at least some of the rendering tasks is performed simultaneously (e.g., in parallel) across the available processing (shader) cores. Generally speaking, the completion time of different rendering tasks can be different, and therefore the rendering tasks of different rendering jobs (of different rendering passes) can therefore be completed out of order.

[0053] Therefore, assigning rendering tasks to processing cores can and preferably usually attempts to balance processing between the available processing (shader) cores of the graphics processor and ensure higher utilization of the available processing (shader) cores. Therefore, the appropriate task issuance circuit (task iterator) of the graphics processor can appropriately control the scheduling and allocation of rendering tasks to processing (shader) cores in order to try to assign rendering tasks to available processing (shader) cores, thereby improving the utilization of the processing (shader) cores.

[0054] However, the present applicants have realised that within a particular rendering pass, or rendering job within a rendering pass, there may still be situations where the utilisation of the available processing (shader) cores is relatively low.

[0055] This is particularly true, for example, near the end of a rendering job, where the rendering job may have relatively few rendering tasks remaining to be completed compared to the number of available processing (shader) cores. A similar situation may also exist at the beginning of a rendering job, where the workload may be concentrated on the fragment "front end" without fully utilizing the processing (shader) cores. Other situations where processing (shader) core utilization may be low may include, for example, situations where the rendering job itself is relatively small and does not include enough rendering tasks to fully utilize all available processing (shader) cores.

[0056] Thus, a "rendering job" is an independent block of processing work, including one or more rendering tasks associated with a specific rendering output, and certain data (processing) dependencies may exist between different rendering jobs in a series of rendering jobs being executed. However, a rendering job may be part of a larger "rendering pass" including multiple different types of rendering jobs associated with the same rendering output. As described above, data (processing) dependencies between rendering jobs may therefore be data (processing) dependencies that exist between rendering jobs for different rendering passes, but may also be data (processing) dependencies between rendering jobs within the same rendering pass.

[0057] Thus, in some more conventional graphics processing arrangements, strict (hard) processing barriers are enforced between such rendering jobs, such that rendering tasks of a later processing job are not released for processing until all rendering tasks of earlier processing jobs on which the rendering tasks of the later processing job depend have completed their processing, for example where there are any possible data (processing) dependencies between the rendering jobs that need to be enforced, such as where a rendering job uses data generated by one or more previous rendering jobs.

[0058] Therefore, if the rendering jobs in the series of rendering jobs are executed strictly one after another, i.e., there are strict (hard) processing barriers between different rendering jobs, and therefore there is no overlap between the processing of different rendering jobs in the processing (shader) cores (as may be done in some more traditional graphics processing devices), then obvious instances of relatively low processing (shader) core utilization may occur, for example and specifically, where the graphics processor is completing processing of an earlier rendering job, but cannot yet start issuing and processing rendering tasks of a subsequent rendering job due to the strict processing barriers between the rendering jobs.

[0059] In order to solve this problem and provide improved graphics processor performance, the present invention therefore provides a novel mechanism to safely allow the processing of separate rendering jobs to overlap in the graphics processor processing (shader) cores, that is, to enable the rendering tasks of different rendering jobs to run simultaneously. For example, in some embodiments of the present invention, the rendering tasks of different rendering jobs can run simultaneously (e.g., in parallel) across different processing (shader) cores of the graphics processor.

[0060] In some embodiments, the graphics processor may be configured so that a particular processing (shader) core can only perform rendering tasks in a single fragment context (i.e., rendering job). In this case, when separate rendering jobs (possibly for separate rendering passes) will overlap in the manner of the present invention, rendering tasks from different rendering jobs can (and will) be issued to different processing (shader) cores for processing. However, in general, rendering tasks of different rendering jobs may also be run simultaneously within the same processing (shader) core, for example, as long as the graphics processor is appropriately configured with a mechanism to ensure forward progress of earlier rendering jobs so that any possible data (processing) dependencies can be properly resolved. In this regard, various arrangements will be possible.

[0061] Since the graphics processor is able to process rendering tasks of separate rendering jobs simultaneously, this means that the graphics processor (task issuing circuit) can start issuing rendering tasks for the second, later rendering job in a series of rendering jobs being executed, and the processing (shader) core can start processing these rendering tasks without having to wait for all rendering tasks of the first rendering job to have completed their processing. Therefore, the present invention advantageously avoids strict (hard) processing barriers between separate rendering jobs.

[0062] Specifically, as will be further explained below, the present invention provides a mechanism that allows a graphics processor to track the completion status of a rendering task of a first earlier rendering job when rendering tasks of a first rendering job and a second rendering job are being executed simultaneously. In a preferred embodiment, the tracking is done using a corresponding "task completion status" data structure associated with the corresponding rendering job being executed.

[0063] As will be further explained below, the method then includes (and the control circuit is configured to) use the "task completion status" data structure associated with the earlier rendering job to control the processing of the rendering tasks of the later rendering job to enforce data (processing) dependencies between different rendering jobs.

[0064] Thus, the processing of the rendering tasks of the second, later rendering job in the series of rendering jobs being executed may be controlled, as needed, using the corresponding "task completion status" data structure associated with the first, earlier rendering job to enforce any data (processing) dependencies between the rendering tasks of the different rendering jobs. In this way, the graphics processor may, and preferably does, ensure that any data (processing) dependencies between the rendering jobs are taken into account (i.e., even when the processing of the rendering tasks of the different rendering jobs overlaps within the processing (shader) core of the graphics processor, so that when the present invention is not applied, there is a risk and possibility that the rendering tasks of the second, later rendering job may exceed the corresponding rendering tasks of the first, earlier rendering job on which it depends, thereby introducing possible artifacts into the output of the second, later rendering job).

[0065] The effect and benefit of doing so is that processing of (at least some) rendering tasks of a second, later rendering job in a series of rendering jobs being executed can be, and preferably is, performed concurrently with processing of rendering tasks of a first, earlier rendering job in the series of rendering jobs, but in such a way as to still ensure "safe" graphics processing operations (i.e., without introducing artifacts into the rendered output generated by earlier or later rendering jobs due to failure to properly account for any such data (processing) dependencies).

[0066] For example, according to the present invention, the rendering tasks within each rendering job in the series of rendering jobs being generated are preferably still processed by the graphics processor processing (shader) core in a set desired rendering order, such as in a normal manner for issuing such rendering tasks, so that all rendering tasks of the first, early rendering job are issued before any rendering tasks of the second, later rendering job are issued. However, as described above, some rendering tasks of the first rendering job may be completed relatively quickly, while other rendering tasks of the first rendering job may take a longer time to complete.

[0067] Therefore, the present invention allows the graphics processor to start issuing and processing rendering tasks of the second later rendering job in a series of rendering jobs relatively early when executing the series of rendering jobs, without having to wait for all rendering tasks of the first earlier rendering job in the series of rendering jobs to complete their processing.

[0068] This thereby allows the processing of the first rendering job and the second rendering job to overlap within the graphics processor processing (shader) core, thereby reducing the possible situation of low utilization of the processing (shader) core, as if the graphics processor has run out of rendering tasks in the first rendering job, so that the rendering tasks of the first rendering job are not sufficient to fully utilize the available processing (shader) cores, the graphics processor according to the present invention can now begin to issue the rendering tasks of the second later rendering job to the graphics processor processing (shader) core to use the available processing (shader) core to process simultaneously with the remaining rendering tasks of the first early rendering job (if a strict (hard) processing barrier is enforced between rendering jobs, the available processing (shader) core will be idle). Therefore, this can improve the utilization of the processing (shader) core, thereby providing more efficient (e.g., faster) graphics processor operation.

[0069] For example, as described above, in some more conventional graphics processing arrangements, strict (hard) processing barriers may be enforced between different rendering jobs, such that the graphics processor cannot issue rendering tasks of a second, later rendering job until all rendering tasks of a first, earlier rendering job have completed their processing. This is because (as described above) there may be (and typically will be) various data (processing) dependencies between different rendering jobs, such that, for example, output (or part thereof) generated by a first, earlier rendering job will be used as input processed by a second, later rendering job. In this case, a rendering task of a second, later rendering job in a series of rendering jobs being executed may therefore need to use data from a data buffer that was (potentially) written by a corresponding rendering task of a previous (first, earlier) rendering job in the series of rendering jobs.

[0070] Such data (processing) dependencies then mean that certain rendering tasks of the second, later rendering job cannot (safely) be executed until processing of corresponding rendering tasks of the first, earlier rendering job has completed, i.e. until it is ensured that required data buffers (render targets) have been updated as required with data required for the second, later rendering job. Thus, when a rendering task of the second, later rendering job needs to access (and can be) written to a data buffer by the first, earlier rendering job, the rendering task of the second rendering job should not be executed until processing of any corresponding rendering tasks of the first rendering job (on which a rendering task of the second rendering job may have a data (processing) dependency) has completed.

[0071] Thus, while different rendering tasks within a given (same) rendering job are typically independent of each other such that they can be processed simultaneously (e.g., in parallel) across graphics processor processing (shader) cores without problems, the processing of a rendering task of a later rendering job in a series of rendering jobs to be executed can, and typically will, depend on the processing of a corresponding rendering task (or in some cases multiple rendering tasks) from an earlier (e.g., and typically previous) rendering job in the series of rendering jobs, such that a rendering task of a later rendering job should not be executed until its dependencies on rendering tasks of earlier rendering jobs have been resolved (i.e., until the rendering tasks of the earlier rendering jobs have completed their processing and written their final outputs such that any data buffers that the later rendering job may need to access can be updated as required by the earlier rendering job, e.g., so that the correct data is subsequently available in the data buffers of the rendering tasks of the later rendering job).

[0072] For example, an example of this would be when a particular rendering job in a series of rendering jobs being executed reads from a data buffer (render target) that a corresponding earlier rendering job in the series of rendering jobs wrote to (a so-called "read-after-write" dependency). A typical example of this could be when performing graphics processing texture operations, where a later rendering task reads from a portion of a texture buffer that was (or can be) written to by a corresponding rendering task of an earlier rendering job (e.g., a fragment rendering job of an earlier rendering pass in a series of rendering passes being executed).

[0073] In this case, a given rendering task of a later rendering job may need to read a portion of a data buffer (render target) that will be filled by a corresponding rendering task for the same portion of the overall rendering output, such as a corresponding region ("tile") of the rendering output of an earlier rendering job. In this case, a rendering task of a later rendering job should therefore not read data from the buffer until the corresponding rendering task of the earlier rendering job has completed, i.e., until that portion of the buffer has been updated (as needed) by the corresponding rendering task of the earlier rendering job.

[0074] Another example is where some state information needs to be passed between rendering jobs, such as variable rate shading parameters that will be reused between (fragment) rendering jobs of different rendering passes, but where the state information or parameter set of a rendering job of a later rendering pass is or can be determined by the processing of a rendering job of an earlier rendering pass (i.e., by executing an appropriate shader program as part of the earlier rendering job). Such state information would typically be read in relatively early in the fragment front end (e.g., via a fragment shader endpoint) (e.g., as a variable rate shading parameter image). In this case, the graphics processor may need to stop processing later rendering tasks within the fragment shader endpoint until the earlier rendering tasks have completed their processing so that the updated state information is available to the later rendering tasks.

[0075] Another example might be for a transaction elimination mechanism that can save memory bandwidth by avoiding writing rendering output (e.g., tiles) to memory when it can be determined that the content just rendered is the same as content already stored in memory. For example, the graphics processor may be operable and configured to write out a suitable signature (which may, for example, and in a preferred embodiment does include a suitable checksum (e.g., a cyclic redundancy check (CRC)) of the output value while writing the rendering output to the memory. Thus, when the next rendering job (e.g., the next rendering pass) is executed, a corresponding signature may be generated, and an appropriate check performed against the stored signature to determine whether the rendering output of the current rendering job is the same as the rendering output currently stored in the memory, in which case the writing out of the current rendering job may be omitted. Likewise, although the signature is not used until written out, in order to hide latency, the reading of the signature (which may, for example, be provided in the form of a "CRC buffer" storing the corresponding checksum (CRC) value, and is preferably so provided) is performed relatively early in the fragment front end (e.g., via a fragment shader endpoint), and therefore the graphics processor may need to stop processing of a rendering task of a later rendering job within the fragment shader endpoint until any rendering tasks of an earlier rendering job on which the rendering task depends have completed their processing and written out such a signature.

[0076] However, there may be various other examples in which the graphics processor, at some point during the graphics processing workload of a particular rendering job, needs to access some data (e.g., from a corresponding data buffer storing such data) that may have been generated or updated during the processing of another (earlier) rendering job in the series of rendering jobs being executed.

[0077] As in some more traditional graphics processing arrangements, strict (hard) processing barriers are enforced between rendering jobs, thereby ensuring that any rendering tasks of a later rendering job that may depend on the output of a first, earlier rendering job will not (and cannot) be issued before the processing of all rendering tasks of the first rendering job is complete, thereby ensuring safe graphics processing operation. However, as described above, this can mean that the utilization of the processing (shader) cores is low for a considerable period of time, for example, particularly towards the end of a rendering job, when relatively few rendering tasks need to be completed, leaving some processing (shader) cores idle, waiting for other processing (shader) cores to complete the processing of rendering tasks assigned to them.

[0078] In contrast, by providing a mechanism for tracking the completion status of rendering tasks within a rendering job, the present invention allows data (processing) dependencies between different rendering jobs to be managed in a more fine-grained manner, for example and preferably at the level of rendering tasks (or groups thereof) within a rendering job, rather than just for the entire rendering job, so that the graphics processor is able to operate and be configured to start issuing and processing rendering tasks of a later rendering job that is dependent on a relatively earlier rendering job, without waiting for all processing of the earlier rendering job to be completed.

[0079] Thus, the present invention advantageously avoids strict (hard) processing barriers between rendering jobs, thereby allowing the processing of different rendering jobs to overlap within the graphics processor processing (shader) core. In addition, this is to ensure that there is still safe graphics processing operation, in particular by ensuring that any data (processing) dependencies between rendering tasks of different rendering jobs are taken into account. Therefore, the management of any such data (processing) dependencies is effectively moved into the graphics processor processing (shader) core and performed under the control of the graphics processor, rather than enforcing strict (hard) processing barriers between rendering jobs in the graphics processor command stream.

[0080] In other words, according to the present invention, when executing a series of rendering jobs, the graphics processor does not need to (and will not) wait for all rendering tasks of the first, early rendering job in the series of rendering jobs to complete their processing before starting to issue and process rendering tasks of the second, late rendering job in the series of rendering jobs, thereby improving graphics processor performance (for example, in terms of throughput and processing (shader) core utilization), but the purpose of doing so is to enforce any required data (processing) dependencies between rendering jobs by controlling the processing, thereby still maintaining safe (i.e., artifact-free) graphics processing operations.

[0081] In this regard, an additional effect and benefit of tracking the completion status of rendering tasks in the manner of the present invention is that, in many cases, when the rendering tasks of later, potentially dependent rendering jobs are released for processing, any possible data (processing) dependencies between the rendering tasks of different rendering jobs will be naturally resolved. That is, in many cases, there is no need to stop processing of most rendering tasks for the second, later rendering job. Therefore, being able to release and process the rendering tasks of the second, later rendering job relatively early (and thus enforce any data (processing) dependencies relatively late) can significantly improve performance because, in many typical cases, the rendering tasks of the later rendering job can be executed immediately without having to stop (control) processing because, typically, there will be no outstanding data (processing) dependencies when the rendering tasks of the later rendering job are released for processing.

[0082] Therefore, the present invention provides a mechanism to ensure that rendering tasks can be stopped when it is necessary to stop rendering tasks, thereby ensuring safe graphics processing operations, but the applicant has recognized that in typical graphics processing applications, it may not actually be necessary to stop processing of most rendering tasks (and in this case, it may only be necessary to check the "task completion status" data structure to confirm that processing does not need to be stopped, but the rendering task can be executed normally without further controlling (e.g., stopping) its processing).

[0083] For example, and preferably, rendering tasks are released for processing in a set (e.g., predefined) order, which is generally consistent between different rendering jobs (and different rendering passes). For example, where the rendering tasks correspond to corresponding rendering tiles for rendering obtained by subdividing the overall output (frame), the rendering tasks (tiles) within each rendering job can generally and preferably be released for processing in a specific order (e.g., a predefined order) (e.g., in Z / Morton order), which is preferably the same as the order of the corresponding rendering tasks of each rendering job in the series of rendering jobs being executed.

[0084] Thus, during a first rendering job, processing may begin by issuing a first rendering task of the first rendering job (the first rendering task preferably corresponds to a first region of a rendering output associated with the first rendering job), and may then proceed to issuing a second rendering task of the first rendering job, and so on, wherein the rendering tasks of the first rendering job are issued in a desired order. Towards the end of the first rendering job, the processing (shader) core may begin to exhaust the rendering tasks of the first rendering job, and thus, as described above, the graphics processor according to the present invention is operable to begin issuing rendering tasks of the next rendering job (i.e., the second, later rendering job) for processing. However, the rendering tasks of the second rendering job are preferably issued in the same order as the rendering tasks corresponding to the first rendering job (e.g., starting again from the corresponding "first" rendering task of the second rendering job (the first rendering task of the second rendering job preferably corresponds to the same rendering output region as the equivalent first rendering task of the first rendering job (but for the rendering output of the second rendering job)), and then proceed to issue the corresponding "second" and further rendering tasks in the same determined order (e.g., a predefined order) as the first rendering job).

[0085] This has the effect that typically (only) the processing of later rendering tasks of a first rendering job may overlap with the processing of earlier rendering tasks of a second rendering job.

[0086] However, the present applicants have recognised that in most cases data (processing) dependencies between rendering jobs will (only) exist between certain corresponding rendering tasks of different rendering jobs (i.e., rendering tasks relating to the same or similar portion of the overall rendering output (e.g., the same region of a frame)). That is, and for example, the processing of a first rendering task of a second rendering job may depend on the processing of a corresponding (equivalent) first rendering task of the first rendering job (e.g., a rendering task that renders the (same) corresponding “tile” during the second rendering job), but often (and typically) does not depend on the processing of any other rendering task (e.g., any other rendered tile). That is, data (processing) dependencies may only relate to specific portions of data buffers (render targets) produced by an earlier rendering job. Thus, by issuing rendering tasks of different rendering jobs in the same consistent order, in many cases any data (processing) dependencies will have been resolved by the time rendering tasks of a later rendering job are issued for processing, since by this time, the processing of the corresponding rendering tasks of the earlier rendering job may have been completed. Therefore, in many cases, there may be no need to stop processing of a particular rendering task at all, because any possible data (processing) dependencies of that particular rendering task on the corresponding rendering task of an earlier rendering job will naturally be resolved by itself when the rendering task is released for processing. However, the present invention provides a "fallback" mechanism to ensure that any required data (processing) dependencies that are not resolved are (and can be) considered to ensure safe graphics processor operation.

[0087] That is, the present invention provides a mechanism for issuing potentially dependent rendering tasks relatively early so that their processing can potentially be performed concurrently with rendering tasks from earlier rendering jobs, but also recognizes that by doing this and managing dependencies within a rendering job (rather than at the level of the rendering job as a whole), in many cases the data (processing) dependencies will already be satisfied when the potentially dependent rendering tasks are issued for processing.

[0088] In this way, in addition to providing fuller utilization of available processing (shader) cores, the present invention can, for example and preferably, provide significant improvements in graphics processing performance, particularly in terms of processing speed / throughput (or in other words, reduced latency), while still ensuring that processing of rendering tasks for later rendering jobs can be stopped (if necessary) to ensure that any dependencies are respected. Another possible benefit is that since different rendering jobs that may require the same data can be executed more closely together, this can potentially increase data locality and therefore improve memory access efficiency.

[0089] Thus, the present invention may provide various benefits compared to other possible approaches.

[0090] As described above, the present invention preferably provides a novel "task completion status" data structure, which is (can be) associated with a specific rendering job (for a specific rendering channel) and can be used to track the completion status of the rendering tasks of the specific rendering job associated with the "task completion status" data structure. Therefore, the "task completion status" data structure of a specific rendering job preferably tracks which rendering tasks to be processed by the rendering job have completed their processing.

[0091] The task completion status of the corresponding rendering task of a given rendering job can be tracked in various suitable ways as needed. For example, in a preferred embodiment, once the output (or all outputs) associated with the rendering task (and which outputs may cause possible data (processing) dependencies) have been properly submitted to the memory, the task is considered to be "completed". Therefore, in a preferred embodiment, a "task completion status" data structure tracks when all memory accesses associated with the rendering task have been completed and the output has been properly submitted to the memory so that subsequent rendering jobs can read the output (i.e., a "consistency point" where all (relevant) observers are guaranteed to see the same copy of the memory location at a specific address). For example, in the case where the graphics processor includes a cache system, the consistency point can typically be the L2 cache, because the later rendering job can read the data directly from the L2 cache without returning the data all the way to the external (main) memory. Therefore, in a preferred embodiment, once (and preferably, as long as) the rendering tasks have completed their processing and performed the final write operation, a suitable task completion status can be signaled, and the suitable task completion status is used to update the "task completion status" data structure accordingly, so as to ensure that any data relied on has been submitted to the memory.

[0092] However, the tracking of task completion status can generally be increased or decreased in complexity as needed. For example, in the case where a rendering task can produce multiple different outputs, a "task completion status" data structure can track the various outputs. In this regard, various arrangements will be possible.

[0093] As will be further explained below, the "task completion status" data structure for a particular rendering job is then or can be used when executing a later rendering job (e.g., the next rendering job) in a series of rendering jobs being executed to control the processing of the rendering tasks of the later rendering job, thereby ensuring that any data (processing) dependencies between different rendering jobs (rendering tasks) are taken into account.

[0094] The "task completion status" data structure of a particular rendering job may therefore be, and preferably is, updated during the rendering job, for example when the rendering tasks of the rendering job in question complete their processing.

[0095] A corresponding (separate) "task completion status" data structure may be assigned to each rendering job currently in progress (in some embodiments) in order to track the completion status of the corresponding rendering task of the rendering job. However, it should be understood that data (processing) dependencies may often be particularly relevant to fragment rendering jobs. Therefore, in some preferred embodiments, a corresponding (separate) "task completion status" data structure is assigned to each fragment rendering job, but other types of rendering jobs (e.g., "compute" jobs related to rendering output) may not be assigned a corresponding "task completion status" data structure. In this regard, various arrangements will be possible.

[0096] In a typical graphics processing application, a rendering job will only depend on rendering jobs in the same or adjacent rendering passes in the entire series of rendering passes being executed. In this case, there may be no need to record which rendering jobs (or rendering passes) have completed their processing, as long as the completion status of the rendering tasks of any rendering job (for any rendering pass) currently in progress is appropriately tracked. Therefore, preferably, once a rendering job has completed (i.e., all rendering tasks of the rendering job have completed their processing), the corresponding "task completion status" data structure for the rendering job can be (and is preferably) discarded. However, if desired, a total record of completed rendering jobs (or rendering passes) can also be stored.

[0097] Accordingly, when the graphics processor begins issuing rendering tasks for a new (i.e., next) rendering job, a corresponding "task completion status" data structure can (and preferably is) allocated as needed for that rendering job, and then filled in accordingly when the rendering tasks of the rendering job in question complete their processing.

[0098] Therefore, in general, when processing of multiple (e.g., two) different rendering jobs (e.g., fragment rendering jobs of two different rendering channels) is performed simultaneously, there may and preferably will be corresponding multiple (e.g., two) (separate) "task completion status" data structures maintained respectively for the multiple (e.g., two) different rendering jobs.

[0099] According to the specific requirements of the present invention, the “task completion status” data structure for a specific rendering job can take any suitable and desired form.

[0100] For example, when rendering tasks are issued according to a set (e.g., predefined) order, a "task completion status" data structure for a particular rendering job may store an indication of the latest (most recent) rendering task that has completed, where all previous rendering tasks for the rendering job have also completed. Thus, based on such an indication, it may be determined that any possible data (processing) dependencies on rendering tasks that precede the rendering task in the set order of issuance earlier than the rendering task storing the indication can be safely ignored because the rendering task has completed. In this case, the "task completion status" data structure may essentially store only a single entry identifying the first rendering task of the rendering job associated with the "task completion status" data structure, knowing that all previous rendering tasks have completed their processing.

[0101] However, in this case, there may be a situation where one (earlier) rendering task takes longer than expected, and even if later rendering tasks have completed their processing, the indication cannot be updated to reflect this, because the indication can only indicate rendering tasks for which all previous rendering tasks have also completed. Therefore, since rendering tasks can complete out of order, it is still necessary to (also) track the completion status of rendering tasks between the moment when all previous rendering tasks are known to have completed their processing and the current rendering task.

[0102] Thus, in preferred embodiments, the "task completion status" data structure can and does store corresponding indicators of whether corresponding rendering tasks (or, in some embodiments, rendering task groups) have completed their processing. The "task completion status" data structure for a rendering job can and preferably does contain a suitable number of entries for storing such indicators.

[0103] In some preferred embodiments, the "task completion status" data structure may include a "scoreboard" that is operable and configured to store, with respect to the rendering task (or rendering task group) of the rendering job in question, respective indicators of whether the rendering task (or all rendering tasks in the rendering task group) have completed their processing. Thus, the "scoreboard" of a given rendering job may be referenced as part of issuing and / or processing rendering tasks of subsequent rendering jobs that may be dependent on the rendering job in question (with which the scoreboard is associated) in order to control the processing of the rendering tasks of the subsequent rendering job accordingly, thereby managing any required data (processing) dependencies.

[0104] In a preferred embodiment, the "task completion status" data structure of the rendering job may therefore include corresponding entries for storing corresponding indicators for each rendering task to be processed for the rendering job in question, so that there is a one-to-one correspondence between the entries in the "task completion status" data structure and the rendering tasks to be performed for the rendering job.

[0105] However, in general, there may be any suitable and desired correspondence between entries in a “task completion status” data structure and rendering tasks of the rendering job to which the “task completion status” data structure is associated.

[0106] In this regard, it should be appreciated that storing a corresponding indicator for each rendering task may require a relatively large area to store the "task completion status" data structure. For example, in a tile-based rendering system, where each rendering task corresponds to a corresponding rendering tile, and where the total output is subdivided into a 64×64 array of rendering tiles, the "task completion status" data structure may therefore need to store 4096 corresponding individual indicators.

[0107] Thus, in some preferred embodiments, corresponding entries in the "task completion status" data structure may store indicators of a plurality of rendering task groups as a whole (each group corresponding to a corresponding set of fewer than all rendering tasks associated with the rendering job in question), in which case the stored indicators indicate when all rendering tasks within the plurality of rendering task groups have completed their processing. In such embodiments where the "task completion status" data structure stores corresponding indicators for rendering task groups, entries of the "task completion status" data structure may generally store corresponding indicators for any appropriately defined rendering task groups, including any appropriate and desired number of rendering tasks. Thus, in an embodiment, the "task completion status" data structure may store corresponding indicators for rendering task groups, such as 4×4, 8×8, 16×16, etc. groups (typically N×M groups) of rendering tasks.

[0108] This means that there may be some "false" dependencies, i.e., dependencies related to specific rendering tasks in a plurality of rendering task groups associated with corresponding entries in the "task completion status" data structure have actually been resolved, but since there are other rendering tasks in the group that have not yet completed, the "task completion status" data structure cannot be used to determine this, and thus the present invention may unnecessarily enforce data (processing) dependencies. On the other hand, storing an indicator of the rendering task group can certainly reduce the area required to store the "task completion status" data structure, and will still provide significant performance improvements (especially since, as mentioned above, in typical graphics processing applications, many (most) data (processing) dependencies can naturally be resolved by themselves).

[0109] In the case where the "task completion status" data structure stores indicators for multiple rendering task groups as a whole, these indicators can take any suitable and desired form. For example, the graphics processor can track the completion status of all rendering tasks for an associated rendering job and combine them into a single (binary) indicator as to whether the rendering task group has completed its processing. Alternatively, and in some preferred embodiments, a corresponding counter can be used that tracks the number of tasks in progress (such that the counter is decremented when a rendering task is completed). In this case, once the counter reaches zero, this can indicate that all rendering tasks for the associated rendering job have been completed.

[0110] Of course, it is also possible to adjust the granularity of the "task completion status" data structure through various other arrangements to balance performance and the area required to store the "task completion status" data structure. Therefore, in general, the "task completion status" data structure can store indicators of the task completion status of a group of one or more rendering tasks, and such indicators can be stored in any suitable and desired form.

[0111] Therefore, the "task completion status" data structure of a particular rendering job (regardless of the specific form of the "task completion status" data structure) can be (and is) updated when the rendering task of the rendering job corresponding to the "task completion status" data structure is completed. The update of the "task completion status" data structure can be triggered in any suitable and desired manner. For example and preferably, when the (final) output associated with the rendering task is written out to, for example, a memory, the "task completion status" data structure is updated with respect to the particular rendering task. That is, when the rendering output of the rendering task is submitted to the memory (as described above), the rendering task is preferably considered to be "completed", and the "task completion status" data structure is preferably updated at this time. Therefore, in a preferred embodiment, whenever a rendering task of a rendering job completes its processing, when the (final) output of the rendering task is written out, this can (and preferably) be signaled to cause an update of the "task completion status" data structure associated with the rendering job. The entry or indicator containing the rendering task can then be updated accordingly based on the situation in which the rendering task completes its processing.

[0112] However, other arrangements may be used to consider when a particular rendering task has completed its processing, such that the "task completion status" data structure can (and should) be updated, and in general, the tracking of completion status can be made more or less complex as needed. For example, in addition to / alternatively to tracking when the (final) rendering output associated with the rendering task is written out, the "task completion status" data structure may also be used to track when the rendering task has completed processing any individual rendering targets (data buffers) that the rendering task may have used.

[0113] The "task completion status" data structure may generally be stored in any suitable and desired manner, depending on the specific requirements of the present invention, and as long as it can be appropriately checked at appropriate points during the graphics processing operation.

[0114] For example, in some embodiments, a global "task completion status" data structure is (for example and preferably) stored in a task iterator (task issuing circuit), which is operable to control the issuing of rendering tasks to multiple processing (shader) cores of a graphics processor (for example, as part of the task issuing circuit of a graphics processor). In this case, any particular processing (shader) core can query the "task completion status" data structure as needed to understand the rendering task being performed by the processing (shader) core. In this case, each processing (shader) core in the processing (shader) core should and preferably be operable to update the same global "task completion status" data structure when / when the processing (shader) core completes the processing of its rendering task. Therefore, in some embodiments, when the processing (shader) core completes its processing of the rendering task, it is preferably signaled back to the task iterator (task issuing circuit) subsequently, and this preferably subsequently triggers the updating of the global "task completion status" data structure.

[0115] However, other arrangements are possible. For example, it is also possible for the processing (shader) cores to store their own local version of the "task completion status" data structure. In this case, the processing (shader) core can then query its own version of the "task completion status" data structure as needed, which can speed up the speed of checking the "task completion status" data structure (e.g., without having to send a message to the task iterator (task issuing circuit) to query the global "task completion status" data structure).

[0116] On the other hand, in this case, a mechanism must be provided to allow different processing (shader) cores to transmit updates between each other. This can be achieved in various suitable ways as needed.

[0117] For example, in one embodiment, this can be accomplished by also maintaining a separate global "task completion status" data structure, similar to the above embodiment, which can then be updated by any one of the processing (shader) cores, as when the processing (shader) core completes its processing of the corresponding rendering task. The global "task completion status" data structure can then be used to update the local version of the "task completion status" data structure for different processing (shader) cores. For example, the global "task completion status" data structure can be periodically broadcast to all processing (shader) cores to allow the processing (shader) core to update its local version of the "task completion status" data structure. Alternatively, in another embodiment, each processing (shader) core can be operable and configured to broadcast updates to all other processing (shader) cores so as to cause the local version of the "task completion status" data structure to be updated as needed.

[0118] In all cases, it is important to note that there is some risk that a given processing (shader) core will access a stale "task completion status" data structure, but this is acceptable as this only delays the rendering task (which is not actually necessary) and still ensures safe graphics processing operations, so this false positive is usually acceptable.

[0119] Various other arrangements for storing and updating the "task completion status" data structure (or structures) would be possible.

[0120] Regardless of the form / mode in which it is stored, the "task completion status" data structure for a particular rendering job can then be (and is) used to control the processing of rendering tasks for a later (e.g., next) rendering job in a series of rendering jobs being executed. Specifically, while the rendering tasks of the (first) earlier rendering job are still being processed, the "task completion status" data structure of the (first) earlier rendering job can be used to control the processing of rendering tasks of the (second) later rendering job, provided that those rendering tasks are posted for processing.

[0121] In this case, when there are data (processing) dependencies between the (first) early rendering jobs, the graphics processor can use the "task completion status" data structure of the (first) early rendering job when issuing and / or processing the rendering tasks of the (second) late rendering job to determine whether any corresponding rendering tasks of the (first) early rendering job on which the processing of the rendering tasks of the (second) late rendering job depends have completed their processing, and therefore determine whether it is necessary to delay the processing of the rendering tasks of the (second) late rendering job.

[0122] Therefore, if the “task completion status” data structure can be used to determine that the processing of any corresponding rendering tasks of the (first) early rendering job on which the processing of the rendering tasks of the (second) late rendering job depends has been completed, then the rendering tasks of the (second) late rendering job can be processed normally accordingly (because the “task completion status” data structure indicates that any data (processing) dependencies have been resolved). On the other hand, if the “task completion status” data structure cannot be used to determine that the processing of any corresponding rendering tasks of the (first) early rendering job on which the processing of the rendering tasks of the (second) late rendering job depends has been completed, then additional control can (and does) be performed on the processing of the late rendering tasks to ensure that the data (processing) dependencies are considered.

[0123] Controlling of the rendering tasks of the second, later rendering job may be performed in any suitable and desired manner, taking into account any data (processing) dependencies between the rendering jobs.

[0124] Preferably, in the case where there is a possible data (processing) dependency between the rendering tasks of the second, later rendering job and the rendering tasks of the first, earlier rendering job, the control of the rendering tasks of the second, later rendering job involves stopping the processing of the rendering tasks of the second, later rendering job until it can be determined that the rendering tasks of the first, earlier rendering job have completed their processing. However, for example and specifically, depending on the type of data (processing) dependency, the control may be performed in various suitable and desired ways.

[0125] For example, the invention may be particularly useful for managing data (processing) dependencies where a rendering task may only write to a particular output once (a "write-once" operation). This is typically the case for tile-based rendering, for example, where each rendering task fills a corresponding entry in a tile buffer, which is then written (exactly) once to the frame buffer when the rendering task is completed.

[0126] In this case, as will be further explained below, processing of a rendering task of a later rendering job may typically only require (and will only) stopping processing of the rendering task of the later rendering job to wait for completion of a rendering task (or multiple rendering tasks) of an earlier rendering job on which it may depend, but once the corresponding rendering task (or multiple tasks) of the earlier rendering job has completed, the rendering task can be safely released for processing (and processed).

[0127] For example, for a "read-after-write" data (processing) dependency, where a later rendering job reads data from a buffer that may have been written to by an earlier rendering job, the processing of the rendering tasks of the later rendering job should and is indeed controlled to ensure that a given rendering task of the later rendering job is not executed until the corresponding (equivalent) rendering task of the earlier rendering job has completed its processing.

[0128] However, this may be checked at any suitable point during the graphics processing workload, eg depending on the data (processing) dependencies in question.

[0129] For example, the graphics processor may delay issuing rendering tasks of a later rendering job (e.g., by stalling the rendering tasks within a task iterator (task issuing circuitry) that controls issuing rendering tasks, such as as part of the graphics processor's task issuing circuitry) until it can (and does) determine using the earlier rendering job's "task completion data structure" that determines that the earlier rendering job's corresponding rendering tasks have completed their processing.

[0130] Delayed release of rendering tasks may also be necessary if the data (processing) dependencies involve data required by the task iterator (task release circuitry) and / or the fragment shader endpoint (e.g., state data for initializing the rendering task). This may include, for example, state information such as variable rate shading parameters read by the fragment shader endpoint. This may also include, for example, signatures (e.g., CRC buffers) that will be used for transaction elimination, which may also be read by the fragment shader endpoint as described above.

[0131] However, in general, the data (processing) dependency may be related to data that is needed later during the graphics processing workload, and in this case, in order to stop the related memory access (as needed) but allow the rendering task to be processed to the related memory access point, it is preferred to control the processing of the rendering task of the later rendering job. Therefore, in a preferred embodiment, the "task completion status" data structure can be checked before any related memory access (for example, immediately before any related memory access) to determine whether it is necessary to stop the memory access of the current rendering job to wait for the completion of the rendering task of the earlier rendering job on which the current rendering job depends.

[0132] For example, and in some preferred embodiments, the graphics processor is operable and configured to check a "task completion status" data structure and, if necessary, control processing of rendering tasks whenever there is a possible memory access, and this may be done at various locations within the graphics processing pipeline.

[0133] Therefore, this can be done by a task iterator (task issuing circuit) that controls the issuance of tasks to the processing (shader) core, so as to control whether rendering tasks are issued to the processing (shader) core (or whether issuing rendering tasks should be stopped).

[0134] However, control may also and preferably be performed within the programmable execution unit of the processing (shader) core, for example during shader program execution. Specifically, whenever a shader program requires memory access, a "task completion status" data structure is preferably checked, and appropriate control (e.g., stalling) is performed as required. This may be, for example, via one or more functional units accessible to the programmable execution unit, such as a texture mapping unit, an attribute unit, or a tile writeback unit.

[0135] This means that the graphics processor can continue to perform useful processing work and provide additional time to naturally resolve data (processing) dependencies before it is possible to stop processing the rendering task for later rendering output. For example, and in a preferred embodiment, even if there are possible data (processing) dependencies in the execution of the shader program, the rendering task can still be processed to that point in the graphics processing pipeline, for example, so that even if the shader program cannot (yet) be executed, the fragment "front-end" processing operations for setting up the fragment shader program can be performed in the background (and this also has the benefit that when the rendering task reaches the fragment shader stage, the data (processing) dependencies may have been cleared).

[0136] Therefore, in various embodiments, the method includes identifying whether there is a potential data dependency between the rendering tasks of the second rendering job and the rendering tasks of the first rendering job, and when it is identified that there is a potential data dependency between the rendering tasks of the second rendering job and the rendering tasks of the first rendering job: controlling the processing of the rendering tasks of the second rendering job to enforce the potential data dependency.

[0137] In this case, identifying whether there is a potential data dependency between the rendering tasks of the second rendering job and the rendering tasks of the first rendering job can be performed before issuing the rendering tasks of the second rendering job for processing, and wherein controlling the processing of the rendering tasks of the second rendering job to enforce the potential data dependency includes stopping issuing the rendering tasks for processing until it is determined based on tracking the completion status of the rendering tasks of the first rendering job that the rendering tasks of the first rendering job on which they depend have completed their processing.

[0138] Additionally / alternatively, identifying whether there is a potential data dependency between the rendering tasks of the second rendering job and the rendering tasks of the first rendering job may be performed during processing of the rendering tasks of the second rendering job and before the rendering tasks of the second rendering job perform a memory access that may depend on output produced by the rendering tasks of the first rendering job, and wherein controlling the processing of the rendering tasks of the second rendering job to enforce the potential data dependency includes halting the memory access until it is determined based on tracking of the completion status of the rendering tasks of the first rendering job that the rendering tasks of the first rendering job on which it depends have completed their processing.

[0139] In this regard, various arrangements would be possible.

[0140] The above discussion mainly concerns so-called "read-after-write" data (processing) dependencies, where a rendering task of a later rendering job may need to read data written by an earlier rendering job.

[0141] The present invention can also be used to manage so-called "write-after-write" data (processing) dependencies, where a later rendering job may write to the same buffer written by an earlier rendering job, so that any writes by the rendering tasks of the later rendering job should be delayed again until the corresponding rendering tasks of the earlier rendering job have completed their writes, and thus the "write-after-read" data (processing) dependency can be handled in much the same way as the above-mentioned "write-after-read" data (processing) dependency (i.e., using the "task completion status" data structure of the present invention). Applications requiring the graphics processor typically specify such write operations in advance (this is usually required by the API), so the graphics processor can easily identify when such a so-called "write-after-write" data (processing) dependency exists.

[0142] However, there may be other types of data (processing) dependencies that cannot be managed using the "task completion status" data structure of the present invention, or it may be necessary or desirable to enforce stricter (harder) processing barriers between rendering jobs. For example, it may be the case that a given output (buffer) associated with a particular rendering task may be overwritten by any other rendering task during the same rendering job, and therefore rendering tasks of later rendering jobs that require data from that buffer cannot be safely executed until all rendering tasks of earlier rendering jobs have completed their processing. In this case, the present invention may, for example, and preferably, enforce stricter (harder) processing barriers between rendering jobs, which may be accomplished, for example, in accordance with the more traditional methods described above.

[0143] A specific example of this would be when a rendering job includes so-called shader "side effects", such as unconstrained load / store operations or atomic operations on shader storage buffer objects, where any rendering task may read or write any object in the shader storage buffer object. In this case, the rendering job is not necessarily a write-once process for the output of the shader storage buffer object. Thus, if a subsequent rendering job has a data (processing) dependency on the shader storage buffer object, the rendering tasks of the subsequent rendering job may not be posted for processing until all the rendering tasks of the earlier rendering job have completed their processing (because any rendering task of the earlier rendering job may write to the shader storage buffer object).

[0144] In such cases, it may be necessary to enforce a stricter (harder) processing barrier between rendering jobs, and it is preferred to do so. Typically, when generating shader programs, the compiler can (preferably) identify the presence of such "side effects". Therefore, the compiler can add a suitable flag (such as a "shader_has_side_effects" flag) to the shader program to signal the presence of such side effects. The graphics processor can therefore use this signaling to identify instances where a stricter (harder) processing barrier needs to be enforced between rendering jobs, thereby preventing the graphics processor from issuing any rendering tasks for processing by a later rendering job until all rendering tasks (including side effects) of an earlier rendering job have completed their processing.

[0145] Another example where it may be necessary to enforce a stricter (harder) processing barrier between rendering jobs may be for so-called "read-after-write" dependencies, where a later rendering job may overwrite a buffer that an earlier rendering job may need to read from. In this case, again, it may not be possible to determine which rendering tasks of the earlier rendering job may still need to read from that buffer when the corresponding rendering task of the later rendering job is issued, since read operations generally do not need to be specified in advance. It may be possible to identify and correctly manage these "read-after-write" dependencies, for example and specifically where the application programmer provides enough information to do so.

[0146] This may not always be guaranteed, and so in some cases it may be preferable or necessary to enforce a stricter (harder) processing barrier between rendering jobs whenever such a "read-after-write" dependency is identified (and so it is preferably done so). Again, these instances can usually be identified in advance, as which input / output buffers will be used and the various access permissions will usually be specified for a given rendering job (and API requirements usually dictate this).

[0147] In various other situations, operation in accordance with the present invention may be inappropriate or undesirable, so being able to fall back to a stricter (harder) processing barrier between rendering jobs may be beneficial.

[0148] In this case, the control of the present invention can therefore be selectively disabled. That is, in some embodiments, the ability of the graphics processor to simultaneously issue and process the rendering tasks of a separate rendering job in a processing (shader) core can be selectively suppressed, so that a stricter (harder) processing barrier can be enforced between rendering jobs when necessary. In other words, in some cases, there may be some situations where it is necessary (or desirable) to enforce a stricter (harder) processing barrier between rendering jobs, and in such cases, the operation according to the present invention can be selectively disabled.

[0149] For example, in some cases, the graphics processor may fall back to a strict (hard) processing barrier, wherein the graphics processor is prevented from issuing any rendering tasks of a later rendering job for processing until all rendering tasks of an earlier rendering job have completed their processing, e.g., as in the more traditional graphics processing arrangements mentioned above. However, it is also possible to suppress certain mechanisms of the present invention, but still allow at least some processing to be performed on rendering tasks of a later rendering job without strictly waiting for all rendering tasks of an earlier rendering job to complete their processing (and this may be done in some preferred embodiments). For example, in situations where it may be desirable to enforce a stricter (harder) processing barrier between rendering jobs, it may still be possible to allow certain processing of a later rendering job to be performed, e.g., and specifically, as long as the associated memory accesses and / or shader programs of the later rendering job are not executed until all rendering tasks of the earlier rendering job have completed their processing.

[0150] In this regard, various arrangements would be possible.

[0151] The graphics processor can therefore be selectively operated in the manner described above. Therefore, in an embodiment, a specific mechanism according to the present invention that allows rendering tasks of a later rendering job to be issued for processing concurrently with rendering tasks of an earlier rendering job can be selectively enabled / disabled, for example and specifically, for certain types of data (processing) dependencies, where it is still necessary to enforce stricter (harder) processing barriers between rendering jobs. That is, in some embodiments, the ability of the graphics processor to simultaneously issue and process rendering tasks of a separate rendering job in a processing (shader) core can be selectively suppressed, so that stricter (harder) processing barriers can be enforced between rendering jobs when necessary.

[0152] In this case, any part of the operation according to the present invention may be selectively enabled / disabled, for example, depending on the specific situation.

[0153] For example, where certain operations according to the invention are selectively disabled between a given pair of rendering jobs, then there may be no need to maintain a "task completion status" data structure for the earlier rendering job, since this will not be used to control the processing of any rendering tasks for the later rendering job (and this is therefore preferably not done). In this regard, various arrangements will be possible.

[0154] Therefore, according to the present invention, for a given rendering job in a series of rendering jobs to be executed, it is preferably identified whether the rendering job has any possible data (processing) dependency on an earlier (e.g., previous) rendering job in the series of rendering jobs in which one or more rendering tasks are still in progress, and when such possible data (processing) dependency is identified, then the processing of the (later) rendering job is controlled accordingly, for example based on the identified data (processing) dependency.

[0155] For example, when it is identified that there is no possible data (processing) dependency between two rendering jobs, the graphics processor can simply issue the rendering tasks of the later rendering job and the rendering tasks of the earlier rendering job at the same time, and no additional control is required to ensure that the data (processing) dependency is considered (because there is no data (processing) dependency that needs to be considered).

[0156] Therefore, when it is identified that there is no potential data dependency between the rendering tasks of the second rendering job and the rendering tasks of the first rendering job, the method preferably includes continuing to process the rendering tasks of the second rendering job.

[0157] However, if the graphics processor identifies that some data (processing) dependencies exist between the two rendering jobs, additional control is performed to ensure that the data (processing) dependencies are respected. The nature of this control may depend on the type of data (processing) dependencies identified.

[0158] For example, as described above, certain types of data (processing) dependencies may exist such that rendering tasks of separate rendering jobs should not be performed concurrently (e.g., because an earlier rendering job included side effects), in which case the control preferably enforces stricter (harder) processing barriers between rendering jobs and prevents any overlap between rendering jobs within the graphics processor processing (shader) cores.

[0159] On the other hand, for data (processing) dependencies of rendering tasks of separate rendering jobs that can potentially be executed simultaneously, that is, as long as the processing (shader) cores enforce any data (processing) dependencies (for example, the so-called "read after write" or "write after write" dependencies as described above), the present invention allows the rendering tasks of a later rendering job to be released for processing without waiting for all rendering tasks of an earlier rendering job to complete their processing, but uses a "task completion status" data structure to ensure that any such data (processing) dependencies are correctly considered, for example and preferably, processing of the rendering tasks of a later rendering job can be stopped as needed until it can be determined from the "task completion status" data structure that the corresponding rendering tasks of the earlier rendering job have been completed.

[0160] Possible data (processing) dependencies between rendering jobs may be identified in any suitable and desired manner.

[0161] For example, for each rendering job, there may be a corresponding set of (output) data buffers (render targets) that the rendering job (may) be written to. Likewise, there may be a corresponding set of (input) data buffers that the rendering job reads from. The API requirements typically require that it is declared in advance which inputs / outputs may be used for a given rendering job (or at least for a particular rendering pass that includes a rendering job). Therefore, it is typically declared in advance which data buffers may be used for a given rendering job, as well as the corresponding memory access permissions of the rendering job in question to these data buffers.

[0162] Therefore, for a given rendering job, if there is no overlap between the set of data buffers that the rendering job needs to access (i.e., read and / or write) and the set of (output) data buffers that the earlier (previous) rendering job (may) write, then there is no possible data (processing) dependency, and thus the rendering job can be safely executed together with the earlier rendering job. On the other hand, if there is an overlap between the set of data buffers that the later rendering job may need to access and the set of data buffers that the earlier (previous) rendering job (may) write, then it means that the later rendering job has a possible data (processing) dependency on the earlier rendering job, and this possible data (processing) dependency needs to be properly managed (and this can be done accordingly using the techniques of the present invention).

[0163] Thus, in an embodiment, identifying whether there is a potential data dependency between the rendering tasks of the second rendering job and the rendering tasks of the first rendering job includes determining whether the rendering tasks of the second rendering job need to access any data buffers potentially written by the rendering tasks of the first rendering job.

[0164] The determination as to whether there is overlap between the set of data buffers that a rendering job needs to access (i.e., read from and / or write to) and the set of (output) data buffers to which an earlier (previous) rendering job (possibly) writes can be performed in any suitable and desired manner. For example, this can be done by storing a bitmap representing the data buffers used by a rendering job (or rendering pass), and then comparing the corresponding bitmaps of different rendering jobs to determine any possible overlap. However, other arrangements are also possible. For example, rather than explicitly comparing all possible data buffers, an appropriate set of probabilistic tests (such as a Bloom filter) can be used to determine whether there is overlap between data buffers between rendering jobs (relying on the fact that as long as there is a possible dependency, false positives are acceptable, and processing is stopped until earlier tasks have completed their processing).

[0165] However, various other arrangements would be possible in this regard.

[0166] Thus, the present invention may generally involve the step of identifying when such possible data (processing) dependencies exist.

[0167] When implementing the present invention, it may also be necessary to identify, for a given rendering task of a later rendering job, which rendering task of a rendering job still being processed the rendering task may depend on.

[0168] In this regard, it should be understood that in some cases, the rendering tasks of a later rendering job may depend (only) on corresponding equivalent rendering tasks of an earlier rendering job (e.g., the same rendering tile, corresponding to the same area of ​​the screen), or on a specific set of rendering tasks of an earlier rendering job that are known in advance, and in some preferred embodiments, the graphics processor is configured to make this the case.

[0169] Thus, in some embodiments, the present invention may be used only when a given rendering task of a later rendering job may depend on a predefined set of rendering tasks of an earlier rendering job, so that control includes checking only the relevant entries of the "task completion status" data structure for the predefined set of rendering tasks (and not other rendering tasks) (and in this case, any data (processing) dependencies between other rendering tasks may be regarded as "side effects").

[0170] However, in principle, the rendering tasks of a later rendering job can depend on any rendering tasks of an earlier rendering task, and as long as they can be properly identified, they can also be properly managed. For example, it is usually not known in advance (in the API) which inputs (images) will be read from, or which parts of these inputs (images) will be read from, and the rendering job can usually specify a large set of inputs from which it may read. The shader program can also choose (for various reasons) during execution whether to actually read from the set of inputs it has access to. Therefore, in a preferred embodiment, when processing the rendering tasks of a later rendering job, during processing (for example, preferably, when memory access is to be performed) it is determined not only whether there is a possible data (processing) dependency on the earlier rendering job, but also which rendering task or tasks of the earlier rendering job the current rendering task may depend on. This can usually be done in the following way, for example, determining whether the later rendering task accesses the same part or area of ​​a specific data buffer (rendering target) that the earlier rendering task also accesses. For example, as described above, data (processing) dependencies may generally exist between rendering tasks associated with the same or similar rendering output regions, and thus determining whether a particular rendering task has a data (processing) dependency on an earlier rendering task is preferably based on whether rendering tasks of different processing jobs are associated with the same or similar rendering output regions relative to each other.

[0171] Control of the rendering tasks of the later rendering tasks may then be performed accordingly, for example, based on the completion status of any (and all) rendering tasks of the earlier rendering tasks on which the processing of the rendering task has been determined to be dependent.

[0172] In some cases, it may be identified that a particular rendering task (e.g., tile) was not processed by an earlier rendering job. For example, in some cases, if a particular tile does not contain any geometry, this may be identified (e.g., during a stitching operation) and rendering of the tile may then be avoided. If it may be identified that a particular rendering task (e.g., tile) was not processed in an earlier rendering job, this means that there may be no data (processing) dependencies for the particular rendering task, and the rendering task may therefore be effectively ignored so as to identify whether there are any possible data (processing) dependencies when executing a later rendering job. Various arrangements will be possible in this regard.

[0173] As mentioned above, the present invention relates in particular to situations where a graphics processor is operated to perform a sequence of rendering jobs, for example and preferably for generating a corresponding sequence of (final) rendering outputs (e.g., frames), for example for display. In some situations, it may be desirable to pause the generation of the (current) rendering output sequence, for example in order to allow the graphics processor to start generating a new, different rendering output sequence. In this case, the pause of the generation of the current sequence of rendering jobs is preferably performed in a controlled manner to allow the rendering jobs of the sequence to be subsequently resumed, preferably without having to start from the beginning of the sequence.

[0174] Specifically, in response to the graphics processor receiving a command to pause processing of a currently executing sequence of rendering jobs, the graphics processor is preferably operable and configured to then execute an appropriate pause mechanism to safely pause its current sequence of rendering jobs. In this regard, the present invention recognizes that if two separate rendering jobs are processed simultaneously, simply pausing both rendering jobs at once when one of the rendering jobs is dependent on the other rendering job may result in a deadlock situation. Therefore, in a preferred embodiment, when such a pause operation is to be performed, an earlier rendering job is allowed to complete its processing before a later rendering job is paused.

[0175] The present invention is generally applicable to any suitable graphics processing system.

[0176] The present invention particularly relates to tile-based rendering systems, in which a rendering output (e.g., a frame) is subdivided into a plurality of rendering tiles for rendering purposes. In this case, each rendering tile can and preferably does correspond to a respective sub-region of the entire rendering output (e.g., a frame) being generated. For example, a rendering tile may correspond to a rectangular (e.g., square) sub-region of the overall rendering output.

[0177] In a preferred embodiment, rendering is performed using rasterization. However, it should be understood that the invention is not necessarily limited to rasterization-based rendering, and may generally be used for other types of rendering, including ray tracing or hybrid ray tracing arrangements.

[0178] The invention is applicable to all forms of output that a graphics processing pipeline may generate, such as frames for display, render-to-texture output, etc. The invention is generally applicable to both graphics and non-graphics (eg, compute) workloads, as well as mixed workloads.

[0179] In some embodiments, the graphics processing system includes and / or communicates with one or more memories and / or memory devices that store data described herein and / or store software for performing the processes described herein. The graphics processing system may also communicate with a host microprocessor and / or with a display for displaying images based on data generated by the graphics processing system.

[0180] In a particularly preferred embodiment, the various functions of the present invention are performed on a single graphics processing platform that generates and outputs rendered data, which is written, for example, to a frame buffer for a display device.

[0181] The present invention may be implemented in any suitable system such as a suitably configured microprocessor-based system. In a preferred embodiment, the present invention is implemented in a computer and / or microprocessor-based system.

[0182] The various functions of the present invention may be performed in any desired and suitable manner. For example, the functions of the present invention may be implemented in hardware or software as needed. Thus, for example, the various functional elements, stages and pipelines of the present invention may include one or more suitable processors, one or more controllers, functional units, circuits / circuitry, processing logic, microprocessor arrangements, etc., such as dedicated hardware elements or processing circuits and / or programmable hardware elements or processing circuits that can be programmed to operate in a desired manner and / or that can be operated to perform various functions, etc.

[0183] It should also be noted here that, as will be appreciated by those skilled in the art, the various functions of the present invention may be repeated and / or performed in parallel on a given processor, etc. Likewise, the various processing stages may share processing circuitry if desired.

[0184] Therefore, the present invention extends to graphics processors and graphics processing platforms, including devices operating according to any one or more aspects of the present invention described herein. Such graphics processors may originally include any one or more or all of the usual functional units included in a graphics processor, etc., depending on any hardware required to perform the specific functions discussed above.

[0185] It will also be appreciated by those skilled in the art that all aspects and embodiments described herein may, and preferably do, appropriately include any one or more or all of the preferred and optional features described herein.

[0186] The method according to the invention may be implemented at least in part using software such as a computer program. Thus, it can be seen that, when considered from other aspects, the invention provides: computer software, which is particularly suitable for performing the method described herein when installed on a data processing device; a computer program element, which includes a computer software code portion for performing the method described herein when the program element is run on a data processing device; and a computer program, which includes code means suitable for performing all the steps of the method described herein when the program is run on a data processing system. The data processor may be a microprocessor system, a programmable FPGA (field programmable gate array), etc.

[0187] The invention also extends to a computer software carrier comprising such software which, when used to operate a graphics processor, a renderer or a microprocessor system comprising a data processing device, causes the combination with the graphics processing device, the processor, the renderer or the system to perform the steps of the method of the invention. Such a computer software carrier may be a physical storage medium, such as a ROM chip, RAM, flash memory, CD ROM or disk, or may be a signal, such as an electronic signal through a wire, an optical signal or a radio signal, such as a signal to a satellite, etc.

[0188] It should also be understood that not all steps of the method of the present invention need to be performed by computer software, and therefore according to a broader aspect, the present invention provides computer software and such software installed on a computer software carrier for performing at least one of the steps of the method set out herein.

[0189] The present invention may therefore be suitably embodied as a computer program product for use with a computer system. Such a specific implementation may include a series of computer-readable instructions fixed on a tangible medium, such as a non-transitory computer-readable medium, for example a disk, CD ROM, ROM, RAM, flash memory or hard disk. It may also include a series of computer-readable instructions that can be transmitted to a computer system via a modem or other interface device through a tangible medium (including but not limited to optical communication lines or analog communication lines) or passively using wireless technology (including but not limited to microwave, infrared or other transmission technology). The series of computer-readable instructions embodies all or part of the functionality previously described herein.

[0190] Those skilled in the art will appreciate that such computer readable instructions may be written in a variety of programming languages ​​for use with many computer architectures or operating systems. In addition, such instructions may be stored using any current or future memory technology (including but not limited to semiconductor, magnetic, or optical technology), or transmitted using any current or future communication technology (including but not limited to optical, infrared, or microwave technology). It is contemplated that such computer program products may be distributed as removable media with accompanying printed or electronic documentation (e.g., shrink-wrapped software), may be preloaded with a computer system, such as on a system ROM or fixed disk, or may be distributed via a network (e.g., the Internet or the World Wide Web) from a server or electronic bulletin board.

[0191] Several preferred embodiments of the present invention will now be described, by way of example only, and with reference to the accompanying drawings, in which:

[0192] Figure 1 An exemplary computer graphics processing system is shown;

[0193] Figure 2 schematically illustrates a graphics processing pipeline capable of operating in the manner of the techniques described herein;

[0194] Figure 3 A more traditional graphics processor operation is shown, where strict processing barriers are enforced between separate rendering jobs;

[0195] Figure 4 shows improved graphics processor operation according to an embodiment of the present invention, wherein separate rendering jobs may overlap;

[0196] Figure 5A shows a "task completion status" data structure according to a first example, which can be used to track the completion status of a rendering task of an associated rendering job;

[0197] Figure 5B shows a "task completion status" data structure according to a second example, which can be used to track the completion status of a rendering task of an associated rendering job;

[0198] Figure 6 shows an example of a so-called "read-after-write" dependency between two rendering passes;

[0199] Figure 7 is a flow chart illustrating how such data dependencies may be controlled according to an embodiment;

[0200] Fig. 8A Shown according to Figure 5A Use of the "task completion status" data structure;

[0201] Figure 8B Shown according to Figure 5B Usage of the "task completion status" data structure;

[0202] Fig. 9 The operation of a graphics processor according to an embodiment is shown;

[0203] Fig.10 shows an example of a so-called "write-after-write" dependency between two rendering passes; and

[0204] Fig.11 Shows examples of shader side effects that can exist within a rendering pass.

[0205] Figure 1 A typical computer graphics processing system is shown. An application 2, such as a game, executing on a host processor (CPU) 1 will require graphics processing operations to be performed by an associated graphics processing unit (GPU) (graphics processor) 3, which implements the graphics processing pipeline. To do this, the application will generate API (application programming interface) calls, which are interpreted by a driver 4 for the graphics processor 3 running on the host processor 1, generating appropriate commands for the graphics processor 3 to generate the graphics output required by the application 2. To facilitate this, a collection of "commands" will be provided to the graphics processor 3 in response to commands from the application 2 running on the host system 1 for graphics output (e.g., to generate a frame to be displayed).

[0206] like Figure 1 As shown, the graphics processing system will also include a suitable memory system 5 for use by the host CPU 1 and the graphics processor 3.

[0207] When a computer graphics image is to be displayed, it is usually first defined as a series of primitives (polygons), which are then divided (rasterized) into graphics fragments and then used for graphics rendering. During normal graphics rendering operations, the renderer will modify (for example) color (red, green and blue, RGB) and transparency (α, a) data associated with each fragment so that the fragment can be displayed correctly. Once these fragments have completely traversed the renderer, their associated data values ​​are stored in memory ready for output, such as for display.

[0208] In embodiments of the present invention, graphics processing is performed in a pipelined fashion, where one or more pipeline stages operate on data to generate a final output (eg, a displayed frame).

[0209] Figure 2 An exemplary graphics processing pipeline 10 that may be executed by graphics processor 3 is shown according to one embodiment. Figure 2 The graphics processing pipeline 10 shown is a "tile-based" rendering system and will therefore produce tiles of output data arrays, such as output frames to be generated. Therefore, an example will now be described in the context of "tile-based" rendering. Figure 2 In the embodiment of the present invention, rasterization is used to perform rendering, as will be further explained below. However, it should be understood that the present invention is not necessarily limited to rasterization-based rendering, and can generally be used for other types of rendering, including ray tracing or hybrid ray tracing arrangements. Likewise, the present invention is not necessarily limited to tile-based rendering, and can also be used for other types of rendering, including immediate mode rendering arrangements.

[0210] The output data array may typically be an output frame intended for display on a display device such as a screen or printer, but may also, for example, comprise a "render to texture" output of a graphics processor or other suitable arrangement.

[0211] Figure 2 1 shows the main elements and pipeline stages of a graphics processing pipeline 10 according to an embodiment of the present invention. As will be appreciated by those skilled in the art, there may be multiple stages in the graphics processing pipeline. Figure 2 Other elements not shown.

[0212] It should also be pointed out here that Figure 2 are schematic only, and in practice, for example, even if the functional units and pipeline stages shown are Figure 2 are schematically shown as individual stages, but they may also share important hardware circuits. Likewise, there is no need to provide Figure 2 Some of the elements depicted in Figure 2 Only one example of a graphics processing pipeline 10 is shown. It should also be understood that Figure 2 Each of the illustrated stages, elements, units, etc. of the graphics processing pipeline may be implemented as desired and will accordingly include, for example, appropriate circuitry and / or processing logic components, etc. for performing the desired operations and functions.

[0213] like Figure 2 The graphics processing pipeline shown will be executed on and implemented by a graphics processing unit (GPU) 3, which will therefore include functional units, processing circuits, etc. that are capable of operating to perform the necessary graphics processing pipeline stages.

[0214] (It will be appreciated that a graphics processing unit (GPU) (graphics processor) 3 may and typically will include Figure 2Various other functional units, processing circuits, etc. not shown in the figure. This may include various functional units, processing circuits, etc. for performing non-graphics processing work. For example, in addition to graphics processing work, the graphics processing unit (GPU) (graphics processor) 3 may also be capable of operating to perform general "computing" operations, and thus may also include various functional units, processing circuits, etc. that are capable of operating to perform such non-graphics processing work. Therefore, although Figure 2 , but in addition to the fragment shader endpoint 21 described below, the shader core may also include a suitable "compute" shader endpoint that is operable and configured to issue a compute task to the execution engine 31 for processing. For example, the shader core may also include other suitable endpoints as needed that are operable and configured to issue other types of tasks to the execution engine 31 for processing. In this regard, various arrangements will be possible. )

[0215] Figure 2 The stages of the graphics processing pipeline are shown after a stitcher (not shown) of the graphics processor has prepared the primitive list (since the graphics processing pipeline 10 is a tile-based graphics processing pipeline).

[0216] (In practice, the stitcher determines which primitives need to be processed for different regions of the output. In embodiments of the invention, these regions may, for example, represent tiles into which the overall output is divided for processing purposes, or sets of multiple such tiles. To do this, the stitcher compares the position of each primitive to be processed with the positions of the regions, and adds the primitive to a corresponding primitive list for each region into which the stitcher determines that the primitive may (likely) fall. Any suitable and desired technique for sorting and binning primitives into tile lists, such as exact binning or bounding box binning or anything in between, may be used for the stitching process.)

[0217] Once the stitchers have finished preparing the primitive lists (the list of primitives to be processed for each region), each stitcher can be rendered according to its associated primitive list.

[0218] To this end, each tile consists of Figure 2 The graphics processing pipeline stages shown are used for processing.

[0219] Therefore a fragment task iterator 20 is provided for dispatching processing work to the graphics processing pipeline 10 .

[0220] Therefore, the fragment task iterator 20 can schedule the graphics processing pipeline to generate a first output, which can be, for example, a frame to be displayed. In an embodiment of the present invention, where the graphics processing pipeline 10 is a tile-based system, where the output has been divided into a plurality of rendering tiles, the graphics processing pipeline 10 iterates over the set of tiles of the first output, thereby rendering each tile in turn.

[0221] like Figure 2 As shown in the figure, the graphics processor 3 includes an overall controller in the form of a job manager circuit (command stream front-end circuit) 35, which is capable of operating to receive tasks for processing by the graphics processor 3 from the host processor 1, and the job manager 35 can then transmit the relevant jobs to the graphics processor and the corresponding elements of the graphics processing pipeline 10 via appropriate buses / interconnects.

[0222] Therefore, if Figure 2 As shown in , the job manager 35 will, among other things, issue fragment processing tasks to the fragment task iterator 20 for the fragment task iterator 20 to then schedule and dispatch the appropriate fragment shading tasks onto the graphics processing pipeline 10 .

[0223] In an embodiment of the present invention, the graphics processing pipeline 10 is implemented with the aid of appropriate processing ("shader") cores. Specifically, Figure 2 As shown, the graphics processor 3 includes a plurality of "shader" cores, each of which is configured to implement a respective parallel instance of the graphics processing pipeline 10. Accordingly, the fragment task iterator 20 is operable and configured to issue tasks to different ones of the shader cores, for example to attempt to balance processing work between the different shader cores.

[0224] (although Figure 2 Not shown, but there may be various other task iterators that control the issuance of "compute" or other tasks, etc.)

[0225] As will be further explained below, each "shader" core includes a fragment "front end" 30, which may typically be implemented in essentially fixed-function hardware and performs the setup for the fragment shader program, and a programmable stage (execution engine 31), which executes the fragment shader program to perform the actual rendering.

[0226] When a rendering task (ie, a tile) is assigned to a given shader core for processing, the tile is then processed (rendered) accordingly (ie, by the graphics processing pipeline 10).

[0227] For a given tile being processed, the primitive list reader (or "polygon list reader") 22 thereby identifies a series of primitives to be processed for that tile (the primitives listed in the primitive list for that tile), and then issues the ordered sequence of primitives for that tile into the graphics processing pipeline 10 for processing.

[0228] The resource allocator 23 then configures and manages the memory space allocation for depth (Z), color etc. and buffers 33 for the tiles of output being generated. For example, these buffers may be provided as part of RAM located on (locally) the graphics processing pipeline (chip).

[0229] The vertex loader 24 then loads the vertices of the primitives, which are then passed into a primitive setup unit (or "triangle setup unit") 25 which operates to determine edge information representing the edges of the primitives from the vertices of the primitives.

[0230] The edge information of the reordered primitives is then passed to the rasterizer 27, which rasterizes the primitives into a set of one or more sampling positions and generates a single graphics fragment with appropriate positions (representing appropriate sampling positions) according to the primitives to render the primitives.

[0231] The fragments generated by the rasterizer 27 are then sent forward to the rest of the pipeline for processing.

[0232] For example, in an embodiment of the present invention, fragments generated by the rasterizer 27 are subjected to an (early) depth (Z) / stencil test 29 to see if any fragments can be discarded (culled) at this stage. To this end, the Z / stencil test stage 29 compares the depth values ​​of the fragments issued from (associated with) the rasterizer 27 with the depth values ​​of the already rendered fragments (these depth values ​​are stored in a depth (Z) buffer that is part of the tile buffer 33) to determine if the new fragment will be blocked by the already rendered fragments. At the same time, an early stencil test is performed.

[0233] Fragments that pass the fragment early Z and stencil test stage 29 may then undergo additional culling operations, such as a “forward pixel elimination” test, such as described in U.S. Patent Application Publication No. 2019 / 0088009 (Arm Limited), before the remaining fragments are passed to the fragment shading stage (in the form of an execution engine 31) for rendering.

[0234] The processing stages including the primitive list reader (or "polygon list reader") 22 up to the (early) depth (Z) / stencil test 29 together constitute the fragment "front end" 30, which is used to set up the data required to perform fragment processing operations performed by the execution engine 31.

[0235] Execution engine 31 then performs appropriate fragment processing operations on fragments that pass the early Z and stencil tests in order to process the fragments to generate appropriate rendered fragment data.

[0236] The fragment processing may include any suitable and desired fragment shading process, such as executing a fragment shader program for the fragment, applying a texture to the fragment, applying fog or other operations to the fragment, etc., to generate appropriate rendered fragment data.

[0237] Therefore, if Figure 2 As shown in, in an embodiment of the present invention, the execution engine 31 includes a programmable execution unit (engine) 32, which is operable to execute a fragment shader program for a corresponding execution thread (where each thread corresponds to a work item for the output being generated, such as a separate fragment) in order to perform the required fragment shading operations, thereby generating the rendered fragment data. In this regard, the execution unit 32 operates in any suitable and desired manner and includes any suitable and desired processing circuitry, etc.

[0238] In embodiments of the present invention, threads of execution may be arranged into thread "groups" or "bundles," where the threads in a group run in lockstep, one instruction at a time, i.e., each thread in the group executes the same single instruction before moving on to the next. In this way, instruction fetch and scheduling resources may be shared among all threads in the group. Such thread groups may also be referred to as "subgroups," "thread warps," and "wavefronts." For convenience, the term "thread group" will be used herein, but this is intended to encompass all equivalent terms and arrangements unless otherwise specified.

[0239] Accordingly, Figure 2 Also shown is a thread group controller in the form of a warp manager 34, which is configured to control the allocation of work items (e.g., fragments) to corresponding thread groups for programmable execution units 32 to perform fragment shading operations, and to issue thread groups to programmable execution units 32 for corresponding thread groups to execute fragment shading programs.

[0240] like Figure 2 As shown, programmable execution unit 32 is also in communication with memory 5 .

[0241] Once fragment shading is complete, the output rendered (shaded) fragment data is written to the tile buffer 33, from which it can be output to a frame buffer (e.g., in memory 5) for display. The depth value of the output fragment is also appropriately written to a Z-buffer within the tile buffer 33. (The tile buffer stores a color buffer and a depth buffer, each of which stores an appropriate color, etc., or Z-value for each sample position represented by the buffer (essentially for each sample position of the rendered tile being processed).) These buffers store arrays of fragment data representing portions (tiles) of the overall output (e.g., an image to be displayed), where corresponding sets of sample values ​​in the buffer correspond to corresponding pixels of the overall output (e.g., each 2×2 set of sample values ​​may correspond to an output pixel, where 4× multisampling is used).

[0242] As mentioned above, the tile buffer 33 is typically provided as part of RAM located on (locally to) the graphics processor.

[0243] Once a tile has been processed for output, the data in the tile buffer may be written back to an external memory output buffer, such as a frame buffer of a display device (not shown), for example in memory 5. (The display device may include, for example, a display including an array of pixels, such as a computer monitor or printer.)

[0244] The next tile is then processed, and so on, until enough tiles have been processed to generate the entire output (e.g., a frame (image) to be displayed). The process is then repeated for the next output (e.g., frame), and so on.

[0245] For a given output (e.g., a frame (image) to be displayed), a graphics processor will perform at least one rendering pass to generate the output. However, in general, the generation of a given output (e.g., a frame (image) to be displayed) may involve a series of multiple rendering passes (where each rendering pass may include multiple rendering jobs to be performed). This may, for example, include a first rendering pass that generates an "intermediate" output (e.g., such as a "render to texture" output), followed by a second rendering pass that uses the output from the first rendering pass when generating its own output (which may be a "final" output (e.g., a frame (image) to be displayed)).

[0246] Thus, a typical command flow for a tile-based graphics processor may include commands to perform a series of rendering passes, such as the following:

[0247] RUN_FRAG_0

[0248] RUN_FRAG_1 ...

[0250] Each RUN_FRAG command corresponds to a respective fragment processing (rendering) job to be performed using the graphics processing pipeline.

[0251] (Note that, although not shown in this example, a rendering channel used to generate a specific rendering output (e.g., a frame) may generally include a variety of different types of rendering jobs, such that a fragment processing (rendering) job within a rendering channel may also be followed by other types of rendering jobs related to the same rendering output. An example of this may be a case where a "compute" job is used to calculate a "fog" effect for the corresponding rendering output generated by the fragment processing (rendering) job. Therefore, the command stream may also generally include various other types of commands for performing other types of rendering jobs as needed.)

[0252] Although each fragment processing (rendering) job represents a separate instance of implementing the graphics processing pipeline, various data (processing) dependencies may exist between different rendering jobs being executed. For example, where one rendering job generates "intermediate" output to be consumed by the next rendering job, then later rendering jobs should generally be executed after earlier rendering jobs are completed. More specifically, in the case of a tile-based rendering system, where later rendering jobs have data (processing) dependencies on earlier rendering jobs, a particular rendering task for rendering a given tile of a later rendering job should not be executed until an equivalent tile of an earlier rendering job has been processed, making the required portion of the rendering output associated with that rendered tile available.

[0253] Thus, as described above, each rendering job typically includes a plurality of rendering tasks, which may, for example, and in the present embodiment correspond to corresponding rendering tiles, into which the output (rendering target) of a rendering pass is subdivided for rendering purposes. Thus, a command stream including a RUN_FRAG command may be received by a job manager circuit (command stream front-end circuit) 35, which identifies the rendering jobs to be executed, then decomposes these rendering jobs into corresponding smaller rendering tasks (e.g., tiles), and provides them to the fragment task iterator 20, for the fragment task iterator 20 to then schedule and dispatch the appropriate fragment shading tasks onto the graphics processing pipeline 10. (The rendering tasks may then be further decomposed into smaller work items within the shader core.)

[0254] In order to ensure that any data (processing) dependencies between rendering jobs are enforced, the job manager circuit (command stream front-end circuit) 35 can, in some more traditional graphics processing arrangements, enforce a strict (hard) processing barrier between rendering jobs (which barrier can and traditionally is enforced between fragment processing (rendering) jobs of different rendering channels, but can also be enforced between different types of rendering jobs within the same rendering channel), for example so that in the example given above, all rendering tasks associated with the RUN_FRAG_0 command will be issued and processed to completion before the job manager circuit (command stream front-end circuit) 35 allows any rendering tasks associated with the RUN_FRAG_1 command to be issued for processing.

[0255] Figure 3 This more traditional graphics processor operation is shown in Figure 1. This approach works well to ensure the security of graphics processing operations. However, as Figure 3 As shown in Figure 1, the warp utilization is relatively low, especially near the end of a rendering job. This is because, near the end of a rendering job, there may be fewer rendering tasks to be executed compared to the number of available shader cores. (Although Figure 3 (not shown in ), but note that warp utilization may also be relatively low during the initial ramp-up period.) However, due to the hard (strict) processing barriers between rendering jobs, the job manager circuit (command stream front-end circuit) 35 cannot start issuing rendering tasks for the next rendering job for processing, and therefore any shader core that has completed its processing may be idle for several cycles, waiting for other shader cores to complete processing their corresponding rendering tasks.

[0256] Thus, embodiments of the present invention provide a mechanism for safely allowing separate rendering jobs to be issued for processing and processed simultaneously, for example using different shader cores (although processing may also be performed simultaneously on the same shader core, for example as long as forward progress of earlier rendering jobs can be guaranteed). Figure 4 This improvement is shown in the figure. As can be seen from the figure, at the end of the first rendering job (RUN_FRAG_0), the shader core is allowed to start processing the rendering tasks of the next rendering job (RUN_FRAG_1), so that the processing of rendering tasks of different rendering jobs can be overlapped between different shader cores, thereby improving the overall shader core utilization.

[0257] Specifically, in order to do this, and to ensure that any data dependencies between rendering jobs can still be enforced as needed, for each rendering job currently in progress, embodiments of the present invention track the completion status of the corresponding rendering task to be executed for the rendering job in question. Based on this tracking, it is possible to control the processing of rendering tasks of a later (e.g., next) rendering job in a series of rendering jobs being executed.

[0258] Figure 5A 2 shows an example of a “task completion status” data structure 50 that can be used to track the completion status of a rendering task of an associated rendering job. Figure 5A As shown in , the “task completion status” data structure 50 in this example takes the form of a tile-based “scoreboard” that is operable to track the completion status of individual rendering tasks into which the output of the rendering job is subdivided.

[0259] For example, where the rendering output is a 1024×1024 image that is subdivided into a 64×64 array of rendering tiles for rendering purposes, the “task completion status” data structure 50 may include a corresponding 64x64 array of entries, each of which stores a (binary) indication of whether the corresponding task (i.e., tile) associated with the entry has completed its processing. Figure 5A As shown, grey entries 51 indicate tasks (tiles) that have completed their processing and therefore have the corresponding bits set (e.g., to “1”) to indicate this, while white entries 52 indicate tasks (tiles) that have not yet completed their processing (and therefore have the corresponding bits cleared (e.g., set to “0”) for them).

[0260] exist Figure 5A In the "task completion status" data structure 50, the "task completion status" data structure 50 therefore includes a bit array that indicates whether each rendering task (tile) to be performed by the rendering job associated with the "task completion status" data structure 50 has completed their processing. Therefore, the corresponding "task completion status" data structure 50 for the rendering job can be allocated within the fragment task iterator 20 at the beginning of the rendering job and then initialized appropriately. For example, any entries in the "task completion status" data structure 50 corresponding to tasks (tiles) residing outside the area to be written by the rendering job can be initially set (for example) to "1" to effectively indicate that those tasks (tiles) have completed their processing (because there is no processing to be performed, so there are no data (processing) dependencies on those tasks (tiles)), while when the "task completion status" data structure 50 is allocated, all other entries are cleared (for example) to "0", so that when the corresponding rendering tasks are completed, those entries are appropriately updated (and set). Figure 5B1 shows an embodiment of a relatively rough "task completion status" data structure, which can be used to track the completion status of rendering tasks associated with rendering jobs, and shows Figure 5A The example shown in shows the same amount of completion, but instead of storing indicators for individual rendering tasks, the "task completion status" data structure indicates for a rendering task group (tile) whether the rendering task group (tile) as a whole has completed its processing. Figure 5B Each entry 61 in the "task completion status" data structure in represents a corresponding set of rendering tasks (tiles) (16 in this example, but tracking can generally be performed at any suitable and desired granularity).

[0261] exist Figure 5B In , instead of storing a (binary) indicator as to whether the group has completed its processing, for each rendering task (tile) group, it is tracked how many rendering tasks (tiles) in the rendering task (tile) group are still being processed. In this case, the reference count stored in the "task completion status" data structure can be initialized to zero when the data structure is allocated. Then, when a rendering task is issued, the reference count of the corresponding entry will be increased. When a rendering task completes its processing, the corresponding reference counter is decremented accordingly, and once the counter of the corresponding rendering task group (tile) returns to zero, this indicates that the rendering task group (tile) has completed its processing. Therefore, in this example, if Figure 5B As shown, most of the rendering task groups (tiles) have completed their processing, as the corresponding counters have been decremented to zero. However, (and as Figure 5A As shown in FIG. 4 , which shows tracking of individual rendering tasks, there are still rendering tasks within the last two groups of rendering tasks that have not yet completed their processing, and therefore the counters for these groups are non-zero.

[0262] Of course, there are various other possible arrangements for tracking the completion status of rendering tasks or groups thereof.

[0263] Whatever form the "task completion status" data structure takes, it can be used accordingly to control the processing of the rendering tasks (tiles) of the next frame to enforce any desired data (processing) dependencies between rendering tasks. For example, Figure 6 shows an example of a so-called "read-after-write" dependency between two render passes, where a later render pass (RenderPass2) needs to read data from render targets (ImageA and ImageB) that were written to by an earlier render pass (RenderPass1).

[0264] In this case, rendering tasks for any rendering jobs of a later rendering pass (RenderPass2) that read from these rendering targets (images) should therefore not be processed until processing of corresponding rendering tasks of an earlier rendering pass (RenderPass1) is completed, i.e., the relevant parts of the rendering targets (images) have been written accordingly by rendering jobs of the earlier rendering passes before any rendering jobs of the later rendering passes read these parts.

[0265] like Figure 6 As shown in , the "task completion status" data structure ("scoreboard") is therefore updated when the rendering tasks of the rendering jobs of the earlier rendering passes are completed. The rendering jobs of the later rendering passes are therefore able to operate and are configured to appropriately check the scoreboard before performing the required read operations to check whether the relevant rendering tasks have completed their processing. If not, the rendering tasks of the rendering jobs of the later rendering passes can be stopped until the required data is available (i.e., until the corresponding rendering tasks of the rendering jobs of the first rendering pass have completed their processing and the scoreboard is updated accordingly to reflect this).

[0266] Figure 7 is a flow chart illustrating how a scoreboard may be used to control such data dependencies according to an embodiment of the present invention. Specifically, Figure 7 The processing of a particular rendering task within a given rendering job of a particular rendering pass is shown. Thus, the rendering task may be scheduled for processing by the task iterator 20 (at step 70), and processing of the rendering task begins (step 71). At some point during the processing of the rendering task, it may be identified that there is a potential data dependency that needs to be checked before performing a read operation, so the dependency reader routine is started (step 72), as shown below.

[0267] First, check whether the read is for a dependent resource (step 73). If not (step 73-no), i.e., there is no data dependency, the read can be performed immediately (step 77), and in doing so, the task is allowed to proceed further.

[0268] On the other hand, if it is identified that there is a potential data dependency, i.e., the read is for a dependent resource (step 73-yes), then the scoreboard of the earlier rendering job that is potentially dependent is checked (step 74) to see if the dependency is released. If it can be determined that the dependency has been resolved (step 75-yes), i.e., because the scoreboard check (at step 74) indicates that the corresponding rendering task of the earlier rendering job has been completed, then the read can be performed (step 77), and this operation is completed, as described above.

[0269] However, if it cannot be determined whether the dependency is resolved (step 75-no), that is, because the scoreboard check (at step 74) indicates that the corresponding rendering task of the earlier rendering job has not completed its processing, the read cannot be performed, so the read is stopped until it can be determined that the dependency has been resolved (step 76). In this case, when the corresponding rendering task of the earlier rendering job is completed, the associated scoreboard of the earlier rendering pass will be updated so that the dependency is resolved, and the stop will be released at some point so that the read can be performed (step 77), and the task can therefore move forward.

[0270] It should be understood that a single rendering task may spawn many execution threads executing a large number of shader programs, and each thread may perform multiple reads. Each of these memory accesses may require the dependency check described above to be performed (i.e., in steps 73 to 77). Once all threads of the rendering task have completed their processing (step 78), the processing of the rendering task can be completed (step 79), where the tile output is written to memory and the corresponding scoreboard of the rendering job is appropriately updated to indicate that the task has completed its processing (step 710). The rendering task is then completed (step 711).

[0271] Fig. 8A Shown according to Figure 5A In this example, when it is necessary to check the scoreboard bit array for the rendering task (step 80, corresponding to Figure 7 ), identify which position in the scoreboard of the earlier rendering job corresponds to the rendering task that the current rendering task may depend on (step 81) (for example, by calculating which bit in the bit array corresponds to the position of the rendering tile associated with the rendering task).

[0272] If the bit is not set, indicating that the corresponding rendering task of the earlier rendering pass has not yet completed its processing (step 82-no), which means that it cannot be determined that the dependency has been resolved, and therefore the reading should stop. Therefore, the process waits until the scoreboard is updated (step 83) and the scoreboard check can pass (step 84). However, if the bit is set, indicating that the corresponding rendering task of the earlier rendering pass has completed its processing, then based on this, it is determined that the dependency has been resolved and the scoreboard check can pass (step 84).

[0273] Figure 8B Shown according to Figure 5B85), it is identified which region in the scoreboard of the earlier pass includes rendering tasks that the current rendering task may depend on (step 86). If the counter for the region has been decremented to zero (step 87-yes), indicating that the corresponding rendering tasks of the earlier rendering pass have completed their processing, then it is determined based on this that the dependency has been resolved and the scoreboard check can be passed (step 89). On the other hand, if the counter for the region is not equal to zero (step 87-no), indicating that there are at least some rendering tasks that have not yet been completed in the region, then a wait is implemented (step 88) until the scoreboard is updated.

[0274] Various other arrangements are possible, for example depending on the configuration of the scoreboard.

[0275] In this way, it is possible to control the processing of rendering tasks within the shader core to enforce data dependencies. That is, rather than strictly enforcing processing barriers between rendering jobs within the job manager circuit (command stream front end circuit) 35, it is possible to move dependency management between rendering jobs into the graphics processor hardware, where dependencies are checked at the point in time when memory access is required. This is Fig. 9 Shown in. Fig. 9 The operation of a graphics processor according to one embodiment is thus illustrated.

[0276] like Fig. 9 As shown, as described above, the task iterator 20 controls the scheduling and sending of tasks to the shader core for processing through the fragment shader endpoint 21. The fragment shader endpoint 21 of the shader core then issues the rendering task for processing by the graphics processing pipeline being implemented by the corresponding shader core, for example by first executing the fragment front-end processing 30, and then passing any remaining fragments to the execution engine 31 to execute the required fragment shader program. The fragment shader program then performs the required rendering.

[0277] There are various moments in the graphics processing pipeline where memory access may be required and memory access may potentially require access to data buffers written by earlier rendering jobs. According to embodiments of the present invention, as described above, scoreboard 90 can therefore be used (and is) whenever such memory access is required in order to enforce any potential data dependencies between rendering jobs.

[0278] For example, Fig. 9As shown in , the fragment shader endpoint 21 may need to read certain state information or parameters (e.g., variable rate shading parameters) or tile signatures for transaction elimination (e.g., in the form of a CRC buffer) from the memory 5. However, these parameters may in turn be generated by earlier rendering work (e.g., within a previous rendering pass). Therefore, the fragment shader endpoint 21 is able to operate and is configured to check the scoreboard 90 to see if there are any potential data dependencies before reading any such parameters, and if so, whether they have been resolved. (Then it can be used as described above Figure 7 This is done by using the slave reader operation shown in . )

[0279] Similarly, the execution engine 31, when executing a shader program, may require memory access at various points during the execution of the shader program. For example, one example of such a situation is where a rendering task needs to read in a texture via the texture mapper unit 310 of the execution engine 31. However, again, the required texture may have been generated by an earlier rendering job within a previous rendering pass, for example as a "render to texture" output. Therefore, the texture mapper unit 310 may perform the texture read via a suitable scoreboard checker 92, which is operable and configured to check whether the dependency has been resolved.

[0280] A similar situation occurs for reading through the attribute unit 312, which may again be performed via the scoreboard checker 92 in the same manner as described above.

[0281] When the rendering task is completed, the tile write back unit 314 may then write the output (render target) accordingly to the memory 5. This is then appropriately signaled to the fragment shader endpoint 21, which in turn may signal this information to the scoreboard 90 to cause an appropriate update of the scoreboard 90 associated with the current rendering job.

[0282] Therefore, if Fig. 9 As shown, the fragment shader endpoint 21 preferably triggers and controls the updating of the scoreboard 90 for the current rendering job. Then, during the processing of the current rendering job, a scoreboard checker 92 within the shader core is operable and configured to check the corresponding scoreboard 90 of the earlier rendering job when necessary to ensure that any data dependencies are resolved before performing any memory access to the potentially dependent resources.

[0283] superior Figure 6 An example of a so-called "read-after-write" dependency between two rendering passes is shown, which can be handled as described above.

[0284] Fig.10shows an example of a so-called "write-after-write" dependency between two rendering passes, where a later rendering pass rewrites a render target (image) that was also written by an earlier rendering pass. Using the above technique, one can usually Figure 6 These dependencies can be handled using the "read-after-write" dependency shown in .

[0285] However, it should be understood that there may be other situations or types of dependencies that cannot be handled in this manner.

[0286] An example where the job manager circuitry (command stream front-end circuitry) 35 may need to enforce a hard processing barrier would be where a rendering pass includes "side effects". Fig.11 Such an example is shown in , where an early rendering pass performs an "image store" operation on a render target (ImageA) that a later rendering pass is to read. This image store operation may exist within any of the rendering tasks of the earlier rendering pass, and this may not be known in advance, and so in this case it is generally unsafe to start processing a later rendering pass until all rendering tasks of the earlier rendering pass have completed.

[0287] Another example of this situation might be a "read-after-write" dependency, where a rendering job of a later rendering pass may write to a rendering target (image) that a rendering job of an earlier rendering pass may need to read from. In this case, the read operation may generally be unconstrained, such that the read may be performed as part of any of the rendering tasks for the rendering jobs of the earlier rendering passes. In this case, it may not be safe to start issuing any rendering tasks for the later rendering pass, and so the job manager circuitry (command stream front-end circuitry) 35 may enforce a stricter (harder) processing barrier.

[0288] Various other examples are possible.

[0289] The above specific embodiments are presented for the purpose of illustration and description. It is not intended to be exhaustive or limit the technology described herein to the precise form disclosed. In accordance with the above teachings, many modifications and variations are possible. The embodiments are selected to best explain the principles of the technology described herein and its practical application, so that other technical personnel in the field can best utilize the technology described herein in various embodiments and through various modifications suitable for the specific purpose envisioned. The scope of the present invention is intended to be defined by the appended claims.

Claims

1. A method of operating a graphics processor while executing a series of rendering jobs, the graphics processor comprising a set of one or more processing cores, the method comprising: When executing a series of rendering jobs: wherein each rendering job in the series of rendering jobs comprises a respective set of one or more rendering tasks to be processed to obtain an overall output of the rendering job, and wherein at least some processing of rendering tasks of a later rendering job in the series of rendering jobs can be dependent on processing of rendering tasks of an earlier rendering job in the series of rendering jobs: issuing a rendering task of a first rendering job in the series of rendering jobs to a processing core of the graphics processor for processing; as well as Tracking the completion status of the rendering task of the first rendering job; The method further comprises: issuing a rendering task of a second, later rendering job in the series of rendering jobs to a processing core of the graphics processor for simultaneous processing with the rendering task of the first rendering job; as well as The processing of the rendering tasks of the second rendering job is controlled based on the tracking of the completion status of the rendering tasks of the first rendering job to enforce a dependency between the first rendering job and the second rendering job.

2. The method according to claim 1 includes identifying whether there is a potential data dependency between the rendering tasks of the second rendering job and the rendering tasks of the first rendering job, and when it is identified that there is a potential data dependency between the rendering tasks of the second rendering job and the rendering tasks of the first rendering job: controlling the processing of the rendering tasks of the second rendering job to enforce the potential data dependency.

3. The method of claim 2, wherein controlling the processing of the rendering tasks of the second rendering job to enforce the potential data dependency comprises stopping the processing of the rendering tasks of the second rendering job until it is determined based on the tracking of the completion status of the rendering tasks of the first rendering job that the rendering tasks of the second rendering job depend on have completed their processing.

4. A method according to claim 2 or 3, wherein the identifying whether there is a potential data dependency between the rendering tasks of the second rendering job and the rendering tasks of the first rendering job is performed before issuing the rendering tasks of the second rendering job for processing, and wherein controlling the processing of the rendering tasks of the second rendering job to enforce the potential data dependency includes stopping issuing the rendering tasks for processing until it is determined based on the tracking of the completion status of the rendering tasks of the first rendering job that the rendering tasks of the second rendering job depend on have completed their processing.

5. A method according to claim 2 or 3, wherein the identifying whether there is a potential data dependency between the rendering task of the second rendering job and the rendering task of the first rendering job is performed during the processing of the rendering task of the second rendering job and before the rendering task of the second rendering job performs a memory access that can depend on the rendering task of the first rendering job, and wherein controlling the processing of the rendering task of the second rendering job to enforce the potential data dependency includes stopping the memory access until it is determined based on the tracking of the completion status of the rendering task of the first rendering job that the rendering task of the second rendering job depends on has completed its processing.

6. The method according to claim 2 or 3, wherein when it is identified that there is no potential data dependency between the rendering tasks of the second rendering job and the rendering tasks of the first rendering job, the method includes continuing to process the rendering tasks of the second rendering job.

7. The method of claim 2 or 3, wherein identifying whether there is a potential data dependency between the rendering task of the second rendering job and the rendering task of the first rendering job comprises determining whether the rendering task of the second rendering job needs to access any data buffer potentially written by the rendering task of the first rendering job.

8. A method according to claim 1, 2 or 3, wherein tracking the completion status of the rendering tasks of the first rendering job uses a task completion status data structure associated with the first rendering job, and the task completion status data structure associated with the rendering job stores corresponding indicators of whether the rendering tasks of the rendering job have completed their processing. 9 . The method of claim 8 , wherein the task completion status data structure associated with the first rendering job is updated when rendering tasks being executed for the first rendering job complete their processing.

10. The method of claim 1, 2 or 3, wherein the graphics processor is configured to perform tile-based graphics processing, wherein a rendering output is subdivided into a plurality of rendering tiles.

11. A graphics processor, comprising: A collection of one or more processing cores; a task issuing circuit operable to control issuing of rendering tasks to the plurality of processing cores for processing; and A control circuit operable and configured to: When the graphics processor is executing a series of rendering jobs: wherein each rendering job in the series of rendering jobs comprises a respective set of one or more rendering tasks to be processed to obtain an overall output of the rendering job, and wherein at least some processing of rendering tasks of a later rendering job in the series of rendering jobs can be dependent on processing of rendering tasks of an earlier rendering job in the series of rendering jobs: controlling the operation of the graphics processor to: permitting the task issuing circuit to begin issuing rendering tasks of a later rendering job in a series of rendering jobs being executed to a processing core of the graphics processor for processing without waiting for all of the rendering tasks of an earlier rendering job in the series of rendering jobs being executed to have completed their processing, so that the rendering tasks of the later rendering job can be processed simultaneously with the rendering tasks of the earlier rendering jobs; The control circuit is further configured to: Tracking the completion status of rendering tasks of at least some rendering jobs, and using the tracking of the completion status of rendering tasks of early rendering jobs in a series of rendering jobs being executed to control the processing of rendering tasks of later rendering jobs in the series of rendering jobs to enforce dependencies between different rendering jobs.

12. The graphics processor of claim 11 , wherein the control circuit is configured to: identify whether there is a potential data dependency between a rendering task of the second rendering job and a rendering task of the first rendering job, and when it is identified that there is a potential data dependency between the rendering task of the second rendering job and the rendering task of the first rendering job: control processing of the rendering task of the second rendering job to enforce the potential data dependency.

13. The graphics processor of claim 12 , wherein controlling the processing of the rendering tasks of the second rendering job to enforce the potential data dependency comprises stopping the processing of the rendering tasks of the second rendering job until it is determined based on the tracking of the completion status of the rendering tasks of the first rendering job that the rendering tasks of the second rendering job depend on have completed their processing.

14. A graphics processor according to claim 12 or 13, wherein the identifying whether there is a potential data dependency between the rendering task of the second rendering job and the rendering task of the first rendering job is performed before the task issuing circuit issues the rendering task of the second rendering job for processing, and wherein controlling the processing of the rendering task of the second rendering job to enforce the potential data dependency includes the task issuing circuit stopping issuing the rendering task for processing until it is determined, based on the tracking of the completion status of the rendering task of the first rendering job, that the rendering task of the second rendering job depends on has completed its processing.

15. The graphics of claim 12 or 13, wherein the identifying whether a potential data dependency exists between the rendering task of the second rendering job and the rendering task of the first rendering job is performed during processing of the rendering task of the second rendering job by a processing core of the graphics processor and before the rendering task of the second rendering job performs a memory access that is dependent on the rendering task of the first rendering job, and wherein controlling the processing of the rendering task of the second rendering job to enforce the potential data dependency includes stopping the memory access until it is determined based on the tracking of the completion status of the rendering task of the first rendering job that the rendering task of the second rendering job depends on has completed its processing.

16. The graphics processor of claim 12 or 13, wherein when it is identified that there is no potential data dependency between the rendering tasks of the second rendering job and the rendering tasks of the first rendering job, the control circuit is configured to allow the rendering tasks of the second rendering job to continue to be processed.

17. A graphics processor according to claim 12 or 13, wherein identifying whether there is a potential data dependency between the rendering task of the second rendering job and the rendering task of the first rendering job includes determining whether the rendering task of the second rendering job needs to access any data buffer potentially written by the rendering task of the first rendering job.

18. A graphics processor according to claim 11, 12 or 13, wherein the control circuit is configured to track the completion status of the rendering tasks of the rendering job using a corresponding task completion status data structure associated with the rendering job, and the task completion status data structure associated with the rendering job stores corresponding indicators of whether the rendering tasks of the rendering job have completed their processing.

19. The graphics processor of claim 18, wherein the task completion status data structure associated with a rendering job is updated when rendering tasks being executed for the rendering job complete their processing.

20. The graphics processor of claim 11, 12 or 13, wherein the graphics processor is configured to perform tile-based graphics processing, wherein a rendering output is subdivided into a plurality of rendering tiles.

21. A computer program product comprising instructions which, when executed by a data processor, will cause the data processor to perform the method according to claim 1, 2 or 3.

Images