Resource allocation in parallel processing systems

Abstract

A method of resource allocation in a parallel processing system is described. The method includes receiving a request to allocate resources for a task, where the request identifies an amount of resources needed to perform only a next block of the task, and when resources are available, allocating to the task the amount of resources needed to perform the next block of the task.

Description

Background Technology

[0001] A graphics processing unit (GPU) comprises a highly parallel architecture designed to efficiently process large amounts of data in parallel. GPUs are commonly used for computer graphics (e.g., rendering images on a screen); however, GPUs can also be used for other operations that benefit from the parallelism they provide.

[0002] In parallel processing systems (e.g., systems that include GPUs and memory), the necessary resources (e.g., registers and / or memory) are allocated to a task when it is ready to be executed. This reduces the idle time of functional units when the number of tasks ready to be executed is large.

[0003] The implementation schemes described below are provided by way of example only and do not constitute a limitation on the ways in which to address any or all of the shortcomings of known resource allocation methods. Summary of the Invention

[0004] This summary is provided to introduce, in a simplified form, a series of concepts further described below in the detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0005] This invention describes a resource allocation method in a parallel processing system. The method includes receiving a request to allocate resources for a task, wherein the request identifies the amount of resources required to execute only the next chunk of the task, and, when resources are available, allocating the amount of resources required to execute the next chunk of the task.

[0006] A first aspect provides a method for allocating resources for a task in a parallel processing system, the method comprising: receiving a request to allocate resources for the task, the request identifying only the amount of resources required to execute the next block of the task, and wherein the task is partitioned into multiple blocks; determining whether the amount of resources required to execute the next block of the task is available; pausing the task in response to determining that the amount of resources required to execute the next block of the task is unavailable; and allocating resources to the task in response to determining that the amount of resources required to execute the next block of the task is available.

[0007] A second aspect provides a method for allocating resources for a task in a parallel processing system, the method comprising: receiving a request to allocate resources for the task, the request identifying both the amount of resources required to execute only the next block of the task and the amount of resources required to execute the entire task, wherein the task is partitioned into multiple blocks; determining whether the amount of resources required to execute the entire task has already been allocated to the task; and in response to determining that the amount of resources required to execute the entire task has not been allocated to the task: determining whether the amount of resources required to execute the entire task is available; and in response to determining that the amount of resources required to execute the entire task is available, allocating resources to the task; and in response to determining that the amount of resources required to execute the entire task is unavailable: determining whether the amount of resources required to execute the next block of the task is available; and in response to determining that the amount of resources required to execute the next block of the task is unavailable, suspending the task; and in response to determining that the amount of resources required to execute the next block of the task is available, allocating resources to the task.

[0008] A third aspect provides a method for allocating resources for a task in a parallel processing system, the method comprising: receiving a request to allocate resources for a task, the request identifying both the amount of resources required to execute only the next block of the task and the amount of resources required to execute the entire task, wherein the task is partitioned into multiple blocks; and in a first operating mode: determining whether the amount of resources required to execute the entire task has been allocated to the task; and in response to determining that the amount of resources required to execute the entire task has not been allocated to the task: determining whether the amount of resources required to execute the entire task is available; and in response to determining that the amount of resources required to execute the entire task is available, allocating resources to the task; and in response to determining that the amount of resources required to execute the entire task is unavailable: determining whether the amount of resources required to execute the next block of the task is available; and in response to determining that... If the resources required to execute the next block of the task are unavailable, the task is paused; and in response to determining that the resources required to execute the next block of the task are available, resources are allocated to the task; and in the second operating mode: it is determined whether the resources required to execute the entire task have already been allocated to the task; and in response to determining that the resources required to execute the entire task have already been allocated to the task, the resource allocation to the task is reduced to the resources required to execute the next block of the task; and in response to determining that the resources required to execute the entire task have not yet been allocated to the task: it is determined whether the resources required to execute the next block of the task are available; and in response to determining that the resources required to execute the next block of the task are unavailable, the task is paused; and in response to determining that the resources required to execute the next block of the task are available, resources are allocated to the task.

[0009] The fourth aspect provides a method for partitioning a program into multiple blocks, the method comprising: analyzing the program’s resource requirements over time against one or more thresholds in a compiler; inserting initial instructions into the program; and inserting instructions into the program in the program execution sequence at a position before the resource requirements first exceed the thresholds.

[0010] The fifth aspect provides a method for partitioning a program into multiple blocks, the method comprising: receiving metadata associated with the program in an instruction sequencer, the metadata identifying one or more instructions in the program and resource requirements associated with those instructions; monitoring the execution of the program; and identifying block boundaries in response to determining that the execution of the program has reached an instruction in which the resource requirements first exceed a threshold in the program execution sequence.

[0011] A sixth aspect provides a parallel processing system comprising: one or more resources used when executing a task, wherein the task is partitioned into multiple blocks; a task scheduler configured to receive a request to allocate resources for a task, the request identifying only the amount of resources required to execute the next block of the task, determining whether the amount of resources required to execute the next block of the task is available, suspending the task in response to determining that the amount of resources required to execute the next block of the task is unavailable, and allocating resources to the task in response to determining that the amount of resources required to execute the next block of the task is available; and one or more execution modules configured to execute each block of the task.

[0012] A seventh aspect provides a parallel processing system comprising: one or more resources used when executing a task, wherein the task is partitioned into multiple blocks; a task scheduler configured to receive a request to allocate resources for the task, the request identifying both the amount of resources required to execute only the next block of the task and the amount of resources required to execute the entire task; determining whether the amount of resources required to execute the entire task has already been allocated to the task; and in response to determining that the amount of resources required to execute the entire task has not been allocated to the task: determining whether the amount of resources required to execute the entire task is available; and in response to determining that the amount of resources required to execute the entire task is available, allocating resources to the task; and in response to determining that the amount of resources required to execute the entire task is unavailable: determining whether the amount of resources required to execute the next block of the task is available; and in response to determining that the amount of resources required to execute the next block of the task is unavailable, suspending the task; and in response to determining that the amount of resources required to execute the next block of the task is available, allocating resources to the task.

[0013] An eighth aspect provides a parallel processing system, comprising: one or more resources used when executing a task, wherein the task is divided into multiple blocks; a task scheduler arranged to receive a request to allocate resources for a task, the request identifying both the amount of resources required to execute only the next block of the task and the amount of resources required to execute the entire task, and further arranged to: determine in a first operating mode whether the amount of resources required to execute the entire task has already been allocated to the task; and in response to determining that the amount of resources required to execute the entire task has not been allocated to the task: determine whether the amount of resources required to execute the entire task is available; and in response to determining that the amount of resources required to execute the entire task is available, allocate resources to the task; and in response to determining that the amount of resources required to execute the entire task is unavailable: determine whether the amount of resources required to execute the next block of the task is available. In response to determining that the resources required to execute the next block of the task are unavailable, the task is paused; and in response to determining that the resources required to execute the next block of the task are available, resources are allocated to the task; and further arranged in a second operating mode: determining whether the resources required to execute the entire task have already been allocated to the task; and in response to determining that the resources required to execute the entire task have already been allocated to the task, reducing the resource allocation to the task to the resources required to execute the next block of the task; and in response to determining that the resources required to execute the entire task have not yet been allocated to the task: determining whether the resources required to execute the next block of the task are available; and in response to determining that the resources required to execute the next block of the task are unavailable, the task is paused; and in response to determining that the resources required to execute the next block of the task are available, resources are allocated to the task.

[0014] The task scheduler can be the task scheduler in the GPU.

[0015] Tasks can be associated with shaders.

[0016] One or more resources may include multiple registers.

[0017] Another aspect provides a parallel processing system configured to execute any of the methods described herein, and computer-readable code configured to cause any of the methods to execute when the code is run.

[0018] The processor described herein can be embodied in hardware on an integrated circuit. A method for manufacturing a processor as described herein can be provided at an integrated circuit manufacturing system. An integrated circuit definition dataset can be provided, which, when processed in an integrated circuit manufacturing system, causes the system to manufacture the processor described herein. A non-transitory computer-readable storage medium storing a computer-readable description of an integrated circuit, which, when processed, causes a layout processing system to generate a circuit layout description used in the integrated circuit manufacturing system to manufacture the processor described herein.

[0019] An integrated circuit manufacturing system may be provided, comprising: a non-transitory computer-readable storage medium storing a computer-readable integrated circuit description describing the processor described herein; a layout processing system configured to process the integrated circuit description to generate a circuit layout description of an integrated circuit embodying the processor; and an integrated circuit generation system configured to manufacture the processor according to the circuit layout description.

[0020] Computer program code for performing any of the methods described herein may be provided. A non-transitory computer-readable storage medium may be provided having computer-readable instructions stored thereon, which, when executed at a computer system, cause the computer system to perform any of the methods described herein.

[0021] As will be apparent to those skilled in the art, the above features can be appropriately combined, and can be combined with any aspect of the examples described herein. Attached Figure Description

[0022] The example will now be described in detail with reference to the accompanying drawings, in which:

[0023] Figure 1 This is a flowchart of a first exemplary method for resource allocation;

[0024] Figure 2 It shows Figure 1 An exemplary implementation of the method;

[0025] Figure 3 This is a flowchart illustrating an exemplary method for partitioning a program;

[0026] Figure 4 and Figure 5 It shows Figure 3 Various exemplary implementations of the method;

[0027] Figure 6 This is a flowchart of a second exemplary method for resource allocation;

[0028] Figure 7 This is a flowchart of a third exemplary method for resource allocation;

[0029] Figure 8 It shows Figure 1 Another exemplary implementation of the method;

[0030] Figure 9 It shows Figure 7 An exemplary implementation of the method;

[0031] Figure 10 This is a flowchart of the fourth exemplary method for resource allocation;

[0032] Figure 11 This is a schematic diagram of the processing system;

[0033] Figure 12 A computer system in which the methods described herein can be implemented is shown; and

[0034] Figure 13 An integrated circuit manufacturing system for generating integrated circuits embodying the processing system described herein is shown.

[0035] The accompanying drawings illustrate various examples. Those skilled in the art will understand that the element boundaries (e.g., boxes, groups of boxes, or other shapes) shown in the drawings represent one example of a boundary. In some examples, it may be that one element can be designed as multiple elements, or multiple elements can be designed as one element. Where appropriate, common reference numerals are used throughout the drawings to indicate similar features. Detailed Implementation

[0036] The following description is given by way of example to enable those skilled in the art to make and use the invention. The invention is not limited to the embodiments described herein, and various modifications to the disclosed embodiments will be readily apparent to those skilled in the art.

[0037] The implementation scheme will now be described using examples only.

[0038] As described above, when a task is ready to execute in a parallel processing system (e.g., a system including a GPU and memory), the required resources (e.g., registers and / or memory and / or other hardware resources, such as the processor) are allocated to that task. This reduces idling of functional units when the number of tasks ready to be executed is large. However, when a program (e.g., a shader) has greater resource requirements, the number of tasks that can be in a "ready" state can be limited (e.g., because there are no resources available to allocate to additional tasks), which can lead to pipeline starvation and thus reduce throughput.

[0039] This document describes various methods and apparatuses for resource allocation (e.g., register allocation, memory allocation, or allocation of other hardware resources such as coprocessors) that enable more tasks to be in a “ready” state at any given time. As described in more detail below, one method involves partitioning a task into multiple blocks based on changes in resource requirements, such that a task in a “ready” state is allocated at least the resources required to execute the next block of that task. At the end of a block, resources not needed by the next block of the task can be released and thus allocated to a different task. If a subsequent block requires additional resources (i.e., resources beyond those already allocated to the task), these resources are requested and, if available, allocated to the task, allowing the task to remain in a “ready” state. However, if the requested additional resources are unavailable, the task will halt until more resources become available and the additional resources required by the next block can be allocated to the task.

[0040] When a task has varying resource requirements over its execution time, the methods described herein can be used to partition the task into blocks based on these varying resource requirements. For tasks without such time-varying resource requirements, known methods can be used to perform resource allocation for the task (e.g., by allocating the resources required to execute the entire task from the outset). The methods described herein can be used in conjunction with such known methods (e.g., as shown in some examples described below).

[0041] The method described herein can be implemented on any system that runs multiple tasks in parallel, whether the parallel operation is achieved using time-slicing (e.g., fine-grained task interleaving, such as by looping between a small number of tasks in a task pool cycle-by-cycle) or by running tasks in parallel hardware. In various examples, the method can be implemented as a system including a GPU and memory, and can be implemented within the GPU. In other examples, the method can be implemented within the CPU.

[0042] By using the methods and apparatus described herein for tasks that can be partitioned into blocks, resources required to execute the entire task can be allocated partially before execution begins. Furthermore, some resources can be released during task execution, ensuring that these resources are not held up for the entire duration of task execution. This partial allocation and / or partial release of resources can improve resource utilization efficiency and increase the number of tasks that can be in a "ready" state at any given time. This, in turn, can increase the throughput of the parallel processing system. Additionally or alternatively, the use of the methods and apparatus described herein can allow parallel processing systems to be designed with fewer resources (without compromising throughput), and thus save power and area.

[0043] In this paper, the term "ready" is used in the context of tasks and refers to those tasks for which all task dependencies have been satisfied and the resources required to execute at least the next block of the task have been allocated to the task.

[0044] The term 'task' is used herein to refer to a group of data items and the work to be performed on those data items. For example, in a Single Instruction Multiple Data (SIMD) processing system, in addition to the dataset to be processed according to a program, a task may also include the program or a reference to the program (e.g., an ALU instruction or the same sequence of references thereof) or associated with the program or a reference to the program, wherein the dataset may include one or more data elements (or data items, e.g., multiple pixels or vertices).

[0045] The term 'program instance' is used herein to refer to the individual instances of code through which a path passes. Thus, a program instance refers to a single data item and a reference (e.g., a pointer) to the program that will be executed on that data item. Therefore, a task can be viewed as comprising multiple program instances (e.g., up to 32 program instances), but in practice, each task requires only a single instance of a common program (or reference). Therefore, there exists a terminology hierarchy where a task comprises one or more program instances.

[0046] This article uses the term "block" in relation to a task, implying that a task comprises one or more blocks. A block differs from a program instance in that a block relates to a portion of the program that the task will execute (rather than to a subset of a group of data items).

[0047] Programs typically perform operations on values ​​stored in registers, and each program instance requires its own copy of each register value. Each program can use (or reference) multiple registers, and multiple tasks can run in parallel on a processing system; therefore, the method described in this paper can be used to provide a flexible way to allocate a relatively large number of registers.

[0048] You can refer to this. Figure 1 A first method for resource allocation is described. This method includes receiving a request from a task for resource allocation, wherein the task has been partitioned into multiple blocks, and the request identifies the resources required to execute the next block of the task (box 102). As described above, a block of a task is an appropriate subset of the task (i.e., not the entire task, but a part of it), and in this first method, the task is partitioned into blocks if resource requirements increase beyond a predefined threshold. If only a single threshold exists, the task is partitioned into two blocks; however, in various examples, more than one threshold may exist, and the task may be partitioned into two or more blocks. The partitioning of a task into blocks is described in more detail below, which can be performed either during or after compilation of the program associated with the task.

[0049] In response to a received request (in box 102), if the resources required to execute the next block are available, they are allocated ("Yes" in box 104, then box 106). This means the next block of the task is now ready to execute, and the task can be placed in (or held in) the "Ready" state. If the resources are unavailable ("No" in box 104), the task is suspended (box 108) until resources for the next block become available and can be allocated (box 106). When a task is suspended (or stopped), it does not enter the "Ready" state, and if the task is already in the "Ready" state, it is removed from that state and placed in another unready state, which may be called the waiting state.

[0050] Since only the resources needed for the next block have been allocated (in box 106), after executing that particular block, such as block n (where the task is partitioned into N tasks, n = [0, ..., N-1] and N > 1), there may not be enough resources to execute the block immediately following the task (e.g., block n+1). Therefore, another request is received that requests the allocation of resources for the next block to be executed (e.g., block n+1). This method is repeated (as follows). Figure 1 (As shown by the arrows from box 106 to box 102), until resources have been allocated for the last block in the task (e.g., block N-1).

[0051] An exemplary implementation of the first method is as follows: Figure 2 As shown. Although this example only illustrates two tasks, Task 0 and Task 1, for illustrative purposes, it should be understood that in reality, there may be a large number of tasks being processed at any given time. Figure 2As shown, requests are received from two tasks (Task 0 and Task 1). Task 0 is partitioned into two blocks (N=2), and the received request (in box 102) is a request for the amount of resources x required to allocate only the first block (block 0 of Task 0) for executing the task. Task 1 is not partitioned into blocks, so the request is a full allocation of resources for the task, also in the amount x. For the purposes of this example, it is assumed that the available resources correspond to 2x, so the resources can be evenly distributed between Task 0 and Task 1 (allocating x to each), and the two tasks can be executed in parallel. When the first block of Task 0 finishes execution (at time t=t1), another request is received from Task 0, requesting resources for the next block (block 1 of Task 0), and this second block requires a large amount of resources, 2x. Since there are not enough resources available for the next block, Task 0 is suspended. When Task 1 completes (at time t=t1+t2), the resources used by Task 1 are released, and additional resources x can be allocated to Task 0 so that Block 1 of Task 0 can be executed. This allows tasks 0 and 1 to complete at time t1+t2+t3. Conversely, if all resources (2x) are initially allocated to task 0, task 1 will stop at time t1+t3, and task 1 will not complete until time 2t1+t2+t3.

[0052] like Figure 2 As the example shows, by allocating only the resources needed for the next block to a task (e.g., to task 0), instead of allocating the resources needed for the entire task, more tasks can be executed in parallel and the overall throughput can be increased because resources, including the execution pipeline, are used more efficiently.

[0053] To implement this method, the task (e.g., at least one task) must be partitioned into multiple blocks based on increasing resource requirements as the task progresses. Task partitioning can also be done if the associated program (e.g., a shader) itself can be partitioned into blocks based on changes in resource requirements. See also... Figures 3 to 5 Describe an exemplary method for partitioning a program (e.g., a shader) into multiple blocks.

[0054] Figure 3 This is a flowchart of an exemplary method for partitioning a program into blocks, which has the effect of partitioning all tasks associated with the program into blocks. As mentioned above, if a task involves executing the program on a set of data items, then that task is associated with the program. This method can be executed by the compiler at compile time, such that all tasks associated with the same program are partitioned in the same way.

[0055] In various examples, the task referred to in this article can be a task associated with a shader executing on the GPU. In other examples, the task can be associated with other programs (i.e., not shaders).

[0056] like Figure 3 As shown, the method includes analyzing the resource requirements of a program for one or more thresholds (box 302), and adding initial instructions to the program at the start (or near the start) of the program, followed by adding additional instructions (box 304) before the resource requirements first exceed each threshold, for example, before the resource requirements first increase from below the threshold to above the threshold during the program execution flow. It is these newly added instructions that partition the program into blocks, where all instructions between the first and second added instructions constitute the first block (block 0), all instructions between the second and third newly added instructions (or at the end of the program, where only two newly added instructions are added) constitute the second block (block 1), and so on. When the program is executed, as described above, each newly added instruction causes a request for resource allocation to be sent to the scheduler (e.g., as...). Figure 1 (Received in frame 102).

[0057] Figure 4 and Figure 5 A diagram illustrating an exemplary resource distribution for the program is shown. Figure 4 Two exemplary distributions 402 and 404 are shown, along with only one threshold (set as x1). A first exemplary program with a higher distribution 402 is divided into two blocks by adding a new instruction before the resource requirement exceeds this threshold (at I1). The point where the program moves from one block to the next (e.g., the location where a new instruction is added) can be referred to as the block boundary. A second exemplary program with a lower distribution 404 is also divided into two blocks at the point before the resource requirement crosses the threshold (at I2). For this second exemplary program, the resource requirement subsequently falls below the threshold, and for this first method (as... Figure 1 As shown in the diagram, this has no effect because the program only partitions the program before the resource requirement first exceeds the threshold in the program execution sequence. Figure 5 A third exemplary distribution 502 and two thresholds (set as x1 and x2) are shown, and the program is divided into three blocks by inserting new instructions (in box 304) at the beginning and before each of positions I4 and I5 in the execution sequence.

[0058] It should be understood that Figure 4 and Figure 5 The graphics are provided only in an interpretive manner and mainly involve the use of... Figure 1 Methods such as register allocation are implementations that allocate resources of only a single type. When using this method to allocate resources of more than one type (e.g., registers and memory) simultaneously, multiple allocations can exist for the same program (e.g., one for registers, one for memory), and a threshold can be defined based on both types (e.g., set to x for registers).i and y for memory i (A combination of thresholds), or there may be multiple thresholds, where a subset of the thresholds is associated with each type of resource.

[0059] The location of the threshold (and thus the partition granularity) can be set based on the architecture of the hardware on which the task will be performed. For example, when the allocated resource is registers, the threshold can be set at a granularity corresponding to the number of memory banks in the memory used to implement the registers, or it can be set at a finer granular level (e.g., every two registers). In various examples, the program can be compiled more than once in different ways, for example with different thresholds, depending on the hardware on which the task will be performed. In this way, different compiled versions of the program can be partitioned in different ways.

[0060] When the program is compiled, it can achieve Figure 3 One approach is to analyze the program after compilation. However, in another example, program analysis can be performed after compilation, and this analysis can be based on a table of instruction counts and resource requirements (e.g., instruction 0, resource requirement = 10, instruction 22, resource requirement = 22, etc.). This table can be provided as metadata along with the program and can be generated during compilation or at another time. With this table, the instruction fetching unit (also known as the instruction sequencer) can count instructions and / or track the program counter, and notify the scheduler when a threshold is crossed and thus a block boundary is reached.

[0061] The first resource allocation method described above involves partially allocating resources to tasks based on the resource requirements of the next block. In addition to, or instead of, performing partial resource allocation, partial resource release can be performed; that is, after executing one block of the task, releasing those resources not needed for the next block of the task. See also... Figure 6 The second resource allocation method described involves the partial allocation and partial release of resources.

[0062] The second method is similar to the first method (e.g.) Figure 1 The process begins in the same manner as shown, receiving a resource allocation request from the task, where the request identifies the resources required to execute the next block of the task (box 102). In response to receiving the request (box 102), if the resources required to execute the next block are available and the remaining resources are released, the resources required to execute the next block are allocated (boxes 602 to 606 and boxes 104 to 108). The remaining resources released will depend on the specific needs of the next block, and the amount released does not necessarily match any combination of previously allocated resource blocks.

[0063] Figure 6An example of a more detailed implementation of this partial allocation and release of resources is shown. For example... Figure 6 As shown, after receiving a request (in box 102), it is determined whether there are any remaining resources (box 602), i.e., resources allocated to a task to execute a previous block (e.g., block n-1) but not needed to execute the next block (e.g., block n). If any remaining resources exist ("Yes" in box 602), they are released (box 604) and can then be allocated to another task. The method then determines whether any additional resources are needed to execute the next block (box 606), and if so, allocates these resources in the same manner as the first method (boxes 104-108). Although Figure 3 The diagram shows the release of remaining resources (in box 604) before allocating additional resources (in box 106), but it should be understood that these two operations can be implemented in other orders or substantially in parallel. As with the first method, the second method is repeated for each block in the task (as indicated by the arrows returning from boxes 606 and 106 to box 102).

[0064] In the case of implementing both partial allocation and partial release, the partitioning of the program (and therefore the task) is implemented in a slightly different manner than described above, as in the second resource allocation method. See again... Figure 3 The method shown, ( Figure 6 The second resource allocation method requires adding additional instructions after the initial instruction, before the point where resource demand exceeds a threshold, and after the point where resource demand falls below a threshold. As previously mentioned, when the program is executed, each newly added instruction causes a request for resource allocation to be sent to the scheduler (e.g., as...). Figure 3 (Received in box 102). In cases where resource demand oscillates around a threshold, other criteria can be specified to limit the number of blocks and / or the minimum block size, thereby limiting the number and / or interval of resource allocation requests sent to the scheduler.

[0065] refer to Figure 4 The lower exemplary resource distribution 404 shown, when using both partial allocation and partial release, can divide the program into three blocks by adding a new instruction before the point where the resource demand exceeds the threshold (at I1), and adding another new instruction after the point where the resource demand subsequently falls below the threshold (at I3).

[0066] It should be understood that even when the program is partitioned by adding instructions whenever resource demand crosses a threshold (whether from below or above), the resource allocation method can be configured to perform only partial allocation (e.g., Figure 3 As shown, boxes 602 and 604 are omitted, and an arrow from box 102 to box 606 is added), or it can be configured to implement both partial allocation and partial release (as shown). Figure 3 (As shown).

[0067] In other examples, resource allocation methods can be configured to achieve only partial release. In such examples, the program can be partitioned into blocks by adding initial instructions (at or near the start of the program) that trigger a request for the maximum amount of resources required to execute the entire program, and then adding additional instructions only after the last point in the program where resource requirements fall below a threshold, and this can be modified... Figure 3 The method omits boxes 606 and 104 through 108, and adds an arrow from box 604 to box 102.

[0068] In the first and second exemplary methods described above, the requests received by the scheduler (in box 102) and the requests triggered by instructions added by the compiler only request the resources needed to allocate the next block of the task. In the third exemplary method of resource allocation, as... Figure 7 As shown, the request additionally identifies the resources required to execute the entire task. This implements a hybrid approach in which resources for the entire task are allocated if available, but if unavailable and resources for the next block are available only, resources for the next block are allocated only.

[0069] like Figure 7 As shown, a request for resource allocation for a task is received (in box 702), where the request includes both resource requirements for the next block of the task and resource requirements for the entire task. For example, in a previous iteration of the method, if resources for the entire task have not yet been allocated (No in box 704), it is determined whether the resources required for the entire task are available (box 706), and if such a resource amount is available (Yes in box 706), the resources required to execute the entire task are allocated (box 708). If the resources required for the entire task are not available (No in box 706), it is determined whether the resources required only for the next block are available (box 104), and if available (Yes in box 104), the resources are allocated to the task (box 106). However, if there are not enough resources for the next block (No in box 104), the task is suspended (box 108) (as described above) until there are enough resources available for at least the next task. As previously stated, the method is repeated for each received allocation request; however, if all resources for the task have already been allocated ("Yes" in box 704), no further allocation of resources is performed.

[0070] In the absence of contentious resource requests, this hybrid approach reduces latency caused by additional resource requests (triggered by additional instructions inserted into the program) and, where tasks require their resources to be allocated in contiguous blocks, also reduces the need to copy data between register / memory locations. See also... Figure 8 and Figure 9 This is described, and a schematic diagram of resource allocation for the task is shown.

[0071] exist Figure 8 In China, use Figure 1 or Figure 6 The method ensures that resources are allocated only for the next block, and is consistent with... Figure 2 The corresponding example differs; there are 3x available resources instead of 2x. Initially, task 0 is allocated the resources needed for the first block (block 0), task 1 is allocated the resources it needs, and there are unallocated resources. When task 0's block 0 completes (at time t1), another request is received from task 0. This request is for allocating the resources needed for the next block (block 1). At this point, although the amount of resources requested by task 0's block 1, 2x, is available, it is not available in consecutive blocks, so task 0 is suspended until task 1 completes and releases its resources. Conversely, as... Figure 9 As shown, if using Figure 7 The method described above, because resources are available, can allocate resources for the entire task 0 from the beginning. Therefore, when the second request is received (after executing block 0), sufficient resources to execute block 1 have already been pre-allocated. However, if no resources are initially available, a smaller amount (x) will be allocated to task 0, allowing both task 0 and task 1 to execute in parallel.

[0072] like Figure 7 The mixing method shown can be further modified as follows: Figure 10 As shown, it includes two operating modes—one mode is to fully allocate resources when resources are available (such as...). Figure 7 As shown), and the second mode allocates resources only for the next block when resources are available (e.g., Figure 1 or Figure 6 As shown in the diagram, and in various examples, if all resources have been allocated, the remaining resources are released. Switching between these two modes can be based on a cancellation criterion (box 1002), which can be defined by any suitable parameter, such as one or more of the following: the occurrence of a deadlock, the number of tasks in execution, the number of suspended tasks, the priority (or relative priority) of a particular task (e.g., suspended tasks compared to tasks requesting resources), etc. If the cancellation criterion is not met (“No” in box 1002), the method proceeds as follows: Figure 7 The process is as shown. However, if the cancellation criterion ("Yes" in box 1002) is met, the method proceeds as follows. Figure 1 or Figure 6As shown, in various examples, the complete allocation of any existing resources is reduced to an additional operation of releasing excess resources, reducing it to only the resources needed for the next task ("Yes" in box 704, followed by box 1004). In other examples, no complete allocation may be reduced (i.e., box 1004 is omitted), and the mode switch may only affect future allocations.

[0073] Although Figure 10 The method is not shown for partial release of resources during operation in the second mode (e.g.) Figure 6 (as shown), but it should be understood that Figure 6 The additional method boxes (boxes 602 to 606) can also be combined with Figure 10 In the second operation mode of the method.

[0074] In use Figure 10 In various examples of the methods, a cancellation criterion can be used to prevent or break deadlocks. For example, high-priority tasks are more likely to be allocated according to the first operating mode (i.e., full allocation if available), while low-priority tasks (or non-high-priority tasks) are more likely to be allocated according to the second operating mode (i.e., block-by-block resource allocation only).

[0075] In use Figure 10 In the case of the above methods, it can be compared with the above references. Figure 3 The same method described divides the program (and therefore the task) into blocks.

[0076] The above resource allocation method can be executed by different parts of the processing system, and the exemplary processing system 1100 in Figure 11 As shown in the diagram. Processing system 1100 includes processor 1102 (e.g., GPU or CPU) and at least one memory 1104. Task scheduler 1106 receives requests for resource allocation and also performs resource allocation. The allocated resources may include one or more registers 1108 (e.g., temporary registers and / or shared registers) within processor 1102 and / or regions within memory 1104. Instances of tasks are also created by scheduler 1106 and then executed by execution module (or pipeline) 1110. Processing system 1100 may also include instruction sequencer 1112, which feeds instructions to execution module 1100. As described above, in the example of partitioning the program (and thus the associated tasks) after compilation, instruction sequencer 1112 may additionally determine when a resource threshold is crossed and notify scheduler 1106 that a block boundary has been reached.

[0077] Figure 12A computer system is shown in which the graphics processing system described herein is implemented. The computer system includes a CPU 1202, a GPU 1204, a memory 1206, and other devices 1214, such as a display 1216, speakers 1218, and a camera 1220. Figure 11 As shown, CPU 1202 and / or GPU 1204 can operate as processor 1102 in processing system 1100, and as described above, memory 1206 can correspond to memory 1104 in processing system 1100. Components of the computer system can communicate with each other via communication bus 1222.

[0078] Figure 2 , Figure 8 and Figure 9 This illustrates resource allocation for tasks or blocks defined as multiples of a common unit x. It should be understood that this is merely exemplary, and in other examples, the threshold between blocks may be defined in other ways.

[0079] Figure 11 and Figure 12 The system is shown as comprising multiple functional blocks. This is merely illustrative and not intended to define a strict division between different logical elements of such an entity. Each functional block may be provided in any suitable manner. It should be understood that the intermediate values ​​described herein as being formed by the processing system do not need to be physically generated by the processing system at any point in time, and may simply represent logical values ​​that conveniently describe the processing performed by the processing system between its inputs and outputs.

[0080] The processing system described herein may be embodied in hardware on an integrated circuit. The processing system described herein may be configured to perform any of the methods described herein. Generally, any of the functions, methods, techniques, or components described above may be implemented in software, firmware, hardware (e.g., a fixed logic circuit system), or any combination thereof. The terms “module,” “function,” “component,” “element,” “cell,” “block,” and “logic” may be used herein to generally denote software, firmware, hardware, or any combination thereof. In the case of a software implementation, a module, function, component, element, cell, block, or logic represents program code that, when executed on a processor, performs a specified task. The algorithms and methods described herein may be executed by one or more processors that execute code that causes the processor to perform the algorithm / method. Examples of computer-readable storage media include random access memory (RAM), read-only memory (ROM), optical disk, flash memory, hard disk storage, and other memory devices that may use magnetic, optical, and other techniques to store instructions or other data and that may be machine-accessible.

[0081] As used herein, the terms computer program code and computer-readable instructions refer to any kind of executable code for a processor, comprising code expressed in machine language, interpreted language, or scripting language. Executable code includes binary code, machine code, bytecode, code defining integrated circuits (e.g., hardware description languages ​​or netlists), and code expressed in programming languages ​​such as C, Java®, or OpenCL®. Executable code can be, for example, any kind of software, firmware, script, module, or library that, when properly executed, processed, interpreted, compiled, or run in a virtual machine or other software environment, causes the processor of a computer system that supports the executable code to perform tasks specified by said code.

[0082] A processor, computer, or computer system can be any kind of device, machine, or special-purpose circuit, or a collection or part thereof, that has processing power enabling it to execute instructions. A processor can be any kind of general-purpose or special-purpose processor, such as a CPU, GPU, system-on-a-chip, state machine, media processor, application-specific integrated circuit (ASIC), programmable logic array, field-programmable gate array (FPGA), physical processing unit (PPU), radio processing unit (RPU), digital signal processor (DSP), general-purpose processor (e.g., general-purpose GPU), microprocessor, any processing unit designed to accelerate tasks other than those of a CPU, etc. A computer or computer system may include one or more processors. Those skilled in the art will recognize that this processing power is incorporated into many different devices; therefore, the term "computer" includes set-top boxes, media players, digital radios, PCs, servers, mobile phones, personal digital assistants, and many other devices.

[0083] This invention also intends to cover software, such as hardware description language (HDL) software, that defines the configuration of hardware as described herein for designing integrated circuits or configuring programmable chips to perform desired functions. Specifically, a computer-readable storage medium may be provided on which computer-readable program code is encoded in the form of an integrated circuit definition dataset, which, when processed (i.e., run) in an integrated circuit manufacturing system, configures the system to manufacture a processor configured to perform any of the methods described herein, or a processing system including any of the devices described herein. The integrated circuit definition dataset may, for example, be an integrated circuit description.

[0084] Therefore, a method for manufacturing a processor as described herein can be provided at an integrated circuit manufacturing system. Furthermore, an integrated circuit definition dataset can be provided, which, when processed in an integrated circuit manufacturing system, enables the method for manufacturing the processor to be executed.

[0085] Integrated circuit definition datasets can be in the form of computer code, such as as a netlist, code for configuring programmable chips, or a hardware description language for defining integrated circuits at any level, including register-transfer level (RTL) code, high-level circuit representations such as Verilog or VHDL, and low-level circuit representations such as OASIS (RTM) and GDSII. Higher-level representations that logically define integrated circuits (e.g., RTL) can be processed at a computer system configured to generate manufacturing definitions of integrated circuits within a software environment that includes definitions of circuit elements and rules for combining those elements to generate manufacturing definitions of integrated circuits defined by said representation. As is typically the case where software executes at a computer system to define a machine, one or more intermediate user steps (e.g., providing commands, variables, etc.) may be required to configure the computer system to generate manufacturing definitions of integrated circuits, executing code that defines the integrated circuits to generate those manufacturing definitions.

[0086] Now refer to Figure 13 This describes an example of processing integrated circuit definition datasets at an integrated circuit manufacturing system in order to configure the system for manufacturing processors.

[0087] Figure 13 An example of an integrated circuit (IC) manufacturing system 1302 is shown, configured to manufacture processors as described in any of the examples herein. Specifically, the IC manufacturing system 1302 includes a layout processing system 1304 and an integrated circuit generation system 1306. The IC manufacturing system 1302 is configured to receive an IC definition dataset (e.g., defining a processor as described in any of the examples herein), process the IC definition dataset, and generate an IC (e.g., embodying a processor as described in any of the examples herein) based on the IC definition dataset. The processing of the IC definition dataset configures the IC manufacturing system 1302 to manufacture integrated circuits embodying processors as described in any of the examples herein.

[0088] The layout processing system 1304 is configured to receive and process an IC definition dataset to determine a circuit layout. Methods for determining a circuit layout based on an IC definition dataset are known in the art and may involve, for example, synthesizing RTL code to determine the gate-level representation of the circuit to be generated, for example, in relation to logic components (e.g., NAND, NOR, AND, OR, MUX, and FLIP-FLOP components). By determining the location information of the logic components, the circuit layout can be determined based on the gate-level representation of the circuit. This can be done automatically or with user intervention to optimize the circuit layout. Once the layout processing system 1304 has determined the circuit layout, it can output the circuit layout definition to the IC generation system 1306. The circuit layout definition may be, for example, a circuit layout description.

[0089] As is known in the art, IC generation system 1306 generates ICs according to a circuit layout definition. For example, IC generation system 1306 can implement a semiconductor device manufacturing process for generating ICs, which may involve a multi-step sequence of photolithography and chemical processing steps, during which electronic circuits are gradually formed on a wafer made of semiconductor material. The circuit layout definition may be in the form of a mask, which can be used in the photolithography process to generate ICs according to the circuit definition. Alternatively, the circuit layout definition provided to IC generation system 1306 may be in the form of computer-readable code, which IC generation system 1306 can use to form a suitable mask for generating ICs.

[0090] The various processes performed by the IC manufacturing system 1302 may all be implemented in one location, for example, by one party. Alternatively, the IC manufacturing system 1302 may be a distributed system, allowing some processes to be performed in different locations and by different parties. For example, some of the following stages may be performed in different locations and / or by different parties: (i) synthesizing RTL code representing an IC definition dataset to form a gate-level representation of the circuit to be generated; (ii) generating a circuit layout based on the gate-level representation; (iii) forming a mask based on the circuit layout; and (iv) using the mask to manufacture the integrated circuit.

[0091] In other examples, processing an integrated circuit definition dataset at an integrated circuit manufacturing system can configure the system to manufacture a processor without processing the IC definition dataset to determine circuit layout. For instance, an integrated circuit definition dataset can define the configuration of a reconfigurable processor, such as an FPGA, and processing the dataset can configure the IC manufacturing system (e.g., by loading configuration data into the FPGA) to generate a reconfigurable processor with the defined configuration.

[0092] In some implementations, when processed in an integrated circuit manufacturing system, an integrated circuit manufacturing definition dataset can enable the integrated circuit manufacturing system to generate devices as described herein. For example, using an integrated circuit manufacturing definition dataset, as described above regarding... Figure 13 The configuration of the integrated circuit manufacturing system described herein enables the production of devices as described in this document.

[0093] In some examples, an integrated circuit definition dataset may include software running on hardware defined at the dataset, or software running in combination with hardware defined at the dataset. Figure 13In the example shown, the IC generation system can also be additionally configured by the integrated circuit definition dataset to load firmware onto the integrated circuit according to the program code defined in the integrated circuit definition dataset during the manufacturing of the integrated circuit, or otherwise provide the integrated circuit with program code to be used with the integrated circuit.

[0094] Those skilled in the art will recognize that the storage device used to store program instructions can be distributed across a network. For example, a remote computer can store an example of a described process as software. A local or terminal computer can access the remote computer and download part or all of the software to run the program. Alternatively, a local computer can download fragments of software as needed, or execute some software instructions at a local terminal while executing others at a remote computer (or computer network). Those skilled in the art will also recognize that, by utilizing conventional techniques known to them, all or part of the software instructions can be executed by dedicated circuitry such as a DSP, a programmable logic array, etc.

[0095] The methods described herein can be executed by a computer having machine-readable software configured to be stored on a tangible storage medium. For example, the software may take the form of a computer program including computer-readable program code for configuring the computer to perform components of the methods, or a computer program including computer program code means adapted to perform all steps of any of the methods described herein when the program is run on the computer and, where the computer program can be implemented on a computer-readable storage medium. Examples of tangible (or non-transitory) storage media include disks, thumb drives, memory cards, etc., and do not include propagation signals. The software may be adapted to execute on a parallel or serial processor, such that the method steps can be executed in any suitable order or simultaneously.

[0096] The hardware components described herein can be generated from a non-transitory computer-readable storage medium on which computer-readable program code is encoded.

[0097] Memory storing machine-executable data used to implement the disclosed aspects can be a non-transitory medium. A non-transitory medium can be volatile or non-volatile. Examples of volatile non-transitory media include semiconductor-based memories such as SRAM or DRAM. Examples of techniques that can be used to implement non-volatile memories include optical and magnetic memory technologies, flash memory, phase-change memory, and resistive RAM.

[0098] A specific reference to "logic" refers to a structure that performs one or more functions. Examples of logic include circuits arranged to perform these functions. For example, such circuits may include transistors and / or other hardware elements available during the manufacturing process. Such transistors and / or other elements may be used to form circuits or structures that implement and / or contain memory, such as registers, flip-flops or latches, logic units such as Boolean operations, mathematical units such as adders, multipliers, or shifters and interconnects, as examples. These elements may be provided as custom circuits or standard cell libraries, macros, or at other levels of abstraction. These elements may be specifically arranged and interconnected. Logic may include circuits with fixed functions, and the circuits may be programmed to perform one or more functions; such programming may be provided from firmware or software updates or control mechanisms. Logic identified as performing a function may also include logic that implements constituent functions or sub-processes. In one example, hardware logic has circuits that implement one or more fixed-function operations, state machines, or processes.

[0099] Compared to known implementations, the implementation of the concepts set forth in this application in devices, apparatuses, modules, and / or systems (and in the methods implemented herein) can lead to performance improvements. Performance improvements may include one or more of increased computational performance, reduced latency, increased throughput, and / or reduced power consumption. During the manufacture of such devices, apparatuses, modules, and systems (e.g., in integrated circuits), trade-offs can be made between performance improvements and physical implementation methods, thereby improving manufacturing methods. For example, a trade-off can be made between performance improvements and layout area, thereby matching the performance of known implementations but using less silicon. This can be accomplished, for example, by reusing functional blocks serially or sharing functional blocks among elements of a device, apparatus, module, and / or system. Conversely, the concepts set forth in this application that lead to improvements in the physical implementation of devices, apparatuses, modules, and systems (such as reduced silicon area) can be traded off for performance improvements. This can be accomplished, for example, by manufacturing multiple instances of a module within a predefined area budget.

[0100] As will be apparent to those skilled in the art, any range or device value given herein may be extended or altered without losing the desired effect.

[0101] It should be understood that the above benefits and advantages may apply to one implementation scheme or multiple implementation schemes. The implementation schemes are not limited to those that address any or all of the stated problems or have any or all of the stated benefits and advantages.

[0102] Any reference to "one" item refers to one or more of those items. The term "comprising" is used herein to mean including identified method blocks or elements, but such blocks or elements do not include an exclusive list, and devices may include additional blocks or elements, and methods may include additional operations or elements. Furthermore, it is not implied that the blocks, elements, and operations themselves are closed.

[0103] The steps described herein can be performed in any suitable order or simultaneously as appropriate. The arrows between the boxes in the diagrams illustrate an exemplary sequence of method steps, but are not intended to exclude other sequences or the parallel execution of multiple steps. Furthermore, individual blocks can be removed from any method without departing from the substance and scope of the subject matter described herein. Some aspects of any of the examples above can be combined with some aspects of any other examples described to form further examples without losing the desired effect. Where elements in the diagrams are shown connected by arrows, it should be understood that these arrows only illustrate an exemplary flow of communication (including data and control messages) between elements. The flow between elements can be in either direction or both.

[0104] The applicant has independently disclosed each individual feature described herein, as well as any combination of two or more such features, to the extent that such features or combinations can be implemented based on the specification as a whole, in accordance with the common knowledge of those skilled in the art, regardless of whether such features or combinations of features solve any problem disclosed herein. In view of the foregoing description, those skilled in the art will understand that various modifications can be made within the scope of this invention.

Claims

1. A method for allocating registers for a task in a parallel processing system, the parallel processing system including a processor, the processor including a plurality of registers, a task scheduler and one or more execution modules, and the task including a set of data items and a program, wherein the data items will be processed according to the program. The task scheduler receives requests to allocate registers for tasks, each request indicating the amount of registers required to execute only the next block of the task and the amount of registers required to execute the entire task. The task is partitioned into multiple blocks based on changes in register requirements over time, and each block is associated with a portion of the program executed by the task. and For each request, the method includes: Determine whether the amount of registers required to execute the entire task has been allocated to the task; and In response to the fact that the amount of registers required to execute the entire task has not yet been allocated to the task: Determine if the amount of registers required to execute the entire task is available; In response to determining that the amount of registers required to execute the entire task is available, the registers are allocated to the task; and In response to the unavailability of the amount of registers required to execute the entire task: The task scheduler determines whether the amount of registers required to execute the next block of the task is available; In response to the determination that the amount of registers required to execute the next block of the task is unavailable, the task is paused; and In response to determining that the amount of registers required to execute the next block of the task is available, the amount of registers is allocated to the task so that the next block of the task can be executed.

2. The method of claim 1, wherein determining whether the amount of registers required to perform the next block of the task is available comprises: Determine whether a contiguous block of registers, including the amount of registers required to perform the task, is available.

3. The method according to claim 1, further comprising: Release any registers that are not needed for the execution of the next block of the task.

4. The method according to any one of claims 1 to 3, further comprising repeating the method for each of the plurality of tasks in the parallel processing system.

5. The method according to any one of claims 1 to 4, wherein the processor is a GPU and the task is associated with a shader.

6. A parallel processing system, comprising a processor, the processor comprising: One or more registers used when performing a task, the task comprising a set of data items and a program, wherein the data items are processed according to the program, and wherein the task is partitioned into multiple blocks based on changes in register requirements over time, and wherein each block is associated with a portion of the program executed by the task; A task scheduler, wherein requests for allocating registers for tasks are received, each request identifying the amount of registers required to execute only the next block of the task and the amount of registers required to execute the entire task, wherein the task scheduler is configured to respond to each request as follows: Determine whether the amount of registers required to execute the entire task has been allocated to the task; and In response to the fact that the amount of registers required to execute the entire task has not yet been allocated to the task: Determine if the amount of registers required to execute the entire task is available; In response to determining that the amount of registers required to perform the entire task is available, the registers are allocated to the task; and In response to the unavailability of the amount of registers required to execute the entire task: Determine whether the amount of registers required to execute the next block of the task is available; In response to the determination that the amount of registers required to execute the next block of the task is unavailable, the task is paused; and In response to determining that the amount of registers required to execute the next block of the task is available, the registers are allocated to the task so that the next block of the task can be executed; as well as One or more execution modules are arranged to execute each block of the task.

7. The parallel processing system of claim 6, wherein the task scheduler is configured to determine whether the amount of registers required to execute the next block of the task is available by: Determine whether a contiguous block of registers, including the amount of registers required to perform the task, is available.

8. The parallel processing system of claim 6, wherein the task scheduler is further arranged as follows: Release any registers that are not needed for the execution of the next block of the task.

9. The parallel processing system of any one of claims 6-8, further comprising an instruction sequencer, wherein the instruction sequencer is configured to access metadata associated with a program associated with the task, monitor the execution of the program, and use the metadata to identify when the execution of the program has reached an instruction in the program whose register requirement first exceeds a threshold in the program execution sequence, and to identify block boundaries, wherein the metadata associated with the program identifies one or more instructions in the program and register requirements associated with those instructions.

Images