Register compression with early deallocation

Abstract

A system, apparatus, and method for implementing register compression with early release are disclosed. The processor includes at least a command processor, a plurality of compute units, a plurality of registers, and a control unit. The registers are statically allocated to a wavefront by the control unit when the wavefront is initiated by the command processor on the compute unit. In response to determining that a first set of registers previously allocated to a first wavefront are no longer needed, the first wavefront executes an instruction to release the first set of registers. The control unit detects the executed instruction and releases the first set of registers to an available pool of registers for potential use by other wavefronts. The control unit can then allocate the first set of registers to a second wavefront for use by threads of the second wavefront while the first wavefront is still active.

Description

[Background technology]

[0001] (Explanation of related technologies) Graphics Processing Units (GPUs) and other multithreaded processing units typically contain multiple processing elements (also called processor cores or compute units) that simultaneously execute multiple instances of a single program on multiple datasets. These instances are called threads, waves, or wavefronts. Several waves are created (or generated) and then dispatched to each processing element within the multithreaded processing unit. A processing unit can contain hundreds of processing elements, allowing thousands of threads to simultaneously execute programs within the unit. In a multithreaded GPU, threads execute different instances of the kernel to perform computations concurrently or in parallel. Threads running concurrently within a multithreaded processing unit share some of the unit's resources. Shared resources include vector general-purpose registers (VGPRs) that store thread state information, local data shares (LDS) used to store thread data, and bandwidth available for moving information between the local cache hierarchy and memory.

[0002] The number of registers allocated to a thread is usually determined at startup. For example, the number of registers allocated to a thread can be included in the corresponding kernel metadata. The number of registers required by a thread can be determined by the compiler, the programmer, at runtime, or by other means. The problem with statically allocating some registers to threads is that the number of available registers is limited, and threads compete with each other for this limited number of registers. Alternatively, a thread may use fewer registers than it was allocated. In such cases, register resources that might be used by other threads remain unused.

[0003] The advantages of the methods and mechanisms described in this specification can be better understood by reference to the following description in conjunction with the accompanying drawings.

Brief Description of the Drawings

[0004] [Figure 1] It is a block diagram of an embodiment of a computing system. [Figure 2] It is a block diagram of another embodiment of a computing system. [Figure 3] It is a block diagram of an embodiment of a graphics processing unit (GPU). [Figure 4] It is a block diagram of an embodiment of a wavefront executing on an SMID unit. [Figure 5] It is a block diagram of an embodiment of a wavefront executing on a SIMD unit. [Figure 6] It is a generalized flowchart showing an embodiment of a method for performing early register release by a wavefront. [Figure 7] It is a generalized flowchart showing an embodiment of a method for releasing a register while waiting for an affirmative response from memory. [Figure 8] It is a generalized flowchart showing an embodiment of a method for performing register compression during early register release.

Modes for Carrying Out the Invention

[0005] The following description includes numerous specific details to provide a full understanding of the methods and mechanisms presented herein. However, those skilled in the art should recognize that various embodiments can be carried out without these specific details. In some examples, well-known structures, components, signals, computer program instructions, and techniques are not shown in detail to avoid obscuring the approaches described herein. For simplicity and clarity, it should be understood that the elements shown in the figures are not necessarily drawn to scale. For example, the dimensions of some elements may be exaggerated relative to others.

[0006] Various systems, devices, and methods for implementing register compression with early release are disclosed herein. In one embodiment, the processor includes at least a command processor, a plurality of compute units, a plurality of registers, and a control unit. The registers are statically allocated to wavefronts by the control unit when a wavefront is started by a command processor on a compute unit. When a first wavefront determines that a first set of registers previously allocated to it is no longer needed, the first wavefront sends a message to the control unit indicating that it can execute an instruction or release the first set of registers. The control unit receives the message and releases the first set of registers into the pool of available registers so that they may be potentially used by other wavefronts. For example, the control unit may allocate the first set of registers to a second wavefront for use by a thread of the second wavefront while the first wavefront is still active.

[0007] Referring to Figure 1, a block diagram of one embodiment of the computing system 100 is shown. In one embodiment, the computing system 100 includes at least processors 105A to 105N, an input / output (I / O) interface 120, a bus 125, a memory controller 130, a network interface 135, a memory device 140, a display controller 150, and a display 155. In other embodiments, the computing system 100 includes other components and / or the computing system 100 is configured in a different manner. Processors 105A to 105N represent any number of processors included in the system 100.

[0008] In one embodiment, processor 105A is a general-purpose processor such as a central processing unit (CPU). In this embodiment, processor 105A runs a driver 110 (e.g., a graphics driver) to communicate with other processors in the system 100 and / or to control operations on one or more of those processors. Note that, depending on the embodiment, the driver 110 can be implemented using any preferred combination of hardware, software and / or firmware. In one embodiment, processor 105N is a data-parallel processor having a highly parallel architecture. Data-parallel processors include graphics processing units (GPUs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc. In some embodiments, processors 105A to 105N include multiple data-parallel processors. In one embodiment, processor 105N is a GPU that provides pixels to the display controller 150 to be sent to the display 155.

[0009] The memory controller 130 represents any number and type of memory controller accessible by processors 105A to 105N. Although the memory controller 130 is shown as being separate from processors 105A to 105N, it should be understood that this represents only one possible embodiment. In other embodiments, the memory controller 130 may be embedded within one or more of processors 105A to 105N, and / or the memory controller 130 may be located on the same semiconductor die as one or more of processors 105A to 105N. The memory controller 130 is coupled to any number and type of memory devices 140. The memory devices 140 represent any number and type of memory devices. For example, the types of memory within the memory devices 140 include Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), NAND flash memory, NOR flash memory, Ferroelectric Random Access Memory (FeRAM), etc.

[0010] The I / O interface 120 represents any number and type of I / O interfaces (e.g., peripheral component interconnect (PCI) bus, PCI Expansion (PCI-X), PCI Express (PCIE) bus, Gigabit Ethernet (GBE) bus, Universal Serial Bus (USB)). Various types of peripheral devices (not shown) are coupled to the I / O interface 120. Such peripheral devices include, but are not limited to, displays, keyboards, mice, printers, scanners, joysticks, other types of game controllers, media recording devices, external storage devices, network interface cards, etc. The network interface 135 can receive and send network messages over the network.

[0011] In various embodiments, the computing system 100 is a computer, laptop, mobile device, game console, server, streaming device, wearable device, or any of various other types of computing systems or devices. Note that the number of components of the computing system 100 varies from embodiment to embodiment. For example, in other embodiments, there may be more or fewer components than those shown in Figure 1. Also note that in other embodiments, the computing system 100 may include other components not shown in Figure 1. In addition, in other embodiments, the computing system 100 is configured in a manner other than that shown in Figure 1.

[0012] Referring to Figure 2, a block diagram of another embodiment of the computing system 200 is shown. In one embodiment, the system 200 includes a GPU 205, system memory 225, and local memory 230. The system 200 may also include other components that are not shown to avoid obscuring the figure. The GPU 205 includes at least a command processor 235, control logic 240, dispatch unit 250, compute units 255A-255N, memory controller 220, global data share 270, level 1 (L1) cache 265, and level 2 (L2) cache 260. In other embodiments, the GPU 205 includes other components, omits one or more of the illustrated components, and has multiple instances of components even if only one instance is shown in Figure 2, and / or is organized in other appropriate ways. In one embodiment, the circuitry of the GPU 205 is included in the processor 105N (in Figure 1).

[0013] In various embodiments, the computing system 200 executes one of various types of software applications. As part of executing a given software application, the host CPU (not shown) of the computing system 200 starts work to be executed on the GPU 205. In one embodiment, the command processor 235 receives a kernel from the host CPU and, using the dispatch unit 250, issues corresponding wavefronts to the computing units 255A-255N. Typically, registers such as the vector general-purpose register (VGPR) 256 of computing unit 255A are statically allocated to wavefronts executed on computing units 255A-255N. However, some wavefronts may not use all of those registers, while other wavefronts fully utilize those registers and wait for more registers to become available. To mitigate this scenario, an active wavefront can execute an instruction to release some or all of those registers into an unused pool that will be reallocated to other wavefronts. This allows VGPR256 and VGPR257 of computing unit 255B and VGPR258 of computing unit 255N to be used more efficiently by the various wavefronts being executed by computing units 255A to 255N. Note that VGPR256 to 258 represent any number of VGPRs.

[0014] Referring to Figure 3, a block diagram of one embodiment of a graphics processing unit (GPU) 300 is shown. In one embodiment, the GPU 300 includes at least SIMDs 310A to 310N, a scheduler unit 345, an instruction buffer 355, and a control unit 360. Note that the GPU 300 may include other logic not shown in Figure 3 to avoid obscuring the diagram. Note also that other processors (e.g., FPGAs, ASICs, DSPs) may include the circuitry shown in the GPU 300.

[0015] In one embodiment, the GPU 300 executes kernel instructions on any number of wavefronts. These instructions are stored in an instruction buffer 355 and scheduled by a scheduler unit 345 for execution on SIMD 310A-310N. In one embodiment, each wavefront contains several work items to be executed on lanes 315A-315N, 320A-320N, and 325A-325N within SIMD 310A-310N. Each lane 315A-315N, 320A-320N, and 325A-325N of SIMD 310A-310N may also be called an “execution unit”.

[0016] In one embodiment, the GPU 300 receives multiple instructions for a wavefront having several work items. When work items are executed on SIMD 310A-310N, each work item is allocated a corresponding portion of vector general-purpose registers (VGPRs) 330A-330N. If a work item can free some of the VGPRs 330A-330N allocated to it, the work item executes an instruction to free those VGPRs. In one embodiment, the control unit 360 receives an indicator that an instruction has been executed, and in response to receiving the indicator, the control unit 360 reallocates these freed VGPRs to one or more work items. This allows other work items to use these VGPRs while the work item that freed the VGPRs remains active. Although the control unit 360 is shown as being outside of SIMD 310A-310N, it should be understood that this represents only one embodiment. In other embodiments, the control unit 360 may be located elsewhere, and / or the control unit 360 may be divided into multiple different control units. For example, in another embodiment, another control unit 360 is located within each SIMD 310A to 310N.

[0017] It should be noted that when the letter "N" appears next to various structures in this specification, it generally indicates any number of elements of that structure (e.g., any number of SIMD310A-310N). In addition, different references in Figure 3 that use the letter "N" (e.g., SIMD310A-310N and lanes 315A-315N) are not intended to indicate that an equal number of different elements are provided (e.g., the number of SIMD310A-310N may differ from the number of lanes 315A-315N).

[0018] Referring to Figure 4, a block diagram of one embodiment of wavefronts 425A-425B running on SIMD unit 400A is shown. In one embodiment, SIMD unit 400A is included within a compute unit (e.g., compute units 255A-255N in Figure 2) along with any number of SIMD units. As shown in Figure 4, SIMD unit 400A includes execution lanes 420A-420B running wavefronts 425A-425B. Only two execution lanes 420A-420B are shown in SIMD unit 400A, but it should be understood that these are shown simply for illustrative purposes. Execution lanes 420A-420B can represent any number of lanes. Also, although 16 VGPRs 440-455 are shown in Figure 4, it should be understood that other embodiments may include a number of VGPRs other than 16.

[0019] In one embodiment, VGPRs 440-447 are assigned to wavefront 425A when wavefront 425A is started, and VGPRs 448-455 are assigned to wavefront 425B when wavefront 425B is started. In one embodiment, control unit 430 controls access to VGPRs and ensures that each wavefront 425A-425B accesses only the registers assigned to that wavefront. However, in various embodiments, one VGPR may use fewer registers than the total number assigned to it, and another wavefront may use all of its assigned registers and operate inefficiently due to the inability to use additional registers. Such cases are shown in Figure 4 with respect to wavefronts 425A-425B.

[0020] As shown in Figure 4, VGPR440-443 are active (i.e., in use) by wavefront 425A, while VPGR444-447 are inactive (i.e., not in use). This is indicated by a dashed line across VGPR440-443, while VPGR444-447 have a clear background. The VGPR legend 460 illustrates the meaning of different shadings of the VGPR for the embodiment shown in Figure 4. In one embodiment, when wavefront 425A detects that a block of registers is unused, wavefront 425A executes an instruction to notify control unit 430 that this block of registers is free and can be reallocated to another wavefront. In the example shown in Figure 4, this block of unused registers includes VPGR444-447. In another embodiment, upon detecting that a block of registers 444-447 is unused, wavefront 425A sends a message to control unit 430 to notify control unit 430. Detecting that the block of registers 444-447 is unused can be done in real time by the compiler, programmer, or by the wavefront 425A, control unit 430, or other circuitry. For example, compilation techniques such as liveness analysis or other methods for identifying dead registers may be used to determine when the register values ​​are no longer needed and the registers can be freed. In some embodiments, this compilation is performed on the kernel at runtime. In such cases, the compiler inserts instructions indicating that the register(s) can be freed and reallocated as needed or desired. In other embodiments, these compilation techniques are performed at different times. In some embodiments, the compiler compiles the code so that registers identified as becoming dead earlier during processing are placed earlier in the allocation range. In this way, the probability that the range of registers becomes available for free earlier is increased. These and other techniques are known and intended.

[0021] When the control unit 430 receives a notification that the blocks of registers 444 to 447 are unused, the control unit 430 updates the register allocation mapping table 435 to indicate that this block of registers 444 to 447 is no longer assigned to the wavefront 425A. Further, the control unit 430 can assign this block of registers 444 to 447 to another wavefront. In the example shown in FIG. 4, all the registers assigned to the wavefront 425B are active. In response to detecting that all the registers assigned to the wavefront 425B are active, or receiving an indicator or notification of this scenario, in one embodiment, the control unit 430 reassigns VPGRs 444 to 447 to the wavefront 425B.

[0022] Referring to FIG. 5, a block diagram of an embodiment of wavefronts 425A to 425B executing on the SIMD unit 400B is shown. The description of FIG. 5 is intended to be a continuation of the description of FIG. 4. Thus, the SIMD unit 400B is intended to represent the SIMD unit 400A (of FIG. 4) at a point in time after the VGPRs 444 to 447 have been reassigned from the wavefront 425A to the wavefront 425B while the wavefront 425A remains active. Also, at the point in time shown in FIG. 5, the VGPRs 444 to 447 are currently being used by the wavefront 4, which is indicated by a dashed line in the background of the blocks representing the VGPRs 444 to 447.

[0023] Referring to FIG. 6, an embodiment of a method 600 for performing early register release by a wavefront is shown. For the sake of explanation, the steps in this embodiment and the steps in FIG. 7 are shown in order. However, it should be noted that in various embodiments of the described method, one or more of the described elements may be executed simultaneously in a different order than that shown, or may be completely omitted. Other additional elements may also be executed as needed. Any of the various systems or devices described herein are configured to implement method 600.

[0024] The processor detects a condition (block 605) for releasing a first set of registers (e.g., VGPRs) of the first wavefront while the first wavefront is still active (i.e., executing instructions). In one embodiment, the condition for releasing the first set of registers is that the first wavefront waits for an acknowledgement (ack) from memory before completing an end. In another embodiment, the condition for releasing the first set of registers is that the first wavefront uses only a portion of its assigned registers. In other embodiments, other conditions for releasing the first set of registers may be detected. In one embodiment, the first set of registers is the entire set of registers assigned to the first wavefront. In another embodiment, the first set of registers is a portion of the registers assigned to the first wavefront.

[0025] Upon detecting a condition, the computing unit sends a message to the control unit that the first wavefront has freed the first set of registers and returned them to the pool of unused registers (block 610). The control unit then marks the freed first set of registers as available to other wavefronts (block 615). Accordingly, one or more other wavefronts can use the freed first set of registers while the first wavefront is still active (block 620). After block 620, method 600 terminates.

[0026] Referring to Figure 7, one embodiment of method 700 for freeing registers while waiting for an acknowledgment from memory is shown. The wavefront performs one or more memory write operations (block 705). Depending on the embodiment, the wavefront includes any number of threads from 1 to N, where N is a positive integer greater than 1. Next, the wavefront waits to receive an acknowledgment (or ack for short) from memory regarding the memory write operation (block 710). While waiting for the ack, the wavefront determines whether there are any further instructions other than waiting for an acknowledgment from memory (condition block 715).

[0027] If there are no further instructions in the wavefront other than waiting for an acknowledgment from memory (condition block 715, "no"), the wavefront sends a VGPR deallocation message to the control unit (block 720). In one embodiment, the VGPR deallocation message is inserted into the wavefront instruction by the compiler. In another embodiment, the VGPR deallocation message is added by the programmer. In one embodiment, the VGPR deallocation message specifies that all VGPRs should be deallocated. In another embodiment, the VGPR deallocation message includes identification information for one or more VGPRs or a range thereof that should be deallocated. For example, in this embodiment, the message includes identification of a first register, offset, and / or range. In other embodiments, other parameters may be included in the VGPR deallocation message.

[0028] Upon receiving the VGPR deallocation message, the control unit reallocates the VGPR to one or more other wavefronts (block 725). After block 725, method 700 terminates. Otherwise, if there are still subsequent instructions for the wavefront to be executed (condition block 715, "yes"), the wavefront continues execution without sending the VGPR deallocation message (block 730). After block 730, method 700 terminates.

[0029] Referring to Figure 8, one embodiment of method 800 for performing register compression during early register deallocation is shown. The processor determines whether the conditions for a thread to begin early deallocation of some registers have been met (block 805). One or more conditions may include having more than a threshold number of inactive registers, waiting for an acknowledgment (ack) from memory, or the thread reaching a particular execution stage. Inactive registers are those that are allocated to a thread but are not used to store values ​​used by the thread. For the purposes of this explanation, we assume that the conditions for performing early deallocation of some registers have been met.

[0030] Depending on whether the conditions for early freeing of some registers have been met, the processor determines whether the thread's active registers can be compressed into contiguous blocks (block 810). For example, if a thread is allocated registers 0 through 63, and most active registers are in the range of registers 0 through 31, but only a few registers are active in the range of 32 through 63, this indicates that the registers can be moved closer to each other so that adjacent blocks can be freed and returned to the register pool for other threads.

[0031] If the active registers can be compressed into contiguous blocks (condition block 815, "yes"), the thread executes one or more move instructions to move data values ​​from active registers in a relatively low-occupancy area of ​​the register range to registers in a relatively high-occupancy area of ​​the register range (block 820). In one embodiment, the active value is moved from a register in the upper part of the register range (e.g., the upper half) to a register in the lower part of the register range (e.g., the lower half). In another embodiment, the active value is moved from a register in the lower part of the register range to a register in the upper part of the register range.

[0032] Depending on the embodiment, different methods can be employed to execute conditional blocks 815 and 820. In one embodiment, the compiler inserts instructions into the thread's code to execute these steps. In this embodiment, the compiler detects the conditions for performing early deallocation, determines whether active registers need to be compressed, and inserts move instructions to move registers to achieve register compression. In another embodiment, the programmer inserts instructions into the thread's code to execute blocks 815 and 820. In a further embodiment, circuitry in hardware executes blocks 815 and 820. For example, in one embodiment, the register window for the thread is reduced to cover only the relatively high-occupancy area of ​​the register range. In this embodiment, the register window for the thread is reduced to no longer cover the relatively low-occupancy area of ​​the register range. This directs future access to registers in this relatively low-occupancy area of ​​the register range to memory. For example, in one embodiment, the thread's register window is reduced from covering registers 0-127 to covering only registers 0-63. Furthermore, any combination of these methods for executing conditional blocks 815 and 820 can be used in a single embodiment. Furthermore, other methods are possible and intended for executing conditional blocks 815 and 820.

[0033] After block 820, the thread executes an instruction to free a portion of the register range and return it to the unused register pool (block 825). Alternatively, in block 825, the thread sends a message or activates a signal to free a portion of the register range and return it to the unused register pool. After block 825, method 800 terminates. If the active registers cannot be compressed into a contiguous block (condition block 815, "no"), the thread executes multiple instructions to free different portions of the register range and return them to the unused register pool (block 830). Alternatively, in block 830, the thread may execute a single instruction to free the largest unused portion of the register range and return it to the unused register pool. After block 830, method 800 terminates.

[0034] In various embodiments, the methods and / or mechanisms described herein are implemented using program instructions for a software application. For example, program instructions that can be executed by a general-purpose or dedicated processor are intended. In various embodiments, such program instructions are expressed in a high-level programming language. In other embodiments, program instructions are compiled from a high-level programming language into binary, intermediate or other form. Alternatively, program instructions describing the behavior or design of hardware are written. Such program instructions are expressed in a high-level programming language such as C. Alternatively, a hardware design language (HDL) such as Verilog is used. In various embodiments, program instructions are stored in any of various non-temporary computer-readable storage media. The storage media are accessible by the computing system in use to provide the computing system with program instructions for program execution. Generally speaking, such a computing system includes at least one memory and one or more processors configured to execute program instructions.

[0035] It should be emphasized that the embodiments described above are merely non-limiting examples of embodiments. A number of variations and modifications will become apparent to those skilled in the art once the above disclosure is fully understood. The following claims are intended to be construed as encompassing all such variations and modifications.

Claims

1. It is a processor, Multiple registers, It includes a control unit, The control unit is Assigning a first set of registers to a first wavefront started on a first computing unit among multiple computing units, Assigning a second set of registers to a second wavefront started on the first computing unit, In response to the first computing unit sending a message to the control unit to release the one or more registers while one or more threads of the first wavefront remain active, one or more registers from a first set of registers are reallocated from the first wavefront to the second wavefront, wherein the second wavefront stores data in the one or more registers from the first set of registers after the one or more registers have been reallocated. It is configured to do, Processor.

2. The first computing unit executes the first wavefront instruction for freeing one or more registers. The processor according to claim 1.

3. The first computing unit is configured to execute an instruction to free all of the first set of registers. The processor according to claim 2.

4. The first wavefront remains active even after the first computing unit has executed the instruction. The processor according to claim 3.

5. The message includes the register base address, size, and identifier (ID) of the first wavefront. The processor according to claim 1.

6. When the first computing unit executes the first wavefront instruction to access the predetermined register after freeing the predetermined register, the control unit is configured to block access to the predetermined register. The processor according to claim 1.

7. It is a method, The control unit assigns a first set of registers to the first wavefront that has been started on the first computing unit, Assigning a second set of registers to a second wavefront started on the first computing unit, The process involves reallocating one or more registers from a first set of registers to a second wavefront in response to the first computing unit sending a message to the control unit to free the one or more registers while one or more threads of the first wavefront remain active, wherein the second wavefront stores data in the one or more registers from the first set of registers after the one or more registers have been reallocated. method.

8. The first computing unit includes executing the first wavefront instruction to free one or more registers, The method according to claim 7.

9. The first computing unit further includes executing an instruction to free all of the first set of registers. The method of claim 8.

10. The first wavefront remains active even after the first computing unit has executed the instruction. The method of claim 9.

11. The message includes the register base address, size, and identifier (ID) of the first wavefront. The method according to claim 7.

12. The method further includes blocking access to the predetermined register when the first computing unit executes the first wavefront instruction for accessing the predetermined register after freeing the predetermined register. The method according to claim 7.

13. It is a system, Memory and The system comprises a processor coupled to the memory, The aforementioned processor, Assigning a first set of registers to the first wavefront started on the computing unit, Assigning a second set of registers to the second wavefront started on the aforementioned computing unit, In response to the compute unit sending a message to the processor to free the one or more registers while one or more threads of the first wavefront remain active, one or more registers from a first set of registers are reallocated to the second wavefront, the second wavefront then stores data in the one or more registers from the first set of registers after the reallocation. It is configured to do, system.

14. The calculation unit executes the first wavefront instruction to free one or more registers. The system according to claim 13.

15. The calculation unit is configured to execute an instruction to release all of the first set of registers. The system according to claim 14.

16. The first wavefront remains active even after the computing unit has executed the instruction. The system according to claim 15.

17. The message includes the register base address, size, and identifier (ID) of the first wavefront. The system according to claim 13.

Images