Method and system for execution in a processor core with a source operand collection cache

Abstract

A method for executing instructions in a processor core implemented with a source operand collection cache coupled to at least one input of a data path located within the processor core, comprising the method: Decoding an instruction to determine a plurality of operands specified by the instruction, For each operand in the plurality of operands: Determine that the operand is not stored in the source operand collection cache, and load the operand from a local register into the source operand collection cache. Configuring the processor core to execute instructions on the operand stored in the source operand collection cache. Updating a cache table after the operand has been loaded into the source operand collection cache to associate a register index corresponding to the operand with a memory location in the source operand collection cache, determining which register indices to use as operands for the next instruction, to determine whether the register value corresponding to that register index is currently stored in the same memory location as a result of scheduling a previous instruction, and if the same memory location stores that register value, another register value from the same register bank is loaded into a different memory location during the same instruction cycle.

Description

BACKGROUND OF THE INVENTIONField of the invention

[0001] The present invention relates generally to processor registers and more specifically to methods for execution in a processor core implemented with a source operand collection cache and a system therefor. Description of the related art

[0002] Parallel processors have multiple independent cores, allowing multiple threads to execute concurrently using different hardware resources. SIMD (single instruction, multiple data) architecture processors execute the same instruction in each of multiple cores, with each core executing on different input data. MIMD (multiple instruction, multiple data) architecture processors execute different instructions in different cores, with each core being provided with different input data. Parallel processors may also be multi-threaded, allowing two or more threads to execute essentially concurrently using the resources of a single processing core (that is, the different threads execute during different clock cycles in the core).

[0003] When a processor schedules the execution of an instruction by a processor core, the processor writes certain values ​​to special registers in a register memory coupled to the processor core. One register may store the opcode that specifies the operation to be performed by the processor core, and additional registers may store operand values ​​that are used as input for the processor core to execute the instruction. To execute an operation, each of the values ​​in the register memory must be written and then coupled to the inputs of the data paths through a crossbar or other data transmission means. Each instruction may require new registers in the register memory to be connected to the inputs at the top of the data path.

[0004] A problem with the above architectures is that configuring the crossbar to couple values ​​stored in the register memory to the inputs located at the top of the data path requires one or more clock cycles to execute. The time required to load each operand introduces latency into the overall processing efficiency. The crossbar may further be configured so that only one operand may be coupled to the inputs of the data path during each clock cycle. US 4 530 050 A describes a processor unit with hit and miss operations on operands. JP 2010-113 470 A discloses a semiconductor circuit having a register in which, in the event of a miss operation, a register file of a cache is loaded.

[0005] Accordingly, it is an object of the present invention to provide an improved technique for loading values ​​from register memory into the inputs of a data path of a processor core. SUMMARY OF THE INVENTION

[0006] This object is achieved by the features of the independent claims. One embodiment provides a method for executing instructions in a processor core implemented with source operand collection caches according to claim 1. The method comprises, among other things, the steps of: decoding an instruction to determine a plurality of operands specified by the instruction, and for each operand in the plurality of operands: determining that a particular operand is not stored in a cache, which cache is coupled to inputs of a data path within the processor core, and loading the particular operand from a local register store associated with the processor core into the cache. The method further comprises the step of configuring the processor core to execute the instruction using operands stored in the cache.

[0007] Yet another embodiment provides a system according to claim 5. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram illustrating a computer system configured to implement one or more aspects of the present disclosure. Fig. 2 is a block diagram of a parallel processing subsystem for the computer system of Fig. 1, according to an embodiment of the present disclosure. Fig. 3A is a block diagram of the frontend of Fig. 2, according to an embodiment of the present disclosure. Fig. Figure 3B is a block diagram of a general processing cluster within one of the parallel processing units of Fig. 2, according to an embodiment of the present disclosure. Fig. Figure 3C is a block diagram of a portion of the streaming multiprocessor of Fig. 3B, according to an embodiment of the present disclosure. Fig. 4 is a block diagram of a portion of the streaming multiprocessor of Fig. 3B, according to another exemplary embodiment of the present disclosure. Fig. 5 shows the operand collector of Fig. 4, according to an exemplary embodiment of the present disclosure. Fig. 6 is a flowchart illustrating a method for executing instructions in a processor core implemented with source operand collection caches, according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0008] The disclosure describes a method and system for source operand collector caching. In one embodiment, a processor includes a register memory that may be coupled to storage elements (i.e., an operand collector) that provide inputs to the processor core's datapath for instruction execution. To reduce the bandwidth between the register memory and the operand collector, operands may be cached and reused in subsequent instructions. Consequently, only a subset of the operands specified by a given instruction may need to be loaded into the operand collector. A scheduler unit maintains a cache table for monitoring the register values ​​currently stored in the operand collector.The scheduler unit may also configure the operand collector to select the particular memory elements coupled to the inputs of the data path for a given instruction, making it possible for operands for two or more instructions to be cached simultaneously. System overview

[0009] Fig. 1 is a block diagram illustrating a computer system 100 configured to implement one or more aspects of the present disclosure. Computer system 100 includes a central processing unit (CPU) 102 and system memory 104 that communicate with each other via an interconnection path, which may include a memory bridge 105. Memory bridge 105, which may be, for example, a northbridge chip, is connected to an I / O (input / output) bridge 107 via a bus or other communication path 106 (for example, a HyperTransport link). I / O bridge 107, which may be, for example, a southbridge chip, receives user input from one or more user input devices 108 (e.g., keyboard, mouse) and passes the input to CPU 102 via communication path 106 and memory bridge 105.A parallel processing subsystem 112 is coupled to the memory bridge 105 via a bus or a second communication path 113 (e.g., a Peripheral Component Interconnect (PCI) Express, an accelerated graphics port, or a HyperTransport link); in one embodiment, the parallel processing subsystem 112 is a graphics subsystem that provides pixels to a display device 110 (e.g., a conventional cathode ray tube or liquid crystal display-based monitor). A system disk 114 is also connected to the I / O bridge 107. A switch 116 provides connections between the I / O bridge 107 and other components, such as a network adapter 118 and various add-in cards 120 and 121.Other components (not explicitly shown), including universal serial bus (USB) or other port connections, compact disc (CD) drives, digital video disc (DVD) drives, movie recorders, and the like, may also be connected to the I / O bridge 107. The various connection paths described in . Fig. 1, including the specifically named communication paths 106 and 113, may be implemented using any suitable protocols, such as PCI Express, AGP ("Accelerated Graphics Port"), HyperTransport, or any other bus or point-to-point communication protocol(s), and connections between different devices may use different protocols, as is known in the art.

[0010] In one embodiment, the parallel processing subsystem 112 includes circuitry optimized for graphics and video processing, including, for example, video output circuitry, and represents a graphics processing unit (GPU). In another embodiment, the parallel processing subsystem 112 includes circuitry optimized for general-purpose processing while maintaining the underlying computational architecture (as described in more detail herein). In yet another embodiment, the parallel processing subsystem 112 may be integrated with one or more other system elements into a single subsystem, such as by combining the memory bridge 105, the CPU 102, and the I / O bridge 107 to form a system-on-chip (SoC).

[0011] It will be understood that the system shown herein is illustrative and that variations and modifications are possible. The interconnect topology, including the number and arrangement of bridges, the number of CPUs 102, and the number of parallel processing subsystems 112, may be varied as desired. For example, in some embodiments, system memory 104 is connected directly to CPU 102 rather than through a bridge, and other devices communicate with system memory 104 via memory bridge 105 and CPU 102. In other alternative topologies, parallel processing subsystem 112 is connected to I / O bridge 107 or directly to CPU 102 rather than to memory bridge 105. In still other embodiments, I / O bridge 107 and memory bridge 105 may be integrated into a single chip rather than existing as one or more discrete devices.Larger embodiments may include two or more CPUs 102 and two or more parallel processing systems 112. The specific components shown herein are optional; for example, any number of expansion cards or peripheral devices may be supported. In some embodiments, the switch 116 is removed, and the network adapter 118 and the expansion cards 120, 121 are connected directly to the I / O bridge 107.

[0012] Fig. 2 shows a parallel processing subsystem 112 according to an embodiment of the present disclosure. The parallel processing subsystem 112 includes, as shown, one or more parallel processing units (PPUs) 202, each of which is coupled to a local parallel processing (PP) memory 204. In general, a parallel processing subsystem includes a number U of PPUs, where U ≥ 1. (Herein, multiple instances of similar objects are identified with reference numerals identifying the object and numbers in parentheses identifying the instance, where necessary.)) The PPUs 202 and the parallel processing memories 204 may be implemented using one or more integrated circuit devices, such as programmable processors, application specific integrated circuits (ASICs), or memory devices, or in any other technically feasible manner.

[0013] In some embodiments, again with reference to both Fig. 1 as well as on Fig. 2, some or all of the PPUs 202 in the parallel processing subsystem 112 may be graphics processors with rendering pipelines that may be configured to perform various operations related to generating pixel data from graphics data provided by the CPU 102 and / or system memory 104 via the memory bridge 105 and the second communication path 113, interacting with local parallel processing memory 204 (which may be used as graphics memory including, for example, a conventional frame buffer) to store and update pixel data, communicating pixel data to the display device 110, and the like. In some embodiments, the parallel processing subsystem 112 may include one or more PPUs 202 operating as graphics processors and one or more PPUs 202 used for general-purpose computations.The PPUs may be identical or different, and each PPU may have dedicated parallel processing memory device(s) or no dedicated parallel processing memory device(s). One or more of the PPUs 202 in the parallel processing system 112 may output data to a display device 110, or each PPU 202 in the parallel processing system 112 may output data to one or more display devices 110.

[0014] In operation, the CPU 102 is the master processor of the computer system 100, which controls and coordinates the operation of other system components. In particular, the CPU 102 issues instructions that control the operation of the PPUs 202. In some embodiments, the CPU 102 writes an instruction stream for each PPU 202 to a (neither Fig. 1 still in Fig. 2) that may be located in system memory 104, parallel processing memory 204, or another memory location accessible to both CPU 102 and PPU 202. A pointer to each data structure is written to a push buffer to initiate processing of the instruction stream in the data structure. PPU 202 reads instruction streams from one or more push buffers and then executes instructions asynchronously relative to the operation of CPU 102. Execution priorities may be set by an application program via device driver 103 for each push buffer to control the scheduling of the various push buffers.

[0015] Now both Fig. 2 as well as on Fig. 1, each PPU 202 includes an I / O (input / output) unit 205 that communicates with the rest of the computer system 100 via communication path 113 that connects to the memory bridge 105 (or, in an alternative embodiment, directly to the CPU 102). The connection of the PPU 202 to the rest of the computer system 100 may also vary. In some embodiments, the parallel processing subsystem 112 is implemented as an expansion card that can be inserted into an expansion slot of the computer system 100. In other embodiments, a PPU 202 may be integrated on a single chip with a bus bridge, such as memory bridge 105 or I / O bridge 107. In still other embodiments, some or all elements of the PPU 202 may be integrated on a single chip with the CPU 102.

[0016] In one embodiment, communication path 113 is a PCI Express link ("PCI Express Link") in which each PPU 202 is assigned dedicated lanes, as is known in the art. Other communication paths may also be used. An I / O unit 205 generates packets (or other signals) for transmission on communication path 113 and also receives any incoming packets (or other signals) from communication path 113, with the incoming packets being routed to appropriate components of PPU 202. Commands related to processing tasks may, for example, be routed to a host interface 206, while commands related to memory operations (e.g., reading from or writing to parallel processing memory 204) may be routed to a memory crossbar unit 210.The host interface 206 reads each burst buffer and outputs the instruction stream stored in the burst buffer to a front end 212.

[0017] Each PPU 202 advantageously implements a highly parallel processing architecture. The PPU 202(0), as shown in detail, includes a processing cluster array 230 comprising a number C of general processing clusters (GPCs) 208, where C ≥ 1. Each GPC 208 is capable of executing a large number (e.g., hundreds or thousands) of threads concurrently, where each thread is an instance of a program. In different applications, different GPCs 208 may be allocated to process different types of programs or to perform different types of computations. The allocation of GPCs 208 may vary depending on the load incurred for each type of program or computation.

[0018] The GPCs 208 receive processing tasks to be executed from a work distribution unit within a task / work unit 207. The work distribution unit receives pointers to computational processing tasks, encoded as task metadata (TMD) and stored in memory. The pointers to TMDs are included in the instruction stream, which is stored as a burst buffer and received from the host interface 206 via the front-end unit 212. Processing tasks, which may be encoded as TMDs, include indices of data to be processed, as well as state parameters and commands that define how the data is to be processed (e.g., which program is to be executed). The task / work unit 207 receives tasks from the front-end 212 and ensures that the GPCs 208 are configured to a valid state before initiating the processing specified by each of the TMDs.A priority may be specified for each TMD used to schedule the execution of processing tasks. Processing tasks may also be obtained from the processing cluster array 230. The TMD may optionally include a parameter controlling whether the TMD should be added to the head or tail of a list of processing tasks (or list of pointers to the processing tasks), providing a further level of control over priority.

[0019] The memory interface 214 includes a number D of partition units 215, each directly coupled to a portion of the parallel processing memory 204, where D ≥ 1. The number of partition units 215 is, as shown, generally equal to the number of dynamic random access memory (DRAM) 220. In other embodiments, the number of partition units 215 may not equal the number of memory devices. Those skilled in the art will understand that DRAM 220 may be replaced by other suitable memory devices and may be of a generally conventional design. A detailed description is therefore omitted. Render targets, such as buffer or structure maps, may be stored across the DRAMs 220, allowing the partition units 215 to write portions of each render target in parallel to efficiently utilize the available bandwidth of the parallel processing memory 204.

[0020] Each of the GPCs 208 may process data to be written to any of the DRAMs 220 within the parallel processing memory 204. The crossbar unit 210 is configured to route the output of each GPC 208 to the input of each partition unit 215 or to another GPC 208 for further processing. The GPCs 208 communicate with the memory interface 214 through the crossbar unit 210 to read from and write to various external storage devices. In one embodiment, the crossbar unit 210 has both a connection to the memory interface 214 to communicate with the I / O unit 205 and a connection to the local parallel processing memory 204, thereby enabling the processing cores within the various GPCs 208 to communicate with the system memory 104 or other memory that is not a local part of the PPU 202. In the Fig. In the embodiment shown in Figure 2, the crossbar unit 210 is directly connected to the I / O unit 205. The crossbar unit 210 may use virtual channels to separate traffic flows between the GPCs 208 and the partition units 215.

[0021] As mentioned, the GPCs 208 can be programmed to perform processing tasks related to a wide variety of applications, including, but not limited to, linear and non-linear data transformations, filtering of video and / or audio data, modeling operations (e.g., the application of physical laws to determine position, velocity, and other properties of objects), image rendering operations (e.g., tessellation shader, vertex shader, geometry shader, and / or pixel shader programs), etc.The PPUs 202 may transfer data from the system memory 104 and / or the local parallel processing memories 204 into an internal (on-chip) memory, process the data, and write the resulting data back to the system memory 104 and / or the local parallel processing memories 204, where such data may be accessed by other system components, including the CPU 102 or other parallel processing subsystems 112.

[0022] A PPU 202 may be equipped with any amount of local parallel processing memory 204, including no local memory, and may use local memory and system memory in any combination. For example, a PPU 202 may be a graphics processor in a unified memory architecture (UMA) embodiment. In such embodiments, little or no dedicated graphics (parallel processing) memory would be provided, and the PPU 202 would use system memory exclusively or almost exclusively. In UMA embodiments, a PPU 202 may be integrated into a bridge chip or processor chip, or may be provided as a discrete chip with a high-speed connector (e.g., PCI Express) connecting the PPU 202 to system memory via a bridge chip or other communication means.

[0023] As mentioned above, any number of PPUs 202 may be included in a parallel processing subsystem 112. For example, multiple PPUs 202 may be provided on a single expansion card, or multiple expansion cards may be connected to the communication path 113, or one or more of the PPUs 202 may be integrated into a bridge chip. The PPUs 202 in a multiple-PPU system may be the same or different from one another. For example, different PPUs 202 may each have a different number of processor cores, different amounts of local parallel processing memory, etc. When multiple PPUs 202 are present, these PPUs may operate in parallel to process data at a greater throughput than is possible with a single PPU 202.Systems having one or more PPUs 202 may be implemented in a variety of configurations and form factors, including desktop, laptop or handheld computers, servers, workstations, game consoles, embedded systems, and the like. Scheduling of multiple concurrent tasks (“Multiple Concurrent Task Scheduling”)

[0024] Multiple processing tasks may be executed concurrently on the GPCs 208, and a processing task may spawn one or more "child" processing tasks during execution. The task / work unit 207 receives the tasks and dynamically schedules the execution of the processing tasks and the child processing tasks using the GPCs 208.

[0025] Fig. Figure 3A is a block diagram of the task / work unit 207 of the Fig. 2, according to one embodiment of the present disclosure. The task / work unit 207 includes a task management unit 300 and the work distribution unit 340. The task management unit 300 organizes the tasks to be scheduled based on execution priority levels. For each priority level, the task management unit 300 stores a list of pointers to the TMDs 322 corresponding to the tasks in the scheduler table 321, which list may be implemented as a linked list. The TMDs 322 may be stored in the PP memory 204 or in the system memory 104. The rate at which the task management unit 300 accepts tasks and stores the tasks in the scheduler table 321 is decoupled from the rate at which the task management unit 300 schedules the execution of tasks. Consequently, the task management unit 300 may collect several tasks before scheduling their execution.The collected tasks may then be scheduled based on priority information or using other techniques, such as round-robin scheduling.

[0026] The work distribution unit 340 includes a task table 345 with slots, each of which may be occupied by the TMD 322 for a task being executed. The task management unit 300 may schedule tasks for execution if there is a free slot in the task table 345. If there is no free slot, a higher priority task that does not occupy a slot may evict a lower priority task that does occupy a slot. When a task is evicted, the task is stopped, and if the task's execution is not complete, a pointer to the task is added to a list of task pointers to be scheduled so that the task's execution can be resumed at a later time. If a child processing task is created during execution of a task, a pointer to the child task is added to the list of task pointers to be scheduled.A child task may be generated by a TMD 322 executing in the processing cluster array 230.

[0027] Unlike a task obtained by the task / work unit 207 from the front end 212, child tasks are obtained from the processing cluster array 230. Child tasks are not inserted into a bump buffer or transmitted to the front end. The CPU 102 is not notified when a child task is created or data for the child task is stored in memory. Another difference between tasks provided by bump buffers and child tasks is that the tasks provided by bump buffers are defined by the application program, while child tasks are dynamically generated during task execution. Overview of task processing

[0028] Fig. Figure 3B is a block diagram of a GPC 208 within one of the PPUs 202 of Fig. 2, according to one embodiment of the disclosure. Each GPC 208 may be configured to execute a large number of threads in parallel, where the term "thread" refers to an instance of a particular program executing on a particular set of input data. In some embodiments, single-instruction, multiple-data (SIMD) instruction issue techniques are used to support parallel execution of a large number of threads without providing multiple independent instruction units.In other embodiments, single-instruction, multiple-thread (SIMT) techniques are used to support parallel execution of a large number of generally synchronized threads, using a common instruction unit configured to issue instructions to a set of processing engines within each of the GPCs 208. Unlike a SIMD execution regime, in which all processing engines typically execute identical instructions, SIMT execution allows different threads to more expeditiously follow diverging execution paths through a given threaded program. Those skilled in the art will understand that a SIMD processing mode is a functional subset of a SIMT processing mode.

[0029] The operation of the GPC 208 is advantageously controlled by a pipeline manager 305, which distributes processing tasks to streaming multiprocessors (SMs) 310. The pipeline manager 305 may also be configured to control a work distribution crossbar 330 by specifying destinations for processed data output from the SMs 310.

[0030] In one embodiment, each GPC 208 comprises a number M of SMs 310, where M ≥ 1 and each SM 310 is configured to process one or more thread groups. Each SM 310 advantageously also comprises an identical set of functional execution units (e.g., execution units and load-store units), which are referred to as execution units 302 and LSUs 303 in Fig. 3C), which may be pipelined, allowing a new instruction to be issued before a previous instruction is complete, as is known in the art. Any combination of functional execution units may be provided. In one embodiment, the functional units support a variety of operations, including integer and floating-point arithmetic (e.g., addition and multiplication), comparison operations, Boolean operations (AND, OR, XOR), bit shifting, and computation of various algebraic functions (e.g., planar interpolation, trigonometric, exponential, and logarithmic functions, etc.); and the same functional unit hardware may be leveraged to perform different operations.

[0031] The series of instructions submitted to a particular GPC 208 constitutes a thread, as defined earlier herein, and the collection of a particular number of threads executing concurrently across the parallel processing engines (not shown) within an SM 310 is referred to as a "warp" or a "thread group." As used herein, a "thread group" refers to a group of threads concurrently executing the same program on different input data, with one thread of the group being associated with a different processing engine within an SM 310. A thread group may contain fewer threads than the number of processing engines within the SM 310, in which case some processing engines will be idle during cycles in which that thread group is processing.A thread group may also contain more threads than the number of processing engines within the SM 310, in which case processing will occur in consecutive cycles. Since each SM 310 can support up to G thread groups simultaneously, it follows that up to G*M thread groups can be executed in the GPC 208 at any given time.

[0032] Furthermore, a plurality of related thread groups may be active at the same time within an SM 310 (in different phases of execution). This collection of thread groups is referred to herein as a "cooperative thread array" (CTA) or "thread array." The size of a particular CTA is equal to m*k, where k is the number of concurrently executing threads in a thread group, which is typically an integer multiple of the number of parallel processing engines within the SM 310, and m is the number of concurrently active thread groups within the SM 310. The size of a CTA is generally determined by the programmer and the amount of hardware resources available to the CTA, such as memory or registers.

[0033] Each SM 310 contains a level one (L1) cache (shown in Fig. 3C) or uses space in a corresponding L1 cache external to the SM 310, which is used to perform load and store operations. Each SM 310 also has access to level two (L2) caches that are shared by all GPCs 208 and may be used to transfer data between threads. Finally, the SMs 310 also have access to external ("off-chip") "global" memory, which may include, for example, parallel processing memory 204 and / or system memory 104. It should be understood that any memory external to the PPU 202 may be used as global memory.A level one point five (L1,5) cache 335 may additionally be included in the GPC 208, which is configured to receive and hold data requested by the SM 310 and retrieved from memory via the memory interface 214, including instructions, uniform data, and constant data, and to deliver the requested data to the SM 310. Embodiments having multiple SMs 310 in the GPC 208 advantageously share common instructions and data stored in the L1,5 cache 335.

[0034] Each GPC 208 may include a memory management unit (MMU) 328 configured to map virtual addresses to absolute ("physical") addresses. In other embodiments, the MMU(s) 328 may be located within the memory interface 214. The MMU 328 includes a set of page table entries (PTEs) used to map a virtual address to an absolute address of a tile and optionally a cache line index. The MMU 328 may include address translation lookaside buffers (TLBs) or address translation lookaside caches that may be located within the multiprocessor SM 310 or the L1 cache or the GPC 208.The physical address is processed to distribute surface data access locality to allow efficient request interleaving between partition units 215. The cache line index may be used to determine whether a request for a cache line is a hit or a miss.

[0035] In graphics and computing applications, a GPC 208 may be configured such that each SM 310 is coupled to a texture unit 315 to perform texture mapping operations, such as determining texture sample positions, reading texture data, and filtering the texture data. Texture data is read from an internal texture L1 cache (not shown) or, in some embodiments, from the L1 cache within the SM 310, and is retrieved as needed from an L2 cache shared between all GPCs 208, the parallel processing memory 204, or the system memory 104. Each SM 310 outputs processed tasks to the work distribution crossbar 330 to provide the processed tasks to another GPC 208 for further processing or to store the processed tasks in an L2 cache, parallel processing memory 204, or system memory 104 via the crossbar unit 210.A pre-ROP (pre-raster operations) 325 is configured to receive data from the SM 310, route data to ROP units within partition units 215, and perform optimizations for color blending, organizing pixel color data, and performing address translations.

[0036] It is understood that the core architecture described herein is illustrative and that variations and modifications are possible. Any number of processing units, for example, SMs 310 or texture units 315, pre-ROPs 325 may be included in a GPC 208. Furthermore, a PPU 202, as described in Fig. 2, any number of GPCs 208, which are advantageously functionally similar to one another, so that the course of execution does not depend on which GPC 208 receives a given processing task. Furthermore, each GPC 208 advantageously operates independently of other GPCs 208, using separate and individual processing units and L1 caches to execute tasks for one or more application programs.

[0037] Average professionals will understand that the Fig. 1, Fig. 2, Fig. 3A and Fig. 3B does not limit the scope of the present invention in any way, and that the techniques described herein may be implemented in any properly configured processing unit, including, without limitation, one or more CPUs, one or more multi-core CPUs, one or more PPUs 202, one or more GPCs 208, one or more graphics or special purpose processing units, or the like, without departing from the scope of the present invention.

[0038] In embodiments of the present invention, it is desirable to use PPU 202 or other processor(s) of a data processing system to perform general computations using thread arrays. Each thread in the thread array is assigned a unique thread identifier ("thread ID") that is accessible to the thread during its execution. The thread ID, which may be defined as a one-dimensional or multi-dimensional numeric value, controls various aspects of the thread's execution process. For example, a thread ID may be used to determine which portion of an input data set a thread should process and / or to determine which portion of an output data set a thread should produce or write.

[0039] A sequence of per-thread instructions may include at least one instruction that specifies cooperative behavior between the representative thread and one or more other threads of the thread array. The sequence of per-thread instructions may, for example, include an instruction to suspend execution of operations for the representative thread at a particular point in the sequence until one or more of the other threads reach that particular point, an instruction for the representative thread to store data in shared memory accessible to one or more of the other threads, an instruction for the representative thread to atomically read and update data stored in shared memory accessible to one or more of the other threads based on their thread IDs, or similar instructions.The CTA program may also include an instruction to calculate an address in the shared memory from which data is to be read, where the address is a function of the thread ID. By defining appropriate functions and providing synchronization techniques, data can be written by one thread of a CTA to a given location in the shared memory and read from that location by another thread of the same CTA in a predictable manner. Consequently, any desired pattern of data sharing between threads can be supported, and any thread in a CTA can share data with any other thread in the same CTA.The extent, if any, of data sharing between threads of a CTA is determined by the CTA program; thus, it is understood that the threads of a CTA in a given application using CTAs may or may not actually share data with each other, depending on the CTA program, and the terms "CTA" and "thread array" are used interchangeably herein.

[0040] Fig. 3C is a block diagram of the SM 310 from Fig. 3B, according to one embodiment of the present disclosure. The SM 310 includes an instruction L1 cache 370 configured to retrieve instructions and constants from memory through L1.5 cache 335. A warp scheduler and instruction unit 312 retrieves instructions and constants from the instruction L1 cache 370 and controls the local register file 304 and functional units of the SM 310 according to the instructions and constants. The functional units of the SM 310 include N execution or processing units 302 and P load-store units (LSU) 303.

[0041] The SM 310 provides internal ("on-chip") data storage with various levels of accessibility. Special registers (not shown) are readable but not writable by the LSU 303 and are used to store information defining the "position" of each thread. In one embodiment, the special registers include one register per thread (or per execution unit 302 in the SM 310) that stores a thread ID; each thread ID register is accessible only to a corresponding execution unit of the execution units 302.The special registers may also contain additional registers readable by all threads executing the same processing task represented by a TMD 322 (or by all LSUs 303), which contain a CTA identifier, the CTA dimensions, the dimensions of a grid to which the CTA belongs (or the queue position if the TMD 322 encodes a queue task rather than a grid task), and an identifier of the TMD 322 to which the CTA is stored.

[0042] If the TMD 322 is a network TMD, then the execution of the TMD 322 causes a fixed number of CTAs to be started and executed to execute the fixed amount of data stored in the queue 525. The number of CTAs is specified as the product of the width, height, and depth of the network. The fixed amount of data may be stored in the TMD 322, or the TMD 322 may store a pointer to the data that will be executed by the CTAs. The TMD 322 also stores a starting address of the program executed by the CTAs.

[0043] If the TMD 322 is a queue TMD, then a queue property of the TMD 322 is used, meaning that the amount of data to be processed is not necessarily fixed. Queue entries store data to be processed by the CTAs associated with the TMD 322. The queue entries may also represent a child task spawned by another TMD 322 during execution of a thread, providing nested parallelism. The execution of the thread or the CTA containing the thread is typically suspended until the execution of the child task is complete. The queue may be stored in or separately from the TMD 322, in which case the TMD stores a queue pointer to the queue.Data generated by the child task can advantageously be written to the queue while the TMD 322 representing the child task is executing. The queue may be implemented as a circular queue, so that the total amount of data is not limited by the size of the queue.

[0044] CTAs belonging to a network have implicit network width, height, and depth parameters that specify the position of the respective CTA within the network. The special registers are written during initialization in response to commands received from the device driver 103 via the frontend 212 and do not change during the execution of a processing task. The frontend 212 schedules the execution of each processing task. Each CTA is associated with a specific TMD 322 for concurrent execution of one or more tasks. A single GPC 208 may also execute multiple tasks concurrently.

[0045] A parameter memory (not shown) stores runtime parameters (constants) that can be read but not written by any other thread within the same CTA (or any LSU 303). In one embodiment, device driver 103 provides parameters to the parameter memory before instructing SM 310 to begin executing a task that uses those parameters. Each thread within each CTA (or each execution unit 302 within SM 310) can access global memory through a memory interface 214. Portions of global memory may be stored in L1 cache 320.

[0046] The local register memory 304 is used by each thread as scratch space; each register is allocated for the exclusive use of one thread, and data in any of the local register memories 304 is accessible only by the thread to which the register is allocated. The local register memory 304 can be implemented as a register memory physically or logically divided into P lanes, each lane having any number of entries (each entry may store, for example, a 32-bit word). One lane is assigned to each of the N execution units 302 and P load-store units (LSUs) 303, and corresponding entries in different lanes can be filled with data for different threads executing the same program to simplify SIMD execution.Different portions of the lanes can be allocated to different thread groups of the G concurrent thread groups, so that a given entry in the local register memory 304 is accessible only to a specific thread. In one embodiment, certain entries in the local register memory 304 are reserved for storing thread identifiers to implement one of the special registers. A uniform L1 cache 375 additionally stores uniform or constant values ​​for each lane of the N execution units 302 and P load-store units (LSUs) 303.

[0047] The shared memory 306 is accessible to threads within a single CTA; in other words, any location in the shared memory 306 is accessible to any thread within the same CTA (or to any processing engine within the SM 310). The shared memory 306 may be implemented as a shared register memory or as shared on-chip memory with an interconnect that allows each processing engine to read and write to and from any location in the shared memory. In other embodiments, a shared state space may map to a per-CTA area of ​​off-chip memory and be cached in the L1 cache 320. The parameter memory may be implemented as a designated section within the same shared register memory or shared cache memory.implementing the shared memory 306, or as a separate shared register memory or internal ("on-chip") cache memory to which the LSUs 303 have read-only access. In one embodiment, the area implementing the parameter memory is also used to store the CTA ID and task ID, as well as CTA and network dimensions or queue position, implementing portions of the special registers. Each LSU 303 in the SM 310 is coupled to a unified address mapping unit 352, which converts an address provided for load and store instructions specified in a unified memory space into an address in each distinct memory space.Consequently, an instruction may be used to access any of the local, shared, or global memory spaces by specifying an address in the unified memory space.

[0048] The L1 cache 320 in each SM 310 can be used to cache private per-thread local data as well as per-application global data. In some embodiments, the per-CTA shared data may be cached in the L1 cache 320. The LSUs 303 are coupled to the shared memory 306 and the L1 cache 320 via a memory and cache interconnect 380. Caching operands

[0049] Fig. 4 is a block diagram of the SM 310 of Fig. 3B, according to another exemplary embodiment of the present disclosure. Although not explicitly shown, the SM 310 of Fig. 4 some or all of the above-described components of the SM 310 from Fig. 3C, in addition to those in Fig. 4 explicitly shown components. As in Fig. 4, the SM 310 includes the warp scheduler and instruction unit 312, the local register memory 304, and one or more functional execution units, such as the execution unit 302 or the LSU 303. The warp scheduler and instruction unit 312 includes a decode unit 450 and a dispatch unit 470. The decode unit 450 receives the next instruction to be dispatched to the execution unit 302. The decode unit 450 performs a full decode of the instruction and transmits the decoded instruction to the dispatch unit 470. The decode unit 450 will, for example, determine the particular type of instruction specified by the opcode in the instruction and the particular register indices used as Operands to the instruction are specified, as well as a register index for storing the result of an instruction.In some embodiments, instructions may be generated in dual or quadruple form, and the decode unit 450 may implement separate logic for each generated instruction. The dispatch unit 470 implements a FIFO and writes the decoded values ​​to the local register memory 304 for execution. In embodiments that generate multiple instructions concurrently, the dispatch unit 470 may dispatch each instruction to a different portion of the functional units of the SM 310.

[0050] In one embodiment, local register memory 304 includes four banks of registers (Bank_0 422, Bank_1 424, Bank_2 426, Bank_3 428). Local register memory may be relatively small in most conventional processing units. For example, the x86 CPU architecture has 8 (32-bit) or 16 (64-bit) registers per processor core. In contrast, each bank of local register memory 304 may contain a large number of registers, such as 256 registers or more, available for use as inputs to execution units 302. Crossbar 420 is configured to connect the various registers in each of the register banks to an operand collector 440 in an execution unit 302. Execution unit 302 implements a datapath for executing an operation. The data path includes the operand collector 440, the arithmetic logic unit (ALU) 452, and a result FIFO 454.The operand collector 440 includes a number of storage elements 441-446 that may be coupled to the inputs of the ALU 452. Each storage element 441-446 may be a flip-flop, latch, or any other technically feasible circuit device capable of temporarily storing a value to be input to the ALU 452. The outputs of the storage elements may be hard-wired to the circuit elements comprising the ALU 452, such as an addition circuit or a floating-point multiplication circuit. The operand collector 440 includes six storage elements 441-446, as shown. In other embodiments, any number of storage elements 441-446 may be implemented in the execution unit 302.

[0051] In conventional processor designs that have relatively small register memories, a crossbar or other interconnect may be configured to couple any register in a local register memory to any of the data path inputs during a single clock cycle. Furthermore, in conventional processors, a number of registers may be connected simultaneously to any of the data path inputs during a single clock cycle, where the number of registers is equal to the number of inputs to the data path. It will be understood that increasing the size of the local register memory 304 to more than 8 or 16 registers may increase the complexity of the crossbar 420. Consequently, in some embodiments, the crossbar 420 may couple only a single register from each register bank (e.g., 422, 424, 426, and 428) to a particular memory element 441-446 during each clock cycle.

[0052] The processor architecture described above (where crossbar 420 couples only a single register per register bank to operand collector 440 during a clock cycle) reduces the size and complexity of crossbar 420, but the processor architecture also introduces various restrictions on the execution of instructions by execution unit 302. For example, the bandwidth between local register memory 304 and operand collector 440 during each clock cycle is equal to the number of register banks in local register memory 304. Local register memory 304 has four register banks, as shown, allowing up to four operands to be written to operand collector 440 during a single clock cycle.However, since operand collector 440 includes six storage elements 441-446 for providing inputs to ALU 452 during each instruction, it may be necessary to load the operands in two consecutive clock cycles (four operands in the first clock cycle and two operands in the second clock cycle). Furthermore, care must be taken to prevent register bank conflicts by storing operand values ​​required for the same instruction in different register banks 422, 424, 426, and 428. For example, if all six operands are stored in Bank_0 422, then six clock cycles are required to load the operands into operand collector 440. In one embodiment, dispatch unit 470 includes logic configured to prevent bank conflicts by ensuring that operands for each instruction are stored in different register banks 422, 424, 426, and 428.In another embodiment, driver 103 may include implementation instructions that cause driver 103 to check for and prevent bank conflicts during compilation of parallel processor instructions before they are passed to PPU 202. Even with the implementation of such techniques, some bank conflicts may not be readily resolved.

[0053] In one embodiment, operand collector 440 implements simultaneous caching of operands associated with multiple instructions within operand collector 440. As shown, operand collector 440 includes one or more storage elements (441(0)-441(R-1); 442(0)-442(R-1); etc.) corresponding to each input to ALU 452. Dispatch unit 470 may include logic to control which sets of storage elements 441(i)-446(i) should be connected to ALU 452 for each instruction. For example, a program executing in ALU 452 may include a set of instructions generated by driver 103 and transmitted to PPU 202, as shown in Table 1. A first instruction may be a fused multiply-add (FMA) instruction that has three operands (R13, R11, R14) and one output (R1).A second instruction may be an addition (ADD) instruction having three operands (R6, R7, R8) and an output (R2). A third instruction may be a multiplication (MUL) instruction having three operands (R6, R11, R13). Operands specified as X are not used as input to the ALU 452 for the given instruction and may be ignored for the purposes of this disclosure. Table 1 Nr. Op-Code Ausgabe Operanden 1 FMA R1 R13, R11, R14, X, X, X 2 ADD R2 R6, R7, R8, X, X, X 3 MUL R3 R6, R11, R13, X, X, X

[0054] The operand collector 440 may reuse an operand from the previous instruction during the next instruction. During the second instruction above, the operands stored in register 6, register 7, and register 8 in the local register memory 304 are, for example, loaded into storage elements 441, 442, and 443, respectively. During the third instruction above, the value in storage element 441 may be reused, and only the operand values ​​stored in register 11 and register 13 in the local register memory 304 are loaded into the operand collector 440. In other words, storage elements 441-446 do not need to be reloaded during each instruction. The dispatch unit 470 may include logic that stores the indices for each register currently loaded into the operand collector 440.The logic included in dispatch unit 470 may be implemented as a lookup table (i.e., a cache table) having a slot corresponding to each storage element 441-446. Each slot may store a value indicating the register index associated with the value stored in the corresponding storage element 441-446. When decoding unit 450 determines which registers (i.e., which register indices) are to be used as operands for the next instruction, dispatch unit 470 will check to determine whether the register value corresponding to that register index is currently stored in the same storage element 441-446 as a result of scheduling a previous instruction.If the same storage element 441-446 stores this register value, another register value from the same register bank may advantageously be loaded into another storage element 441-446 during the same instruction cycle.

[0055] In another embodiment, operand collector 440 may store multiple sets of register values ​​for a given datapath input of ALU 452 (i.e., operands 1-6 stored in storage elements 441-446, respectively). In such embodiments, operand collector 440 may include a set of R storage elements 441(0), 441(1), ..., 441(R-1) for each datapath input of ALU 452. Accordingly, the operands from the resulting R instructions may be buffered in operand collector 440 and reused in subsequent instructions to reduce the bandwidth required to load operands from local register memory 304 into operand collector 440.For example, if R is equal to two, then during the first instruction described above in Table 1, register 13 is loaded into storage element 441(0), register 11 is loaded into storage element 442(0), and register 14 is loaded into storage element 443(0). Subsequently, during the second instruction, register 6 is loaded into storage element 441(1), register 7 is loaded into storage element 442(1), and register 8 is loaded into storage element 443(1). During the third instruction, the first operand value (register 6) is stored in storage element 441(1) and the second operand value (register 11) is stored in storage element 442(0).

[0056] In some embodiments, a given subset of memory elements 441(i)-446(i) loaded during a given instruction must be simultaneously selected for use with the subsequent instruction. In other words, during the third instruction discussed above, the dispatch unit could reuse either the memory elements 441(0)-446(0) loaded during the first instruction or the memory elements 441(1)-446(1) loaded during the second instruction, but not the first memory element loaded during the second instruction 441(1) (i.e., storing the value in register 6) or the second memory element loaded during the first instruction 442(0) (i.e., storing the value in register 11).In other embodiments, the dispatch unit 470 may include logic for selecting between each individual storage element 441-446, thereby allowing register values ​​stored in different sets of storage elements 441(i)-446(i) during different instructions to be reused in a subsequent instruction.

[0057] It is understood that cache coherence may be monitored to ensure that the values ​​in storage elements 441-446 are equal to the values ​​in the corresponding registers in local register memory 304. In one embodiment, the slots in the cache table store an index value that specifies the register corresponding to the value stored in the associated storage element 441-446. Whenever a new value is written to a register in local register memory 304, the cache table is always searched to determine if an index corresponding to that register is currently stored in operand collector 440. If operand collector 440 contains an entry corresponding to that register, the cache table is updated by removing that index (i.e., the value in that slot from the cache table is set to zero to invalidate the data in operand collector 440).Consequently, if this register is then specified as an operand in a subsequent instruction, the dispatch unit 470 will reload the value from the local register memory 304 into the operand collector 440, ensuring that invalid values ​​are not provided to the ALU 452.

[0058] Fig. 5 shows the operand collector 440 of the Fig. 4, according to an exemplary embodiment of the present disclosure. As shown in Fig. 5, the operand collector 440 contains the storage elements 441-446 of the Fig. 4. As described above, the storage elements 441-446 may implement a cache for storing register values ​​coupled to the data path inputs of the ALU 452. The crossbar 420 is configured by the dispatch unit 470 to couple various registers in the local register memory 304 to the storage elements 441-446. The operand collector 440 also includes a plurality of multiplexers 511-516. The multiplexer 511 has a first input coupled to a first storage element 441(0) and a second input coupled to a second storage element 441(1). The multiplexer 511 also has an output coupled to a first data path input of the ALU 452 that provides a first operand value to the logic of the ALU 452. The multiplexer 512 has a first input coupled to a first storage element 442(0) and a second input coupled to a second storage element 442(1).Multiplexer 512 also has an output coupled to a second data path input of ALU 452, which provides a second operand value to the logic of ALU 452. Multiplexers 513, 514, 515, and 516 have respective connections similar to those described above with respect to multiplexers 511 and 512 for the third, fourth, fifth, and sixth data path inputs of ALU 452.

[0059] The dispatch unit 470 configures the multiplexers 511, 512, 513, 514, 515, and 516 to select between the different sets of memory elements 441(i)-446(i). The control signal coupled to the multiplexers 511-516 may, as shown in Fig. 5, may be coupled to each of the multiplexers to select all of the memory elements 441(i)-446(i) corresponding to the same previously loaded instruction. In alternative embodiments, dispatch unit 470 may supply a different control signal to each of the multiplexers 511-516 so that memory elements 441(i)-446(i) corresponding to different instructions may be coupled to the data path inputs of ALU 452.

[0060] In alternative embodiments, operand collector 440 may include different interconnect logic that allows each of the storage elements 441-446 to be connected to each of the data path inputs of ALU 452. For example, one or more crossbars may be implemented within operand collector 440 instead of multiplexers 511-516. The one or more crossbars may allow, as an example, storage element 442(1) to be connected to the fourth data path input of ALU 452 and storage element 443(0) to be connected to the first data path input of ALU 452. In other words, an operand will result in an operand cache 440 hit if the operand has previously been loaded into any of the storage elements of operand cache 440.Thus, with reference to the instructions given above in Table 1, all three operations specified by the third instruction may be reused because the three operands specified by the third instruction were previously loaded into at least one of the memory elements of the operand cache 440 during the two previous instructions.

[0061] Fig. 6 is a flowchart illustrating a method 600 for executing instructions in a processor core implemented with source operand collection caches, according to an exemplary embodiment of the present disclosure. Although the method steps are described in connection with the systems of Fig.1, 2, 3A-3C, 4 and 5, those of ordinary skill in the art will understand that any system configured to perform the method steps in any order is included within the scope of the present disclosure.

[0062] The method 600 begins at step 610, where the SM 310 decodes an instruction to determine a plurality of operands specified by the instruction. Each operand may be specified as an index corresponding to a register within the local register memory 304. At step 612, the SM 310 determines whether one or more of the operands are cached in the operand collector 440. The SM 310 may maintain a cache table that stores indexes corresponding to the registers stored in the operand collector 440. The dispatch unit 470 may compare the decoded register indexes corresponding to the operands specified by the instruction with the indexes stored in the cache table.

[0063] At step 614, the SM 310 configures the crossbar 420 to couple the specified registers in the local register memory 304 to the storage elements 441-446 in the operand collector 440 for each operand not stored in the operand collector 440. At step 616, the SM 310 updates the cache table to reflect the register indices corresponding to the register values ​​stored in the operand collector 440. At step 618, the SM 310 configures the ALU 452 to execute the instruction using the operands stored in the operand collector 440 as inputs to the ALU 452. At step 620, the SM 310 determines whether there are any further instructions scheduled for execution in the ALU 452. If further instructions are scheduled for execution, then the method 600 returns to step 610 where the next instruction is decoded.However, if no further instructions are scheduled for execution, then method 600 is terminated.

[0064] An embodiment of the disclosure may be implemented as a program product for use with a computer system. The program(s) of the program product define(s) functions of the embodiments (including the methods described herein) and may be embodied on a variety of computer-readable storage media.Example computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., read-only storage devices within a computer, such as non-writable compact disc (CD-ROM) discs readable by a CD-ROM drive, flash memory, non-writable memory (ROM) chips, or any other type of solid-state non-volatile semiconductor memory) on which information is permanently stored; and (ii) writable storage media (e.g., floppy disks in a floppy disk drive or hard disk drive, or any type of solid-state random-access semiconductor memory) on which alterable information is stored.

Claims

[1] A method for executing instructions in a processor core implemented with a source operand collection cache coupled to at least one input of a data path located within the processor core, the method comprising: Decoding an instruction to determine a plurality of operands specified by the instruction, for each operand in the plurality of operands: determining that the operand is not stored in the source operand collection cache, and loading the operand from a local register store into the source operand collection cache, Configuring the processor core to execute instructions over the operand stored in the source operand collection cache. Updating a cache table after the operand has been loaded into the source operand collection cache to associate a register index corresponding to the operand with a storage element in the source operand collection cache, determining which register indices to use as operands for a next instruction, determining whether the register value corresponding to that register index is currently stored in a same storage element as a result of scheduling a previous instruction, and if the same storage element stores that register value, loading a different register value from the same register bank into a different storage element during the same instruction cycle. [2] The method of claim 1, wherein the local register memory comprises a plurality of register banks. [3] The method of claim 2, further comprising configuring a crossbar to couple a register in a first register bank of the plurality of register banks to a first storage element in the source operand collection cache. [4] The method of claim 1, wherein the source operand collection cache is dimensioned to simultaneously store the plurality of operands specified by the instruction and a plurality of operands specified by another decoded instruction. [5] A system comprising: a processor core having a data path for executing instructions, a source operand collection cache coupled to at least one input of the data path, a local register memory having a plurality of register banks coupled to the source operand collection cache, and a scheduling unit configured to: decode an instruction to determine a plurality of operands specified by the instruction, for each operand in the plurality of operands: determining that the operand is not stored in the source operand collection cache and loading the operand from a local register store into the source operand collection cache, and to configure the processor core to execute instructions over the operand stored in the source operand collection cache, the system further comprising a cache table, and the scheduling unit further configured to update the cache table after the operand has been loaded into the source operand collection cache, to associate a register index corresponding to the operand with a storage element in the source operand collection cache, determining which register indices are to be used as operands for a next instruction, to determine whether the register value corresponding to that register index is currently stored in a same storage element as a result of scheduling a previous instruction, and if the same storage element stores that register value,another register value is loaded from the same register bank into another memory element during the same instruction cycle. [6] The system of claim 5, further comprising a crossbar configured to couple a register in a first register bank of the plurality of register banks to a first storage element in the source operand collection cache. [7] The system of claim 5, wherein the source operand collection cache is dimensioned to simultaneously store the plurality of operands specified by the instruction and a plurality of operands specified by another decoded instruction.

Images