Unloading function flow

Abstract

The present application describes a technique for offloading a function stream. In some examples, a function is a sequence of instructions, and a stream is a sequence of functions. In some examples, a coprocessor handles functions and / or function streams provided by a main processor. In some examples, a coprocessor includes a plurality of execution resources, which include at least one or more of a direct memory access (DMA) engine, an atomic engine, and a collective engine.

Description

Technical Field

[0001] The present application relates generally to the field of processors, and more particularly to offloading function flows. Background Art

[0002] The Advanced Graphic Intelligence Logical Computing Environment (AGILE) project aims to design a system that targets workflows that exhibit ultra-sparse graph properties. These workflows can take advantage of specialized operations that exploit common behaviors in graph algorithms, thereby saturating both network and memory bandwidth - a common performance goal for applications with low compute intensity. Summary of the invention

[0003] According to one aspect of the present application, a device is provided, comprising: a function identifier controller for handling function definition instructions, wherein a function is a sequence of instructions including at least one direct memory access operation or a collective operation; an instruction memory for storing functions; an execution engine for reading functions from the instruction memory; and multiple execution resources for executing the read functions, the multiple execution resources including at least a direct memory access (DMA) engine, an atomic engine, and a collective engine.

[0004] According to another aspect of the present application, a system is provided, comprising: a processor for unloading instructions to a coprocessor; the coprocessor comprising: a function identifier controller for handling function definition instructions, wherein a function is a sequence of instructions including at least one direct memory access operation or a collective operation; an instruction memory for storing functions; an execution engine for reading functions from the instruction memory; and a plurality of execution resources for executing the read functions, the plurality of execution resources comprising at least a direct memory access (DMA) engine, an atomic engine, and a collective engine. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Various examples according to the present disclosure will be described with reference to the accompanying drawings, in which:

[0006] Figure 1 An example of a processing element (PE) consisting of an offload coprocessor alongside a local pipeline is illustrated.

[0007] Figure 2 An example of a coprocessor is illustrated.

[0008] Figure 3 An example of the flow of operation of a function ID controller is illustrated.

[0009] Figure 4Illustrated are examples of a function ID table and the flow of opening an ID buffer given different request types and conditions within the buffer.

[0010] Figure 5 An example of a process of allocating and determining information stored in each slot of a memory partition table is illustrated.

[0011] Figure 6 The functional process leading to the determination of the instruction memory address generated by the instruction word request of the instance during the execution of the instance is shown.

[0012] Figure 7 An example of a flow diagram illustrating the operation of an example controller.

[0013] Figure 8 An example computing system is illustrated.

[0014] Fig. 9 Illustrated is a block diagram of an example processor and / or system on a chip (SoC) that may have one or more cores and with an integrated memory controller.

[0015] Fig.10 The block diagram of illustrates a computing system configured to implement one or more aspects of the examples described herein.

[0016] Fig.11A An example of a parallel processor is illustrated.

[0017] Fig. 11B An example of a block diagram of a partition unit is illustrated.

[0018] Fig. 11C An example of a block diagram of a processing cluster within a parallel processing unit is illustrated.

[0019] Fig.11D An example of a graphics multiprocessor is illustrated, where the graphics multiprocessor is coupled with a pipeline manager of a processing cluster.

[0020] Figures 12A-12C Additional graphics multiprocessors are illustrated according to examples.

[0021] Fig.13 A parallel computing system according to some examples is shown.

[0022] Figures 14A-14B Illustrated is a hybrid logical / physical view of a factorized parallel processor according to examples described herein.

[0023] FIG. 15(A) is a block diagram illustrating both an example in-order pipeline and an example register renaming, out-of-order issue / execution pipeline according to some examples.

[0024] FIG. 15(B) is a block diagram illustrating both an example in-order architecture core and an example register renaming, out-of-order issue / execution architecture core to be included in a processor, according to an example.

[0025] Fig.16 Examples of execution unit circuitry, such as execution unit circuitry, are illustrated.

[0026] Fig.17 is a block diagram of a register architecture according to some examples.

[0027] Fig.18 An example of an instruction format is illustrated.

[0028] Fig.19 An example of an addressing information field is illustrated.

[0029] Fig. 20 An example of the first prefix is ​​illustrated.

[0030] 21(A)-(D) illustrate examples of how the R, X, and B fields of the first prefix are used.

[0031] 22(A)-(B) illustrate examples of the second prefix.

[0032] Fig.23 An example of the third prefix is ​​illustrated.

[0033] Figure 24A-24B Thread execution logic including an array of processing elements employed in a graphics processor core is illustrated according to examples described herein.

[0034] Fig.25 Additional execution units are illustrated according to one example.

[0035] Fig.26 The block diagram illustrates a graphics processor instruction format according to some examples.

[0036] Fig. 27 is a block diagram of another example of a graphics processor.

[0037] Fig.28A The block diagram illustrates a graphics processor command format according to some examples.

[0038] Fig.28B The block diagram illustrates a graphics processor command sequence according to one example.

[0039] Fig.29 The block diagram illustrates the use of a software instruction converter according to an example to convert binary instructions in a source ISA to binary instructions in a target ISA.

[0040] Fig.30 The block diagram illustrates an IP core development system according to some examples, which can be used to manufacture integrated circuits to perform operations. DETAILED DESCRIPTION

[0041] The present disclosure relates to methods, apparatus, systems, and non-transitory computer-readable storage media for offloading functions.

[0042] In some previous work on graph optimization architectures, a massively multithreaded approach was adopted, where each thread executed customized graph operations (e.g., direct memory access (DMA)-like operations) separately in the pipeline, and serialization was maintained between graph operations issued in the pipeline via fence instructions. In some other work, coprocessors adjacent to the pipeline were utilized. In most cases, the offloaded instructions were executed as functions. However, the core pipeline still maintained serialization of functions by monitoring the coprocessor state before issuing any subsequent functions.

[0043] Multithreading schemes introduce additional programming costs, including load balancing considerations, inter-thread synchronization, and shared memory management. These are problems with known software solutions and generally increase runtime overhead and degrade application performance.

[0044] The example detailed herein describes a coprocessor implementation that supports software-defined serialization of offloaded functions consisting of one or more DMA and collective operations. Although the DMA within a function may be serialized, the function instance itself supports serialization (or parallel execution) based on the programmer's requirements. The offload function, the creation of offload function instances, the construction of the instance "stream", and the programmer's visibility into the "stream" state are exposed via custom ISA extensions.

[0045] The example detailed in this paper meets the ultra-sparse graph requirements by implementing custom direct memory access (DMA) operations with graph traversal behavior and near-memory target computation capabilities. These DMA operations saturate network and memory bandwidth by maintaining a large number of requests to hide the latency of long-range random pointer chasing behavior throughout the system.

[0046] DMA operations are exposed to programmers through instructions of the instruction set architecture (ISA). Most of these instructions include up to five operands. In some examples, the DMA ISA uses 64-bit instruction words.

[0047] In some examples, the coprocessor is coupled to a main processor that allows ISA extensions (e.g., a 32-bit processor such as a RISC-V core), but does not have good DMA support or collective support. In addition, in some examples, the DMA ISA cannot be implemented as an extension in the main processor without using one of various known methods, including variable instruction length encoding or multiple instruction operations (i.e., dma.copy.1 followed by dma.copy.2). Although generally supported within the specification of the main processor, these methods increase pipeline complexity and the total logic in the instruction fetch / decode / execution control will also increase, resulting in increased area / power and timing pressures. Therefore, DMA, collectives, etc. are moved to the coprocessor.

[0048] One goal when operating on sparse graph workloads is to maintain enough outstanding requests to keep the memory bandwidth saturated, and to a similar extent the network bandwidth saturated. This is achieved by using "custom" DMA operations, which may have thousands of unique memory requests within a single operation. Combinations of these complex DMAs can be used in a variety of ways, including double buffering of vertex and edge lists, parallel walks of the graph as part of a breadth-first search (BFS), or pushing / pulling information between a node and its neighbors.

[0049] In many cases, serialization is required between DMA operations to establish correct functionality. For example, an initial DMA would fetch a list of vertices from remote memory to local memory, and then any subsequent DMA operations would fetch an edge list for each vertex, and then walk their respective edge lists (or collect from each edge). If serialization was always applied between all DMAs issued from the coprocessor, this would limit performance by artificially limiting the total number of outstanding memory requests from the DMA engine, and would not saturate memory bandwidth - especially in situations where the remote access rate to other sockets on the system is high. A more performance-optimized approach would be to allow serialization between offloaded operations when needed, but to execute independent serialization sequences concurrently to achieve a much higher total number of outstanding memory requests.

[0050] Implementing DMA via an offload coprocessor allows for more complex operations covering a variety of graph analysis primitives, and can maintain high performance for sparse operations. Providing the ability to offload and execute multiple DMAs concurrently or serially allows for functional flexibility to execute more high-performance pure DMA functions that require little to no supervision overhead from the pipeline.

[0051] The examples detailed herein allow one or more of: chaining of DMA and collective operations (e.g., within a single function), reusing a function in multiple instances (each instance having a unique set of inputs, allowing the function to be executed on different data sets), serializing instances in a stream, limited host processor supervision, and so on.

[0052] Figure 1 An example of a processing element (PE) consisting of an offload coprocessor next to a local pipeline is illustrated. Coprocessor 111 has an instruction request / response interface with the execution stage 103 of local pipeline 101, and an interface with the system network for remote load / store requests. For example, instruction data (e.g., opcode, data, etc.) and instruction requests can be sent from pipeline 101 to coprocessor 111. Coprocessor 111 can perform the requested operation(s) and respond with data (RESP_DATA) and an indication that it has completed its task (INST_RESP).

[0053] The internal SRAM within the coprocessor 111 is visible within the memory map, and there is space allocated for instruction memory and temporary register space to be used during the execution of a function. The coprocessor 111 cannot directly access the contents of the register file of the pipeline 101. All addresses of instructions running on the coprocessor 111 are virtual. In addition, in some examples, all memory accesses from the coprocessor are non-cached accesses.

[0054] In some examples, the programming scheme for coprocessor 111 is conceptually broken down into two parts. The first part is creating a function for execution by coprocessor 111. The second part is the process of handling function instances, including initialization, execution, and management of instances on coprocessor 111.

[0055] In some examples, a function in coprocessor 111 is an instruction within the coprocessor 111 ISA - or a sequence of instructions, ending with a termination token. This ISA includes various DMA, collective, and remote atomic operations. Coprocessor 111 allows a user to program multiple individual functions that can be executed by coprocessor 111.

[0056] To create coprocessor 111 functions in some examples, the ISA includes a set of pipeline 101 extensions. The first instruction of pipeline 101 is "function.open" (any other name of the instruction detailed herein may be used). This instruction reserves space for functions to be registered in coprocessor 111 for all pipelines associated with the thread. This instruction takes a function identifier as input (e.g., an arbitrary 64-bit value, which may be a function address for ease of programming). Software can then use the assigned function identifier as input to other function instructions. If the identifier submitted with the instruction has already been used, the handling instruction will return an error code to the return register specified in the instruction.

[0057] The next instruction to submit to a function is "function.inst". Processing of this instruction writes the instruction to the function. This instruction takes as input a pipeline register that contains a valid coprocessor 111 instruction.

[0058] The instruction to close the function creation is "function.close". - This process indicates the end of the creation of the currently opened function.

[0059] The directive to delete a function is "function.delete". Disposition of this directive invalidates the function defined for the given function identifier.

[0060] The following points provide general rules for function construction:

[0061] Once a function is opened (when function.open is issued), all subsequent function.inst instructions will add instructions for that function only, in order. The same release pipeline must not build multiple function IDs concurrently.

[0062] Instructions issued sequentially within the same function are "chained", meaning the hardware maintains serialization. If operations are expected to be issued in parallel, they must be in different functions.

[0063] Functions are “closed” in two scenarios: issuing function.close for an ID, or issuing a new function.open — this will implicitly close the opened function.

[0064] Streams are used to queue operations (function instances) to be processed by coprocessor 111 in a first-in, first-out manner. Operations within a stream are executed to completion by coprocessor 111 in a sequential order. Operations queued in different streams may be executed independently based on hardware resource availability. Streams are represented by arbitrary values ​​provided and managed by software.

[0065] After the function is correctly constructed, the application will create an instance to execute the function code. The function instance is created by queuing several operations at the coprocessor to allocate the function instance, set its parameters, and schedule it for execution. In some examples, multiple unique instances of the same function can be created. In some examples, multiple instances can be executed concurrently in the coprocessor 111, but the software will be provided with the ability to serialize different instances by using streams. In some examples, each instance will be assigned a set of temporary registers to enter initial values ​​(and used as intermediate values).

[0066] To create a coprocessor instance, the ISA includes a set of coprocessor extensions. The allocate instruction ("instance.alloc") instruction queues an instance allocation at a given stream of the coprocessor 111. The instruction input parameters are a function ID representing the code to be executed, and a stream ID. This operation can be queued on an empty stream, or can be after an instance.exec or instance.cancel from a previous different instance.

[0067] The set parameter instruction ("instance.setarg") queues a set parameter operation at a given stream of the coprocessor. This instruction copies the value from the pipeline register to the temporary register space (32 registers mapped as r0-r31) for the current instance setup. The input operands are the target temporary register identifier (value 0 to 31) and the value to be stored (the contents of the pipeline register). This operation logically follows instance.alloc or instance.setarg.

[0068] The Execute Instance instruction ("instance.exec") queues the current instance for execution at the given stream of the coprocessor. The input operands to this ISA are the instance ID and the stream ID - that is, if the instance is part of an instance stream, it will be blocked on completion of a previously submitted instance of the same stream. This operation logically follows instance.alloc or instance.setarg.

[0069] The cancel instance instruction ("instance.cancel") cancels the instance currently queued at the given stream of the coprocessor. All operations queued for the given stream are canceled. Before the cancel operation returns, all operations already in progress for the given stream are quiesced and their resources are reclaimed. The input operation object to this ISA is the stream ID. This operation may be queued at any time, even on an empty stream, in which case it returns immediately.

[0070] Once an instance has started its execution phase in coprocessor 111, completion of the instance may be queried via the following instructions.

[0071] Instance poll instruction ("instance.poll"): For a given stream ID, the hardware will return whether the stream is empty (all queued operations have completed), non-empty (some operations are still queued), or an error has occurred. When an error occurs, the coprocessor will raise an exception, and software can query the function ID that failed and the type of operation that failed (alloc, setarg, exec). Software can then queue an instance.cancel operation to clear the stream and queue new operations.

[0072] Stall instruction ("instance.wait"): For a given stream ID, the pipeline will stall forward until all previously queued operations have completed.

[0073] The following table provides ISA parameters and descriptions for all pipeline 101 (e.g., RISC-V) instruction extensions described.

[0074]

[0075]

[0076] Figure 2 An example of a coprocessor is illustrated. In some examples, this coprocessor is coprocessor 111. Input instruction decoder 201 acts as an interface for receiving requests from a partner pipeline (e.g., pipeline 101). Input instruction decoder 201 determines the instruction type of the incoming request and forwards the request to function ID controller 203 (for function.* instructions) or instance ID controller 205 (for instance.* instructions).

[0077] The function ID management subsystem within the coprocessor 111 consists of a function ID controller 203 , a function ID table 207 (or other data structures) and an instruction memory partition table 211 .

[0078] The function ID controller 203 accepts the function.* instruction, performs the modification on the state of the target function, and stores the submitted 64-bit coprocessor instruction into the function memory 209 .

[0079] Function ID table 207 stores software assigned identifiers (IDs) and status bits for functions. The following table provides a breakdown of the fields within a single entry of function ID table 207. Since function IDs are managed by software, the table entry slots do not correspond to the IDs themselves. Instead, when a read or modify request hits function ID table 207, a search is performed on all entries in the table to identify any possible matching IDs. In some examples, the table is stored in content-addressable memory (CAM) and the search is a CAM search.

[0080]

[0081] Figure 3 An example of the flow of operation of the function ID controller is illustrated. The order of operations through the function ID controller 203 depends on the opcode received from the input instruction decoder 201. Note that in some examples, the function ID controller 203 is pipelined.

[0082] As shown, when a function.* instruction is received, it is searched in the function ID table 207 to determine whether the function ID exists, and if so, a certain action is performed, while if the function ID does not exist (or the function is not open), a different action is performed. Examples of possible actions are described. For example, for the function.inst instruction, when the ID exists, the instruction is written to the function memory 209.

[0083] The number of entries in the function ID table 207 is determined by software requirements. Although software has the ability to create / delete functions via extension of the main processor (pipeline 101), the greater the number of slots in the function ID table 207, the more concurrently active functions are allowed.

[0084] For function opening, closing and deletion, access to the function ID table 207 is not expected to be too frequent, and delay is not critical. However, the function.inst operation will be more frequent. In some examples, for these operations, a single slot open ID buffer will be placed in front of the function ID table 207. It serves as a temporary storage area for the opened function ID, and the function ID controller 203 uses it to check whether the ID being modified is currently open. Figure 4 An example of the function ID table 207 and the flow of the open ID buffer is illustrated given different request types and conditions within the buffer (ie, ID match / no match and whether the appropriate ID is open or not). The function ID controller 203 accesses the open ID buffer looking for a buffer match.

[0085] For an open operation, if there is a buffer match, and the ID is already open, nothing will change. If there is a buffer match, and the ID is not marked as open, the buffer entry will be marked as open.

[0086] For the open operation, if there is a buffer ID match and the ID is already open, there will be no changes. If there is a buffer match and the ID is not marked as open, the buffer entry will be marked as open. When there is no buffer ID match, the function ID is written to the buffer and marked as open. If there is an existing ID marked as open in the buffer, the ID in the function ID table 207 is marked as closed. If there is no existing ID marked as open in the buffer, no changes will be made to the function ID table 207.

[0087] For an "inst" operation, if there is a buffer ID match, the request is responded to with an indication that the command write can proceed. If there is no ID match, a response is sent indicating that the ID is invalid.

[0088] For a close operation, if there is a buffer ID match, the ID in the buffer is marked as closed, and so is the ID in the function ID table 207. If there is no buffer ID match, the operation ends on this instruction.

[0089] For a delete operation, if there is a buffer ID match, the ID in the buffer is marked as invalid, and so is the ID in the function ID table 207. If there is no buffer ID match, the ID in the function ID table 207 is marked as invalid.

[0090] The design goal of the coprocessor 111 is to limit the latency of instance execution to prevent the offload latency cost of DMA, collectives, and atomics from becoming a performance limiting factor. In some examples, SRAM instruction memory local to the coprocessor is used to limit instruction fetch latency during instance execution. While this removes the overhead of instruction cache implementation, it limits the total instruction memory used for coprocessor functions to the capacity of this local memory, and also places the burden on software to actively manage the memory of each coprocessor.

[0091] In some examples, in order to reduce the software burden on instruction memory management, the coprocessor 111 implements a memory partition management scheme for the instruction memory space of the function (e.g., using the function ID controller 204, the instance ID controller 203 and / or a state machine associated with the memory partition table 211). In this scheme, a partition is a set of n consecutive words in the instruction memory space. When a function is created, a partition will be assigned to the function. The memory partition table 211 is used to track the function ID associated with each partition. As instructions are added to the function, the space of the assigned partition will fill up. If the function ends before filling up the partition, the remaining space of the partition will not be used (will not be assigned to another function). If the function fills up its partition and continues to add more instructions, a new partition will be assigned, and a pointer to this new partition will be included in the partition table entry of the previous partition of the function. The new partition may not be the next consecutive address of the memory (depending on the available partitions). Once a function is deleted (e.g., using function.delete), all partitions assigned to the function ID will be released so as to be reassigned to the new function. The following table shows an example of entries of the memory partition table 211 .

[0092]

[0093]

[0094] Figure 5 An example of a process for allocating and determining the information held in each slot of the memory partition table 211 is illustrated. This illustration corresponds to receiving an instruction during function creation. Similar to the earlier process, the received function is checked for a match in the partition table 211. Example details of what to do for matches and mismatches are illustrated.

[0095] The partition table 211 also plays a key role when the instance has begun execution and instructions are being read from the instruction memory during execution. Specifically, the instance controller 4B03 sends two values ​​to the partition table 211 when making a memory request: the ID of the function that the instance is executing; and the program counter (PC) value of the instance. The partition table 211 can be used in conjunction with these values ​​to find the location of this value within the appropriate partition. Then an instruction memory 209 request can be made, and the instruction word is returned to the instance controller 205.

[0096] Figure 6The functional process leading to the determination of the instruction memory address generated by the instruction word request of the instance during the instance execution is shown. This process can be performed by one or more of the function ID controller 204, the instance ID controller 203 and / or the state machine associated with the memory partition table 211. For example, the instruction is read from the instruction memory 209 to be executed (e.g., by one or more of the DMA engine 221, the atomic engine 223 and / or the collective engine 225). The arbiter 227 is used to determine what operation should be performed on the network.

[0097] In some examples, the atomic engine 223 supports various dual memory operations commonly seen in parallel, multi-threaded, and multi-processor applications. The disclosed "dual memory" instructions use two addresses in load / store / atomic operations. These instructions are presented in the form of dual ((D)ual) memory operations, including read ((R)ead), write ((W)rite), or atomic ((X)atomic) operations, such as: DRR, DRW, DWW, DRX, DWX. These operations are "atomic" operations, that is, both memory addresses involved are updated, and no intervening operations are allowed during the dual address update.

[0098] In some examples, the collective engine (CENG) 225 includes circuits for maintaining one or more state machines to manage the execution of collective operations. In some embodiments, CENG 225 includes a hardware state machine that manages the execution of collective operations, whether in the form of barriers, reductions, broadcasts, or multicasts. In some embodiments, CENG 225 is a simplified asynchronous offload engine that can support any architecture platform and ISA. It presents a unified interface that allows broadcasts, multicasts, reductions, and barriers between user (software) defined core sets. Examples of collectives can include instructions for collection, dispersion, reduction, barriers, broadcasts, multicasts, reductions, and the like.

[0099] As shown, for a read instruction, the partition table 211 is checked for an ID match. If there is a match, the PC value is used to at least identify the target partition. If there is a target partition (and its sub-words in some examples), the target memory address is calculated and the read instruction is sent to be executed by the execution engine. If there is no partition match, an indication of an invalid PC is sent to the execution engine.

[0100] If there is no ID match in the partition table 211, an indication of an invalid PC is sent to the execution engine.

[0101] In some examples, the instruction memory 209 within the coprocessor 111 is a 1-read-1-write static random access memory (SRAM) that can accommodate 64-bit wide entries, each entry being an instruction word.

[0102] The instance management subsystem within coprocessor 111 consists of instance controller 203, flow table 213, and temporary register memory 215. Instance controller 203 accepts inst.* instructions, performs appropriate modifications to the state of the target instance, and submits the instance to execution engine 217 once the instance meets the execution conditions.

[0103] Streams are used to queue instances to be processed in a first-in, first-out manner. Operations within a stream are executed to completion by coprocessor 111 in a sequential order. Operations queued in different streams may be executed independently based on hardware resource availability. In some examples, streams are represented by values ​​provided and managed by software.

[0104] To manage serialization of instances within a flow, coprocessor 111 includes flow table 213. In some examples, each slot in the flow table holds information for a single instance. Examples of entries (slots) are shown in the table below.

[0105]

[0106] A single entry of the flow table 213 holds one or more of the following: an instance ID, a flow ID, an indication of whether the instance is the latest entry in the flow, a pointer to the next instance in the flow (if it exists), and an indication of whether the instance is ready to execute (e.g., all parameters have been set and instance.exec has been issued). If the instance is ready to execute but is blocked by the instance before it in the flow, the flow table controller sends the instance to the execution engine input queue once it is next to be issued.

[0107] Figure 7 An example of a flow chart of the operation of an instance controller is illustrated. The order of operations by the instance controller 203 depends on the opcode received from the decoder 201 .

[0108] In some examples, as part of the execution of a function instance, the coprocessor 111 enables the programmer to commit values ​​to temporary registers 215. This allows the programmer to set initial parameters for the operations (DMA, collective, and atomic) executed within the instance. This temporary register space 215 can be a local SRAM private to each coprocessor 111. The temporary register memory 215 provides data registers for each possible instance.

[0109] A particular data entry within the temporary register space 215 is indexed according to the instance's slot in the flow table 213. Once an instance is assigned to a flow table slot, the base address in the temporary register memory 215 is set according to the slot ID. The registers within the instance are then indexed consecutively in that partition of memory. In some examples, this memory can only be accessed from the pipeline via the instance.setarg instruction.

[0110] Once functions and instances of these functions are properly initialized, constructed, and set up for execution, the instances will be passed to the execution engine 217 for processing of the instances. Specifically, the execution will fetch each instruction from the local instruction memory 209, fetch the register data contents 215, and issue the complete instruction (with input fields) to the instruction sequencing manager 219. Additional responsibilities of the execution engine 217 may include tracking incomplete instances and providing information back to the flow table 213 once an instance is completed.

[0111] In some examples, an instance is passed to the execution engine if the following criteria are met: * The instance has been properly constructed on a valid function, i.e., instance.alloc, followed by various instance.setarg operations, followed by instance.exec. * The instance is not blocked by a previous instance in the flow. * The execution engine input queue is not full and can accept new instance requests.

[0112] All synchronization of instances within a stream is handled externally to the execution engine 217, and instances that do not meet the above requirements will not be provided to the execution engine 217. As instances are submitted to the execution engine 217, they are placed in a queue where they are processed by the execution engine 217. When the execution engine 217 starts operating on an instance, it will fetch and issue all instructions belonging to that instance before operating on the next instance in the queue.

[0113] In some examples, the execution engine 217 performs the following operations on each instance:

[0114] 1) Receive an instance from the input queue. The received information includes the function ID and instance ID.

[0115] 2) Send the first instruction request to the instruction memory partition table. The information sent to the table includes the function ID and a "PC" value of 0 (the base address of the first partition of the function).

[0116] 3) Send a command packet to the sequence queue.

[0117] The execution engine 217 provides services for the instructions. Since all instruction operations are read, there is no data dependency between instructions. The end of the instance will be indicated to the execution engine 217 through the return value from the instruction memory partition table 211. After receiving this indication, the execution engine 217 will wait for the sequencing queue 219 to accept all instructions, mark the instance as "issued, waiting for completion" in the flow table 212, and end the execution on this instance.

[0118] The sequencing queue 219 serializes instructions within instances that belong to the same chain. In some examples, software can define chains of instructions that force serialization. If instructions within an instance are not programmed into a chain, there will be no serialization between them, and they can be issued to the target engine(s) at the same time.

[0119] In some examples, to create a chain, the following steps are performed:

[0120] 1. The execution engine 217 sends a chain.init instruction including a chain ID to the order queue 219 (the engine 217 will also send an instance ID - to uniquely identify the chain).

[0121] 2. All instructions after chain.init are serialized in the order in which they are submitted to the sequencing queue 219.

[0122] 3. The chain will end once the execution engine 217 sends chain.end. This will not include the ID, but will close the chain currently opened by this instance.

[0123] In some examples, if the chain is not ended and a new chain.init instruction arrives in the order queue, the currently open chain is implicitly "ended". Instructions issued without a previous chain.init (i.e., not in any chain) will execute independently. The following table illustrates the slots in the order queue 219.

[0124]

[0125] Once an instruction is ready to be issued to the target engine (it is the next in the chain to be executed), the in-order queue 219 will take the following steps:

[0126] 1) Check the number of operands of the instruction. All operands in coprocessor instructions are reads, so the number of operands will determine the number of registers to be read from the temporary register store.

[0127] 2) Make (one or more) read requests.

[0128] 3) Once the data is returned, a complete command packet with all data fields is constructed.

[0129] 4) Send the command packet to the target engine (eg, DMA engine 221, collective engine 225, or atomic engine 223).

[0130] After issuing the instruction to the target engine, the queue slot will be reserved until the operation is completed. Once the operation is completed, an instruction will be sent back to the execution engine 217 to track the progress of the instance, and then the flow table 213 will be notified.

[0131] Some examples utilize the instruction formats described herein. Some examples are implemented in one or more computer architectures, cores, accelerators, etc. Some examples are generated or are IP cores. Some examples utilize simulation and / or translation.

[0132] Example Architecture

[0133] The example computer architecture is described in detail below. Other system designs and configurations for laptops, desktop computers, handheld personal computers (personal computers, PCs), personal digital assistants, engineering workstations, servers, decomposed servers, network devices, network hubs, switches, routers, embedded processors, digital signal processors (digitalsignal processors, DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular phones, portable media players, handheld devices, and various other electronic devices known in the art are also appropriate. In short, various systems or electronic devices that can include processors and / or other execution logic disclosed herein are generally appropriate.

[0134] Example System

[0135] Figure 8 An example computing system is illustrated. Multiprocessor system 800 is an interfaced system and includes multiple processors or cores, including a first processor 870 and a second processor 880 coupled via an interface 850 (e.g., a point-to-point (PP) interconnect, a fabric, and / or a bus). In some examples, the first processor 870 and the second processor 880 are isomorphic. In some examples, the first processor 870 and the second processor 880 are heterogeneous. Although example system 800 is shown as having two processors, the system may have three or more processors, or may be a single processor system. In some examples, the computing system is a system on a chip (SoC).

[0136] Processors 870 and 880 are shown as including integrated memory controller (IMC) circuits 872 and 882, respectively. Processor 870 also includes interface circuits 876 and 878; similarly, second processor 880 includes interface circuits 886 and 888. Processors 870, 880 may exchange information via interface 850 using interface circuits 878, 888. IMCs 872 and 882 couple processors 870, 880 to respective memories, namely memory 832 and memory 834, which may be part of the main memory locally attached to the respective processors.

[0137] Processors 870, 880 can each utilize interface circuits 876, 894, 886, 898 to exchange information with a network interface (NW I / F) 890 via respective interfaces 852, 854. The network interface 890 (e.g., one or more of an interconnect, bus, and / or structure, a chipset in some examples) can optionally exchange information with a coprocessor 838 via interface circuit 892. In some examples, the coprocessor 838 is a special-purpose processor, such as a high-throughput processor, a network or communication processor, a compression engine, a graphics processor, a general purpose graphics processing unit (GPGPU), a neural-network processing unit (NPU), an embedded processor, and the like.

[0138] A shared cache (not shown) may be included in either processor 870, 880, or external to both processors but connected to the processors via an interface (e.g., a PP interconnect) such that if a processor is placed in a low power mode, local cache information of either or both processors may also be stored in the shared cache.

[0139] The network interface 890 can be coupled to the first interface 816 via the interface circuit 896. In some examples, the first interface 816 can be an interface such as a Peripheral Component Interconnect (PCI) interconnect, a PCI Express interconnect, or another I / O interconnect. In some examples, the first interface 816 is coupled to a power control unit (PCU) 817, which can include circuits, software, and / or firmware to perform power management operations with respect to the processors 870, 880, and / or the coprocessor 838. The PCU 817 provides control information to a voltage regulator (not shown) so that the voltage regulator generates an appropriate regulated voltage. The PCU 817 also provides control information to control the generated operating voltage. In various examples, the PCU 817 can include various power management logic units (circuits) to perform hardware-based power management. Such power management may be entirely processor controlled (e.g., controlled by various processor hardware and may be triggered by workload and / or power constraints, thermal constraints, or other processor constraints), and / or power management may be performed in response to an external source (e.g., a platform or power management source or system software).

[0140] PCU 817 is illustrated as existing as logic separate from processor 870 and / or processor 880. In other cases, PCU 817 may execute on a given one or more cores in the core (not shown) of processor 870 or 880. In some cases, PCU 817 may be implemented as a microcontroller (dedicated or general purpose) or other control logic that is configured to execute its own dedicated power management code (sometimes referred to as P-code). In yet other examples, the power management operations to be performed by PCU 817 may be implemented outside the processor, such as by a separate power management integrated circuit (PMIC) or another component external to the processor. In yet other examples, the power management operations to be performed by PCU 817 may be implemented within BIOS or other system software.

[0141] Various I / O devices 814 and a bus bridge 818 can be coupled to the first interface 816, which couples the first interface 816 to the second interface 820. In some examples, (one or more) additional processors 815 are coupled to the first interface 816, such as a coprocessor, a high-throughput integrated many-core (MIC) processor, a GPGPU, an accelerator (such as a graphics accelerator or a digital signal processing (DSP) unit), a field programmable gate array (FPGA), or any other processor. In some examples, the second interface 820 can be a low pin count (LPC) interface. Various devices can be coupled to the second interface 820, including, for example, a keyboard and / or mouse 822, a communication device 827, and a storage circuit 828. The storage circuit 828 can be one or more non-transitory machine-readable storage media as described below, such as a disk drive or other mass storage device, which in some examples may include instructions / codes and data 830 and can implement the storage device `ISAB03. In addition, audio I / O 824 can be coupled to second interface 820. Note that other architectures are possible in addition to the point-to-point architecture described above. For example, a system such as multiprocessor system 800 can implement a multi-drop interface or other such architectures instead of a point-to-point architecture.

[0142] Example core architectures, processors, and computer architectures.

[0143] Processor cores can be implemented in different ways, for different purposes, and in different processors. For example, implementations of these cores may include: 1) general-purpose in-order cores for general-purpose computing purposes; 2) high-performance general-purpose out-of-order cores for general-purpose computing purposes; 3) special-purpose cores, mainly for graphics and / or scientific (throughput) computing purposes. Implementations of different processors may include: 1) a CPU including one or more general-purpose in-order cores for general-purpose computing purposes and / or one or more general-purpose out-of-order cores for general-purpose computing purposes; and 2) a coprocessor including one or more special-purpose cores mainly for graphics and / or scientific (throughput) computing purposes. These different processors result in different computer system architectures, which may include: 1) coprocessors on separate chips from the CPU; 2) coprocessors on separate dies in the same package as the CPU; 3) coprocessors on the same die as the CPU (in which case such coprocessors are sometimes referred to as dedicated logic, such as integrated graphics and / or scientific (throughput) logic, or as dedicated cores); and 4) systems on a chip (SoC), which may be included on the same die as the described CPU (sometimes referred to as application core(s) or application processor(s), the coprocessors described above, and additional functionality. An example core architecture is described next, followed by a description of an example processor and computer architecture.

[0144] Fig. 9 A block diagram of an example processor and / or SoC 900 is illustrated, which may have one or more cores and an integrated memory controller. The processor 900 illustrated in solid line blocks has a single core 902(A), a system agent unit circuit 910, and a set of one or more interface controller unit circuits 916, while the optionally added dashed line blocks illustrate an alternative processor 900 as having multiple cores 902(A)-(N), a set of one or more integrated memory control unit circuits 914 in the system agent unit circuit 910, dedicated logic 908, and a set of one or more interface controller unit circuits 916. Note that the processor 900 may be Figure 8 One of the processors 870 or 880 or the coprocessors 838 or 815.

[0145] Thus, different implementations of processor 900 may include: 1) a CPU, where the dedicated logic 908 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores, not shown), and cores 902(A)-(N) are one or more general purpose cores (e.g., general purpose in-order cores, general purpose out-of-order cores, or a combination of the two); 2) a coprocessor, where cores 902(A)-(N) are a large number of dedicated cores primarily for graphics and / or scientific (throughput) purposes; and 3) a coprocessor, where cores 902(A)-(N) are a large number of general purpose in-order cores. Thus, processor 900 may be a general purpose processor, a coprocessor, or a dedicated processor, such as a network or communication processor, a compression engine, a graphics processor, a GPGPU (general purpose graphics processing unit), a high throughput many integrated core (MIC) coprocessor (including 30 or more cores), an embedded processor, etc. The processor may be implemented on one or more chips. Processor 900 may be part of one or more substrates and / or may be implemented on one or more substrates using any of a variety of process technologies, such as complementary metal oxide semiconductor (CMOS), bipolar CMOS (BiCMOS), P-type metal oxide semiconductor (PMOS), or N-type metal oxide semiconductor (NMOS).

[0146] The memory hierarchy includes one or more levels of cache unit circuits 904(A)-(N) within the cores 902(A)-(N), a set of one or more shared cache unit circuits 906, and external memory (not shown) coupled to the set of integrated memory controller unit circuits 914. The set of one or more shared cache unit circuits 906 may include one or more intermediate level caches, such as level 2 (L2), level 3 (L3), level 4 (L4), or other levels of cache, such as last level cache (LLC), and / or combinations thereof. While in some examples an interface network circuit 912 (e.g., a ring interconnect) provides an interface to dedicated logic 908 (e.g., integrated graphics logic), the set of shared cache unit circuits 906, and the system agent unit circuit 910, alternative examples use any number of well-known techniques to provide interfaces to these units. In some examples, coherency is maintained between one or more circuits in the shared cache unit circuits 906 and the cores 902(A)-(N). In some examples, interface controller unit circuitry 916 couples the cores 902 to one or more other devices 918, such as one or more I / O devices, storage, one or more communication devices (e.g., wireless network, wired network, etc.), and the like.

[0147] In some examples, one or more of the cores 902(A)-(N) have multi-threading capabilities. The system agent unit circuit 910 includes those components that coordinate and operate the cores 902(A)-(N). The system agent unit circuit 910 may include, for example, a power control unit (PCU) circuit and / or a display unit circuit (not shown). The PCU may be (or may include) the logic and components required to adjust the power state of the cores 902(A)-(N) and / or the dedicated logic 908 (e.g., integrated graphics logic). The display unit circuit is used to drive one or more externally connected displays.

[0148] The cores 902(A)-(N) may be homogeneous with respect to instruction set architecture (ISA). Alternatively, the cores 902(A)-(N) may be heterogeneous with respect to ISA; that is, a subset of the cores 902(A)-(N) may be capable of executing one ISA, while other cores may be capable of executing only a subset of the ISA or may be capable of executing another ISA.

[0149] Fig.10The block diagram of 1000 illustrates a computing system 1000 configured to implement one or more aspects of the examples described herein. The computing system 1000 includes a processing subsystem 1001 having one or more processors 1002, and a system memory 1004, which communicate via an interconnect path, which may include a memory hub 1005. The memory hub 1005 may be a separate component within a chipset component, or may be integrated within the one or more processors 1002. The memory hub 1005 is coupled to an I / O subsystem 1011 via a communication link 1006. The I / O subsystem 1011 includes an I / O hub 1007, which may enable the computing system 1000 to receive input from one or more input devices 1008. In addition, the I / O hub 1007 may enable a display controller included in the one or more processors 1002 to provide output to one or more display devices 1010A. In some examples, one or more display devices 1010A coupled to I / O hub 1007 may include local, internal, or embedded display devices.

[0150] For example, the processing subsystem 1001 includes one or more parallel processors 1012, which are coupled to the memory hub 1005 via a bus or other communication link 1013. The communication link 1013 can be one of any number of standard-based communication link technologies or protocols, such as but not limited to PCI Express, or can be a vendor-specific communication interface or communication structure. One or more parallel processors 1012 can form a computationally concentrated parallel or vector processing system that can include a large number of processing cores and / or processing clusters, such as an integrated many-core (MIC) processor. For example, one or more parallel processors 1012 form a graphics processing subsystem that can output pixels to one of one or more display devices 1010A coupled via an I / O hub 1007. One or more parallel processors 1012 may also include a display controller and a display interface (not shown) to enable direct connection to one or more display devices 1010B.

[0151] Within the I / O subsystem 1011, a system storage unit 1014 may be connected to the I / O hub 1007 to provide a storage mechanism for the computing system 1000. An I / O switch 1016 may be used to provide an interface mechanism to enable connections between the I / O hub 1007 and other components (e.g., a network adapter 1018 and / or a wireless network adapter 1019 that may be integrated into the platform) and various other devices that may be added via one or more additional devices 1020. (One or more) additional devices 1020 may also include, for example, one or more external graphics processor devices, graphics cards, and / or computing accelerators. The network adapter 1018 may be an Ethernet adapter or another wired network adapter. The wireless network adapter 1019 may include one or more of Wi-Fi, Bluetooth, near field communication (NFC), or other network devices including one or more wireless radios.

[0152] Computing system 1000 may include other components not explicitly shown, including USB or other port connections, optical storage drives, video capture devices, etc., which may also be connected to I / O hub 1007 . Fig.10 The communication paths interconnecting the various components in the system may be implemented using any suitable protocol, such as a PCI (Peripheral Component Interconnect) based protocol (e.g., PCI Express), or any (one or more) other bus or point-to-point communication interface and / or protocol, such as NVLink high-speed interconnect, Compute Express Link TM (CXL TM) (e.g., CXL.mem), Infinity Fabric (IF), Ethernet (IEEE 802.3), remote direct memory access (RDMA), InfiniBand, Internet Wide Area RDMA Protocol (iWARP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), quick UDP Internet Connections (QUIC), RDMA over Converged Ethernet (RoCE), Intel QuickPath Interconnect (QPI), Intel Ultra Path Interconnect (UPI), Intel On-Chip System Fabric (IOSF), Omnipath, HyperTransport, Advanced Microcontroller Bus Architecture (AMBA) Interconnect, OpenCAPI, Gen-Z, Cache Coherent Interconnect for Accelerators (Cache Coherent Interconnect) For Accelerators, CCIX), 3GPP Long Term Evolution (LTE) (4G), 3GPP 5G, and variants thereof, or wired or wireless interconnect protocols known in the art. In some examples, data can be copied or stored to a virtualized storage node using a protocol such as non-volatile memory express (NVMe) over Fabrics (NVMe-oF) or NVMe.

[0153] One or more parallel processors 1012 may include circuits optimized for graphics and video processing, such as video output circuits, and constitute a graphics processing unit (GPU). Alternatively or additionally, one or more parallel processors 1012 may include circuits optimized for general-purpose processing while retaining the underlying computing architecture described in more detail herein. The components of the computing system 1000 may be integrated with one or more other system elements on a single integrated circuit. For example, one or more parallel processors 1012, a memory hub 1005, (one or more) processors 1002, and an I / O hub 1007 may be integrated into a system-on-chip (SoC) integrated circuit. Alternatively, the components of the computing system 1000 may be integrated into a single package to form a system-in-package (SIP) configuration. In some examples, at least a portion of the components of the computing system 1000 may be integrated into a multi-chip module (MCM), which may be interconnected with other multi-chip modules to form a modular computing system.

[0154] It will be appreciated that the computing system 1000 shown herein is illustrative only and may be varied and modified. The connection topology, including the number and arrangement of bridges, the number of (one or more) processors 1002, and the number of (one or more) parallel processors 1012, may be modified as desired. For example, the system memory 1004 may be directly connected to the (one or more) processors 1002, rather than through a bridge, while other devices communicate with the system memory 1004 via the memory hub 1005 and the (one or more) processors 1002. In other alternative topologies, the (one or more) parallel processors 1012 are connected to the I / O hub 1007 or directly to one of the one or more processors 1002, rather than to the memory hub 1005. In other examples, the I / O hub 1007 and the memory hub 1005 may be integrated into a single chip. Two or more groups of processors 1002 may also be attached via multiple sockets, which may be coupled to two or more instances of (one or more) parallel processors 1012.

[0155] Some specific components shown herein are optional and may not be included in all implementations of computing system 1000. For example, any number of add-in cards or peripherals may be supported, or some components may be eliminated. In addition, some architectures may have different Fig.10 Different terminology is used for similar components shown in FIG. 1004. For example, in some architectures, memory hub 1005 may be referred to as a north bridge, while I / O hub 1007 may be referred to as a south bridge.

[0156] Fig.11A An example of a parallel processor 1100 is shown. The parallel processor 1100 may be a GPU, a GPGPU, or the like as described herein. The various components of the parallel processor 1100 may be implemented using one or more integrated circuit devices, such as a programmable processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The illustrated parallel processor 1100 may be Fig.10 One or more of the parallel processor(s) 1012 shown in .

[0157] The parallel processor 1100 includes a parallel processing unit 1102. The parallel processing unit includes an I / O unit 1104 that enables communication with other devices (including other instances of the parallel processing unit 1102). The I / O unit 1104 can be directly connected to other devices. For example, the I / O unit 1104 is connected to other devices by using a hub or switch interface (e.g., a memory hub 1005). The connection between the memory hub 1005 and the I / O unit 1104 forms a communication link 1013. Within the parallel processing unit 1102, the I / O unit 1104 is connected to a host interface 1106 and a memory crossbar 1116, wherein the host interface 1106 receives commands for performing processing operations, and the memory crossbar 1116 receives commands for performing memory operations.

[0158] When the host interface 1106 receives the command buffer via the I / O unit 1104, the host interface 1106 can direct the work operations of executing these commands to the front end 1108. In some examples, the front end 1108 is coupled with a scheduler 1110, which is configured to distribute commands or other work items to the processing cluster array 1112. Before the task is distributed to the processing clusters of the processing cluster array 1112, the scheduler 1110 ensures that the processing cluster array 1112 is correctly configured and in a valid state. The scheduler 1110 can be implemented via firmware logic executed on a microcontroller. The microcontroller-implemented scheduler 1110 can be configured to perform complex scheduling and work distribution operations at coarse and fine granularity, thereby achieving fast preemption and context switching of threads executed on the processing cluster array 1112. Preferably, the host software can prove the workload via one of a plurality of graphics processing doorbells so as to be scheduled on the processing cluster array 1112. In other examples, polling for new workloads or interrupts can be used to identify or indicate the availability of work to be executed. The scheduler 1110 logic within the scheduler microcontroller may then automatically distribute the workload across the array of processing clusters 1112 .

[0159] The processing cluster array 1112 may include up to "N" processing clusters (e.g., cluster 1114A, cluster 1114B to cluster 1114N). Each cluster 1114A-1114N of the processing cluster array 1112 may execute a large number of concurrent threads. The scheduler 1110 may use various scheduling and / or work distribution algorithms to assign work to the clusters 1114A-1114N of the processing cluster array 1112, which may vary depending on the workload that occurs for each type of program or calculation. Scheduling may be handled dynamically by the scheduler 1110, or may be partially assisted by compiler logic during the compilation of program logic that is configured for execution by the processing cluster array 1112. Optionally, different clusters 1114A-1114N of the processing cluster array 1112 may be assigned to process different types of programs or perform different types of calculations.

[0160] Processing cluster array 1112 may be configured to perform various types of parallel processing operations. For example, processing cluster array 1112 is configured to perform general-purpose parallel computing operations. For example, processing cluster array 1112 may include logic to perform processing tasks including filtering video and / or audio data, performing modeling operations (including physics operations), and performing data transformations.

[0161] The processing cluster array 1112 is configured to perform parallel graphics processing operations. In such examples where the parallel processor 1100 is configured to perform graphics processing operations, the processing cluster array 1112 may include additional logic to support the execution of such graphics processing operations, including but not limited to texture sampling logic for performing texture operations, and tessellation logic and other vertex processing logic. In addition, the processing cluster array 1112 may be configured to execute shader programs related to graphics processing, such as but not limited to vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. The parallel processing unit 1102 may transfer data from the system memory via the I / O unit 1104 for processing. During processing, the transferred data may be stored in an on-chip memory (e.g., parallel processor memory 1122) during processing and then written back to the system memory.

[0162] In examples where parallel processing units 1102 are used to perform graphics processing, scheduler 1110 may be configured to divide the processing workload into tasks of approximately the same size so that graphics processing operations can be better distributed to multiple clusters 1114A-1114N of processing cluster array 1112. In some of these examples, portions of processing cluster array 1112 may be configured to perform different types of processing. For example, a first portion may be configured to perform vertex shading and topology generation, a second portion may be configured to perform tessellation and geometry shading, and a third portion may be configured to perform pixel shading or other screen space operations to produce a rendered image for display. Intermediate data generated by one or more of clusters 1114A-1114N may be stored in a buffer to allow the intermediate data to be transferred between clusters 1114A-1114N for further processing.

[0163] During operation, the processing cluster array 1112 may receive processing tasks to be performed via the scheduler 1110, which receives commands defining the processing tasks from the front end 1108. For graphics processing operations, the processing tasks may include an index of data to be processed, such as surface (patch) data, primitive data, vertex data, and / or pixel data, and state parameters and commands defining how the data is to be processed (e.g., what program to execute). The scheduler 1110 may be configured to retrieve an index corresponding to a task, or may receive an index from the front end 1108. The front end 1108 may be configured to ensure that the processing cluster array 1112 is configured to a valid state before a workload specified by an incoming command buffer (e.g., a batch buffer, a push buffer, etc.) is initiated.

[0164] Each of the one or more instances of the parallel processing unit 1102 may be coupled to a parallel processor memory 1122. The parallel processor memory 1122 may be accessed via a memory crossbar 1116, which may receive memory requests from the processing cluster array 1112 and the I / O unit 1104. The memory crossbar 1116 may access the parallel processor memory 1122 via a memory interface 1118. The memory interface 1118 may include a plurality of partition units (e.g., partition unit 1120A, partition unit 1120B, up to partition unit 1120N), each of which may be coupled to a portion of the parallel processor memory 1122 (e.g., a memory cell). The number of partition units 1120A-1120N may be configured to be equal to the number of memory cells, such that the first partition unit 1120A has a corresponding first memory cell 1124A, the second partition unit 1120B has a corresponding second memory cell 1124B, and the Nth partition unit 1120N has a corresponding Nth memory cell 1124N. In other examples, the number of partition units 1120A-1120N may not be equal to the number of memory devices.

[0165] The memory units 1124A-1124N may include various types of memory devices, including dynamic random-access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory. Optionally, the memory units 1124A-1124N may also include 3D stacked memory, including but not limited to high bandwidth memory (HBM). Those skilled in the art will appreciate that the specific implementation of the memory units 1124A-1124N may be varied and may be selected from one of various conventional designs. Render targets (such as frame buffers or texture maps) may be stored across the memory units 1124A-1124N, thereby allowing the partition units 1120A-1120N to write portions of each render target in parallel to efficiently use the available bandwidth of the parallel processor memory 1122. In some examples, a local instance of the parallel processor memory 1122 may not be included, and a unified memory design utilizing system memory in combination with local cache memory may be employed.

[0166] Optionally, any of the clusters 1114A-1114N of the processing cluster array 1112 has the ability to process data to be written to any memory unit 1124A-1124N within the parallel processor memory 1122. The memory crossbar 1116 can be configured to transmit the output of each cluster 1114A-1114N to any partition unit 1120A-1120N or another cluster 1114A-1114N, which can perform additional processing operations on the output. Each cluster 1114A-1114N can communicate with a memory interface 1118 via the memory crossbar 1116 to read from or write to various external memory devices. In one example having a memory crossbar 1116, the memory crossbar 1116 has connections to a memory interface 1118 for communicating with the I / O unit 1104, and connections to a local instance of a parallel processor memory 1122, thereby enabling processing units within different processing clusters 1114A-1114N to communicate with system memory or other memory that is not local to the parallel processing unit 1102. In general, the memory crossbar 1116 may be capable of separating traffic flows between the clusters 1114A-1114N and the partition units 1120A-1120N, for example, using virtual channels.

[0167] Although a single instance of parallel processing unit 1102 is illustrated within parallel processor 1100, any number of instances of parallel processing unit 1102 may be included. For example, multiple instances of parallel processing unit 1102 may be provided on a single add-in card, or multiple add-in cards may be interconnected. For example, parallel processor 1100 may be an add-in device, such as Fig.10 Additional device 1020, which can be a graphics card, for example, a discrete graphics card including one or more GPUs, one or more memory devices, and a device-to-device or network or structure interface. Different instances of parallel processing unit 1102 can be configured to interoperate, even if different instances have different numbers of processing cores, different numbers of local parallel processor memories, and / or other configuration differences. Optionally, some instances of parallel processing unit 1102 may include floating point units with higher precision relative to other instances. Systems including one or more instances of parallel processing unit 1102 or parallel processor 1100 can be implemented in various configurations and form factors, including but not limited to desktop, laptop or handheld personal computers, servers, workstations, game consoles, and / or embedded systems. The coordinator can utilize one or more of the following to form a composite node for workload execution: decomposed processor resources, cache resources, memory resources, storage resources, and networking resources.

[0168] In some examples, the parallel processing unit 1102 can be partitioned into multiple instances. These multiple instances can be configured to execute workloads associated with different clients in an isolated manner, so that a predetermined quality of service can be provided for each client. For example, each cluster 1114A-1114N can be separated and isolated from other clusters, thereby allowing the processing cluster array 1112 to be divided into multiple computing partitions or instances. In this configuration, the workload executed on the isolated partition can be protected from the impact of failures or errors associated with different workloads executed on different partitions. Partition units 1120A-1120N can be configured to implement dedicated and / or isolated paths to memory for clusters 1114A-1114N associated with each computing partition. This data path isolation enables the computing resources within the partition to communicate with one or more assigned memory units 1124A-1124N without being disturbed by the activities of other partitions.

[0169] Fig. 11B is a block diagram of the partition unit 1120. The partition unit 1120 may be Fig.11A 1120N. As shown, partition unit 1120 includes L2 cache 1121, frame buffer interface 1125, and ROP 1126 (raster operation unit). L2 cache 1121 is a read / write cache that is configured to perform load and store operations received from memory crossbar 1116 and ROP 1126. Read misses and emergency writeback requests are output by L2 cache 1121 to frame buffer interface 1125 for processing. Updates can also be sent to the frame buffer for processing via frame buffer interface 1125. In some examples, frame buffer interface 1125 interfaces with one of the storage units in the parallel processor memory, such as Fig.11A Partition unit 1120 may additionally or alternatively interface with one of the memory units in the parallel processor memory via a memory controller (not shown).

[0170] In graphics applications, ROP 1126 is a processing unit that performs raster operations (e.g., stencil, z-test, blending, etc.). ROP 1126 then outputs processed graphics data, which is stored in graphics memory. In some examples, ROP 1126 includes or is coupled to a codec 1127 that includes compression logic to compress depth or color data written to memory or L2 cache 1121 and decompress depth or color data read from memory or L2 cache 1121. The compression logic can be lossless compression logic that utilizes one or more of a variety of compression algorithms. The type of compression performed by codec 1127 can vary based on the statistical characteristics of the data to be compressed. For example, in some examples, incremental color compression is performed on depth and color data on a per-tile basis. In some examples, codec 1127 includes compression and decompression logic that can compress and decompress computational data associated with machine learning operations. For example, codec 1127 can compress sparse matrix data for sparse machine learning operations. The codec 1127 may also compress sparse matrix data encoded in a sparse matrix format (e.g., coordinate list encoding (COO), compressed sparse row (CSR), compressed sparse column (CSC), etc.) to generate compressed and encoded sparse matrix data. The compressed and encoded sparse matrix data may be decompressed and / or decoded before being processed by a processing element, or the processing element may be configured to consume compressed, encoded, or compressed and encoded data for processing.

[0171] ROP 1126 may be included in each processing cluster (e.g., Fig.11A 1114A-1114N) rather than within partition unit 1120. In this example, read and write requests for pixel data rather than pixel fragment data are transmitted through memory crossbar 1116. The processed graphics data can be displayed on a display device, such as Fig.10 One of the one or more display devices 1010A-1010B is routed to be further processed by (one or more) processors 1002, or is routed to be further processed by (one or more) processors 1002. Fig.11A One of the processing entities within parallel processor 1100 is further processed.

[0172] Fig. 11C is a block diagram of a processing cluster 1114 within a parallel processing unit. For example, a processing cluster is Fig.11AAn instance of one of the processing clusters 1114A-1114N. Processing cluster 1114 can be configured to execute many threads in parallel, wherein the term "thread" refers to an instance of a specific program executed on a specific input data set. Optionally, a single instruction, multiple data (SIMD) instruction issuance technology can be used to support the parallel execution of a large number of threads without providing multiple independent instruction units. Alternatively, a single instruction, multiple threads (single-instruction, multiple-thread, SIMT) technology can be used to support the parallel execution of a large number of generally synchronized threads, wherein a common instruction unit is used, which is configured to issue instructions to a group of processing engines in each processing cluster. Unlike the SIMD execution mechanism (where all processing engines usually execute the same instructions), SIMT execution allows different threads to more easily follow the divergent execution path through a given thread program. Those skilled in the art will understand that the SIMD processing mechanism represents a functional subset of the SIMT processing mechanism.

[0173] The operation of the processing cluster 1114 may be controlled via a pipeline manager 1132, which distributes processing tasks to SIMT parallel processors. Fig.11A The scheduler 1110 receives instructions and manages the execution of these instructions via the graphics multiprocessor 1134 and / or the texture unit 1136. The illustrated graphics multiprocessor 1134 is an exemplary instance of a SIMT parallel processor. However, various types of SIMT parallel processors of different architectures may be included in the processing cluster 1114. One or more instances of the graphics multiprocessor 1134 may be included in the processing cluster 1114. The graphics multiprocessor 1134 may process data, and the data cross bar 1140 may be used to distribute the processed data to one of a plurality of possible destinations, including other shader units. The pipeline manager 1132 may facilitate the distribution of processed data by specifying a destination for the processed data to be distributed via the data cross bar 1140.

[0174] Each graphics multiprocessor 1134 within a processing cluster 1114 may include a set of identical function execution logic (e.g., arithmetic logic units, load-store units, etc.). The function execution logic may be configured in a pipelined manner, where new instructions may be issued before previous instructions are completed. The function execution logic supports a variety of operations, including integer and floating point arithmetic, comparison operations, Boolean operations, shifts, and calculation of various algebraic functions. The same functional unit hardware may be utilized to perform different operations, and any combination of functional units may be present.

[0175] The instructions transmitted to the processing cluster 1114 constitute threads. A group of threads executed on a group of parallel processing engines is a thread group. A thread group executes the same program on different input data. Each thread in a thread group can be assigned to a different processing engine in the graphics multiprocessor 1134. The number of threads included in the thread group may be less than the number of processing engines in the graphics multiprocessor 1134. When the number of threads included in the thread group is less than the number of processing engines, one or more of the processing engines may be idle during the cycle in which the thread group is processed. The number of threads included in the thread group may also be more than the number of processing engines in the graphics multiprocessor 1134. When the number of threads included in the thread group is more than the number of processing engines in the graphics multiprocessor 1134, processing can be performed in consecutive clock cycles. Optionally, multiple thread groups can be executed simultaneously on the graphics multiprocessor 1134.

[0176] The graphics multiprocessors 1134 may include internal cache memory to perform load and store operations. Alternatively, the graphics multiprocessors 1134 may forgo internal cache and use cache memory (e.g., level 1 (L1) cache 1148) within the processing cluster 1114. Each graphics multiprocessor 1134 may also have access to a partition unit (e.g., Fig.11A 1120N) that are shared between all processing clusters 1114 and can be used to transfer data between threads. Graphics multiprocessor 1134 can also access off-chip global memory, which can include one or more of local parallel processor memory and / or system memory. Any memory external to parallel processing unit 1102 can be used as global memory. Embodiments in which processing cluster 1114 includes multiple instances of graphics multiprocessor 1134 can share common instructions and data, which can be stored in L1 cache 1148.

[0177] Each processing cluster 1114 may include an MMU 1145 (memory management unit) that is configured to map virtual addresses to physical addresses. In other examples, one or more instances of MMU 1145 may exist in Fig.11A1148 of the memory interface 1118. The MMU 1145 includes a set of page table entries (PTEs) for mapping virtual addresses to physical addresses of tiles, and optionally also includes a cache line index. The MMU 1145 may include an address translation lookaside buffer (TLB) or cache, which may be present in the L1 cache 1148 of the graphics multiprocessor 1134 or processing cluster 1114. The physical address is processed to distribute the surface data access location to allow efficient request interleaving between partition units. The cache line index can be used to determine whether a request for a cache line is a hit or a miss.

[0178] In graphics and compute applications, the processing clusters 1114 may be configured such that each graphics multiprocessor 1134 is coupled to a texture unit 1136 that is used to perform texture mapping operations, such as determining texture sample locations, reading texture data, and filtering texture data. Texture data is read from an internal texture L1 cache (not shown), or in some examples from an L1 cache within the graphics multiprocessor 1134, and is retrieved from an L2 cache, local parallel processor memory, or system memory as needed. Each graphics multiprocessor 1134 outputs processed tasks to a data crossbar 1140 to provide the processed tasks to another processing cluster 1114 for further processing, or stores the processed tasks in an L2 cache, local parallel processor memory, or system memory via a memory crossbar 1116. A preROP 1142 (pre-raster operation unit) is configured to receive data from the graphics multiprocessor 1134, direct the data to a ROP unit, which may communicate with a partition unit (e.g., Fig.11A The preROP 1142 unit may perform color blending optimization, organize pixel color data, and perform address translation.

[0179] It will be appreciated that the core architecture described herein is illustrative only and is subject to variation and modification. Any number of processing units may be included within a processing cluster 1114, e.g., a graphics multiprocessor 1134, a texture unit 1136, a preROP 1142, and the like. Additionally, while only one processing cluster 1114 is shown, a parallel processing unit as described herein may include any number of instances of a processing cluster 1114. Optionally, each processing cluster 1114 may be configured to operate independently of other processing clusters 1114 using separate and distinct processing units, L1 caches, L2 caches, and the like.

[0180] Fig.11DAn example of a graphics multiprocessor 1134 is shown, where the graphics multiprocessor 1134 is coupled to a pipeline manager 1132 of a processing cluster 1114. The graphics multiprocessor 1134 has an execution pipeline that includes, but is not limited to, an instruction cache 1152, an instruction unit 1154, an address mapping unit 1156, a register file 1158, one or more general purpose graphics processing unit (GPGPU) cores 1162, and one or more load / store units 1166. The GPGPU cores 1162 and the load / store units 1166 are coupled to a cache memory 1172 and a shared memory 1170 via a memory and cache interconnect 1168. The graphics multiprocessor 1134 may also include a tensor and / or ray tracing core 1163 that includes hardware logic for accelerating matrix and / or ray tracing operations.

[0181] The instruction cache 1152 may receive a stream of instructions to be executed from the pipeline manager 1132. These instructions are cached in the instruction cache 1152 and dispatched by the instruction unit 1154 for execution. The instruction unit 1154 may dispatch instructions as a thread group (e.g., a warp), each thread of the thread group being assigned to a different execution unit within the GPGPU core 1162. Instructions may access any of the local, shared, or global address spaces by specifying an address within the unified address space. The address mapping unit 1156 may be used to translate addresses in the unified address space into different memory addresses accessible by the load / store unit 1166.

[0182] The register file 1158 provides a set of registers for the functional units of the graphics multiprocessor 1134. The register file 1158 provides temporary storage for operands of data paths connected to the functional units of the graphics multiprocessor 1134 (e.g., GPGPU core 1162, load / store unit 1166). The register file 1158 may be divided between each functional unit so that each functional unit may be allocated a dedicated portion of the register file 1158. For example, the register file 1158 may be divided between different warps executed by the graphics multiprocessor 1134.

[0183] The GPGPU cores 1162 may each include a floating point unit (FPU) and / or an integer arithmetic logic unit (ALU) that is used to execute instructions of the graphics multiprocessor 1134. In some implementations, the GPGPU cores 1162 may include hardware logic that may otherwise exist within the tensor and / or ray tracing cores 1163. The GPGPU cores 1162 may be similar in architecture or may be different in architecture. For example, in some examples, a first portion of the GPGPU core 1162 includes a single-precision FPU and an integer ALU, while a second portion of the GPGPU core includes a double-precision FPU. Optionally, the FPU may implement the IEEE 754-2008 standard for floating-point arithmetic or enable variable-precision floating-point arithmetic. The graphics multiprocessor 1134 may also include one or more fixed-function or special-function units to perform specific functions, such as copying rectangles or pixel blending operations. One or more GPGPU cores may also include fixed or special-function logic.

[0184] GPGPU core 1162 may include SIMD logic capable of executing a single instruction to multiple sets of data. Optionally, GPGPU core 1162 may physically execute SIMD4, SIMD8, and SIMD16 instructions, and logically execute SIMD1, SIMD2, and SIMD32 instructions. The SIMD instructions of the GPGPU core may be generated by a shader compiler at compile time, or automatically generated when executing a program written and compiled for a single program multiple data (SPMD) or SIMT architecture. Multiple threads of a program configured for a SIMT execution model may be executed via a single SIMD instruction. For example, in some examples, eight SIMT threads performing the same or similar operation may be executed in parallel via a single SIMD8 logic unit.

[0185] The memory and cache interconnect 1168 is an interconnect network that connects each functional unit of the graphics multiprocessor 1134 to the register file 1158 and the shared memory 1170. For example, the memory and cache interconnect 1168 is a crossbar interconnect that allows the load / store unit 1166 to implement load and store operations between the shared memory 1170 and the register file 1158. The register file 1158 can operate at the same frequency as the GPGPU core 1162, so that the data transfer latency between the GPGPU core 1162 and the register file 1158 is very low. The shared memory 1170 can be used to implement communication between threads executed on the functional units within the graphics multiprocessor 1134. The cache memory 1172 can be used, for example, as a data cache to cache texture data communicated between the functional units and the texture unit 1136. The shared memory 1170 can also be used as a program management cache. The shared memory 1170 and the cache memory 1172 can be coupled to the data crossbar 1140 to enable communication with other components of the processing cluster. In addition to automatically cached data stored in cache memory 1172, threads executing on GPGPU core 1162 can also programmatically store data in shared memory.

[0186] Figures 12A-12C Additional graphics multiprocessors are illustrated according to examples. Figures 12A-12B Graphics multiprocessors 1225, 1250 are shown, which are connected to Fig. 11C The graphics multiprocessor 1134 of FIG. 1134 is related to and can be used to replace one of them. Therefore, any feature disclosed herein in conjunction with the graphics multiprocessor 1134 also discloses the corresponding combination with (one or more) graphics multiprocessors 1225, 1250, but is not limited to this. Fig. 12C A graphics processing unit (GPU) 1280 is illustrated that includes a collection of specialized graphics processing resources arranged into multi-core groups 1265A-1265N that correspond to graphics multiprocessors 1225, 1250. The illustrated graphics multiprocessors 1225, 1250 and multi-core groups 1265A-1265N may be streaming multiprocessors (SMs) that are capable of executing a large number of execution threads simultaneously.

[0187] Relative to Fig.11D Graphics multiprocessor 1134, Fig. 12AThe graphics multiprocessor 1225 includes multiple additional instances of execution resource units. For example, the graphics multiprocessor 1225 may include multiple instances of instruction units 1232A-1232B, register files 1234A-1234B, and (one or more) texture units 1244A-1244B. The graphics multiprocessor 1225 also includes multiple groups of graphics or compute execution units (e.g., GPGPU cores 1236A-1236B, tensor cores 1237A-1237B, ray tracing cores 1238A-1238B) and multiple groups of load / store units 1240A-1240B. The execution resource units have a shared instruction cache 1230, texture and / or data cache memory 1242, and a shared memory 1246.

[0188] The various components may communicate via the interconnect structure 1227. The interconnect structure 1227 may include one or more crossbar switches to enable communication between the various components of the graphics multiprocessor 1225. The interconnect structure 1227 may be a separate high-speed network fabric layer on which each component of the graphics multiprocessor 1225 is stacked. The components of the graphics multiprocessor 1225 communicate with remote components via the interconnect structure 1227. For example, the cores 1236A-1236B, 1237A-1237B, and 1238A-1238B may each communicate with the shared memory 1246 via the interconnect structure 1227. The interconnect structure 1227 may arbitrate communications within the graphics multiprocessor 1225 to ensure fair bandwidth allocation between components.

[0189] Fig. 12B The graphics multiprocessor 1250 includes multiple groups of execution resources 1256A-1256D, wherein each group of execution resources includes multiple instruction units, register files, GPGPU cores and load storage units, such as Fig.11D and Fig. 12A . Execution resources 1256A-1256D may work in conjunction with (one or more) texture units 1260A-1260D to perform texture operations while sharing instruction cache 1254 and shared memory 1253. For example, execution resources 1256A-1256D may share instruction cache 1254 and shared memory 1253, as well as multiple instances of texture and / or data caches 1258A-1258B. The various components may be connected to the same Fig. 12A The interconnect structure 1227 communicates with a similar interconnect structure 1252 .

[0190] Those skilled in the art will understand that Figure 1 , Figures 11A-11D and Figures 12A-12BThe architecture described in the description is merely illustrative and does not limit the scope of the current examples. Thus, the techniques described herein may be implemented on any appropriately configured processing unit, including but not limited to one or more mobile application processors, one or more desktop or server central processing units (CPUs) (including multi-core CPUs), one or more parallel processing units (e.g., Fig.11A parallel processing unit 1102), and one or more graphics processors or special purpose processing units without departing from the scope of the examples described herein.

[0191] A parallel processor or GPGPU as described herein may be communicatively coupled to a host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general purpose GPU (GPGPU) functions. The GPU may be communicatively coupled to a host processor / core via a bus or other interconnect (e.g., a high-speed interconnect, such as PCIe, NVLink, or other known protocols, standardized protocols, or proprietary protocols). In other examples, the GPU may be integrated with the core on the same package or chip and communicatively coupled to the core via an internal processor bus / interconnect (i.e., inside a package or chip). Regardless of how the GPU is connected, the processor core may assign work to the GPU in the form of a sequence of commands / instructions contained in a work descriptor. The GPU then uses dedicated circuits / logic to efficiently process these commands / instructions.

[0192] Fig. 12C A graphics processing unit (GPU) 1280 is illustrated that includes a collection of specialized graphics processing resources arranged into multi-core groups 1265A-1265N. Although details are provided for only a single multi-core group 1265A, it will be appreciated that other multi-core groups 1265B-1265N may be equipped with the same or similar sets of graphics processing resources. The detailed descriptions related to multi-core groups 1265A-1265N may also apply to any of the graphics multiprocessors 1134, 1225, and 1250 described herein.

[0193] As shown, multi-core group 1265A may include a set of graphics cores 1270, a set of tensor cores 1271, and a set of ray tracing cores 1272. Scheduler / dispatcher 1268 schedules and dispatches graphics threads for execution on various cores 1270, 1271, 1272. A set of register files 1269 may store operand values ​​used by cores 1270, 1271, and 1272 when executing graphics threads. These registers may include, for example, integer registers for storing integer values, floating point registers for storing floating point values, vector registers for storing packed data elements (integer and / or floating point data elements), and tile registers for storing tensor / matrix values. Tile registers may be implemented as a combined set of vector registers.

[0194] One or more combined level 1 (L1) caches and shared memory units 1273 store graphics data such as texture data, vertex data, pixel data, light data, bounded volume data, etc. locally within each multi-core group 1265A. One or more texture units 1274 can also be used to perform texture rendering operations, such as texture mapping and sampling. A level 2 (L2) cache 1275 shared by all or a subset of the multi-core groups 240A-240N can store graphics data and / or instructions for multiple concurrent graphics threads. As shown, the L2 cache 1275 can be shared between multiple multi-core groups 1265A-1265N. One or more memory controllers 1267 couple the GPU 1280 to the memory 1266, which can be system memory (e.g., DRAM) and / or dedicated graphics memory (e.g., GDDR6 memory).

[0195] Input / output (I / O) circuitry 1263 couples GPU 1280 to one or more I / O devices 1262, such as a digital signal processor (DSP), a network controller, or a user input device. On-chip interconnects may be used to couple I / O devices 1262 to GPU 1280 and memory 1266. One or more I / O memory management units (IOMMUs) 1264 of I / O circuitry 1263 couple I / O devices 1262 directly to memory 1266. Optionally, IOMMUs 1264 manage multiple sets of page tables to map virtual addresses to physical addresses in system memory 1266. I / O devices 1262, CPU(s) 1261, and GPU(s) 1280 may then share the same virtual address space.

[0196] In one implementation of IOMMU 1264, IOMMU 1264 supports virtualization. In this case, it can manage a first set of page tables to map guest / graphics virtual addresses to guest / graphics physical addresses, and a second set of page tables to map guest / graphics physical addresses to system / host physical addresses (e.g., within system memory 1266). The base address of each of the first and second sets of page tables can be stored in control registers and swapped out at context switches (e.g., to enable a new context to access the associated set of page tables). Although in Fig. 12CNot shown, each core 1270, 1271, 1272 and / or multi-core group 1265A-1265N may include a translation lookaside buffer (TLB) to cache guest virtual to guest physical translations, guest physical to host physical translations, and guest virtual to host physical translations.

[0197] (One or more) CPU 1261, GPU 1280 and I / O device 1262 can be integrated on a single semiconductor chip and / or chip package. The illustrated memory 1266 can be integrated on the same chip, or can be coupled to a memory controller 1267 via an off-chip interface. In one implementation, the memory 1266 includes GDDR6 memory that shares the same virtual address space with other physical system-level memories, but the underlying principles described herein are not limited to this particular implementation.

[0198] Tensor Core 1271 may include multiple execution units specifically designed to perform matrix operations, which are basic computational operations used to perform deep learning operations. For example, synchronized matrix multiplication operations can be used for neural network training and reasoning. Tensor Core 1271 can perform matrix processing using a variety of operand precisions, including single-precision floating point (e.g., 32 bits), half-precision floating point (e.g., 16 bits), integer words (16 bits), bytes (8 bits), and half bytes (4 bits). For example, a neural network implementation extracts features of each rendered scene, possibly combining details from multiple frames to construct a high-quality final image.

[0199] In a deep learning implementation, parallel matrix multiplication work can be scheduled to execute on the tensor core 1271. The training of neural networks in particular requires a large number of matrix dot product operations. To process the inner product formula of Nx Nx N matrix multiplication, the tensor core 1271 may include at least N dot product processing elements. Before the matrix multiplication begins, a complete matrix is ​​loaded into the tile register, and at least one column of the second matrix is ​​loaded in each cycle within N cycles. In each cycle, N dot products are processed.

[0200] Matrix elements can be stored with different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4), depending on the specific implementation. Different precision modes can be specified for the tensor core 1271 to ensure that the most efficient precision is used for different workloads (e.g., reasoning workloads, which can tolerate quantization to bytes and nibbles). Supported formats also include 64-bit floating point (FP64) and non-IEEE floating point formats, such as the bfloat16 format (e.g., brain floating point), which is a 16-bit floating point format with a sign bit, eight exponent bits, and eight significand bits, of which seven bits are explicitly stored. An example includes support for reduced precision tensor-floating point (TF32) mode, which performs calculations using the range of FP32 (8 bits) and the precision of FP16 (10 bits). Reduced precision TF32 operations can be performed on FP32 inputs and produce FP32 outputs with higher performance relative to FP32 and higher precision relative to FP16. In some examples, one or more 8-bit floating point formats (FP8) are supported.

[0201] In some examples, the tensor core 1271 supports a sparse operation mode of a matrix, in which the vast majority of values ​​are zero. The tensor core 1271 includes support for sparse input matrices encoded in a sparse matrix representation (e.g., coordinate list encoding (COO), compressed sparse rows (CSR), compressed sparse columns (CSC), etc.). In the case where the sparse matrix representation can be further compressed, the tensor core 1271 also includes support for compressed sparse matrix representation. Compressed, encoded and / or compressed and encoded matrix data and associated compression and / or encoding metadata can be read by the tensor core 1271, and non-zero values ​​can be extracted. For example, for a given input matrix A, non-zero values ​​can be loaded from a compressed and / or encoded representation of at least a portion of matrix A. Based on the position of the non-zero value in matrix A (which can be determined based on the index or coordinate metadata associated with the non-zero value), the corresponding value in the input matrix B can be loaded. Depending on the operation to be performed (e.g., multiplication), if the corresponding value is a zero value, the operation of loading the value from the input matrix B can be bypassed. In some examples, the pairing of values ​​for certain operations (e.g., multiplication operations) may be pre-scanned by the scheduler logic, and only operations between non-zero inputs are scheduled. Depending on the size of matrices A and B and the operations to be performed, the output matrix C may be dense or sparse. In the case where the output matrix C is sparse, depending on the configuration of the tensor core 1271, the output matrix C may be output in a compressed format, sparse coding, or compressed sparse coding.

[0202] The ray tracing core 1272 can accelerate ray tracing operations for both real-time ray tracing and non-real-time ray tracing implementations. Specifically, the ray tracing core 1272 may include a ray traversal / intersection circuit for performing ray traversal using a bounding volume hierarchy (BVH) and identifying intersections between rays and primitives enclosed within the BVH volume. The ray tracing unit 1272 may also include circuits for performing depth testing and culling (e.g., using a Z buffer or a similar arrangement). In one implementation, the ray tracing core 1272 performs traversal and intersection operations in conjunction with the image denoising technique described herein, at least a portion of which may be performed on the tensor core 1271. For example, the tensor core 1271 may implement a deep learning neural network to perform denoising on frames generated by the ray tracing core 1272. However, (one or more) CPUs 1261, graphics core 1270, and / or ray tracing core 1272 may also implement all or part of a denoising and / or deep learning algorithm.

[0203] In addition, as described above, a distributed denoising scheme may be employed, wherein the GPU 1280 is located in a computing device coupled to other computing devices via a network or high-speed interconnect. In such a distributed scheme, the interconnected computing devices may share neural network learning / training data to increase the speed at which the entire system learns to perform denoising for different types of image frames and / or different graphics applications.

[0204] The ray tracing core 1272 can handle all BVH traversals and / or ray primitive intersections so that the graphics core 1270 is not overloaded with thousands of instructions per ray. For example, each ray tracing core 1272 includes a first set of dedicated circuits for performing bounding box tests (e.g., for traversal operations) and / or a second set of dedicated circuits for performing ray-triangle intersection tests (e.g., traversed intersecting rays). Thus, for example, the multi-core group 1265A can simply launch a ray probe, and the ray tracing core 1272 independently performs ray traversals and intersections and returns hit data (e.g., hits, misses, multiple hits, etc.) to the thread context. While the ray tracing core 1272 performs traversals and intersection operations, the other cores 1270, 1271 are freed up to perform other graphics or computational work.

[0205] Optionally, each ray tracing core 1272 may include a traversal unit that performs BVH test operations and / or an intersection unit that performs ray-primitive intersection tests. The intersection unit generates a "hit", "miss", or "multiple hits" response and provides it to the appropriate thread. During traversal and intersection operations, execution resources of other cores (e.g., graphics core 1270 and tensor core 1271) are freed up to perform other forms of graphics work.

[0206] In some examples described below, a hybrid rasterization / ray tracing scheme is used in which the work is distributed between graphics core 1270 and ray tracing core 1272.

[0207] The ray tracing core 1272 (and / or other cores 1270, 1271) may include hardware support for ray tracing instruction sets, such as Microsoft's DirectX Ray Tracing (DXR), which includes the DispatchRays command, as well as ray generation, nearest hit, any hit, and miss shaders, which enable unique shader and texture sets to be assigned to each object. Another ray tracing platform that the ray tracing core 1272, graphics core 1270, and tensor core 1271 can support is the Vulkan API (e.g., Vulkan 1.1.85 and later). However, it should be noted that the underlying principles described herein are not limited to any particular ray tracing ISA.

[0208] In general, the various cores 1272, 1271, 1270 may support a ray tracing instruction set that includes instructions / functions for one or more of ray generation, nearest hit, any hit, ray-primitive intersection, per-primitive and hierarchical bounding box construction, misses, visits, and exceptions. More specifically, some examples include ray tracing instructions for performing one or more of the following functions:

[0209] Ray Generation - Ray generation instructions can be executed for each pixel, sample, or other user-defined work assignment.

[0210] Closest Hit—A closet hit instruction can be executed to locate the closest intersection point of a ray with a primitive within the scene.

[0211] Any Hit—Any Hit instruction identifies multiple intersections of a ray with primitives within the scene, potentially identifying a new closest intersection point.

[0212] Intersection - The intersection commands perform ray-primitive intersection tests and output the results.

[0213] Per-primitive bounding box construction—This instruction builds a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure).

[0214] Misses—Indicates that the ray missed all geometry within the scene or within a specified area of ​​the scene.

[0215] Visits—Indicates the subvolumes that the ray will traverse.

[0216] Exceptions—Includes various types of exception handlers (eg, called for various error conditions).

[0217] In some examples, the ray tracing core 1272 may be adapted to accelerate general computing operations that may be accelerated using computing techniques similar to ray intersection testing. A computing framework may be provided that enables shader programs to be compiled into low-level instructions and / or primitives that perform general computing operations via the ray tracing core. Typical computing problems that may benefit from computing operations performed on the ray tracing core 1272 include calculations involving the propagation of beams, waves, rays, or particles within a coordinate space. Interactions associated with the propagation may be calculated relative to geometry or meshes within the coordinate space. For example, calculations associated with the propagation of electromagnetic signals through an environment may be accelerated using instructions or primitives executed via the ray tracing core. Diffraction and reflection of signals by objects in the environment may be calculated as a direct ray tracing analogy.

[0218] The ray tracing core 1272 can also be used to perform calculations that are not directly similar to ray tracing. For example, the ray tracing core 1272 can be used to accelerate mesh projection, mesh refinement, and volume sampling calculations. General coordinate space calculations, such as nearest neighbor calculations, can also be performed. For example, by defining a bounding box in the coordinate space around a given point, a set of points near the point can be found. The BVH and ray detection logic within the ray tracing core 1272 can then be used to determine the set of intersections within the bounding box. These intersections constitute the origin and the nearest neighbors of the origin. Calculations performed using the ray tracing core 1272 can be performed in parallel with calculations performed on the graphics core 1272 and the tensor core 1271. The shader compiler can be configured to compile a compute shader or other general graphics processing program into a low-level primitive that can be processed in parallel on the graphics core 1270, the tensor core 1271, and the ray tracing core 1272.

[0219] Building larger and larger silicon dies is challenging for a variety of reasons. As silicon dies get larger, manufacturing yields become smaller, and process technology requirements for different components may diverge. On the other hand, to achieve high-performance systems, key components should be connected to each other through high-speed, high-bandwidth, low-latency interfaces. These conflicting requirements pose challenges to high-performance chip development.

[0220] The embodiments described herein provide a technique for decomposing the architecture of an on-chip integrated circuit system into a plurality of different chiplets, which can be packaged into a common housing. In some examples, a graphics processing unit or a parallel processor is composed of different silicon chiplets manufactured separately. A chiplet is an integrated circuit that is at least partially packaged, and the integrated circuit includes different logic units that can be assembled into a larger package with other chiplets. Different groups of chiplets with different IP core logics can be assembled into a single device. In addition, active insert technology can be used to integrate chiplets into a base die or base chiplet. The concept described herein makes it possible to interconnect and communicate between different forms of IP within a GPU. The development of IPs on different processes can be mixed. This avoids the complexity of converging multiple IPs, especially on large SoCs with multiple styles of IPs, to the same process.

[0221] Enabling the use of multiple process technologies improves time to market and provides a cost-effective way to create multiple product SKUs. For customers, this means getting products that better meet their requirements in a cost-effective and timely manner. In addition, disaggregated IP is more suitable for being independently power-gated, and components that are not used on a given workload can be powered off, reducing overall power consumption.

[0222] Fig.13 A parallel computing system 1300 according to some examples is shown. In some examples, the parallel computing system 1300 includes a parallel processor 1320, which can be a graphics processor or a computing accelerator as described herein. The parallel processor 1320 includes a global logic unit 1301, an interface 1302, a thread dispatcher 1303, a media unit 1304, a group of computing units 1305A-1305H, and a cache / memory unit 1306. The global logic unit 1301 includes global functions of the parallel processor 1320 in some examples, including device configuration registers, a global scheduler, power management logic, etc. The interface 1302 may include a front-end interface of the parallel processor 1320. The thread dispatcher 1303 can receive a workload from the interface 1302 and dispatch the threads of the workload to the computing units 1305A-1305H. If the workload includes any media operations, at least a portion of these operations can be performed by the media unit 1304. The media unit may also offload some operations to the compute units 1305A-1305H. The cache / memory unit 1306 may include cache memory (eg, L3 cache) and local memory (eg, HBM, GDDR) for the parallel processors 1320.

[0223] Figures 14A-14BIllustrated is a hybrid logical / physical view of a factorized parallel processor according to examples described herein. Fig.14A A decomposed parallel computing system 1400 is illustrated. Fig. 14B A core 1430 of the decomposed parallel computing system 1400 is illustrated.

[0224] like Fig.14A As shown, the decomposed computing system 1400 may include a parallel processor 1420, wherein various components of the parallel processor SOC are distributed on multiple core particles. Each core particle can be a different IP core of independent design, and the IP core is configured to communicate with other core particles via one or more shared interfaces. Core particles include, but are not limited to, computing core particles 1405, media core particles 1404, and memory core particles 1406. Each core particle can be manufactured separately using different process technologies. For example, the computing core particle 1405 can be manufactured using the smallest or most advanced process technology available at the time of manufacture, while the memory core particle 1406 or other core particles (e.g., I / O, networking, etc.) can be manufactured using larger or less advanced process technologies.

[0225] Various cores may be bonded to the base die 1410 and configured to communicate with each other and logic within the base die 1410 via an interconnect layer 1412. In some examples, the base die 1410 may include a global logic 1401, which may include a scheduler 1411 and a power management 1421 logic unit, an interface 1402, a dispatch unit 1403, and an interconnect fabric module 1408 coupled or integrated with one or more L3 cache bank groups 1409A-1409N. The interconnect fabric 1408 may be an inter-core fabric integrated into the base die 1410. The logic cores may use the fabric 1408 to pass messages between the various cores. In addition, the L3 cache bank groups 1409A-1409N in the base die and / or the L3 cache bank groups within the memory core 1406 may cache data read from the DRAM cores within the memory core 1406 and sent to these DRAM cores and to the system memory of the host.

[0226] In some examples, global logic 1401 is a microcontroller that can execute firmware to perform scheduler 1411 and power management 1421 functions for parallel processor 1420. The microcontroller that executes the global logic can be customized based on the target use case of parallel processor 1420. Scheduler 1411 can perform global scheduling operations for parallel processor 1420. Power management 1421 functions can be used to enable or disable individual cores within a parallel processor when they are not in use.

[0227] The various cores of parallel processor 1420 may be designed to perform specific functions that would otherwise be integrated into a single die in existing designs. A set of compute cores 1405 may include a cluster of compute units (e.g., execution units, streaming multiprocessors, etc.) that include programmable logic to execute compute or graphics shader instructions. Media cores 1404 may include hardware logic to accelerate media encoding and decoding operations. Memory cores 1406 may include volatile memory (e.g., DRAM) and one or more SRAM cache memory banks (e.g., L3 banks).

[0228] like Fig. 14B As shown, each core grain 1430 may include common components and application-specific components. The core grain logic 1436 within the core grain 1430 may include specific components of the core grain, such as an array of streaming multiprocessors, computing units, or execution units described herein. The core grain logic 1436 may be coupled to an optional cache or shared local memory 1438, or may include a cache or shared local memory within the core grain logic 1436. The core grain 1430 may include a fabric interconnect node 1442 that receives commands via an inter-core grain structure. Commands and data received via the fabric interconnect node 1442 may be temporarily stored in an interconnect buffer 1439. Data sent to and received from the fabric interconnect node 1442 may be stored in the interconnect cache 1440. Power control 1432 and clock control 1434 logic may also be included within the core grain. The power control 1432 and clock control 1434 logic may receive configuration commands via the structure and may configure dynamic voltage and frequency scaling for the core grain 1430. In some examples, each coredie may have independent clock and power domains and may be clock-gated and power-gated independently of other coredies.

[0229] At least a portion of the components within the illustrated core particle 1430 may also be included in a device embedded in Fig.14A The logic within the base die 1410 of the fabric may include a version of the fabric interconnect node 1442. The independently clockable or power gated base die logic may include a version of the power control 1432 and / or clock control 1434 logic.

[0230] Thus, while various examples described herein use the term SOC to describe a device or system having a processor and associated circuitry (e.g., input / output ("I / O") circuitry, power delivery circuitry, memory circuitry, etc.) monolithically integrated into a single integrated circuit ("IC") die or chip, the present disclosure is not limited in this regard. For example, in various examples of the present disclosure, a device or system may have one or more processors (e.g., one or more processor cores) and associated circuitry (e.g., input / output ("I / O") circuitry, power delivery circuitry, etc.) that are arranged in a disaggregated collection of discrete dies, tiles, and / or cores (e.g., one or more discrete processor core dies are arranged adjacent to one or more other dies (e.g., memory dies, I / O dies, etc.)). In such disaggregated devices and systems, the various dies, tiles, and / or cores may be physically and electrically coupled together by a packaging structure that includes, for example, various packaging substrates, interposers, active interposers, photonic interposers, interconnect bridges, and the like. A disaggregated collection of discrete dies, tiles, and / or cores may also be part of a system-in-package ("SoP").

[0231] Example core architectures—in-order and out-of-order core block diagrams.

[0232] The block diagram of FIG. 15(A) illustrates both an example in-order pipeline and an example register renaming, out-of-order issue / execution pipeline according to some examples. The block diagram of FIG. 15(B) illustrates both an example in-order architecture core and an example register renaming, out-of-order issue / execution architecture core to be included in a processor according to an example. The solid line boxes in FIG. 15(A)-(B) illustrate the in-order pipeline and the in-order core, while the optionally added dashed line boxes illustrate the register renaming, out-of-order issue / execution pipeline and core. Considering that the in-order aspect is a subset of the out-of-order aspect, the out-of-order aspect will be described.

[0233] In FIG. 15(A), a processor pipeline 1500 includes a fetch stage 1502, an optional length decoding stage 1504, a decode stage 1506, an optional allocation (Alloc) stage 1508, an optional rename stage 1510, a dispatch (also called dispatch or issue) stage 1512, an optional register read / memory read stage 1514, an execute stage 1516, a write back / memory write stage 1518, an optional exception handling stage 1522, and an optional commit stage 1524. One or more operations may be performed in each of these processor pipeline stages. For example, during the fetch stage 1502, one or more instructions are fetched from an instruction memory, and during the decode stage 1506, the fetched one or more instructions may be decoded, an address using a forwarding register port (e.g., a load store unit (LSU) address) may be generated, and branch forwarding (e.g., an immediate offset or a link register (LR)) may be performed. In some examples, decode stage 1506 and register read / memory read stage 1514 can be combined into one pipeline stage. In some examples, during execute stage 1516, decoded instructions can be executed, LSU address / data pipelining to an Advanced Microcontroller Bus (AMB) interface can be performed, multiplication and addition operations can be performed, arithmetic operations with branch results can be performed, and the like.

[0234] As an example, the example register renaming, out-of-order issue / execution architecture core of FIG. 15(B) may implement pipeline 1500 in the following manner: 1) instruction fetch circuit 1538 performs fetch and length decoding stages 1502 and 1504; 2) decoding circuit 1540 performs decoding stage 1506; 3) renaming / allocator unit circuit 1552 performs allocation stage 1508 and renaming stage 1510; 4) (one or more) scheduler circuit 1556 performs scheduling stage 1512; 5) (one or more) physical registers The register file circuit 1558 and the memory unit circuit 1570 perform the register read / memory read phase 1514; (one or more) execution clusters 1560 perform the execute phase 1516; 6) the memory unit circuit 1570 and (one or more) physical register file circuits 1558 perform the write back / memory write phase 1518; 7) various circuits may be involved in the exception handling phase 1522; and 8) the retirement unit circuit 1554 and (one or more) physical register file circuits 1558 perform the commit phase 1524.

[0235] 15(B) shows that the processor core 1590 includes a front end unit circuit 1530 coupled to an execution engine unit circuit 1550, and both are coupled to a memory unit circuit 1570. The core 1590 can be a reduced instruction set architecture computing (RISC) core, a complex instruction set architecture computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As another option, the core 1590 can be a special-purpose core, such as a network or communication core, a compression engine, a coprocessor core, a general purpose computing graphics processing unit (GPGPU) core, a graphics core, and the like.

[0236] The front end unit circuit 1530 may include a branch prediction circuit 1532 coupled to an instruction cache circuit 1534 coupled to an instruction translation lookaside buffer (TLB) 1536 coupled to an instruction fetch circuit 1538 coupled to a decode circuit 1540. In some examples, the instruction cache circuit 1534 is included in the memory unit circuit 1570 instead of the front end circuit 1530. The decode circuit 1540 (or decoder) may decode the instruction and generate one or more micro-operations, microcode entry points, micro-instructions, other instructions, or other control signals as outputs, which are decoded from the original instruction, or otherwise reflect the original instruction, or are derived from the original instruction. The decode circuit 1540 may also include an address generation unit (AGU, not shown) circuit. In some examples, the AGU uses the forwarded register port to generate the LSU address and may further perform branch forwarding (e.g., immediate offset branch forwarding, LR register branch forwarding, etc.). The decode circuit 1540 may be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLA), microcode read only memory (ROM), and the like. In some examples, the core 1590 includes a microcode ROM (not shown) or other medium that stores microcode for certain macroinstructions (e.g., in the decode circuit 1540 or otherwise within the front end circuit 1530). In some examples, the decode circuit 1540 includes a micro-op or operation cache (not shown) to save / cache decoded operations, micro-tags, or micro-operations generated during decoding or other stages of the processor pipeline 1500. The decode circuit 1540 may be coupled to the rename / allocator unit circuit 1552 in the execution engine circuit 1550.

[0237] The execution engine circuit 1550 includes a rename / allocator unit circuit 1552, which is coupled to a retirement unit circuit 1554 and a set of one or more scheduler circuits 1556. The scheduler circuit 1556 represents any number of different schedulers, including reservation stations, central instruction windows, and the like. In some examples, the scheduler circuit(s) 1556 may include an arithmetic logic unit (ALU) scheduler / scheduling circuit, an ALU queue, an address generation unit (AGU) scheduler / scheduling circuit, an AGU queue, and the like. The scheduler circuit(s) 1556 are coupled to the physical register file(s) circuit(s) 1558. Each of the physical register file circuits 1558 represents one or more physical register files, different ones of which store one or more different data types, such as scalar integers, scalar floating points, packed integers, packed floating points, vector integers, vector floating points, status (e.g., an instruction pointer that is the address of the next instruction to be executed), and the like. In some examples, the (one or more) physical register file circuits 1558 include vector register unit circuits, write mask register unit circuits, and scalar register unit circuits. These register units can provide architectural vector registers, vector mask registers, general registers, etc. The (one or more) physical register file circuits 1558 are coupled to the retirement unit circuit 1554 (also called the retirement queue) to demonstrate various ways that register renaming and out-of-order execution can be implemented (e.g., using (one or more) reorder buffers (ROB) and (one or more) retirement register files; using (one or more) future stacks, (one or more) history buffers, and (one or more) retirement register files; using register maps and pools of registers; etc.). The retirement unit circuit 1554 and the (one or more) physical register file circuits 1558 are coupled to (one or more) execution clusters 1560. The (one or more) execution clusters 1560 include a set of one or more execution unit circuits 1562 and a set of one or more memory access circuits 1564. The execution unit circuit(s) 1562 may perform various arithmetic, logical, floating point, or other types of operations (e.g., shifts, additions, subtractions, multiplications) on various types of data (e.g., scalar integer, scalar floating point, packed integer, packed floating point, vector integer, vector floating point). While some examples may include several execution units or execution unit circuits dedicated to a particular function or set of functions, other examples may include only one execution unit circuit or multiple execution units / execution unit circuits that all perform all functions.Scheduler circuit(s) 1556, physical register file circuit(s) 1558, and execution cluster(s) 1560 are shown as possibly multiple because some examples create separate pipelines for certain types of data / operations (e.g., a scalar integer pipeline, a scalar floating point / packed integer / packed floating point / vector integer / vector floating point pipeline, and / or a memory access pipeline, each with its own scheduler circuit(s), physical register file circuit(s), and / or execution cluster—and in the case of a separate memory access pipeline, only the execution cluster of that pipeline has memory access unit circuit(s) 1564 in some examples of the implementation). It should also be understood that where separate pipelines are used, one or more of these pipelines may be out-of-order issue / execution, while the rest are in-order.

[0238] In some examples, the execution engine unit circuit 1550 can perform load store unit (LSU) address / data pipelining to an advanced microcontroller bus (AMB) interface (not shown), as well as address stage and writeback, data stage loads, stores, and branches.

[0239] A set of memory access circuits 1564 are coupled to the memory cell circuit 1570, which includes a data TLB circuit 1572, which is coupled to a data cache circuit 1574, which is coupled to a level 2 (L2) cache circuit 1576. In some examples, the memory access circuit 1564 may include a load unit circuit, a storage address unit circuit, and a storage data unit circuit, each of which is coupled to the data TLB circuit 1572 in the memory cell circuit 1570. The instruction cache circuit 1534 is further coupled to the level 2 (L2) cache circuit 1576 in the memory cell circuit 1570. In some examples, the instruction cache 1534 and the data cache 1574 are combined into a L2 cache circuit 1576, a level 3 (L3) cache circuit (not shown), and / or a single instruction and data cache in the main memory (not shown). The L2 cache circuit 1576 is coupled to one or more other levels of cache and ultimately to the main memory.

[0240] Core 1590 may support one or more instruction sets (e.g., x86 instruction set architecture (optionally with some extensions that have been added with newer versions); MIPS instruction set architecture; ARM instruction set architecture (optionally with optional additional extensions, such as NEON)) that include the instruction(s) described herein. In some examples, core 1590 includes logic to support packed data instruction set architecture extensions (e.g., AVX1, AVX2), allowing operations used by many multimedia applications to be performed using packed data.

[0241] Example execution unit circuit.

[0242] Fig.16 An example of (one or more) execution unit circuits is illustrated, such as (one or more) execution unit circuits 1562 of FIG. 15 (B). As shown, (one or more) execution unit circuits 1562 may include one or more ALU circuits 1601, optional vector / single instruction multiple data (single instruction multiple data, SIMD) circuits 1603, load / store circuits 1605, branch / jump circuits 1607, and / or floating-point unit (FPU) circuits 1609. ALU circuits 1601 perform integer arithmetic and / or Boolean operations. Vector / SIMD circuits 1603 perform vector / SIMD operations on packed data (e.g., SIMD / vector registers). Load / store circuits 1605 execute load and store instructions to load data from memory into registers, or store data from registers to memory. Load / store circuits 1605 may also generate addresses. Branch / jump circuits 1607 cause branches or jumps to a certain memory address depending on the instruction. FPU circuit 1609 performs floating point arithmetic. The width of execution unit circuit(s) 1562 varies depending on the example and may range, for example, from 16 bits to 1024 bits. In some examples, two or more smaller execution units are logically combined to form a larger execution unit (e.g., two 128-bit execution units are logically combined to form a 256-bit execution unit).

[0243] Example register architecture.

[0244] Fig.171 is a block diagram of a register architecture 1700 according to some examples. As shown, the register architecture 1700 includes a vector / SIMD register 1710, whose width varies from 128 bits to 1024 bits. In some examples, the vector / SIMD register 1710 is physically 512 bits, and only some low-order bits are used depending on the mapping. For example, in some examples, the vector / SIMD register 1710 is a 512-bit ZMM register: the low-order 256 bits are used for the YMM register, and the low-order 128 bits are used for the XMM register. Therefore, there is an overlay of registers. In some examples, the vector length field selects between the maximum length and one or more other shorter lengths, wherein each such shorter length is half the length of the previous length. Scalar operations are operations performed on the lowest-order data element position in the ZMM / YMM / XMM register; higher-order data element positions are either maintained as the same as before the instruction, or are zeroed, depending on the example.

[0245] In some examples, the register architecture 1700 includes a write mask / predicate register 1715. For example, in some examples, there are 8 write mask / predicate registers (sometimes referred to as k0 to k7), each of which is 16 bits, 32 bits, 64 bits, or 128 bits in size. The write mask / predicate register 1715 can allow merging (e.g., allowing any set of elements in the destination to be protected from update during the execution of any operation) and / or zeroing (e.g., a zeroing vector mask allows any set of elements in the destination to be zeroed during the execution of any operation). In some examples, each data element position in a given write mask / predicate register 1715 corresponds to a data element position at the destination. In other examples, the write mask / predicate register 1715 is scalable and consists of a set number of enable bits for a given vector element (e.g., 8 enable bits for each 64-bit vector element).

[0246] The register architecture 1700 includes a plurality of general purpose registers 1725. These registers may be 16 bits, 32 bits, 64 bits, etc., and may be used for scalar operations. In some examples, these registers are referred to by the names RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, and R8 to R15.

[0247] In some examples, register architecture 1700 includes a scalar floating-point (FP) register file 1745, which is used to perform scalar floating-point operations on 32 / 64 / 80-bit floating-point data using x87 instruction set architecture extensions, or as an MMX register to perform operations on 64-bit packed integer data, and to store operands for some operations performed between MMX and XMM registers.

[0248] One or more flag registers 1740 (e.g., EFLAGS, RFLAGS, etc.) store status and control information for arithmetic, comparison, and system operations. For example, one or more flag registers 1740 can store condition code information, such as carry, parity, auxiliary carry, zero, sign, and overflow. In some examples, one or more flag registers 1740 are referred to as program status and control registers.

[0249] Segment registers 1720 contain segment points for accessing memory. In some examples, these registers are referred to by the names CS, DS, SS, ES, FS, and GS.

[0250] Model specific registers or machine specific registers (MSR) 1735 control and report processor performance. Most MSRs 1735 handle system-related functions and are inaccessible to applications. For example, MSRs may provide control of one or more of the following: performance monitoring counters, debug extensions, memory type range registers, thermal management and power management, instruction specific support, and / or processor feature / mode support. Machine check registers 1760 consist of control, status, and error reporting MSRs for detecting and reporting hardware errors. (One or more) control registers 1755 (e.g., CR0-CR4) determine the operating mode of the processor (e.g., processors 870, 880, 838, 815, and / or 900) and the characteristics of the task currently being executed. In some examples, MSRs 1735 are a subset of control registers 1755.

[0251] One or more instruction pointer registers 1730 store instruction pointer values. Debug registers 1750 control and allow monitoring of debug operations of the processor or core.

[0252] Memory (mem) management registers 1765 specify the location of data structures used in protected mode memory management. These registers may include global descriptor table registers (GDTR), interrupt descriptor table registers (IDTR), task registers, and local descriptor table registers (LDTR) registers.

[0253] Alternative examples may use wider or narrower registers. In addition, alternative examples may use more, less or different register files and registers. Register architecture 1700 may be used, for example, in register file / memory `ISAB08, or in physical register file circuit 1558.

[0254] Instruction set architecture.

[0255] An instruction set architecture (ISA) may include one or more instruction formats. A given instruction format may define various fields (e.g., the number of bits, the position of the bits) to specify the operation to be performed (e.g., opcode) and the (one or more) operands and / or other (one or more) data fields (e.g., mask) on which the operation is to be performed, etc. Some instruction formats are further decomposed by the definition of instruction templates (or subformats). For example, an instruction template of a given instruction format may be defined as having different subsets of the fields of the instruction format (the included fields are usually in the same order, but at least some have different bit positions because there are fewer included fields) and / or defined as having a given field interpreted in a different manner. Thus, each instruction of the ISA is expressed using a given instruction format (and, if defined, expressed in a given instruction template in the instruction format), and includes fields for specifying the operation and the operand. For example, the example ADD instruction has a specific opcode and instruction format, the instruction format includes an opcode field to specify the opcode, and includes an operand field to select the operand (source 1 / destination and source 2); and the appearance of this ADD instruction in the instruction stream will have specific content in the operand field that selects the specific operand. In addition, although the following description is made in the context of the x86 ISA, it is within the knowledge of those skilled in the art to apply the teachings of the present disclosure to other ISAs.

[0256] Example instruction format.

[0257] The examples of the (one or more) instructions described herein may be implemented in different formats. In addition, example systems, architectures, and pipelines are described in detail below. The examples of (one or more) instructions may be executed on these systems, architectures, and pipelines, but are not limited to those described in detail.

[0258] Fig.18 An example of an instruction format is illustrated. As shown, an instruction may include multiple components, including, but not limited to, one or more fields for: one or more prefixes 1801, an opcode 1803, addressing information 1805 (e.g., a register identifier, memory addressing information, etc.), a displacement value 1807, and / or an immediate value 1809. Note that some instructions utilize some or all of the fields of the format, while other instructions may only use the opcode 1803 field. In some examples, the illustrated order is the order in which the fields are to be encoded, however, it should be understood that in other examples, the fields may be encoded in another order, combined, etc.

[0259] (One or more) prefix fields 1801 modify the instruction when used. In some examples, one or more prefixes are used to repeat string instructions (e.g., 0xF0, 0xF2, 0xF3, etc.), provide section override (e.g., 0x2E, 0x36, 0x3E, 0x26, 0x64, 0x65, 0x2E, 0x3E, etc.), perform bus lock operations, and / or change operation objects (e.g., 0x66) and address sizes (e.g., 0x67). Some instructions require mandatory prefixes (e.g., 0x66, 0xF2, 0xF3, etc.). Some of these prefixes can be considered as "traditional" prefixes. Other prefixes (one or more examples of which are described in detail herein) indicate and / or provide further capabilities, such as specifying specific registers, etc. These other prefixes are usually followed by the "traditional" prefix.

[0260] The opcode field 1803 is used to at least partially define the operation to be performed when the instruction is decoded. In some examples, the length of the primary opcode encoded in the opcode field 1803 is one, two or three bytes. In other examples, the primary opcode can be other lengths. Extra 3-bit opcode fields are sometimes encoded in another field.

[0261] The addressing information field 1805 is used to address one or more operands of the instruction, such as a location in memory or one or more registers. Fig.19An example of an addressing information field 1805 is illustrated. In this illustration, an optional MOD R / M byte 1902 and an optional Scale, Index, Base (SIB) byte 1904 are shown. The MOD R / M byte 1902 and SIB byte 1904 are used to encode up to two operands of the instruction, each operand being a direct register or an effective memory address. Note that these fields are optional, i.e., not all instructions include one or more of these fields. The MOD R / M byte 1902 includes a MOD field 1942, a register (reg) field 1944, and an R / M field 1946.

[0262] The content of MOD field 1942 distinguishes between memory access and non-memory access modes. In some examples, when MOD field 1942 has a binary value of 11 (11b), register direct addressing mode is utilized, otherwise register indirect addressing mode is used.

[0263] Register field 1944 may encode a destination register operand or a source register operand, or may encode an opcode extension and not be used to encode any instruction operand. The contents of register field 1944 specify the location (in a register or in memory) of the source or destination operand directly or through address generation. In some examples, register field 1944 is supplemented with additional bits from a prefix (e.g., prefix 1801) to allow for greater addressing.

[0264] R / M field 1946 may be used to encode an instruction operand that references a memory address, or may be used to encode a destination register operand or a source register operand. Note that in some examples, R / M field 1946 may be combined with MOD field 1942 to specify an addressing mode.

[0265] SIB byte 1904 includes a scale field 1952, an index field 1954, and a base address field 1956 for address generation. Scaling field 1952 indicates a scaling factor. Index field 1954 specifies the index register to be used. In some examples, index field 1954 is complemented with additional bits from a prefix (e.g., prefix 1801) to allow for larger addressing. Base address field 1956 specifies the base address register to be used. In some examples, base address field 1956 is complemented with additional bits from a prefix (e.g., prefix 1801) to allow for larger addressing. In practice, the contents of scale field 1952 allow the contents of index field 1954 to be scaled for memory address generation (e.g., for memory addresses using 2 缩放 * address generation of index + base address).

[0266] Some addressing forms utilize a displacement value to generate a memory address. For example, a memory address may be generated based on 2 scale * index + base address + displacement, index * scale + displacement, r / m + displacement, instruction pointer (RIP / EIP) + displacement, register + displacement, etc. The displacement may be a value of 1 byte, 2 bytes, 4 bytes, etc. In some examples, the displacement field 1807 provides this value. In addition, in some examples, the use of a displacement factor is encoded in the MOD field of the addressing information field 1805, which indicates a compressed displacement scheme, for which the displacement value is calculated and stored in the displacement field 1807.

[0267] In some examples, the immediate value field 1809 specifies an immediate value for the instruction. The immediate value may be encoded as a 1-byte value, a 2-byte value, a 4-byte value, and so on.

[0268] Fig. 20 An example of a first prefix 1801(A) is illustrated. In some examples, the first prefix 1801(A) is an example of a REX prefix. Instructions using this prefix can specify general registers, 64-bit packed data registers (e.g., single instruction, multiple data (SIMD) registers or vector registers), and / or control registers and debug registers (e.g., CR8-CR15 and DR8-DR15).

[0269] Instructions using the first prefix 1801(A) can specify up to three registers using a 3-bit field, depending on the format: 1) using the reg field 1944 and the R / M field 1946 of the MOD R / M byte 1902; 2) using the MOD R / M byte 1902 with the SIB byte 1904, including using the reg field 1944 as well as the base field 1956 and the index field 1954; or 3) using the register field of the opcode.

[0270] In the first prefix 1801 (A), bit positions 7:4 of the payload byte are set to 0100. Bit position 3 (W) can be used to determine the operand size, but cannot determine the operand width alone. Therefore, when W=0, the operand size is determined by the code segment descriptor (CS.D), and when W=1, the operand size is 64 bits.

[0271] Note that adding another bit allows addressing of 16 (2 4 ) registers, while the separate MOD R / M reg field 1944 and MOD R / MR / M field 1946 can only address 8 registers each.

[0272] In the first prefix 1801(A), bit position 2 (R) may be an extension of the MOD R / M reg field 1944 and may be used to modify the MOD R / M reg field 1944 when the field encodes a general register, a 64-bit packed data register (e.g., an SSE register), or a control or debug register. When the MOD R / M byte 1902 specifies other registers or defines an extended opcode, R is ignored.

[0273] Bit position 1 (X) may modify the SIB byte index field 1954 .

[0274] Bit position 0 (B) may modify the base address in R / M field 1946 or SIB byte base address field 1956 of MOD R / M; or it may modify the opcode register field used to access a general register (eg, general register 1725).

[0275] 21(A)-(D) illustrate examples of how the R, X, and B fields of the first prefix 1801(A) are used. FIG. 21(A) illustrates that when the SIB byte 1904 is not used for memory addressing, the R and B from the first prefix 1801(A) are used to extend the reg field 1944 and the R / M field 1946 of the MOD R / M byte 1902. FIG. 21(B) illustrates that when the SIB byte 1904 is not used, the R and B from the first prefix 1801(A) are used to extend the reg field 1944 and the R / M field 1946 of the MOD R / M byte 1902 (register-register addressing). FIG. 21(C) illustrates that when the SIB byte 1904 is used for memory addressing, the R, X, and B from the first prefix 1801(A) are used to extend the reg field 1944 and the index field 1954 and the base address field 1956 of the MOD R / M byte 1902. 21(D) illustrates that when a register is encoded in the opcode 1803, the B from the first prefix 1801(A) is used to extend the reg field 1944 of the MOD R / M byte 1902.

[0276] Figure 22 (A)-(B) illustrates an example of a second prefix 1801 (B). In some examples, the second prefix 1801 (B) is an example of a VEX prefix. The second prefix 1801 (B) encoding allows instructions to have more than two operands, and allows SIMD vector registers (e.g., vector / SIMD registers 1710) to be longer than 64 bits (e.g., 128 bits and 256 bits). The use of the second prefix 1801 (B) provides a syntax for three operands (or more). For example, a previous two-operand instruction performs an operation such as A=A+B, which overwrites the source operand. The use of the second prefix 1801 (B) enables the operands to perform non-destructive operations, such as A=B+C.

[0277] In some examples, the second prefix 1801(B) has two forms: a two-byte form and a three-byte form. The two-byte second prefix 1801(B) is mainly used for 128-bit, scalar, and some 256-bit instructions; while the three-byte second prefix 1801(B) provides a compact replacement for 3-byte opcode instructions and the first prefix 1801(A).

[0278] Figure 22 (A) illustrates an example of a two-byte form of the second prefix 1801 (B). In some examples, the format field 2201 (byte 0 2203) contains the value C5H. In some examples, byte 1 2205 includes an "R" value in bit [7]. This value is the complement of the "R" value of the first prefix 1801 (A). Bit [2] is used to specify the length (L) of the vector (wherein the value of 0 is a scalar or 128-bit vector, and the value of 1 is a 256-bit vector). Bit [1:0] provides an opcode extensibility equivalent to some traditional prefixes (e.g., 00=no prefix, 01=66H, 10=F3H, and 11=F2H). Bits [6:3], shown as vvvv, can be used to: 1) encode a first source register operand, which is specified in inverted (complement) form, valid for instructions with 2 or more source operands; 2) encode a destination register operand, which is specified in complement form, used for certain vector shifts; or 3) not encode any operand, the field is reserved and should contain a value, such as 1111b.

[0279] Instructions using this prefix may use MOD R / MR / M field 1946 to encode instruction operands that reference memory addresses, or to encode destination register operands or source register operands.

[0280] Instructions using this prefix may use MOD R / M reg field 1944 to encode a destination register operand or a source register operand, or be treated as an opcode extension and not used to encode any instruction operand.

[0281] For instruction syntax supporting four operands, three of the four operands are encoded by vvvv, MOD R / MR / M field 1946, and MOD R / M reg field 1944. Bits [7:4] of immediate value field 1809 are then used to encode the third source register operand.

[0282] Figure 22 (B) illustrates an example of a three-byte form of the second prefix 1801 (B). In some examples, the format field 2211 (byte 0 2213) contains the value C4H. Byte 1 2215 includes "R", "X" and "B" in bit [7:5], which are the complements of these values ​​of the first prefix 1801 (A). Bit [4:0] (shown as mmmmm) of byte 1 2215 includes content that encodes one or more implicit leading opcode bytes as needed. For example, 00001 means 0FH leading opcode, 00010 means 0F38H leading opcode, 00011 means 0F3AH leading opcode, and so on.

[0283] The use of bit [7] of byte 2 2217 is similar to W of the first prefix 1801 (A), including helping to determine the size of the operand that can be promoted. Bit [2] is used to specify the length (L) of the vector (where a value of 0 is a scalar or 128-bit vector, and a value of 1 is a 256-bit vector). Bits [1:0] provide opcode extensibility equivalent to some traditional prefixes (e.g., 00=no prefix, 01=66H, 10=F3H, and 11=F2H). Bits [6:3], shown as vvvv, can be used to: 1) encode the first source register operand, which is specified in inverted (inverse) form, valid for instructions with 2 or more source operands; 2) encode the destination register operand, which is specified in inverse form for certain vector shifts; or 3) do not encode any operand, this field is reserved and should contain a value, such as 1111b.

[0284] Instructions using this prefix may use MOD R / MR / M field 1946 to encode instruction operands that reference memory addresses, or to encode destination register operands or source register operands.

[0285] Instructions using this prefix may use MOD R / M reg field 1944 to encode a destination register operand or a source register operand, or be treated as an opcode extension and not used to encode any instruction operand.

[0286] For instruction syntax supporting four operands, three of the four operands are encoded by vvvv, MOD R / MR / M field 1946, and MOD R / M reg field 1944. Bits [7:4] of immediate value field 1809 are then used to encode the third source register operand.

[0287] Fig.23 An example of a third prefix 1801(C) is illustrated. In some examples, the third prefix 1801(C) is an example of an EVEX prefix. The third prefix 1801(C) is a four-byte prefix.

[0288] The third prefix 1801(C) can encode 32 vector registers (e.g., 128-bit, 256-bit, and 512-bit registers) in 64-bit mode. In some examples, write masks / operation masks are used (see the discussion of registers in previous figures, e.g., Fig.17 ) or predicate instructions utilize this prefix. The operation mask register allows conditional processing or selection control. Operation mask instructions whose source / destination operands are operation mask registers and treat the contents of the operation mask register as a single value are encoded using the second prefix 1801 (B).

[0289] The third prefix 1801(C) may encode functionality specific to the instruction class (e.g., a packed instruction with “load+operation” semantics may support an embedded broadcast functionality, a floating-point instruction with rounding semantics may support a static rounding functionality, a floating-point instruction with non-rounding arithmetic semantics may support a “suppress all exceptions” functionality, etc.).

[0290] The first byte of the third prefix 1801 (C) is the format field 2311, which in some examples has a value of 62 H. The subsequent bytes are called payload bytes 2315-2319, and together form a 24-bit value of P[23:0], providing specific capabilities in the form of one or more fields (described in detail herein).

[0291] In some examples, P[1:0] of payload byte 2319 is the same as the two mm bits in the lower order. In some examples, P[3:2] is reserved. Bit P[4] (R') allows access to the upper 16 vector register sets when combined with P[7] and the reg field 1944 of MOD R / M. When SIB type addressing is not required, P[6] can also provide access to the upper 16 vector registers. P[7:5] consists of R, X, and B, which are operand specifier modifier bits for vector registers, general registers, and memory addressing, and when combined with the MOD R / M register field 1944 and the MOD R / MR / M field 1946, allows access to the next set of 8 registers beyond the lower 8 registers. P[9:8] provides opcode extensibility equivalent to some traditional prefixes (e.g., 00=no prefix, 01=66H, 10=F3H, and 11=F2H). P

[10] is a fixed value of 1 in some examples. P[14:11], shown as vvvv, can be used to: 1) encode a first source register operand, which is specified in inverted (complement) form, valid for instructions with 2 or more source operands; 2) encode a destination register operand, which is specified in complement form, used for certain vector shifts; or 3) not encode any operand, the field is reserved and should contain a value, such as 1111b.

[0292] P

[15] is similar to W of the first prefix 1801 (A) and the second prefix 1811 (B), and can be used as an opcode extension bit or an operand size increase.

[0293] P[18:16] specifies the index of a register in an operation mask (write mask) register (e.g., write mask / predicate register 1715). In some examples, a particular value aaa=000 has special behavior, implying that no operation mask is used for this particular instruction (this can be implemented in a variety of ways, including using an operation mask hardwired to all ones or hardware that bypasses the masking hardware). When merging, the vector mask allows any set of elements in the destination to be protected from updates during the execution of any operation (specified by the basic and enhanced operations); in some other examples, the old value of each element of the destination where the corresponding mask bit has a value of 0 is retained. In contrast, when zeroing, the vector mask allows any set of elements in the destination to be zeroed during the execution of any operation (specified by the basic and enhanced operations); in some examples, the elements of the destination are set to 0 when the corresponding mask bit has a value of 0. A subset of this functionality is the ability to control the vector length of the operation being performed (i.e., the span of elements modified, from first to last); however, the elements modified do not have to be contiguous. Thus, the operation mask field allows for a subset of vector operations, including loads, stores, arithmetic, logical, and so on. While in the described examples, the contents of the operation mask field selects which of several operation mask registers contains the operation mask to be used (thus the contents of the operation mask field indirectly identify the masking to be performed), alternative examples alternatively or additionally allow the contents of the mask write field to directly specify the masking to be performed.

[0294] P

[19] can be combined with P[14:11] to encode a second source vector register in a non-destructive source syntax that can utilize P

[19] to access the upper 16 vector registers. P

[20] encodes various features that vary in different classes of instructions and can affect the meaning of the vector length / rounding control specifier field (P[22:21]). P

[23] indicates support for merge-write masking (e.g., when set to 0) or support for zeroing and merge-write masking (e.g., when set to 1).

[0295] An example of the encoding of registers in instructions using the third prefix 1801 (C) is detailed in the table below.

[0296]

[0297] Table 1: 32-register support in 64-bit mode

[0298]

[0299]

[0300] Table 2: Encoding register specifiers in 32-bit mode

[0301] Table 3: Operation Mask Register Specifier Encoding

[0302] Graphics Execution Unit

[0303] Figure 24- Fig. 24B FIG24 illustrates thread execution logic 2400 including an array of processing elements employed in a graphics processor core according to examples described herein. Fig. 24B Elements having the same reference numeral (or name) as elements in any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto. Fig.24A represents an execution unit within a general-purpose graphics processor, and Fig. 24B It represents the execution unit that can be used in the computing accelerator.

[0304] like Fig.24A As shown in, in some examples, the thread execution logic 2400 includes a shader processor 2402, a thread dispatcher 2404, an instruction cache 2406, a scalable execution unit array including a plurality of execution units 2408A-2408N, a sampler 2410, a shared local memory 2411, a data cache 2412, and a data port 2414. In some examples, the scalable execution unit array can be dynamically scaled by enabling or disabling one or more execution units (e.g., any one of execution units 2408A, 2408B, 2408C, 2408D to 2408N-1 and 2408N) based on the computational requirements of the workload. In some examples, the included components are interconnected via an interconnect structure linked to each component. In some examples, the thread execution logic 2400 includes one or more connections to a memory (e.g., a system memory or a cache memory) through one or more of the instruction cache 2406, the data port 2414, the sampler 2410, and the execution units 2408A-2408N. In some examples, each execution unit (e.g., 2408A) is an independently programmable general-purpose computing unit that is capable of executing multiple simultaneous hardware threads while processing multiple data elements in parallel for each thread. In various examples, the array of execution units 2408A-2408N is scalable to include any number of individual execution units.

[0305] In some examples, execution units 2408A-2408N are primarily used to execute shader programs. Shader processor 2402 can process various shader programs and dispatch execution threads associated with the shader programs via thread dispatcher 2404. In some examples, the thread dispatcher includes logic that arbitrates thread initiation requests from graphics and media pipelines and instantiates the requested threads on one or more execution units in execution units 2408A-2408N. For example, a geometry pipeline can dispatch vertex, subdivision, or geometry shaders to thread execution logic for processing. In some examples, thread dispatcher 2404 can also handle runtime thread derivation requests from executing shader programs.

[0306] In some examples, the instruction set supported by execution units 2408A-2408N includes native support for many standard 3D graphics shader instructions, so that when executing shader programs from graphics libraries (e.g., Direct 3D and OpenGL), only minimal translation is required. The execution unit supports vertex and geometry processing (e.g., vertex programs, geometry programs, vertex shaders), pixel processing (e.g., pixel shaders, fragment shaders), and general processing (e.g., compute and media shaders). Each of execution units 2408A-2408N is capable of multi-issue single instruction multiple data (SIMD) execution and multi-threaded operations enable efficient execution environments in the face of higher latency memory accesses. Each hardware thread within each execution unit has a dedicated high-bandwidth register stack and associated independent thread state. Execution is multi-issued to the pipeline at each clock, capable of integer, single-precision and double-precision floating-point operations, SIMD branching capabilities, logical operations, transcendental operations, and other miscellaneous operations. While waiting for data from one of the memory or shared functions, the dependency logic within the execution unit 2408A-2408N causes the waiting thread to sleep until the requested data has been returned. While the waiting thread is sleeping, hardware resources can be used to process other threads. For example, during the delay period associated with the vertex shader operation, the execution unit can perform operations for pixel shaders, fragment shaders, or other types of shader programs, including different vertex shaders. Various examples can be applied to use execution by using single instruction multiple threads (SIMT) as an alternative to using SIMD or in addition to using SIMD. References to SIMD cores or operations can also be applied to SIMT or to SIMD combined with SIMT.

[0307] Each of the execution units 2408A-2408N operates on an array of data elements. The number of data elements is the "execution size," or number of lanes of an instruction. An execution lane is a logical execution unit used for data element access, masking, and flow control within an instruction. The number of lanes may be independent of the number of physical arithmetic logic units (ALUs) or floating point units (FPUs) of a particular graphics processor. In some examples, the execution units 2408A-2408N support integer and floating point data types.

[0308] The execution unit instruction set includes SIMD instructions. Various data elements can be stored in registers as compact data types, and the execution unit will process various elements based on the data size of the element. For example, when a 256-bit wide vector is operated, the 256 bits of the vector are stored in registers, and the execution unit operates the vector as four independent 64-bit compact data elements (data elements of four words (QW) size), eight independent 32-bit compact data elements (data elements of double words (DW) size), sixteen independent 16-bit compact data elements (data elements of word (W) size), or thirty-two independent 8-bit data elements (data elements of byte (B) size). However, different vector widths and register sizes are possible.

[0309] In some examples, one or more execution units may be combined into a fused execution unit 2409A-2409N having a thread control logic (2407A-2407N) shared by the fused EUs. Multiple EUs may be fused into EU groups. Each EU in the fused EU group may be configured to execute a separate SIMD hardware thread. The number of EUs in the fused EU group may vary according to the example. In addition, various SIMD widths may be executed for each EU, including but not limited to SIMD8, SIMD16, and SIMD32. Each fused graphics execution unit 2409A-2409N includes at least two execution units. For example, the fused execution unit 2409A includes a first EU 2408A, a second EU 2408B, and a thread control logic 2407A shared by the first EU 2408A and the second EU 2408B. Thread control logic 2407A controls threads executing on fused graphics execution unit 2409A, allowing each EU within fused execution units 2409A-2409N to execute using a common instruction pointer register.

[0310] One or more internal instruction caches (e.g., 2406) are included in the thread execution logic 2400 to cache thread instructions for the execution unit. In some examples, one or more data caches (e.g., 2412) are included to cache thread data during thread execution. Threads executed on the execution logic 2400 can also store explicitly managed data in the shared local memory 2411. In some examples, a sampler 2410 is included to provide texture sampling for 3D operations and media sampling for media operations. In some examples, the sampler 2410 includes a specialized texture or media sampling function to process texture or media data during the sampling process and then provide the sampled data to the execution unit.

[0311] During execution, the graphics and media pipeline sends thread initiation requests to the thread execution logic 2400 via thread derivation and dispatch logic. Once a set of geometric objects has been processed and rasterized into pixel data, the pixel processor logic (e.g., pixel shader logic, fragment shader logic, etc.) within the shader processor 2402 is called to further calculate output information and cause the results to be written to the output surface (e.g., color buffer, depth buffer, stencil buffer, etc.). In some examples, the pixel shader or fragment shader calculates the values ​​of various vertex attributes to be interpolated on the rasterized object. In some examples, the pixel processor logic within the shader processor 2402 then executes the pixel or fragment shader program provided by the application programming interface (API). To execute the shader program, the shader processor 2402 dispatches the thread to an execution unit (e.g., 2408A) via the thread dispatcher 2404. In some examples, the shader processor 2402 uses the texture sampling logic in the sampler 2410 to access texture data in a texture atlas stored in memory. Arithmetic operations on texture data and input geometry data compute pixel color data for each geometry fragment or discard one or more pixels from further processing.

[0312] In some examples, data port 2414 provides a memory access mechanism for thread execution logic 2400 to output processed data to memory for further processing on a graphics processor output pipeline. In some examples, data port 2414 includes or is coupled to one or more cache memories (e.g., data cache 2412) to cache data for memory access via the data port.

[0313] In some examples, execution logic 2400 may also include a ray tracer 2405, which may provide ray tracing acceleration functionality. Ray tracer 2405 may support a ray tracing instruction set that includes instructions / functions for ray generation.

[0314] Fig. 24B 2426, and a set of dedicated integer SIMD ALUs 2435. The GRF 2424 and ARF 2426 include a set of general register files and architectural register files associated with each simultaneous hardware thread that may be active in the graphics execution unit 2408. In some examples, the per-thread architectural state is maintained in the ARF 2426, while data used during thread execution is stored in the GRF 2424. The execution state of each thread, including the instruction pointer of each thread, can be saved in thread-specific registers in the ARF 2426.

[0315] In some examples, the graphics execution unit 2408 has an architecture that is a combination of Simultaneous Multi-Threading (SMT) and fine-grained Interleaved Multi-Threading (IMT). The architecture has a modular configuration that can be fine-tuned at design time based on the target number of simultaneous threads and the number of registers per execution unit, where the execution unit resources are divided between the logic for executing multiple simultaneous threads. The number of logical threads that the graphics execution unit 2408 can execute is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread.

[0316] In some examples, the graphics execution unit 2408 may issue multiple instructions together, each of which may be a different instruction. The thread arbiter 2408 of the graphics execution unit thread 2408 may dispatch the instruction to the send unit 2430, the branch unit 2432, or one of (one or more) SIMD FPUs 2434 for execution. Each execution thread may access 128 general registers within the GRF 2424, each of which may store 32 bytes, which may be accessed as a SIMD 8 element vector of 32-bit data elements. In some examples, each execution unit thread is capable of accessing 4K bytes within the GRF 2424, but the example is not limited thereto, and more or less register resources may be provided in other examples. In some examples, the graphics execution unit 2408 is partitioned into seven hardware threads, which may independently perform computing operations, but the number of threads per execution unit may also vary according to the example. For example, in some examples, up to 16 hardware threads are supported. In an example where seven threads may access 4K bytes, the GRF 2424 may store a total of 28K bytes. With 16 threads accessing 4K bytes, a total of 64K bytes can be stored in the GRF 2424. Flexible addressing modes allow registers to be addressed together, effectively building wider registers or representing strided rectangular block data structures.

[0317] In some examples, memory operations, sampler operations, and other longer latency system communications are dispatched via "send" instructions, which are executed by the message passing send unit 2430. In some examples, branch instructions are dispatched to a dedicated branch unit 2432 to facilitate SIMD divergence and eventual convergence.

[0318] In some examples, the graphics execution unit 2408 includes one or more SIMD floating point units (FPUs) 2434 to perform floating point operations. In some examples, (one or more) FPUs 2434 also support integer calculations. In some examples, (one or more) FPUs 2434 can SIMD perform up to M 32-bit floating point (or integer) operations, or SIMD perform up to 2M 16-bit integer or 16-bit floating point operations. In some examples, at least one of (one or more) FPUs provides extended math capabilities to support high throughput transcendental math functions and double-precision 64-bit floating point. In some examples, there is also a set of 8-bit integer SIMD ALUs 2435, and these ALUs can be specifically optimized to perform operations associated with machine learning calculations.

[0319] In some examples, an array of multiple instances of graphics execution unit 2408 can be instantiated in graphics sub-core groupings (e.g., sub-slices). To achieve scalability, product architects can select the exact number of execution units for each sub-core grouping. In some examples, execution unit 2408 can execute instructions across multiple execution lanes. In another example, each thread executed on graphics execution unit 2408 is executed on a different lane.

[0320] Fig.25 An additional execution unit 2500 according to one example is illustrated. In some examples, the execution unit 2500 includes a thread control unit 2501, a thread state unit 2502, an instruction fetch / prefetch unit 2503, and an instruction decode unit 2504. The execution unit 2500 also includes a register file 2506 that stores registers that can be assigned to hardware threads within the execution unit. The execution unit 2500 also includes a dispatch unit 2507 and a branch unit 2508. In some examples, the operations of the dispatch unit 2507 and the branch unit 2508 can be similar to Fig. 24B The sending unit 2430 and the branching unit 2432 of the graphics execution unit 2408.

[0321] The execution unit 2500 also includes a computing unit 2510, which includes a plurality of different types of functional units. In some examples, the computing unit 2510 includes an ALU unit 2511, which includes an array of arithmetic logic units. The ALU unit 2511 can be configured to perform 64-bit, 32-bit, and 16-bit integer and floating-point operations. Integer and floating-point operations can be performed simultaneously. The computing unit 2510 may also include a systolic array 2512, and a math unit 2513. The systolic array 2512 includes a network of data processing units with a width of W and a depth of D, which can be used to perform vector or other data parallel operations in a systolic manner. In some examples, the systolic array 2512 can be configured to perform matrix operations, such as matrix dot product operations. In some examples, the systolic array 2512 supports 16-bit floating-point operations, as well as 8-bit and 4-bit integer operations. In some examples, the systolic array 2512 can be configured to accelerate machine learning operations. In such an example, systolic array 2512 may be configured to support the bfloat 16-bit floating point format. In some examples, math unit 2513 may be included to perform a specific subset of math operations in a more efficient, lower power manner than ALU unit 2511. Math unit 2513 may include a variation of math logic that may be found in the shared functional logic of the graphics processing engine provided in other examples (e.g., Figure 4In some examples, math unit 2513 may be configured to perform 32-bit and 64-bit floating point operations.

[0322] The thread control unit 2501 includes logic for controlling the execution of threads within the execution unit. The thread control unit 2501 may include thread arbitration logic to start, stop and preempt the execution of threads within the execution unit 2500. The thread state unit 2502 may be used to store the thread state of threads assigned to be executed on the execution unit 2500. Storing thread states within the execution unit 2500 enables the threads to be quickly preempted when they become blocked or idle. The instruction acquisition / prefetch unit 2503 may obtain instructions from the instruction cache of the higher-level execution logic (e.g., the instruction cache 2406 shown in Figure 24). The instruction acquisition / prefetch unit 2503 may also issue a prefetch request to load instructions into the instruction cache based on the analysis of the currently executed thread. The instruction decoding unit 2504 may be used to decode instructions to be executed by the computing unit. In some examples, the instruction decoding unit 2504 may be used as a secondary decoder to decode complex instructions into component micro-operations.

[0323] Execution unit 2500 also includes a register file 2506 that can be used by hardware threads executing on execution unit 2500. Registers in register file 2506 can be divided among logic used to execute multiple simultaneous threads within compute unit 2510 of execution unit 2500. The number of logical threads that can be executed by graphics execution unit 2500 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread. The size of register file 2506 can vary between examples based on the number of hardware threads supported. In some examples, register renaming can be used to dynamically allocate registers to hardware threads.

[0324] Fig.26 2600 according to some examples. In one or more examples, the graphics processor execution unit supports an instruction set having instructions in multiple formats. Solid line boxes illustrate components that are generally included in execution unit instructions, while dashed lines include components that are optional or included only in a subset of instructions. In some examples, the instruction format 2600 described and illustrated are macroinstructions because they are instructions provided to the execution unit, rather than micro-operations generated from instruction decoding once the instructions are processed.

[0325] In some examples, the graphics processor execution unit natively supports instructions in 128-bit instruction format 2610. Based on the selected instruction, instruction options, and the number of operands, a 64-bit compact instruction format 2630 can be used for some instructions. The native 128-bit instruction format 2610 provides access to all instruction options, while some options and operations are restricted in the 64-bit format 2630. The native instructions available in the 64-bit format 2630 vary from example to example. In some examples, a portion of the instruction is compressed using a set of index values ​​in the index field 2613. The execution unit hardware references a set of compression tables based on the index values, and uses the compression table output to reconstruct the native instructions of the 128-bit instruction format 2610. Instructions of other sizes and formats may also be used.

[0326] For each format, the instruction opcode 2612 defines the operation to be performed by the execution unit. The execution unit executes each instruction in parallel on multiple data elements of each operand. For example, in response to an addition instruction, the execution unit performs simultaneous addition operations on each color channel representing a texture element or a picture element. By default, the execution unit executes each instruction on all data channels of the operand. In some examples, the instruction control field 2614 enables control of certain execution options, such as channel selection (e.g., predicate) and data channel order (e.g., swizzle). For instructions 2610 in a 128-bit instruction format, an execution size field 2616 limits the number of data channels that will be executed in parallel. In some examples, the execution size field 2616 is not available for a 64-bit compressed instruction format 2630.

[0327] Some execution unit instructions have up to three operands, including two source operands src0 2620, src1 2622, and one destination 2618. In some examples, the execution unit supports dual destination instructions, where one destination is implicit. Data manipulation instructions may have a third source operand (e.g., SRC2 2624), where the instruction opcode 2612 determines the number of source operands. The last source operand of an instruction may be an immediate value (e.g., hard-coded) passed with the instruction.

[0328] In some examples, the 128-bit instruction format 2610 includes an access / address mode field 2626 that specifies, for example, whether direct register addressing mode or indirect register addressing mode is used. When direct register addressing mode is used, the register address of one or more operands is provided directly by bits in the instruction.

[0329] In some examples, the 128-bit instruction format 2610 includes an access / address mode field 2626 that specifies the address mode and / or access mode of the instruction. In some examples, the access mode is used to define the data access alignment of the instruction. Some examples support access modes including 16-byte aligned access modes and 1-byte aligned access modes, where the byte alignment of the access mode determines the access alignment of the instruction operand. For example, in a first mode, the instruction can use byte-aligned addressing for the source operand and the destination operand, and in a second mode, the instruction can use 16-byte aligned addressing for all source operands and the destination operand.

[0330] In some examples, the address mode portion of the access / address mode field 2626 determines whether the instruction uses direct addressing or indirect addressing. When using direct register addressing mode, the bits in the instruction directly provide the register address of one or more operands. When using indirect register addressing mode, the register address of one or more operands can be calculated based on the address register value and the address immediate field in the instruction.

[0331] In some examples, instructions are grouped based on the opcode 2612 bit field to simplify opcode decoding 2640. For 8-bit opcodes, bits 4, 5, and 6 allow the execution unit to determine the type of opcode. The precise opcode grouping shown is just an example. In some examples, the move and logic opcode group 2642 includes data movement and logic instructions (e.g., move (mov), compare (cmp)). In some examples, the move and logic group 2642 shares five most significant bits (mostsignificant bit, MSB), wherein the form of the move (mov) instruction is 0000xxxxb, and the form of the logic instruction is 0001xxxxb. The flow control instruction group 2644 (e.g., call, jump (jmp)) includes instructions in the form of 0010xxxxb (e.g., 0x20). The miscellaneous instruction group 2646 includes a mixture of instructions, including synchronization instructions (e.g., wait, send) in the form of 0011xxxxb (e.g., 0x30). The parallel math instruction group 2648 includes component-wise arithmetic instructions (e.g., addition, multiplication (mul)) of the form 0100xxxxb (e.g., 0x40). The parallel math group 2648 performs arithmetic operations in parallel across data channels. The vector math group 2650 includes arithmetic instructions (e.g., dp4) of the form 0101xxxxb (e.g., 0x50). The vector math group performs arithmetic operations, such as dot product calculations on vector operands. In some examples, the illustrated opcode decode 2640 may be used to determine which portion of the execution unit will be used to execute the decoded instruction. For example, some instructions may be designated as systolic instructions to be executed by a systolic array. Other instructions, such as ray tracing instructions (not shown), may be routed to a ray tracing core or ray tracing logic within a slice or partition of the execution logic.

[0332] Graphics Pipeline

[0333] Fig. 27 is a block diagram of another example of a graphics processor 2700 . Fig. 27 Elements having the same reference numeral (or name) as elements in any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

[0334] In some examples, graphics processor 2700 includes geometry pipeline 2720, media pipeline 2730, display engine 2740, thread execution logic 2750, and rendering output pipeline 2770. In some examples, graphics processor 2700 is a graphics processor within a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown), or by commands issued to graphics processor 2700 via ring interconnect 2702. In some examples, ring interconnect 2702 couples graphics processor 2700 to other processing components, such as other graphics processors or general-purpose processors. Commands from ring interconnect 2702 are interpreted by command stream processor 2703, which provides instructions to individual components of geometry pipeline 2720 or media pipeline 2730.

[0335] In some examples, command stream processor 2703 directs the operation of vertex fetcher 2705, which reads vertex data from memory and executes vertex processing commands provided by command stream processor 2703. In some examples, vertex fetcher 2705 provides vertex data to vertex shader 2707, which performs coordinate space transformation and lighting operations on each vertex. In some examples, vertex fetcher 2705 and vertex shader 2707 execute vertex processing instructions by dispatching execution threads to execution units 2752A-2752B via thread dispatcher 2731.

[0336] In some examples, execution units 2752A-2752B are arrays of vector processors with instruction sets for performing graphics and media operations. In some examples, execution units 2752A-2752B have an attached L1 cache 2751, which is either per array or shared between arrays. The cache can be configured as a data cache, an instruction cache, or a single cache that is partitioned to contain data and instructions in different partitions.

[0337] In some examples, the geometry pipeline 2720 includes a tessellation component to perform hardware accelerated tessellation of 3D objects. In some examples, the programmable hull shader 2711 configures the tessellation operation. The programmable domain shader 2717 provides back-end evaluation of the tessellation output. The tessellation 2713 operates under the direction of the hull shader 2711 and contains special purpose logic to generate a set of detailed geometric objects based on the coarse geometric model provided as input to the geometry pipeline 2720. In some examples, if tessellation is not used, the tessellation components (e.g., the hull shader 2711, the tessellation 2713, and the domain shader 2717) can be bypassed.

[0338] In some examples, the complete geometric object may be processed by the geometry shader 2719 via one or more threads dispatched to the execution units 2752A-2752B, or may go directly to the clipper 2729. In some examples, the geometry shader operates on the entire geometric object, rather than on vertices or vertex patches as in previous stages of the graphics pipeline. If tessellation is disabled, the geometry shader 2719 receives input from the vertex shader 2707. In some examples, the geometry shader 2719 may be programmed by a geometry shader program to perform geometry tessellation when the tessellation unit is disabled.

[0339] Before rasterization, the clipper 2729 processes the vertex data. The clipper 2729 can be a fixed function clipper, or can be a programmable clipper with clipping and geometry shader functions. In some examples, the rasterizer and depth test component 2773 in the render output pipeline 2770 dispatches a pixel shader to convert the geometric object into a representation of each pixel. In some examples, the pixel shader logic is included in the thread execution logic 2750. In some examples, the application can bypass the rasterizer and depth test component 2773 and access the unrasterized vertex data via the stream output unit 2723.

[0340] Graphics processor 2700 has an interconnect bus, interconnect structure or some other interconnect mechanism that allows data and messages to be passed between the main components of the processor. In some examples, execution units 2752A-2752B and associated logic units (e.g., L1 cache 2751, sampler 2754, texture cache 2758, etc.) are interconnected via data port 2756 to perform memory access and communicate with the rendering output pipeline components of the processor. In some examples, sampler 2754, cache 2751, 2758 and execution units 2752A-2752B each have a separate memory access path. In some examples, texture cache 2758 may also be configured as a sampler cache.

[0341] In some examples, the rendering output pipeline 2770 includes a rasterizer and a depth test component 2773, which converts vertex-based objects into associated pixel-based representations. In some examples, the rasterizer logic includes a windower / masker unit to perform fixed-function triangle and linear rasterization. In some examples, associated rendering buffers 2778 and depth buffers 2779 are also available. Pixel operation components 2777 perform pixel-based operations on data, but in some cases, pixel operations associated with 2D operations (e.g., bit block image transfer with blending) are performed by the 2D engine 2741, or replaced by a display controller 2743 using an overlay display plane when displayed. In some examples, a shared L3 cache 2775 is available for all graphics components, thereby allowing data to be shared without using main system memory.

[0342] In some examples, the graphics processor media pipeline 2730 includes a media engine 2737 and a video front end 2734. In some examples, the video front end 2734 receives pipeline commands from the command stream processor 2703. In some examples, the media pipeline 2730 includes a separate command stream processor. In some examples, the video front end 2734 processes the media commands before sending the commands to the media engine 2737. In some examples, the media engine 2737 includes a thread spawning function to spawn a thread for dispatching to the thread execution logic 2750 via the thread dispatcher 2731.

[0343] In some examples, the graphics processor 2700 includes a display engine 2740. In some examples, the display engine 2740 is external to the processor 2700 and is coupled to the graphics processor via a ring interconnect 2702 or some other interconnect bus or structure. In some examples, the display engine 2740 includes a 2D engine 2741 and a display controller 2743. In some examples, the display engine 2740 contains dedicated logic that can operate independently of the 3D pipeline. In some examples, the display controller 2743 is coupled to a display device (not shown), which can be a system-integrated display device, such as in a laptop computer, or can be an external display device attached via a display device connector.

[0344] In some examples, the geometry pipeline 2720 and the media pipeline 2730 may be configured to perform operations based on multiple graphics and media programming interfaces, rather than specifically for any one application programming interface (API). In some examples, driver software for a graphics processor translates API calls specific to a particular graphics or media library into commands that can be processed by the graphics processor. In some examples, support is provided for the Open Graphics Library (OpenGL), the Open Computing Language (OpenCL), and / or the Vulkan graphics and computing APIs, all of which are from the Khronos Group. In some examples, support may also be provided for the Direct3D library from Microsoft. In some examples, a combination of these libraries may be supported. Support may also be provided for the Open Source Computer Vision Library (OpenCV). If it is possible to map from the pipeline of a future API to the pipeline of a graphics processor, future APIs with compatible 3D pipelines will also be supported.

[0345] Graphics pipeline programming

[0346] Fig.28A ' is a block diagram illustrating a graphics processor command format 2800 according to some examples. Fig.28B The block diagram of 2810 illustrates a graphics processor command sequence 2810 according to one example. Fig.28A The solid-line boxes in the figure illustrate components that are generally included in the graphics commands, while the dashed lines include components that are optional or included only in a subset of the graphics commands. Fig.28A An exemplary graphics processor command format 2800 includes a data field for identifying a client 2802, a command operation code (opcode) 2804, and command data 2806. In some commands, a sub-opcode 2805 and a command size 2808 are also included.

[0347] In some examples, client 2802 specifies a client unit of a graphics device that processes command data. In some examples, a graphics processor command parser checks the client field of each command to determine the conditions for further processing of the command and routes the command data to the appropriate client unit. In some examples, a graphics processor client unit includes a memory interface unit, a rendering unit, a 2D unit, a 3D unit, and a media unit. Each client unit has a corresponding processing pipeline for processing commands. Once a command is received by a client unit, the client unit reads an opcode 2804 and, if present, a subopcode 2805 to determine the operation to be performed. The client unit executes the command using the information in data field 2806. For some commands, it is expected that there is a clear command size 2808 to specify the size of the command. In some examples, the command parser automatically determines the size of at least some commands based on the command opcode. In some examples, commands are aligned via multiples of double words. Other command formats can be used.

[0348] Fig.28B The flowchart in 28 illustrates an exemplary graphics processor command sequence 2810. In some examples, software or firmware of a data processing system featuring an example of a graphics processor uses a version of the command sequence shown to set up, execute, and terminate a set of graphics operations. The sample command sequence is shown and described for example purposes only, as the examples are not limited to these particular commands or this command sequence. In addition, commands can be issued as batch commands in a command sequence so that the graphics processor will process the sequence of commands at least partially concurrently.

[0349] In some examples, graphics processor command sequence 2810 may begin with pipeline flush command 2812 to cause any active graphics pipeline to complete the currently pending commands for that pipeline. In some examples, 3D pipeline 2822 and media pipeline 2824 do not operate concurrently. Pipeline flush is performed to cause active graphics pipelines to complete any pending commands. In response to pipeline flush, the command parser of the graphics processor will suspend command processing until the active graphics engine completes pending operations and the associated read cache is invalidated. Optionally, any data in the rendering cache that is marked as "dirty" can be flushed to memory. In some examples, pipeline flush command 2812 can be used for pipeline synchronization or before placing the graphics processor in a low power state.

[0350] In some examples, when a command sequence requires the graphics processor to explicitly switch between pipelines, pipeline select command 2813 is used. In some examples, unless the context is to issue commands for both pipelines, pipeline select command 2813 is only required once within the execution context before issuing pipeline commands. In some examples, pipeline flush command 2812 is required immediately before a pipeline switch is made via pipeline select command 2813.

[0351] In some examples, pipeline control commands 2814 configure the operation of the graphics pipeline and are used to program 3D pipeline 2822 and media pipeline 2824. In some examples, pipeline control commands 2814 configure pipeline states for active pipelines. In some examples, pipeline control commands 2814 are used for pipeline synchronization and to clear data from one or more cache memories within an active pipeline before processing a batch of commands.

[0352] In some examples, return buffer state commands 2816 are used to configure a set of return buffers for each pipeline to write data. Some pipeline operations require allocating, selecting, or configuring one or more return buffers into which these operations write intermediate data during processing. In some examples, the graphics processor also uses one or more return buffers to store output data and perform cross-thread communication. In some examples, return buffer state 2816 includes selecting the size and number of return buffers for a set of pipeline operations.

[0353] The remaining commands in the command sequence differ based on the active pipeline of the operation.Based on the pipeline determination 2820 , the command sequence is tailored for either the 3D pipeline 2822 starting from the 3D pipeline state 2830 or the media pipeline 2824 starting from the media pipeline state 2840 .

[0354] The commands to configure 3D pipeline state 2830 include 3D state setup commands for vertex buffer state, vertex element state, constant color state, depth buffer state, and other state variables to be configured before 3D primitive commands are processed. The values ​​of these commands are determined at least in part based on the specific 3D API in use. In some examples, 3D pipeline state 2830 commands can also selectively disable or bypass certain pipeline elements if they are not to be used.

[0355] In some examples, the 3D primitive 2832 command is used to submit a 3D primitive to be processed by the 3D pipeline. The command and associated parameters passed to the graphics processor via the 3D primitive 2832 command are forwarded to the vertex acquisition function in the graphics pipeline. The vertex acquisition function uses the 3D primitive 2832 command data to generate a vertex data structure. The vertex data structure is stored in one or more return buffers. In some examples, the 3D primitive 2832 command is used to perform vertex operations on the 3D primitive via a vertex shader. In order to process the vertex shader, the 3D pipeline 2822 dispatches the shader execution thread to the graphics processor execution unit.

[0356] In some examples, the 3D pipeline 2822 is triggered via an execute 2834 command or event. In some examples, a register write triggers the command execution. In some examples, the execution is triggered via a "go" or "kick" command in a command sequence. In some examples, the command execution is triggered using a pipeline synchronization command to flush the command sequence through the graphics pipeline. The 3D pipeline will perform geometry processing on the 3D primitives. Once the operation is completed, the resulting geometric objects are rasterized and the pixel engine colors the resulting pixels. For these operations, additional commands that control pixel shading and pixel backend operations may also be included.

[0357] In some examples, the graphics processor command sequence 2810 follows the media pipeline 2824 path when performing media operations. In general, the specific use and programming of the media pipeline 2824 depends on the media or computing operations to be performed. Specific media decoding operations can be offloaded to the media pipeline during media decoding. In some examples, the media pipeline can also be bypassed, and the media decoding can be performed in whole or in part using resources provided by one or more general-purpose processing cores. In some examples, the media pipeline also includes elements for general-purpose graphics processor unit (GPGPU) operations, where the graphics processor is used to perform SIMD vector operations using compute shader programs that are not explicitly related to the rendering of graphics primitives.

[0358] In some examples, the media pipeline 2824 is configured in a manner similar to the 3D pipeline 2822. A set of commands to configure the media pipeline state 2840 are dispatched or placed into a command queue before the media object commands 2842. In some examples, the commands for the media pipeline state 2840 include data used to configure the media pipeline elements that will be used to process the media objects. This includes data used to configure the video decoding and video encoding logic within the media pipeline, such as the encoding or decoding format. In some examples, the commands for the media pipeline state 2840 also support the use of one or more pointers to "indirect" state elements, which contain a batch of state settings.

[0359] In some examples, media object command 2842 provides a pointer to a media object for processing by the media pipeline. The media object includes a memory buffer containing video data to be processed. In some examples, all media pipeline states must be valid before issuing media object command 2842. Once the pipeline state is configured and media object command 2842 is queued, media pipeline 2824 is triggered via execute command 2844 or an equivalent execution event (e.g., a register write). The output from media pipeline 2824 can then be post-processed by operations provided by 3D pipeline 2822 or media pipeline 2824. In some examples, GPGPU operations are configured and executed similarly to media operations.

[0360] Program code can be applied to input information to perform the functions described herein and generate output information. Output information can be applied to one or more output devices in a known manner. For the present application, a processing system includes any system having a processor, such as a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a microprocessor, or any combination thereof.

[0361] Program code can be implemented in a process-oriented or object-oriented high-level programming language to communicate with the processing system. If desired, program code can also be implemented in assembly or machine language. In fact, the mechanism described herein is not limited to any specific programming language in scope. In any case, the language can be a compiled language or an interpreted language.

[0362] Examples of the mechanisms disclosed herein can be implemented in hardware, software, firmware, or a combination of these implementation approaches. Examples can be implemented as computer programs or program codes, executed on a programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0363] These machine-readable storage media may include, but are not limited to, non-transitory tangible arrangements of items manufactured or formed by a machine or device, including storage media such as: hard disks, any other type of disk (including floppy disks, optical disks, compact disk read-only memory (CD-ROM), compact disk rewritable (CD-RW), and magneto-optical disks), semiconductor devices (e.g., read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM), phase change memory (PCM)), magnetic cards or optical cards, or any other type of medium suitable for storing electronic instructions.

[0364] Therefore, examples also include non-transitory tangible machine-readable media containing instructions or design data, such as hardware description languages ​​(HDL), that define the structures, circuits, devices, processors and / or system features described herein. Such examples may also be referred to as program products.

[0365] Emulation (including binary translation, code deformation, etc.).

[0366] In some cases, the instruction converter can be used to convert instructions from a source instruction set architecture to a target instruction set architecture. For example, the instruction converter can translate (e.g., using static binary translation, dynamic binary translation including dynamic compilation), deform, simulate or otherwise convert instructions to one or more other instructions to be processed by the core. The instruction converter can be implemented with software, hardware, firmware or a combination thereof. The instruction converter can be on the processor, outside the processor, or partly on the processor and partly outside the processor.

[0367] Fig.29The block diagram of illustrates the use of a software instruction converter according to an example, which is used to convert binary instructions in a source ISA into binary instructions in a target ISA. In the illustrated example, the instruction converter is a software instruction converter, but alternatively, the instruction converter can be implemented in software, firmware, hardware, or various combinations thereof. Fig.29 It is shown that a program in a high-level language 2902 can be compiled using a first ISA compiler 2904 to generate first ISA binary code 2906 that can be natively executed by a processor having at least one first ISA core 2916. The processor having at least one first ISA core 2916 represents any processor that is capable of executing or otherwise processing (1) a substantial portion of the first ISA or (2) a program that is natively executed by a processor having at least one first ISA core. The first ISA compiler 2904 represents a compiler operable to generate first ISA binary code 2906 (e.g., object code) that can be executed on a processor 2916 having at least one first ISA core with or without additional linking processing. Similarly, Fig.29 It is shown that a program in a high-level language 2902 can be compiled using an alternative ISA compiler 2908 to generate an alternative ISA binary code 2910, which can be executed natively by a processor 2914 without a first ISA core. An instruction converter 2912 is used to convert the first ISA binary code 2906 into code that can be executed natively by a processor 2914 without a first ISA core. This converted code will not necessarily be the same as the alternative ISA binary code 2910; however, the converted code will implement the overall operation and be composed of instructions from the alternative ISA. Thus, the instruction converter 2912 represents software, firmware, hardware, or a combination thereof that allows a processor or other electronic device that does not have a first ISA processor or core to execute the first ISA binary code 2906 through simulation, emulation, or any other process.

[0368] IP Core Implementation

[0369] One or more aspects of at least some examples may be implemented by representative code stored on a machine-readable medium, which represents and / or defines logic within an integrated circuit such as a processor. For example, a machine-readable medium may include instructions representing various logics within a processor. When read by a machine, the instructions may cause the machine to make logic to perform the techniques described herein. This form of expression, known as an "IP core," is a reusable unit of logic for an integrated circuit that may be stored on a tangible machine-readable medium as a hardware model describing the structure of the integrated circuit. The hardware model may be provided to various customers or manufacturing facilities, which load the hardware model on a manufacturing machine that manufactures the integrated circuit. An integrated circuit may be made so that the circuit performs the operations described in association with any of the examples described herein.

[0370] Fig.30 The block diagram of illustrates an IP core development system 3000 according to some examples, which can be used to manufacture integrated circuits to perform operations. The IP core development system 3000 can be used to generate modular, reusable designs that can be included in larger designs or used to construct entire integrated circuits (e.g., SOC integrated circuits). The design facility 3030 can generate software simulations 3010 of the IP core design in a high-level programming language (e.g., C / C++). The software simulation 3010 can be used to design, test, and verify the behavior of the IP core using a simulation model 3012. The simulation model 3012 can include functional, behavioral, and / or timing simulations. A register transfer level (RTL) design 3015 can then be created or synthesized from the simulation model 3012. The RTL design 3015 is an abstraction of the behavior of an integrated circuit that models the flow of digital signals between hardware registers, including associated logic executed using the modeled digital signals. In addition to the RTL design 3015, a lower level design at the logic level or transistor level can also be created, designed, or synthesized. Thus, the specific details of the initial design and simulation can vary.

[0371] The RTL design 3015 or equivalent may also be synthesized by the design facility into a hardware model 3020, which may take the form of a hardware description language (HDL) or some other representation of physical design data. The HDL may be further simulated or tested to verify the IP core design. The IP core design may be stored to be delivered to a third party fabrication facility 3065 using a non-volatile memory 2240 (e.g., a hard disk, a flash memory, or any other non-volatile storage medium). Alternatively, the IP core design may be transmitted via a wired connection 3050 or a wireless connection 3060 (e.g., via the Internet). The fabrication facility 3065 may then fabricate an integrated circuit based at least in part on the IP core design. The fabricated integrated circuit may be configured to perform operations according to at least some of the examples described herein.

[0372] References to "some examples," "an example," and the like indicate that the examples being described may include a particular feature, structure, or characteristic, but that not every example may include the particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same example. In addition, when a particular feature, structure, or characteristic is described in conjunction with one example, it should be considered that it is within the knowledge of those skilled in the art to implement such feature, structure, or characteristic in conjunction with other examples (whether or not explicitly described).

[0373] 1. A device comprising:

[0374] a function identifier controller for handling function definition instructions, wherein a function is a sequence of instructions including at least one direct memory access operation or collective operation;

[0375] Instruction memory, used to store functions;

[0376] an execution engine, configured to read functions from the instruction memory; and

[0377] A plurality of execution resources are used to execute the read function, wherein the plurality of execution resources at least include a direct memory access (DMA) engine, an atomic engine, and a collective engine.

[0378] 2. The apparatus of Example 1, wherein the first instruction in the function definition instructions is an instruction to reserve space for a function in the instruction memory.

[0379] 3. The apparatus of Example 2, wherein the second instruction in the function definition instruction is an instruction to write one of a DMA instruction, an atomic instruction, or a collective instruction to the function.

[0380] 4. The apparatus as described in Example 3, wherein the third instruction in the function definition instruction is an instruction to close an opened function.

[0381] 5. The apparatus of Example 4, wherein the fourth instruction in the function definition instruction is an instruction to delete a function.

[0382] 6. The apparatus of any one of Examples 1-5, further comprising:

[0383] An instance identifier controller is used to handle instructions for configuring the use of a function flow, where a function flow is a plurality of sequential functions.

[0384] 7. The apparatus of Example 6, wherein the first instruction of the instructions for configuring the use of a function stream is an instruction for allocating a function to a stream.

[0385] 8. The apparatus of Example 7, wherein the second instruction in the instructions for configuring the use of the function flow is an instruction for setting parameters for the execution of the flow.

[0386] 9. An apparatus as described in Example 8, wherein a third instruction of the instructions for configuring the use of the function stream is an instruction to queue the stream for execution by the execution resource.

[0387] 10. The apparatus of example 9, wherein a fourth instruction of the instructions for configuring the use of a function stream is an instruction for dequeuing a stream for execution by the execution resource.

[0388] 11. The apparatus of Example 10, wherein the fifth instruction of the instructions for configuring the use of the function flow is an instruction for polling the execution status of the flow.

[0389] 12. The apparatus of example 11, wherein a sixth instruction of the instructions for configuring the use of a function flow is an instruction for stalling the flow.

[0390] 13. A system comprising:

[0391] processor, used to offload instructions to a coprocessor;

[0392] The coprocessor comprises:

[0393] a function identifier controller for handling function definition instructions, wherein a function is a sequence of instructions including at least one direct memory access operation or collective operation;

[0394] Instruction memory, used to store functions;

[0395] an execution engine, configured to read functions from the instruction memory; and

[0396] A plurality of execution resources are used to execute the read function, wherein the plurality of execution resources at least include a direct memory access (DMA) engine, an atomic engine, and a collective engine.

[0397] 14. The system of Example 13, wherein a first instruction of the function definition instructions is an instruction to reserve space in the instruction memory for a function.

[0398] 15. The system of Example 14, wherein the second instruction in the function definition instructions is an instruction to write one of a DMA instruction, an atomic instruction, or a collective instruction to a function.

[0399] 16. The system of Example 15, wherein the third instruction in the function definition instruction is an instruction to close an opened function.

[0400] 17. The system of Example 16, wherein the fourth instruction in the function definition instructions is an instruction to delete a function.

[0401] 18. The system of any of Examples 13-17, further comprising:

[0402] An instance identifier controller is used to handle instructions for configuring the use of a function flow, where a function flow is a plurality of sequential functions.

[0403] 19. The system of any of Examples 13-18, wherein the processor is a RISC-V processor.

[0404] 20. The system of Example 19, wherein the function definition instruction is a RISC-V instruction.

[0405] In addition, in the various examples described above, unless otherwise specifically noted, selection language such as phrases "at least one of A, B, or C" or "A, B and / or C" should be understood to refer to A, B, or C, or any combination thereof (i.e., A and B, A and C, B and C, and A, B and C).

[0406] Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the present disclosure as set forth in the claims.

Claims

1. A device comprising: a function identifier controller for handling function definition instructions, wherein a function is a sequence of instructions including at least one direct memory access operation or collective operation; Instruction memory, used to store functions; an execution engine, configured to read functions from the instruction memory; and A plurality of execution resources are used to execute the read function, wherein the plurality of execution resources at least include a direct memory access (DMA) engine, an atomic engine, and a collective engine.

2. The device according to claim 1, wherein: The first instruction in the function definition instructions is an instruction for reserving space for a function in the instruction memory.

3. The device as claimed in claim 2, wherein: The second instruction in the function definition instructions is an instruction for writing one of a DMA instruction, an atomic instruction, or a collective instruction to the function.

4. The device as claimed in claim 3, wherein: The third instruction in the function definition instruction is an instruction for closing the opened function.

5. The device according to claim 4, wherein: The fourth instruction in the function definition instructions is an instruction for deleting a function.

6. The device according to any one of claims 1 to 5, further comprising: An instance identifier controller is used to handle instructions for configuring the use of a function flow, where a function flow is a plurality of sequential functions.

7. The device according to claim 6, wherein: The first of the instructions for configuring the use of a function stream is an instruction for allocating a function into a stream.

8. The device according to claim 7, wherein: A second instruction of the instructions for configuring the use of a function flow is an instruction for setting parameters for execution of the flow.

9. The device of claim 8, wherein: A third instruction of the instructions for configuring use of a function stream is an instruction for queuing a stream for execution by the execution resource.

10. The device of claim 9, wherein: A fourth instruction of the instructions for configuring the use of a function stream is an instruction for dequeuing a stream for execution by the execution resource.

11. The device of claim 10, wherein: The fifth instruction in the instructions for configuring the use of a function flow is an instruction for polling the execution status of a flow.

12. The device of claim 11, wherein: A sixth instruction of the instructions for configuring the use of a function stream is an instruction for stalling a stream.

13. A system comprising: processor, used to offload instructions to a coprocessor; The coprocessor comprises: a function identifier controller for handling function definition instructions, wherein a function is a sequence of instructions including at least one direct memory access operation or collective operation; Instruction memory, used to store functions; an execution engine, configured to read functions from the instruction memory; and A plurality of execution resources are used to execute the read function, wherein the plurality of execution resources at least include a direct memory access (DMA) engine, an atomic engine, and a collective engine.

14. The system of claim 13, wherein: The first instruction in the function definition instructions is an instruction for reserving space for a function in the instruction memory.

15. The system of claim 14, wherein: The second instruction in the function definition instructions is an instruction for writing one of a DMA instruction, an atomic instruction, or a collective instruction to the function.

16. The system of claim 15, wherein: The third instruction in the function definition instruction is an instruction for closing the opened function.

17. The system of claim 16, wherein: The fourth instruction in the function definition instructions is an instruction for deleting a function.

18. The system of any one of claims 13 to 17, further comprising: An instance identifier controller is used to handle instructions for configuring the use of a function flow, where a function flow is a plurality of sequential functions.

19. The system of any one of claims 13 to 18, wherein: The processor is a RISC-V processor.

20. The system of claim 19, wherein: The function definition instruction is a RISC-V instruction.

Images