Systems, methods, and apparatus for heterogeneous computing

Abstract

The present invention provides systems, methods, and apparatus for heterogeneous computing. [Solution] The hardware heterogeneous scheduler dispatches instructions for execution on one or more of a plurality of heterogeneous processing elements. The instructions correspond to code fragments processed by one or more of the plurality of heterogeneous processing elements and are native instructions for at least one of the one or more of the plurality of heterogeneous processing elements.

Description

Technical Field

[0001] The present disclosure generally relates to the field of computing devices, and more specifically, to heterogeneous computing methods, devices, and systems.

Background Art

[0002] In current computers, the CPU executes general computing tasks such as executing application software and operating systems. Computing tasks specialized in a certain technical field, for example, graphics and image processing, are processed by graphics processors, image processors, digital signal processors, and fixed-function accelerators. In current heterogeneous machines, each type of processor is programmed in various ways.

[0003] In the era of big data processing, compared with today's general-purpose processors, lower energy and higher performance are required. Accelerators (for example, either a custom fixed-function unit or a custom programmable unit) are helpful to meet these requirements. This field has undergone rapid evolution in both algorithms and workloads, and the set of available accelerators is difficult to predict in advance, highly likely to branch across stock units within a product type, and evolve with the product type.

Brief Description of Drawings

[0004] Embodiments will be readily understood by the following detailed description in conjunction with the accompanying drawings. For ease of explanation, like reference numerals designate like structural elements. The embodiments are shown by way of example and are not intended to be limited to the figures of the accompanying drawings.

[0005] [Figure 1] It represents an execution environment of heterogeneous multiprocessing.

[0006] [Figure 2] This represents the execution environment for heterogeneous multiprocessing.

[0007] [Figure 3] This document presents an exemplary implementation of a heterogeneous scheduler.

[0008] [Figure 4] This document describes an embodiment of system booting and device discovery for a computer system.

[0009] [Figure 5] This example shows thread transitions based on the mapping of program phases to three types of processing elements.

[0010] [Figure 6] This is an exemplary implementation flow executed by a heterogeneous scheduler.

[0011] [Figure 7] This example demonstrates a method for selecting thread destinations using a heterogeneous scheduler.

[0012] [Figure 8] This document illustrates the concept of using striped mapping for logical IDs.

[0013] [Figure 9] This example demonstrates the use of striped mapping for logical IDs.

[0014] [Figure 10] Here is an example of a core group.

[0015] [Figure 11] This document provides an example of how threads can be executed in a system that utilizes a binary translator switching mechanism.

[0016] [Figure 12] An exemplary method for core allocation of hot code to an accelerator is shown.

[0017] [Figure 13] An exemplary method for possible core allocation for wake-up or write to a page directory base register event is shown.

[0018] [Figure 14] An example of a serial phase thread is shown.

[0019] [Figure 15] An exemplary method for potential core allocation for thread response to a sleep command event is shown.

[0020] [Figure 16] An exemplary method for potential core allocation for a thread in response to a phase change event is shown.

[0021] [Figure 17] An example of code describing an acceleration region is shown.

[0022] [Figure 18] An embodiment of a method for execution using ABEGIN in a hardware processor core is shown.

[0023] [Figure 19] An embodiment of a method for execution using AEND in a hardware processor core is shown.

[0024] [Figure 20] A system for providing ABEGIN / AEND equivalence using pattern matching is shown.

[0025] [Figure 21] An embodiment of a method for a non-accelerated description thread exposed to pattern recognition is shown.

[0026] [Figure 22] This document presents an embodiment of a method for non-accelerated description threads subjected to pattern recognition.

[0027] [Figure 23] This document illustrates various types of memory dependencies, their semantics, ordering requirements, and use cases.

[0028] [Figure 24] An example of a memory data block pointed to by the ABEGIN instruction is shown.

[0029] [Figure 25] An example of memory 2503 configured to use ABEGIN / AEND semantics is shown.

[0030] [Figure 26] This section provides examples of different modes of operation for execution using ABEGIN / AEND.

[0031] [Figure 27] This section provides examples of different modes of operation for execution using ABEGIN / AEND.

[0032] [Figure 28] Further details regarding one embodiment are provided below.

[0033] [Figure 29] An embodiment of the accelerator is shown.

[0034] [Figure 30] This describes a computer system including an accelerator and one or more computer processor chips coupled to a processor via a multiprotocol link.

[0035] [Figure 31]The device bias flow according to the embodiment is shown.

[0036] [Figure 32] An exemplary process according to one embodiment is shown.

[0037] [Figure 33] This describes the process when an operand is released from one or more I / O devices.

[0038] [Figure 34] This document describes an example using two different types of work queues.

[0039] [Figure 35] This example shows a data streaming accelerator (DSA) device having multiple work queues that receive descriptors submitted via an I / O fabric interface.

[0040] [Figure 36] Two work queues are shown.

[0041] [Figure 37] Another configuration using engines and grouping is shown.

[0042] [Figure 38] An example of the descriptor is shown.

[0043] [Figure 39] An example of a completion record is shown.

[0044] [Figure 40] Exemplary non-op descriptors and no-op completion records are shown.

[0045] [Figure 41] An exemplary batch descriptor and no-op completion record are shown.

[0046] [Figure 42] An exemplary drain descriptor and drain completion record are shown.

[0047] [Figure 43] An exemplary memory move descriptor and memory move completion record are shown.

[0048] [Figure 44] An example of a fill descriptor is shown.

[0049] [Figure 45] An exemplary comparison descriptor and comparison completion record are shown.

[0050] [Figure 46] An exemplary comparative intermediate descriptor is shown.

[0051] [Figure 47] An example of a creation data record descriptor and a creation difference record completion record are shown.

[0052] [Figure 48] This shows the format for differential recording.

[0053] [Figure 49] An exemplary fitted difference record descriptor is shown.

[0054] [Figure 50] This document presents one example of using the creation difference recording and adaptation difference recording operations.

[0055] [Figure 51] An exemplary memory copy descriptor using dualcast and a memory copy completion record using dualcast are shown.

[0056] [Figure 52] This shows the CRC generation descriptor and CRC generation in the dataset.

[0057] [Figure 53]An example copy using a CRC generation descriptor is shown.

[0058] [Figure 54] An example of a DIF insertion descriptor and a DIF insertion completion record are shown.

[0059] [Figure 55] An exemplary DIF strip descriptor and DIF strip completion record are shown.

[0060] [Figure 56] An exemplary DIF update descriptor and DIF update completion record are shown.

[0061] [Figure 57] An exemplary cache flush descriptor is shown.

[0062] [Figure 58] This shows the 64-byte enqueue storage data generated by ENQCMD.

[0063] [Figure 59] This document describes an embodiment of a method executed by the processor to process the MOVDIRI instruction.

[0064] [Figure 60] This document describes an embodiment of the method executed by the processor to process the MOVDIRI64B instruction.

[0065] [Figure 61] This document describes an embodiment of a method executed by a processor to process the ENCQMD instruction.

[0066] [Figure 62] The format for the ENQCMDS instruction is shown below.

[0067] [Figure 63]This document describes an embodiment of a method executed by a processor to process the ENCQMD instruction.

[0068] [Figure 64] This document describes an embodiment of a method executed by the processor to process the UMONITOR instruction.

[0069] [Figure 65] This document describes an embodiment of a method executed by the processor to process the UMWAIT instruction.

[0070] [Figure 66] This document describes an embodiment of a method executed by the processor to process the TPAUSE instruction.

[0071] [Figure 67] An example of execution using the UMWAIT and UMONITOR instructions is shown.

[0072] [Figure 68] An example of execution using the TPAUSE and UMONITOR instructions is shown.

[0073] [Figure 69] This demonstrates an exemplary implementation in which an accelerator is coupled to multiple cores in a communicative manner via a cache-coherent interface.

[0074] [Figure 70] Another diagram shows the accelerator, which includes a data management unit, multiple processing elements, and high-speed on-chip storage, as well as the other components mentioned above.

[0075] [Figure 71] This section shows an example set of processes performed by the processing element.

[0076] [Figure 72a] To generate the vector y, an example of multiplication between sparse matrices for the vector x is illustrated.

[0077] [Figure 72b] This shows the CSR representation of matrix A, where each value is stored as a (value, row index) pair.

[0078] [Figure 72c] This shows the CSC representation of matrix A using (value, column index) pairs.

[0079] [Figure 73a] The pseudocode for the calculation pattern is shown below. [Figure 73b] The pseudocode for the calculation pattern is shown below. [Figure 73c] The pseudocode for the calculation pattern is shown below.

[0080] [Figure 74] This shows a processing flow for one embodiment of a data management unit and processing elements.

[0081] [Figure 75a] The paths related to the spMspV_csc and scale_update operations are highlighted (using dotted lines).

[0082] [Figure 75b] This shows the path related to the spMdV_csr operation.

[0083] [Figure 76a] An example of representing a graph as an adjacency matrix is ​​shown. [Figure 76b] An example of representing a graph as an adjacency matrix is ​​shown.

[0084] [Figure 76c] The vertex program is shown below.

[0085] [Figure 76d] The following is an example of program code for executing the vertex program.

[0086] [Figure 76e] The GSPMV formulation is shown.

[0087] [Figure 77] The framework is presented.

[0088] [Figure 78] This indicates that customizable logical blocks are provided within each PE.

[0089] [Figure 79] This shows the processing for each accelerator tile.

[0090] [Figure 80a] This summarizes the customizable parameters for one example of the template.

[0091] [Figure 80b] The following are tuning considerations.

[0092] [Figure 81] Here is one of the most common sparse matrix formats.

[0093] [Figure 82] This document outlines the steps involved in an example of sparse matrix-dense vector multiplication using the CRS data format.

[0094] [Figure 83] An example of an accelerator including an accelerator logic die and one or more stacks of DRAM is shown.

[0095] [Figure 84a] This shows one embodiment of an accelerator logic chip, viewed from above, facing the DRAM die stack. [Figure 84b] This shows one embodiment of an accelerator logic chip, viewed from above, facing the DRAM die stack.

[0096] [Figure 85] This provides a general overview of the photo processing and printing (DPE) process.

[0097] [Figure 86] An example of a blocking scheme is shown.

[0098] [Figure 87] This shows a block descriptor.

[0099] [Figure 88] This shows a 2x2 matrix that matches within the buffer of a single dot product engine.

[0100] [Figure 89] This shows one example of hardware within a dot product engine that uses this format.

[0101] [Figure 90] This shows the contents of the match logic unit that performs the capture.

[0102] [Figure 91] This document details the dot product engine design that supports sparse matrix-sparse vector multiplication according to the embodiment.

[0103] [Figure 92] An example using a specific value is shown.

[0104] [Figure 93] This section demonstrates how sparse-dense and sparse-sparse dot product engines can be combined to produce a dot product engine capable of handling both types of calculations.

[0105] [Figure 94a] This demonstrates socket swapping using a 12-accelerator stack.

[0106] [Figure 94b]This shows a multi-chip package (MCP) implementation using a set of processors / cores and eight stacks.

[0107] [Figure 95] This shows the accelerator stack.

[0108] [Figure 96] This shows a potential accelerator layout intended to be located beneath a WIO3 DRAM stack, which includes 64 dot product engines, 8 vector caches, and an integrated memory controller.

[0109] [Figure 97] This compares seven DRAM technologies.

[0110] [Figure 98a] This shows a stacked DRAM. [Figure 98b] This shows a stacked DRAM.

[0111] [Figure 99] A list of breadth-first search (BFS) methods is shown.

[0112] [Figure 100] The format of the descriptor used to define a lambda function according to one embodiment is shown below.

[0113] [Figure 101] The lower 6 bytes of the header word in the embodiment are shown.

[0114] [Figure 102] Matrix value buffer, matrix index buffer, and vector value buffer are shown.

[0115] [Figure 103] This document details one embodiment of a lambda data path.

[0116] [Figure 104] An example of instruction encoding is shown.

[0117] [Figure 105] This indicates the encoding for a specific set of instructions.

[0118] [Figure 106] The encoding of an exemplary comparative predicate is shown.

[0119] [Figure 107] An embodiment using bias is shown.

[0120] [Figure 108a] This shows memory-mapped I / O (MMIO) space registers used with work queue-based implementations. [Figure 108b] This shows memory-mapped I / O (MMIO) space registers used with work queue-based implementations.

[0121] [Figure 109] Here is an example of matrix multiplication.

[0122] [Figure 110] This demonstrates the processing of the octoMADD instruction using a binary tree reduction network.

[0123] [Figure 111] This document describes an embodiment of a method executed by a processor to process a multiply-accumulate instruction.

[0124] [Figure 112] This document describes an embodiment of a method executed by a processor to process a multiply-accumulate instruction.

[0125] [Figure 113(A)] This shows exemplary hardware for executing the MADD instruction. [Figure 113(B)]This shows exemplary hardware for executing the MADD instruction. [Figure 113(C)] This shows exemplary hardware for executing the MADD instruction.

[0126] [Figure 114] This document provides an example of a hardware heterogeneous scheduler circuit and its interaction with memory.

[0127] [Figure 115] An example of a software heterogeneous scheduler is shown.

[0128] [Figure 116] This document illustrates an embodiment of a method for discovering a post-system boot device.

[0129] [Figure 117] This shows an example of moving threads within shared memory.

[0130] [Figure 118] This document illustrates an exemplary method for thread movement that can be performed by a heterogeneous scheduler.

[0131] [Figure 119] As described in detail above, this is a block diagram of a processor that presents an abstract execution environment.

[0132] [Figure 120] This is a simplified block diagram illustrating an exemplary multi-chip configuration.

[0133] [Figure 121] A block diagram is shown representing at least a portion of the system, including an exemplary implementation of multi-chip link (MCL).

[0134] [Figure 122] An exemplary block diagram of an exemplary logic PHY in MCL is shown.

[0135] [Figure 123] A simplified block diagram illustrates an alternative representation of the logic used to implement MCL.

[0136] [Figure 124] This shows an example of execution when ABEGIN / AEND is not supported.

[0137] [Figure 125] This is a block diagram of the register architecture relating to one embodiment of the present invention.

[0138] [Figure 126A] This block diagram shows both an exemplary in-order pipeline and an exemplary register renaming out-of-order issue / execution pipeline according to embodiments of the present invention.

[0139] [Figure 126B] This block diagram shows both an exemplary embodiment of an in-order architecture core included in a processor according to an embodiment of the present invention and an exemplary register renaming out-of-order issue / execution architecture core.

[0140] [Figure 127A] A more specific and illustrative block diagram of an in-order core architecture is shown, where a core would be one of several logical blocks (including other cores of the same and / or different types) within a chip. [Figure 127B] A more specific and illustrative block diagram of an in-order core architecture is shown, where a core would be one of several logical blocks (including other cores of the same and / or different types) within a chip.

[0141] [Figure 128]This is a block diagram of a processor according to an embodiment of the present invention, which may have one or more cores, an integrated memory controller, and integrated graphics.

[0142] [Figure 129] A block diagram of a system according to an embodiment of the present invention is shown.

[0143] [Figure 130] This is a block diagram of a first more specific exemplary system according to an embodiment of the present invention.

[0144] [Figure 131] This is a block diagram of a second, more specific exemplary system according to an embodiment of the present invention.

[0145] [Figure 132] This is a block diagram of an SoC according to an embodiment of the present invention.

[0146] [Figure 133] This block diagram illustrates the use of a software instruction converter, according to an embodiment of the present invention, which converts binary instructions in a source instruction set to binary instructions in a target instruction set. [Modes for carrying out the invention]

[0147] The following detailed description includes references to the accompanying drawings that form part of this specification, where similar reference numerals throughout refer to similar parts, and is illustrated using exemplary embodiments that may be put into practice. Other embodiments may be used, and it should be understood that structural or logical modifications may be made without departing from the scope of this disclosure. Accordingly, the following detailed description should not be taken as restrictive, and the scope of embodiments is defined by the accompanying claims and their equivalents.

[0148] Various operations may be described sequentially as a number of distinct actions or processes in the manner that is most helpful in understanding the subject matter described in the claims. However, the order of description should not be interpreted as suggesting that these processes are necessarily order-dependent. In particular, these processes do not have to be performed in the order presented. The processes described may be performed in a different order than in the embodiments described. In additional embodiments, various additional processes may be performed, and / or the processes described may be omitted.

[0149] For the purposes of this disclosure, the term "A and / or B" means (A), (B), or (A and B). For the purposes of this disclosure, the term "A, B, and / or C" means (A), (B), (C), (A and B), (A and C), (B and C), or (A, B and C).

[0150] In the description, the terms “in one embodiment” or “in multiple embodiments” may be used to refer to one or more of the same or different embodiments, respectively. Furthermore, terms such as “comprising,” “including,” and “having” are synonymous when used in reference to embodiments of the present disclosure.

[0151] As described in the background technology section, the existence of a wide range of stock units and platforms that implement various mixtures of accelerators makes it challenging to deploy accelerator solutions and manage the complexity of accelerators for portable use. Furthermore, considering the vast number of operating systems (and versions, patches, etc.), deploying accelerators using a device driver model presents limitations, including hurdles to adoption stemming from developers' efforts in big data processing, portability, and stringent performance requirements. Accelerators are typically hardware devices (circuits) that perform functions more efficiently than software running on a general-purpose processor. For example, hardware accelerators can be used to improve the execution of specific algorithms / tasks (e.g., video encoding or decoding, specific hash functions, etc.) or classes of algorithms / tasks (e.g., machine learning, sparse data manipulation, cryptography, graphics, physics, regular expressions, packet processing, artificial intelligence, digital signal processing, etc.). Examples of accelerators include, but are not limited to, graphics processing units ("GPUs"), fixed-function field-programmable gate array ("FPGA") accelerators, and fixed-function application-specific integrated circuits ("ASICs"). Note that in some embodiments, the accelerator may be a general-purpose central processing unit ("CPU") where the CPU is more efficient than other processors in the system.

[0152] The power consumption of a given system (e.g., a system-on-a-chip ("SoC"), processor stock unit, rack, etc.) can be consumed by processing elements on only a portion of the available silicon area. This makes it advantageous to build various specialized hardware blocks that reduce energy consumption for specific processes, even though not all hardware blocks can be active simultaneously.

[0153] Embodiments of systems, methods, and apparatus for selecting processing elements (e.g., cores or accelerators) to handle threads, interface with processing elements, and / or manage power consumption within a heterogeneous multiprocessor environment are described in detail. For example, in various embodiments, a heterogeneous multiprocessor is configured (e.g., by design or by software) to dynamically move threads between different types of processing elements of the heterogeneous multiprocessor based on the characteristics of the corresponding workloads of threads and / or processing elements, to provide a program interface to one or more processing elements, to translate code for execution on a particular processing element, and to select a communication protocol to use with the selected processing element based on the characteristics of the workload and the selected processing element or a combination thereof.

[0154] In a first embodiment, the workload dispatch interface, i.e., the heterogeneous scheduler, presents a homogeneous multiprocessor programming model to the system programmer. In particular, this embodiment allows programmers to develop software or equivalent abstractions targeting a specific architecture, while facilitating continuous improvements to the underlying hardware without requiring corresponding changes to the developed software.

[0155] In a second embodiment, a multiprotocol link enables a first entity (such as a heterogeneous scheduler) to communicate with a large number of devices using protocols associated with communication. This replaces the need for separate links for device communication. In particular, the link has three or more protocols dynamically multiplexed on the link. For example, a common link supports protocols comprising: 1) a Producer / Consumer, Discover, Configure, Interrupt (PDCI) protocol that enables device discovery, device configuration, error reporting, interrupts, DMA-style data transfer, and various services, as may be defined in one or more proprietary or industry standards (e.g., PCI Express specification or equivalent alternative); 2) a Caching Agent Coherence (CAC) protocol that enables devices to issue coherent read and write requests to processing elements; and 3) a Memory Access (MA) protocol that enables processing elements to access the local memory of another processing element.

[0156] In a third embodiment, thread scheduling, migration, or emulation, or parts thereof, are performed based on the thread's phase. For example, the data-parallel phase of a thread is typically scheduled or migrated to a SIMD core, the thread-parallel phase of a thread is typically scheduled or migrated to one or more scalar cores, and the serial phase is typically scheduled or migrated to an out-of-order core. Each of the core types minimizes either the energy or latency considered for thread scheduling, migration, or emulation. Emulation may be used when scheduling or migration is not possible or advantageous.

[0157] In a fourth embodiment, threads or parts thereof are opportunistically offloaded to an accelerator. In particular, the thread's accelerator start (ABEGIN) and accelerator end (AEND) instructions, or parts thereof, and bookend instructions, may be executable on an accelerator. If an accelerator is not available, the instructions between ABEGIN and AEND are then executed as usual. However, if an accelerator is available, it is preferable to use the accelerator (e.g., with less power), and the instructions between ABEGIN and AEND are then converted for execution on that accelerator and scheduled for execution on that accelerator. As a result, the use of the accelerator is opportunistic.

[0158] In the fifth embodiment, a thread or part thereof is analyzed for (opportunistic) offloading to an accelerator without using ABEGIN or AEND. Software or hardware pattern matching is performed on the thread or part thereof for code that may be executable on the accelerator. If the accelerator is not available, or if the thread or part thereof itself does not serve the purpose of executing on the accelerator, the instructions of the thread are executed as usual. However, if an accelerator is available, it is preferable to use the accelerator (e.g., using less power), and the instructions are then translated for execution on that accelerator and scheduled for execution on that accelerator. As a result, the use of the accelerator is opportunistic.

[0159] In a sixth embodiment, a transformation of the code fragment (part of the thread) is performed to better suit the selected destination processing element. For example, the code fragment may be transformed to 1) utilize a different instruction set, 2) be more parallelized, 3) be less parallelized (serialized), 4) have parallelized data (e.g., vectorized), and / or 5) have less parallelized data (e.g., de-vectorized).

[0160] In the seventh embodiment, a work queue (either shared or dedicated) receives descriptors that define the scope of work performed by the device. A dedicated work queue stores descriptors for a single application, while a shared work queue stores descriptors submitted by multiple applications. A hardware interface / arbiter dispatches descriptors from the work queue to the accelerator processing engine according to a specific arbitration policy (e.g., based on the processing requirements of each application and the QoS / fairness policy).

[0161] In the eighth embodiment, the improvement over dense matrix multiplication considers the multiplication of two-dimensional matrices in conjunction with the execution of a single instruction. Multiple packed data (SIMD, vector) sources are multiplied against a single packed data source. In some examples, a binary tree is used for multiplication.

[0162] Figure 1 illustrates a heterogeneous multiprocessing execution environment. In this example, a first type of code fragment (e.g., one or more instructions associated with a software thread) is received by the heterogeneous scheduler 101. The code fragment may be any number of source code representations, including, for example, machine code, intermediate representations, bytecode, and text-based code (high-level languages, such as assembly code and source code of C++). The heterogeneous scheduler 101 presents a homogeneous multiprocessor programming model (for example, so that all threads appear to the user and / or operating system as if they were running on a scalar core), determines the workload type (program phase) of an incoming code fragment, selects the type of processing element (scalar, out-of-order (OOO), single-instruction multiple data (SIMD), or accelerator) corresponding to the determined workload type, processes the workload (e.g., scalar for thread-parallel code, OOO for serial code, SIMD for data-parallel, and accelerator for data-parallel), and schedules the code fragment for processing by the corresponding processing element. In the specific embodiment shown in Figure 1, the processing element type is The system includes a scalar core 103 (e.g., an in-order core), a single-instruction multiple data (SIMD) core 105 that performs operations on packed data operands whose registers contain multiple sequentially stored data elements, a low-latency out-of-order core 107, and an accelerator 109. In some embodiments, the scalar core 103, the single-instruction multiple data (SIMD) core 105, and the low-latency out-of-order core 107 are located within a heterogeneous processor, while the accelerator 109 is located outside of this heterogeneous processor. However, it should be noted that various different configurations of the processing elements may be utilized. In some embodiments, the heterogeneous scheduler 101 converts or interprets an received code fragment or part thereof into a format corresponding to a selected type of processing element.

[0163] Processing elements 103-109 may support different instruction set architectures (ISAs). For example, an out-of-order core may support a first ISA, and an in-order core may support a second ISA. This second ISA may be a (sub-or super)set of the first ISA, or it may be different. Furthermore, processing elements may have different microarchitectures. For example, a first out-of-order core may support a first microarchitecture, and an in-order core may support a different second microarchitecture. Note that even within a particular type of processing element, the ISA and microarchitecture may be different. For example, a first out-of-order core may support a first microarchitecture, and a second out-of-order core may support a different microarchitecture. Instructions are "native" to a particular ISA in that they are part of the ISA. Native instructions execute on a particular microarchitecture without requiring external modification (e.g., translation).

[0164] In some embodiments, one or more processing elements are integrated onto a single die, for example, as a system-on-a-chip (SoC). Such embodiments may benefit from, for example, improved communication latency, manufacturing / cost, reduced pin count, and smaller platform size. In other embodiments, the processing elements are packaged together, thereby not needing to be on a single die, and achieving one or more of the SoC benefits referred to above. These embodiments may further benefit from, for example, different processing technologies optimized for each processing element type, smaller die sizes for improved yield, and integration of proprietary intellectual property blocks. Some conventional multi-package limitations may make it difficult to communicate with different devices when they are added. The multiprotocol links described herein minimize or mitigate this challenge by presenting a common interface to different types of devices, such as to the user, operating system ("OS"), etc.

[0165] In some embodiments, the heterogeneous scheduler 101 is implemented in software stored in a computer-readable medium (e.g., memory) for execution on a processor core (e.g., OOO core 107). In these embodiments, the heterogeneous scheduler 101 is referred to as a software heterogeneous scheduler. This software may implement a binary translator, a runtime ("JIT") compiler, an OS 117 for scheduling the execution of threads containing code fragments, a pattern matcher, internal module components, or a combination thereof.

[0166] In some embodiments, the heterogeneous scheduler 101 is implemented in hardware as a circuit and / or a finite state machine executed by the circuit. In these embodiments, the heterogeneous scheduler 101 is referred to as a hardware heterogeneous scheduler.

[0167] From the perspective of a program (e.g., OS117, emulation layer, hypervisor, secure monitor, etc.), each type of processing element 103-109 utilizes the shared memory address space 115. In some embodiments, the shared memory address space 115 selectively contains two types of memory, memory 211 and memory 213, as shown in Figure 2. In such embodiments, the types of memory may be distinguished in various ways, including, but are not limited to, differences in memory location (e.g., located on different sockets, etc.), differences in corresponding interface standards (e.g., DDR4, DDR5, etc.), differences in power requirements, and / or differences in the underlying memory technology used (e.g., high-bandwidth memory (HBM), synchronous DRAM, etc.).

[0168] The shared memory address space 115 is accessible by each type of processing element. However, in some embodiments, different types of memory may be preferentially allocated to different processing elements, for example, based on the needs of the workload. For example, in some embodiments, the platform's firmware interface (e.g., BIOS or UEFI) or memory storage includes fields indicating the types of memory resources available on the platform and / or the commonalities of processing elements with respect to a particular address range or memory type.

[0169] The heterogeneous scheduler 101 utilizes this information when analyzing threads to determine where a thread will execute at a given point in time. Typically, a thread management mechanism examines all available information to inform an informed decision, depending on how it manages existing threads. This can reveal itself in numerous ways. For example, threads running on a particular processing element that share a physically close address range commonality with that processing element may be given priority processing over threads under the normal circumstances under which they would run on that processing element.

[0170] Another example is that a thread that would benefit from a particular memory type (e.g., a faster version of DRAM) might physically move its data to that memory type and adjust its memory references in the code to point to a portion of the shared address space. For example, a thread on SIMD core 105 might utilize a second memory type 213, but it might be moved away from this utilization if accelerator 109 is active and requires that memory type 213 (or at least requires the portion allocated to the thread on SIMD core 105).

[0171] An exemplary scenario is when memory is physically closer to one processing element than to another. A common example is when an accelerator is directly connected to a different type of memory than the core.

[0172] In these examples, it is typically the OS that initiates the data movement. However, there is nothing preventing lower levels (e.g., a heterogeneous scheduler) from performing this function on their own or with assistance from another component (e.g., the OS). Whether data from the previous processing element is flushed and page table entries are invalidated depends on the implementation and disadvantages of performing the data movement. If the data is not likely to be used immediately, it may be more feasible to simply copy it from storage rather than move the data from one memory type to another.

[0173] Figures 117(A) and 117(B) illustrate an example of movement for threads in shared memory. In this example, two types of memory share an address space, each having its own range of addresses within that space. In 117(A), the shared memory 11715 contains a first type of memory 11701 and a second type of memory 11707. The first type of memory 11701 has a first address range 11703, which is within the range of addresses dedicated to thread 11705. The second type of memory 11707 has a second address range 11709.

[0174] At some point during the execution of thread 1 11705, the heterogeneous scheduler makes a decision to move thread 1 11705 to use addresses in the first type of memory 11701 before the second thread 11711 is assigned to thread 1 11705. This is shown in Figure 117(B). In this example, thread 1 11705 is reallocated to the second type of memory 11707 and given a new set of addresses to use. However, this is not necessarily the case. Note that the difference in memory types can be physical or spatial (for example, based on distance to the PE).

[0175] Figure 118 shows an exemplary method for thread movement that can be performed by a heterogeneous scheduler. In 11801, a first thread is instructed to run on a first processing element ("PE"), such as a core or accelerator, using a first type of memory in the shared memory space. For example, in Figure 117(A), this is thread 1.

[0176] At some point later, a request to execute a second thread is received at 11803. For example, an application, the OS, etc., requests a hardware thread to be executed.

[0177] In 11805, it is determined that the second thread should be executed on a second PE using the first type of memory in the shared address space. For example, the second thread runs on an accelerator directly coupled to the first type of memory, and this execution (including freeing the memory used by the first thread) is more efficient than allowing the second thread to use the second type of memory.

[0178] In some embodiments, in 11807, the data of the first thread is moved from a first type of memory to a second type of memory. This does not necessarily occur when it is more efficient to simply stop the execution of the first thread and start another thread in its place.

[0179] In 11809, the Translation Lookaside Buffer (TLB) entry associated with the first thread is disabled. Furthermore, in most embodiments, data flushing is performed.

[0180] At 11811, the second thread is directed to the second PE and is allocated to a range of addresses in the first type of memory that was previously allocated to the first thread.

[0181] Figure 3 shows an exemplary embodiment of the heterogeneous scheduler 301. In some examples, the scheduler 301 is part of the runtime system. As shown, the program phase detector 313 receives a code fragment and identifies one or more characteristics of the code fragment to determine whether the execution of the corresponding program phase is best characterized as serial, data parallel, or thread parallel. An example of how this determination is made is described in detail below. As described in detail with respect to Figure 1, the code fragment may be in any number of source code representation forms.

[0182] For repetitive code fragments, the pattern matcher 311 identifies this "hot" code and, in some examples, also identifies corresponding characteristics indicating that the workload associated with the code fragment may be better suited to being processed by a different processing element. Further details regarding the pattern matcher 311 and its operation are described below, for example, in the context of Figure 20.

[0183] The selector 309 selects a target processing element to execute the native representation of the received code fragment, at least in part, based on the characteristics of the processing elements and the thermal and / or power information provided by the power manager 307. The selection of the target processing element may be as simple as possible in choosing the one that best fits the code fragment (i.e., the match between the workload characteristics and the processing element's capabilities), but may also take into account the system's current power consumption level (e.g., if provided by the power manager 307), the availability of the processing elements, the amount of data to move from one type of memory to the other (and the disadvantages associated with doing so), etc. In some embodiments, the selector 309 is a finite state machine implemented in or executed by hardware circuitry.

[0184] In some embodiments, the selector 309 also selects a corresponding link protocol for communicating with the target processing element. For example, in some embodiments, the processing element utilizes a corresponding common link interface that can dynamically multiplex or encapsulate multiple protocols relating to the system fabric or point-to-point interconnection. For example, in certain embodiments, supported protocols include: 1) a Producer / Consumer, Discover, Configure, Interrupt (PDCI) protocol that enables device discovery, device configuration, error reporting, interrupts, DMA-style data transfer, and various services, as may be defined in one or more proprietary or industry standards (e.g., PCI Express specification or equivalent alternative); 2) a Caching Agent Coherence (CAC) protocol that enables a device to issue coherent read and write requests to a processing element; and 3) a Memory Access (MA) protocol that enables a processing element to access the local memory of another processing element. The selector 309 makes a selection among these protocols based on the type of request to be communicated to the processing element. For example, producer / consumer, discovery, configuration, or interrupt requests use the PDCI protocol, cache coherence requests use the CAC protocol, and local memory access requests use the MA protocol.

[0185] In some embodiments, the thread includes a marker indicating the phase type, and therefore, a phase detector is not used. In some embodiments, the thread includes implicit or explicit requests regarding the processing element type, link protocol, and / or memory type. In these embodiments, selector 309 utilizes this information in its selection process. For example, the selection by selector 309 may be overridden by the thread and / or the user.

[0186] Depending on the implementation, the heterogeneous scheduler may include one or more converters that process incoming code fragments and generate corresponding native encodings for target processing elements. For example, the heterogeneous scheduler may include a converter that translates a first type of machine code to a second type of machine code, and / or a JIT compiler that translates an intermediate representation into a format native to the target processing element. Alternatively or further, the heterogeneous scheduler may include a pattern matcher that identifies recurring code fragments (i.e., "hot" code) and caches one or more native encodings for the code fragments or corresponding microoperations. Each of these optional components is shown in Figure 3. In particular, the heterogeneous scheduler 301 includes a converter 303 and a JIT compiler 305. When the heterogeneous scheduler 301 performs operations on object code or an intermediate representation, the JIT compiler 305 is invoked to translate the incoming code fragment into a format native to one or more of the target processing elements 103, 105, 107, and 109. When the heterogeneous scheduler 301 performs operations on machine code (binary) (for example, when converting from one instruction set to another), the binary translator 303 converts the received code fragment into machine code native to one or more of the target processing elements. In an alternative embodiment, the heterogeneous scheduler 301 may exclude one or more of these components.

[0187] For example, in some embodiments, a binary translator is not included. This can result in increased programming complexity, as the program needs to consider potentially available accelerators, cores, etc., instead of letting the scheduler handle this. For example, the program might need to include code for routines in different formats. However, in some embodiments, if there is no binary translator, there is a JIT compiler that accepts the code at a higher level and performs the necessary translations. If a pattern matcher is present, hot code may be further detected to find the code that should be executed by a particular processing element.

[0188] For example, in some embodiments, a JIT compiler is not included. This can also result in increased programming complexity, as the program must first be compiled into machine code for a specific ISA instead of having the scheduler handle this. However, in some embodiments, if a binary translator is available and a JIT compiler is not, the scheduler may perform translations between ISAs, as described in detail below. If a pattern matcher is present, hot code may be further detected to find the code that should be executed in a particular processing element.

[0189] For example, in some embodiments, pattern matchers are not included. This can also result in decreased efficiency, as the code that may have been moved is likely to remain in the core and may not be efficient for the particular task being executed.

[0190] In some embodiments, there is no binary translator, JIT compiler, or pattern matcher. In these embodiments, only phase detection or explicit requests for moving threads are used for thread / processing element allocation / migration.

[0191] Referring again to Figures 1-3, the heterogeneous scheduler 101 may be implemented in hardware (e.g., circuitry), software (e.g., executable program code), or any combination thereof. Figure 114 shows an example of a hardware heterogeneous scheduler circuit and its interaction with memory. The heterogeneous scheduler may be created in many different forms, including, but not limited to, a field-programmable gate array (FPGA) based or application-specific integrated circuit (ASIC) based state machine, an embedded microcontroller coupled to memory that internally stores software providing the functions described in detail herein, a logic circuit with other subcomponents (e.g., a data hazard detection circuit), and / or software executed by an out-of-order core (e.g., a state machine), software executed by a scalar core (e.g., a state machine), software executed by a SIMD core (e.g., a state machine), or a combination thereof. In the illustrated example, the heterogeneous scheduler is a circuit 11401 comprising one or more components that perform various functions. In some embodiments, this circuit 11401 is part of the processor core 11419, but it may also be part of the chipset.

[0192] The thread / processing element (PE) tracker 11403 maintains the status of each thread running on the system and each PE (e.g., PE availability, current power consumption, etc.). For example, the tracker 11403 maintains the active, idle, or inactive status of data structures such as tables.

[0193] In some embodiments, the pattern matcher 11405 identifies “hot” codes, accelerator codes, and / or codes requesting PE assignments. Further details regarding this matching will be provided later.

[0194] The PE information 11411 stores information about what PEs (and their types) are in the system and what can be scheduled by the OS or the like.

[0195] Although described in detail above as separate components within the heterogeneous scheduler circuit 11401, on the other hand, the components may be combined and / or moved outside the heterogeneous scheduler circuit 11401.

[0196] The memory 11413 coupled to the heterogeneous scheduler circuit 11401 may include software to provide additional functionality (by the core and / or the heterogeneous scheduler circuit 11401). For example, the software pattern matcher 11417 may be used to identify "hot" code, accelerator code, and / or code that requests a PE allocation. For example, the software pattern matcher 11417 compares a code sequence to a predetermined set of patterns stored in the memory. The memory may also store a converter that converts code from one instruction set to another (e.g., from one instruction set to accelerator-based instructions or primitives).

[0197] These components supply the selector 11411, which selects the PE to execute a thread, with information such as what link protocol is used and what migrations should occur if there are already threads running on a PE, etc. In some embodiments, the selector 11411 is a finite state machine implemented within a hardware circuit or executed by a hardware circuit.

[0198] In some embodiments, the memory 11413 may include that one or more converters 11415 (e.g., binary, JIT compiler, etc.) are stored in the memory to convert thread code into different formats for the selected PE.

[0199] Figure 115 shows an example of a software heterogeneous scheduler. A software heterogeneous scheduler may be implemented in many different forms, including, but is not limited to, a field-programmable gate array (FPGA)-based or application-specific integrated circuit (ASIC)-based state machine, an embedded microcontroller coupled to memory that stores software providing the functions described in detail herein, a logic circuit with other subcomponents (e.g., a data hazard detection circuit), and / or software executed by an out-of-order core (e.g., a state machine), software executed by a scalar core (e.g., a state machine), software executed by a SIMD core (e.g., a state machine), or a combination thereof. In the illustrated example, the software heterogeneous scheduler is stored in memory 11413. As a result, memory 11413 coupled to the processor core 11419 contains software executed (by the core) to schedule threads. In some embodiments, the software heterogeneous scheduler is part of the OS.

[0200] Depending on the implementation, the thread / processing element (PE) tracker 11403 within the core maintains the status (e.g., PE availability, current power consumption, etc.) for each thread running in the system and each PE, or this is done in software using the thread / PE tracker 11521. For example, the tracker maintains the active, idle, or inactive status of data structures such as tables.

[0201] In some embodiments, the pattern matcher 11417 identifies "hot" codes and / or codes requesting PE assignments. Further details regarding this matching will be provided later.

[0202] PE information 11409 and / or 11509 stores information about what PEs are in the system and what can be scheduled by the OS, etc.

[0203] Software pattern matcher 11417 may be used to identify “hot” code, accelerator code, and / or code that requests PE allocation.

[0204] The thread / PE tracker, processing element information, and / or pattern matching are supplied to the selector 11411, which selects the PE on which to execute a thread, determining what link protocol to use and what transitions should occur if there are already threads running in the PE or elsewhere. In some embodiments, the selector 11411 is a finite state machine implemented and executed by the processor core 11419.

[0205] Memory 11413 may, for example in some embodiments, include storing one or more converters 11415 (e.g., binary, JIT compiler, etc.) in order to convert thread code into different formats for selected PEs.

[0206] During operation, the OS uses a heterogeneous scheduler (e.g., heterogeneous scheduler 101, 301, etc.) that presents an abstraction of the execution environment to schedule the threads to be processed and then executes them.

[0207] The table below summarizes the potential abstraction features (i.e., what the program refers to), potential design freedoms and architectural optimizations (i.e., what is hidden from the programmer), and the potential benefits or reasons for providing specific features in the abstraction. [Table 1]

[0208] In some exemplary implementations, the heterogeneous scheduler, in combination with other hardware and software resources, presents a complete programming model that does everything and supports all programming techniques (e.g., compilers, intrinsic functions, assembly, libraries, JIT, offloading, devices). Other exemplary implementations present alternative execution environments that are compatible with those provided by other processor development companies, such as ARM Holdings, MIPS, IBM, or their licensors or adopters.

[0209] Figure 119 is a block diagram of a processor presenting an abstract execution environment, as detailed above. In this example, the processor 11901 includes several different core types, e.g., those described in detail in Figure 1. Each (wide) SIMD core 11903 includes a fused multiply-accumulate (FMA) circuit supporting dense arithmetic primitives, its own cache (e.g., L1 and L2), purpose-specific execution circuitry, and storage for thread state.

[0210] Each latency-optimized (OOO) core 11913 includes a fused multiply-accumulate (FMA) circuit, its own cache (e.g., L1 and L2), and out-of-order execution circuitry.

[0211] Each scalar core 11905 includes a fused multiply-accumulate (FMA) circuit, its own cache (e.g., L1 and L2), and storage for purpose-specific execution and thread states. Typically, a scalar core 11905 supports enough threads to cover memory latency. In some embodiments, the number of SIMD cores 11903 and latency-optimized cores 11913 is smaller compared to the number of scalar cores 11905.

[0212] In some embodiments, one or more accelerators 11917 are included. These accelerators 11917 may be fixed function or FPGA-based. Alternatively or in addition to these accelerators 11917, in some embodiments, the accelerator 11917 is external to the processor.

[0213] Processor 11901 also includes a last level cache (LLC) 11907 that is shared by cores within the processor and potentially any accelerators. In some embodiments, LLC 11907 includes circuitry for fast atomics.

[0214] One or more interconnects 11915 couple the cores and accelerators to each other and to an external interface. For example, in some embodiments, a mesh-type interconnect couples various cores.

[0215] Memory controller 11909 couples the cores and / or accelerators to memory.

[0216] A plurality of input / output interfaces (e.g., PCIe, common link, described in detail below) 11911 connect processor 11901 to external devices, such as other processors and accelerators.

[0217] FIG. 4 shows an embodiment for system boot and device discovery of a computer system. Knowledge about the system includes, for example, knowledge such as which cores are available, how much memory is available, and memory locations associated with the cores, which is utilized by a heterogeneous scheduler. In some embodiments, this knowledge is built using Advanced Configuration and Power Interface (ACPI).

[0218] At 401, the computer system is booted.

[0219] In step 403, a query is performed for configuration settings. For example, in some BIOS-based systems, when the system boots, the BIOS checks the system's operation and prepares the computer for operation by querying its own memory banks for drive and other configuration settings.

[0220] In step 405, plug-in components are searched for. For example, the BIOS searches for any plug-in components in the computer, sets up pointers (interrupt vectors) in memory, and accesses their routines. The BIOS accepts requests from device drivers and application programs regarding interface connections with hardware and other peripheral devices.

[0221] In 407, data structures for system components (e.g., core, memory, etc.) are generated. For example, the BIOS typically generates hardware device and peripheral device configuration information that the OS interfaces with. Furthermore, ACPI defines a flexible and scalable hardware interface to the system board, enabling the computer to turn its peripherals on and off to improve power management, especially in portable devices such as notebook computers. The ACPI specification includes hardware interfaces, software interfaces (APIs), and data structures that, if implemented, support OS-oriented configuration and power management. Software designers can use ACPI to integrate power management functions across the entire computer system, including hardware, operating systems, and application software. This integration allows the OS to determine which devices are active and which handle all power management resources for computer subsystems and peripherals.

[0222] In step 409, the operating system (OS) is loaded and gains control. For example, when the BIOS completes its boot routine, it hands over control to the OS. In the case of ACPI, the BIOS hands over control of the computer to the OS and exports a data structure containing the ACPI namespace to the OS, which may be represented graphically as a tree. The namespace acts as a directory of ACPI devices connected to the computer and contains objects that further define and provide status information for each ACPI device. Each node in the tree is associated with a device, while nodes, subnodes, and leaves represent objects that control the device and return specific information to the OS, as defined in the ACPI specification, when evaluated by the OS. The OS or a driver accessed by the OS may include a set of functions for enumerating and evaluating namespace objects. The OS can be said to have evaluated an object when it calls a function and returns a value for an object in the ACPI namespace.

[0223] In some examples, the available devices change. For example, accelerators, memory, etc., may be added. An embodiment of the method for post-system boot device discovery is shown in Figure 116. For example, an embodiment of this method may be used to discover accelerators added to the system after booting. In 11601, an indication of a connected device being powered on or reset is received. For example, an endpoint device may be plugged into a PCIe slot or reset by the OS, for example.

[0224] In 11603, link training is performed using the connected devices to initialize them. For example, PCIe link training is performed to establish link configuration parameters such as link width, lane polarity, and / or the maximum supported data rate. In some embodiments, the performance of the connected devices is stored (e.g., in an ACPI table).

[0225] In step 11605, once the initialization of the connected device is complete, a ready message is sent from the connected device to the system.

[0226] In 11607, the ready status bit for the connected device is set to indicate that the device is ready to configure.

[0227] In 11609, an initialized, connected device is configured. In some embodiments, the device and the OS agree on an address for the device (e.g., a memory-mapped I / O (MMIO) address). The device provides a device descriptor that includes one or more of the following: vendor identification number (ID), device ID, model number, serial number, characteristics, resource requirements, etc. The OS may determine additional operation and configuration parameters for the device based on the descriptor data and system resources. The OS may generate configuration queries. The device may respond to the device descriptor. The OS then generates configuration data and sends this data to the device (e.g., through PCI hardware). This may include setting a base address register that defines the address space associated with the device.

[0228] After system knowledge has been established, the OS uses a heterogeneous scheduler (e.g., heterogeneous schedulers 101, 301, etc.) to schedule and execute the threads to be processed. The heterogeneous scheduler then dynamically and transparently maps the code fragments of each thread to the most appropriate type of processing element (e.g., to the user and / or the OS), thereby potentially avoiding the need to build hardware for legacy architecture mechanisms and potentially exposing microarchitectural details to system programmers or the OS.

[0229] In some cases, the most suitable type of processing element is determined based on the performance of the processing element and the execution characteristics of the code fragment. Generally, a program and its associated threads may have different execution characteristics depending on the workload being processed at a given time. Exemplary execution characteristics, or execution phases, include, for example, data-parallel phases, thread-parallel phases, and serial phases. The following table identifies these phases and summarizes their characteristics. The table also includes exemplary workloads / operations, exemplary hardware useful for processing each phase type, and typical purposes for the phases and hardware used. [Table 2]

[0230] In some embodiments, the heterogeneous scheduler is configured to select either thread migration or emulation. In configurations where each type of processing element can handle any type of workload (if emulation is required to do so), the most suitable processing element is selected for each program phase based on one or more criteria, including, for example, the workload's latency requirements, the increased execution latency associated with emulation, and the power and thermal characteristics and constraints of the processing elements. As will be described in detail later, the selection of the suitable processing element is achieved in some embodiments by detecting the presence of SIMD instructions or vectorizable code within a code fragment, taking into account the number of threads currently running.

[0231] Moving threads between processing elements is not without its drawbacks. For example, data may need to be moved from a shared cache to a lower-level cache, and both the original and receiving processing elements will flush their pipelines to adapt to the move. Depending on the situation, in some embodiments, heterogeneous schedulers implement hysteresis to avoid very frequent migrations (e.g., by setting thresholds for one or more criteria referenced above, or a subset of the same). In some embodiments, hysteresis is implemented by limiting thread migrations so as not to exceed a predefined rate (e.g., 1 migration per millisecond). Thus, the rate of such migrations is limited to avoid excessive overloads resulting from code generation, synchronization, and data migration.

[0232] In some embodiments, for example, as a preferred approach for a particular thread, if the heterogeneous scheduler does not select a transition, the heterogeneous scheduler emulates the missing functionality for the thread in the assigned processing element. For example, in embodiments that maintain a constant total number of threads available to the operating system, the heterogeneous scheduler may emulate multithreading if the number of available hardware threads is oversubscribed (e.g., in a wide concurrent multithreading core). On a scalar or latency core, one or more SIMD instructions of a thread are converted to scalar instructions, or on a SIMD core, more threads are spawned and / or instructions are converted to utilize packed data.

[0233] Figure 5 illustrates an example of thread transitions based on a mapping of program phases to three types of processing elements. As illustrated, the three types of processing elements include latency optimization (e.g., out-of-order core, accelerator, etc.), scalar (processing one data item per instruction time), and SIMD (processing multiple data elements per instruction). Typically, this mapping is performed per thread or per code fragment by a heterogeneous scheduler in a manner transparent to the programmer and the operating system.

[0234] In one embodiment, a heterogeneous scheduler is used to map each phase of a workload to the most appropriate type of processing element. Ideally, this reduces the need to build hardware for legacy functions and avoids exposing microarchitectural details, as the heterogeneous scheduler presents a complete programming model that supports multiple code types, including compiled code (machine code), intrinsic functions (programming language logical constructs that map directly to processor or accelerator instructions), assembly code, libraries, intermediate (JIT-based), offload (movement from one machine type to another), and device-specific.

[0235] In a particular configuration, the default selection for the target processing element is the processing element that optimizes latency.

[0236] Referring again to Figure 5, in the serial phase execution 501 for the workload, it is initially processed by one or more latency-optimized processing elements. If a phase shift is detected (for example, in a dynamic way that the code parallelizes more data, such as seen by the type of instructions obtained in the code before or during execution, or before execution), the workload is moved to one or more SIMD processing elements to complete the data-parallel phase execution 503. Furthermore, the execution scheduling and / or transformations are typically cached. The workload is then moved back to one or more latency-optimized processing elements, or a second set of one or more latency-optimized processing elements, to complete the next serial phase execution 505. Next, the workload is moved to one or more scalar cores to process the thread-parallel phase execution 507. Then, the workload is moved back to one or more latency-optimized processing elements to complete the next serial phase execution 509.

[0237] This illustrated example demonstrates a return to a latency-optimized core, while a heterogeneous scheduler may continue execution for any subsequent phases in one or more corresponding types of processing elements until the thread is terminated. In some embodiments, processing elements utilize a work queue to store incomplete tasks. As a result, tasks do not have to start immediately, but are executed when these spots in the queue appear.

[0238] Figure 6 shows an exemplary execution flow performed by a heterogeneous scheduler, e.g., heterogeneous scheduler 101. This flow illustrates the selection of processing elements (e.g., cores). As illustrated, the heterogeneous scheduler receives a code fragment. In some embodiments, but not limited to, an event occurs that includes one or more instructions indicating a thread wake-up command, a write to a page directory base register, a sleep command, a thread phase change, and a desired reallocation.

[0239] In 601, the heterogeneous scheduler determines whether a code fragment (e.g., a code fragment in a serial or parallel phase) is parallel based, for example, on the dependencies of detected data, instruction type, and / or control flow instructions. For example, a thread full of SIMD code would be considered parallel. If the code fragment is not suitable for parallel processing, the heterogeneous scheduler selects one or more latency-sensitive operational elements (e.g., OOO cores) to process the code fragment in the serial phase execution 603. Typically, OOO cores have (deep) inference and dynamic scheduling and usually have lower watt performance compared to simpler alternatives.

[0240] In some embodiments, latency-sensitive operational elements typically consume more power and die space than scalar cores, so there are no available latency-sensitive operational elements. In these embodiments, only scalar, SIMD, and accelerator cores are available.

[0241] In 605, with respect to parallel code fragments, parallelizable code fragments, and / or vectorizable code fragments, the heterogeneous scheduler determines the type of parallelism of the code. In 607, with respect to thread-parallel code fragments, the heterogeneous scheduler selects a thread-parallel processing element (e.g., a multiprocessor scalar core). A thread-parallel code fragment contains independent instruction sequences that can be executed simultaneously on separate scalar cores.

[0242] Data-parallel code occurs when each processing element performs the same task on a different number of data points. Data-parallel code can take the form of different data layouts: packed and random. The data layout is determined in 609. Random data may be assigned to SIMD processing elements, but requires the use of gather instructions 613 to retrieve data from different memory locations, spatial computation arrays 615 (which spatially map computations onto an array of small programmable processing elements, e.g., an FPGA array), or arrays of scalar processing elements 617. Packed data is assigned in 611 to SIMD processing elements or processing elements that use dense arithmetic primitives.

[0243] In some embodiments, the code fragment is transformed to better suit the selected destination processing element. For example, the code fragment may be transformed to 1) utilize a different instruction set, 2) be more parallelized, 3) be less parallelized (serialized), 4) have parallelized data (e.g., vectorized), and / or 5) have less parallelized data (e.g., de-vectorized).

[0244] After a processing element is selected, the code fragment is sent to one of the processing elements that has been determined to be executed.

[0245] Figure 7 shows an example of a method for thread destination selection by a heterogeneous scheduler. In some embodiments, this method is performed by a binary translator. At 701, the thread to be evaluated or a code fragment thereof is received. In some embodiments, but not limited to, an event occurs that includes one or more instructions indicating a thread wake-up command, a write to a page directory base register, a sleep command, a thread phase change, and a desired reallocation.

[0246] In 703, a determination is made as to whether a code fragment is to be offloaded to an accelerator. For example, a code fragment that is sent to an accelerator. The heterogeneous scheduler may know that this is a corrective action if the code contains code that identifies a request to use an accelerator. This request may be an identifier indicating that a region of code may be executed on an accelerator or executed natively (e.g., ABEGIN / AEND as described herein), or an explicit command to use a specific accelerator.

[0247] In some embodiments, the code fragment is transformed in 705 to better suit the selected destination processing element. For example, the code fragment may be transformed to 1) utilize a different instruction set, 2) be more parallelized, 3) be less parallelized (serialized), 4) have parallelized data (e.g., vectorized), and / or 5) have less parallelized data (e.g., de-vectorized).

[0248] Typically, in 707, translated threads are cached for later use. In some embodiments, the binary translator locally caches translated threads so that they are available for future use by the binary translator. For example, if the code becomes "hot" (executed repeatedly), the cache provides a mechanism for future use without the disadvantage of the translation (although there may be a transmission cost).

[0249] In 709, the (translated) thread is sent to a destination processing element for processing (e.g., offloaded). In some embodiments, the translated thread is cached by the receiver so that it is locally available for future use. Furthermore, if the receiver or binary translator determines that the code is "hot", this caching allows for faster execution with less energy used.

[0250] In 711, the heterogeneous scheduler determines whether a code fragment (e.g., a code fragment in a serial or parallel phase) is parallel based, for example, on the dependencies of detected data, instruction type, and / or control flow instructions. For example, a thread full of SIMD code would be considered parallel. If the code fragment is not suitable for parallel processing, the heterogeneous scheduler selects one or more latency-sensitive operational elements (e.g., OOO cores) to process the code fragment in the serial phase execution 713. Typically, OOO cores have (deep) inference and dynamic scheduling, and therefore may have better watt performance compared to scalar alternatives.

[0251] In some embodiments, latency-sensitive operational elements typically consume more power and die space than scalar cores, so there are no available latency-sensitive operational elements. In these embodiments, only scalar, SIMD, and accelerator cores are available.

[0252] In 715, with respect to parallel code fragments, parallelizable code fragments, and / or vectorizable code fragments, the heterogeneous scheduler determines the type of parallelism of the code. In 717, with respect to thread-parallel code fragments, the heterogeneous scheduler selects a thread-parallel processing element (e.g., a multiprocessor scalar core). A thread-parallel code fragment contains independent instruction sequences that can be executed simultaneously on separate scalar cores.

[0253] Data parallel code occurs when each processing element performs the same task on a different number of data points. Data parallel code can take the form of different data layouts, such as packed and random. The data layout is determined in 719. Random data may be assigned to SIMD processing elements, but it is necessary to use a gather instruction 723, a spatial calculation array 725, or an array of scalar processing elements 727. Packed data is assigned in 721 to a SIMD processing element or processing element that uses dense arithmetic primitives.

[0254] In some embodiments, transformations of unoffloaded code fragments are performed to better suit the destination processing element being determined. For example, a code fragment may be transformed to 1) utilize a different instruction set, 2) be more parallelized, 3) be less parallelized (serialized), 4) have data parallelized (e.g., vectorized), and / or 5) have data less parallelized (e.g., unvectorized).

[0255] After a processing element is selected, the code fragment is sent to one of the processing elements that has been determined to be executed.

[0256] The OS refers to the total number of potentially available threads, regardless of whether cores and accelerators have access to them. In the following description, each thread is enumerated by a thread identifier (ID) called a logical ID. In some embodiments, the operating system and / or heterogeneous scheduler use logical IDs to map threads to a specific processing element type (e.g., core type), processing element ID, and thread IDs on that processing element (e.g., a tuple of core type, core ID, thread ID). For example, a scalar core has a core ID and one or more thread IDs, a SIMD core has a core ID and one or more thread IDs, an OOO core has a core ID and one or more thread IDs, and / or an accelerator has a core ID and one or more thread IDs.

[0257] Figure 8 illustrates the concept of using striped mapping for logical IDs. Striped mapping may be used by a heterogeneous scheduler. In this example, there are eight logical IDs and three core types, each with one or more threads. Typically, the mapping from logical IDs to (core IDs, thread IDs) is calculated using division and modulo and may be fixed to maintain commonality among software threads. The mapping from logical IDs to (core types) is performed flexibly by the heterogeneous scheduler to adapt to future new core types that are readily available to the OS.

[0258] Figure 9 shows an example of using striped mapping for logical IDs. In this example, logical IDs 1, 4, and 5 are mapped to the first core type, and all other logical IDs are mapped to the second core type. The third core type is not used.

[0259] In some embodiments, core type groupings are created. For example, a “core group” tuple may consist of one OOO tuple and accelerator core tuples whose scalars, SIMDs, and logical IDs map to the same OOO tuple. Figure 10 shows an example of a core group. Typically, serial phase detection and thread migration are performed within the same core group.

[0260] Figure 11 shows an example of how threads are executed in a system that utilizes a binary translator switching mechanism. In 1101, the threads are executed on a core. The core may be any of the types described in detail herein, including an accelerator.

[0261] At 1103, potential core reallocation events occur at some point during thread execution. Exemplary core reallocation events include, but are not limited to, one or more instructions indicating a thread wake-up command, a write to a page directory base register, a sleep command, a thread phase change, and a desired reallocation to a different core.

[0262] In 1105, the event is processed, and a decision is made depending on whether or not there is a change in the core allocation. An example of how a particular core allocation may be processed is described in detail below.

[0263] In some embodiments, core (re)allocation is subject to one or more limiting factors, such as migration rate limits and power consumption limits. Migration rate limits are tracked per core type, core ID, and thread ID. Once a thread is assigned to a target (core type, core ID, thread ID), a timer is started and maintained by a binary translator. No other threads can migrate to the same target until the timer expires. As a result, a thread may migrate away from its current core before the timer expires, but not the other way around.

[0264] As will be described in detail, as more core types (including accelerators) are added to computing systems (either on-die or off-die), attention to power consumption limits is likely to increase. In some embodiments, the instantaneous power consumed by all running threads on all cores is calculated. If the calculated power consumption exceeds a threshold, new threads are simply assigned to lower-power cores, such as SIMD, scalar, and dedicated accelerator cores, and one or more threads are forced to migrate from OOO cores to lower-power cores. Note that in some embodiments, power consumption limits take precedence over migration rate limits.

[0265] Figure 12 illustrates an exemplary method for assigning hot code to cores for an accelerator. In 1201, a determination is made that the code is "hot." A hot portion of the code may refer to a part of the code that is better suited to running on one core via another, based on considerations such as power, performance, heat, other known processor criteria, or a combination thereof. This determination may be made using any number of techniques. For example, a dynamic binary optimizer may be used to monitor thread execution. Hot code may be detected based on a counter value that records the dynamic execution frequency of static code, such as during program execution. In an embodiment where one core is an OOO core and the other core is an in-order core, the hot portion of the code may then refer to a hotspot of program code that is better suited to running on a serial core and potentially has more available resources for execution of high-repetition sections. Often, sections of code with high-repetition patterns can be optimized to run more efficiently on an in-order core. Essentially, in this example, cold code (low iteration) is distributed to the native OOO core, while hot code (high iteration) is distributed to a software-managed in-order core. The hot portion of the code may be identified statically, dynamically, or a combination thereof. In the first case, the compiler or user may determine that a section of program code is hot code. In one embodiment, the decoding logic within the core is adapted to decode a hot code identifier instruction from the program code, which identifies the hot portion of the program code. Fetching or decoding such an instruction may trigger the translation and / or execution of the hot section of code on the core. In another example, code execution is profiled execution, based on profile characteristics—power and / or performance metrics associated with the execution—and a region of program code may be identified as hot code. Similar to hardware operations, monitoring code may run on one core to perform monitoring / profiling of program code running on other cores.It should be noted that such monitoring code may be code held in a storage structure within the core or in a system including the processor. For example, monitoring code may be microcode or other code held in the core's storage structure. As yet another example, static identification of hot code is done implicitly. However, dynamic profiling of program code execution can ignore static identification of code regions as hot, and this type of static identification is often referred to as compiler or user implicit, which dynamic profiling may consider when determining which cores are suitable for code distribution. Furthermore, as with the characteristics of dynamic profiling, identification of code regions as hot is not limited to sections of code always being identified as hot. After transformation and / or optimization, a transformed version of the code section is executed.

[0266] In step 1203, an appropriate accelerator is selected. The binary translator, virtual machine monitor, or operating system makes this selection based on the available accelerators and the desired performance. In many cases, an accelerator is better suited to running hot code with improved performance per watt than a larger, more common core.

[0267] In 1205, the hot code is sent to the selected accelerator. This transmission utilizes the appropriate connection type, as described in detail herein.

[0268] Finally, in step 1207, the hot code is received and executed by the selected accelerator. During execution, the hot code may be evaluated for assignment to different cores.

[0269] Figure 13 illustrates an exemplary method for possible core allocation for wake-up or write events in response to page directory-based register events. For example, this shows how to determine the phase of a code fragment. At 1301, either a wake-up event or a page directory-based register (e.g., task switching) event is detected. For example, a wake-up event occurs due to an interrupt received by a stopped thread or the end of a waiting state. A write to the page directory-based register may indicate the start or end of a serial phase. Typically, this detection occurs on the core running the binary translator.

[0270] In 1303, the number of cores that share the same page table base pointer as a thread that has woken up or experienced task switching is counted. In some embodiments, the table is used to map logical IDs to specific heterogeneous cores. The table is indexed by logical ID. Each entry in the table includes a flag indicating whether the logical ID is currently active or stopped, a flag indicating whether SIMD or scalar cores are preferred, a page table base address (e.g., CR3), a value indicating the type of core to which the logical ID is currently mapped, and a counter that limits the migration rate.

[0271] Threads belonging to the same process share the same address space, page table, and page directory base register values.

[0272] In step 1305, a determination is made based on whether the counted number of cores is greater than 1. This count determines whether a thread is in a serial or parallel phase. If the count is 1, then the thread experiencing the event is in the serial phase 1307. As a result, a serial phase thread is a thread that has a unique page directory base register value among all threads in the same core group. Figure 14 shows an example of a serial phase thread. As illustrated, the process has one or more threads, and each process has its own allocated address.

[0273] In 1313 or 1315, if a thread experiencing an event is not assigned to an OOO core, it is migrated to the OOO core, and any existing threads on the OOO core are migrated to a SIMD or scalar core. If a thread experiencing an event is assigned to an OOO core, it remains there in many situations.

[0274] If the count is greater than 1, then at 1309, the thread experiencing the event is in a parallel phase, and the type of parallel phase is determined. If the thread experiencing the event is in a data parallel phase and the thread is not assigned to a SIMD core, then the thread is assigned to a SIMD core; otherwise, at 1313, if it is already there, the thread is kept on a SIMD core.

[0275] When a thread experiencing an event is in the data parallel phase, if the thread is not assigned to a SIMD core, it is assigned to a SIMD core; otherwise, in 1313, if it is already there, the thread is kept on a SIMD core.

[0276] If a thread experiencing an event is in the thread parallel phase and the thread is not assigned to a scalar core, then the thread is assigned to a scalar core; otherwise, if it is already there in 1315, then the thread remains on the scalar core.

[0277] Furthermore, in some embodiments, a flag is set for the thread's logical ID to indicate that the thread is running.

[0278] Figure 15 illustrates an exemplary method for potential core allocation for thread responses to sleep command events. For example, this shows how to determine the phase of a code fragment. At 1501, a sleep event affecting the thread is detected. For example, a stop, wait entry, and timeout or pause command occur. Typically, this detection occurs on the core running the binary translator.

[0279] In some embodiments, at 1503, a flag indicating that a thread is running is cleared for the thread's logical ID.

[0280] In 1505, the number of threads on cores that share the same page table base pointer as the sleep thread is counted. In some embodiments, the table is used to map logical IDs to specific heterogeneous cores. The table is indexed by logical ID. Each entry in the table includes a flag indicating whether the logical ID is currently active or stopped, a flag indicating whether SIMD or scalar cores are preferred, the page table base address (e.g., CR3), a value indicating the type of core to which the logical ID is currently mapped, and a counter that limits the migration rate. We refer to the first executing thread (with any page table base pointer) from the group.

[0281] In step 1507, a decision is made based on whether the OOO core in the system is idle or not. An idle OOO core has no OS threads actively running.

[0282] If the page table base pointer is entirely shared by one thread within a core group, then at 1509, the sharing thread is moved from a SIMD or scalar core to an OOO core. If the page table base pointer is shared by more than one thread, then at 1511, the first executing thread of the aforementioned group is the thread that was moved from a SIMD or scalar core to an OOO core to make space for the released thread (to run in the location of the first executing thread).

[0283] Figure 16 illustrates an exemplary method for possible core allocation for threads in response to phase change events. For example, this shows how to determine the phase of a code fragment. At 1601, a potential phase change event is detected. Typically, this detection occurs on the core running the binary translator.

[0284] At step 1603, a decision is made based on whether the thread's logical ID is valid on the scalar core and whether SIMD instructions exist. If no such SIMD instructions exist, the thread continues execution as normal. However, if SIMD instructions exist in the thread running on the scalar core, the thread is then migrated to the SIMD core at step 1605.

[0285] At step 1607, a decision is made based on whether the thread's logical ID is valid on the SIMD core and whether or not SIMD instructions exist. If SIMD instructions exist, the thread then continues execution as usual. However, if there are no SIMD instructions present in the thread running on the SIMD core, the thread then moves to the scalar core at step 1609.

[0286] As explained throughout this discussion, accelerators accessible from binary translators can offer more efficient execution (including more energy-efficient execution). However, enabling programming for each potentially available accelerator may be a challenging task, if not impossible.

[0287] Embodiments described herein in detail utilize descriptive instructions that explicitly indicate the start and end of potential accelerator-based execution for a portion of a thread. If no accelerator is available, the code between descriptive instructions executes without accelerator use. In some embodiments, the code between these instructions may mitigate some semantics of the core being executed.

[0288] Figure 17 shows an example of code describing an acceleration region. The first instruction in this region is the Acceleration Start (ABEGIN) instruction 1701. In some embodiments, the ABEGIN instruction grants permission to enter a relaxed (sub)mode of execution for non-accelerator cores. For example, in some embodiments, the ABEGIN instruction allows the programmer or compiler to indicate in the instruction's fields which submode features differ from the standard mode. Exemplary features include, but are not limited to, ignoring self-rewriting code (SMC), weakening memory consistency model restrictions (e.g., relaxing storage ordering requests), altering floating-point semantics, altering performance monitoring (perfmon), altering the use of architecture flags, etc. In some embodiments, SMC is a write to a memory location in a code segment currently cached by the processor that invalidates the associated cache line (or line). If the write affects a prefetch instruction, the prefetch queue is invalidated. This latter check is based on the linear address of the instruction. If the target instruction has already been decoded, writing or snooping an instruction within a code segment present in the trace cache invalidates the entire trace cache. The SMC may be ignored by adjusting the SMC detection circuit in the translation lookaside buffer. For example, memory consistency model limits may be changed by changing settings in one or more registers or tables (e.g., memory type range register or page attribute table). For example, when changing floating-point semantics, the way floating-point execution circuits perform floating-point calculations is changed through the use of one or more control registers that control the operation of these circuits (e.g., setting the floating-point arithmetic unit (FPU) control word register). Floating-point semantics that may be changed include, but are not limited to, rounding modes, how exception masks and status flags are handled, flush-to-zero, denormalization settings, and precision (e.g., single precision, double precision, extended precision) controls.Furthermore, in some embodiments, the ABEGIN instruction takes into account an explicit accelerator type preference so that a preferred type of accelerator is selected if one is available.

[0289] Non-accelerator code 1703 follows the ABEGIN instruction 1701. This code is native to the system's processor core. In the worst case, if no accessor is available or ABEGIN is not supported, this code will be executed directly on the core. However, in some embodiments, a submode is used for its execution.

[0290] The presence of the Accelerated Termination (AEND) instruction 1705 causes the execution of the accelerator to be gated on the processor core until it appears that the accelerator has completed its execution. Effectively, the use of ABEGIN and AEND allows the programmer to opt in / out with the execution of the accelerator and / or relaxed mode.

[0291] Figure 18 shows an embodiment of the method for execution using ABEGIN in a hardware processor core. At 1801, the thread's ABEGIN instruction is fetched. As previously mentioned, the ABEGIN instruction typically contains one or more fields used to define different (sub)modes of execution.

[0292] In 1803, the fetched ABEGIN instruction is decoded using a decoding circuit. In some embodiments, the ABEGIN instruction is decoded into a microoperation.

[0293] In 1805, the decoded ABEGIN instruction is executed by the execution circuit in such a way that the thread enters a different mode of execution (which may be explicitly defined by one or more fields of the ABEGIN instruction) for an instruction that follows the ABEGIN instruction but precedes the AEND instruction. This different mode of execution may be on an accelerator or on an existing core, depending on the availability and selection range of accelerators. In some embodiments, the selection of an accelerator is performed by a heterogeneous scheduler.

[0294] In 1807, subsequent non-AEND instructions are executed in a different mode of execution. When an accelerator is used for execution, the instruction may first be translated into a different instruction set by a binary translator.

[0295] Figure 19 shows an embodiment of the method for execution using AEND in a hardware processor core. At 1901, the AEND instruction is fetched.

[0296] In 1903, the fetched AEND instruction is decoded using a decoding circuit. In some embodiments, AEND is decoded into a microoperation.

[0297] In 1905, the decoded AEND instruction is executed by the execution circuit and returns from a different mode of execution previously set by the ABEGIN instruction. This different mode of execution may be on the accelerator or on an existing core, depending on the availability and range of accelerators.

[0298] In 1807, subsequent non-AEND instructions are executed in their original mode of execution. If an accelerator is used for execution, the instruction may first be translated into a different instruction set by a binary translator.

[0299] Figure 124 shows an example of execution when ABEGIN / AEND is not supported. At 12401, the ABEGIN instruction is fetched. At 12403, it is determined that ABEGIN is not supported. For example, CPUID indicates that it is not supported.

[0300] In the absence of support, typically, at 12405, a no-props (NOP) operation that does not modify the context associated with the thread is performed. At 12407, since there is no change in the execution mode, the instruction following the unsupported ABEGIN is executed as usual.

[0301] In some embodiments, an equivalent use of ABEGIN / AEND is implemented using at least pattern matching. This pattern matching may be hardware, software, and / or both. Figure 20 shows a system that provides an ABEGIN / AEND equivalent using pattern matching. The illustrated system includes a scheduler 2015 (e.g., a heterogeneous scheduler as described in detail above) which includes a converter 2001 (e.g., a binary translator, JIT, etc.) stored in memory 2005. Core circuitry 2007 executes the scheduler 2015. The scheduler 2015 receives a thread 2019 which may or may not have an explicit ABEGIN / AEND instruction.

[0302] The scheduler 2015 manages the software-based pattern matcher 2003, performs traps and context switches during offloading, manages the user-space storage area (described in detail later), and generates or translates accelerator code 2011. The pattern matcher 2003 recognizes the (predefined) code sequences stored in memory obtained in incoming threads 2019 that are not written using ABEGIN / AEND, although it may benefit from the use of accelerators and / or relaxed execution states. Typically, each pattern is stored in the converter 2001, but at least accessible by the pattern matcher 2003. The selector 2019 functions as described in detail earlier.

[0303] Scheduler 2015 may offer performance monitoring features. For example, if the code does not have a complete pattern match, Scheduler 2015 may recognize that the code may need further relaxation of requirements to be more efficient and adjust the operating mode associated with the thread as needed. The relationships between operating modes are described in detail above.

[0304] Scheduler 2015 also performs one or more of the following: cycling cores within the ABEGIN / AEND region, cycling accelerators that are activated or stalled, counting ABEGIN calls, delaying accelerator queuing (synchronous processing), and monitoring memory / cache statistics. In some embodiments, binary translator 2001 includes accelerator-specific code used to interpret accelerator code that may be useful in identifying bottlenecks. The accelerator executes this translated code.

[0305] In some embodiments, the core circuit 2007 includes a hardware pattern matcher 2009 that uses stored patterns 2017 to recognize (predefined) code sequences within a received thread 2019. Typically, this pattern matcher 2009 is lighter and seeks out areas that are easier to represent (e.g., rep movs) compared to a software pattern matcher 2003. The recognized code sequences may be converted for use by an accelerator by the scheduler 2015 and / or may result in a relaxation of the operating mode for the thread.

[0306] The system is coupled to one or more accelerators 2013 that receive and execute accelerator code 2011.

[0307] Figure 21 shows an embodiment of the method for a non-accelerated description thread subjected to pattern recognition. The method is performed by a system including at least one type of pattern matcher.

[0308] In some embodiments, a thread is executed at 2101. Typically, this thread runs on a non-accelerator core. Instructions from the running thread are supplied to the pattern matcher. However, instructions from the thread may be supplied to the pattern matcher before any execution.

[0309] In 2103, patterns within a thread are recognized (detected). For example, a software-based pattern matcher or hardware pattern matcher circuit finds patterns that are typically associated with available accelerators.

[0310] In 2105, the recognized pattern is translated for use by an available accelerator. For example, a binary translator translates the pattern into accelerator code.

[0311] The converted code is transferred to an accelerator available on 2107 for execution.

[0312] Figure 22 shows an embodiment of the method for a non-accelerated description thread subjected to pattern recognition. This method is performed by a system including at least one type of pattern matcher, as shown in the system of Figure 20.

[0313] In some embodiments, a thread is executed at 2201. Typically, this thread runs on a non-accelerator core. Instructions from the running thread are supplied to the pattern matcher. However, instructions from the thread may be supplied to the pattern matcher before any execution.

[0314] In 2203, patterns within a thread are recognized (detected). For example, a software-based pattern matcher or hardware pattern matcher circuit finds patterns that are typically associated with available accelerators.

[0315] In 2205, the binary translator adjusts the operating mode associated with the thread to use the mitigation request based on the recognized pattern. For example, the binary translator utilizes the settings associated with the recognized pattern.

[0316] As described in detail, in some embodiments, parallel regions of code are demarcated by ABEGIN and AEND instructions. Within ABEGIN / AEND blocks, independence of specific memory load and store operations is guaranteed. Other loads and stores consider potential dependencies. This allows implementations to parallelize blocks with little or no checks for memory dependencies. In any case, serial execution of a block is permitted because it is included in the possible ways of executing the block. The binary translator performs static dependency analysis to create instances of parallel execution and maps these instances to hardware. Static dependency analysis may parallelize iterations of outer, middle, or inner loops. Slicing is implementation-dependent. The implementation of ABEGIN / AEND extracts parallelism in the size most appropriate for the implementation.

[0317] ABEGIN / AEND blocks may contain multiple levels of nested loops. Implementations are free to choose the amount of parallel execution supported or fall back to serial execution. ABEGIN / AEND provides parallelism over a much larger area than SIMD instructions. For certain types of code, ABEGIN / AEND enables more efficient hardware implementations than multithreading.

[0318] Through the use of ABEGIN / AEND, programmers and / or compilers can fall back to conventional serial execution by the CPU core if the parallelization criteria are not met. When running on conventional out-of-order CPU cores, ABEGIN / AEND reduces the power requirements of memory areas and memory ordering buffers (MOBs) as a result of relaxed memory ordering.

[0319] Within an ABEGIN / AEND block, the programmer defines memory dependencies. Figure 23 illustrates various types of memory dependencies 2301, their semantics 2303, ordering requirements 2305, and use cases 2307. Furthermore, some semantics apply to instructions within an ABEGIN / AEND block, depending on the implementation. For example, in some embodiments, register dependencies are permitted, but modifications to registers do not persist beyond AEND. Additionally, in some embodiments, an ABEGIN / AEND block must be entered at ABEGIN and terminated at AEND (or similarly entered based on pattern recognition), without any branches to or from the ABEGIN / AEND block. Finally, typically, instruction streams cannot be modified.

[0320] In some embodiments, the ABEGIN instruction includes a source operand containing a pointer to a memory data block. This data memory block contains a lot of information that is used by the runtime and core circuitry to process the code within the ABEGIN / AEND block.

[0321] Figure 24 shows an example of a memory data block pointed to by the ABEGIN instruction. As shown, depending on the implementation, the memory data block includes fields for sequence number 2401, block class 2403, implementation identifier 2405, state storage area size 2407, and local storage area size 2409.

[0322] Sequence number 2401 indicates how much (parallel) computation the processor has performed before the interrupt. The software initializes sequence number 2401 to zero before executing ABEGIN. ABEGIN tracks the progress of execution by writing non-zero values ​​to sequence number 2401. When complete, AEND writes zero to it, reinitializing sequence number 2401 for its next use.

[0323] A predefined block class identifier 2403 (i.e., GUID) specifies a predefined ABEGIN / AEND block class. For example, DMULADD and DGEMM may be predefined as block classes. Using a predefined class, a binary translator does not need to parse the binary to perform mapping analysis for heterogeneous hardware. Instead, the converter (e.g., binary translator) performs the pre-generated conversion for this ABEGIN / AEND class by simply obtaining the input values. The code enclosed with ABEGIN / AEND simply functions as code used to execute this class in a non-specialized core.

[0324] The implementation ID field 2405 indicates the type of execution hardware used. ABEGIN execution updates this field 2405 to indicate the type of heterogeneous hardware used. This helps implementations migrate ABEGIN / AEND code to machines with different acceleration hardware types or no accelerators at all. This field allows for possible transformations of the stored context to adapt to the target implementation. Specifically, the emulator is used to execute the code after an interruption and migration to a machine without the same accelerator type, until it exits AEND. This field 2405 also allows the system to dynamically reallocate ABEGIN / AEND blocks to different heterogeneous hardware within the same machine, even if interrupted during execution of the ABEGIN / AEND block.

[0325] The state storage area field 2407 indicates the size and format of the state storage area, which are implementation-specific. An implementation ensures that the implementation-specific portion of the state storage area does not exceed a certain maximum value specified in CPUID. Typically, the execution of an ABEGIN instruction triggers writes to the state storage area of ​​general-purpose and packed data registers, which are modified within the ABEGIN / AEND block, associated flags, and additional implementation-specific states. Multiple instances of registers may be written to facilitate parallel execution.

[0326] Local storage area 2409 is allocated as a local storage area. The amount of storage to reserve is typically specified as an immediate operand to ABEGIN. When the ABEGIN instruction is executed, a write operation is performed to a specific register (e.g., R9) using the address of local storage 2409. In case of failure, this register is made to point to a sequence number.

[0327] Each instance of parallel execution receives a unique local storage area 2409. The address differs for each instance of parallel execution. For serial execution, one storage area is allocated. Local storage area 2409 provides temporary storage beyond the architecture's general-purpose and packed data registers. Local storage area 2409 should not be accessed outside of the ABEGIN / AEND blocks.

[0328] Figure 25 shows an example of memory 2503 configured to use ABEGIN / AEND semantics. Hardware that supports ABEGIN / AEND and utilizes this memory 2503 (e.g., various processing elements described herein) is not shown. As will be described in detail, memory 2503 includes a stored state area 2507 containing indications of the registers 2501, flags 2505, and implementation-specific information 2511 to be used. In addition, local storage 2509 is stored in memory 2503 for each parallel execution instance.

[0329] Figure 26 shows examples of different modes of operation for execution using ABEGIN / AEND. Typically, this method is performed by a combination of entities, such as a converter and an execution circuit. In some embodiments, threads are converted before entering this mode.

[0330] In 2601, different execution modes include, for example, relaxed execution modes (with or without accelerators). Normally, this mode is entered through the execution of the ABEGIN instruction. However, as described in detail above, this mode can also be entered through pattern matching. Entering this mode involves a reset of the sequence number.

[0331] At 2603, a write operation is performed to the saved state area. For example, general-purpose and packed data registers to be modified, associated flags, and additional implementation-specific information are written to this area. This area allows for the resumption or rollback of execution in the event of a malfunction (e.g., an interrupt) within the block.

[0332] In version 2605, a local storage area is reserved for each parallel execution instance. As described in detail above, the size of this area is indicated by the state storage area field.

[0333] In 2607, the progress of a block is tracked while it is being executed. For example, if an instruction is successfully executed and then retired, the block's sequence number is updated.

[0334] At 2609, a decision is made depending on whether the AEND instruction has been reached (for example, to determine whether the block has completed). If the AEND instruction has not been reached, then at 2613, the local storage area is updated with the intermediate results. Execution retrieves from these results if possible. However, in some cases, at 2615, a rollback to before ABEGIN / AEND occurs. For example, if an exception or interrupt occurs during the execution of the ABEGIN / AEND block, the instruction pointer points to the ABEGIN instruction, and the R9 register points to the memory data block to be updated with the intermediate results. Upon resumption, the state stored in the memory data block is used to resume at the correction point. Furthermore, if the initial part of the memory data block containing the state storage area does not exist or is inaccessible, a page fault is triggered. For loads and stores to the local storage area, the usual method is used, i.e., a page fault is reported on the first access to a page that does not exist or is inaccessible. In some cases, non-accelerated processing elements are used upon resumption.

[0335] In 2611, if the block completes successfully, the discarded registers are then restored to their original state along with the flags. Only the memory state differs after the block.

[0336] Figure 27 shows examples of different modes of operation for execution using ABEGIN / AEND. Typically, this method is performed by a combination of entities, such as a binary converter and an execution circuit.

[0337] In 2701, different execution modes include, for example, relaxed execution modes (with or without accelerators). Normally, this mode is entered through the execution of the ABEGIN instruction. However, as detailed above, this mode can also be entered through pattern matching. Entering this mode involves a reset of the sequence number.

[0338] At 2703, a write operation is performed to the saved state area. For example, the general-purpose and packed data registers to be modified, associated flags, and additional implementation-specific information are written there. This area allows for the resumption or rollback of execution in the event of a malfunction (e.g., an interrupt) within the block.

[0339] In version 2705, a local storage area is reserved for each parallel execution instance. As detailed above, the size of this area is indicated by the state storage area field.

[0340] In 2706, the code within the block is transformed for execution.

[0341] In 2707, the progress of a converted block is tracked during its execution. For example, if an instruction successfully executes and then retires, the block's sequence number is updated.

[0342] At 2709, a decision is made depending on whether the AEND instruction has been reached (for example, to determine whether the block has completed). If the AEND instruction has not been reached, then at 2713, the local storage area is updated with the intermediate results. Execution retrieves from these results if possible. However, in some cases, at 2715, a rollback to before ABEGIN / AEND occurs. For example, if an exception or interrupt occurs during the execution of the ABEGIN / AEND block, the instruction pointer points to the ABEGIN instruction, and the R9 register points to the memory data block to be updated with the intermediate results. Upon resumption, the state stored in the memory data block is used to resume at the correction point. Furthermore, if the initial part of the memory data block, including the state storage area, does not exist or is inaccessible, a page fault is triggered. For loads and stores to the local storage area, the usual method is used, i.e., a page fault is reported on the first access to a page that does not exist or is inaccessible. In some cases, non-accelerated processing elements are used upon resumption.

[0343] If the block completes successfully, then at 2711, the discarded registers are restored to their original state along with the flags. Only the memory state differs after the block.

[0344] As described above, in some embodiments, a common link (calling a Multiprotocol Common Link (MCL)) is used to reach devices (e.g., processing elements described in Figures 1 and 2). In some embodiments, these devices are seen as PCI Express (PCIe) devices. This link has three or more protocols dynamically multiplexed on the link. For example, the common link supports protocols consisting of: 1) a Producer / Consumer, Discover, Configure, Interrupt (PDCI) protocol that enables device discovery, device configuration, error reporting, interrupts, DMA-style data transfer and various services, as may be defined in one or more proprietary or industry standards (e.g., PCI Express Specification or equivalent alternatives); 2) a Caching Agent Coherence (CAC) protocol that enables devices to issue coherent read and write requests to processing elements; and 3) a Memory Access (MA) protocol that enables processing elements to access the local memory of another processing element. While specific examples of these protocols are provided (e.g., Intel® On-Chip System Fabric (IOSF), In-Die Interconnect (IDI), Scalable Memory Interconnect 3+ (SMI3+)), the underlying principles of the present invention are not limited to any particular set of protocols.

[0345] Figure 120 is a simplified block diagram 12000 showing an exemplary multichip configuration 12005 including two or more chips or dies (e.g., 12010, 12015) that are communicated together using an exemplary multichip link (MCL) 12020. While Figure 120 shows an example of two (or more) dies interconnected using an exemplary MCL 12020, it should be understood that the principles and features described herein with respect to the implementation of MCLs can be applied to any interconnection or link connecting dies (e.g., 12010) and other components, including, among many potential examples, connecting two or more dies (e.g., 12010, 12015), connecting dies (or chips) to other component off-dies, connecting dies to other devices or die off-packages (e.g., 12005), and connecting dies (e.g., 12010) and other components, including BGA packages, patch implementations on interposers (POINT).

[0346] In some examples, larger components (e.g., dies 12010, 12015) may themselves be IC systems, such as a system-on-a-chip (SoC), a multiprocessor chip, or other components on a device, for example, on a single die (e.g., 12010, 12015), including multiple components (12026-12030 and 12040-12045) such as cores and accelerators. MCL12020 offers flexibility for building complex and diverse systems from potentially multiple separate components and systems. For example, dies 12010 and 12015 may each be manufactured, or otherwise supplied by two different entities. Furthermore, the dies and other components may include interconnects or other communication fabrics (e.g., 12031, 12050) that themselves provide infrastructure for communication between components (e.g., 12026-12030 and 12040-12045) within the device (e.g., 12010, 12015, respectively). The various components and interconnects (e.g., 12031, 12050) may support or use multiple different protocols. In addition, communication between the dies (e.g., 12010, 12015) may potentially involve transactions between various components on the die via multiple different protocols.

[0347] The Multi-Chip Link (MCL) embodiment supports multiple package options, multiple I / O protocols, and reliability, availability, and maintainability (RAS) features. Furthermore, the physical layer (PHY) can include physical electrical and logical layers and can support longer channel lengths, including, in some cases, channel lengths exceeding approximately 45 mm. In some embodiments, exemplary MCLs can operate at high data rates, including data rates exceeding 8–10 Gb / s.

[0348] In one exemplary embodiment of MCL, the PHY electrical layer improves upon conventional multi-channel interconnection solutions (e.g., multi-channel DRAM I / O) and extends data rates and channel configurations, among many other potential examples, particularly with numerous features including, for example, coordinated intermediate rail termination, low-power active crosstalk rejection, circuit redundancy, bit-by-bit duty cycle correction and deskew, line coding and transmitter equalization.

[0349] In one exemplary embodiment of MCL, the PHY logic layer is implemented to further support the ability to extend data rates and channel configurations while also enabling interconnection that transmits multiple protocols across the electrical layer (e.g., electrical layer functionality). Such embodiments provide and define a modular common physical layer designed to be protocol-independent and potentially work with any existing or future interconnection protocol.

[0350] Referring to Figure 121, the simplified block diagram 12100 represents at least a portion of a system including an exemplary implementation of a multi-chip link (MCL). The MCL may be implemented using physical electrical connections (e.g., wires implemented as lanes) connecting a first device 12105 (e.g., a first die containing one or more subcomponents) to a second device 12110 (e.g., a second die containing one or more other subcomponents). In the specific example shown in the high-level representation of Figure 12100, all signals (within channels 12115, 12120) may be unidirectional, and lanes may provide data signals having both upstream and downstream data transfer. The block diagram 12100 in Figure 121 refers to the first component 12105 as an upstream component, the second component 12110 as a downstream component, the physical lanes of the MCL used when transmitting data as a downstream channel 12115, and the lanes used to receive data (from component 12110) as an upstream channel 12120. It should be understood that the MCL between devices 12105 and 12110 may be used by each device to both transmit and receive data between them.

[0351] In one exemplary embodiment, the MCL can provide a physical layer including electrical MCL physical layers (PHYs) 12125a,b (or collectively referred to as 12125) and executable logic implementation MCL logic PHYs 12130a,b (or collectively referred to as 12130). The electrical or physical PHY 12125 provides the physical connection over which data is communicated between devices 12105, 12110. Signaling and regulating components and logic can be implemented in conjunction with the physical PHY 12125 to establish high data rate and channel configuration capabilities for the link, some applications relating to tightly clustered physical connections over lengths of approximately 45 mm or longer. The logic PHY 12130 includes a clock, link state management (e.g., link layers 12135a, 12135b) and circuitry to facilitate protocol multiplexing between potentially multiple different protocols used for communication over the MCL.

[0352] In one exemplary embodiment, the physical PHY 12125 includes a set of data lanes for each channel (e.g., 12115, 12120) through which in-band data is transmitted. In this specific example, 50 data lanes are provided for each of the upstream and downstream channels 12115, 12120, but other numbers of lanes may be used if permitted by layout and power constraints, desired application, device constraints, etc. Each channel may further include one or more dedicated lanes for channel-related strobe or clock signals, one or more dedicated lanes for channel-related active signals, one or more dedicated lanes for stream signals, and one or more dedicated lanes for link state machine management or sideband signals. The physical PHY may further include a sideband link 12140, which in some examples, among many others, may be a bidirectional low-frequency control signal link used to coordinate state transitions and other attributes of the MCL connecting devices 12105, 12110.

[0353] As described above, multiple protocols are supported using the MCL implementation. In practice, multiple independent transaction layers 12150a, 12150b may be provided in each device 12105, 12110. For example, each device 12105, 12110 may support and utilize two or more protocols, including PCI, PCIe, and CAC. CAC is an on-die coherent protocol used to communicate between the core, the last-level cache (LLC), memory, graphics, and I / O controller. Other protocols, including Ethernet® protocol, InfiniBand protocol, and other PCIe fabric-based protocols, may also be supported. Among many examples, combinations of logical and physical PHYs can also be used as inter-die interconnects connecting a SerDes PHY (PCIe, Ethernet®, InfiniBand, or other high-speed SerDes) on one die to its higher layers implemented on another die.

[0354] The logic PHY 12130 supports multiplexing between these multiple protocols in the MCL. For example, a dedicated stream lane may be used to assert an encoded stream signal that identifies which protocol applies to data transmitted substantially simultaneously on the channel's data lane. Furthermore, the logic PHY 12130 negotiates with various types of link state transitions that various protocols may support or require. In some examples, an LSM_SB signal transmitted via a channel's dedicated LSM_SB lane may be used in conjunction with a sideband link 12140 to communicate and negotiate link state transitions between devices 12105 and 12110. In addition, other functions for link training, error detection, skew detection, deskue, and conventional interconnects may be replaced or controlled in part using the logic PHY 12130. For example, a valid signal transmitted via one or more dedicated valid signal lanes in each channel may be used, among many other examples, to signal link activity, detect skew and link errors, and implement other features. In the specific example shown in Figure 121, multiple active lanes are provided for each channel. For example, data lanes within a channel may be bundled or clustered (physically and / or logically), and active lanes may be provided for each cluster. Furthermore, multiple straw lanes may be provided in some cases, and in particular among many examples, to provide dedicated strobe signals for each cluster within multiple data lane clusters in a channel.

[0355] As described above, the logic PHY 12130 negotiates and manages link control signals transmitted between devices connected by the MCL. In some embodiments, the logic PHY 12130 includes a link-layer packet (LLP) generation circuit 12160 that transmits (i.e., in-band) link-layer control messages via the MCL. Such messages may be transmitted via a channel's data lane, which has a stream lane that identifies the data as link-layer messaging, such as link-layer control data, among many other examples. Link-layer messages enabled with the LLP module 12160 assist in the negotiation and operation of link-layer state transitions, power management, loopback, disable, and recentering scramble, among other link-layer features between devices 12105, 12110, 12135a, and 12135b, respectively.

[0356] Referring to Figure 122, a simplified block diagram 12200 is shown illustrating an exemplary logic PHY of an exemplary MCL. The physical PHY 12205 may be connected to a die containing the logic PHY 12210 and additional logic supporting the link layer of the MCL. In this example, the die may further include logic supporting multiple different protocols on the MCL. For example, in the example in Figure 122, PCIe logic 12215 is provided together with CAC logic 12220, and as a result, the dies can communicate using either PCIe or CAC over the same MCL connecting the two dies, among many potentially numerous examples, including examples where more than two protocols, or protocols other than PCIe and CAC, are supported via the MCL. The various protocols supported between the dies can provide varying levels of service and features.

[0357] The logic PHY 12210 may include link state machine management logic 12225 for negotiating link state transitions in relation to requests from higher layer logic of the die (e.g., received via PCIe or CAC). In some embodiments, the logic PHY 12210 may further include link test and debug logic (e.g., 12230). As described above, exemplary MCLs can support control signals transmitted between dies via the MCL to facilitate high-performance and power-efficient functionality, independent of the MCL protocol (in particular among its many exemplary functions). For example, the logic PHY 12210 can support the generation and transmission, as well as reception and processing of valid signals, stream signals, and LSM sideband signals in relation to the transmission and reception of data via dedicated data lanes, as described in the example above.

[0358] In some embodiments, multiplexing (e.g., 12235) and demultiplexing (e.g., 12240) logic may be contained within or otherwise accessible from the logical PHY 12210. For example, multiplexing logic (e.g., 12235) may be used to identify data (e.g., embodied as packets, messages, etc.) transmitted over the MCL. Multiplexing logic 12235 can identify the protocol governing the data and generate an encoded stream signal to identify the protocol. For example, in one exemplary embodiment, the stream signal may be encoded as two 1-byte hexadecimal symbols (e.g., CAC:FFh;PCIe:F0h;LLP:AAh;Sideband:55h, etc.) and transmitted within the same window (e.g., a 1-byte time period window) for data governed by the identified protocol. Similarly, the demultiplexing logic 12240 can be used to interpret the incoming stream signal, decode the stream signal, and identify the protocol applied to the data received simultaneously with the stream signal on the data lane. The demultiplexing logic 12240 can then apply (or ensure) the link layer processing specific to the protocol, allowing the data to be processed by the corresponding protocol logic (e.g., PCIe logic 12215 or CAC logic 12220).

[0359] The logic PHY 12210 may further include link-layer packet logic 12250, which can be used to handle various link control functions, including power management tasks, loopback, disable, recentering, and scrambling. Among its many functions, the LLP logic 12250 can facilitate link-layer-to-link-layer messaging via MCLP. Data corresponding to LLP signaling can also be identified by stream signals transmitted on a dedicated stream signal lane encoded to identify the LLP data in its data lane. Multiplexing and demultiplexing logics (e.g., 12235, 12240) can also be used to generate and interpret stream signals corresponding to LLP traffic and to have such traffic processed by appropriate die logic (e.g., LLP logic 12250). Similarly, some embodiments of MCLP may include dedicated sidebands (e.g., sideband 12255 and support logic), such as asynchronous and / or low-frequency sideband channels, among many other examples.

[0360] The logic PHY logic 12210 may further include link state machine management logic capable of generating and receiving (and using) link state management messaging via a dedicated LSM sideband lane. For example, among many potential examples, the LSM sideband lane may be used to perform handshakes to proceed to a link training state and to exit a power management state (e.g., L1 state). LSM sideband signals may be asynchronous signals in that they do not align with the link's data, enable, and stream signals, but instead correspond to signaling state transitions and coordinate the link state machine between two dies or chips connected by the link. Providing a dedicated LSM sideband lane may, in some examples, allow for the elimination of conventional squelch and receive detection circuitry in the analog front end (AFE), among many exemplary benefits.

[0361] Referring to Figure 123, the simplified block diagram 12300 illustrates another representation of the logic used to implement the MCL. As an example, the logic PHY 12210 is provided with a defined logic PHY interface (LPIF) 12305 that can interface with the physical layer of the exemplary MCL to one of several different protocols (e.g., PCIe, CAC, PDCI, MA, etc.) 12315, 12320, 12325 and a signaling mode (e.g., sideband). In some embodiments, the multiplexing and arbitration logic 12330 can also be provided as a separate layer from the logic PHY 12210. In one example, the LPIF 12305 may be provided as an interface on both sides of this MuxArb layer 1230. The logic PHY 12210 can interface with the physical PHY (e.g., the analog front-end (AFE) 12205 of the MCL PHY) through another interface.

[0362] LPIF allows for the removal of PHYs (logic and electrical / analog) from the upper layers (e.g., 12315, 12320, 12325) so that completely different PHYs can be implemented beneath the transparent LPIF. This helps promote modularity, allows for reuse in the design, and, among many other examples, ensures that the upper layers remain intact when the underlying signaling technology PHY is updated. Furthermore, LPIF can define numerous signals to enable multiplexing / demultiplexing, LSM management, error detection and handling, and other functions of the logic PHY. As an example, the following table summarizes at least some of the signals that can be defined for an exemplary LPIF. [Table 3]

[0363] As mentioned in the table, in some embodiments, the alignment mechanism may be provided through the AlignReq / AlignAck handshake. For example, if the physical layer enters recovery mode, some protocols may lose packet framing. Packet alignment may be corrected, for example, to ensure corrected framing identification by the link layer. If the physical layer enters recovery mode, it may assert a StallReq signal, and as a result, the link layer asserts a stall signal when it is ready to forward the newly aligned packet. Physical layer logic may sample both stall and valid to determine whether a packet is aligned or not. Among the many potential implementations, including other alternative implementations that use valid to assist packet alignment, the physical layer may continue driving to eject link layer packets until stall and valid have been sampled and asserted.

[0364] Various fault tolerances can be defined for signals on the MCL. For example, fault tolerances can be defined for valid, stream, LSM sideband, low-frequency sideband, link-layer packets, and other types of signals. Fault tolerance for packets, messages, and other data transmitted through dedicated data lanes on the MCL may be based on a specific protocol governing the data. In some embodiments, error detection and handling mechanisms may be provided, among many potential examples, cyclic redundancy checks (CRC) and retry buffers. For example, with respect to PCIe packets transmitted through the MCL, a 32-bit CRC may be used for PCIe transaction layer packets (TLPs) (with guaranteed delivery, e.g., through a regeneration mechanism), and a 16-bit CRC may be used for PCIe link-layer packets (which may be designed to have high loss, e.g., no regeneration is applied). Furthermore, with respect to PCIe framing tokens, a specific Hamming distance (e.g., a 4(4) Hamming distance) may be defined for the token identifier, among many other examples, and parity and a 4-bit CRC may also be used. On the other hand, for CAC packets, a 16-bit CRC may be used.

[0365] In some embodiments, fault tolerance is defined for link layer packets (LLPs) that utilize a valid signal to transition from low to high (i.e., 0 to 1) (e.g., to support guarantee bits and symbol locking). Furthermore, in one example, among several defined characteristics that can be used as a basis for determining faults in LLP data on the MCL, a certain number of consecutive identical LLPs may be defined to be transmitted, and responses may be expected for each request, with a requester that retries after a timeout. In further examples, fault tolerance may be induced for a valid signal, and spread across a time-period window or entire symbol through the valid signal (e.g., by keeping a valid signal high for eight UIs). Furthermore, errors or faults in a stream signal may be prevented, among several examples, by maintaining a Hamming distance with respect to the encoding value of the stream signal.

[0366] The implementation of the logical PHY includes error detection, error reporting, and error handling logic. In some embodiments, the exemplary MCL's logical PHY may include logic to detect PHY layer deframing errors (e.g., on active and stream lanes), sideband errors (e.g., related to LSM state transitions), and errors within the LLP (e.g., critical to LSM state transitions), among many other examples. Some error detection / resolution may be delegated to higher-layer logic, such as PCIe logic adapted to detect PCIe-specific errors, among many other examples.

[0367] In the case of a deframing error, in some embodiments, one or more mechanisms may be provided through error handling logic. Deframing errors may be handled based on the relevant protocol. For example, in some embodiments, the link layer may signal the error to trigger a retry. Deframing may also cause a re-realignment of the logic PHY deframing. Furthermore, among many techniques, re-centering of the logic PHY may be performed, and the symbol / window lock may be reacquired. Centering may, in some examples, include a PHY that moves the receiver clock phase to the optimal point for detecting incoming data. In this context, "optimal" may mean having the most margin for noise and clock jitter. Among many examples, re-centering may include a simplified centering function that is performed, for example, when the PHY wakes up from a low-power state.

[0368] Other types of errors may be associated with other error handling techniques. For example, errors detected in the sideband may be caught through a timeout mechanism for the corresponding state (e.g., in an LSM). The error may be logged, and the link state machine may then transition to a reset. The LSM can remain in the reset state until a restart command is received from the software. In another example, an LLP error, such as a link control packet error, may be handled using a timeout mechanism that allows the LLP sequence to be restarted if no acknowledgment for the LLP sequence is received.

[0369] In some embodiments, each of the above protocols is a variation of PCIe. PCIe devices communicate using a common address space associated with the bus. This address space is either the bus address space or the PCIe address space. In some embodiments, PCIe devices use addresses in an internal address space that may differ from the PCIe address space.

[0370] The PCIe specification defines a mechanism that allows PCIe devices to expose their local memory (or a portion of it) to the bus, thereby enabling the CPU, or other devices attached to the bus that directly access that memory. Typically, each PCIe device is allocated a dedicated area within the PCIe address space called the PCI Base Address Register (BAR). Furthermore, the addresses exposed by the device are mapped to their respective addresses within the PCI BAR.

[0371] In some embodiments, a PCIe device (e.g., an HCA) uses an input / output memory mapping unit (IOMMU) to translate its internal address to a PCIe bus address. In other embodiments, a PCIe device may use a PCI Address Translation Service (ATS) to perform address translation and resolution. In some embodiments, tags, such as a processing address space ID (PASID) tag, are used to define addresses that are translated to belong to the virtual address space of a particular processing.

[0372] Figure 28 shows additional details relating to one embodiment. As shown in the embodiment described above, this embodiment includes an accelerator 2801 having an accelerator memory 2850 coupled via a multiprotocol link 2800 to a host processor 2802 having a host memory 2860. As already mentioned, the accelerator memory 2850 may utilize a different memory technology than the host memory 2860 (for example, the accelerator memory may be HBM or stacked DRAM, while the host memory may be SDRAM).

[0373] Multiplexers 2811 and 2812 are shown to highlight the fact that the multiprotocol link 2800 is a dynamically multiplexed bus that supports PCDI, CAC, and MA protocol (e.g., SMI3+) traffic, each of which can be forwarded to different functional components within accelerator 2801 and host processor 2802. These protocols may include, but are not limited to, IOSF, IDI, and SMI3+. In one embodiment, the PCIe logic 2820 of accelerator 2801 includes a local TLB 2822 for caching virtual-to-physical address translation for use by one or more accelerator cores 2830 when executing commands. As already mentioned, virtual memory space is distributed between accelerator memory 2850 and host memory 2860. Similarly, the PCIe logic on the host processor 2802 includes an I / O memory management unit (IOMMU) 2810 for managing memory access to the PCIe I / O device 2806, and in one embodiment, an accelerator 2801. As shown in the figure, the PCIe logic 2820 on the accelerator and the PCIe logic 2808 on the host processor communicate using the PCDI protocol to perform functions such as device discovery, register access, device configuration and initialization, interrupt handling, DMA processing, and address translation services (ATS). As already mentioned, the IOMMU 2810 on the host processor 2802 may operate primarily for the purpose of controlling and coordinating these functions.

[0374] In one embodiment, the accelerator core 2830 includes a processing engine (element) that performs functions required by the accelerator. Furthermore, the accelerator core 2830 may include a host memory cache 2834 for locally caching pages stored in host memory 2860 and an accelerator memory cache 2832 for caching pages stored in accelerator memory 2850. In one embodiment, the accelerator core 2830 communicates with the coherence and cache logic 2807 of the host processor 2802 via the CAC protocol to ensure that the cache lines shared between the accelerator 2801 and the host processor 2802 maintain coherence.

[0375] The bias / coherence logic 2840 of accelerator 2801 implements various device / host biasing techniques described herein (e.g., page-level granularity) to ensure data coherence while reducing unnecessary communication over the multiprotocol link 2800. As illustrated, the bias / coherence logic 2840 communicates with the coherence and cache logic 2807 of host processor 2802 using MA memory transactions (e.g., SMI3+). The coherence and cache logic 2807 is responsible for maintaining the coherence of data stored in its LLC 2809, host memory 2860, accelerator memory 2850 and caches 2832, 2834, and the separate caches of core 2805.

[0376] In summary, one embodiment of accelerator 2801 appears as a PCIe device to software running on host processor 2802 and is accessed via the PDCI protocol (effectively reformatting the PCIe protocol for multiplexed buses). Accelerator 2801 may participate in shared virtual memory using the accelerator device TLB and standard PCIe Address Translation Service (ATS). The accelerator may also be treated as a coherence / memory agent. Certain functions (e.g., ENQCMD, MOVDIR, described below) are available on PDCI (e.g., for work submission), while the accelerator may cache host data on the accelerator and in certain bias transition flows using CAC. Access from the host to accelerator memory (or host bias access from the accelerator) may use the MA protocol as described.

[0377] As shown in Figure 29, in one embodiment, the accelerator includes PCI configuration registers 2902 and MMIO registers 2906, which can be programmed to provide access to a device backend resource 2905. In one embodiment, the base address for MMIO register 2906 is identified by a set of base address registers (BARs) 2901 in PCI configuration space. Unlike previous implementations, the embodiment of the data streaming accelerator (DSA) described herein does not implement multiple channels or PCI functionality, and therefore the device has only one instance for each register. However, there may be more than one DSA device on a single platform.

[0378] Examples may provide additional performance not described herein, or registers may be debugged. It should be considered that any such registers will be implementation-specific.

[0379] PCI configuration space access is performed as aligned 1-byte, 2-byte, or 4-byte access. For rules regarding accessing unimplemented registers and reserved bits in PCI configuration space, refer to the PCI Express-based specification.

[0380] MMIO space access to the BAR0 area (function, configuration, and status registers) shall be performed as aligned 1-byte, 2-byte, 4-byte, or 8-byte accesses. 8-byte access should only be used for 8-byte registers. Software should not read from or write to unimplemented registers. MMIO space access to the BAR2 and BAR4 areas shall be performed as 64-byte accesses using the ENQCMD, ENQCMDS, or MOVDIR64B instructions (described in detail below). ENQCMD or ENQCMDS should be used to access work queues (SWQs) configured to be shared, and MOVDIR64B must be used to access work queues (DWQs) configured to be dedicated.

[0381] One embodiment of the DSA PCI configuration space implements three 64-bit BAR2901s. The device control register (BAR0) is a 64-bit BAR containing the physical base address of the device control register. These registers provide information about device performance, device configuration and enablement controls, and device status. The size of the BAR0 area depends on the size of the interrupt message storage 2904. The size is calculated by adding the number of interrupt message storage entries 2904 × 16 to 32KB and rounding up to the next power of 2. For example, if the device supports 1024 interrupt message storage entries 2904, the interrupt message storage is 16KB and the size of BAR0 is 64KB.

[0382] BAR2 is a 64-bit BAR containing the physical base addresses of privileged and unprivileged portals. Each portal is 64 bytes in size and occupies a separate 4KB page. This allows portals to be independently mapped to different address spaces using the CPU page table. Portals are used to submit descriptors to the device. Privileged portals are used by kernel-mode software, and unprivileged portals are used by user-mode software. The number of unprivileged portals is equal to the number of supported work queues. The number of privileged portals is the number of work queues (WQs) × (size of the MSI-X table - 1). The address of the portal used to submit a descriptor allows the device to determine which WQ to place the descriptor in, whether the portal is privileged or unprivileged, and which MSI-X table entries can be used for completion interrupts. For example, if the device supports 8 WQs, the WQ for a given descriptor is (portal address >> 12) and 0x7. If portal address >> 15 is 0, the portal is unprivileged. Otherwise, the portal is privileged, and the MSI-X2903 table index used for completion interrupts is portal address >> 15. Bits 5:0 must be 0. Bits 11:6 are ignored. Therefore, any 64-byte aligned address on the page can be used with the same effect.

[0383] When configured using the Work Queue Configuration (WQCFG) register, descriptor submissions using an unprivileged portal target the WQ's occupancy threshold. Descriptor submissions using a privileged portal do not target this threshold. Descriptor submissions to SWQs must be submitted using ENQCMD or ENQCMDS. Other write operations to SWQ portals are ignored. Descriptor submissions to DWQs must be submitted using a 64-byte write operation. The software uses MOVDIR64B to ensure a continuous 64-byte write. ENQCMD or ENQCMDS to a disabled or dedicated WQ portal returns a retry. Other write operations to DWQ portals are ignored. Any read operations to the BAR2 address space return all 1s. Kernel-mode descriptors should be submitted using a privileged portal to receive a completion interrupt. If a kernel-mode descriptor is submitted using an unprivileged portal, there is no completion interrupt that can be requested. User-mode descriptors may be submitted using either a privileged or unprivileged portal.

[0384] The number of portals in the BAR2 area is the number of WQs supported by the device multiplied by the size of the MSI-X2903 table. The size of the MSI-X table is typically the number of WQs plus 1. Therefore, for example, if the device supports 8 WQs, the useful size of BAR2 would be 8 × 9 × 4KB = 288KB. The total size of BAR2 would be rounded up to the next power of 2, or 512KB.

[0385] BAR4 is a 64-bit BAR containing the physical base address of the guest portal. Each guest portal is 64 bytes in size and resides in a separate 4KB page. This allows portals to be independently mapped to different address spaces using the CPU Extended Page Table (EPT). If the Interrupt Message Storage Support field in GENCAP is 0, this BAR is not implemented.

[0386] A guest portal may be used by guest kernel-mode software to submit a descriptor to the device. The number of guest portals is the number of entries in the interrupt message storage multiplied by the number of supported WQs. The address of the guest portal used to submit the descriptor allows the device to determine the WQ for the descriptor and allows the interrupt message storage entry to be used to generate a completion interrupt for descriptor completion (if it is a kernel-mode descriptor and the request completion interrupt flag is set for the descriptor). For example, if the device supports eight WQs, the WQs for a given descriptor are (guest portal address >> 12) and 0x7, and the interrupt table entry index used for the completion interrupt is guest portal address >> 15.

[0387] In one embodiment, MSI-X is only PCIe interrupt functionality provided by the DSA, and where the DSA does not implement legacy PCI interrupts or MSI. Details of this register structure are subject to the PCI Express specification.

[0388] In one embodiment, three PCI Express functions control address translation. Only specific combinations of values ​​for these functions may be supported, as shown in Table A. The values ​​are checked when the enable bit in the general control register (GENCTRL) is set to 1. [Table 4]

[0389] If any of these functions are modified by software while the device is enabled, the device may shut down and an error will be reported to the software error register.

[0390] In one embodiment, the software configures a PASID function to control whether the device performs address translation using PASID. When PASID is disabled, only physical addresses may be used. When PASID is enabled, virtual or physical addresses may be used depending on the IOMMU configuration. When PASID is enabled, both the Address Translation Service (ATS) and the Page Request Service (PRS) should be enabled.

[0391] In one embodiment, the software configures an ATS function to control whether the device should translate addresses before performing memory access. If address translation is enabled in the IOMMU2810, ATS must be enabled in the device to obtain acceptable system performance. If address translation is not enabled in the IOMMU2810, ATS must be disabled. When ATS is disabled, only physical addresses may be used, and all memory accesses are performed using untranslated accesses. If PASID is enabled, ATS must be enabled.

[0392] In one embodiment, the software configures a PRS function to control whether the device can request a page if address translation fails. When PASID is enabled, PRS must be enabled, and when PASID is disabled, PRS must be disabled.

[0393] In some embodiments, a virtual memory space is utilized that is seamlessly shared among one or more processor cores, accelerator devices, and / or other types of processing devices (e.g., I / O devices). In particular, one embodiment utilizes a shared virtual memory (SVM) architecture in which the same virtual memory space is shared among cores, accelerator devices, and / or other processing devices. Furthermore, some embodiments include a heterogeneous form of physical system memory addressed using a common virtual memory space. The heterogeneous form of physical system memory may use different physical interfaces to connect to the DSA architecture. For example, an accelerator device may be directly coupled to local accelerator memory such as high-bandwidth memory (HBM), and each core may be directly coupled to host physical memory such as dynamic random-access memory (DRAM). In this example, the shared virtual memory (SVM) is mapped to a combined physical memory of HBM and DRAM so that accelerators, processor cores, and / or other processing devices can access HBM and DRAM using a consistent set of virtual memory addresses.

[0394] The accelerators for these and other features are described in detail below. For the purposes of overview, different implementations may include one or more of the following infrastructure features.

[0395] Shared Virtual Memory (SVM): In some embodiments, SVM is supported, allowing user-level applications to directly submit commands to the DSA using virtual addresses in descriptors. The DSA may support translating virtual addresses to physical addresses using an input / output memory management unit (IOMMU), which includes handling page faults. The virtual address range referenced by the descriptor may span multiple pages distributed across multiple heterogeneous memory types. Furthermore, in one embodiment, the use of physical addresses is also supported, as long as the data buffer is contiguous in physical memory.

[0396] Partial Descriptor Completion: With SVM support, operation may encounter page faults during address translation. In some cases, the device may terminate processing of the corresponding descriptor at the point of encountering a fault and provide the software with a completion record indicating partial completion and fault information, allowing the software to take corrective actions and retry operation after resolving the fault.

[0397] Batch Processing: In some implementations, it is supported to submit descriptors in "batches." A batch descriptor points to a substantially contiguous set of work descriptors (i.e., descriptors containing the actual data processing). When processing batch descriptors, the DSA fetches the work descriptors from specific memory and processes them.

[0398] Stateless Devices: In one embodiment, the descriptor is designed such that all information is needed to process the descriptor, which is contained within the descriptor payload itself. This allows the device to store minimal client-specific state, improving its scalability. One exception, when used, is the completion interrupt message, which is configured by trusted software.

[0399] Cache allocation control: This allows applications to specify whether to write to the cache or bypass the cache and write directly to memory. In one embodiment, completion records are always written to the cache.

[0400] Shared Work Queue (SWQ) Support: As described in detail below, in some embodiments, scalable work submissions are supported through a shared work queue (SWQ) using enqueue command (ENQCMD) and enqueue command (ENQCMDS) instructions. In this embodiment, the SWQ is shared by multiple applications.

[0401] Dedicated Work Queue (DWQ) Support: In some embodiments, there is support for high-throughput work submission through a dedicated work queue (DWQ) using the MOVDIR64B instruction. In this embodiment, the DWQ is dedicated to a specific application.

[0402] QoS Support: In some embodiments, it is possible to specify a Quality of Service (QoS) level for each work queue (e.g., by a kernel driver). Different work queues may then be assigned to different applications, allowing work from different applications to be dispatched from the work queues with different priorities. Work queues may be programmed to use specific channels for fabric QoS.

[0403] Biased cache coherence mechanism

[0404] In one embodiment, accelerator performance is improved using directly attached memory such as stacked DRAM or HBM, and application development for applications using the accelerator with directly attached memory is simplified. In this embodiment, the accessor-attached memory is mapped as part of the system memory and can be accessed using shared virtual memory (SVM) technology (for example, as used in current IOMMU implementations), without suffering the typical performance drawbacks associated with the full system's cache coherence.

[0405] The ability to access accessor-attached memory as part of system memory, without the cumbersome cache coherence overhead, provides a beneficial operating environment for accelerator offloading. The ability to access memory as part of system address mapping allows host software to set up operands and access computation results without the overhead of conventional I / O DMA data copying. Such conventional copies are all inefficient compared to simple memory access, relating to driver calls, interrupts, and memory-mapped I / O (MMIO) access. Simultaneously, the ability to access accessor-attached memory without cache coherence overhead can be critical to the execution time of offloaded computations. For example, with substantial streaming write memory traffic, cache coherence overhead can halve the effective write bandwidth seen by the accelerator. The efficiency of operand setup, the efficiency of the resulting access, and the efficiency of the accelerator computation all play a role in determining how well accelerator offloading is working. The cost of offloading (for example, setting up operands and obtaining results) is very high, and offloading may have no effect at all, or it may limit the accelerator to only very large jobs. The efficiency with which the accelerator performs the calculations may have the same effect.

[0406] In one embodiment, different memory access and coherence techniques are applied depending on the entity initiating the memory access (e.g., accelerator, core, etc.) and the memory being accessed (e.g., host memory or accelerator memory). These techniques are generally referred to as a “coherence bias” mechanism that provides accessor-attached memory, and two sets of cache coherence flows are used: one optimizes efficient accelerator access to its attached memory, and the other optimizes host access to accessor-attached memory and shared accelerator / host access to accessor-attached memory. Furthermore, two techniques are included for switching between these flows: one driven by application software and the other driven by independent hardware implicits. In both sets of coherence flows, the hardware maintains full cache coherence.

[0407] As generally shown in Figure 30, in one embodiment, a computer system is applied that includes an accelerator 3001 and one or more computer processor chips having a processor core and I / O circuitry 3003, wherein the accelerator 3001 is coupled to the processor via a multiprotocol link 2800. In one embodiment, the multiprotocol link 3010 is a dynamically multiplexed link that supports several different protocols, including, but not limited to, those described in detail above. However, it should be noted that the underlying principles of the present invention are not limited to any particular set of protocols. Furthermore, it should be noted that the accelerator 3001 and the core I / O 3003 may be integrated on the same semiconductor chip or on different semiconductor chips, depending on the implementation.

[0408] In the illustrated embodiment, the accelerator memory bus 3012 connects the accelerator 3001 to the accelerator memory 3005, and a separate host memory bus 3011 connects the core I / O 3003 to the host memory 3007. As already stated, the accelerator memory 3005 may have high-bandwidth memory (HBM) or stacked DRAM (some examples of these are described herein), and the host memory 3007 may have DRAM, for example, double data-rate synchronous dynamic random access memory (e.g., DDR3 SDRAM, DDR4 SDRAM, etc.). However, the underlying principles of the present invention are not limited to any particular type of memory or memory protocol.

[0409] In one embodiment, both the accelerator 3001 and the “host” software running on the processing core in the processor chip 3003 access the accelerator memory 3005 using two separate sets of protocol flows, referred to as “host-biased” flows and “device-biased” flows. As described below, one embodiment supports multiple options for modulating and / or selecting protocol flows for specific memory accesses.

[0410] The coherence bias flow is partially implemented on two protocol layers of the multiprotocol link 3010 between the accelerator 3001 and one of the processor chips 3003, namely the CAC protocol layer and the MA protocol layer. In one embodiment, the coherence bias flow is enabled by (a) using existing opcodes in the CAC protocol in a new manner, (b) adding new opcodes to the existing MA standard, and (c) adding support for the MA protocol to the multiprotocol link 3001 (before the link included only CAC and PCDI). It should be noted that the multiprotocol link is not limited to simply supporting CAC and MA. In one embodiment, it is simply required to support at least those protocols.

[0411] As used herein, the “host bias” flow shown in Figure 30 is a set of flows that centralize all requests, including requests from the accelerator itself, to the accelerator memory 3005 through a standard coherence controller 3009 in the processor chip 3003 to which the accelerator 3001 is mounted. This causes the accelerator 3001 to take a detour to access its own memory, but ensures that access from both the accelerator 3001 and the processor core I / O 3003 is maintained coherently using the processor’s standard coherence controller 3009. In one embodiment, the flow issues requests to the processor’s coherence controller 3009 via a multiprotocol link using the CAC opcode, in the same or similar manner in which the processor core 3009 issues requests to the coherence controller 3009. For example, the processor chip's coherence controller 3009 may issue UPI and CAC coherence messages (e.g., snoops) resulting from requests from the accelerator 3001 to all peer processor core chips (e.g., 3003) and internal processor agents on behalf of the accelerator, to the same extent as requests from the processor core 3003. In this embodiment, coherence is maintained between the data accessed by the accelerator 3001 and the processor core I / O 3003.

[0412] In one embodiment, the coherence controller 3009 also conditionally issues memory access messages to the accelerator's memory controller 3006 via the multiprotocol link 2800. These messages are also similar to messages containing new opcodes that force the data to be returned to the processor's coherence controller 3009 on the multiprotocol link 2800, and as a result, instead of being returned to the accelerator 3001 as a CAC response via the multiprotocol link 2800, the coherence controller 3009 sends it to a memory controller local to these processor dies, allowing the data to be returned directly to an agent inside the accelerator 3001.

[0413] In one embodiment of the “host bias” mode shown in Figure 30, all requests from the processor core 3003 targeting the accessor-attached memory 3005 are sent directly to the processor coherence controller 3009, just as they would be if targeting the normal host memory 3007. The coherence controller 3009 may apply these standard cache coherence algorithms and send these standard cache coherence messages in the same way they do for access from the accelerator 3001 and in the same way they do for access to the normal host memory 3007. The coherence controller 3009 also conditionally sends MA commands over the multiprotocol link 2800 for this class of requests, in which case the MA flow returns data over the multiprotocol link 2800.

[0414] The “device bias” flow shown in Figure 31 is a flow that allows accelerator 3001 to access its local associated memory 3005 without querying the host processor’s cache coherence controller 3007. More specifically, these flows allow accelerator 3001 to access its local associated memory via memory controller 3006 without sending a request via multiprotocol link 2800.

[0415] In "device bias" mode, requests from the processor core I / O 3003 are issued as described above for "host bias," but the MA portion of these flows is completed differently. In "device bias" mode, processor requests to the accessor-attached memory 3005 are completed as if they had been issued as "uncached" requests. This "uncached" convention is employed to ensure that data subject to device bias flows is never cached in the processor's cache hierarchy. This is because it allows accelerator 3001 to access device bias data in its memory 3005 without querying the cache coherence controller 3009 on the processor.

[0416] In one embodiment, support for “uncached” processor core 3003 access flows is implemented on the processor’s CAC bus with a globally monitored once-use (“GO-UO”) response. This response returns a portion of the data to the processor core 3003, instructing the processor to use only the data value for the time being. This prevents data caching and satisfies the requirements of the “uncached” flow. In systems with cores that do not support GO-UO responses, the “uncached” flow may be implemented using a multi-message response sequence on the MA layer of the multiprotocol link 2800 and on the processor core 3003’s CAC bus.

[0417] Specifically, when a processor core is able to target a "device bias" page in accelerator 3001, the accelerator sets up several states to block future requests from the accelerator to the target cache line and sends a special "device bias hit" response over the MA layer of the multiprotocol link 2800. In response to this MA message, the processor's cache coherence controller 3009 returns data to the requesting processor core 3003, immediately followed by a snoop invalidation message. If the processor core 3003 acknowledges that snoop invalidation is complete, the cache coherence controller 3009 sends another special MA "device bias Bock complete" message back to accelerator 3001 on the MA layer of the multiprotocol link 2800. This completion message causes accelerator 3001 to clear the aforementioned blocking state.

[0418] Figure 107 shows an embodiment using bias. In one embodiment, the selection of a device and a host bias flow is driven by a bias tracker data structure which may be maintained as a bias table 10707 in accelerator memory 3005. This bias table 10707 may be a page-granular structure (i.e., controlled by the granularity of memory pages) containing 1 or 2 bits for each accelerator-attached memory page. The bias table 10707 may be implemented in the stolen memory range of accelerator-attached memory 3005 with or without using an in-accelerator bias cache 10703 (for example, caching frequently / recently used entries of the bias table 10707). Alternatively, the entire bias table 10707 may be maintained in accelerator 3001.

[0419] In one embodiment, the bias table entry associated with each access to the accessor-attached memory 3005 is accessed before the actual access to the accelerator memory, causing the following actions to be performed. A local request from accelerator 3001 to find these pages within the device bias is directly transferred to accelerator memory 3005. A local request from accelerator 3001 to find these pages within the host bias is forwarded to processor 3003 as a CAC request over multiprotocol link 2800. The MA request from processor 3003, which finds these pages within the device bias, completes the request using the "uncached" flow described above. • The MA request from processor 3003, which finds these pages within the host bias, completes the request like a normal memory read.

[0420] The bias state of a page can be altered by either a software-based mechanism, a hardware-assisted software-based mechanism, or, in the case of a limited set, purely by a hardware-based mechanism.

[0421] One mechanism for changing the bias state employs API calls (e.g., OpenCL) to sequentially invoke the accelerator's device driver, which sends messages (or enqueues command descriptors) to accelerator 3001 instructing it to change the bias state, and for some transitions, performs cache flushing on the host. Cache flushing is required for the transition from host bias to device bias, but not for the reverse transition.

[0422] In some cases, it is difficult for software to determine when to make a bias transition API call and when to identify a page request bias transition. In such cases, the accelerator may implement a bias transition (implicit) mechanism to detect the need for a bias transition and send a message to its driver indicating it. The implicit mechanism may be as simple as a mechanism corresponding to a bias table lookup that triggers the accelerator's access to a host bias page or the host's access to a device bias page and signals the event to the accelerator's driver via an interrupt.

[0423] Note that in some embodiments, a second bias state bit may be required to enable the bias transition state value. This allows the system to continue accessing memory pages while those pages are in the process of handling a bias change (i.e., the cache is partially flushed and incremental cache contamination due to subsequent requests must be suppressed).

[0424] An exemplary process according to one embodiment is shown in Figure 32. The process may be implemented on the systems and processor architectures described herein, but is not limited to any particular system or processor architecture.

[0425] In 3201, a specific set of pages is placed within the device bias. As already mentioned, this can be achieved by updating the entries for these pages in the bias table to indicate that the pages are within the device bias (for example, by setting bits associated with each page). In one embodiment, once set within the device bias, it is guaranteed that the pages are not cached in the host cache memory. In 3202, the pages are allocated from device memory (for example, software allocates the pages by initiating a driver / API call).

[0426] In 3203, the operand is pushed to a page allocated from the processor core. In one embodiment, this is achieved by software using an API call to flip the host bias n operand page (e.g., via an OpenCL API call). No data copying or cache flushing is required, and the operand data may terminate at this stage at any point in the host cache hierarchy.

[0427] In 3204, the accelerator device generates results using operands. For example, it may execute commands to process data directly from its local memory (e.g., 3005 above). In one embodiment, the software uses the OpenCL API to flip the operand page back to the device bias (e.g., update the bias table). As a result of the API call, the work descriptor is submitted to the device (e.g., via sharing on a dedicated work queue, as described below). The work descriptor may instruct the device to flush the operand page from the host cache, resulting in a cache flush (e.g., performed using CLFLUSH in the CAC protocol). In one embodiment, the accelerator runs without host-related coherence overhead and dumps the data to the results page.

[0428] In step 3205, the result is retrieved from the allocated page. For example, in one embodiment, the software makes one or more API calls (e.g., via the OpenCL API) to flip the result page to the host bias. This action may change some bias conditions but does not result in any coherence or cache flushing. The host processor core can then access, cache, and share the result data as needed. Finally, in step 3206, the allocated page is released (e.g., via the software).

[0429] A similar process in which operands are released from one or more I / O devices is shown in Figure 33. In 3301, a particular set of pages is placed in device bias. As already mentioned, this can be achieved by updating the entries for these pages in the bias table to indicate that the pages are in device bias (e.g., by setting bits associated with each page). In one embodiment, once set in device bias, it is guaranteed that the pages are not cached in host cache memory. In 3302, the pages are allocated from device memory (e.g., software allocates the pages by initiating a driver / API call).

[0430] In 3303, the operand is pushed to a page allocated by the I / O agent. In one embodiment, this is achieved by the I / O agent and software posting a DMA request to the I / O agent using unallocated storage to write the data. In one embodiment, the data is never allocated to the host cache hierarchy, and the target page remains within the device bias.

[0431] In 3304, the accelerator device generates results using operands. For example, software may submit work to the accelerator device without the need for page transitions (i.e., pages remain within the device bias). In one embodiment, the accelerator device runs without host-related coherence overhead, and the accelerator dumps the data to a results page.

[0432] In 3305, the I / O agent retrieves the result from the assigned page (for example, under instructions from the software). For example, the software may post a DMA request to the I / O agent. If the source page remains in the device bias, no page transition is required. In one embodiment, the I / O bridge uses an RdCurr (current read) request to grab a non-cacheable copy of the data from the result page.

[0433] In some embodiments, a work queue (WQ) holds “descriptors” submitted by software, an arbiter and fairness policy used to implement Quality of Service (QoS), a processing engine for processing descriptors, an address translation and caching interface, and a memory read / write interface. A descriptor defines the scope of work to be performed. As illustrated in Figure 34, in one embodiment there are two different types of work queues: a dedicated work queue 3400 and a shared work queue 3401. The dedicated work queue 3400 stores descriptors for a single application 3413, while the shared work queue 3401 stores descriptors submitted by multiple applications 3410-3412. A hardware interface / arbiter 3402 dispatches descriptors from work queues 3400-3401 to the accelerator processing engine 3405 according to a specific arbitration policy (e.g., based on the processing requirements of each application 3410-3413 and the QoS / fairness policy).

[0434] Figures 108a–108B show the memory-mapped I / O (MMIO) space registers used with work queue-based implementations. Version register 10807 reports the version of this architecture specification supported by the device.

[0435] The general function register (GENCAP) 10808 defines the general functions of the device, such as the maximum transfer size and maximum batch size. Table B lists the various parameters and values ​​that can be specified in the GENCAP register. [Table 5]

[0436] In one embodiment, the Work Queue Function Register (WQCAP) 10810 defines the functions of the work queue, such as support for dedicated and / or shared modes of operation, the number of engines, and the number of work queues. Table C below lists the various parameters and values ​​that can be configured. [Table 6]

[0437] In one embodiment, the Operation Function Register (OPCAP) 10811 is a bitmask that defines the operation types supported by the device. Each bit corresponds to an operation type with the same code as the bit position. For example, bit 0 of this register corresponds to a No-op operation (code 0). The bit is set when the operation is supported and cleared when the operation is not supported. [Table 7]

[0438] In one embodiment, the general configuration register (GENCFG) 10812 defines the virtual channel (VC) steering tag. See Table E below. [Table 8]

[0439] In one embodiment, the general control register (GENCTRL) 10813 indicates whether or not an interrupt was generated in response to a hardware or software error. See Table F below. [Table 9]

[0440] In one embodiment, the device enable register (ENABLE) stores an error code, an indicator indicating whether the device is enabled or not, and a device reset value. For further details, see Table G below. [Table 10]

[0441] In one embodiment, the interrupt cause register (INTCAUSE) stores a value indicating the cause of the interrupt. See Table H below. [Table 11]

[0442] In one embodiment, the command register (CMD) 10814 is used to submit drain WQ, drain PASID, and drain all commands. The abort area indicates whether the requested operation is a drain or an abort. Before writing to this register, the software may ensure that any command has completed before being submitted through this register. Before writing to this register, the software may also configure a command configuration register and, if a completion record is requested, a command completion record address register.

[0443] The drain-all command drains or aborts all outstanding descriptors in all WQs and all engines. The drain-PASID command drains or aborts descriptors using a specific PASID in all WQs and all engines. Drain-WQ drains or aborts all descriptors in a specific WQ. Depending on the implementation, any drain command may wait for the completion of other descriptors in addition to the descriptor that needs to be waited for.

[0444] If the abort area is 1, the software is requesting that the affected descriptors be discarded. However, the hardware may complete some or all of them further. If the descriptors are discarded, there is no completion record to write, and no completion interrupts generated for those descriptors. Some or all of the other memory accesses may still occur.

[0445] Command completion is indicated by clearing the status field of this register (if requested) by generating a completion interrupt. When completion is signaled, all affected descriptors are either completed or discarded, and no further address translations, memory reads, memory writes, or interrupts are generated due to any affected descriptors. See Table I below. [Table 12]

[0446] In one embodiment, the software error status register (SWERROR) 10815 stores several different types of errors, including errors that occur when a descriptor is submitted, errors that occur when translating the completion record address within the descriptor, errors that occur when validating the descriptor if the completion record address valid flag within the descriptor is 0, and errors that occur while processing the descriptor, such as a page fault if the completion record address valid flag within the descriptor is 0. See Table J below. [Table 13] JPEG2026086720000015.jpg124170

[0447] In one embodiment, the hardware error status register (HWERROR) 10816 uses the same method as the software error status register (see above).

[0448] In one embodiment, the Group Configuration Register (GRPCFG) 10817 stores configuration data for each work queue / engine group (see Figures 36-37). In particular, the Group Configuration Table is an array of registers in BAR0 that controls the mapping of work queues to engines. There are as many groups as there are engines, but the software may configure the number of groups required. Each active group contains one or more work queues and one or more engines. Any unused group must have both a WQ field and an engine field equal to 0. Descriptors submitted to any WQ in a group may be processed by any engine in that group. Each active work queue must be in a single group. An active work queue is one in which the WQ size field of the corresponding WQCFG register is non-zero. Any engine not in a group is inactive.

[0449] Each GRPCFG register 10817 may be divided into three sub-registers, each sub-register being one or more 32-bit words (see Tables K-M). These registers may be read-only while the device is enabled. They are also read-only if the WQCAP work queue configuration support field is 0.

[0450] The offset of the sub-registers within BAR0 is such that 0 ≤ G < number of engines for each group G, and in one embodiment it is as follows: [Table 14] [Table 15] [Table 16]

[0451] In one embodiment, the Work Queue Configuration Register (WQCFG) 10818 stores data that defines the operation of each work queue. The WQ configuration table is an array of 16-byte registers in BAR0. The number of WQ configuration registers is equal to the number of WQ fields in WQCAP.

[0452] Each 16-byte WQCFG register is divided into four 32-bit sub-registers, which may be read from or written to using aligned 64-bit read or write operations.

[0453] Each WQCFG-A subregister is read-only while the device is enabled or when the WQCAP work queue configuration support field is 0.

[0454] Each WQCFG-B is writable at any time unless the WQCAP work queue configuration support field is 0. If the WQ threshold field contains a value greater than the WQ size while WQ is enabled, WQ is not enabled and the WQ error code is set to 4. If the WQ threshold field is written with a value greater than the WQ size while WQ is enabled, WQ is disabled and the WQ error code is set to 4.

[0455] While WQ is enabled, each WQCFG-C subregister is read-only. It may be written before or at the same time as setting WQ Enable to 1. If the Work Queue Configuration Support field of WQCAP is 0, the following fields, namely WQ Mode, WQ Fault on Block Enable, and WQ Priority, are always read-only. Even if the Work Queue Configuration Support field of WQCAP is 0, the following fields of WQCFG-C, namely WQ PASID and WQ U / S, are writable when WQ is not enabled.

[0456] Each WQCFG-D sub-register is writable at any time. However, if the device is not enabled, it is an error that sets WQ Enable to 1.

[0457] If WQ Enable is set to 1, both the WQ Enable and WQ Error Code fields are cleared. Then, either WQ Enable or WQ Error Code is set to a non-zero value indicating whether WQ was successfully enabled or not.

[0458] The sum of the WQ size fields of all WQCFG registers cannot exceed the sum of the WQ size fields in GENCAP. This constraint is checked when the device is enabled. WQs with a WQ size field of 0 cannot be enabled, and all other fields of such WQCFG registers are ignored. While the device is enabled, the WQ size fields are read-only. See Table N for data on each of the sub-registers. [Table 17] JPEG2026086720000020.jpg191170

[0459] In one embodiment, a work queue occupancy interrupt control register 10819 (one for each work queue (WQ)) allows software to request an interrupt when the work queue occupancy rate drops to a specific threshold. If the WQ occupancy interrupt enable for a WQ is 1, and the current WQ occupancy rate is at or below the WQ occupancy limit, the following actions may occur: 1. The WQ occupancy interrupt enable field is cleared. 2. Bit 3 of the interrupt reason register is set to 1. 3. If bit 3 of the interrupt reason register was 0 before step 2, an interrupt is generated using MSI-X table entry 0. 4. If the register is written with enable=1 and limit ≥ current WQ occupancy rate, an interrupt is generated immediately. As a result, if the register is written with enable=1 and limit ≥ WQ size, an interrupt is always generated immediately. [Table 18]

[0460] In one embodiment, a work queue status register (one per WQ) 10820 specifies the number of current entries in each WQ. This number can change whenever descriptors are submitted to or dispatched from the queue, making it unreliable to determine whether a WQ is free or not.

[0461] In one embodiment, MSI-X entry 10821 stores MSI-X table data. The offset and number of entries are determined by the MSI-X functionality. The proposed number of entries is the number of WQs plus 2.

[0462] In one embodiment, the MSI-X unprocessed bit array 10822 stores the offset and number of entries in the MSI-X function.

[0463] In one embodiment, the interrupt message storage entry 10823 stores interrupt messages in a table structure. The format of this table is similar to the MSI-X table format specified by PCIe, but its size is not limited to 2048 entries. However, in some embodiments, the size of this table may vary between different DSA implementations and may be less than 2048 entries. In one embodiment, the number of entries is located in the interrupt message storage size field of a general function register. If the interrupt message storage support function is 0, this table is not presented. For the DSA to support a large number of virtual machines or containers, the size of the supported table needs to be considerably larger.

[0464] In one embodiment, the format of each entry in IMS is described in Table P below. [Table 19]

[0465] Figure 35 shows an embodiment of a data streaming accelerator (DSA) device having multiple work queues 3511-3512 that receive descriptors submitted via an I / O fabric interface 3501 (e.g., the multiprotocol link 2800 described above). The DSA uses the I / O fabric interface 3501 to receive downstream work requests from clients (processor cores, peer input / output (IO) agents (such as network interface controllers (NICs)) and / or software chain offload requests, etc.) and for upstream read, write, and address translation operations. The illustrated embodiment includes an arbiter 3513 that performs arbitration between work queues and dispatches work descriptors to one of a plurality of engines 3550. The processing of the arbiter 3513 and work queues 3511-1012 may be configured through a work queue configuration register 3500. For example, the arbiter 3513 may be configured to implement various QoS and / or fairness policies in order to dispatch descriptors from each of the work queues 3511-1012 to each of the engines 3550.

[0466] In one embodiment, some of the descriptors queued in work queues 3511-3512 are batch descriptors 3515 that contain / identify batches of work descriptors. The arbiter 3513 transfers the batch descriptors to a batch processing unit 3516 that processes the batch descriptors by reading an array of descriptors 3518 from memory, using addresses translated through a translation cache 3520 (potentially other address translation services on the processor). Once the physical addresses are identified, a data read / write circuit 3540 reads the batch of descriptors from memory.

[0467] The second arbiter 3519 performs arbitration between batches of work descriptors 3518 provided by the batch processing unit 3516 and individual work descriptors 3514 obtained from work queues 3511-3512, and outputs the work descriptors to the work descriptor processing unit 3530. In one embodiment, the work descriptor processing unit 3530 has stages for reading memory (via the data R / W unit 3540), performing requested operations on the data, generating output data, and writing output data, completion records, and interrupt messages (via the data R / W unit 3540).

[0468] In one embodiment, the work queue configuration allows software to configure each WQ (via the WQ configuration register 3500) as either a shared work queue (SWQ) that receives descriptors using the non-posted ENQCMD / S instruction, or a dedicated work queue (DWQ) that receives descriptors using the posted MOVDIR64B instruction. As already mentioned above with respect to Figure 34, the DWQ may process work descriptors and batch descriptors submitted from a single application, while the SWQ may be shared among multiple applications. The WQ configuration register 3500 also allows software to control which WQ3511-3512 supplies which accelerator engine 3550, and the priority associated with the WQ3511-3512 supplying each engine. For example, an ordered set of priorities may be defined (e.g., high, medium, low: 1, 2, 3, etc.), and descriptors may generally be dispatched before lower-priority work queues, or more frequently from higher-priority work queues than from lower-priority work queues. For example, using two work queues identified as high-priority and low-priority, for each set of 10 descriptors to be dispatched, 8 of the 10 descriptors may be dispatched from the high-priority work queue, while 2 of the 10 descriptors may be dispatched from the low-priority work queue. Various other techniques may be used to achieve different priority levels between work queues 3511-3512.

[0469] In one embodiment, the data streaming accelerator (DSA) is software compatible with the PCI Express configuration mechanism, implementing the PCI header and extension space within its configuration mapping register set. The configuration registers can be programmed from the root complex via CFC / CF8 or MMCFG. Similarly, all internal registers may also be accessible via a JTAG or SM bus interface.

[0470] In one embodiment, the DSA device uses memory-mapped registers to control its operations. Function, configuration, and work submission registers (portals) are accessible through the MMIO area defined by the BAR0, BAR2, and BAR4 registers (described below). Each portal may reside on a separate 4K page so that they can be independently mapped to different address spaces (clients) using the processor page table.

[0471] As already mentioned, the software defines the work on the DSA through descriptors. Descriptors specify the type of operation on the DSA and execute data and status buffer addresses, immediate operands, completion attributes, etc. (Further details and information regarding the descriptor format are described below). Completion attributes specify the address to which the completion record is written and the information required to generate a selective completion interrupt.

[0472] In one embodiment, the DSA avoids maintaining client-specific state on the device. All information for processing the descriptor is contained within the descriptor itself. This improves its ability to be shared between user-mode applications, as well as between different virtual machines (machine containers) within a virtualized system.

[0473] A descriptor may contain an operation and associated parameters (called a work descriptor), or it may contain the address of an array of work descriptors (called a batch descriptor). The software prepares the descriptor in memory and submits it to the device's work queues (WQs) 3511-3512. Depending on the WQ mode and the client's privilege level, the descriptor is submitted to the device using the MOVDIR64B, ENQCMD, or ENQCMDS instruction.

[0474] Each WQ3511-3512 has a fixed number of slots, which can become full under heavy load. In one embodiment, the device provides necessary feedback to help the software implement flow control. The device dispatches descriptors from work queues 3511-3512 and submits them to the engine for further processing. If the engine 3550 encounters a particular failure or error that results in an abort when completing a descriptor, it notifies the host software by either writing to a completion record in host memory or issuing an interrupt, or both.

[0475] In one embodiment, each work queue is accessible via multiple registers, each located in a separate 4KB page within the device MMIO space. For each WQ, one work submission register is called the "unprivileged portal" and is mapped to user space used by user-mode clients. Another work submission register is called the "privileged portal" and is used by kernel-mode drivers. The remaining one is the guest portal and is used by kernel-mode clients within a virtual machine.

[0476] As already mentioned, each work queue 3511-3512 can be configured to operate in one of two modes: dedicated or shared. The DSA exposes function bits in the work queue function register to indicate support for dedicated and shared modes. The DSA also exposes control to the work queue configuration register 3500 to configure each WQ to operate in one of the modes. The mode of a WQ can only be changed while the WQ is disabled, i.e., (WQCFG enable = 0). Further details of the WQ function register and WQ configuration register are described below.

[0477] In one embodiment, in shared mode, a DSA client submits a descriptor to a work queue using the ENQCMD or ENQCMDS instruction. ENQCMD and ENQCMDS use a 64-byte non-posted write and wait for a response from the device before completing. The DSA returns "success" if there is space in the work queue (e.g., for the requesting client / application), or "retry" if the work queue is full. The ENQCMD and ENQCMDS instructions may return the status of the command submission with a zero flag (0 indicates success, 1 indicates retry). Using the ENQCMD and ENQCMDS instructions, multiple clients can submit descriptors directly and simultaneously to the same work queue. The device provides this feedback so that clients can indicate whether or not these descriptors have been received.

[0478] In shared mode, the DSA may reserve a portion of the SWQ capacity for submissions via a privileged portal for kernel-mode clients. Work submissions via an unprivileged portal are accepted until the number of descriptors in the SWQ reaches a threshold set for the SWQ. Work submissions via a privileged portal are accepted until the SWQ is full. Work submissions via a guest portal are limited by the same threshold as for unprivileged portals.

[0479] If the ENQCMD or ENQCMDS instruction returns "success," the descriptor is received by the device and queued for processing. If the instruction returns "retry," the software may attempt to resubmit the descriptor to the SWQ, or, if it is a user-mode client using an unprivileged portal, it may request the kernel-mode driver to submit the descriptor on behalf of the user-mode client using a privileged portal. This helps avoid denials of service and provides a guarantee of forward progress. Alternatively, if the SWQ is full, the software may use other methods (e.g., performing work using the CPU).

[0480] The client / application is identified by the device using a 20-bit ID called the Processing Address Space ID (PASID). The PASID is used by the device to look up addresses within the device TLB1722 and send address translation or page requests to the IOMMU1710 (e.g., via the Multiprotocol Link 2800). In shared mode, the PASID used with each descriptor is contained in the descriptor's PASID field. In one embodiment, ENQCMD copies the PASID of the current thread from a specific register (e.g., PASID MSR) to the descriptor, while ENQCMDS allows software in supervisor mode to copy the PASID to the descriptor.

[0481] In "dedicated" mode, a DSA client may use the MOVDIR64B instruction to submit a descriptor to the device's work queue. MOVDIR64B uses a 64-byte posted write, and the instruction completes faster due to the posted nature of the write operation. With respect to the dedicated work queue, the DSA may expose the total number of slots in the work queue and provide flow control in software. The software is responsible for tracking the number of descriptors submitted and completed in order to detect when the work queue is full. If the software mistakenly submits a descriptor to the dedicated WQ when there are no free slots in the work queue, the descriptor is dropped and an error may be recorded (for example, in a software error register).

[0482] The MOVDIR64B instruction does not write the PASID as the ENQCMD or ENQCMDS instructions do, so the PASID field in the descriptor cannot be used in dedicated mode. The DSA may ignore the PASID field in the descriptor submitted to the dedicated work queue and instead uses the WQ PASID field of the WQ configuration register 3500 for address translation. In one embodiment, the WQ PASID field is set by the DSA driver when configuring the work queue in dedicated mode.

[0483] In dedicated mode, multiple clients / applications do not share a single DWQ; however, a DSA device can be configured to have multiple DWQs, each of which can be independently assigned to a client. Furthermore, DWQs can be configured to have the same or different QoS levels for different performance levels provided for different clients / applications.

[0484] In one embodiment, the data streaming accelerator (DSA) includes two or more engines 3550 that process descriptors submitted to work queues 3511-1012. One embodiment of the DSA architecture includes four engines numbered 0 through 3. Engines 0 and 1 can each utilize up to the full bandwidth of the device (e.g., 30 GB / s for reading and 30 GB / s for writing). Of course, the combined bandwidth for all engines is also limited to the maximum bandwidth available to the device.

[0485] In one embodiment, the software uses group configuration registers to group WQ3511-3512 and engine 3550. Each group contains one or more WQs and one or more engines. The DSA may use any engine in a group to process descriptors posted to any WQ in a group, and each WQ and each engine may be in only one group. The number of groups may be the same as the number of engines, so each engine may be in a separate group, but not all groups need to be used if any group contains more than one engine.

[0486] The DSA architecture allows for great flexibility when configuring work queues, groups, and engines, although the hardware may be narrowly designed for use in specific configurations. Engines 0 and 1 may be configured in one of two different ways, depending on software requirements. One recommended configuration is to place both engines 0 and 1 within the same group. The hardware uses either engine to process descriptors from any work queue within the group. In this configuration, if one engine experiences a stall due to high-latency memory address translation or a page fault, the other engine can continue to operate, maximizing the overall device throughput.

[0487] Figure 36 shows two work queues 3621-3622 and 3623-3624 within each group 3611 and 3612, respectively, but there may be any number up to the maximum number of supported WQs. The WQs within a group may be shared WQs with different priorities, one shared WQ and other dedicated WQs, or multiple dedicated WQs with the same or different priorities. In the illustrated example, group 3611 is serviced by engines 0 and 1 3601, and group 3612 is serviced by engines 2 and 3 3602.

[0488] As shown in Figure 37, in another configuration using engine 0 3700 and engine 1 3701, they are placed in separate groups 3710 and 3711, respectively. Similarly, group 2 3712 is assigned to engine 2 3702, and group 3 is assigned to engine 3 3703. Furthermore, group 0 3710 consists of two work queues 3721 and 3722, group 1 3711 consists of work queue 3723, work queue 2 3712 consists of work queue 3724, and group 3 3713 consists of work queue 3725.

[0489] The software may choose this configuration if it wants to reduce the possibility of latency-sensitive operations being blocked behind other operations. In this configuration, the software submits latency-sensitive operations to work queue 3723 connected to engine 1 3702, and other operations to work queues 3721-3722 connected to engine 0 3700.

[0490] Engines 2 3702 and 3 3703 may be used, for example, to write to high-bandwidth non-volatile memory such as phase-change memory. The bandwidth function of these engines may be sized to match the expected write bandwidth of this type of memory. In this use, bits 2 and 3 of the engine configuration register should be set to 1, indicating that virtual channel 1 (VC1) should be used for traffic from these engines.

[0491] In platforms lacking high-bandwidth non-volatile memory (e.g., phase-change memory), or where the DSA device is not used to write to this type of memory, engines 2 and 3 may remain unused. However, provided that the submitted operation can tolerate limited bandwidth, the software may use them as additional low-latency paths.

[0492] When each descriptor arrives at the head of the work queue, it may be removed by the scheduler / arbiter 3513 and transferred to one of the engines in the group. For a batch descriptor 3515 that points to a work descriptor 3518 in memory, the engine fetches the array of work descriptors from memory (i.e., using the batch processing unit 3516).

[0493] In one embodiment, for each work descriptor 3514, the engine 3550 prefetches the translation for the completion record address and passes the operation to the work descriptor processing unit 3530. The work descriptor processing unit 3530 uses devices TLB1722 and IOMMU1710 to read the source data, perform a specific operation, and write the destination data back to memory for source and destination address translation. When the operation is complete, the engine writes the completion record to the pre-translated completion address, if requested by the work descriptor, and generates an interrupt.

[0494] In one embodiment, multiple work queues in a DSA may be used to provide multiple levels of quality of service (QoS). The priority of each WQ may be specified in the WQ configuration register 3500. The priority of a WQ is relative to other WQs in the same group (for example, the priority level of a WQ existing alone in a group is meaningless). Work queues in a group may have the same or different priorities. However, since a single SWQ will serve the same purpose, it is meaningless to configure multiple shared WQs with the same priority in the same group. The scheduler / arbiter 3513 dispatches work descriptors from work queues 3511-3512 to the engine 3550 according to these priorities.

[0495] Figure 38 shows one embodiment of a descriptor 1300 that includes an operation field 3801 defining the operation to be executed, several flags 3802, a processing address space identifier (PASID) field 3803, a completion record address field 3804, a source address field 3805, a destination address field 3806, a completion interrupt field 3807, a transfer size field 3808, and a field 3809 that is (potentially) specific to one or more operations. In one embodiment, there are three flags: completion record address valid, request completion record, and request completion interrupt.

[0496] Common fields include both trusted and untrusted fields. Trusted fields are always trusted by the DSA device because they are input by the CPU on the host or by privileged (ring 0 or VMM) software. Untrusted fields are supplied directly by the DSA client.

[0497] In one embodiment, the trusted fields include the PASID field 3803, the reserved field 3811, and the U / S (User / Supervisor) field 3810 (i.e., 4 bytes starting at an offset of 0). If the descriptor is submitted using the ENQCMD instruction, these fields in the source descriptor may be ignored. Values ​​contained in the MSR (e.g., PASID MSR) may be placed in these fields before the descriptor is sent to the device.

[0498] In one embodiment, when a descriptor is submitted using the ENQCMDS instruction, these fields in the source descriptor are initialized by software. If the PCI Express PASID function is not enabled, the U / S field 3810 is set to 1 and the PASID field 3803 is set to 0.

[0499] If the descriptor is submitted using the MOVDIR64B instruction, these fields within the descriptor may be ignored. The device instead uses the WQ U / S and WQ PASID fields of the WQ config register 3500.

[0500] These fields may be ignored for any descriptor in the batch. The corresponding fields of batch descriptor 3515 are used for each descriptor 3518 in the batch. Table Q provides a description and bit position for each of these trusted fields. [Table 20]

[0501] The following table R is a list of actions performed in one embodiment according to the descriptor's operation field 3801. [Table 21]

[0502] The following table S lists the flags used in one embodiment of the descriptor. [Table 22] JPEG2026086720000026.jpg159170

[0503] In one embodiment, the completion record address 3804 defines the address of the completion record. The completion record may be 32 bytes, and the completion record address is aligned on a 32-byte boundary. If the completion record address valid flag is 0, this field is reserved. If the requested completion record flag is 1, the completion record is written to this address when the operation is completed. If the requested completion record is 0, the completion record is written to this address only if there is a page fault or error.

[0504] For any operation that results in a comparison or other outcome, the completion record address valid and the request completion record flag should both be 1, and the completion record address should be valid.

[0505] For any operation using a virtual address, the completion record address should be valid for determining whether the request completion record flag is set, and as a result, the completion record may be written in the event of a page fault or error.

[0506] For best results, this field should be enabled for all descriptors, as it allows the device to report errors to the software that submitted the descriptor. If this flag is 0 and an unexpected error occurs, the error is reported to the SWERROR register, and the software that submitted the request may not be notified of the error.

[0507] If the completion queue enable flag is set in the batch descriptor, the completion log address field 3804 ignores the descriptor in the batch and uses the completion queue address in the batch descriptor instead.

[0508] In one embodiment, for an operation to read data from memory, the source address field 3805 specifies the address of the source data. There is no alignment requirement for the source address. For an operation to write data to memory, the destination address field 3806 specifies the address of the destination buffer. There is no alignment requirement for the destination address. For some operation types, this field is used as the address of a second source buffer.

[0509] In one embodiment, the transfer size field 3808 indicates the number of bytes to be read from the source address to perform the operation. The maximum value of this field may be 232-1, but the maximum possible transfer size may be smaller and should be determined from the maximum transfer size field of a general function register. The transfer size should not be 0. For many operation types, there are no alignment requirements for the transfer size. Exceptions are mentioned in the operation descriptions.

[0510] In one embodiment, if the Use Interrupt Message Storage flag is 1, the Completion Interrupt Processing field 3807 defines the interrupt message storage entry used to generate the completion interrupt. The value of this field should be less than the value of the Interrupt Message Storage Size field in GENCAP. In one embodiment, the Completion Interrupt Processing field 3807 is reserved under any of the following conditions: the Use Interrupt Message Storage flag is 0, the Request Completion Interrupt flag is 0, the U / S bit is 0, the Interrupt Message Storage Support field of the General Function Register is 0, or the descriptor has been submitted via the guest portal.

[0511] As shown in Figure 39, one embodiment of the completion record 3900 is a 32-byte structure in memory that the DSA writes when an operation is completed or an error occurs. The completion record address should be 32 bytes aligned.

[0512] This section describes the fields of the completion record that are common to many operation types. Each operation type description includes a completion record diagram if the format differs. Additional operation-specific fields are described further below. The completion record 3900 can always be 32 bytes, even if none of the required fields are present. The completion record 3900 contains enough information to continue the operation if it was partially completed due to a page fault.

[0513] The completion record may be implemented as a 32-byte aligned structure in memory (identified by the completion record address 3804 of descriptor 3800). The completion record 3900 includes a completion status field 3904 indicating whether the operation was completed or not. If the operation was completed successfully, the completion record may include the results of the operation, if any, depending on the type of operation. If the operation was not completed successfully, the completion record includes failure or error information.

[0514] In one embodiment, the status field 3904 reports the completion status of the descriptor. The software should initialize this field to 0 so that it can detect when the completion record has been written. [Table 23]

[0515] Table T above provides various status codes and is relevant to the description of one embodiment.

[0516] Table U below shows a fault code 3903 available in one embodiment, which includes a first bit indicating whether the fault address was read or written, and a second bit indicating whether the fault access was in user mode or supervisor mode. [Table 24]

[0517] In one embodiment, if the completion record 3900 was for a descriptor submitted as part of a batch, the index field 3902 contains the batch index of the descriptor that generated this completion record. For batch descriptors, this field may be 0xff. For other descriptors that are not part of a batch, this field may be reserved.

[0518] In one embodiment, if an operation is partially completed due to a page fault, the byte completion field 3901 contains the number of source bytes processed before the fault occurred. All source bytes represented by this count are fully processed, and the results are written to the destination address according to the operation type, as necessary. For some operation types, this field may be used if the operation is stopped before completion for some reason other than a fault. If the operation is fully completed, this field may be set to 0.

[0519] For operation types where the output size cannot be easily determined from this value, the completion record also includes the number of bytes written to the destination address.

[0520] If an operation is partially completed due to a page fault, this field contains the address that caused the fault. As a general rule, all descriptors should have a valid completion record address 3804, and the valid completion record address flag should be 1. Several exceptions to this rule are described below.

[0521] In one embodiment, the first byte of the completion record is the status byte. All status values ​​written by the device are non-zero. The software should initialize the status field of the completion record to 0 before submitting the descriptor, so that it can indicate when the device has written to the completion record. Initializing the completion record also ensures that it is mapped and therefore the device will not encounter a page fault when accessing it.

[0522] The request completion record flag indicates to the device that a completion record should be written even if the operation was successful. If this flag is not set, the device will only write a completion record if there are errors.

[0523] Descriptor completion can be detected by software using one of the following methods:

[0524] 1. Poll the completion record and wait for the status field to become non-zero.

[0525] 2. Block using a UMONITOR / UMWAIT instruction (as described herein) on the completion record address until it is written or until a timeout occurs. The software should then check whether the status field is non-zero to determine whether the operation is complete.

[0526] 3. For kernel-mode descriptors, request an interrupt when the operation is complete.

[0527] 4. If a descriptor is in a batch, set the fence flag in subsequent descriptors in the same batch. Completion of a descriptor with a fence or any subsequent descriptor in the same batch indicates completion of all descriptors preceding the fence.

[0528] 5. If a descriptor is present in a batch, the completion of the batch descriptor that initialized the batch indicates the completion of all descriptors within the batch.

[0529] 6. Issue a drain descriptor or drain command and wait for it to complete.

[0530] If the completion status indicates partial completion due to a page fault, the completion record indicates how much of the process was completed before the failure occurred (if any) and the virtual address where the failure occurred. The software may choose to return the failure to a normal state (by touching the fault address from the processor) and resubmit the remainder of the work in a new descriptor, or complete the remainder of the work in the software. Faults on the descriptor list and completion record addresses are handled differently, as described in more detail below.

[0531] In one embodiment of DSA, only message signaling interrupts are supported. DSA provides two types of interrupt message storage: (a) an MSI-X table enumerated through the MSI-X function that stores interrupt messages used by the host driver, and (b) a device-specific interrupt message storage (IMS) table that stores interrupt messages used by the guest driver.

[0532] In one embodiment, interrupts may be generated for three types of events: (1) completion of a kernel-mode descriptor, (2) completion of a drain or abort command, and (3) an error posted in a software or hardware error register. There is a separate interrupt enable for each type of event. Interrupts resulting from errors and completion of abort / drain commands are generated using entry 0 in the MSI-X table. The interrupt reason register may be read by software to determine the reason for the interrupt.

[0533] For kernel-mode descriptors that complete (e.g., descriptors with a U / S field of 1), the interrupt message used depends on how the descriptor was submitted and the interrupt message storage flag used within the descriptor.

[0534] Completion interrupt messages for kernel-mode descriptors submitted via a privileged portal are generally entries in the MSI-X table, determined by the portal address. However, if the interrupt message storage support field in GENCAP is 1, descriptors submitted via a privileged portal may override this behavior by setting the use interrupt message storage flag within the descriptor. In this case, the completion interrupt processing field within the descriptor is used as an index to the interrupt message storage.

[0535] A completion interrupt message for a kernel-mode descriptor submitted via the guest portal is an entry in the interrupt message storage, determined by the portal address.

[0536] Interrupts generated by DSA are processed through interrupt remapping and posting hardware, as configured by the kernel or VMM software. [Table 25]

[0537] As already mentioned, DSA supports submitting multiple descriptors at once. A batch descriptor contains the address of an array of work descriptors in host memory and the number of elements in that array. An array of work descriptors is called a "batch". The use of batch descriptors allows a DSA client to submit multiple work descriptors using a single ENQCMD, ENQCMDS, or MOVDIR64B instruction, potentially improving overall throughput. DSA enforces limits on the number of work descriptors in a batch. This limit is indicated in the Maximum Batch Size field in the common function registers.

[0538] Batch descriptors are submitted to the work queue in the same way as other work descriptors. When a batch descriptor is processed by a device, the device reads an array of work descriptors from memory and then processes each work descriptor. Work descriptors are not necessarily processed in order.

[0539] The PASID 3803 and U / S flag of a batch descriptor are used for all descriptors within the batch. The PASID and U / S fields 3810 in descriptors within a batch are ignored. Each work descriptor within a batch can identify a completion record address 3804, just as a directly submitted work descriptor does. Alternatively, a batch descriptor can identify a "completion queue" address to which the device writes completion records for all work descriptors from the batch. In this case, the completion record address field 3804 in the batch descriptor is ignored. The completion queue should be one entry larger than the total number of descriptors, so that there is space for completion records for all descriptors in the batch and for the batch descriptor. Completion records are generated in the order in which the descriptors complete, and do not have to be the same as the order in which they appear in the descriptor array. Each completion record includes an index of the descriptor within the batch that generated that completion record. An index of 0xff is used for the batch descriptor itself. An index of 0 is used for directly submitted descriptors other than batch descriptors. Some descriptors within a batch do not require a completion record to be generated if they do not request one and complete successfully. In this case, the number of completion records written to the completion queue may be less than the number of descriptors in the batch. Completion records for batch descriptors (if requested) are written to the completion queue after completion records for all descriptors in the batch.

[0540] A batch descriptor does not define a completion queue; a completion record for the batch descriptor (if requested) is written to its own completion record address after all descriptors in the batch have completed. The completion record for a batch descriptor includes an indication of whether any descriptors in the batch completed with a status that does not correspond to success. This allows the software to check only the completion record for the batch descriptor in the normal case where all descriptors in the batch complete successfully.

[0541] A completion interrupt may be requested by one or more work descriptors in the batch, as needed. A completion record for a batch descriptor (if requested) is written after the completion records and completion interrupts for all descriptors in the batch. A completion interrupt for a batch descriptor (if requested) is generated after the completion record for the batch descriptor, just like for any other descriptor.

[0542] Batch descriptors do not need to be included in a batch. Nested or chained descriptor arrays are not supported.

[0543] By default, the DSA does not guarantee any ordering while work descriptors are being executed. Descriptors can be dispatched and completed in any order the device deems appropriate to maximize throughput. Therefore, if ordering is required, the software must explicitly order it. For example, the software may submit a descriptor, wait for a completion record or interrupt from the descriptor to ensure completion, and then submit the next descriptor.

[0544] The software can also specify the ordering of descriptors within a batch defined by the batch descriptor. Each work descriptor has a fence flag. If set, the fence ensures that processing of the descriptor does not begin until the previous descriptor in the same batch has completed. This allows a descriptor with a fence to consume data generated by the previous descriptor in the same batch.

[0545] The descriptor completes after all writes generated by the operation have become globally observable, after the destination has been read back if requested, after the write to the completion record has become globally observable if required, and after a completion interrupt has been generated if requested.

[0546] If any descriptor in a batch completes with a status that does not correspond to success, for example, if it partially completes due to a page fault, any subsequent descriptors with a fence flag equal to 1, and any subsequent descriptors in the batch, are discarded. The completion record for the batch descriptor used to submit the batch indicates how many descriptors have completed. Any descriptor that partially completes and for which a completion record is generated is counted when it is completed. Only discarded descriptors are considered incomplete.

[0547] Fences also ensure ordering for completion records and interrupts. For example, a No-op descriptor with a configured fence and a requested completion interrupt will generate an interrupt after all preceding descriptors in the batch have completed (and their completion records have been written, if required). Completion record writes are always ordered behind data writes generated by the same work descriptor, and completion interrupts (if requested) are always ordered behind completion record writes for the same work descriptor.

[0548] A drain descriptor allows a client to wait for all descriptors belonging to its own PASID to complete. It can be used as a fence operation for the entire PASID. A drain operation completes when all preceding descriptors with that PASID have completed. Drain descriptors can be used by software to request a single completion record or interrupt for the completion of all their descriptors. Drain descriptors are normal descriptors submitted to a normal work queue. Drain descriptors may be included in a batch. (A fence flag may be used within a batch to wait for preceding descriptors within the batch to complete.)

[0549] The software must ensure that after a drain descriptor is submitted and before it completes, there are no descriptors with a specific PASID to be submitted to the device. If additional descriptors are submitted, it is not specified whether the drain operation waits for the additional descriptors to complete. This could result in a long drain operation. Even if the device does not wait for the additional descriptors to complete, some of the additional descriptors may complete before the drain operation is complete. Thus, drain is different from a fence in that a fence ensures that no subsequent operations begin until all preceding operations are complete.

[0550] In one embodiment, abort / drain commands are submitted by privileged software (OS kernel or VMM) by writing to the abort / drain register. Upon receiving one of these commands, the DSA waits for the completion of a particular descriptor (described below). Once the command is complete, the software can verify that there are no more descriptors in a particular category that are pending on the device.

[0551] In one embodiment, there are three types of drain commands: drainall, drainPASID, and drainWQ. Each command has an abort flag indicating a device that may discard any unprocessed descriptors rather than process them to completion.

[0552] The drain-all command waits for all descriptors submitted before the drain-all command to complete. Descriptors submitted after the drain-all command may be in progress when the drain-all command completes. While the drain-all command waits for previous descriptors to complete, the device may begin working on new descriptors.

[0553] The drainPASID command waits for all descriptors associated with a given PASID. Once the drainPASID command completes, there are no more descriptors for that PASID in the device. The software may ensure that there are no descriptors with a given PASID to submit to the device after the drainPASID command is submitted and before it completes. Otherwise, the behavior is undefined.

[0554] The drain WQ command waits for all descriptors submitted to a specific work queue. The software can ensure that no descriptors are submitted to the WQ after the drain WQ command has been submitted and before it has completed.

[0555] If an application or VM using a DSA is paused, it may have outstanding descriptors submitted to the DSA. This work must be completed, so the client is in a coherent state that can be resumed later. The DrainPASID and DrainAll commands are used by the OS or VMM to wait for any outstanding descriptors. The DrainPASID command is used for applications or VMs that were using a single PASID. The DrainAll command is used for VMs that are using multiple PASIDs.

[0556] When an application using DSA exits or is terminated by the operating system (OS), the OS must ensure that there are no outstanding descriptors before it can free or reuse the address space, allocated memory, and PASID. To dispose of any outstanding descriptors, the OS uses the drainPASID command with the PASID of the client being terminated, and sets the abort flag to 1. Upon receiving this command, the DSA discards all descriptors belonging to that PASID without further processing.

[0557] In one embodiment of DSA, a mechanism is provided to identify quality of service for dispatching work from multiple Work Queries (WQs). DSA allows software to divide the total WQ space into multiple WQs. Each WQ may be assigned a different priority for dispatching work. In one embodiment, the DSA scheduler / arbiter 3513 dispatches work from WQs such that higher-priority WQs serve more work than lower-priority WQs. However, DSA ensures that higher-priority WQs do not exhaust lower-priority WQs. As already mentioned, various prioritization schemes may be adopted based on implementation requirements.

[0558] In one embodiment, a WQ configuration register table is used to configure WQs. The software can configure the number of active WQs to match the number of desired QoS levels. The software configures each WQ by programming the WQ size and several additional parameters into the WQ configuration register table. This effectively divides the entire WQ space into the desired number of WQs. Unused WQs have a size of 0.

[0559] Errors can be broadly categorized into two types: 1) relevant errors that occur when processing a descriptor for a specific PASID, and 2) non-relevant errors that are effectively broad and not specific to a particular PASID. The DSA attempts to avoid, as much as possible, situations where an error from one PASID stops or affects other PASIDs. PASID-specific errors are reported in the completion record for their respective descriptors, unless the error lies in the completion record itself (e.g., a page fault on a completion record address).

[0560] Errors within a descriptor submission or in the descriptor completion record may be reported to the host driver via the software error register (SWERROR). Hardware errors may be reported via the hardware error register (HWERROR).

[0561] In one embodiment of DSA, the following checks are performed when the enable bit in the device enable register is set to 1. • Bus master enable is set to 1. • A combination of PASID, ATS, and PRS functions is enabled (see Table 6-3, Section 6.1.3). • The sum of the WQ size fields of all WQCFG registers is not greater than the total WQ size. For each GRPCFG register, either both WQ and the engine field are 0, or both are non-zero. Each WQ whose size field in the WQCFG register is non-zero belongs to one group. • Each WQ whose size field in the WQCFG register is zero does not belong to any group. Each engine belongs to only one group.

[0562] If any of these checks fail, the device will not be enabled, and an error code will be recorded in the error code field of the device enable register. These checks may be performed in any order. Therefore, the indication of one type of error does not necessarily mean that other errors exist. The same configuration error may result in different error codes at different times or in different versions of the device. If no checks fail, the device will be enabled, and the enable field will be set to 1.

[0563] The device performs the following checks when the WQ enable bit in the WQCFG register is set to 1. • The device is enabled (i.e., the enable field in the device enable register is 1). • The WQ size field is non-zero. • The WQ threshold is not greater than the WQ size field. The WQ Mode field selects a supported mode. That is, if the Shared Mode Support field in WQCAP is 0, the WQ Mode is 1, or if the Dedicated Mode Support field in WQCAP is 0, the WQ Mode is 0. If both the Shared Mode Support and Dedicated Mode Support fields are 1, either of the WQ Mode values ​​is permitted. • If the block support bit for a fault in GENCAP is 0, the block enable field for the WQ fault is 0.

[0564] If any of these checks fail, the WQ is not enabled, and an error code is recorded in the WQ error code field of the WQ configuration register 3500. These checks may be performed in any order. Therefore, the indication of one type of error does not suggest that other errors exist. The same configuration error may result in different error codes at different times or in different versions of the device. If no checks fail, the device is enabled, and the WQ enable field is set to 1.

[0565] In one embodiment, the DSA performs the following checks when a descriptor is received. The WQ identified by the register address used to submit the descriptor is the active WQ (the size field in the WQCFG register is non-zero). If this check fails, an error is logged in the software error register (SWERROR). • If the descriptor was submitted to a shared WQ, It was submitted along with ENQCMD or ENQCMDS. If this check fails, an error is logged in SWERROR. • If privileges are not granted or the descriptor was submitted via the guest portal, the current queue occupancy is not greater than the WQ threshold. If this check fails, a retry response is returned. • If the descriptor was submitted via a privileged portal, the current queue occupancy is less than the WQ size. If this check fails, a retry response is returned. • If the descriptor was submitted to a dedicated WQ, It was submitted using MOVDIR64B. • Queue occupancy is smaller than the WQ size.

[0566] If any of these checks fail, an error is logged in SWERROR.

[0567] In one embodiment, the device performs the following checks on each descriptor as each descriptor is processed. • The value in the operation code field corresponds to a supported operation. This involves checking that the operation is valid in the context in which it was submitted. For example, a batch descriptor within a batch would be treated as an invalid operation code. • No reserved flags are set. This includes flags where the corresponding function bit in the GENCAP register is 0. • No unsupported flags are set. This includes flags reserved for use with specific operations. For example, the fence bit is reserved in descriptors that are enqueued directly rather than as part of a batch. This also includes flags that are disabled within the configuration, such as the fault block flag, which is reserved when the fault block enable field in the WQCFG register is 0. • The requested flags are set. For example, the request completion record flag must be 1 in the descriptor for the comparison operation. • The number of reserved fields is 0. This includes any fields for which no specific action is defined. In some implementations, it is not necessary to check all reserved fields, but the software should clear all unused fields for maximum compatibility. In batch descriptors, the total number of descriptors field is not greater than the maximum batch size field in the GENCAP register. • (Applicable to descriptor types) The transfer size, source size, maximum differential record size, differential record size, and maximum destination size are not greater than the maximum transfer size field in the GENCAP register. In a memory copy descriptor using dualcast, bits 11:0 of ​​the two destination addresses are the same. • If the "Use interrupt message storage" flag is set, the completion interrupt processing will be less than the interrupt message storage size.

[0568] In one embodiment, the completion record address 3804 cannot be translated, the descriptor 3800 is discarded, and an error is recorded in the software error register. Otherwise, if any of these checks fail, the completion record is written with a status field indicating the type of check that failed, and byte completion is set to 0. A completion interrupt is generated if requested.

[0569] These checks may be performed in any order. Therefore, the indication of one type of error in the completion record does not necessarily imply the presence of other errors. The same invalid descriptor may report different error codes at different times or on different versions of the device.

[0570] Reserved fields 3811 within a descriptor may be categorized into three types: fields that are always reserved, fields that are reserved under certain conditions (e.g., based on the function, configuration fields, how the descriptor was submitted, or the values ​​of other fields in the descriptor itself), and fields that are reserved based on the operation type. The following table lists the conditions under which a field is reserved. [Table 26] [Table 27] JPEG2026086720000032.jpg251170

[0571] As already mentioned, DSA supports the use of either physical or virtual addresses. The use of virtual addresses shared with the processing performed on the processor core is called shared virtual memory (SVM). To support SVM, the device provides a PASID when performing address translation, which handles page faults that occur when no translation exists for an address. However, the device itself does not distinguish between virtual and physical addresses. This distinction is controlled by the programming of the IOMMU1710.

[0572] In one embodiment, the DSA supports Address Translation Service (ATS) and Page Request Service (PRS) PCI Express functions, as shown in Figure 28, which illustrates a PCIe logic 2820 communicating with a PCIe logic 2808 using PCDI in order to utilize ATS. The ATS describes the behavior of a device during address translation. When a descriptor enters the descriptor processing unit, device 2801 may request a translation for the address in the descriptor. If there is a hit in device TLB 2822, the device uses the corresponding host physical address (HPA). If there is a failure or authorization failure, in one embodiment of the DSA 2801, it sends an address translation request to IOMMU 2810 (i.e., across the multiprotocol link 2800) for translation. The IOMMU 2810 then searches for the translation by traversing the respective page tables and may return an address translation response containing the translated address and valid authorization. Device 2801 then stores the translation in device TLB 2822 and uses the corresponding HPA for operation. If the IOMMU2810 is unable to find a translation in its page table, it may return an address translation response indicating that no translation is available. If the IOMMU2810's response indicates no translation is available, or indicates a valid permission that does not include the permission requested by the operation, it is considered a page fault.

[0573] The DSA device 2801 may encounter a page fault at one of the following addresses: 1) completion log address 3804, 2) descriptor list address within a batch descriptor, or 3) source buffer or destination buffer address. The DSA device 2801 may block until the page fault is resolved, or it may complete the descriptor early and return partial completion to the client. In one embodiment, the DSA device 2801 always blocks page faults on completion log address 3804 and descriptor list address.

[0574] If the DSA blocks a page fault, it reports the page fault to the IOMMU2810 as a Page Request Service (PRS) request for service by the OS page fault handler. The IOMMU2810 may also notify the OS via an interrupt. The OS validates the address, and if the check is successful, creates a mapping in the page table and returns a PRS response through the IOMMU2810.

[0575] In one embodiment, each descriptor 3800 has a fault block flag indicating whether the DSA2801 should return partial completion or block if a page fault occurs on the source or destination buffer address. If the fault block flag is 1 and a fault is encountered, the descriptor that encountered the fault is blocked until a PRS response is received. Other operations behind the faulty descriptor may also be blocked.

[0576] If the fault block is 0 and the page fault occurs on the source or destination buffer address, the device stops the operation and writes a partial completion status to the completion record along with the fault address and progress information. When client software receives a completion record indicating partial completion, it has the option to restore the fault on the processor to a normal state (for example, by touching the page) and submit a new work descriptor along with the remaining work.

[0577] Alternatively, the software can complete the remaining work on the processor. The Block Support field for Fault in the General Function Register (GENCAP) may indicate device support for this function, and the Block Enable field for Fault in the Work Queue Configuration Register allows the VMM or kernel driver to control whether or not an application is allowed to use the function.

[0578] Device page faults can be relatively costly. In fact, the cost of working for a device page fault may be higher than the cost of working for a processor page fault. Even if the device performs partial work completion instead of blocking the fault upon failure, it still requires software intervention to work for the page fault and resubmit the work, thus incurring further overhead. Therefore, for optimal performance, it is preferable for software to minimize device page faults without incurring pinning and unpinning overhead.

[0579] Batch descriptor lists and source data buffers are typically generated by software immediately before they are submitted to the device. Therefore, their addresses are less likely to fail due to temporal locality. However, completion descriptors and destination data buffers are more likely to fail if they are not touched by software before being submitted to the device. Such failures can be minimized by software explicitly "writing touching" these pages before submission.

[0580] If, during a device TLB invalidation request, an address to be invalidated is being used in a descriptor processing unit, the device waits for the engine to use the address before completing the invalidation request.

[0581] Additional descriptor types

[0582] In some embodiments, one or more of the following additional descriptor types may be used.

[0583] No-op

[0584] Figure 40 shows an exemplary non-op descriptor 4000 and no-op completion record 4001. The no-op operation 4005 does not perform a DMA operation. It may request a completion record and / or a completion interrupt. If it is in a batch, a fence flag may be specified to ensure that the completion of the no-op descriptor occurs after the completion of all previous descriptors in the batch.

[0585] batch

[0586] Figure 41 shows an exemplary batch descriptor 4100 and no-op completion record 4101. The batch operation 4108 queues multiple descriptors at once. The descriptor list address 4102 is the address of a contiguous array of work descriptors to be processed. In one embodiment, each descriptor in the array is 64 bytes. The descriptor list address 4102 is 64-byte aligned. The total number of descriptors 4103 is the number of descriptors in the array. A set of descriptors in an array is called a "batch". The maximum number of descriptors allowed in a batch is given in the maximum batch size field in GENCAP.

[0587] The PASID 4104 and U / S flag 4105 in a batch descriptor are used for all descriptors in the batch. The PASID 4104 and U / S flag fields 4105 in a batch descriptor are ignored. If the completion queue enable flag in batch descriptor 4100 is set, the completion record address enable flag must be 1, and the completion queue address field 4106 contains the address of the completion queue used for all descriptors in the batch. In this case, the completion record address field 4106 in a batch descriptor is ignored. If the completion queue support field in a general function register is 0, the completion queue enable flag is reserved.

[0588] If the completion queue enable flag in a batch descriptor is 0, the completion record for each descriptor in the batch is written to the completion record address 4106 within that descriptor. In this case, if the request completion record flag in the batch descriptor is 1, the completion queue address field is used exclusively as the completion record address 4106 for the batch descriptor.

[0589] The status field 4110 of the batch completion record 4101 indicates success if all descriptors in the batch have completed; otherwise, it indicates that one or more descriptors have completed with a status that does not correspond to success. The descriptor completion field 4111 of the completion record contains the total number of descriptors in the batch that were processed, regardless of whether they were successful or not. Descriptor completion 4111 may be less than the total number of descriptors 4103 if there are fences in the batch or if a page fault occurred while reading the batch.

[0590] drain

[0591] Figure 42 shows an exemplary drain descriptor 4200 and drain completion record 4201. Drain operation 4208 waits for the completion of all outstanding descriptors associated with PASID 4202 in the work queue to which drain descriptor 4200 was submitted. This descriptor may be used during a normal shutdown by processes using the device. To wait for all descriptors associated with PASID 4202, the software should submit a separate drain operation to each work queue in which PASID 4202 was used. The software should ensure that there are no descriptors with a particular PASID 4202 to be submitted to the work queue after drain descriptor 4201 has been submitted and before it has completed.

[0592] Drain descriptor 4201 does not need to be included in the batch and will be treated as an unsupported operation type. The drain should define a request completion record or a request completion interrupt. The completion notification will occur after other descriptors have completed.

[0593] Memory Movement

[0594] Figure 43 shows an exemplary memory move descriptor 4300 and memory move completion record 4301. The memory move operation 4308 copies memory from source address 4302 to destination address 4303. The number of bytes copied is given by the transfer size 4304. There are no alignment requirements for the memory address or transfer size. If the source and destination regions overlap, the memory copy is performed as if the entire source buffer were copied to a temporary space and then copied to the destination buffer. This can be achieved by reversing the direction of the copy if the start of the destination buffer overlaps with the end of the source buffer.

[0595] If the operation is partially completed due to a page fault, the direction field 4310 of the completion record is 0 if the copy was performed starting from the beginning of the source and destination buffers, and the direction field is 1 if the copy direction was reversed.

[0596] To resume the operation after a partial completion, if the direction is 0, the source and destination address fields 4302-4303 in the consecutive descriptor should be incremented by byte completion, and the transfer size should be reduced by byte completion 4311. If the direction is 1, the transfer size 4304 should be reduced by byte completion 4311, but the source and destination address fields 4302-4303 should be the same as in the original descriptor. Note that if a subsequent partial completion occurs, the direction field 4310 does not have to be the same as for the first partial completion.

[0597] Full (Fill)

[0598] Figure 44 shows an exemplary fill descriptor 4400. The memory fill operation 4408 fills the memory at destination address 4406 with the value in the pattern field 4405. The pattern size may be 8 bytes. To use a smaller pattern, the software must duplicate the pattern in the descriptor. The number of bytes written is given by the transfer size 4407. The transfer size does not need to be a multiple of the pattern size. There are no alignment requirements for the destination address or transfer size. If the operation is partially completed due to a page fault, the byte completion field in the completion record contains the number of bytes written to the destination before the fault occurred.

[0599] comparison

[0600] Figure 45 shows an exemplary comparison descriptor 4500 and comparison completion record 4501. The comparison operation 4508 compares the memory at address 4504 of source 1 with the memory at address 4505 of source 2. The number of bytes to compare is given by the transfer size 4506. There is no alignment request for the memory address or transfer size 4506. The completion record address must be valid, and the completion record address flags must be 1. The result of the comparison is written to the result field 4510 of the completion record 4501. A value of 0 indicates that the two memory regions match, and a value of 1 indicates that they do not match. If result 4510 is 1, the byte completion field 4511 of the completion record shows the byte offset of the first difference. If the operation is partially completed due to a page fault, the result is 0. If a difference is detected, that difference will be reported instead of a page fault.

[0601] If the operation is successful and the check result flag is 1, the status field 4512 of the completion record is set according to the result and predicted result, as shown in the table below. This allows subsequent descriptors in the same batch as the fence flag to continue or stop batch execution based on the comparison result. [Table 28]

[0602] Comparison Intermediate

[0603] Figure 46 shows an exemplary comparison intermediate descriptor 4600. The comparison intermediate process 4608 compares the memory at source address 4601 with the value in pattern field 4602. The pattern size is 8 bytes. To use a smaller pattern, the software must duplicate the pattern in the descriptor. The number of bytes to be compared is given by the transfer size 4603. The transfer size does not need to be a multiple of the pattern size. The completion record address valid and requested completion record flags must be 1. The completion record address 4604 must be valid. The result of the comparison is written to the result field of the completion record. A value of 0 indicates that the memory region matches the pattern, and a value of 1 indicates that it does not match. If the result is 1, the byte completion field of the completion record indicates the location of the first difference. It may not be the exact byte location, but it is guaranteed not to be greater than the first difference. If the operation is partially completed due to a page fault, the result is 0. If a difference is detected, that difference will be reported instead of a page fault. In one embodiment, the completion record format for the comparison intermediate and the behavior of the check results and prediction results are the same as for the comparison.

[0604] Creation difference record

[0605] Figure 47 shows an exemplary creation differential record descriptor 4700 and creation differential record completion record 4701. The creation differential record operation 4708 compares the memory at address 4705 of source 1 with the memory at address 4702 of source 2 to generate a differential record containing the information necessary to update source 1 to match source 2. The number of bytes to be compared is given by the transfer size 4703. As described below, the transfer size is limited by the maximum offset that can be stored in the differential record. There are no alignment requests for memory addresses or transfer sizes. The completion record address valid and request completion record flags must be 1, and the completion record address 4704 must be valid.

[0606] The maximum size of the differential record is given by the maximum differential record size of 4709. The maximum differential record size of 4709 should be a multiple of the difference size (10 bytes) and must not be greater than the maximum transfer size within GENCAP. The actual size of the differential record depends on the number of differences found between source 1 and source 2, which is written to the differential record size field 4710 of the completed record. If the space required for the differential record exceeds the maximum differential record size of 4709 specified in the descriptor, the operation is completed with a partial differential record.

[0607] The result of the comparison is written to the result field 4711 of the completion record 4701. If the two regions match exactly, the result is 0, the difference record size is 0, and the byte completion is 0. If the two regions do not match, the complete set of differences is written to the difference record, the result is 1, the difference record size includes the sum of all differences obtained, and the byte completion is 0. If the two regions do not match and the space required to record all differences exceeds the maximum difference record size, the result is 2, the difference record size 4710 includes the size of the set of differences written to the difference record (typically equal to or approximately equal to the difference record size specified in the descriptor), and the byte completion 4712 includes the number of bytes compared before the space in the difference record was exceeded.

[0608] If the operation is partially completed due to a page fault, as described in the previous paragraph, the result 4711 is either 0 or 1, the byte completion 4712 includes the number of bytes compared before the page fault occurred, and the differential record size includes the space used in the differential record before the page fault occurred.

[0609] The format of the differential record is shown in Figure 48. The differential record contains an array of deltas. Each delta contains a 2-byte offset 4801 and an 8-byte block of data 4802 from source 2 that is different from the corresponding 8 bytes in source 1. The total size of the differential record is a multiple of 10. Since offset 4801 is a 16-bit field representing a multiple of 8 bytes, the maximum offset that can be represented is 0x7FFF8, and therefore the maximum transfer size is 0x80000 bytes (512KB).

[0610] If the operation is successful and the check result flag is 1, the status field of the completion record is set according to the result and predicted result, as shown in the table below. This allows subsequent descriptors in the same batch as the fence flag to continue or stop batch execution based on the result of differential record creation. Bits 7:2 of the predicted result are ignored. [Table 29]

[0611] Compatibility Difference Record

[0612] Figure 49 shows an exemplary alignable differential record descriptor 4901. The alignable differential record operation 4902 applies the differential record to the memory content at destination address 4903. The differential record address 4904 is the address of the differential record created by the create differential record operation 4902, which completed with a result equal to 1. The differential record size 4905 is the size of the differential record as reported in the completion record of the create differential record operation 4902. Destination address 4903 is the address of the buffer containing the same content as the memory at source 1's address, if the differential record had been created. The transfer size 4906 is the same transfer size used if the differential record had been created. After the alignable differential record operation 4902 is completed, the memory at destination address 4903 matches the content that was in the memory at source 2's address if the differential record had been created. There are no alignment requirements for the memory address or transfer size.

[0613] If a page fault occurs during the conforming differential record operation 4902, the byte completion field of the completed record will contain the number of bytes of differential record that successfully applied to the destination. If the software chooses to submit another descriptor to resume the operation, the successive descriptor will contain the same destination address 4903 as the original. The differential record address 4904 should be incremented by the byte completion (and thus point to the first unapplied differential), and the differential record size 4905 should be decreased by the byte completion.

[0614] Figure 50 shows an example of using the creation differential record and adaptation differential record operations. First, the creation differential record operation 5001 is executed. The creation differential record operation 5001 reads two source buffers—source 1 and 2—and writes a differential record 5010 to its completion record 5003, recording the actual differential record size 5004. The adaptation differential record operation 5005 retrieves the contents of the differential record written by the creation differential record operation 5001, along with its size and a copy of the source 1 data, and updates the destination buffer 5015 to be a replica of the original source 2 buffer. The creation differential record operation includes a maximum differential record size 5002.

[0615] Memory copy using dual cast

[0616] Figure 51 shows an exemplary memory copy descriptor 5100 using dualcast and a memory copy completion record 5102 using dualcast. The memory copy using dualcast operation 5104 copies memory from source address 5105 to both destination address 5106 and destination address 5107. The number of bytes copied is given by the transfer size 5108. There are no alignment requirements for the source address or transfer size. Bits 11:0 of ​​the two destination addresses 5106-5107 should be the same.

[0617] If the source region overlaps with either of the destination regions, the memory copy is performed as if the entire source buffer were copied to a temporary space and then copied to the destination buffer. This can be done by reversing the direction of the copy if the beginning of the destination buffer overlaps with the end of the source buffer. If the source region overlaps with both destination regions, or if the two destination regions overlap, it is an error. If the operation is partially completed due to a page fault, the copy operation stops after writing the same number of bytes to both destination regions, and the direction field 5110 of the completion record is 0 if the copy is performed starting from the beginning of both the source and destination buffers, and the direction field is 1 if the direction of the copy is reversed.

[0618] To resume the operation after a partial completion, if direction 5110 is 0, the source 5105 and both destination address fields 5106-5107 in the consecutive descriptor should be incremented by byte completion 5111, and the transfer size 5108 should be reduced by byte completion 5111. If direction is 1, the transfer size 5108 should be reduced by byte completion 5111, but the source 5105 and destination 5106-5107 address fields should be the same as in the original descriptor. Note that if a subsequent partial completion occurs, the direction field 5110 does not have to be the same as for the first partial completion.

[0619] Cyclic Redundancy Check (CRC) generation

[0620] Figure 52 shows an exemplary CRC generation descriptor 5200 and CRC generation 5201. The CRC generation process 5204 calculates the CRC in memory at the source address. The number of bytes used for CRC calculation is given by the transfer size 5205. There is no alignment request for the memory address or transfer size 5205. The completion record address valid and request completion record flags must be 1, and the completion record address 5206 must be valid. The calculated CRC value is written to the completion record.

[0621] If an operation is partially completed due to a page fault, the partial CRC result is written to the completion record along with the page fault information. When the software corrects the fault and resumes the operation, this partial result must be copied to the CRC seed field of the continuary descriptor. Otherwise, the CRC seed field should be 0.

[0622] Copy using CRC generation

[0623] Figure 53 shows an exemplary copy using the CRC generation descriptor 5300. The copy using the CRC generation process 5305 copies memory from source address 5302 to destination address 5303 and calculates the CRC for the copied data. The number of bytes copied is given by the transfer size 5304. There are no alignment requirements for memory addresses or transfer sizes. If the source and destination regions overlap, it is an error. The completion record address valid and the request completion record flag must be 1, and the completion record address must be valid. The calculated CRC value is written to the completion record.

[0624] If an operation is partially completed due to a page fault, the partial CRC result is written to the completion record along with the page fault information. When the software corrects the fault and resumes the operation, this partial result must be copied to the CRC seed field of the continuous descriptor. Otherwise, the CRC seed field should be 0. In one embodiment, the completion record format for copying using CRC generation is the same as the format for CRC generation.

[0625] Inserting a Data Integrity Field (DIF)

[0626] Figure 54 shows an exemplary DIF insertion descriptor 5400 and DIF insertion completion record 5401. The DIF insertion operation 5405 copies memory from source address 5402 to destination address 5403, calculates the data integrity field (DIF) on the source data, and inserts the DIF into the output data. The number of source bytes copied is given by the transfer size 5406. The DIF calculation is performed on each block of source data, for example, 512, 520, 4096, or 4104 bytes. The transfer size should be a multiple of the source block size. The number of bytes written to the destination is the transfer size plus 8 bytes for each source block. There are no alignment requirements for memory addresses. If the source and destination areas overlap, it is an error. If the operation is partially completed due to a page fault, the updated values ​​of the reference tag and application tag are written to the completion record along with the page fault information. If the software corrects the fault and resumes the operation, these fields may be copied to a contiguous descriptor.

[0627] DIF strip

[0628] Figure 55 shows an exemplary DIF strip descriptor 5500 and DIF strip completion record 5501. The DIF strip operation 5505 copies memory from source address 5502 to destination address 5503, calculates a data integrity field (DIF) on the source data, and compares the calculated DIF with the DIF contained in the data. The number of source bytes to be read is given by the transfer size 5506. The DIF calculation is performed on each block of source data, which may be 512, 520, 4096, or 4104 bytes. The transfer size should be a multiple of the source block size plus 8 bytes for each source block. The number of bytes written to the destination is the transfer size minus 8 bytes for each source block. There are no alignment requirements for memory addresses. If the source and destination areas overlap, it is an error. If the operation is partially completed due to a page fault, the updated values ​​of the reference tag and application tag are written to the completion record along with the page fault information. When the software corrects the fault and resumes operation, these fields may be copied to a contiguous descriptor.

[0629] DIF update

[0630] Figure 56 shows an exemplary DIF update descriptor 5600 and DIF update completion record 5601. A memory move using the DIF update operation 5605 copies memory from source address 5602 to destination address 5603, calculates a data integrity field (DIF) on the source data, and compares the calculated DIF with the DIF contained in the data. Simultaneously, the DIF on the source data is calculated using the destination DIF field in the descriptor, and the calculated DIF is inserted into the output data. The number of source bytes to be read is given by the transfer size 5606. DIF calculations are performed on each block of source data, which may be 512, 520, 4096, or 4104 bytes. The transfer size 5606 should be a multiple of the source block size plus 8 bytes for each source block. The number of bytes written to the destination is the same as the transfer size 5606. There are no alignment requirements for memory addresses. If the source and destination regions overlap, it is an error. If an operation is partially completed due to a page fault, the updated values ​​of the source and destination reference tags, and the application tag, are written to the completion record along with the page fault information. If the software corrects the fault and resumes the operation, these fields may be copied to a sequential descriptor.

[0631] Table AA below shows the DIF flags used in one embodiment. Table BB shows the source DIF flags used in one embodiment, and Table CC shows the destination DIF flags used in one embodiment. [Table 30]

[0632] Source DIF flag [Table 31]

[0633] Destination DIF flag [Table 32]

[0634] In one embodiment, the DIF result field reports the status of the DIF operation. This field may be defined only for DIF strip and update operations, and only if the status field in the completion record is success or success with an incorrect predicate. The following table DD shows exemplary DIF result field codes. [Table 33]

[0635] The F detection condition is met if one of the following conditions shown in Table EE is true. [Table 34]

[0636] If the operation is successful and the check result flag is 1, the status field in the completion record is set according to the DIF result, as shown in Table FF below. This allows subsequent descriptors in the same batch as the fence flag to continue or stop batch execution based on the result of the operation. [Table 35]

[0637] Cash Flush

[0638] Figure 57 shows an exemplary cache flush descriptor 5700. Cache flush operation 5705 flushes the processor cache at the destination address. The number of bytes flushed is given by the transfer size 5702. The transfer size does not need to be a multiple of the cache line size. There are no alignment requirements for the destination address or transfer size. Any cache line partially covered by the destination area is flushed.

[0639] If the destination cache filter is 0, the affected cache line may be disabled from each level of the cache hierarchy. The cache line contains the modified data at any level of the cache hierarchy, and the data is written back to memory. This is similar to the behavior of the CLFLUSH instruction implemented in some processors.

[0640] If the destination cache filter is 1, the modified cache line is written to main memory but not evicted from the cache. This is similar to the behavior of the CLWB instruction in some processors.

[0641] A term accelerator is sometimes used herein to offload or perform any type of computation or I / O task by referring to a loosely coupled agent that may be used by software running on the host processor. Depending on the type of accelerator and the model of use, these may be tasks that perform data movement to memory or storage, computation, communication, or any combination thereof.

[0642] "Loose coupling" refers to how these accelerators are exposed and accessed by the host software. Specifically, they are not exposed as processor ISA extensions, but rather as PCI Express countable endpoint devices on the platform. Loose coupling allows these agents to accept work requests from the host software and operate asynchronously with respect to the host processor.

[0643] The "accelerator" may be a programmable agent (e.g., a GPU / GPGPU), a fixed-function agent (e.g., a compression or encryption engine), or a reconfigurable agent such as a field-programmable gate array (FPGA). Some of these may be used for computation offloading, while others (e.g., RDMA or host fabric interfaces) may be used for packet processing, communication, storage, or message passing operations.

[0644] Accelerator devices may be physically integrated at different levels, including on-die (i.e., the same die as the processor), on-package, on the chipset, or on the motherboard, or they may be separate PCIe-connected devices. Regarding integrated accelerators, even if they are enumerated as PCI Express endpoint devices, some of these accelerators may be coherently mounted (to an on-die coherent fabric or an external coherent interface), while others may be mounted to an internal non-coherent interface or an external PCI Express interface.

[0645] Conceptually, "accelerators" and high-performance I / O device controllers are similar. What distinguishes them are features such as integrated / shared virtual memory, the ability to operate on paged memory, user-mode work submission, task scheduling / preemption, and support for low-latency synchronization. As a result, accelerators can be considered a new, improved category of high-performance I / O devices.

[0646] Off-road operation model

[0647] The accelerator off-road operation model can be broadly classified into three use categories.

[0648] 1. Streaming: In the streaming offload model, small units of work are streamed to an accelerator at a high rate. A typical example of this use is a network data plane that performs various types of packet processing at high rates.

[0649] 2. Low Latency: For some offload uses, the latency of offload operations (both dispatching tasks to accelerators and the accelerators performing them) is of significant importance. An example of this use is a low-latency message passing configuration model that includes remote retrieval, put, and atomic operations over the host fabric.

[0650] 3. Scalable: Scalable offloading refers to the direct access of computing accelerator services to a large number of client applications (within and via virtual machines) (e.g., from the highest ring in a tiered protection domain, such as Ring 3) without the constraints imposed by accelerator devices such as multiple work queues or multiple doorbells supported on the device. Some of the accelerator devices and processor interconnects described herein fall into this category. Such scalability, applicable to computing offload devices that support work time-sharing / scheduling, such as GPUs, GPGPUs, FPGAs, or compression accelerators, or message passing, utilizes enterprise databases and other systems with significant scalability requirements for lock-free operations.

[0651] Work dispatch across off-road models

[0652] Each of the off-road operation models described above faces its own unique work dispatch challenges, as outlined below.

[0653] 1. Work dispatch for the use of streaming offload

[0654] A typical work dispatch model for streaming involves using a work queue residing in memory. Specifically, the device configures the location and size of the work queue in memory. The hardware implements a doorbell (tail pointer) register that is updated by software when a new work element is added to the work queue. The hardware reports the current head pointer to the software to enhance producer-consumer flow control on the work queue elements. A typical model for streaming involves software checking for free space in the work queue by examining the head and tail pointers cached in software (often maintained in host memory by hardware to avoid the overhead of software-based UC MMIO reads), adding new work elements to the work queue residing in memory, and updating the tail pointer using a doorbell register write to the device.

[0655] A doorbell write is typically a 4-byte or 8-byte uncached (UC) write to MMIO. On some processors, a UC write is a serialized operation that blocks all younger stores in the processor pipeline because the UC write is issued before it is posted by the platform, ensuring that older stores are globally monitored before the UC write (required for producer-consumer use) is issued. Typical latency for UC write operations on Xeon server processors is on the order of 80-100 nanoseconds, limiting streaming offload performance during the time all younger store operations are blocked by the core.

[0656] One approach to addressing the serialization of younger storage following UC doorbell writes is to use write combining (WC) storage operations for doorbell writes (due to the weak ordering of WC), but using WC storage for doorbell writes presents several challenges. The doorbell write size (typically a DWORD or QWORD) is smaller than the cache line size. These partial writes introduce additional latency due to the processor maintaining the partial writes in its write combining buffer (WCB) for potential write combining opportunities, and thus introduce latency for doorbell writes issued from the processor. Software can issue these through explicit storage fences, causing the same serialization to younger storage as with UC doorbells.

[0657] Another issue with WC-mapped MMIO is that erroneous predictions and speculative reads (using MOVNTDQA) are exposed to WC-mapped MMIO (which have registers that can affect the read side). This is cumbersome for devices because they need to host WC-mapped doorbell registers in separate pages from the rest of the UC-mapped MMIO registers. This also presents challenges in virtualization use, as VMM software can no longer ignore guest memory types and force UC mapping to arbitrary device MMIO exposed to the guest using EPT page tables.

[0658] The MOVDIRI instructions described herein address the above limitations of using UC or WC storage for doorbell writing in the use of these streaming offloads.

[0659] 2. Work dispatch for low-latency off-road use

[0660] Some types of accelerator devices are highly optimized to complete requested operations with minimal latency. Unlike streaming accelerators (optimized for throughput), these accelerators generally implement device-hosted work queues (exposed through device MMIO) to avoid DMA read latency in fetching work elements (and, in some cases, the same data buffer) from memory-hosted work queues. Instead, host software submits work to the device-hosted work queue exposed through device MMIO by directly writing work descriptors (and, in some cases, data as well). Examples of such devices include host fabric controllers, remote DMA (RDMA) devices, and newer storage controllers such as Non-Volatile Memory (NVM) Express. The use of device-hosted work queues presents few challenges with respect to existing ISAs.

[0661] To avoid the serialization overhead of UC writes, the MMIO address of a device-hosted work queue is typically mapped as a WC. This exposes the same challenges as a doorbell mapped to a WC for streaming accelerators.

[0662] Furthermore, using WC storage to a device-hosted work queue requires the device to adhere to the write atomicity behavior of several processors. For example, the atomicity of write operations up to 8 bytes in size only guarantees that some processors write within cache line boundaries (and for lock operations), but does not define the atomicity of any guaranteed write completion. Write operation atomicity is the granularity at which processor storage operations are monitored by other agents, and is a characteristic of the processor instruction set architecture and coherency protocol. Write completion atomicity is the granularity at which uncacheable storage operations are monitored by the receiver (the memory controller in the case of memory, or the device in the case of MMIO). Write completion atomicity is stronger than write operation atomicity and is a platform feature as well as a processor instruction set architecture. Without write completion atomicity, a processor instruction performing an N-byte uncacheable storage operation may be received as multiple (tone) write transactions by a device-hosted work queue. Currently, device hardware must protect against such tone writes by tracking each word or data of the work descriptor written to the device-hosted work queue.

[0663] The MOVDIR64B instruction described herein addresses the above limitations by supporting 64-byte writes with guaranteed atomicity of 64-byte write completion. MOVDIR64B is also useful for other applications, such as writing to persistent memory (NVM attached to a memory controller) and replicating data through the system via an opaque bridge (NTB).

[0664] 3. Work dispatch for scalable off-road use

[0665] The traditional approach to submitting work from an application to an I / O device involves making system calls to the kernel I / O stack, which forwards requests to the I / O controller device through the kernel device driver. While this approach is scalable (any number of applications can share the service of a device), it often introduces latency and overhead of the serialized kernel I / O stack, which becomes a performance bottleneck for high-performance devices and accelerators.

[0666] To support low-overhead work dispatch, some high-performance devices enable direct work dispatch to the device and support direct Ring 3 access to check work completion. In this model, several device resources (doorbell, work queue, completion queue, etc.) are allocated and mapped to the application's virtual address space. Once mapped, Ring 3 software (user-mode drivers or libraries) can directly dispatch work to the accelerator. For devices supporting Shared Virtual Memory (SVM) functionality, the doorbell and work queue are set up by the kernel-mode driver to identify the processing address space identifier (PASID) of the application processing to which the doorbell and work queue are mapped. When processing a work item dispatched through a particular work queue, the device uses the respective PASID configured for that virtual work queue for physical address translation through the I / O Memory Management Unit (IOMMU).

[0667] One of the challenges with DirectRing 3 work submission is the scalability issue. The number of application clients that can directly submit work to an accelerator device depends on the number of queues / doorbells (or device-hosted work queues) supported by the accelerator device. This is because the number of these resources supported by the accelerator device design is fixed, as doorbells or device-hosted work queues are statically allocated / mapped to application clients. Some accelerator devices attempt to "address" this scalability challenge by over-accommodating their doorbell resources (by dynamically removing and reattaching doorbells for application requests), but this is often cumbersome and difficult to scale. The limited doorbell / work queue resources are further constrained for devices that support I / O virtualization (e.g., Single Root I / O Virtualization (SR-IOV)), as these need to be segmented across various virtual functions (VFs) that are allocated to different virtual machines.

[0668] Scaling issues are of paramount importance to high-performance message-passing accelerators used by enterprise applications such as databases for lockless operations (some of which have RDMA devices supporting 64K to 1M queue pairs), and to compute accelerators that support sharing accelerator resources across tasks submitted by numerous clients.

[0669] The ENQCMD / S instruction described herein addresses the scaling limitations mentioned above in order to enable an unlimited number of clients to subscribe to and share the resources of the work queue on the accelerator.

[0670] One embodiment includes a new type of storage operation by a processor core, including direct storage and enqueued storage.

[0671] In one embodiment, direct storage is generated by the MOVDIRI and MOVDIR64B instructions described herein.

[0672] Caching Capability: Similar to UC and WC storage, direct storage is not cacheable. If a direct storage is issued to a cached address, the line is written back (if modified) and invalidated from the cache before the direct storage.

[0673] Memory Ordering: Similar to WC storage, direct storage is weakly ordered. Specifically, they are not ordered to different addresses than older WB / WC / NT storage, CLFLUSHOPT, and CLWB. Younger WB / WC / NT storage, CLFLUSHOPT, or CLWB to different addresses can bypass older direct storage. Direct storage to the same address is always ordered using older storage (including direct storage) to the same address. Direct storage is fenced by any operation that strengthens storage fencing (e.g., SFENCE, MFENCE, UC / WP / WT storage, locks, IN / OUT instructions, etc.).

[0674] Write combining: Direct storage behaves differently from normal WC storage. Specifically, direct storage is subject to immediate eviction from the write combine buffer and, consequently, cannot be combined with a younger storage (including direct storage) to the same address. Older WC / NT storages held in the write combine buffer may be combined with a younger direct storage to the same address, and uses that need to avoid such combinations must explicitly store a fenced WC / NT storage before performing a direct storage to the same address.

[0675] Atomicity: Direct storage supports atomicity of write completion with respect to the write size of the instruction issuing the direct storage. For MOVDIR64B, if the destination is 4-byte aligned (or 8-byte aligned), the atomicity of write completion is 4 bytes (or 8 bytes). For MOVDIR64B, the destination is forced to be 64-byte aligned, and the atomicity of write completion is 64 bytes. Atomicity of write completion ensures that the direct storage is not torn across multiple write transactions, such as those handled by the memory controller or root complex. Root complex implementations on processors that support direct storage ensure that the direct storage is transferred over the external PCI Express fabric (and the internal I / O fabric within the SoC that follows PCI Express ordering) as a single non-torn posted write transaction. Read operations to memory locations from any agent (processor or non-processor agent) refer to either all or none of the data written by the instruction issuing the direct storage operation.

[0676] Ignoring Destination Memory Type: Direct storage ignores the destination address memory type (including UC / WP types) and always follows weak ordering. This allows software to map device MMIO as a UC and access specific registers (e.g., doorbell or device host type work queue registers) using a direct storage instruction (MOVDIRI or MOVDIR64B) while continuing to access other registers that may have strict serialization requirements using normal MOV operations that follow UC ordering for each memory type of the mapped UC. This also allows direct storage instructions to be operated from within guest software, while virtual machine monitor (VMM) software (which does not have device-specific knowledge) maps guest-exposed MMIO as a UC in the processor-extended page table (EPT) and ignores the guest memory type.

[0677] SoCs that support direct storage must ensure the atomicity of write completion for direct storage, as follows:

[0678] Direct Storage to Main Memory: With regard to direct storage to main memory, the coherent fabric and system agents ensure that all data bytes within a direct storage are issued to the home agent or other global observability (GO) point in response to a memory request as a single (non-torn) write transaction. For platforms supporting persistent memory, the home agent, memory controller, memory-side cache, inline memory encryption engine, memory bus (e.g., DDR-T) that mounts the persistent memory, and the persistent memory controller must each support the same or higher granularity of atomic write completion for direct storage. Thus, software can perform 64-byte direct storage to memory (volatile or persistent) using MOVDIR64B, and it can be guaranteed that all 64-byte writes are processed atomically by all agents. Similar to normal writes to persistent memory, if software needs to explicitly commit to persistence, it performs the direct storage using a fence / commit / fence sequence.

[0679] Direct Storage to Memory-Mapped I / O: With respect to direct storage to memory-mapped I / O (MMIO), the coherent fabric and system agent must ensure that all data bytes in a direct storage are issued to the root complex (a global observability point for requests to MMIO) as a single (non-torn) write transaction. The root complex implementation must ensure that each direct storage is processed and transferred as a single (non-torn) posted write transaction on the internal I / O fabric where the PCI Express Root Complex Integration Endpoint (RCIEP) and Root Port (RP) are installed. The PCI Express Root Port and Switch Port must transfer each direct storage as a single posted write transaction. Atomicity of write completion is not specified or guaranteed for direct storage targeting devices on or behind secondary bridges (e.g., legacy PCI, PCI-X bridges) or secondary buses (e.g., USB, LPC, etc.).

[0680] It should be noted that some SoC implementations already guarantee the atomicity of write completion for WC write requests. Specifically, partial line WC writes (WCiL) and full line WC writes (WCiLF) are already handled with atomicity of write completion by the system agent, memory controller, root complex, and I / O fabric. In such implementations, the processor does not need to distinguish between direct writes and WC writes, and the difference in behavior between direct storage and WC storage resides within the processor core. Therefore, no changes are proposed to the internal or external fabric specifications for direct writes.

[0681] The handling of direct writes received via PCI Express endpoints or RCIEPs is specific to the device implementation. Depending on the device's programming interface, the device and its drivers may require certain registers (e.g., doorbell registers or device-host type work queue registers) to always be written using direct store instructions (e.g., MOVDIR64B), and may process these atomically within the device. Writes to other registers on the device may be processed by the device without any consideration or expectation of atomicity. With respect to RCIEPs, if registers with atomic write requirements are implemented for access via sideband or private wire interfaces, such implementations must ensure the atomic write characteristic through implementation-specific means.

[0682] In one embodiment, the enqueue storage is generated by the ENQCMD and ENQCMDS instructions described herein. The designated target of the enqueue storage is a shared work queue (SWQ) on the accelerator device. In one embodiment, the enqueue storage has the following characteristics:

[0683] Non-posted: Enqueue storage generates a 64-byte non-postwrite transaction to the target address and receives a completion response indicating success or retry status. The success / retry status returned in the completion response may be returned to the software by the ENQCMD / S instruction (e.g., with a zero flag).

[0684] Caching Capability: In one embodiment, enqueued storage is not cacheable. Platforms that support enqueued storage enforce that enqueued non-posted writes are forwarded only to an address (MMIO) range that is explicitly enabled to accept these storages.

[0685] Memory Ordering: Enqueue storage may update the architecture state (e.g., zero flag) that has a non-posted write complete status. Therefore, at most one enqueue storage may remain unprocessed from a given logical processor. In this sense, an enqueue storage from a logical processor cannot pass another enqueue storage issued from the same logical processor. Enqueue storage does not order to a different address than older WB / WC / NT storage, CLFLUSHOPT, or CLWB. Software that needs to enforce such ordering may use explicit storage fencing after such storage and before the enqueue storage. Enqueue storage is always ordered to the same address as the older storage.

[0686] Alignment: The ENQCMD / S instruction forces the enqueue storage destination address to be 64-byte aligned.

[0687] Atomicity: Enqueue storage generated by the ENQCMD / S instruction supports 64-byte write-to-completion atomicity. Write-to-completion atomicity ensures that the enqueue storage is not torn into multiple transactions, such as those handled by the root complex. Root complex implementations on processors supporting enqueue storage ensure that each enqueue storage is transferred to the endpoint device as a single (non-torn) 64-byte non-post-write transaction.

[0688] Ignoring Destination Memory Type: Similar to direct storage, enqueue storage ignores the destination address memory type (including UC / WP type) and always follows the ordering described above. This allows software to map device MMIO as UC and continue accessing shared work queue (SWQ) registers using ENQCMD / S instructions while continuing to access other registers using normal MOV instructions or through direct storage (MOVDIRI or MOVDIR64B) instructions. This also allows enqueue storage instructions to be executed from within guest software, while VMM software (which does not have device-specific knowledge) maps guest-exposed MMIO as UC in the Processor Extended Page Table (EPT) and ignores the guest memory type.

[0689] Platform considerations for enqueue storage

[0690] In some embodiments, a specific set of platform-integrated devices supports shared work queue (SWQ) functionality. These devices may be mounted to the root complex through an internal I / O fabric. These devices may be exposed to host software as either PCI Express root complex integrated endpoints (RCIEPs) or PCI Express endpoint devices behind virtual root ports (VRPs).

[0691] Platforms supporting integrated devices with SWQ should restrict the forwarding of enqueue non-posted write requests on the internal I / O fabric to only those devices. This is to ensure that the new transaction type (enqueue non-posted write) is not treated as an ill-formed transaction layer packet (TLP) by an endpoint device that is unaware of the enqueue.

[0692] Enqueue storage to all other addresses (including the main memory address range and all other memory-mapped address ranges) is terminated by the platform, and a normal (non-error) response is returned to the issuing processor with a retry completion status. Since unprivileged software (ring 3 software in VMX non-root mode, or ring 0 software) can generate enqueue-nonposted write transactions by executing the ENQCMD / S instruction, there are no platform errors generated at the end of such enqueue storage.

[0693] The root complex implementation should ensure that enqueue storage is processed and transferred as a single (non-torn) non-posted write transaction on the internal I / O fabric to integrated devices that support SWQ.

[0694] Platform performance evaluation

[0695] This section describes some performance considerations in the system agent and the enqueue storage process performed by the system agent.

[0696] Relaxed ordering for the allocation of System Agent Tracker (TOR) entries for enqueue storage.

[0697] To maintain memory integrity, system agent implementations typically enforce strict ordering for requests to cache line addresses (when allocating TOR entries) for coherent memory and MMIO. While this is necessary to support overall ordering for coherent memory access, this strict ordering for enqueue storage introduces performance issues. This is because enqueue storage targets a shared work queue (SWQ) on the device, thereby commonly causing multiple logical processors with the same destination SWQ address to issue enqueue storage requests. Furthermore, unlike regular storage posted to the system agent, enqueue storage is non-posted and incurs latency similar to reads. To bypass the condition that only one unprocessed enqueue storage is allowed for a shared work queue, system agent implementations need to relax the strict ordering for enqueue storage requests to the same address and instead allow TOR allocations for multiple in-flight enqueue storages to the same address. Since a logical processor can issue at most one enqueue storage at a time, system agents / platforms can process each enqueue storage independently without worrying about ordering.

[0698] Support for multiple unprocessed enqueued non-posted writes in the I / O bridge agent.

[0699] I / O bridge implementations typically limit the number of non-posted (read) requests supported in downstream paths to a small number of recipients (often a single request). This is because processor reads to MMIO (usually UC reads) do not incur significant performance risks for most uses and support large queue depths for the buffers required to read the returned data, thus increasing hardware costs. Applying this limited queuing to enqueue non-posted writes can detriment performance, as enqueue storage is typically expected to be used for work dispatch to accelerator devices. I / O bridge implementations are recommended to support increased queue depths (a certain percentage of the number of logical processors, since a logical processor can only have one pending enqueue storage request at a time) for improved enqueue non-posted write bandwidth. Unlike read requests, enqueue storage does not incur hardware costs for data buffers because enqueue non-posted write completion simply returns a completion status (success vs. retry) and does not return any data.

[0700] Virtual channel support for enqueue non-posted lights

[0701] Unlike typical memory read and write requests on an I / O bus with producer-consumer ordering requests (as identified, for example, by PCI Express Transaction Ordering), enqueued non-posted writes do not make ordering requests on the I / O bus. This allows the use of non-VC0 virtual channels to issue enqueued non-posted writes and return their completion. The benefit of using non-VC0 channels is that enqueued non-posted write completions can have better latency (fewer cycles delaying the core) by avoiding ordering from the device to the host behind upstream posted writes on VC0. Implementations are advised to minimize enqueued non-posted completion latency by carefully considering the use of integrated devices.

[0702] Enqueue Non-Posted Light Intermediate Stop

[0703] To handle specific flow control in high-latency situations (e.g., waking up internal links or power management on a locked flow), intermediate agents (such as system agents or I / O bridges) can drop a legitimate enqueue storage request and return completion along with a retry response to the issuing core. The software issuing the enqueue storage does not have direct visibility if the retry response is from an intermediate agent or target, or if the software performs a normal retry (potentially with some backoff).

[0704] Implementations that perform such intermediate pauses must take great care to ensure that such behavior does not expose any arbitrary denial-of-service attacks to software clients sharing the SWQ.

[0705] Support for shared work queues on endpoint devices

[0706] Figure 34 illustrates the concept of a shared work queue (SWQ), which allows multiple non-collaborative software agents (applications 3410-3412) to submit work through the shared work queue 3401 using the ENQCMD / S instruction described herein.

[0707] The following considerations apply to endpoint devices that implement a shared work queue (SWQ).

[0708] SWQs and their enumeration: A device physical function (PF) may support one or more SWQs. Each SWQ is 64 bytes aligned within the device MMIO address range and is enqueuable and non-posted write accessible through a size register (referred to here as SWQ_REG). It is recommended that each such SWQ_REG on the device reside in a unique system page size (4KB) area. The device driver for the device is responsible for reporting / enumerating the SWQ functionality, the number of supported SWQs, and the corresponding SWQ_REG addresses to the software through an appropriate software interface. The driver may selectively report the depth of supported SWQs for software tuning or informational purposes (although this is not essential for the accuracy of the functionality). For devices supporting multiple physical functions, it is recommended that each physical function independently support an SWQ.

[0709] SWQ support on a single root I / O virtualization (SR-IOV) device: A device supporting SR-IOV may independently support SWQs for each virtual function (VF) exposed through the SWQ_REG in its respective VF base address register (BAR). The point of this design is to consider the maximum performance isolation for work submissions across VFs and may be suitable for a small to medium number of VFs. For devices supporting a large number of VFs (where independent SWQs for each VF are impractical), a single SWQ may be shared across multiple VFs. Even in this case, each VF has its own private SWQ_REG in its VF BAR, except that it is supplemented by a common SWQ across the VFs that share the SWQ. For such device designs, the VFs that share the SWQ may be statically determined by the hardware design, or the mapping between the SWQ_REGs of a given VF to the SWQ instance may be dynamically set up / toned down through the physical function and its driver. Device designs that share an SWQ across VFs require special attention to QoS and protection against denial-of-service attacks, as will be discussed later in this section. When sharing an SWQ across VFs, care must be taken in the device design to identify which VF received an accepted enqueue request to the SWQ. When dispatching work requests from the SWQ, the device should ensure that the upstream request is properly tagged with the requester ID (bus / device / function#) of each VF (in addition to the PASID transmitted in the enqueue request payload).

[0710] Enqueue Non-Posted Write Addresses: Endpoint devices supporting SWQ are required to accept enqueue non-posted writes to any address forwarded through these PF or VF memory BARs. For any enqueue non-posted write request received by an endpoint device to an address that is not an SWQ_REG address, the device may be required to return completion with a retry completion status (MRS) instead of treating it as an error (e.g., an ill-formed TLP). This may be done to ensure that the use of ENQCMD / S instructions by unprivileged (ring 3 or ring 0 VMX guest) software to mistakenly or maliciously issue enqueue storage to a non-SWQ_REG address on an SWQ-enabled device cannot result in reporting a non-fatal or fatal error with platform-specific error handling results.

[0711] Handling of non-enqueue requests to SWQ_REG: Endpoint devices supporting SWQ may drop non-enqueue requests (normal memory writes and reads) to SWQ_REG addresses without authorization, without treating them as fatal or non-fatal errors. Read requests to SWQ_REG addresses may return a successful completion response (as opposed to a UR or CA) with all 1s for the requested data bytes. Normal memory (posted) write requests to SWQ_REG addresses are simply dropped without any action by the endpoint device. This may be done to ensure that unprivileged software cannot generate normal read and write requests to SWQ_REG addresses in a way that causes them to mistakenly or maliciously report non-fatal or fatal errors along with platform-specific error handling results.

[0712] SWQ Queue Depth and Storage: SWQ queue depth and storage are specific to the device implementation. Device design should ensure that sufficient queue depth is supported for the SWQ to achieve maximum device utilization. Storage for the SWQ may be implemented on the device. Integrated devices on an SoC may utilize stolen main memory (non-OS visible private memory reserved for device use) as SWQ buffer overflow, enabling a larger SWQ queue depth than would be achieved using on-device storage. For such designs, the use of buffer overflow is transparent to software, as the device hardware decides when to overflow (as opposed to dropping enqueue requests and sending a retry complete status), fetch from buffer overflow for command execution, and maintain ordering requests specific to any given command. For all applications, the use of such buffer overflow is equivalent to a separate device using DRAM with a locally attached device for SWQ storage. Device designs involving buffer overflows in stolen memory require careful consideration to ensure that such stolen memory is protected by the assigned device from any access other than reading and writing the buffer overflow.

[0713] Non-blocking SWQ behavior: For performance reasons, device implementations should respond promptly to enqueue-non-posted write requests with a success or retry completion status and should not block enqueue completion of SWQ capacity to be released to accept requests. Decisions to accept or reject enqueue requests to an SWQ may be based on capacity, QoS / occupancy, or other policies. Several exemplary QoS considerations are described below.

[0714] SWQ QoS Considerations: For enqueue non-posted writes targeting SWQ_REG addresses, endpoint devices may apply authorization controls to decide whether to accept (and send a success completion status) or drop (and send a retry completion status) each request to the SWQ. Authorization controls may be device and usage-specific, and specific policies supported / enforced by hardware may be exposed to software through the physical function (PF) driver interface. Since the SWQ is a shared resource with multiple producer clients, device implementations must ensure adequate protection against denial-of-service attacks across producers. QoS for the SWQ refers only to the acceptance of work requests to the SWQ (through enqueue requests) and is orthogonal to any QoS applied by device hardware regarding how QoS is applied to share the device's execution resources when processing work requests submitted by different producers. Several exemplary approaches to configuring endpoint devices to enforce authorization policies for accepting enqueue requests to the SWQ are described below. These are recorded for illustrative purposes only, and the selection of the exact embodiment is device-specific.

[0715] In one embodiment, the MOVDIRI instruction uses direct storage to move a double-word integer in the source operand (second operand) to the destination operand (first operand). The source operand may be a general-purpose register. The destination operand may be a 32-bit memory location. In 64-bit mode, the default processing size of the instruction is 32 bits. MOVDIRI defines the destination so that it is double-word or quad-word aligned.

[0716] Direct storage can be implemented by using the write-combining (WC) memory type protocol for writing data. Using this protocol, the processor neither writes data to the cache hierarchy nor fetches the corresponding cache line from memory into the cache hierarchy. If the destination address is cached, the line is written back (if modified) and invalidated from the cache before direct storage. Unlike storage with non-temporary suggestions, which allow uncached (UC) and write-protected (WP) memory types to override non-temporary suggestions, direct storage always adheres to the WC memory type protocol, regardless of the destination address memory type (including UC and WP types).

[0717] Unlike WC storage and storage using non-temporary suggestion, direct storage is subject to immediate eviction from the write combining buffer and, consequently, cannot be combined with younger storage (including direct storage) to the same address. Older WC and non-temporary storage held in the write combine buffer may be combined with younger direct storage to the same address.

[0718] The WC protocol used for direct storage follows a weakly ordered memory consistency model, so fencing should follow the MOVDIRI instruction to enforce ordering when necessary.

[0719] Direct storage issued by MOVDIRI to a destination is aligned to a 4-byte boundary, guaranteeing atomicity of 4-byte write completion. This means that data reaches the destination in a single non-tone 4-byte (or 8-byte) write transaction. If the destination does not match the write size, direct storage issued by MOVDIRI reaches the destination in two parts. Each part of such split direct storage does not merge with the younger storage, but can reach the destination in any order.

[0720] Figure 59 illustrates an embodiment of the method executed by the processor to process the MOVDIRI instruction. For example, hardware details are used here.

[0721] At 5901, an instruction is fetched. For example, MOVDIRI is fetched. The MOVDIRI instruction includes an opcode (and, in some embodiments, a prefix), a destination field representing the destination operand, and a source field representing the source register operand.

[0722] In 5903, the fetched instruction is decoded. F...

Claims

1. A first set of multiple cores having a first microarchitecture, A second set of multiple cores having a second microarchitecture different from the first microarchitecture, The interconnections coupled to the first and second multiple cores, An accelerator that performs matrix operations is coupled to the aforementioned interconnection, Equipped with, The aforementioned accelerator is An array of multiply-accumulate units that can operate to perform a multiply-accumulate operation on a first set of data elements of a first matrix and a second set of data elements of a second matrix in response to a multiply-accumulate instruction, A plurality of memories associated with the array of the sum-of-products units, which store the first plurality of data elements and the second plurality of data elements, It has, Each sum-of-products unit in the array of sum-of-products units is: A multiplication circuit that multiplies each data element of the first subset of data elements with the corresponding data element of the second subset of data elements to generate a corresponding set of products, An addition circuit that adds up the aforementioned multiple products to generate corresponding result data elements for multiple result data elements, It has, An integrated circuit in which at least one of the first or second plurality of cores is configured to execute program code for scheduling the matrix processing operations.

2. In the array of the sum-of-products units, the first sum-of-products unit generates a first plurality of result data elements using a first plurality of sum-of-products data elements. The first plurality of result data elements become a second plurality of accumulation data elements for a second sum-of-products unit in the array of sum-of-products units. The integrated circuit according to claim 1, wherein the second sum-of-products unit generates a second plurality of result data elements using the second plurality of sum-of-products data elements.

3. The integrated circuit according to claim 1 or 2, further comprising at least one scalar core that performs scalar operations, coupled to the interconnection.

4. The first plurality of data elements each have data elements of a first block of a first block size, The integrated circuit according to any one of claims 1 to 3, wherein the second plurality of data elements have data elements of a second block having the size of the first block.

5. Each of the first plurality of cores has a single instruction multiple data (SIMD) core, The integrated circuit according to any one of claims 1 to 4, wherein each of the second plurality of cores has an out-of-order execution core.

6. The integrated circuit according to any one of claims 1 to 5, wherein the plurality of memories have a plurality of different types of on-chip memories, each including one or more cache memories.

7. The integrated circuit according to claim 6, further comprising a coherence circuit for maintaining the coherence of data stored in at least some of the plurality of different types of on-chip memories.

8. The subset of the on-chip memory is associated with the accelerator, The integrated circuit according to claim 6 or 7, wherein each on-chip memory in the subset of on-chip memories is associated with one of the sum-of-products units.

9. The integrated circuit according to claim 8, further comprising a controller for loading the first plurality of data elements and the second plurality of data elements from an external memory into one or more on-chip memories of a subset of the on-chip memory.

10. The integrated circuit according to claim 9, wherein the controller stores the plurality of result data elements from one or more on-chip memories among a subset of the on-chip memory in the external memory.

11. The integrated circuit according to any one of claims 1 to 10, further comprising an FPGA-based accelerator coupled to the aforementioned interconnection.

Images