Systems, methods, and apparatus for heterogeneous computing

Abstract

The disclosure relates to systems, methods, and apparatus for heterogeneous computing. In some embodiments, a hardware heterogeneous scheduler dispatches instructions for execution on one or more of a plurality of heterogeneous processing elements, the instructions corresponding to code fragments to be processed by the one or more of the plurality of heterogeneous processing elements, the instruction is a native instruction for at least one of one or more of the plurality of heterogeneous processing elements.

Description

Technical Field

[0001] This disclosure generally pertains to the field of computing devices, and more specifically to heterogeneous computing methods, devices, and systems. Background Technology

[0002] In today's computers, the CPU performs general-purpose computing tasks, such as running application software and operating systems. Specialized computing tasks, such as graphics and image processing, are handled by graphics processors, image processors, digital signal processors, and fixed-function accelerators. In today's heterogeneous machines, each type of processor is programmed differently.

[0003] The era of big data processing demands higher performance with lower energy consumption compared to today's general-purpose processors. Accelerators (e.g., custom fixed-function units or custom programmable units) help meet these needs. Because the field is rapidly evolving in terms of algorithms and workloads, the available accelerator set is difficult to predict a priori and is highly likely to vary between existing units in product generation and evolve with product generations. Attached Figure Description

[0004] The embodiments will be readily understood from the following detailed description taken in conjunction with the accompanying drawings. For ease of explanation, similar reference numerals denote similar structural elements. In the figures, embodiments are shown exemplary and not limitingly.

[0005] Figure 1 It represents a heterogeneous multiprocessing execution environment;

[0006] Figure 2 It represents a heterogeneous multiprocessing execution environment;

[0007] Figure 3 An exemplary implementation of a heterogeneous scheduler is shown;

[0008] Figure 4 An embodiment of system booting and device discovery for a computer system is illustrated;

[0009] Figure 5 An example of thread migration based on the mapping from program phases to three types of processing elements is shown;

[0010] Figure 6 This is an exemplary implementation flow executed by a heterogeneous scheduler;

[0011] Figure 7 An example of a method for thread destination selection in a heterogeneous scheduler is shown;

[0012] Figure 8 This illustrates the concept of using stripe mapping for logical IDs;

[0013] Figure 9 An example of using stripe mapping for logical IDs is shown;

[0014] Figure 10 An example of a core group is shown;

[0015] Figure 11 An example of a thread execution method in a system that utilizes a binary converter switching mechanism is shown;

[0016] Figure 12 An exemplary method for hot code allocation to an accelerator is shown;

[0017] Figure 13 An exemplary method for waking up or writing to a page directory base address register event is shown;

[0018] Figure 14 An example of a serial phase thread is shown;

[0019] Figure 15 An exemplary method for potential core allocation in response to a sleep command event is shown;

[0020] Figure 16 An exemplary method for potential core allocation to a thread in response to a phase change event is shown;

[0021] Figure 17 An example of code depicting the acceleration region is shown;

[0022] Figure 18 An embodiment of the execution method using ABEGIN in a hardware processor core is shown;

[0023] Figure 19 An embodiment of the execution method using AEND in a hardware processor core is shown;

[0024] Figure 20 A system is shown that uses pattern matching to provide ABEGIN / AEND equivalences;

[0025] Figure 21 An embodiment of an execution method for a non-accelerated drawing thread exposed to pattern recognition is shown;

[0026] Figure 22 An embodiment of an execution method for a non-accelerated drawing thread exposed to pattern recognition is shown;

[0027] Figure 23 It illustrates different types of memory dependencies, their semantics, ordering requirements, and use cases;

[0028] Figure 24 An example of a memory data block pointed to by the ABEGIN instruction is shown;

[0029] Figure 25 An example of a memory 2503 configured to use ABEGIN / AEND semantics is shown;

[0030] Figure 26 Examples of methods using ABEGIN / AEND to operate in different execution modes are shown;

[0031] Figure 27 Examples of methods using ABEGIN / AEND to operate in different execution modes are shown;

[0032] Figure 28 Additional details of one implementation are shown;

[0033] Figure 29 An embodiment of the accelerator is shown;

[0034] Figure 30 A computer system is shown, which includes an accelerator and one or more computer processor chips coupled to a processor via a multi-protocol link;

[0035] Figure 31 The device bias process according to an embodiment is illustrated;

[0036] Figure 32 An exemplary process according to one implementation is shown;

[0037] Figure 33 This illustrates the process of releasing operands from one or more I / O devices;

[0038] Figure 34 Implementations using two different types of work queues are shown;

[0039] Figure 35 An implementation of a data stream accelerator (DSA) device is shown, which includes multiple work queues that receive descriptors submitted via an I / O structure interface;

[0040] Figure 36 Two work queues are shown;

[0041] Figure 37 Another configuration using engines and groups is shown;

[0042] Figure 38 An implementation of the descriptor is shown;

[0043] Figure 39 An implementation method for completing the record is shown;

[0044] Figure 40An exemplary no-operation descriptor and no-operation completion record are shown;

[0045] Figure 41 An exemplary batch descriptor and no-operation-complete record are shown;

[0046] Figure 42 An exemplary discharge descriptor and discharge completion record are shown;

[0047] Figure 43 An exemplary memory move descriptor and memory move completion record are shown;

[0048] Figure 44 An example fill descriptor is shown;

[0049] Figure 45 An exemplary comparison descriptor and comparison completion record are shown;

[0050] Figure 46 An exemplary comparison immediate descriptor is shown;

[0051] Figure 47 An example of creating a data record descriptor and creating an incremental record completion record is shown;

[0052] Figure 48 The format of the incremental record is shown;

[0053] Figure 49 An example application incremental record descriptor is shown;

[0054] Figure 50 One implementation is shown using the operations of creating and applying incremental records;

[0055] Figure 51 An exemplary memory copy with a dual translation descriptor and a memory copy with a dual translation completion record are shown;

[0056] Figure 52 An example CRC generation descriptor and CRC generation completion record are shown;

[0057] Figure 53 An exemplary copy with a CRC generation descriptor is shown;

[0058] Figure 54 An exemplary DIF insert descriptor and DIF insert completion record are shown;

[0059] Figure 55 An exemplary DIF stripe descriptor and DIF stripe completion record are shown;

[0060] Figure 56 An example DIF update descriptor and DIF update completion record are shown;

[0061] Figure 57 An example cache refresh descriptor is shown;

[0062] Figure 58 This shows 64 bytes of queued storage data generated by ENQCMD;

[0063] Figure 59 An embodiment of a method executed by a processor for processing MOVDIRI instructions is shown;

[0064] Figure 60 An embodiment of a processor-executed method for processing MOVDIRI64B instructions is shown;

[0065] Figure 61 An embodiment of a processor-executed method for processing ENCQMD instructions is shown;

[0066] Figure 62 The format of the ENQCMDS command is shown;

[0067] Figure 63 An embodiment of a method executed by a processor for processing ENCQMDs instructions is shown;

[0068] Figure 64 An embodiment of a processor-executed method for processing UMONITOR instructions is shown;

[0069] Figure 65 An embodiment of a processor-executed method for processing UMWAIT instructions is shown;

[0070] Figure 66 An embodiment of a method executed by a processor for processing TPAUSE instructions is shown;

[0071] Figure 67 An example of execution using the UMWAIT and UMONITOR instructions is shown;

[0072] Figure 68 An example of execution using the TPAUSE and UMONITOR instructions is shown;

[0073] Figure 69 An exemplary implementation is shown, in which the accelerator is communicatively coupled to multiple cores via a cache coherence interface;

[0074] Figure 70 Another view of the accelerator and other previously described components is shown, including a data management unit, multiple processing elements, and fast on-chip memory;

[0075] Figure 71An exemplary set of operations performed by the processing element is shown;

[0076] Figure 72A shows an example of a multiplication between a sparse matrix A and a vector x to produce a vector y;

[0077] Figure 72B shows the CSR representation of matrix A, where each value is stored as a (value, row index) pair;

[0078] Figure 72C shows the CSC representation of matrix A using (value, column index) pairs;

[0079] Figures 73A, 73B, and 73C show the pseudocode for each computational mode;

[0080] Figure 74 The processing flow of one embodiment of the data management unit and processing element is shown;

[0081] Figure 75a Highlight the paths of the spMspV_csc and scale_update operations (using dashed lines);

[0082] Figure 75b The path of the spMdV_csr operation is shown;

[0083] Figure 76a -b shows an example of representing a graph as an adjacency matrix;

[0084] Figure 76c The vertex procedure is shown;

[0085] Figure 76d Exemplary program code for executing the vertex program is shown;

[0086] Figure 76e The GSPMV formula is shown;

[0087] Figure 77 The framework is shown;

[0088] Figure 78 This shows the customizable logic blocks provided within each PE;

[0089] Figure 79 The operation of each accelerator block is shown;

[0090] Figure 80a This document summarizes the customizable parameters for one implementation of the template.

[0091] Figure 80b The adjustment considerations are shown;

[0092] Figure 81 This illustrates one of the most common sparse matrix formats;

[0093] Figure 82 The steps involved in implementing sparse matrix-dense vector multiplication using the CRS data format are shown.

[0094] Figure 83 An implementation of an accelerator is shown, which includes an accelerator logic die and one or more DRAM stacks;

[0095] Figures 84A-B illustrate one implementation of an accelerator logic chip, oriented through a DRAM die stack from a top perspective view.

[0096] Figure 85 Provides an advanced overview of DPE;

[0097] Figure 86 An implementation of the block-based scheme is shown;

[0098] Figure 87 The block descriptor is shown;

[0099] Figure 88 The diagram shows a two-row matrix within a buffer suitable for a single dot product engine;

[0100] Figure 89 One implementation of the hardware in a dot product engine using this format is shown;

[0101] Figure 90 The contents of the matching logic unit that performs the capture are shown;

[0102] Figure 91 Details of a dot product engine design supporting sparse matrix-sparse vector multiplication according to one embodiment are shown.

[0103] Figure 92 An example using a specific value is shown;

[0104] Figure 93 This demonstrates how to combine sparse-dense and sparse-sparse dot product engines to produce a dot product engine that can handle both types of computation.

[0105] Figure 94a An embodiment with a socket replacement featuring 12 accelerator stacks is shown;

[0106] Figure 94b An implementation with a processor / core set and eight stacked multi-chip package (MCP) is shown;

[0107] Figure 95 The accelerator stack is shown;

[0108] Figure 96 A potential layout for an accelerator located under a WIO3 DRAM stack is shown, including 64 dot product engines, 8 vector caches, and an integrated memory controller;

[0109] Figure 97 Seven DRAM technologies were compared;

[0110] Figure 98a -b indicates stacked DRAM;

[0111] Figure 99 A list of breadth-first search (BFS) results is shown;

[0112] Figure 100 The format of a descriptor for specifying a Lambda function is shown according to one embodiment;

[0113] Figure 101 The lower six bytes of the header word in the embodiment are shown;

[0114] Figure 102 The matrix value buffer, matrix index buffer, and vector value buffer are shown.

[0115] Figure 103 Details of one implementation of the Lambda data path are shown;

[0116] Figure 104 An implementation method for instruction encoding is shown;

[0117] Figure 105 It shows the encoding of a specific instruction set;

[0118] Figure 106 The encoding of an exemplary comparison predicate is shown;

[0119] Figure 107 An example using offset is shown;

[0120] Figure 108A -B shows the memory-mapped I / O (MMIO) space registers used with work queue-based implementations;

[0121] Figure 109 An example of matrix multiplication is shown;

[0122] Figure 110 The octoMADD instruction operation with a binary tree reduction network is shown;

[0123] Figure 111 An embodiment of a method executed by a processor for processing multiplication and addition instructions is shown;

[0124] Figure 112 An embodiment of a method executed by a processor for processing multiplication and addition instructions is shown;

[0125] Figure 113A-C illustrates exemplary hardware for executing MADD instructions;

[0126] Figure 114 An example of a hardware heterogeneous scheduler circuit and its interaction with memory is shown;

[0127] Figure 115 An example of a software heterogeneous scheduler is shown;

[0128] Figure 116 An embodiment of a method for system post-boot device discovery is illustrated;

[0129] Figure 117 (A)-(B) show examples of thread movement in shared memory;

[0130] Figure 118 An exemplary method for thread movement that can be executed by a heterogeneous scheduler is shown;

[0131] Figure 119 This is a block diagram of a processor configured to present the abstract execution environment detailed above;

[0132] Figure 120 This is a simplified block diagram illustrating an exemplary multi-chip configuration;

[0133] Figure 121 A block diagram illustrating at least a portion of a system including an exemplary implementation of a multi-chip link (MCL) is shown;

[0134] Figure 122 A block diagram of an exemplary logic PHY for an exemplary MCL is shown;

[0135] Figure 123 A simplified block diagram is shown, illustrating another representation of the logic used to implement MCL;

[0136] Figure 124 An example of execution is shown when ABEGIN / AEND is not supported;

[0137] Figure 125 This is a block diagram of a register architecture according to an embodiment of the present invention;

[0138] Figure 126A This is a block diagram illustrating an exemplary ordered pipeline and an exemplary register renaming, out-of-order release / execution pipeline according to embodiments of the present invention;

[0139] Figure 126B This is a block diagram illustrating an exemplary embodiment of an ordered architecture core to be included in a processor and an exemplary register renaming, out-of-order release / execution architecture core according to an embodiment of the present invention;

[0140] Figure 127A -B shows a block diagram of a more specific exemplary ordered core architecture, which will be one of several logical blocks in the chip (including other cores of the same type and / or different types);

[0141] Figure 128 This is a block diagram of a processor according to an embodiment of the present invention, which may have more than one core, may have an integrated memory controller, and may have an integrated graphics.

[0142] Figure 129 A block diagram of a system according to an embodiment of the present invention is shown;

[0143] Figure 130 This is a block diagram of a first more specific exemplary system according to an embodiment of the present invention;

[0144] Figure 131 This is a block diagram of a second, more specific, exemplary system according to an embodiment of the present invention;

[0145] Figure 132 This is a block diagram of a SoC according to an embodiment of the present invention; and

[0146] Figure 133 This is a block diagram illustrating the comparative use of a software instruction converter to convert binary instructions in a source instruction set into binary instructions in a target instruction set, according to an embodiment of the present invention. Detailed Implementation

[0147] The following detailed description is made with reference to the accompanying drawings, which form part of the description, wherein similar reference numerals indicate similar parts throughout, and exemplary embodiments that can be practiced are shown in the drawings. It should be understood that other embodiments can be utilized, and structural or logical changes can be made, without departing from the scope of this disclosure. Therefore, the following detailed description should not be considered limiting, and the scope of the embodiments is defined by the appended claims and their equivalents.

[0148] Multiple operations are described as a series of separate actions or operations in a manner most conducive to understanding the claimed subject matter. However, the order in which they are described should not be construed as implying that these operations must be sequentially related. Specifically, these operations may not be performed in the presented order. The operations may be performed in an order different from that described in the embodiment. In other embodiments, multiple additional operations may be performed and / or the operations may be omitted.

[0149] For the purposes of this disclosure, the phrase "A and / or B" means (A), (B), or (A and B). For the purposes of this disclosure, the phrase "A, B, and / or C" means (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C).

[0150] This specification may use the phrases "in one embodiment" or "in an embodiment," which may refer to one or more of the same or different embodiments, respectively. Furthermore, the terms "comprising," "including," "having," etc., as used with respect to embodiments of this disclosure are synonyms.

[0151] As discussed in the background section, deploying accelerator solutions and managing the complexity of portable accelerator utilization can be challenging due to the existence of various stock units and platforms implementing different hybrid accelerators. Furthermore, deploying accelerators via a device driver model has limitations, including barriers to adoption due to developer effort, non-portability, and the stringent performance requirements of big data processing, given the diversity of operating systems (and versions, patches, etc.). Accelerators are typically hardware devices (circuits) that perform functions more efficiently than software running on a general-purpose processor. For example, hardware accelerators can be used to improve the execution of specific algorithms / tasks (e.g., video encoding or decoding, specific hash functions, etc.) or classes of algorithms / tasks (e.g., machine learning, sparse data manipulation, cryptography, graphics, physics, regular expressions, packet processing, artificial intelligence, digital signal processing, etc.). Examples of accelerators include, but are not limited to, graphics processing units (“GPUs”), fixed-function field-programmable gate arrays (“FPGAs”) accelerators, and fixed-function application-specific integrated circuits (“ASICs”). Note that in some implementations, an accelerator can be a general-purpose central processing unit (“CPU”) if that CPU is more efficient than other processors in the system.

[0152] The power budget for a given system (e.g., a system-on-a-chip (“SOC”), processor inventory unit, rack, etc.) may be consumed by processing elements located on only a small fraction of the available silicon area. This makes it advantageous to build various dedicated hardware blocks that reduce the energy consumption of specific operations, even if not all hardware blocks can operate simultaneously.

[0153] Detailed embodiments of systems, methods, and apparatuses for selecting processing elements (e.g., cores or accelerators) to process threads, interface with processing elements, and / or manage power consumption within a heterogeneous multiprocessor environment are described. For example, in various embodiments, the heterogeneous multiprocessor is configured (e.g., by design or by software) to dynamically migrate threads between different types of processing elements in the heterogeneous multiprocessor based on the characteristics of the corresponding workloads of the threads and / or processing elements; to provide programming interfaces for one or more processing elements; to translate code for execution on a specific processing element; to select a communication protocol to be used with the selected processing element based on the workload and characteristics of the selected processing element; or to implement a combination of these operations.

[0154] In the first aspect, the workload distribution interface, or heterogeneous scheduler, presents a homogeneous multiprocessor programming model to system programmers. Specifically, this aspect enables programmers to develop software for specific architectures or equivalent abstractions, while facilitating continuous improvements to the underlying hardware without requiring corresponding changes to the developed software.

[0155] Secondly, multi-protocol links allow a first entity (such as a heterogeneous scheduler) to communicate with multiple devices using protocols associated with the communication. This replaces the need for separate links for device communication. In particular, the link has three or more protocols dynamically multiplexed on it. For example, a common link supports the following protocols: 1) a producer / consumer, discovery, configuration, and interrupt (PDCI) protocol to enable device discovery, device configuration, error reporting, interruption, DMA-style data transfer, and various services, such as those specified in one or more proprietary or industry standards (e.g., the PCI Express specification or equivalent alternatives); 2) a cache agent coherence (CAC) protocol to enable devices to issue consistent read and write requests to processing elements; and 3) a memory access (MA) protocol to enable processing elements to access the local memory of another processing element.

[0156] Thirdly, the scheduling, migration, or emulation of threads or portions thereof is done based on the thread's phase. For example, the data-parallel phase of a thread is typically scheduled or migrated to a SIMD core; the thread-parallel phase is typically scheduled or migrated to one or more scalar cores; and the serial phase is typically scheduled or migrated to out-of-order cores. Energy or latency is minimized for each core type, both of which are considered for thread scheduling, migration, or emulation. Emulation can be used if scheduling or migration is impossible or unfavorable.

[0157] Fourthly, threads or portions thereof are opportunistically offloaded to accelerators. Specifically, the instructions between the thread's accelerator start (ABEGIN) and accelerator end (AEND) instructions or portions thereof are bookends that can be executed on the accelerator. If the accelerator is unavailable, the instructions between ABEGIN and AEND will execute as usual. However, when the accelerator is available and it is desired to use it (e.g., to use less power), the instructions between ABEGIN and AEND are translated to be executed on that accelerator and scheduled for execution on that accelerator. Therefore, the use of accelerators is opportunistic.

[0158] In the fifth aspect, the analysis involves offloading threads or portions thereof to an accelerator (opportunistically) without using ABEGIN or AEND. For code executable on the accelerator, software or hardware pattern matching is performed on the thread or portion thereof. If the accelerator is unavailable, or if the thread or portion thereof is unsuitable for accelerator execution, the thread's instructions are executed as usual. However, when an accelerator is available and it is desired to use it (e.g., to use less power), the instructions are translated to be executed on that accelerator and scheduled for execution on that accelerator. Therefore, the use of an accelerator is opportunistic.

[0159] In the sixth aspect, code snippets (part of a thread) are transformed to better suit the chosen destination processing element. For example, a code snippet may be: 1) transformed to utilize a different instruction set, 2) made more parallel, 3) made less parallel (serialized), 4) made data parallel (e.g., vectorized), and / or 5) made less data parallel (e.g., unvectorized).

[0160] In the seventh aspect, a work queue (shared or dedicated) receives descriptors that define the scope of work to be performed by the device. A dedicated work queue stores descriptors for a single application, while a shared work queue stores descriptors submitted by multiple applications. The hardware interface / arbitrator dispatches descriptors from the work queue to the accelerator processing engine based on a specified arbitration policy (e.g., based on the processing requirements of each application and QoS / fairness policies).

[0161] In the eighth aspect, improvements to dense matrix multiplication allow for two-dimensional matrix multiplication using the execution of a single instruction. Multiple packed data (SIMD, vector) sources are multiplied by a single packed data source. In some cases, binary trees are used for multiplication.

[0162] Figure 1This is a representation of a heterogeneous multiprocessor execution environment. In this example, the heterogeneous scheduler 101 receives code fragments of a first type (e.g., one or more instructions associated with a software thread). The code fragments can be in any form of source code representation, including, for example, machine code, intermediate representation, bytecode, text-based code (e.g., assembly code, source code of a high-level language such as C++), etc. The heterogeneous scheduler 101 presents a homogeneous multiprocessor programming model (e.g., making all threads appear to the user and / or operating system as executing on a scalar core) and determines the workload type (program phase) of the received code fragment, selects a type of processing element (scalar, out-of-order (OOO), single instruction multiple data (SIMD), or accelerator) corresponding to the determined workload type to handle the workload (e.g., scalar for thread-parallel code, OOO for serial code, SIMD for data-parallel code, and accelerator for data-parallel code), and schedules the code fragments to be processed by the appropriate processing element. Figure 1 In the specific embodiments shown, the processing element types include a scalar core 103 (such as an ordered core), a single instruction multiple data (SIMD) core 105 that operates on packed data operands, wherein registers have multiple data elements stored contiguously, a low-latency, out-of-order core 107, and an accelerator 109. In some embodiments, the scalar core 103, the SIMD core 105, the low-latency, out-of-order core 107 are in a heterogeneous processor, and the accelerator 109 is external to the heterogeneous processor. However, it should be noted that various different processing element arrangements can be used. In some embodiments, the heterogeneous scheduler 101 converts or interprets received code segments or portions thereof into a format corresponding to the selected type of processing element.

[0163] Processing elements 103-109 can support different instruction set architectures (ISAs). For example, an out-of-order core can support a first ISA, while an ordered core can support a second ISA. This second ISA can be a set (subset or superset) of the first ISA, or it can be different. Additionally, processing elements can have different microarchitectures. For example, a first out-of-order core supports a first microarchitecture, while an ordered core supports a different second microarchitecture. Note that even within a specific type of processing element, the ISA and microarchitecture can differ. For example, a first out-of-order core can support a first microarchitecture, while a second out-of-order core can support a different microarchitecture. Instructions are "inherent" to a specific ISA because they are part of the ISA. Inherent instructions execute on a specific microarchitecture without external changes (e.g., translations).

[0164] In some implementations, one or more processing elements are integrated on a single die, for example, as a System-on-a-Chip (SoC). Such implementations can benefit, for example, from improved communication latency, manufacturing / cost, reduced pin count, and platform miniaturization. In other implementations, the processing elements are packaged together, thereby achieving one or more of the benefits of the SoC mentioned above without being on a single die. These implementations can further benefit, for example, from different processing technologies optimized for each processing element type, smaller die size for increased yield, and integration of proprietary intellectual property blocks. Communicating with added different devices can be challenging in some traditional multi-package constraints. The multi-protocol links discussed herein minimize or mitigate this challenge by presenting a common interface for different types of devices to users, operating systems (“OS”), etc.

[0165] In some implementations, the heterogeneous scheduler 101 is implemented in software stored in a computer-readable medium (e.g., memory) for execution on a processor core (e.g., 000 core 107). In these implementations, the heterogeneous scheduler 101 is referred to as a software heterogeneous scheduler. This software may implement a binary converter, a just-in-time (“JIT”) compiler, and an OS 117 to schedule the execution of threads, which include code snippets, pattern matchers, module components therein, or combinations thereof.

[0166] In some implementations, the heterogeneous scheduler 101 is implemented in hardware as a circuit and / or a finite state machine executed by the circuit. In these implementations, the heterogeneous scheduler 101 is referred to as a hardware heterogeneous scheduler.

[0167] From a programming perspective (e.g., OS117, emulation layer, supervisor, security monitor, etc.), each type of processing element 103-109 utilizes a shared memory address space 115. In some embodiments, the shared memory address space 115 may optionally include two types of memory, memory 211 and memory 213, such as... Figure 2 As shown. In such implementations, memory types can be distinguished in various ways, including but not limited to: differences in memory location (e.g., located on different sockets), differences in corresponding interface standards (e.g., DDR4, DDR5, etc.), differences in power requirements, and / or differences in the underlying memory technology used (e.g., high-bandwidth memory (HBM), synchronous DRAM, etc.).

[0168] The shared memory address space 115 can be accessed by each type of processing element. However, in some embodiments, different types of memory may preferably be assigned to different processing elements, for example, based on workload requirements. For example, in some embodiments, the platform firmware interface (e.g., BIOS or UEFI) or memory storage device includes fields for indicating the type of memory resources available in the platform and / or processing element affinity for certain address ranges or memory types.

[0169] The heterogeneous scheduler 101 uses this information when analyzing threads to determine where a thread should be executed at a given point in time. Typically, thread management mechanisms look at all available information to make informed decisions about how to manage existing threads. This can manifest in several ways. For example, a thread executing on a specific processing element that has an affinity for an address range that is physically closer to the processing element may be given priority over a thread that would normally execute on that processing element.

[0170] Another example is allowing a thread that benefits from a specific memory type (e.g., a faster version of DRAM) to physically move its data to that memory type and adjust memory references in the code to point to that portion of the shared address space. For instance, while a thread on SIMD core 205 can use second memory type 213, that thread can be removed from that use when accelerator 209 is active and needs that memory type 213 (or at least the portion allocated to the thread on SIMD core 205).

[0171] An exemplary scenario is when the memory is physically closer to a processing element (compared to other processing elements). A common scenario is that the accelerator is directly connected to a different type of memory than the core.

[0172] In these examples, the OS typically initiates the data movement. However, nothing prevents a lower level (e.g., a heterogeneous scheduler) from performing this function on its own or with the help of another component (e.g., the OS). Whether to flush the data of previously processed elements and invalidate page table entries depends on the implementation and the penalty for performing the data movement. If the data is unlikely to be used immediately, it may be more feasible to simply copy it from memory rather than moving it from one memory type to another.

[0173] Figure 117(A)-(B) illustrate an example of thread movement in shared memory. In this example, two types of memory share an address space, each type having its own address range within that space. In 117(A), shared memory 11715 includes a first type of memory 11701 and a second type of memory 11707. The first type of memory 11701 has a first address range 11703, and within this range are addresses dedicated to thread 1 11705. Memory type 11707 has a second address range 11709.

[0174] At some point during the execution of thread 1 11705, the heterogeneous scheduler makes a decision to move thread 11705, causing the second thread 11711 to use the address in memory 11701 of the first type previously allocated to thread 1 11705. This is in Figure 117 (B) is shown. In this example, thread 1 11705 is reassigned to memory type 2 11707 and given a new address set to use; however, this is not always the case. Note that the difference between memory types can be physical or spatial (e.g., based on distance to PE).

[0175] Figure 118 An exemplary method for thread movement, which can be executed by a heterogeneous scheduler, is shown. At 11801, a first thread is guided to execute on a first processing element (“PE”), such as a core or accelerator, using a first type of memory in the shared memory space. For example, in Figure 117 (A) This is thread 1.

[0176] At a later time, a request to execute a second thread is received at 11803. For example, an application, the OS, etc., requests to execute a hardware thread.

[0177] At address 11805, it is determined that the second thread should execute on the second PE using memory of type 1 in the shared address space. For example, the second thread will run on an accelerator directly coupled to memory of type 1, and this execution (including releasing memory used by the first thread) is more efficient than having the second thread use memory of type 2.

[0178] In some embodiments, at 11807, the data of the first thread is moved from a first type of memory to a second type of memory. This is not necessarily the case if it would be more efficient to simply stop the execution of the first thread and start another thread at its location.

[0179] At 11809, the Translation Backing Buffer (TLB) entry associated with the first thread is invalidated. Additionally, in most embodiments, a data refresh is performed.

[0180] At 11811, the second thread is directed to the second PE and allocated an address range in the first type of memory previously allocated to the first thread.

[0181] Figure 3 An exemplary implementation of a heterogeneous scheduler 301 is shown. In some instances, scheduler 301 is part of a runtime system. As shown, program phase detector 313 receives code fragments and identifies one or more features of the code fragments to determine whether the corresponding program execution phase is best characterized as serial, data-parallel, or thread-parallel. Examples of how this is determined are detailed below. (See also: Regarding...) Figure 1 In detail, code snippets can be any number of source code representations.

[0182] For recurring code snippets, pattern matcher 311 identifies the "hot" code and, in some instances, also identifies characteristics that indicate the workload associated with the code snippet might be better suited for processing on different processing elements. For example, below... Figure 20 Further details relating to the pattern matcher 311 and its operation are described in the context of this paper.

[0183] Selector 309 selects a target processing element to execute a native representation of the received code segment, based at least in part on the characteristics of the processing element and thermal and / or power information provided by power manager 307. Selecting a target processing element can be simplified to selecting the most suitable target processing element for the code segment (i.e., a match between workload characteristics and processing element capabilities), but may also take into account the current power consumption level of the system (e.g., which may be provided by power manager 307), the availability of processing elements, the amount of data moved from one type of memory to another (and the associated penalties for doing so), etc. In some embodiments, selector 309 is a finite state machine implemented in or executed by hardware circuitry.

[0184] In some embodiments, selector 309 also selects a suitable link protocol for communicating with the target processing element. For example, in some embodiments, the processing element utilizes a suitable common link interface capable of dynamically multiplexing or encapsulating multiple protocols on a system architecture or point-to-point interconnect. For example, in some embodiments, supported protocols include: 1) a Producer / Consumer, Discovery, Configuration, Interruption (PDCI) protocol for device discovery, device configuration, error reporting, interruption, DMA-style data transfer, and various services, such as those specified in one or more proprietary or industry standards (e.g., the PCI Express specification or equivalent alternatives); 2) a Cache Agent Coherence (CAC) protocol that enables devices to issue consistent read and write requests to the processing element; and 3) a Memory Access (MA) protocol that enables a processing element to access the local memory of another processing element. Selector 309 selects among these protocols based on the type of request to be delivered to the processing element. For example, producer / consumer, discovery, configuration, or interrupt requests use the PDCI protocol, cache coherence requests use the CAC protocol, and local memory access requests use the MA protocol.

[0185] In some implementations, the thread includes a flag indicating the stage type, thus eliminating the need for a stage detector. In some implementations, the thread includes prompts or explicit requests for the processing element type, link protocol, and / or memory type. In these implementations, selector 309 utilizes this information during its selection process. For example, the selection by selector 309 can be overridden by the thread and / or the user.

[0186] According to an implementation, the heterogeneous scheduler may include one or more converters to process received code segments and generate corresponding native encoding for the target processing element. For example, the heterogeneous scheduler may include converters for converting machine code of a first type to machine code of a second type; and / or a just-in-time compiler for converting intermediate representations to the native format of the target processing element. Alternatively or additionally, the heterogeneous scheduler may include a pattern matcher for identifying recurring code segments (i.e., “hot” code) and caching one or more native encodings of the code segments or corresponding micro-operations. Each of these optional components in Figure 3As shown in the diagram. Specifically, the heterogeneous scheduler 301 includes a converter 303 and a just-in-time (JIT) compiler 305. When the heterogeneous scheduler 301 operates on target code or intermediate representations, the JIT compiler 305 is invoked to convert the received code fragments into the native format of one or more target processing elements 103, 105, 107, 109. When the heterogeneous scheduler 301 operates on machine code (binary), the binary converter 303 converts the received code fragments into the native machine code of one or more target processing elements (e.g., when converting from one instruction set to another). In alternative embodiments, one or more of these components may be omitted from the heterogeneous scheduler 301.

[0187] For example, in some embodiments, a binary converter is not included. This can lead to increased programming complexity because the program needs to consider potentially available accelerators, cores, etc., rather than letting the scheduler handle the problem. For example, the program might need to include code for routines used in different formats. However, in some embodiments, when a binary converter is absent, a JIT compiler exists that accepts higher-level code and performs the necessary transformations. When a pattern matcher is present, hot code can still be detected to find the code that should run on a specific processing element.

[0188] For example, in some embodiments, a JIT compiler is not included. This can also lead to increased programming complexity because the program needs to be compiled into machine code for a specific ISA first, rather than having the scheduler handle that. However, in some embodiments, when a binary converter is present and there is no JIT compiler, the scheduler can perform conversions between ISAs, as described below. When a pattern matcher is present, hot code can still be detected to find the code that should run on a specific processing element.

[0189] For example, in some embodiments, a pattern matcher is not included. This can also lead to inefficiency, as code that may have been moved is more likely to remain on a less efficient core used for the specific task being run.

[0190] In some embodiments, there is no binary converter, JIT compiler, or pattern matcher. In these embodiments, only phase detection or explicit requests for moving threads are utilized in thread / processing element allocation / migration.

[0191] Refer again Figure 1-3 The heterogeneous scheduler 101 can be implemented in hardware (e.g., circuitry), software (e.g., executable program code), or any combination thereof. Figure 114An example of a hardware heterogeneous scheduler circuit and its interaction with memory is shown. Heterogeneous schedulers can be manufactured in many different ways, including but not limited to: as a state machine based on a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC); as an embedded microcontroller coupled to memory (with software stored in the memory to provide the functions detailed herein), logic circuitry including other sub-components (e.g., data hazard detection circuitry, etc.); and / or as software executed by an out-of-order core (e.g., a state machine), software executed by a scalar core (e.g., a state machine), software executed by a SIMD core (e.g., a state machine), or a combination thereof. In the example shown, the heterogeneous scheduler is circuit 11401, which includes one or more components for performing various functions. In some embodiments, circuit 11401 is part of processor core 11419; however, it can be part of a chipset.

[0192] The thread / processing element (PE) tracker 11403 maintains the state of each thread executing in the system and in each PE (e.g., PE availability, its current power consumption, etc.). For example, the tracker 11403 maintains the active, idle, or inactive state in a data structure such as a table.

[0193] In some embodiments, pattern matcher 11405 identifies “hot” codes, accelerator codes, and / or codes requesting PE allocation. More details about this matching will be provided later.

[0194] PE information 11409 stores information about what PE (and its type) is in the system and can be scheduled by the OS, etc.

[0195] Although the above details are of separate components within the heterogeneous scheduler circuit 11401, the components may be combined and / or moved outside the heterogeneous scheduler circuit 11401.

[0196] The memory 11413 coupled to the heterogeneous scheduler circuit 11401 may include software (by the core and / or the heterogeneous scheduler circuit 11401) for executing additional functionality. For example, a software pattern matcher 11417 may be used, which identifies “hot” code, accelerator code, and / or code requesting PE allocation. For example, the software pattern matcher 11417 compares a sequence of code against a predetermined set of patterns stored in the memory. The memory may also store converters to translate code from one instruction set to another (e.g., from one instruction set to accelerator-based instructions or primitives).

[0197] These components feed information to selector 11411, which makes choices regarding: the PE of the execution thread, the link protocol used, and what migration should occur if a thread is already executing on that PE, etc. In some embodiments, selector 11411 is a finite state machine implemented in or executed by hardware circuitry.

[0198] For example, in some implementations, memory 11413 may also include one or more converters 11415 (e.g., binary, JIT compiler, etc.) stored in memory to convert thread code into different formats for selected PEs.

[0199] Figure 115 An example of a software heterogeneous scheduler is shown. Software heterogeneous schedulers can be manufactured in many different ways, including but not limited to: as a state machine based on a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC); as an embedded microcontroller coupled to memory (the memory stores software to provide the functionality detailed herein), including logic circuitry comprising other sub-components (e.g., data hazard detection circuitry, etc.); and / or as software executed by an out-of-order core (e.g., a state machine), software executed by a scalar core (e.g., a state machine), software executed by a SIMD core (e.g., a state machine), or a combination thereof. In the example shown, the software heterogeneous scheduler is stored in memory 11413. Thus, memory 11413 coupled to processor core 11419 includes software for scheduling threads (executed by the core). In some embodiments, the software heterogeneous scheduler is part of the operating system.

[0200] Depending on the implementation, the thread / processing element (PE) tracker 11403 in the core maintains the state of each thread executing in the system and each PE (e.g., PE availability, its current power consumption, etc.) or this is performed in software using the thread / PE tracker 11521. For example, the tracker maintains the active, idle, or inactive state in a data structure such as a table.

[0201] In some embodiments, pattern matcher 11405 identifies “hot” codes and / or codes requesting PE assignment. More details about this matching will be provided later.

[0202] PE information 11409 and / or 11509 stores information about what PE is in the system and can be scheduled by the OS, etc.

[0203] The software pattern matcher 11417 can be used to identify "hot" codes, accelerator codes, and / or codes requesting PE allocation.

[0204] Thread / PE trackers, processing element information, and / or pattern matching are fed to selector 11411, which makes selections regarding: the PE executing the thread, the link protocol used, and what migration should occur if a thread is already executing on that PE. In some embodiments, selector 11411 is a finite state machine implemented by processor core 11419.

[0205] For example, in some implementations, memory 11413 may also include one or more converters 11415 (e.g., binary, JIT compiler, etc.) stored in memory to convert thread code into different formats for selected PEs.

[0206] In operation, the OS uses heterogeneous schedulers (e.g., heterogeneous schedulers 101, 301) to schedule and process threads, which presents an abstraction of the execution environment.

[0207] The table below summarizes the potential abstraction features (i.e., what the program sees), potential design freedom and architectural optimizations (i.e., what is hidden from the programmer), and the potential benefits or reasons for providing specific features in the abstraction.

[0208]

[0209] surface

[0210] In some exemplary implementations, the heterogeneous scheduler, combined with other hardware and software resources, presents a complete programming model that runs all content and supports all programming technologies (e.g., compilers, intrinsics, assembly, libraries, JIT, offload, devices). Other exemplary implementations present alternative execution environments that conform to those provided by other processor development companies (e.g., ARM Holdings, Ltd., MIPS, IBM, or their licensors or adopters).

[0211] Figure 119 This is a block diagram of a processor configured to present the abstract execution environment detailed above. In this example, processor 11901 includes several different core types, such as Figure 1 As described in detail in the document. Each (wide) SIMD core 11903 includes fused multiply-accumulate / add (FMA) circuitry supporting intensive arithmetic primitives, its own caches (e.g., L1 and L2), dedicated execution circuitry, and memory for thread state.

[0212] Each latency-optimized (OOO) core 11913 includes fused multiply-accumulate / add (FMA) circuitry, its own caches (e.g., L1 and L2), and out-of-order execution circuitry.

[0213] Each scalar core 11905 includes fused multiply-accumulate / add (FMA) circuitry, its own caches (e.g., L1 and L2), dedicated execution, and storage of thread state. Typically, the scalar core 11905 supports enough threads to cover memory latency. In some implementations, the number of SIMD cores 11903 and latency-optimized cores 11913 is less than the number of scalar cores 11905.

[0214] In some embodiments, one or more accelerators 11905 are included. These accelerators 11905 may be fixed-function or FPGA-based. Alternatively, or in addition to these accelerators 11905, in some embodiments, the accelerators 11905 are external to the processor.

[0215] Processor 11901 also includes a last-level cache (LLC) 11907 shared by the cores, as well as any accelerators that may be present in the processor. In some embodiments, LLC 11907 includes circuitry for fast atoms.

[0216] One or more interconnects 11915 couple the cores and accelerators to each other and to external interfaces. For example, in some embodiments, a mesh interconnect couples various cores.

[0217] The memory controller 11909 couples the core and / or accelerator to the memory.

[0218] Multiple input / output interfaces (e.g., PCIe, the common link detailed below) 11911 connect the processor 11901 to external devices such as other processors and accelerators.

[0219] Figure 4 An embodiment of system booting and device discovery for a computer system is illustrated. The heterogeneous scheduler uses knowledge of the system, including, for example, which cores are available, how much memory is available, and the memory location relative to the cores. In some embodiments, this knowledge is constructed using the Advanced Configuration and Power Interface (ACPI).

[0220] At position 401, the computer system is booted.

[0221] A configuration setting query is performed at address 403. For example, in some BIOS-based systems, when booting, the BIOS tests the system and prepares the computer for operation by querying its own memory to find drivers and other configuration settings.

[0222] At point 405, a search for plug-in components is performed. For example, the BIOS searches for any plug-in components in the computer and sets pointers (interrupt vectors) in memory to access these routines. The BIOS accepts requests from device drivers and applications that interface with hardware and other peripherals.

[0223] At section 407, data structures for system components (e.g., kernel, memory, etc.) are generated. For example, the BIOS typically generates configuration information for hardware devices and peripherals, based on which the OS interfaces with the attached devices. Furthermore, ACPI defines a flexible and scalable hardware interface for the system board and enables the computer to turn its peripherals on and off to improve power management, especially in portable devices such as laptops. The ACPI specification includes hardware interfaces, software interfaces (APIs), and data structures that, when implemented, support configuration and power management as directed by the OS. Software designers can use ACPI to integrate power management functionality throughout the entire computer system, including hardware, operating system, and application software. This integration allows the OS to determine which devices are active and manage all power management resources for the computer subsystems and peripherals.

[0224] At 409, the operating system (OS) is loaded and gains control. For example, once the BIOS completes its boot routine, it transfers control to the OS. When the ACPI BIOS transfers control of the computer to the OS, the BIOS exports a data structure containing the ACPI namespace to the OS, which can be represented as a tree in a graph. The namespace acts as a directory of ACPI devices connected to the computer and includes objects that further define or provide status information for each ACPI device. Each node in the tree is associated with a device, while nodes, child nodes, and leaves represent objects that, when evaluated by the OS, will control the device or return specified information to the OS, as defined by the ACPI specification. The OS or the drivers accessed by the OS may include a set of functions that enumerate and evaluate namespace objects. When the OS calls a function to return a value for an object in the ACPI namespace, the OS is said to evaluate that object.

[0225] In some cases, the available devices may change. For example, accelerators, memory, etc., may be added. Figure 116 The diagram illustrates an embodiment of a method for post-boot device discovery. For example, embodiments of this method can be used to discover accelerators added to the system after boot. An indication is received at 11601 that a connected device has been powered on or reset. For example, an endpoint device may be inserted into a PCIe slot, or it may be reset, for example, by the OS.

[0226] At 11603, link training is performed in conjunction with the connected devices, and the connected devices are initialized. For example, PCIe link training is performed to establish link configuration parameters, such as link width, channel polarity, and / or maximum supported data rate. In some embodiments, the capabilities of the connected devices are stored (e.g., in an ACPI table).

[0227] When the connected device completes initialization, a ready message is sent from the connected device to the system at 11605.

[0228] At 11607, set the connected device ready status bit to indicate that the device is ready for configuration.

[0229] Configure the initialized connected device at 11609. In some embodiments, the device and the OS agree on the device's address (e.g., a memory-mapped I / O (MMIO) address). The device provides a device descriptor that includes one or more of the following: vendor identifier (ID), device ID, model number, serial number, features, resource requirements, etc. The OS can determine additional operational and configuration parameters for the device based on the descriptor data and system resources. The OS can generate a configuration query. The device can respond with the device descriptor. The OS then generates configuration data and sends it to the device (e.g., via PCI hardware). This may include setting a base address register to define the address space associated with the device.

[0230] After building knowledge of the system, the OS uses a heterogeneous scheduler (e.g., heterogeneous scheduler 101, 301) to schedule and process threads. The heterogeneous scheduler then dynamically and transparently (e.g., to the user and / or the OS) maps the code snippets of each thread to the most appropriate type of processing element, potentially avoiding the need to build hardware for traditional architectural features, and potentially avoiding the need to expose the details of the microarchitecture to the system programmer or the OS.

[0231] In some examples, the most suitable type of processing element is determined based on the capabilities of the processing element and the execution characteristics of the code snippet. Typically, programs and associated threads can have different execution characteristics depending on the workload being processed at a given point in time. Exemplary execution characteristics or execution phases include, for example, data-parallel phases, thread-parallel phases, and serial phases. The table below lists these phases and summarizes their characteristics. The table also includes exemplary workloads / operations, exemplary hardware useful in processing each phase type, and typical goals for the phases and hardware used.

[0232]

[0233] surface

[0234] In some implementations, the heterogeneous scheduler is configured to select between thread migration and simulation. In configurations where each type of processing element can handle any type of workload (sometimes requiring simulation to do so), the most suitable processing element is selected for each program stage based on one or more criteria. These criteria include, for example, the latency requirements of the workload, the increased execution latency associated with simulation, the power and thermal characteristics of the processing element, and constraints. As will be described in detail later, in some implementations, the selection of a suitable processing element is accomplished by considering the number of running threads and detecting the presence of SIMD instructions or vectorizable code in the code snippet.

[0235] Moving threads between processing elements is not without penalty. For example, data may need to be moved from a shared cache to a lower-level cache, and both the originating and receiving processing elements will flush their pipelines to accommodate the move. Therefore, in some implementations, heterogeneous schedulers implement hysteresis to avoid overly frequent migrations (e.g., by setting thresholds for one or more of the criteria mentioned above or a subset thereof). In some embodiments, hysteresis is implemented by limiting thread migrations to no more than a predefined rate (e.g., once per millisecond). Thus, the migration rate is limited to avoid excessive overload due to code generation, synchronization, and data migration.

[0236] In some embodiments, such as when the heterogeneous scheduler does not select migration as the preferred method for a particular thread, the heterogeneous scheduler emulates the missing functionality of the thread in the allocated processing elements. For example, in embodiments where the total number of threads available to the operating system remains constant, the heterogeneous scheduler may emulate multithreading when the number of available hardware threads (e.g., in a wide synchronous multithreaded core) is oversubscribed. On scalar or deferred cores, one or more SIMD instructions of a thread are translated into scalar instructions, or more threads and / or translation instructions are generated on SIMD cores to utilize packed data.

[0237] Figure 5 An example of thread migration based on a mapping from program phases to three types of processing elements is shown. As illustrated, the three types of processing elements include latency-optimized (e.g., out-of-order cores, accelerators, etc.), scalar (processing one data item at a time per instruction), and SIMD (processing multiple data elements per instruction). Typically, this mapping is executed by a heterogeneous scheduler on a per-thread or per-code-segment basis in a manner transparent to the programmer and the operating system.

[0238] One implementation uses a heterogeneous scheduler to map each stage of the workload to the most suitable type of processing element. Ideally, this alleviates the need to build hardware for legacy functions and avoids exposing microarchitectural details, as the heterogeneous scheduler provides a complete programming model supporting multiple code types, such as: compiled code (machine code), intrinsic functions (programming language constructs that map to processor or accelerator instructions), assembly code, libraries, intermediate code (JIT-based), offloading (moving from one machine type to another), and device-specific code.

[0239] In some configurations, the default selection for the target processing element is a latency-optimized processing element.

[0240] Refer again Figure 5 Initially, the workload is processed in a serial execution phase 501 on one or more latency-optimized processing elements. Upon detecting a phase shift (e.g., moving dynamically as the code becomes more data-parallel, or moving before execution, as seen by the type of instructions found in the code before or during execution), the workload is migrated to one or more SIMD processing elements to complete the data-parallel execution phase 503. Additionally, execution scheduling and / or transitions are typically cached. Subsequently, the workload is migrated back to one or more latency-optimized processing elements, or to a second set of one or more latency-optimized processing elements, to complete the next serial execution phase 505. Next, the workload is migrated to one or more scalar cores for processing in a thread-parallel execution phase 507. Then, the workload is migrated back to one or more latency-optimized processing elements to complete the next serial execution phase 509.

[0241] While this illustrative example shows a return to the core of latency optimization, a heterogeneous scheduler can continue executing any subsequent execution phases on one or more corresponding processing elements until the thread is terminated. In some implementations, processing elements utilize work queues to store tasks to be completed. Therefore, a task may not start immediately but is executed as its position in the queue is reached.

[0242] Figure 6 This is an exemplary implementation flow executed, for example, by a heterogeneous scheduler (e.g., heterogeneous scheduler 101). The flow describes the selection of processing elements (e.g., cores). As shown, the heterogeneous scheduler receives a code snippet. In some embodiments, events have occurred, including but not limited to: thread wake-up commands; writing to the page directory base address register; sleep commands; thread phase changes; and one or more instructions indicating the need for reallocation.

[0243] At 601, the heterogeneous scheduler determines whether parallelism exists within a code segment (e.g., whether it's a code segment in a serial or parallel phase), for example, based on detected data dependencies, instruction types, and / or control flow instructions. For instance, threads filled with SIMD code will be considered parallel. If the code segment is not suitable for parallel processing, the heterogeneous scheduler selects one or more latency-sensitive processing elements (e.g., OOO cores) to process the code segment in serial execution phase 603. Typically, OOO cores feature (deep) speculative and dynamic scheduling and generally have lower performance per watt compared to simpler alternatives.

[0244] In some embodiments, no latency-sensitive processing elements are available because they typically consume more power and die space than scalar cores. In these embodiments, only scalar, SIMD, and accelerator cores are available.

[0245] For parallel code segments, parallelizable code segments, and / or vectorizable code segments, the heterogeneous scheduler determines the type of parallelism at 605. For thread-parallel code segments, at 607, the heterogeneous scheduler selects a thread-parallel processing element (e.g., a multiprocessor scalar core). Thread-parallel code segments consist of independent sequences of instructions that can be executed concurrently on separate scalar cores.

[0246] Data parallelism occurs when each processing element performs the same task on different data segments. Data parallelism can have different data layouts: packed and random. The data layout is determined at 609. Random data can be assigned to SIMD processing elements, but this requires the use of collection instructions 613 for fetching data from different memory locations, a spatial computation array 615 (which spatially maps computations onto a small array of programmable processing elements, such as an FPGA array), or a scalar processing element array 617. At 611, packed data is assigned to either the SIMD processing element or the processing element using dense arithmetic primitives.

[0247] In some embodiments, code snippets are transformed to better suit the selected destination processing element. For example, the code snippets are: 1) transformed to utilize a different instruction set, 2) made more parallel, 3) made less parallel (serialized), 4) made data parallel (e.g., vectorized), and / or 5) made less data parallel (e.g., unvectorized).

[0248] After selecting a processing element, the code snippet is delivered to one of the selected processing elements for execution.

[0249] Figure 7An example of a method for thread destination selection in a heterogeneous scheduler is shown. In some embodiments, the method is performed by a binary converter. At 701, the thread to be evaluated or a code segment thereof is received. In some embodiments, events have occurred, including but not limited to: a thread wake-up command; a write to the page directory base address register; a sleep command; a thread phase change; and one or more instructions indicating the need for reallocation.

[0250] At 703, it determines whether to unload the code snippet to the accelerator. For example, whether to send the code snippet to the accelerator. When the code includes code that identifies the expectation of using the accelerator, the heterogeneous scheduler knows this is the correct operation. This expectation can be an identifier indicating whether the code region can be executed on the accelerator or locally (e.g., ABEGIN / AEND as described herein) or an explicit command to use a specific accelerator.

[0251] In some embodiments, a code snippet is transformed at 705 to better suit the selected destination processing element. For example, the code snippet is: 1) transformed to utilize a different instruction set, 2) made more parallel, 3) made less parallel (serialized), 4) made data parallel (e.g., vectorized), and / or 5) made less data parallel (e.g., unvectorized).

[0252] Typically, the converted thread is cached at 707 for later use. In some embodiments, the binary converter caches the converted thread locally, making it available for future use by the binary converter. For example, if the code becomes "hot" code (repeatedly executed), the cache provides a mechanism for future use without conversion penalties (although there may be transfer costs).

[0253] At 709, the (converted) thread is transferred (e.g., unloaded) to the destination processing element for processing. In some embodiments, the converted thread is cached by the receiver so that it is locally available for future use. Similarly, this cache will use less energy to achieve faster execution if the receiver or binary converter determines the code to be "hot" code.

[0254] At 711, the heterogeneous scheduler determines whether parallelism exists within a code segment (e.g., whether the code segment is in a serial or parallel phase), for example, based on detected data dependencies, instruction types, and / or control flow instructions. For instance, threads filled with SIMD code will be considered parallel. If the code segment is not suitable for parallel processing, the heterogeneous scheduler selects one or more latency-sensitive processing elements (e.g., OOO cores) to process the code segment in the serial execution phase 713. Typically, OOO cores have (deep) speculative and dynamic scheduling and therefore may have better performance per watt compared to scalar alternatives.

[0255] In some embodiments, no latency-sensitive processing elements are available because they typically consume more power and die space than scalar cores. In these embodiments, only scalar, SIMD, and accelerator cores are available.

[0256] For parallel code segments, parallelizable code segments, and / or vectorizable code segments, the heterogeneous scheduler determines the type of parallelism at 715. For thread-parallel code segments, at 717, the heterogeneous scheduler selects a thread-parallel processing element (e.g., a multiprocessor scalar core). Thread-parallel code segments consist of independent sequences of instructions that can be executed concurrently on separate scalar cores.

[0257] Data parallelism occurs when each processing element performs the same task on different data segments. Data parallelism can have different data layouts: packed and random. The data layout is determined at 719. Random data can be assigned to SIMD processing elements, but this requires the use of collection instructions 723, spatial computation array 725, or scalar processing element array 727. At 721, packed data is assigned to SIMD processing elements or processing elements using dense arithmetic primitives.

[0258] In some embodiments, the unloaded code snippets are transformed to better suit the determined destination processing element. For example, the code snippets are: 1) transformed to utilize a different instruction set, 2) made more parallel, 3) made less parallel (serialized), 4) made data parallel (e.g., vectorized), and / or 5) made less data parallel (e.g., unvectorized).

[0259] After selecting a processing element, the code snippet is delivered to one of the selected processing elements for execution.

[0260] The OS sees the total number of potentially available threads, regardless of which cores and accelerators are accessible. In the following description, each thread is enumerated by a thread identifier (ID) called a logical ID. In some implementations, the operating system and / or a heterogeneous scheduler utilizes a logical ID, a processing element ID, and a thread ID on that processing element (e.g., a tuple of core type, core ID, and thread ID) to map threads to a specific processing element type (e.g., core type). For example, a scalar core has a core ID and one or more thread IDs; a SIMD core has a core ID and one or more thread IDs; an OOO core has a core ID and one or more thread IDs; and / or an accelerator has a core ID and one or more thread IDs.

[0261] Figure 8This illustrates the concept of using striped mapping for logical IDs. Striped mapping can be used by a heterogeneous scheduler. In this example, there are 8 logical IDs and 3 core types, each with one or more threads. Typically, the mapping from logical IDs to (core IDs, thread IDs) is calculated using division and modulo, and can be fixed to maintain software thread affinity. The mapping from logical IDs to (core types) is flexibly performed by the heterogeneous scheduler to accommodate future new core types accessible to the OS.

[0262] Figure 9 This example demonstrates the use of striped mapping for logical IDs. In the example, logical IDs 1, 4, and 5 are mapped to the first core type, and all other logical IDs are mapped to the second core type. No third core type is used.

[0263] In some implementations, core types are grouped. For example, a "core group" tuple may consist of an OOO tuple and all scalar, SIMD, and accelerator core tuples whose logical IDs are mapped to the same OOO tuple. Figure 10 An example of a core group is shown. Typically, serial phase detection and thread migration are performed within the same core group.

[0264] Figure 11 An example of thread execution methods in a system utilizing a binary converter switching mechanism is shown. At 1101, the thread is executing on a core. A core can be any type detailed herein, including accelerators.

[0265] At some point during thread execution, a potential core reallocation event occurs at 1103. Exemplary core reallocation events include, but are not limited to: thread wake-up commands; writes to the page directory base address register; sleep commands; thread phase changes; and one or more instructions indicating a desired reallocation to a different core.

[0266] At 1105, the event is processed and it is determined whether there has been a change in the core allocation. The following details an exemplary method related to processing a specific core allocation.

[0267] In some embodiments, core (re)assignment is subject to one or more limiting factors, such as migration rate limits and power limits. Migration rate limits are tracked for each core type, core ID, and thread ID. Once a thread is assigned to a target (Core type, core ID, thread ID), the binary converter starts and maintains a timer. No other threads will be migrated to the same target until the timer expires. Therefore, while a thread may migrate from its current core before the timer expires, the reverse is not true.

[0268] As detailed, power consumption may become an increasing concern as more core types (including accelerators) are added to computing systems (on-die or off-die). In some embodiments, the instantaneous power consumed by all running threads on all cores is calculated. When the power consumption of the calculation exceeds a threshold, new threads are assigned only to lower-power cores such as SIMD, scalar, and dedicated accelerator cores, and one or more threads are forcibly migrated from the 000 core to the lower-power core. Note that in some implementations, power consumption constraints take precedence over migration rate constraints.

[0269] Figure 12An exemplary method for core allocation of hot code on an accelerator is illustrated. At 1203, code is determined to be "hot". A hot portion of code can refer to a section of code that is more suitable for execution on a core compared to another core, based on considerations such as power, performance, heat, other known processor metrics, or a combination thereof. This determination can be made using any number of techniques. For example, a dynamic binary optimizer can be used to monitor thread execution. Hot code can be detected based on counter values ​​that record the dynamic execution frequency of static code during program execution. In an embodiment where one core is an OOO core and the other is an ordered core, a hot portion of code can refer to a hot spot of program code that is more suitable for execution on the serial core, potentially having more available resources for executing highly repetitive segments. Typically, code segments with high repetition patterns can be optimized for more efficient execution on the ordered core. Essentially, in this example, cold code (low repetition rate) is distributed to the local OOO core, while hot code (high repetition rate) is distributed to the software-managed ordered core. Hot portions of code can be identified statically, dynamically, or a combination of both. In the first case, the compiler or user can determine that a piece of code is hot code. In one embodiment, decoding logic in the core is adapted to decode hot code identifier instructions from the program code, which are used to identify hot parts of the program code. The extraction or decoding of such instructions can trigger the transformation and / or execution of the hot code segment on the core. In another example, the code execution is profiled execution, and regions of program code can be identified as hot code based on the characteristics of the profile—power and / or performance metrics associated with the execution. Similar to hardware operations, monitoring code can be executed on one core to perform monitoring / profiling of program code executing on another core. Note that this monitoring code can be code stored in the core's internal storage structure or code stored in the system, including the processor. For example, monitoring code can be microcode or other code stored in the core's storage structure. As yet another example, static identification of hot code is used as a hint. However, dynamic profiling of program code execution can ignore the static identification of code regions as hot code regions; this type of static identification is often referred to as a compiler or user hint that dynamic profiling can consider when determining which core is suitable for code distribution. Furthermore, as with dynamic profiling, identifying a code region as a hot code region does not limit the identification of that code segment as a hot code segment indefinitely. After transformation and / or optimization, the transformed version of the code segment is executed.

[0270] Choose an appropriate accelerator at point 1203. This selection is based on the available accelerators, the virtual machine monitor, or the operating system, and the desired performance. In many cases, an accelerator is better suited for executing hot code with better performance per watt than a larger, more general-purpose core.

[0271] At 1205, the hot code is transferred to the selected accelerator. This transfer uses the appropriate connection type as detailed in this article.

[0272] Finally, at position 1207, the hot code is received and executed by the selected accelerator. During execution, the hot code can be evaluated to allocate it to different cores.

[0273] Figure 13 An exemplary method for a potential core allocation for a wake-up or write event to the page directory base address register is shown. For example, this illustrates the determination of a phase in a code segment. At 1301, a wake-up event or a page directory base address register (e.g., task switch) event is detected. For example, a wake-up event occurs in response to an interrupt received by a paused thread or a waiting state exit. A write to the page directory base address register can indicate the start or stop of a serial phase. Typically, this detection occurs on the core performing the binary converter.

[0274] At address 1303, the number of cores sharing the same page table base pointer with threads that are being woken up or undergoing task switching is counted. In some implementations, a table is used to map logical IDs to specific heterogeneous cores. This table is indexed by logical IDs. Each entry in the table contains a flag indicating whether the logical ID is currently running or paused, a flag indicating whether a SIMD or scalar core is preferred, a page table base address (e.g., CR3), a value indicating the type of core the logical ID is currently mapped to, and a counter used to limit the migration rate.

[0275] Threads belonging to the same process share the same address space, page table, and page directory base address register value.

[0276] At 1305, the count of cores is determined to be greater than 1. This count determines whether a thread is in the serial or parallel phase. When the count is 1, the thread experiencing this event is in the serial phase 1311. Therefore, a serial phase thread is a thread that has a unique page directory base address register value among all threads in the same core group. Figure 14 An example of a serial phase thread is shown. As illustrated, a process has one or more threads, and each process has its own allocated address.

[0277] When a thread experiencing an event is not assigned to an OOO core, it is migrated to an OOO core, and existing threads on the OOO core are migrated to a SIMD or scalar core at 1313 or 1315. When a thread experiencing an event is assigned to an OOO core, it remains there in most cases.

[0278] When the count is greater than 1, the thread experiencing the event is in the parallel phase, and the type of the parallel phase is determined at 1309. At 1313, when the thread experiencing the event is in the data parallel phase, if the thread is not assigned to a SIMD core, it is assigned to a SIMD core; otherwise, if the thread is already on a SIMD core, it remains on the SIMD core.

[0279] At 1313, when a thread experiencing an event is in the data parallel phase, if the thread is not assigned to a SIMD core, it is assigned to a SIMD core; otherwise, if the thread is already on a SIMD core, it remains on the SIMD core.

[0280] At 1315, when a thread experiencing an event is in the thread-parallel phase, if the thread is not assigned to a scalar core, it is assigned to a scalar core; otherwise, if the thread is already on a scalar core, it remains on the scalar core.

[0281] Additionally, in some implementations, a flag is set for the logical ID of a thread to indicate that the thread is running.

[0282] Figure 15 An exemplary method for determining potential core allocations for a thread in response to a sleep command event is illustrated. For example, this illustrates the phase of determining a code segment. At 1501, a sleep event affecting the thread is detected. For example, an abort, wait-in-and-timeout, or pause command has occurred. Typically, this detection occurs on the core executing the binary converter.

[0283] In some embodiments, at 1503, the logical ID for a thread clears a flag indicating that the thread is running.

[0284] At position 1505, the number of threads sharing the same page table base pointer with the sleeping thread is counted. In some implementations, a table is used to map logical IDs to specific heterogeneous cores. This table is indexed by the logical ID. Each entry in this table contains a flag indicating whether the logical ID is currently running or paused, a flag indicating whether a SIMD or scalar core is preferred, the page table base address (e.g., CR3), a value indicating the type of core the logical ID is currently mapped to, and a counter used to limit the migration rate. Note the first running thread from this group (with any page table base pointer).

[0285] At point 1507, determine if the OOO core in the system is idle. An idle OOO core does not have any running OS threads.

[0286] At 1509, if the page table base pointer is shared by just one thread in the core group, that thread is moved from the SIMD or scalar core to the OOO core. If the page table base pointer is shared by more than one thread, the first running thread in that group is moved from the SIMD or scalar core to the OOO core at 1511 to make room for the awakened thread (which executes at the location of the first running thread).

[0287] Figure 16 An exemplary method for determining a potential core allocation for a thread in response to a stage change event is shown. For example, this illustrates determining the stage of a code segment. At 1601, a potential stage change event is detected. Typically, this detection occurs on the core performing the binary converter.

[0288] At 1603, it is determined whether the thread's logical ID is running on a scalar core and whether SIMD instructions exist. If no such SIMD instructions exist, the thread continues execution as normal. However, if SIMD instructions exist in a thread running on a scalar core, the thread is migrated to a SIMD core at 1605.

[0289] At 1607, it is determined whether the thread's logical ID is running on the SIMD core and whether the SIMD instruction does not exist. If the SIMD instruction exists, the thread continues to execute as normal. However, if the thread running on the SIMD core does not have the SIMD instruction, the thread is migrated to the scalar core at 1609.

[0290] As noted throughout the specification, accelerators accessible from the binary converter can provide more efficient execution (including more energy-efficient execution). However, being able to program each available accelerator can be a difficult (if not impossible) task.

[0291] This document describes in detail an embodiment of using description instructions to explicitly mark the start and end of a portion of an execution thread based on a potential accelerator. When no accelerator is available, the code between the description instructions is executed without using an accelerator. In some implementations, the code between these instructions may relax some of the semantics of the core on which it runs.

[0292] Figure 17An example of code describing an accelerated region is shown. The first instruction in this region is the ABEGIN instruction 1701. In some embodiments, the ABEGIN instruction allows entry into a relaxed execution (sub)mode regarding non-accelerator cores. For example, in some implementations, the ABEGIN instruction allows the programmer or compiler to indicate in a field of the instruction which features of the submode differ from the standard mode. Exemplary features include, but are not limited to, one or more of the following: ignoring self-modifying code (SMC), weakening memory consistency model constraints (e.g., relaxing memory ordering requirements), changing floating-point semantics, changing performance monitoring (perfmon), changing architecture flag usage, etc. In some implementations, SMC is a write to a memory location in a code segment currently cached in the processor, causing the associated cache line (or lines) to be invalidated. If the write affects a prefetch instruction, the prefetch queue is invalidated. The subsequent check is based on the linear address of the instruction. If the target instruction has been decoded and resides in the trace cache, a write or snooping of an instruction in the code segment invalidates the entire trace cache. SMC can be ignored by translating the SMC detection circuitry in the translation back buffer. For example, memory coherence model constraints can be altered by changing settings in one or more registers or tables (e.g., memory type range registers or page attribute tables). For instance, when changing floating-point semantics, the way floating-point execution circuitry performs floating-point calculations can be changed by using one or more control registers that control the behavior of these circuits (e.g., setting the floating-point unit (FPU) control word register). Floating-point semantics that can be changed include, but are not limited to, rounding modes, how exception masks and status flags are handled, zeroing, setting non-standard values, and precision (e.g., single, double, and extended) control. Additionally, in some embodiments, the ABEGIN instruction allows for explicit accelerator type preferences, such that if a preferred type of accelerator is available, that preferred type of accelerator will be selected.

[0293] Non-accelerator code 1703 follows the ABEGIN instruction 1701. This code is native to the system's processor core. In the worst case, if no accelerator is available or ABEGIN is not supported, this code will execute as is on the core. However, in some implementations, a submode is used for execution.

[0294] By including the Accelerated End (AEND) instruction 1705, execution is gated on the processor core until the accelerator appears to have completed its execution. In effect, the use of ABEGIN and AEND allows programmers to choose to join / leave using the accelerator and / or relax the execution mode.

[0295] Figure 18An example of an execution method using ABEGIN in a hardware processor core is illustrated. At 1801, the ABEGIN instruction for the thread is extracted. As previously mentioned, the ABEGIN instruction typically includes one or more fields for defining different execution (sub) modes.

[0296] At 1803, the extracted ABEGIN instruction is decoded using a decoding circuit. In some embodiments, the ABEGIN instruction is decoded into a micro-operation.

[0297] At 1805, the decoded ABEGIN instruction is executed by the execution circuitry to put the thread into a different mode (which can be explicitly defined by one or more fields of the ABEGIN instruction) for instructions following the ABEGIN instruction but preceding the AEND instruction. Depending on the availability and selection of the accelerator, this different execution mode can be on the accelerator or on an existing core. In some embodiments, the accelerator is chosen to be executed by a heterogeneous scheduler.

[0298] At 1807, subsequent non-AEND instructions are executed in a different execution mode. When the accelerator is used for execution, the instructions can first be converted by a binary converter into different instruction sets.

[0299] Figure 19 An example of an execution method using AEND in a hardware processor core is shown. At 1901, the AEND instruction is extracted.

[0300] At 1903, the extracted AEND instruction is decoded using a decoding circuit. In some embodiments, AEND is decoded into a micro-operation.

[0301] At 1905, the decoded AEND instruction is executed by the execution circuitry to restore the execution mode previously set by the ABEGIN instruction. This different execution mode can be on an accelerator or on an existing core, depending on accelerator availability and selection.

[0302] At 1807, subsequent non-AEND instructions are executed in raw execution mode. When the accelerator is used for execution, the instructions can first be converted by a binary converter into different instruction sets.

[0303] Figure 124 This example demonstrates execution when ABEGIN / AEND is not supported. At 12401, the ABEGIN instruction is fetched. At 12403, it is determined that ABEGIN is not supported. For example, CPUID indicates that support is not available.

[0304] When no support is available, a no-op is typically performed at 12405, which does not change the context associated with the thread. Since the execution mode remains unchanged, instructions following the unsupported ABEGIN are executed as usual at 12407.

[0305] In some embodiments, at least pattern matching is used to achieve an equivalent use of ABEGIN / AEND. This pattern matching can be based on hardware, software, and / or both. Figure 20 A system using pattern matching to provide ABEGIN / AEND equivalences is illustrated. The system includes a scheduler 2015 (e.g., a heterogeneous scheduler as detailed above), which includes a converter 2001 (e.g., a binary converter, JIT, etc.) stored in memory 2005. Core circuitry 2007 executes scheduler 2015. Scheduler 2015 receives thread 2019, which may or may not have explicit ABEGIN / AEND instructions.

[0306] Scheduler 2015 manages the software-based pattern matcher 2003, executes traps and context switches during unloading, manages user-space save regions (detailed later), and generates or converts to accelerator code 2011. Pattern matcher 2003 identifies (predefined) sequences of code stored in memory that are found in receiving threads 2019 that may benefit from accelerator use and / or relaxed execution states, but are not described using ABEGIN / AEND. Typically, the patterns themselves are stored in converter 2001, but are at least accessible by pattern matcher 2003. Selector 2019 functions as previously detailed.

[0307] Scheduler 2015 also offers performance monitoring features. For example, if the code doesn't have a perfect pattern match, Scheduler 2015 recognizes that the code may still need to relax requirements to be more efficient and adjusts the operation patterns associated with threads accordingly. The relationships between operation patterns have been described in detail above.

[0308] Scheduler 2015 also performs one or more of the following operations: causing cores to loop in the ABEGIN / AEND region, keeping accelerator loops active or stopped, counting ABEGIN calls, delaying accelerator queuing (synchronous processing), and monitoring memory / cache statistics. In some embodiments, binary converter 2001 includes accelerator-specific code for interpreting accelerator code, which can be used to identify bottlenecks. The accelerator executes this converted code.

[0309] In some embodiments, core circuitry 2007 includes a hardware pattern matcher 2009 to identify (predefined) code sequences in received thread 2019 using stored patterns 2017. Typically, this pattern matcher 2009 is lightweight compared to a software pattern matcher 2003 and seeks simple representation regions (e.g., rep moves). The identified code sequences can be transformed by scheduler 2015 for use with an accelerator, and / or may result in a relaxation of the thread's operating mode.

[0310] Coupled to the system are one or more accelerators 2013, which receive accelerator code 2011 for execution.

[0311] Figure 21 An embodiment of an execution method for a non-accelerated description thread exposed to pattern recognition is shown. This method is executed by a system comprising at least one type of pattern matcher.

[0312] In some embodiments, a thread is executed at 2101. Typically, this thread executes on a non-accelerator core. The instructions for executing the thread are fed into the pattern matcher. However, the instructions for the thread may be fed into the pattern matcher before any execution.

[0313] At 2103, patterns within the thread are identified (detected). For example, software-based pattern matchers or hardware pattern matcher circuitry find patterns that are typically associated with available accelerators.

[0314] At 2105, the identified pattern is converted for use with available accelerators. For example, a binary converter translates the pattern into accelerator code.

[0315] At 2107, the converted code is sent to an available accelerator for execution.

[0316] Figure 22 An embodiment of an execution method for a non-accelerated description thread exposed to pattern recognition is shown. This method comprises, for example, […]. Figure 20 The system execution of at least one type of pattern matcher in the system.

[0317] In some embodiments, a thread is executed at 2201. Typically, this thread executes on a non-accelerator core. Instructions for executing the thread are fed into the pattern matcher. However, the thread's instructions may be fed into the pattern matcher before any execution.

[0318] At 2203, patterns within the thread are identified (detected). For example, software-based pattern matchers or hardware pattern matcher circuitry find patterns that are typically associated with available accelerators.

[0319] At 2205, the binary converter adjusts the operating mode associated with the thread to use relaxed requirements based on the identified mode. For example, the binary converter utilizes settings associated with the identified mode.

[0320] As detailed, in some embodiments, parallel regions of code are defined by ABEGIN and AEND instructions. Within an ABEGIN / AEND block, there are guarantees of independence for certain memory load and store operations. Other loads and stores allow for potential dependencies. This allows implementations to parallelize blocks with little or no checking of memory dependencies. In all cases, serial execution of the block is permitted because serial cases are included among the possible ways to execute the block. The binary converter performs static dependency analysis to create instances of parallel execution and maps these instances to hardware. Static dependency analysis can parallelize iterations of outer, intermediate, or inner loops. Slicing is implementation-dependent. The ABEGIN / AEND implementation extracts parallelism at a size optimal for the implementation.

[0321] ABEGIN / AEND blocks can contain multiple levels of nested loops. Implementations are free to choose the amount of parallel execution supported, or fall back to serial execution. ABEGIN / AEND provides parallelism over a much larger area than SIMD instructions. For certain types of code, ABEGIN / AEND allows for hardware implementations that are more efficient than multithreading.

[0322] By using ABEGIN / AEND, programmers and / or compilers can fall back to traditional serial execution by the CPU core if parallelization standards are not met. When executing on a traditional out-of-order CPU core, ABEGIN / AEND reduces the area and power requirements of the memory sort buffer (MOB) due to the relaxed memory sorting.

[0323] Within the ABEGIN / AEND block, the programmer specifies memory dependencies. Figure 23 Different types of memory dependencies 2301, their semantics 2303, ordering requirements 2305, and use cases 2307 are illustrated. Furthermore, depending on the implementation, some semantics apply to instructions within an ABEGIN / AEND block. For example, in some embodiments, register dependencies are allowed, but modifications to registers do not persist beyond AEND. Additionally, in some embodiments, one must enter the ABEGIN / AEND block at ABEGIN and exit at AEND (or enter a similar state based on pattern recognition), without branches entering / exiting the ABEGIN / AEND block. Finally, instruction flow modifications are generally not permitted.

[0324] In some implementations, the ABEGIN instruction includes a source operand, which includes a pointer to a block of memory data. This data block contains a wealth of information used at runtime and by core circuitry to process the code within the ABEGIN / AEND block.

[0325] Figure 24 An example of a memory data block pointed to by the ABEGIN instruction is shown. As shown, depending on the implementation, the memory data block includes fields for serial number 2401, block class 2403, implementation identifier 2405, saved state area size 2407, and local storage area size 2409.

[0326] Serial number 2401 indicates how much (parallel) computation the processor had performed before the interrupt. The software initializes serial number 2401 to zero before executing ABEGIN. Execution of ABEGIN writes a non-zero value to serial number 2401 to track execution progress. Upon completion, execution of AEND writes zero to reinitialize serial number 2401 for its next use.

[0327] The predefined block class identifier 2403 (i.e., GUID) specifies the predefined ABEGIN / AEND block class. For example, DMULADD and DGEMM can be predefined as block classes. Using predefined classes, the binary converter does not need to analyze the binary file to perform heterogeneous hardware mapping analysis. Instead, the converter (e.g., a binary converter) simply performs pre-generated transformations for that ABEGIN / AEND class by taking the input values. The code enclosed in ABEGIN / AEND is only used for code that executes this type of transformation on a non-dedicated core.

[0328] The Implementation ID field 2405 indicates the type of execution hardware being used. Execution of ABEGIN updates this field 2405 to indicate the type of heterogeneous hardware being used. This facilitates the migration of ABEGIN / AEND code to machines with different acceleration hardware types or no accelerators at all. This field allows for possible conversions of saved contexts to match the target implementation. Alternatively, an emulator may be used to execute the code until it exits AEND after a migration that occurs when the ABEGIN / AEND code is interrupted and migrated to a machine without the same accelerator type. This field 2405 also allows the system to dynamically reassign ABEGIN / AEND blocks to different heterogeneous hardware within the same machine, even if the execution of the ABEGIN / AEND block is interrupted in the middle.

[0329] State storage area field 2407 indicates the size and format of the implementation-specific state storage area. The implementation will ensure that the implementation-specific portion of the state storage area does not exceed a certain maximum value specified in CPUID. Typically, execution of the ABEGIN instruction results in the writing of the state storage areas, associated flags, and additional implementation-specific states of the general and packed data registers to be modified within the ABEGIN / AEND block. To facilitate parallel execution, multiple register instances can be written.

[0330] Local storage region 2409 is allocated as local storage. The amount of storage to be reserved is typically specified as the immediate operand of ABEGIN. When the ABEGIN instruction is executed, a specific register (e.g., R9) is written using the address of local storage region 2409. If a fault occurs, this register is made to point to the sequence number.

[0331] Each instance executing in parallel receives a unique local storage region 2409. This address will be different for each instance executing in parallel. In serial execution, a storage region is allocated. Local storage region 2409 provides temporary storage beyond the general-purpose and packed data registers of the architecture. Local storage region 2409 should not be accessed outside of the ABEGIN / AEND blocks.

[0332] Figure 25 An example of a memory 2503 configured to use ABEGIN / AEND semantics is shown. Hardware (such as the various processing elements described herein) supporting ABEGIN / AEND and utilizing this memory 2503 is not shown. As detailed, the memory 2503 includes a state storage area 2507, which includes indications 2501 regarding registers to be used, flags 2505, and implementation-specific information 2511. Additionally, local storage 2509 for each parallel execution instance is stored in the memory 2503.

[0333] Figure 26 Examples of methods using ABEGIN / AEND to operate in different execution modes are shown. Typically, this method is executed by a combination of entities such as converters and execution circuitry. In some embodiments, the thread is switched before entering this mode.

[0334] At 2601, a different execution mode is entered, such as a relaxed execution mode (with or without the accelerator). This mode is typically entered by executing the ABEGIN instruction; however, as mentioned above, it can also be entered due to pattern matching. Entering this mode involves resetting the sequence number.

[0335] Write to the saved state area at 2603. For example, write the general and packed data registers to be modified, associated flags, and additional implementation-specific information. This area allows for restarting execution or rollback if an error occurs in the block (e.g., an interrupt).

[0336] A local storage area is reserved for each parallel execution instance at position 2605. The size of this area is determined by the state storage area field detailed above.

[0337] During block execution, the block's progress is tracked at point 2607. For example, when the instruction executes successfully and exits, the block's sequence number is updated.

[0338] At 2609, it is determined whether the AEND instruction has been reached (e.g., to determine if the block is complete). If not, the local storage area is updated at 2613 with the intermediate results. If possible, execution picks from these results; however, in some cases, a rollback occurs at 2615 prior to ABEGIN / AEND. For example, if an exception or interrupt occurs during the execution of the ABEGIN / AEND block, the instruction pointer will point to the ABEGIN instruction, and the R9 register will point to the memory data block updated with the intermediate results. Upon recovery, the state saved in the memory data block will be used to restore at the correct point. Additionally, if the initial portion of the memory data block (up to and including the state-saving area) is missing or inaccessible, a page fault is raised. For loads and writes to the local storage area, page faults are reported in the usual manner, i.e., upon first access to a non-existent or inaccessible page. In some cases, non-accelerator processing elements will be used upon restarting.

[0339] If the block completes successfully, the reserved registers and flags are restored at 2611. Only the memory state will differ after the block.

[0340] Figure 27 Examples of methods using ABEGIN / AEND to operate in different execution modes are shown. Typically, this method is executed by a combination of entities such as binary converters and execution circuitry.

[0341] At 2701, a different execution mode is entered, such as a relaxed execution mode (with or without the accelerator). This mode is typically entered by executing the ABEGIN instruction; however, as mentioned above, it can also be entered due to pattern matching. Entering this mode involves resetting the sequence number.

[0342] Write to the saved state area at 2703. For example, write the general and packed data registers to be modified, associated flags, and additional implementation-specific information. This area allows for restarting execution or rollback if an error occurs in the block (e.g., an interrupt).

[0343] A local storage area is reserved for each parallel execution instance at position 2705. The size of this area is determined by the state storage area field detailed above.

[0344] At 2706, the code within the translation block is prepared for execution.

[0345] During the execution of the transformed block, the block's progress is tracked at 2707. For example, when the instruction executes successfully and exits, the block's sequence number is updated.

[0346] At 2709, it is determined whether the AEND instruction has been reached (e.g., to determine if the block is complete). If not, the local storage area is updated at 2713 with the intermediate results. If possible, execution picks from these results; however, in some cases, a rollback occurs at 2715 prior to ABEGIN / AEND. For example, if an exception or interrupt occurs during the execution of the ABEGIN / AEND block, the instruction pointer will point to the ABEGIN instruction, and the R9 register will point to the memory data block updated with the intermediate results. Upon recovery, the state saved in the memory data block will be used to restore at the correct point. Additionally, if the initial portion of the memory data block (up to and including the state-saving area) is missing or inaccessible, a page fault is raised. For loads and writes to the local storage area, page faults are reported in the usual manner, i.e., upon first access to a non-existent or inaccessible page. In some cases, non-accelerator processing elements will be used upon restarting.

[0347] If the block completes successfully, the reserved registers and flags are restored at position 2711. Only the memory state will differ after the block.

[0348] As described above, in some implementations, a common link (referred to as a multiprotocol common link (MCL)) is used to reach devices (such as... Figure 1 and 2(The processing elements described herein). In some embodiments, these devices are considered PCI Express (PCIe) devices. Three or more protocols are dynamically multiplexed on this link. For example, protocols supported by the common link include: 1) a Producer / Consumer Discovery, Configuration, Interruption (PDCI) protocol to enable device discovery, device configuration, error reporting, interruption, DMA-style data transfer, and various services, such as those specified in one or more proprietary or industry standards (e.g., the PCI Express specification or equivalent alternatives); 2) a Cache Agent Coherence (CAC) protocol that enables devices to issue consistent read and write requests to processing units; and 3) a Memory Access (MA) protocol that enables processing elements to access the local memory of another processing element. While specific examples of these protocols are provided below (e.g., Intel System-on-Chip Architecture (IOSF), In-Die Interconnect (IDI), Scalable Memory Interconnect 3+ (SMI3+)), the basic principles of the invention are not limited to any particular set of protocols.

[0349] Figure 120 A simplified block diagram 12000 illustrating an exemplary multi-chip configuration 12005 includes two or more chips or dies (e.g., 12010, 12015) communicatively connected using an exemplary multi-chip link (MCL) 12020. Although Figure 120 An example of two (or more) dies interconnected using the exemplary MCL 12020 is shown; however, it should be understood that the principles and features described herein with respect to the implementation of the MCL can be applied to any interconnect or link connecting a die (e.g., 12010) to other components, including connecting two or more dies (e.g., 12010, 12015), connecting a die (or chip) to another component outside the die, connecting a die to another device or die outside the package (e.g., 12005), connecting a die to a BGA package, implementations of point-on-intermediate layer (POINT), and other possible examples.

[0350] In some cases, larger components (e.g., dies 12010, 12015) may themselves be IC systems (e.g., system-on-a-chip (SoC)), multiprocessor chips, or other components on a device (e.g., on a single die (e.g., 12010, 12015) including multiple components such as cores, accelerators, etc., (12025-12030 and 12040-12045). MCL 12020 provides the flexibility to build complex and varied systems from potentially multiple discrete components and systems. For example, each of dies 12010, 12015 may be manufactured by two different entities or otherwise provided. Furthermore, dies and other components may themselves include interconnects or other communication structures (e.g., 12035, 12050) that provide the infrastructure for communication between components (e.g., 12025-12030 and 12040-12045) within the device (e.g., 12010, 12015, respectively). Various components and interconnects (e.g., 12035, 12050) support or use a variety of different protocols. Furthermore, communication between dies (e.g., 12010, 12015) can potentially include transactions between various components on the die via multiple different protocols.

[0351] Multi-chip link (MCL) embodiments support multiple packaging options, multiple I / O protocols, and reliability, availability, and serviceability (RAS) characteristics. Furthermore, the physical layer (PHY) can include physical electrical and logic layers and can support longer channel lengths, including channel lengths up to and in some cases exceeding approximately 45 mm. In some implementations, exemplary MCLs can operate at high data rates, including data rates exceeding 8-10 Gb / s.

[0352] In one exemplary implementation of MCL, the PHY electrical layer improves upon conventional multi-channel interconnect solutions (e.g., multi-channel DRAM I / O) by extending data rates and channel configurations, for example through a number of features, including, for example, regulated mid-rail termination, low-power active crosstalk cancellation, circuit redundancy, bit duty cycle correction and skew cancellation, line coding and transmitter equalization, and other possible examples.

[0353] In one exemplary implementation of MCL, a PHY logical layer is implemented such that it further facilitates (e.g., electrical layer features) data rate and channel configuration, while also enabling interconnects to route multiple protocols across electrical layers. Such an implementation provides and defines a modular common physical layer that is protocol-agnostic and constructed to work with any potential existing or future interconnect protocols.

[0354] Go to Figure 121A simplified block diagram 12100 is shown, representing at least a portion of a system including an exemplary implementation of a multi-chip link (MCL). The MCL can be implemented by connecting a first device 12105 (e.g., a first die including one or more sub-components) to a second device 12110 (e.g., a second die including one or more other sub-components) using physical electrical connections (e.g., lines implemented as pathways). In the specific example shown in the high-level representation of Figure 12100, all signals (in channels 12115, 12120) can be unidirectional, and pathways can be provided for data signals to have upstream and downstream data transmission. Although... Figure 121 The block diagram 12100 designates the first component 12105 as the upstream component and the second component 12110 as the downstream component. The physical path of the MCL used to transmit data is designated as the downstream channel 12115, and the path used to receive data (from component 12110) is designated as the upstream channel 12120. However, it should be understood that the MCL between devices 12105 and 12110 can be used by each device to transmit and receive data between devices.

[0355] In one exemplary embodiment, the MCL may provide a physical layer (PHY) including an electrical MCL physical layer (PHY) 12125a, b (or collectively referred to as 12125) and executable logic implementing MCL logic PHYs 12130a, b (or collectively referred to as 12130). The electrical or physical PHY 12125 provides a physical connection through which data is transmitted between devices 12105, 12110. Signal conditioning components and logic may be implemented in conjunction with the physical PHY 12125 to establish high data rates and channel configuration capabilities for links, which in some applications involve tight cluster physical connections of approximately 45 mm or longer. The logic PHY 12130 includes circuitry for facilitating clocking, link state management (e.g., for link layers 12135a, 12135b), and protocol multiplexing between potentially multiple different protocols communicating via the MCL.

[0356] In one exemplary embodiment, the physical PHY 12125 includes a set of data paths for each channel (e.g., 12115, 12120) on which in-band data is transmitted. In this particular example, 50 data paths are provided in each of the upstream and downstream channels 12115, 12120, but any other number of paths may be used where layout and power constraints, desired application, device constraints, etc., allow. Each channel may also include one or more dedicated paths for channel gating or clocking, one or more dedicated paths for channel active signals, one or more dedicated paths for streaming signals, and one or more dedicated paths for link state machine management or sideband signals. The physical PHY may also include a sideband link 12140, which in some examples may be a bidirectional low-frequency control signal link for coordinating state transitions and other properties of MCL connection devices 12105, 12110, and other examples are also possible.

[0357] As described above, an implementation using MCL is used to support multiple protocols. In practice, multiple independent transaction processing layers 12150a, 12150b can be provided at each device 12105, 12110. For example, each device 12105, 12110 can support and utilize two or more protocols, such as PCI, PCIe, CAC, etc. CAC is a coherence protocol used on a die for communication between the core, last-level cache (LLC), memory, graphics, and I / O controllers. Other protocols can also be supported, including Ethernet, Infiniband, and other PCIe-based protocols. A combination of logical and physical PHYs can also be used as a die-to-die interconnect to connect a SerDes PHY (PCIe, Ethernet, Infiniband, or other high-speed SerDes) on one die to a higher layer implemented on another die, and other examples are also possible.

[0358] The logic PHY 12130 supports multiplexing among these multiple protocols on the MCL. For example, a dedicated flow path can be used to enable coded stream signals that identify which protocol will be applied to data transmitted substantially simultaneously on the data path of the channel. Furthermore, the logic PHY 12130 negotiates various types of link state transitions that different protocols can support or request. In some cases, LSM_SB signals transmitted via a dedicated LSM_SB path of the channel, along with sideband link 12140, can be used to communicate and negotiate link state transitions between devices 12105 and 12110. Additionally, link training, error detection, skew detection, skew cancellation, and other functions of traditional interconnects can be partially replaced or controlled using the logic PHY 12130. For example, valid signals transmitted via one or more dedicated valid signal paths in each channel can be used to signal link activity, detect skew, link errors, and implement other features, and other examples are possible. Figure 121 In specific examples, each channel provides multiple valid paths. For instance, data paths within a channel can be bundled or clustered (physically and / or logically), and a valid path can be provided for each cluster. Furthermore, in some cases, multiple gating paths can be provided to provide a dedicated gating signal for each cluster in a channel with multiple data paths within a cluster; other examples are also possible.

[0359] As described above, the logic PHY 12130 negotiates and manages link control signals transmitted between devices connected by the MCL. In some embodiments, the logic PHY 12130 includes a Link Layer Packet (LLP) generation circuit 12160 to transmit link layer control messages via the MCL (i.e., in-band). These messages can be transmitted via the channel's data path, where the path identification data is a link layer-to-link layer message pass, such as link layer control data, and other examples may also exist. Link layer messages enabled using the LLP module 12160 facilitate the negotiation and execution of link layer state transitions, power management, loopback, disable, re-centering, scrambling, and other link layer features between the link layers 12135a and 12135b of devices 12105 and 12110, respectively.

[0360] Go to Figure 122 A simplified block diagram 12200 is shown, illustrating an exemplary logical PHY for an exemplary MCL. A physical PHY 12205 can be connected to a die, which includes a logical PHY 12210 and additional logic supporting the link layer of the MCL. In this example, the die may also include logic supporting multiple different protocols over the MCL. For example, in... Figure 122In the example, PCIe logic 12215 and CAC logic 12220 are provided, allowing the dies to communicate using PCIe or CAC through the same MCL connecting the two dies. Many other examples are also possible, including those supporting more than two protocols besides PCIe and CAC on the MCL. The various protocols supported between dies can provide different levels of service and features.

[0361] The logic PHY 12210 may include link state machine management logic 12225 for negotiating link state transitions in conjunction with requests from the die's upper-layer logic (e.g., received via PCIe or CAC). In some implementations, the logic PHY 12210 may also include link testing and debugging logic (e.g., 12230). As described above, the exemplary MCL may support control signals transmitted between dies via the MCL to facilitate the MCL's protocol agnostic, high-performance, and power-saving characteristics (among other example features). For example, the logic PHY 12210 may support the generation and transmission, as well as the reception and processing, of valid signals, streaming signals, and LSM sideband signals related to sending and receiving data via a dedicated data path, as described in the examples above.

[0362] In some implementations, multiplexing (e.g., 12235) and demultiplexing (e.g., 12240) logic may be included in logic PHY 12210, or may otherwise be accessible to logic PHY 12210. For example, multiplexing logic (e.g., 12235) may be used to identify data to be sent to the MCL (e.g., embodied as packets, messages, etc.). Multiplexing logic 12235 may identify the protocol managing the data and generate a stream signal encoded to identify the protocol. For example, in one exemplary implementation, the stream signal may be encoded as bytes of two hexadecimal symbols (e.g., CAC: FFh; PCIe: F0h; LLP: AAh; sideband: 55h; etc.) and may be sent during the same window (e.g., a byte time window) of data controlled by the identified protocol. Similarly, demultiplexing logic 12240 may be used to interpret the input stream signal to decode the stream signal and identify the protocol applied to data received simultaneously with the stream signal on the data path. Then, the demultiplexing logic 12240 can apply (or ensure) protocol-specific link layer processing and allow data to be processed by the appropriate protocol logic (e.g., PCIe logic 12215 or CAC logic 12220).

[0363] The logic PHY 12210 may also include link-layer packet logic 12250, which can be used to handle various link control functions, including power management tasks, loopback, disable, recenter, scrambling, etc. LLP logic 12250 can facilitate link-layer to link-layer messages through MCLP, as well as other features. Data corresponding to LLP signaling can also be identified by flow signals transmitted on a dedicated flow signaling path, which is encoded to identify LLP data in the data path. Multiplexing and demultiplexing logic (e.g., 12235, 12240) can also be used to generate and interpret flow signals corresponding to LLP services, and to enable such services to be processed by appropriate die logic (e.g., LLP logic 12250). Similarly, some implementations of MCLP may include dedicated sidebands (e.g., sideband 12255 and supporting logic), such as asynchronous and / or low-frequency sideband channels, and other examples are also possible.

[0364] The logic PHY logic 12210 may also include link state machine management logic, which can generate and receive (and use) link state management messages via a dedicated LSM sideband path. For example, the LSM sideband path can be used to perform handshakes to advance the link training state, exit power management states (e.g., L1 state), etc. The LSM sideband signal can be asynchronous, as it is not aligned with the link's data, valid, and stream signals, but rather corresponds to signaling state transitions and aligns the link state machine between the two dies or chips connected via the link; other examples are also possible. In some examples, providing a dedicated LSM sideband path can allow for the elimination of traditional squelch and receive detection circuitry in the analog front-end (AFE), among other exemplary benefits.

[0365] Go to Figure 123 A simplified block diagram 12300 is shown, illustrating another representation of the logic used to implement the MCL. For example, a logical PHY 12210 is provided with a defined logical PHY interface (LPIF) 12305, through which any of several different protocols (e.g., PCIe, CAC, PDCI, MA, etc.) 12315, 12320, 12325 and signaling modes (e.g., sidebands) can be connected to the physical layer of the example MCL. In some implementations, multiplexing and arbitration logic 12330 may also be provided as a separate layer from the logical PHY 12210. In one example, the LPIF 12305 may be provided as an interface on either side of the MuxArb layer 1230. The logical PHY 12210 may interface with a physical PHY (e.g., the analog front-end (AFE) 12205 of the MCL PHY) via another interface.

[0366] LPIF can extract PHY (logic and electrical / analog) from upper layers (e.g., 12315, 12320, 12325), allowing for the implementation of entirely different PHYs under an LPIF that is transparent to the upper layers. This can help promote modularity and reuse in design, as the upper layer can remain intact or have alternatives when updating the lower-level signaling technology PHY. Furthermore, LPIF can define multiple signals, enabling multiplexing / demultiplexing, LSM management, error detection and handling, and other functionalities of the logic PHY. For example, the following table summarizes at least some of the signals that can be defined for an exemplary LPIF:

[0367]

[0368] As shown in the table, in some implementations, alignment mechanisms can be provided via the AlignReq / AlignAck handshake. For example, some protocols may lose framing when the physical layer enters recovery. For instance, packet alignment can be corrected to ensure proper framing identification at the link layer. The physical layer can activate the StallReq signal upon entering recovery, so that the link layer activates the Stall signal when ready to transmit new aligned packets. The physical layer logic can sample Stall and Valid to determine if packets are aligned. For example, the physical layer can continue driving the trdy to drain link layer packets until Stall and Valid are sampled as valid signals. Other implementations are also possible, including alternative implementations that use Valid to assist packet alignment.

[0369] Various fault tolerances can be defined for signals on the MCL. For example, fault tolerance can be defined for active, stream, LSM sideband, low-frequency sideband, link layer packets, and other types of signals. Fault tolerance for packets, messages, and other data transmitted through the dedicated data path of the MCL can be based on a specific protocol for managing the data. In some implementations, error detection and handling mechanisms can be provided, such as cyclic redundancy check (CRC), retry buffers, and other possible examples. As an example, for PCIe packets transmitted through the MCL, a 32-bit CRC can be used for PCIe transaction layer packets (TLP) (with guaranteed delivery (e.g., via replay mechanism)), and a 16-bit CRC can be used for PCIe link layer packets (which may be constructed to be lossy (e.g., without replay)). Furthermore, for PCIe framing tokens, a specific Hamming distance can be defined for the token identifier (e.g., a Hamming distance of four (4)); in other examples, parity check and a 4-bit CRC can also be used. On the other hand, for CAC packets, a 16-bit CRC can be used.

[0370] In some implementations, fault tolerance is defined for link layer packets (LLPs) that utilize the transition of valid signals from low to high (i.e., from 0 to 1) (e.g., to help ensure bit and symbol locking). Furthermore, in one example, a specific number of consecutive identical LLPs can be defined for transmission, and a response to each request can be expected, where the requester retryes after a response timeout. There may also be other defined features that can be used as a basis for determining faults in the LLP data on the MCL. In other examples, fault tolerance can be provided for valid signals, for example, by extending the valid signal or symbols over the entire time window or symbol (e.g., by keeping the valid signal high over 8 UIs). Additionally, errors or faults in the streaming signal can be prevented by maintaining the Hamming distance between the encoded values ​​of the streaming signal, and other examples exist.

[0371] Implementations of the logical PHY include error detection, error reporting, and error handling logic. In some implementations, the logical PHY of an example MCL may include logic for detecting PHY layer deframe errors (e.g., on active and circulating paths), sideband errors (e.g., related to LSM state transitions), errors in the LLP (e.g., critical for LSM state transitions), and so on. Some error detection / solutions may be delegated to higher-layer logic, such as PCIe logic suitable for detecting PCIe-specific errors, or other examples may exist.

[0372] In the event of a deframe error, in some implementations, one or more mechanisms can be provided through error handling logic. Deframe errors can be handled based on the protocol involved. For example, in some implementations, the error can be notified to the link layer to trigger a retry. Deframe can also result in the logical PHY realigning the deframed frame. Furthermore, logical PHY recentering can be performed, and symbol / window locking can be regained, among other techniques. In some examples, centering may include the PHY moving the receiver clock phase to the optimal point for detecting input data. In this case, "optimal" may refer to the point with the maximum tolerance for noise and clock jitter. Recentering may include simplified centering functions, for example, performed when the PHY wakes up from a low-power state, and other examples are possible.

[0373] Other types of errors may involve different error handling techniques. For example, errors detected in the sideband can be captured through a timeout mechanism for the corresponding state (e.g., LSM). The error can be logged, and the link state machine can then transition to a reset. The LSM can remain in the reset state until a restart command is received from the software. In another example, LLP errors (e.g., link control packet errors) can be handled with a timeout mechanism that can restart the LLP sequence if no acknowledgment is received.

[0374] In some embodiments, each of the above protocols is a variant of PCIe. PCIe devices communicate using a common address space associated with the bus. This address space is either the bus address space or the PCIe address space. In some embodiments, PCIe devices use addresses in an internal address space that may differ from the PCIe address space.

[0375] The PCIe specification defines a mechanism that allows PCIe devices to expose their local memory (or a portion thereof) to the bus, enabling a CPU or other device connected to the bus to directly access their memory. Typically, a dedicated region within the PCIe address space, known as the PCIe Base Address Register (BAR), is allocated to each PCIe device. Furthermore, the addresses exposed by the device are mapped to corresponding addresses in the PCI BAR.

[0376] In some embodiments, a PCIe device (e.g., an HCA) uses an Input / Output Memory Mapping Unit (IOMMU) to translate between its internal addresses and PCIe bus addresses. In other embodiments, a PCIe device may use a PCI Address Translation Service (ATS) to perform address translation and resolution. In some embodiments, a label such as a Process Address Space ID (PASID) label is used to specify the address to be translated to belong to the virtual address space of a particular process.

[0377] Figure 28 Additional details of one implementation are shown. As in the above implementation, this implementation includes an accelerator 2801 having accelerator memory 2850 coupled to a host processor 2802 having host memory 2860 via a multiprotocol link 2800. As mentioned above, the accelerator memory 2850 may use a different memory technology than the host memory 2860 (e.g., the accelerator memory may be HBM or stacked DRAM, while the host memory may be SDRAM).

[0378] Multiplexers 2811 and 2812 are shown to highlight that the multiprotocol link 2800 is a dynamic multiplexing bus supporting PCDI, CAC, and MA protocol (e.g., SMI3+) traffic, where each traffic can be routed to different functional units within accelerator 2801 and host processor 2802. These protocols may include IOSF, IDI, and SMI3+ as examples, not limitations. In one embodiment, the PCIe logic 2820 of accelerator 2801 includes a local TLB 2822 for caching virtual-to-physical address translations for use by one or more accelerator cores 2830 during command execution. As described above, the virtual memory space is distributed between accelerator memory 2850 and host memory 2860. Similarly, the PCIe logic on host processor 2802 includes an I / O memory management unit (IOMMU) 2810 for managing PCIe I / O devices 2806 and memory accesses of accelerator 2801 in one embodiment. As shown in the PCIe logic 2820 on the accelerator and the PCIe logic 2808 on the main processor, they communicate using the PCDI protocol to perform functions such as device discovery, register access, device configuration and initialization, interrupt handling, DMA operations, and address translation service (ATS). As mentioned above, the IOMMU 2810 on the main processor 2802 can operate as the central point for the control and coordination of these functions.

[0379] In one embodiment, accelerator core 2830 includes a processing engine (element) that performs the functions required by the accelerator. Additionally, accelerator core 2830 may include a host memory cache 2834 for locally caching pages stored in host memory 2860 and an accelerator memory cache 2832 for caching pages stored in accelerator memory 2850. In one embodiment, accelerator core 2830 communicates with coherence and caching logic 2807 of host processor 2802 via a CAC protocol to ensure consistency of cache lines shared between accelerator 2801 and host processor 2802.

[0380] The offset / consistency logic 2840 of accelerator 2801 implements various device / host offset techniques described herein (e.g., at the page level) to ensure data consistency while reducing unnecessary communication on the multiprotocol link 2800. As shown, the offset / consistency logic 2840 communicates with the consistency and cache logic 2807 of host processor 2802 using MA memory transactions (e.g., SMI3+). The consistency and cache logic 2807 is responsible for maintaining the consistency of data stored in its LLC 2809, host memory 2860, accelerator memory 2850, and caches 2832, 2834, as well as the various caches of core 2805.

[0381] In summary, one implementation of accelerator 2801 presents itself as a PCIe device to software executing on host processor 2802, accessed via the PDCI protocol (effectively a PCIe protocol reformatted for multiplexed buses). Accelerator 2801 can participate in shared virtual memory using accelerator device TLB and standard PCIe Address Translation Service (ATS). The accelerator can also be viewed as a coherence / memory agent. Certain capabilities (e.g., ENQCMD, MOVDIR described below) are available on PDCI (e.g., for job submission), while the accelerator can use CAC to cache host data at the accelerator and in certain offset translation streams. Access to accelerator memory from the host (or host offset access from the accelerator) can be performed using the MA protocol described.

[0382] like Figure 29 As shown, in one embodiment, the accelerator includes a PCI configuration register 2902 and an MMIO register 2906, which can be programmed to provide access to device backend resources 2905. In one embodiment, the base address of the MMIO register 2906 is specified by a set of base address registers (BARs) 2901 in the PCI configuration space. Unlike previous embodiments, one embodiment of the data flow accelerator (DSA) described herein does not implement multiple channels or PCI functionality, thus each register has only one instance in the device. However, there can be more than one DSA device in a single platform.

[0383] Implementations may provide additional performance or debug registers not described herein. Any such registers should be considered implementation-specific.

[0384] PCI configuration space access is performed as aligned 1, 2, or 4-byte accesses. For rules regarding access to unimplemented registers and reserved bits in the PCI configuration space, see the PCI Express Basic Specification.

[0385] MMIO space accesses to the BAR0 region (capabilities, configuration, and status registers) are performed as aligned 1, 2, 4, or 8-byte accesses. 8-byte accesses should only be used for 8-byte registers. Software should not read from or write to unimplemented registers. MMIO space accesses to the BMI 2 and BAR4 regions should be performed as 64-byte accesses using the ENQCMD, ENQCMDS, or MOVDIR64B instructions (described in detail below). ENQCMD or ENQCMDS should be used to access work queues configured as shared (SWQ), and MOVDIR64B must be used to access work queues configured as dedicated (DWQ).

[0386] One implementation of the DSA PCI configuration space uses three 64-bit BARs 2901. The Device Control Register (BAR0) is a 64-bit BAR that contains the physical base address of the Device Control Register. These registers provide information about device capabilities, configuration and enabling of the device, and device status. The size of the BAR0 region depends on the size of the interrupt message store 2904. The size is 32KB plus the number of interrupt message store entries 2904 multiplied by 16, rounded up to the next power of 2. For example, if the device supports 1024 interrupt message store entries 2904, the interrupt message store is 16KB, and the size of BAR0 is 64KB.

[0387] BAR2 is a 64-bit BAR that contains the physical base addresses of privileged and non-privileged entries. Each entry is 64 bytes in size and resides on a separate 4KB page. This allows entries to be mapped independently to different address spaces using CPU page tables. Entries are used to submit descriptors to the device. Privileged entries are used by kernel-mode software, and non-privileged entries are used by user-mode software. The number of non-privileged entries is the same as the number of supported work queues. The number of privileged entries is the number of work queues (WQ) × (MSI-X-table-size - 1). The address of the entry used to submit a descriptor allows the device to determine which work queue (WQ) to put the descriptor into, whether the entry is privileged or non-privileged, and which MSI-X table entry can be used to complete the interrupt. For example, if the device supports 8 WQs, the WQ for a given descriptor is (entry address >> 12) & 0x7. If entry address >> 15 is 0, the entry is non-privileged; otherwise, it is privileged, and the MSI-X 2903 table index used to complete the interrupt is entry address >> 15. Bit 5:0 must be 0. Ignore bits 11:6; therefore, any 64-byte aligned address on the page can be used with the same effect.

[0388] Descriptor commits using non-privileged entries are subject to the WQ occupancy threshold, such as when configured using the Work Queue Configuration (WQCFG) register. Descriptor commits using privileged entries are not subject to this threshold. Descriptor commits to SWQs must be made using ENQCMD or ENQCMDS. Any additional write operations to SWQ entries are ignored. Descriptor commits to DWQs must be made using a 64-byte write operation. The software uses MOVDIR64B to guarantee lossless 64-byte writes. ENQCMD or ENQCMDS to a disabled or dedicated WQ entry will return a retry. Any additional write operations to DWQ entries are ignored. Any read operation to the BAR2 address space returns all 1s. Kernel-mode descriptors should be committed using privileged entries to receive completion interrupts. If a kernel-mode descriptor is committed using a non-privileged entry, a completion interrupt cannot be requested. User-mode descriptors can be committed using either privileged or non-privileged entries.

[0389] The number of entries in the BAR2 area is the number of WQs supported by the device multiplied by the MSI-X 2903 table size. The MSI-X table size is typically the number of WQs plus one. Therefore, for example, if the device supports 8 WQs, the usable size of BAR2 will be 8 × 9 × 4 KB = 288 KB. The total size of BAR2 will be rounded up to the next power of 2, which is 512 KB.

[0390] BAR4 is a 64-bit BAR that contains the physical base address of the visitor entry. Each visitor entry is 64 bytes in size and resides in a separate 4KB page. This allows entries to be mapped independently to different address spaces using the CPU Extended Page Table (EPT). This BAR is not implemented if the interrupt message storage support field in GENCAP is 0.

[0391] Guest kernel-mode software can use guest entries to submit descriptors to the device. The number of guest entries is the number of entries in the interrupt message store multiplied by the number of supported WQs. The address of the guest entry used to submit the descriptor allows the device to determine the descriptor's WQ and the interrupt message store entry used to generate a completion interrupt for the descriptor (if it is a kernel-mode descriptor and if the request completion interrupt flag is set in the descriptor). For example, if the device supports 8 WQs, the WQ of a given descriptor is (guest entry address >> 12) & 0x7, and the interrupt table entry index for the completion interrupt is guest entry address >> 15.

[0392] In one implementation, MSI-X is the only PCIe interrupt capability provided by the DSA, and the DSA does not implement traditional PCI interrupts or MSI. Details of this register structure are in the PCI Express specification.

[0393] In one implementation, three PCI Express capabilities control address translation. Only certain combinations of values ​​for these capabilities are supported, as shown in Table A. These values ​​are checked when the enable bit in the General Purpose Control Register (GENCTRL) is set to 1.

[0394]

[0395]

[0396] Table A

[0397] If any of these capabilities are changed by software when the device is enabled, the device can be paused and an error will be reported in the software error register.

[0398] In one implementation, the software configures the PASID capability to control whether the device uses PASID to perform address translation. If PASID is disabled, only physical addresses can be used. If PASID is enabled, virtual or physical addresses can be used, depending on the IOMMU configuration. If PASID is enabled, the Address Translation Service (ATS) and Page Request Service (PRS) should also be enabled.

[0399] In one implementation, the software configures the ATS capability to control whether the device should translate addresses before performing memory accesses. If address translation is enabled in the IOMMU 2810, ATS must be enabled in the device to achieve acceptable system performance. If address translation is not enabled in the IOMMU 2810, ATS must be disabled. If ATS is disabled, only physical addresses can be used, and all memory accesses are performed using untranslated accesses. If PASID is enabled, ATS must be enabled.

[0400] In one implementation, the software configures the PRS capability to control whether the device can request a page in the event of an address translation failure. If PASID is enabled, PRS must be enabled; if PASID is disabled, PRS must be disabled.

[0401] Some implementations utilize a virtual memory space seamlessly shared among one or more processor cores, accelerator devices, and / or other types of processing devices (e.g., I / O devices). In particular, one implementation utilizes a shared virtual memory (SVM) architecture, where the same virtual memory space is shared among cores, accelerator devices, and / or other processing devices. Additionally, some implementations include heterogeneous forms of physical system memory addressed using a common virtual memory space. Heterogeneous forms of physical system memory can use different physical interfaces to connect to the DSA architecture. For example, accelerator devices can be directly coupled to local accelerator memory, such as high-bandwidth memory (HBM), and each core can be directly coupled to host physical memory, such as dynamic random-access memory (DRAM). In this example, the shared virtual memory (SVM) is mapped to a combined physical memory of HBM and DRAM so that accelerators, processor cores, and / or other processing devices can access HBM and DRAM using a consistent set of virtual memory addresses.

[0402] These and other accelerator features are described in detail below. In brief, different implementations may include one or more of the following infrastructure features:

[0403] Shared Virtual Memory (SVM)Some implementations support SVM, which allows user-level applications to submit commands to the DSA directly using virtual addresses in the descriptor. The DSA can support the translation of virtual addresses to physical addresses using an Input / Output Memory Management Unit (IOMMU) that includes page fault handling. The range of virtual addresses referenced by the descriptor can span multiple pages across multiple heterogeneous memory types. Alternatively, one implementation also supports the use of physical addresses, provided that the data buffer is contiguous in physical memory.

[0404] Partial descriptor completion With SVM support, operations may encounter page faults during address translation. In some cases, the device can terminate processing of the corresponding descriptor at the point where the error occurs and provide the software with a completion log indicating partial completion and fault information, allowing the software to take remedial action and retry the operation after resolving the error.

[0405] Batch processing Some implementations support submitting descriptors in "batches". A batch descriptor points to a nearly contiguous set of working descriptors (i.e., descriptors containing the actual data operations). When processing batch descriptors, DSA retrieves working descriptors from designated memory and processes them.

[0406] Stateless devices In one implementation, the descriptor is designed such that all the information required to process the descriptor comes from the descriptor payload itself. This allows the device to store very little client-specific state, thereby improving its scalability. One exception is the completion interrupt message, which is configured by trusted software at the time of use.

[0407] Cache allocation control This allows applications to specify whether to write to the cache or bypass the cache and write directly to memory. In one implementation, completion records are always written to the cache.

[0408] Shared Work Queues (SWQ) support As described in detail below, some implementations use queuing commands (ENQCMD) and queuing commands (ENQCMDS) to support scalable job submission via a shared work queue (SWQ). In this implementation, the SWQ is shared by multiple applications.

[0409] Dedicated Work Queues (DWQ) support In some implementations, high-throughput job submissions via a dedicated work queue (DWQ) using the MOVDIR64B instruction are supported. In this implementation, the DWQ is dedicated to a specific application.

[0410] QoS supportSome implementations allow specifying a Quality of Service (QoS) level for each work queue (e.g., specified by the kernel driver). It can then assign different work queues to different applications, allowing work from different applications to be dispatched from work queues with different priorities. Work queues can be programmed to use specific channels for structured QoS.

[0411] Offset cache consistency mechanism

[0412] One implementation improves the performance of accelerators with directly connected memory (e.g., stacked DRAM or HBM) and simplifies application development for applications utilizing accelerators with directly connected memory. This implementation allows the memory attached to the accelerator to be mapped as part of the system memory and accessed using shared virtual memory (SVM) techniques (e.g., those used in current IOMMU implementations) without encountering the typical performance drawbacks associated with full system cache coherency.

[0413] The ability to access accelerator-attached memory as part of system memory without incurring heavy cache coherence overhead provides a beneficial operating environment for accelerator offloading. This ability to access memory as part of the system address map allows host software to set operands and access computation results without the overhead of traditional I / O DMA data copying. This traditional copying involves driver calls, interrupts, and memory-mapped I / O (MMIO) accesses, all of which are inefficient compared to simple memory access. Simultaneously, the ability to access accelerator-attached memory without cache coherence overhead can be critical to the execution time of offloading computations. For example, in scenarios with heavy streaming writes to memory, cache coherence overhead can halve the effective write bandwidth seen by the accelerator. The efficiency of operand setting, the efficiency of result access, and the efficiency of accelerator computation all play a role in determining the effectiveness of accelerator offloading. If the cost of offloading the work (e.g., setting operands; obtaining results) is too high, offloading may not be worthwhile at all, or it may limit the accelerator to very large tasks. The efficiency of accelerator computation execution can have the same effect.

[0414] One implementation applies different memory access and coherence techniques based on the entity initiating the memory access (e.g., accelerator, core, etc.) and the accessed memory (e.g., host memory or accelerator memory). These techniques, often referred to as a "coherence offset" mechanism, provide two sets of cache coherence streams for the memory attached to the accelerator: one optimized for efficient accelerator access to its attached memory, and the second optimized for host access to the accelerator-attached memory and shared accelerator / host access to the accelerator-attached memory. Furthermore, it includes two techniques for switching between these streams: one driven by application software and the other driven by autonomous hardware hints. In both sets of coherence streams, the hardware maintains full cache coherence.

[0415] like Figure 30 As generally shown, one embodiment is applicable to a computer system including an accelerator 3001 and one or more computer processor chips having a processor core and I / O circuitry 3003, wherein the accelerator 3001 is coupled to the processor via a multi-protocol link 3010. In one embodiment, the multi-protocol link 3010 is a dynamically multiplexed link supporting multiple different protocols, including but not limited to the protocols detailed above. However, it should be noted that the basic principles of the invention are not limited to any particular set of protocols. Additionally, it should be noted that, depending on the embodiment, the accelerator 3001 and the core I / O 3003 may be integrated on the same semiconductor chip or on different semiconductor chips.

[0416] In the illustrated embodiment, accelerator memory bus 3012 couples accelerator 3001 to accelerator memory 3005, and a separate host memory bus 3011 couples core I / O 3003 to host memory 3007. As described above, accelerator memory 3005 may include high-bandwidth memory (HBM) or stacked DRAM (some examples of which are described herein), and host memory 3007 may include DRAM, such as double data rate synchronous dynamic random access memory (e.g., DDR3 SDRAM, DDR4 SDRAM, etc.). However, the basic principles of the invention are not limited to any particular type of memory or memory protocol.

[0417] In one implementation, both the accelerator 3001 and the "host" software running on the processing core within the processor chip 3003 use two distinct sets of protocol streams (referred to as the "host offset" stream and the "device offset" stream) to access the accelerator memory 3005. As described below, one implementation supports multiple options for modulating and / or selecting the protocol stream for a particular memory access.

[0418] The consistency offset flow is implemented in part on two protocol layers on the multiprotocol link 3010 between accelerator 3001 and a processor chip 3003: the CAC protocol layer and the MA protocol layer. In one implementation, the consistency offset flow is enabled by: (a) using existing opcodes in the CAC protocol in a new manner, (b) adding new opcodes to the existing MA standard, and (c) adding support for the MA protocol to the multiprotocol link 3001 (previously, the link only included CAC and PCDI). Note that the multiprotocol link is not limited to supporting CAC and MA; in one implementation, only support for at least those protocols is required.

[0419] As used in this article, such as Figure 30 As shown, the "host offset" flow is a collection of requests, including those from the accelerator itself, that are aggregated into the accelerator memory 3005 by the standard coherence controller 3009 in the processor chip 3003 to which the accelerator 3001 is attached. This allows the accelerator 3001 to take a roundabout path to access its own memory, but allows access from both the accelerator 3001 and the processor core I / O 3003 to be kept consistent using the processor's standard coherence controller 3009. In one implementation, the flow makes requests to the processor's coherence controller 3009 via a multiprotocol link using CAC opcodes, in the same or similar manner as the processor core 3009 makes requests to the coherence controller 3009. For example, the processor chip's coherence controller 3009 may issue UPI and CAC coherence messages (e.g., snooping) generated by requests from the accelerator 3001 to all peer processor core chips (e.g., 3003) and internal processor agents representing the accelerator, just as they would to requests from the processor core 3003. In this way, consistency is maintained between the data accessed by the accelerator 3001 and the processor core I / O 3003.

[0420] In one implementation, the consistency controller 3009 also conditionally sends memory access messages to the accelerator's memory controller 3006 via the multiprotocol link 3010. These messages are similar to those sent by the consistency controller 3009 to its processor die-local memory controller and include new opcodes that allow data to be returned directly to the agent inside the accelerator 3001, instead of forcing the data to be returned to the processor's consistency controller 3009 via the multiprotocol link 3010, and then returned to the accelerator 3001 as a CAC response via the multiprotocol link 3010.

[0421] exist Figure 30In one implementation of the “host offset” mode shown, all requests from processor core 3003 to accelerator-attached memory 3005 are sent directly to processor coherence controller 3009, just as they would to normal host memory 3007. Coherence controller 3009 can apply its standard cache coherence algorithm and send its standard cache coherence messages, just as it would for accesses from accelerator 3001, and just as it would for accesses to normal host memory 3007. Coherence controller 3009 also conditionally sends MA commands via multiprotocol link 3010 for such requests, although in this case, the MA circulates through multiprotocol link 3010 to return data.

[0422] like Figure 31 As shown, "device offset" flows are flows that allow accelerator 3001 to access its locally attached memory 3005 without consulting the cache coherence controller 3007 of the main processor. Specifically, these flows allow accelerator 3001 to access its locally attached memory via memory controller 3006 without sending requests via multiprotocol link 3010.

[0423] In "Device Offset" mode, requests from processor core I / O 3003 are issued according to the description of "Host Offset" above, but are completed differently in the MA portion of their streams. When in "Device Offset" mode, processor requests to the memory 3005 to which the accelerator is attached are completed as if they were issued as "Uncached" requests. This "Uncached" convention ensures that data subject to device offset streams is never cached in the processor's cache hierarchy. It is this fact that allows the accelerator 3001 to access device offset data in its memory 3005 without consulting the cache coherence controller 3009 on the processor.

[0424] In one implementation, support for "unbuffered" processor core 3003 access flows is achieved using a global observe, use-once ("GO-UO") response on the processor's CAC bus. This response returns a data statement to processor core 3003, instructing the processor to use the data's value only once. This prevents data caching and satisfies the requirements of the "unbuffered" flow. In systems with cores that do not support GO-UO responses, the "unbuffered" flow can be implemented using a multi-message response sequence on the MA layer of the multiprotocol link 3010 and on the processor core 3003 CAC bus.

[0425] Specifically, when a processor core is found to be targeting a "device offset" page at accelerator 3001, the accelerator sets a state to block future requests from the accelerator to the target cache line and sends a specific "device offset hit" response at the MA layer of multiprotocol link 3010. In response to this MA message, the processor's cache coherence controller 3009 returns data to the requesting processor core 3003 and immediately follows up with a listen invalidation message. When processor core 3003 confirms the listen invalidation is complete, cache coherence controller 3009 sends another specific MA "device offset block complete" message back to accelerator 3001 at the MA layer of multiprotocol link 3010. This completion message causes accelerator 3001 to clear the aforementioned blocking state.

[0426] Figure 107 An embodiment using offsets is illustrated. In one implementation, the selection between device and host offset streams is driven by an offset tracker data structure, which can be maintained as an offset table 10707 in accelerator memory 3005. This offset table 10707 can be a page-granular structure (i.e., controlled at the memory page level), comprising 1 or 2 bits for each memory page attached to the accelerator. Offset table 10707 can be implemented within the stolen memory range of the accelerator-attached memory 3005, with or without an offset cache 10703 in the accelerator (e.g., for caching frequently / recently used entries of offset table 10707). Alternatively, the entire offset table 10707 can be maintained within accelerator 3001.

[0427] In one implementation, the offset table entry associated with each access to the memory 3005 attached to the accelerator is accessed before the actual access to the accelerator memory, resulting in the following operation:

[0428] Local requests from accelerator 3001 to locate their pages in the device offset are directly forwarded to accelerator memory 3005.

[0429] • Local requests from accelerator 3001 to find their pages in the host offset are forwarded as CAC requests to processor 3003 via multiprotocol link 3010.

[0430] • MA requests from processor 3003 that locate their pages in the device offset complete the request using the aforementioned "uncached" stream.

[0431] • MA requests from processor 3003 to locate their pages in the host offset are completed as normal memory reads.

[0432] The offset state of a page can be changed through the following mechanisms: software-based mechanisms, hardware-assisted software-based mechanisms, or purely hardware-based mechanisms for a finite set of cases.

[0433] One mechanism for changing the offset state involves API calls (e.g., OpenCL), which in turn call the accelerator's device driver. This driver then sends a message to the accelerator 3001 (or queues a command descriptor), thus guiding it to change the offset state. For some transitions, a cache refresh operation is performed on the host. A cache refresh operation is required for transitions from host offset to device offset, but not for the reverse transition.

[0434] In some cases, software may struggle to determine when to make an offset translation API call and identify the page requiring an offset translation. In such situations, the accelerator can implement an offset translation "hint" mechanism, where it detects the need for an offset translation and sends a message to its driver instructing it to perform the translation. This hint mechanism might be as simple as a mechanism for responding to offset table lookups, which is triggered when the accelerator accesses a host offset page or the host accesses a device offset page, and signals the event to the accelerator driver via an interrupt.

[0435] Note that some implementations may require a second offset status bit to enable the offset transition status value. This allows the system to continue accessing memory pages while these pages are in the process of offset changing (i.e., when the cache is partially flushed, incremental cache pollution due to subsequent requests must be suppressed).

[0436] Figure 32 An exemplary process according to one implementation is illustrated herein. This process can be implemented on the system and processor architecture described herein, but is not limited to any particular system or processor architecture.

[0437] At 3201, a specific set of pages is placed in the device offset. As described above, this can be achieved by updating the entries for these pages in the offset table to indicate that these pages are in the device offset (e.g., by setting the bit associated with each page). In one implementation, once set to the device offset, it is guaranteed that the pages are not cached in the host cache memory. At 3202, pages are allocated from device memory (e.g., software allocates pages by initiating driver / API calls).

[0438] At 3203, the operand is pushed from the processor core to the allocated page. In a single implantation, this is done in software using API calls to flip the operand page to the host offset (e.g., via OpenCL API calls). No data copying or cache flushing is required, and the operand data may end at any arbitrary point in the host cache hierarchy at this stage.

[0439] At 3204, the accelerator device uses operands to generate results. For example, it can execute commands and process data directly from its local memory (e.g., 3005 discussed above). In one implementation, the software uses the OpenCL API to flip operand pages back to the device offset (e.g., update the offset table). As a result of the API call, a work descriptor is submitted to the device (e.g., via a shared dedicated work queue as described below). The work descriptor can instruct the device to flush operand pages from the host cache, resulting in a cache flush (e.g., performed using CLFLUSH over the CAC protocol). In one implementation, the accelerator executes and dumps data to the result page without host-related consistency overhead.

[0440] At 3205, the result is retrieved from the allocated page. For example, in one implementation, the software makes one or more API calls (e.g., via the OpenCL API) to flip the result page to the host offset. This operation may cause some offset state changes, but does not result in any consistency or cache refresh operations. The host processor core can then access, cache, and share the result data as needed. Finally, at 3206, the allocated page is released (e.g., via software).

[0441] Figure 33 A similar process for releasing operands from one or more I / O devices is illustrated. At 3301, a specific set of pages is placed in a device offset. As described above, this can be done by updating the entries for these pages in the offset table to indicate that these pages are in a device offset (e.g., by setting the bit associated with each page). In one implementation, once set to a device offset, it is guaranteed that the pages are not cached in the host cache memory. At 3302, pages are allocated from device memory (e.g., software allocates pages by initiating a driver / API call).

[0442] At 3303, the operand is pushed from the I / O agent to the allocated page. In a single implantation, this is accomplished by the software making a DMA request to the I / O agent and the I / O agent writing data using non-allocated storage. In one implementation, the data is never allocated to the host cache hierarchy, and the target page remains at the device offset.

[0443] At 3304, the accelerator device uses operands to generate results. For example, software can submit work to the accelerator device; no page translation is required (i.e., pages remain in the device offset). In one implementation, the accelerator device executes without host-related consistency overhead, and the accelerator dumps data to the resulting page.

[0444] At 3305, the I / O agent retrieves the result from the allocated page (e.g., under software guidance). For example, the software might make a DMA request to the I / O agent. Since the source page is retained in the device offset, page translation is not required. In one implementation, the I / O bridge uses an RdCurr (Read Current) request to fetch a non-cached copy of the data from the result page.

[0445] In some implementations, the work queue (WQ) holds software-submitted "descriptors," serves as an arbitrator for implementing Quality of Service (QoS) and fairness policies, a processing engine for handling descriptors, address translation and caching interfaces, and a memory read / write interface. A descriptor defines the scope of work to be performed. For example... Figure 34 As shown, in one implementation, there are two different types of work queues: a dedicated work queue 3400 and a shared work queue 3401. The dedicated work queue 3400 stores descriptors for a single application 3413, while the shared work queue 3401 stores descriptors submitted by multiple applications 3410-3412. A hardware interface / arbitrator 3402 dispatches descriptors from work queues 3400-3401 to the accelerator processing engine 3405 according to a specified arbitration policy (e.g., based on the processing requirements of each application 3410-3413 and a QoS / fairness policy).

[0446] Figure 108A -B indicates the memory-mapped I / O (MMIO) space registers used with work queue-based implementations. Version register 10807 reports the version of this architecture specification supported by the device.

[0447] The General Capability Register (GENCAP) 10808 specifies the general capabilities of the device, such as maximum transfer size, maximum batch size, etc. Table B lists the various parameters and values ​​that can be specified in the GENCAP register.

[0448]

[0449]

[0450] Table B

[0451] In one implementation, the Work Queue Capability Register (WQCAP) 10810 specifies the capabilities of the work queues, such as support for dedicated and / or shared operating modes, the number of engines, and the number of work queues. Table C below lists various configurable parameters and values.

[0452]

[0453]

[0454] Table C

[0455] In one implementation, the Operation Capability Register (OPCAP) 10811 is a bitmask used to specify the types of operations supported by the device. Each bit corresponds to an operation type with the same code as the bit position. For example, bit 0 of the register corresponds to no operation (code 0). If the operation is supported, the bit is set; if the operation is not supported, the bit is cleared.

[0456]

[0457] Table D

[0458] In one implementation, the general configuration register (GENCFG) 10812 specifies the virtual channel (VC) boot label. See Table E below.

[0459]

[0460]

[0461] Table E

[0462] In one implementation, the general control register (GENCTRL) 10813 indicates whether an interrupt is generated in response to a hardware or software error. See Table F below.

[0463]

[0464] Table F

[0465] In one implementation, the device enable register (ENABLE) stores error codes, an indicator of whether the device is enabled, and a device reset value. See Table G below for more details.

[0466]

[0467]

[0468] Table G

[0469] In one implementation, the interrupt cause register (INTCAUSE) stores a value indicating the interrupt cause. See Table H below.

[0470]

[0471]

[0472] Table H

[0473] In one implementation, the command register (CMD) 10814 is used to submit discharge WQ, discharge PASID, and discharge All commands. The abort field indicates whether the requested operation is discharge or abort. Before writing to this register, the software can ensure that any commands previously submitted through this register have been completed. Before writing to this register, if a completion record is requested, the software can configure the command configuration register and the command completion record address register.

[0474] The `Expel All` command exits or aborts all incomplete descriptors in all WQs and all engines. The `Expel PASID` command exits or aborts descriptors using the specified PASID in all WQs and all engines. The `Expel WQ` command exits or aborts all descriptors in the specified WQ. Depending on the implementation, any `Expel` command can wait for the completion of other descriptors, except for those that need to be waited for.

[0475] If the abort field is 1, the software requests that the affected descriptor be abandoned. However, the hardware may still complete part or all of it. If a descriptor is abandoned, no completion record is written, and no completion interrupt is generated for that descriptor. Some or all other memory accesses may occur.

[0476] The completion of a command is indicated by generating a completion interrupt (if requested) and by clearing the status field of the register. When completion is signaled, all affected descriptors have been completed or abandoned, and no further address translations, memory reads, memory writes, or interrupts will be generated due to any affected descriptors. See Table I below.

[0477]

[0478]

[0479] Table I

[0480] In one implementation, the software error status register (SWERROR) 10815 stores various types of errors, such as: errors when committing a descriptor; errors when translating the completion record address in a descriptor; errors when verifying a descriptor if the completion record address validity flag in the descriptor is 0; and errors when processing a descriptor (e.g., page faults) if the completion record address validity flag in the descriptor is 0. See Table J below.

[0481]

[0482]

[0483]

[0484] Table J

[0485] In one implementation, the hardware error status register (HWERROR) 10816 is configured in a manner similar to that of the software error status register (see above).

[0486] In one implementation, the group configuration register (GRPCFG) 10817 stores configuration data for each work queue / engine group (see [link]). Figures 36-37 Specifically, the group configuration table is a register array in BAR0 that controls the mapping of work queues to engines. There are the same number of groups as engines, but the software can configure the desired number of groups. Each active group contains one or more work queues and one or more engines. Any unused group must have both the WQs and Engines fields equal to 0. Descriptors submitted to any WQ in a group can be processed by any engine in that group. Each active work queue must be in a single group. An active work queue is the work queue whose corresponding WQ size field in the WQCFG register has a non-zero value. Any engine not in a group is inactive.

[0487] Each GRPCFG register 10817 can be divided into three sub-registers, each of which is one or more 32-bit words (see Table KM). These registers can be read-only when the device is enabled. They are also read-only if the WQCAP work queue configuration support field is 0.

[0488] In one implementation, for each group G, 0 ≤ G < number of engines, the offsets of the sub-registers in BAR0 are as follows:

[0489]

[0490] Table K

[0491]

[0492] Table L

[0493]

[0494] Table M

[0495] In one implementation, the Work Queue Configuration Register (WQCFG) 10818 stores data specifying the operations for each work queue. The WQ configuration table is an array of 16-byte registers in BAR0. The number of WQ configuration registers matches the number of WQ fields in WQCAP.

[0496] Each 16-byte WQCFG register is divided into four 32-bit sub-registers, which can also be read or written using aligned 64-bit read or write operations.

[0497] Each WQCFG-A sub-register is read-only when the device is enabled or the WQCAP work queue configuration supports a field of 0.

[0498] Unless the WQCAP work queue configuration supports a field of 0, each WQCFG-B can be written to at any time. If the WQ threshold field contains a value larger than the WQ size when WQ is enabled, WQ is not enabled and the WQ error code is set to 4. If a value larger than the WQ size is written to the WQ threshold field when WQ is enabled, WQ is disabled and the WQ error code is set to 4.

[0499] Each WQCFG-C sub-register is read-only when WQ is enabled. It can be written to before or at the same time as setting WQ enable to 1. If the WQCAP work queue configuration support field is 0, the following fields are always read-only: WQ mode, WQ error blocking enabled, and WQ priority. Even if the WQCAP work queue configuration support field is 0, the following fields of WQCFG-C are writable when WQ is not enabled: WQ PASID and WQ U / S.

[0500] Each WQCFG-D sub-register is writable at all times. However, setting WQ enable to 1 when the device is not enabled will result in an error.

[0501] When WQ Enable is set to 1, both the WQ Enabled and WQ Error Code fields are cleared. Subsequently, either WQ Enabled or the WQ Error Code will be set to a non-zero value to indicate whether WQ was successfully enabled.

[0502] The sum of the WQ size fields of all WQCFG registers must not exceed the total WQ size field in GENCAP. This constraint is checked when enabling the device. WQs with a WQ size field of 0 cannot be enabled, and all other fields of these WQCFG registers are ignored. The WQ size field is read-only when enabling the device. See Table N for data for each sub-register.

[0503]

[0504]

[0505]

[0506] Table N

[0507] In one implementation, the work queue occupancy interrupt control register 10819 (one for each work queue (WQ)) allows software to request an interrupt when the work queue occupancy drops to a specified threshold. When the WQ occupancy interrupt is enabled to 1 and the current WQ occupancy is equal to or less than the WQ occupancy limit, the following operations can be performed:

[0508] 1. Clear the WQ occupancy interruption enable field.

[0509] 2. Set the 3rd bit of the interrupt reason register to 1.

[0510] 3. If the third bit of the interrupt cause register is 0 before step 2, then an interrupt is generated using MSI-X table entry 0.

[0511] 4. If a register is written to enable = 1 and the limit is greater than or equal to the current WQ occupancy rate, an interrupt is generated immediately. Therefore, if a register is written to enable = 1 and the limit is greater than or equal to the WQ size, an interrupt is always generated immediately.

[0512]

[0513] Table O

[0514] In one implementation, a work queue status register (one per WQ) 10820 specifies the number of entries currently in each WQ. This number can change whenever a descriptor is submitted to or dispatched from the queue, so it cannot be relied upon to determine whether there is space in the WQ.

[0515] In one implementation, MSI-X entry 10821 stores MSI-X table data. The offset and entry number are within the MSI-X capabilities. The recommended entry number is the WQ number plus 2.

[0516] In one implementation, the MSI-X pending bit array 10822 stores the offset and number of entries in the MSI-X capability.

[0517] In one implementation, interrupt message storage entry 10823 stores interrupt messages in a table structure. The table format is similar to that of an MSI-X table defined in PCIe, but its size is not limited to 2048 entries. However, the table size can vary between different DSA implementations and may have fewer than 2048 entries in some implementations. In one implementation, the number of entries is in the interrupt message storage size field of the general capability register. If the interrupt message storage support capability is 0, this table does not exist. For DSA to support a large number of virtual machines or containers, the supported table size needs to be large.

[0518] In one implementation, the format of each entry in the IMS is illustrated in Table P below:

[0519]

[0520] Table P

[0521] Figure 35 One embodiment of a Data Stream Accelerator (DSA) device is illustrated, comprising multiple work queues 3511-3512 that receive descriptors submitted via an I / O architecture interface 3501 (e.g., a multiprotocol link 2800 as described above). The DSA uses the I / O architecture interface 3501 to receive downstream work requests (e.g., processor cores, peer-to-peer input / output (I / O) agents (e.g., network interface controllers (NICs)) and / or software link offloading requests) from clients, as well as upstream read, write, and address translation operations. The illustrated embodiment includes an arbitrator 3513 that arbitrates between work queues and assigns work descriptors to one of multiple engines 3550. The operation of the arbitrator 3513 and work queues 3511-1012 can be configured via a work queue configuration register 3500. For example, the arbitrator 3513 can be configured to implement various QoS and / or fairness policies for assigning descriptors from each work queue 3511-1012 to each engine 3550.

[0522] In one implementation, some descriptors queued in work queues 3511-3512 are batch descriptors 3515, which contain / identify a batch of work descriptors. Arbitrator 3513 forwards the batch descriptors to batch processing unit 3516, which processes the batch descriptors by reading the descriptor array 3518 from memory using addresses translated by translation cache 3520 (or other address translation services possible on the processor). Once the physical address is identified, data read / write circuitry 3540 reads the batch descriptor from memory.

[0523] The second arbitrator 3519 arbitrates between the batch work descriptors 3518 provided by the batch processing unit 3516 and the individual work descriptors 3514 retrieved from the work queues 3511-3512, and outputs the work descriptors to the work descriptor processing unit 3530. In one embodiment, the work descriptor processing unit 3530 has the following stages: reading from memory (via data R / W unit 3540), performing the requested operation on the data, generating output data and writing output data (via data R / W unit 3540), completing the recording, and interrupting the message.

[0524] In one implementation, work queue configuration allows software to configure each WQ (via WQ configuration register 3500) as either a shared work queue (SWQ) that receives descriptors using non-published ENQCMD / S instructions or a dedicated work queue (DWQ) that receives descriptors using published MOVDIR64B instructions. (See above regarding...) Figure 34 The DWQ can handle both work descriptors and batch descriptors submitted from a single application, while the SWQ can be shared across multiple applications. The WQ configuration register 3500 also allows software control over which WQs 3511-3512 are fed to which accelerator engine 3550 and the relative priority of WQs 3511-3512 feeding to each engine. For example, an ordered set of priorities (e.g., high, medium, low; 1, 2, 3, etc.) can be specified, and descriptor dispatch from higher-priority work queues can typically occur before or more frequently than dispatch from lower-priority work queues. For example, for two work queues identified as high-priority and low-priority, for every 10 descriptors to be dispatched, 8 out of 10 descriptors can be dispatched from the high-priority work queue, while 2 out of 10 descriptors can be dispatched from the low-priority work queue. Various other techniques can be used to implement different priorities between work queues 3511-3512.

[0525] In one implementation, the Data Stream Accelerator (DSA) is software compatible with the PCI Express configuration mechanism and implements the PCI header and expansion space in its configuration mapping register set. The configuration registers can be programmed via CFC / CF8 or MMCFG in the root complex. All internal registers are also accessible via JTAG or SMBus interfaces.

[0526] In one implementation, the DSA device uses memory-mapped registers to control its operation. Capability, configuration, and work commit registers (entries) are accessed via the MMIO regions defined by registers BAR0, BAR2, and BAR4 (described below). Each entry can reside on a separate 4K page so that they can be independently mapped to different address spaces (clients) using processor page tables.

[0527] As mentioned above, the software specifies the operation of the DSA through descriptors. The descriptor specifies the type of operation the DSA will perform, the data address and status buffer, immediate operands, completion attributes, etc. (descriptor format details and specifications are described below). The completion attributes specify the address to write the completion record, as well as the information needed to generate an optional completion interrupt.

[0528] In one implementation, DSA avoids maintaining client-specific state on the device. All information about the processing descriptor resides within the descriptor itself. This improves its shareability between user-mode applications and between different virtual machines (or machine containers) within a virtualization system.

[0529] A descriptor can contain operations and associated parameters (called a working descriptor), or it can contain the addresses of an array of working descriptors (called a batch descriptor). The software prepares the descriptors in memory and submits them to the device's work queues (WQs) 3511-3512. Depending on the WQ mode and the client's privilege level, the descriptors are submitted to the device using the MOVDIR64B, ENQCMD, or ENQCMDS instructions.

[0530] Each WQ 3511-3512 has a fixed number of time slots, and therefore may become full under heavy load. In one implementation, the device provides the necessary feedback to help the software implement flow control. The device dispatches descriptors from work queues 3511-3512 and submits them to the engine for further processing. When the engine 3550 completes a descriptor or encounters some fault or error that leads to abort, it notifies the host software by writing a completion record to host memory, issuing an interrupt, or both.

[0531] In one implementation, each work queue is accessed via multiple registers, each residing in a separate 4KB page of the device's MMIO space. One work commit register for each WQ is called the "non-privileged entry" and is mapped to user space used by user-mode clients. Another work commit register is called the "privileged entry" and is used by kernel-mode drivers. The remainder are guest entries and are used by kernel-mode clients within the virtual machine.

[0532] As described above, each work queue 3511-3512 can be configured to operate in one of two modes, dedicated or shared. The DSA exposes capability bits in the work queue capability register to indicate support for dedicated and shared modes. It also exposes controls in the work queue configuration register 3500 to configure each WQ to operate in one of these modes. The mode of a WQ can only be changed when the WQ is disabled, i.e. (WQCFG.Enabled = 0). Further details of the WQ capability register and WQ configuration register are described below.

[0533] In one implementation, in shared mode, the DSA client submits a descriptor to a work queue using the ENQCMD or ENQCMDS instruction. ENQCMD and ENQCMDS use a 64-byte non-published write and wait for a response from the device before completion. If there is space in the work queue, DSA returns "success" (e.g., sent to the requesting client / application), or "retry" if the work queue is full. The ENQCMD and ENQCMDS instructions can return the status of the command submission in a zero flag (0 for success, 1 for retry). Using the ENQCMD and ENQCMDS instructions, multiple clients can submit descriptors directly and simultaneously to the same work queue. Because the device provides this feedback, the client can determine whether its descriptor has been accepted.

[0534] In shared mode, DSA can reserve some SWQ capacity for kernel-mode clients submitted via privileged entry points. It accepts work submissions via non-privileged entry points until the number of descriptors in the SWQ reaches the threshold configured for the SWQ. It also accepts work submissions via privileged entry points until the SWQ is full. Work submissions via guest entry points are subject to the threshold in the same way as those via non-privileged entry points.

[0535] If the ENQCMD or ENQCMDS instruction returns "success," the descriptor has been accepted by the device and queued for processing. If the instruction returns "retry," the software can attempt to resubmit the descriptor to the SWQ, or, if it is a user-mode client using a non-privileged entry, it can request the kernel-mode driver to submit the descriptor on its behalf using a privileged entry. This helps avoid denial-of-service and provides forward progress guarantees. Alternatively, if the SWQ is full, the software can use other methods (e.g., using the CPU to perform work).

[0536] Clients / applications are identified by the device using a 20-bit ID called the Process Address Space ID (PASID). The device uses the PASID to look up addresses in the device TLB 1722 and send address translations or page requests to the IOMMU 1710 (e.g., via a multiprotocol link 2800). In shared mode, the PASID used with each descriptor is contained in the descriptor's PASID field. In one implementation, ENQCMD copies the current thread's PASID from a specific register (e.g., PASIDMSR) to the descriptor, while ENQCMDS allows supervisor-mode software to copy the PASID to the descriptor.

[0537] In "Dedicated" mode, the DSA client can use the MOVDIR64B instruction to submit descriptors to the device work queue. MOVDIR64B uses 64-byte published writes, and due to the published nature of write operations, the instruction completes faster. For dedicated work queues, DSA can expose the total number of slots in the work queue and relies on software to provide flow control. The software is responsible for tracking the number of submitted and completed descriptors to detect the fullness of the work queue. If the software mistakenly submits a descriptor to the dedicated WQ when there is no space in the work queue, the descriptor is discarded and the error can be logged (e.g., in the software error register).

[0538] Because the MOVDIR64B instruction does not populate the PASID like the ENQCMD or ENQCMDS instructions, the PASID field in the descriptor cannot be used in dedicated mode. The DSA can ignore the PASID field in descriptors submitted to a dedicated work queue and use the WQ PASID field of the WQ configuration register 3500 instead for address translation. In one implementation, the WQPASID field is set by the DSA driver when the work queue is configured in dedicated mode.

[0539] Although in dedicated mode, a single DWQ is not shared by multiple clients / applications, a DSA device can be configured to have multiple DWQs, and each DWQ can be independently assigned to a client. Furthermore, DWQs can be configured to have the same or different QoS levels to provide different performance levels for different clients / applications.

[0540] In one implementation, the Data Streaming Accelerator (DSA) comprises two or more engines 3550, whose processing is submitted to descriptors in work queues 3511-1012. One implementation of the DSA architecture includes four engines, numbered 0 to 3. Engines 0 and 1 are each capable of utilizing the full bandwidth of the device (e.g., 30 GB / s for reads and 30 GB / s for writes). Of course, the combined bandwidth of all engines is also limited to the maximum bandwidth available to the device.

[0541] In one implementation, the software uses group configuration registers to configure WQs 3511-3512 and engines 3550 into groups. Each group contains one or more WQs and one or more engines. The DSA can use any engine in a group to process descriptors published to any WQ in that group, and each WQ and each engine can be in only one group. The number of groups can be the same as the number of engines, so each engine can be in a separate group, but if any group contains more than one engine, it is not necessary to use all groups.

[0542] While the DSA architecture allows for significant flexibility in configuring work queues, groups, and engines, the hardware can be designed specifically for a particular configuration. Engines 0 and 1 can be configured in one of two different ways, depending on the software requirements. A recommended configuration is to place engines 0 and 1 in the same group. The hardware uses either engine to process descriptors from any work queue in the group. In this configuration, if one engine stops due to high-latency memory address translation or a page fault, the other engine can continue operating, maximizing the overall throughput of the device.

[0543] Figure 36 The two work queues 3621-3622 and 3623-3624 in each group 3611 and 3612 are shown respectively, but any number up to the maximum supported number of WQs can exist. WQs in a group can be shared WQs with different priorities, or one shared WQ and other dedicated WQs, or multiple dedicated WQs with the same or different priorities. In the example shown, group 3611 is served by engines 0 and 1 (3601), and group 3612 is served by engines 2 and 3 (3602).

[0544] like Figure 37 As shown, another configuration using engines 0 3700 and 1 3701 is to place them in separate groups 3710 and 3711, respectively. Similarly, group 2 3712 is assigned to engine 2 3702, and group 3 is assigned to engine 3 3703. Furthermore, group 0 3710 includes two work queues 3721 and 3722; group 1 3711 includes work queue 3723; work queue 2 3712 includes work queue 3724; and group 3 3713 includes work queue 3725.

[0545] The software can select this configuration when it wants to reduce the likelihood that a latency-sensitive operation will be blocked after other operations. In this configuration, the software submits latency-sensitive operations to work queue 3723 connected to engine 1 3702 and submits other operations to work queues 3721-3722 connected to engine 0 3700.

[0546] For example, engines 2 (3702) and 3 (3703) can be used to write to high-bandwidth non-volatile memory such as phase-change memory. The bandwidth capabilities of these engines can be adjusted to match the expected write bandwidth of this type of memory. For this use, bits 2 and 3 of the engine configuration register should be set to 1, indicating that virtual channel 1 (VC1) should be used for traffic from these engines.

[0547] In platforms without high-bandwidth non-volatile memory (e.g., phase-change memory) or when the DSA device is not used for writing to this type of memory, engines 2 and 3 can be unused. However, if the submitted operations can tolerate limited bandwidth, the software can use them as additional low-latency paths.

[0548] When each descriptor arrives at the head of the work queue, it can be removed by the scheduler / arbitrator 3513 and forwarded to one of the engines in that group. For batch descriptors 3515, which refer to work descriptors 3518 in memory, the engine retrieves the array of work descriptors from memory (i.e., using batch processing unit 3516).

[0549] In one implementation, for each working descriptor 3514, engine 3550 prefetches the translation of the completion record address and passes the operation to working descriptor processing unit 3530. Working descriptor processing unit 3530 uses device TLB 1722 and IOMMU 1710 to perform source and destination address translation, reads source data, performs the specified operation, and writes the destination data back to memory. When the operation is complete, the engine writes the completion record to the pre-translated completion address and generates an interrupt when a working descriptor is requested.

[0550] In one implementation, multiple work queues of the DSA can be used to provide multi-tiered Quality of Service (QoS). The priority of each WQ can be specified in the WQ configuration register 3500. The priority of a WQ is relative to other WQs in the same group (e.g., the priority of a single WQ in the group is meaningless). Work queues in a group can have the same or different priorities. However, it is meaningless to configure multiple shared WQs with the same priority in the same group, as a single WQ can be used for the same purpose. The scheduler / arbitrator 3513 dispatches work descriptors from work queues 3511-3512 to engine 3550 according to the priority of the work queues.

[0551] Figure 38One embodiment of descriptor 1300 is shown, which includes an operation field 3801 for specifying the operation to be performed, multiple flags 3802, a process address space identifier (PASID) field 3803, a completion record address field 3804, a source address field 3805, a destination address field 3806, a completion interruption field 3807, a transfer size field 3808, and (possibly) one or more operation-specific fields 3809. In one embodiment, three flags are present: completion record address valid, request completion record, and request completion interruption.

[0552] Public fields include trusted and untrusted fields. DSA devices always trust trusted fields because they are populated by the CPU or privileged (ring 0 or VMM) software on the host. Untrusted fields are provided directly by the DSA client.

[0553] In one implementation, the trusted fields include a PASID field 3803, a reserved field 3811, and a U / S (User / Supervisor) field 3810 (i.e., 4 bytes starting at offset 0). These fields in the source descriptor can be ignored when submitting the descriptor using the ENQCMD instruction. Values ​​contained in the MSR (e.g., PASIDMSR) can be placed in these fields before sending the descriptor to the device.

[0554] In one implementation, when a descriptor is submitted using the ENQCMDS instruction, these fields in the source descriptor are initialized by software. If PCI Express PASID capability is not enabled, the U / S field 3810 is set to 1 and the PASID field 3803 is set to 0.

[0555] When submitting a descriptor using the MOVDIR64B instruction, these fields in the descriptor can be ignored. The device instead uses the WQ U / S and WQ PASID fields of the WQ configuration register 3500.

[0556] These fields can be ignored for any descriptor in the batch. The corresponding fields for batch descriptor 3515 are used for each descriptor 3518 in the batch. Table Q provides a description and bit position for each trusted field.

[0557]

[0558] Table Q (Descriptor Trust Field)

[0559] Table R below lists the operations performed according to the descriptor's operation field 3801 in one implementation.

[0560]

[0561]

[0562] Table R (Operation Type)

[0563] Table S below lists the flags used in one implementation of the descriptor.

[0564]

[0565]

[0566]

[0567] Table S (Mark)

[0568] In one implementation, completion record address 3804 specifies the address of the completion record. The completion record can be 32 bytes, and the completion record address is aligned to a 32-byte boundary. This field is reserved if the completion record address validity flag is 0. If the request completion record flag is 1, the completion record is written to this address when the operation completes. If the request completion record flag is 0, the completion record is only written to this address in the event of a page fault or error.

[0569] For any operation that produces a result (such as a comparison), the completion record address should be valid and the request completion record flag should be 1, and the completion record address should be valid.

[0570] For any operation using a virtual address, the completion record address should be valid, regardless of whether the request completion record flag is set, so that a completion record can be written in the event of a page fault or error.

[0571] For best results, this field should be valid in all descriptors, as it allows the device to report errors to the software that submitted the descriptor. If this flag is 0 and an unexpected error occurs, the error is reported to the SWERROR register, and the software that submitted the descriptor may not be notified of the error.

[0572] If the completion queue enable flag is set in the batch descriptor, the completion record address field 3804 will be ignored for descriptors in the batch; instead, the completion queue address in the batch descriptor will be used.

[0573] In one implementation, for operations reading data from memory, the source address field 3805 specifies the address of the source data. There are no alignment requirements for the source address. For operations writing data to memory, the destination address field 3806 specifies the address of the destination buffer. There are no alignment requirements for the destination address. For some operation types, this field is used as the address of a second source buffer.

[0574] In one implementation, the transfer size field 3808 indicates the number of bytes to be read from the source address to perform the operation. The maximum value of this field can be 2^32-1, but the maximum allowed transfer size can be smaller and must be determined from the maximum transfer size field of the general capability register. The transfer size should not be 0. For most operation types, there are no alignment requirements for the transfer size. Exceptions are noted in the operation instructions.

[0575] In one implementation, if the interrupt message storage flag is 1, the completion interrupt handle field 3807 specifies the interrupt message storage entry to be used to generate the completion interrupt. The value of this field should be less than the value of the interrupt message storage size field in GENCAP. In one implementation, the completion interrupt handle field 3807 is retained under any of the following conditions: the interrupt message storage flag is 0; the request completion interrupt flag is 0; the U / S bit is 0; the interrupt message storage support field of the general capability register is 0; or the descriptor is submitted via a guest entry.

[0576] like Figure 39 As shown, one implementation of the completion record 3900 is a 32-byte structure in memory, which the DSA writes to when the operation is complete or an error is encountered. The completion record address should be 32-byte aligned.

[0577] This section describes the fields of the completion record common to most operation types. If the format differs, the description for each operation type includes a completion record diagram. Additional operation-specific fields are further described below. Even if not all fields are needed, the completion record 3900 can always be 32 bytes. The completion record 3900 contains sufficient information to continue operation if partial completion occurs due to a page fault.

[0578] The completion record can be implemented as a 32-byte aligned structure in memory (identified by the completion record address 3804 of descriptor 3800). The completion record 3900 contains a completion status field 3904 to indicate whether the operation has been completed. If the operation was completed successfully, the completion record may contain the result of the operation (if any), depending on the operation type. If the operation was not completed successfully, the completion record contains fault or error information.

[0579] In one implementation, the status field 3904 reports the completion status of the descriptor. The software should initialize this field to 0 so that it can detect when a completion record is written.

[0580]

[0581] Table T (Completion Record Status Code)

[0582] Table T above provides various status codes and related descriptions for one implementation method.

[0583] Table U below shows error code 3903 available in one implementation, including a first bit indicating whether the error address was read or written and a second bit indicating whether the error access was in user mode or supervisor mode.

[0584]

[0585] Table U (Completion Record Error Code)

[0586] In one implementation, if the completion record 3900 is used for a descriptor submitted as part of a batch, the index field 3902 contains an index in the batch that generated the descriptor of the completion record. For batch descriptors, this field can be 0xff. For any other descriptor that is not part of a batch, this field can be retained.

[0587] In one implementation, if the operation was partially completed due to a page fault, the byte completion field 3901 contains the number of source bytes processed before the error occurred. This count indicates that all source bytes have been fully processed and the result has been written to the destination address, depending on the operation type. For some operation types, this field may also be used when the operation stops before completion for some reason other than an error. If the operation is fully completed, this field can be set to 0.

[0588] For operation types whose output size is not readily determined from this value, the completion record also includes the number of bytes written to the destination address.

[0589] If the operation was partially completed due to a page fault, this field contains the address that caused the error. As a general rule, all descriptors should have a valid completion record address (3804), and the completion record address validity flag should be 1. Some exceptions to this rule are described below.

[0590] In one implementation, the first byte of the completion record is a status byte. All status values ​​written by the device are non-zero. The software should initialize the status field of the completion record to 0 before committing the descriptor to be able to determine when the device has written to the completion record. Initializing the completion record also ensures that it is mapped so that the device does not encounter page faults when accessing the completion record.

[0591] The request completion record flag instructs the device that a completion record should be written even if the operation is successfully completed. If this flag is not set, the device will only write a completion record if an error occurs.

[0592] This can be accomplished using any of the following methods to detect descriptors via software:

[0593] 1. Polling completes the record, and the waiting status field becomes non-zero.

[0594] 2. Use the UMONITOR / UMWAIT command (as described in this article) on the completion record address to prevent it from being written until a timeout occurs. The software should then check if the status field is non-zero to determine if the operation has been completed.

[0595] 3. For kernel mode descriptors, request an interrupt when the operation is complete.

[0596] 4. If the descriptor is in a batch, set the guardrail flag in subsequent descriptors within the same batch. The completion of a descriptor with a guardrail, or any subsequent descriptor in the same batch, indicates the completion of all descriptors prior to the guardrail.

[0597] 5. If the descriptor is in a batch, the completion of the batch descriptor that starts the batch indicates the completion of all descriptors in the batch.

[0598] 6. Issue an eject descriptor or eject command and wait for it to complete.

[0599] If the completion status indicates partial completion due to a page fault, the completion record indicates how much processing (if any) was completed before the error was encountered, and the virtual address where the error occurred. The software can choose to fix the error (by accessing the faulty address from the processor) and resubmit the remainder of the work in a new descriptor, or complete the remaining work using the software. Errors at the descriptor list and completion record addresses are handled differently, as described in more detail below.

[0600] One implementation of DSA supports interrupts only for message signal notifications. DSA provides two types of interrupt message storage: (a) an MSI-X table, which stores interrupt messages used by the host driver through an MSI-X capability enumeration; and (b) a device-specific interrupt message storage (IMS) table, which stores interrupt messages used by the guest driver.

[0601] In one implementation, interrupts can be generated for three types of events: (1) completion of a kernel-mode descriptor; (2) completion of a discharge or abort instruction; and (3) errors published in a software or hardware error register. For each type of event, a separate interrupt is enabled. Interrupts resulting from errors and the completion of abort / discharge commands are generated using entry 0 in the MSI-X table. The cause of the interrupt can be determined by reading the interrupt cause register via software.

[0602] The interrupt message used for the completion of a kernel-mode descriptor (e.g., a descriptor with a U / S field of 1) depends on how the descriptor was committed and the interrupt message storage flag in the descriptor.

[0603] Completion interrupt messages for kernel-mode descriptors submitted via privileged entry points are typically entries in the MSI-X table, determined by the entry address. However, if the interrupt message storage support field in GENCAP is 1, this behavior can be overridden by setting the use of interrupt message storage flag in the descriptor. In this case, the completion interrupt handle field in the descriptor is used as an index to the interrupt message storage.

[0604] The completion interrupt message for a kernel mode descriptor submitted through a guest entry is an entry in the interrupt message store, determined by the entry address.

[0605] Interrupts generated by DSA are handled by interrupt remapping and release hardware configured by the kernel or VMM software.

[0606]

[0607] Table V

[0608] As mentioned above, DSA supports submitting multiple descriptors at once. A batch descriptor contains the address of an array of working descriptors in host memory, along with the number of elements in the array. This array of working descriptors is called a "batch." The use of batch descriptors allows DSA clients to submit multiple working descriptors using a single ENQCMD, ENQCMDS, or MOVDIR64B instruction, potentially improving overall throughput. DSA imposes a limit on the number of working descriptors in a batch. This limit is indicated in the Maximum Batch Size field of the General Capability Register.

[0609] Batch descriptors are submitted to the work queue in the same way as other work descriptors. When the device processes a batch descriptor, it reads the array of work descriptors from memory and then processes each work descriptor. Work descriptors are not necessarily processed in order.

[0610] The PASID 3803 and U / S flags of the batch descriptor are used for all descriptors in the batch. The PASID and U / S fields 3810 in the batch descriptors are ignored. Each work descriptor in the batch can specify a completion record address 3804, just like directly submitted work descriptors. Alternatively, the batch descriptor can specify a "completion queue" address, where completion records from all work descriptors in the batch are written by the device. In this case, the completion record address field 3804 in the batch descriptors is ignored. The completion queue should have one more entry than the descriptor count, so for each descriptor in the batch, there is space for the completion record to increment the batch descriptor count. Completion records are generated in the order in which the descriptors complete, which may differ from the order in which they appear in the descriptor array. Each completion record includes the index of the descriptor in the batch that generated the completion record. Index 0xff is used for the batch descriptor itself. Index 0 is used for directly submitted descriptors other than batch descriptors. Some descriptors in the batch may not generate a completion record if they did not request a completion record and they completed successfully. In this scenario, the number of completion records written to the completion queue may be less than the number of descriptors in the batch. The completion records for batch descriptors (if requested) are written to the completion queue after the completion records for all descriptors in the batch.

[0611] If a batch descriptor does not specify a completion queue, the batch descriptor's completion record (if requested) is written to its own completion record address after all descriptors in the batch have completed. The batch descriptor's completion record contains an indication of whether any descriptor in the batch completed in a state other than success. Normally, this allows the software to view only the batch descriptor's completion record after all descriptors in the batch have completed successfully.

[0612] If needed, one or more work descriptors in a batch may also request a completion interruption. The completion record for the batch descriptor (if requested) is written after the completion records and completion interruptions for all descriptors in the batch. The completion interruption for the batch descriptor (if requested) is generated after the completion record for the batch descriptor, just like for any other descriptor.

[0613] Batch descriptors may not be included in a batch. Nested or linked arrays of descriptors are not supported.

[0614] By default, DSA does not guarantee any ordering when executing work descriptors. Descriptors can be dispatched and completed in any order that the device deems appropriate to maximize throughput. Therefore, if ordering is required, the software must explicitly order them; for example, the software can submit a descriptor, wait for a completion record or interruption from the descriptor to ensure completion, and then submit the next descriptor.

[0615] The software can also specify the order of descriptors within a batch specified by the batch descriptor. Each job descriptor has a guard flag. When set, the guard ensures that processing of that descriptor will not begin until the previous descriptors in the same batch are completed. This allows descriptors with guards to use data generated by previous descriptors in the same batch.

[0616] The descriptor completes after all writes to the global observable generated by the operation; after a readback to the destination, if requested; after all writes to the global observable are completed, if requested; and after a production completion interruption, if requested.

[0617] If any descriptor in a batch completes in a state that is not equal to success—for example, if it partially completes due to a page fault—then subsequent descriptors with a guardrail flag equal to 1, as well as any subsequent descriptors in the batch, are discarded. The completion record for the batch descriptors used to submit the batch indicates the number of completed descriptors. Any descriptor that partially completes and generates a completion record is counted as completed. Only discarded descriptors are considered not completed.

[0618] Guardrails also ensure the ordering of completion records and interruptions. For example, a No-op descriptor with guardrails and a request completion terminal setting will cause an interruption to be generated after all preceding descriptors in the batch have completed (and their completion records have been written if necessary). Completion record writes are always ordered after data writes generated by the same working descriptor, and completion interruptions (if requested) are always ordered after completion record writes of the same working descriptor.

[0619] An expulsion is a descriptor that allows a client to wait for all descriptors belonging to its own PASID to complete. It can be used as a guardrail operation for an entire PASID. The expulsion operation completes when all previous descriptors for that PASID have completed. An expulsion descriptor can be a single completion record or an interrupt used by software to request completion of all its descriptors. Expulsions are regular descriptors submitted to the regular work queue. Expulsion descriptors may not be included in a batch. (Guardrail flags can be used in batches to wait for the completion of previous descriptors in the batch.)

[0620] The software must ensure that no descriptor with a specified PASID is submitted to the device after the discharge descriptor is submitted but before its completion. If additional descriptors are submitted, it is not specified whether the discharge operation waits for the completion of these additional descriptors. This can cause the discharge operation to take a long time. Even if the device does not wait for the additional descriptors to complete, some additional descriptors may still complete before the discharge operation finishes. In this way, discharge differs from fencing, which ensures that subsequent operations do not begin until all previous operations have completed.

[0621] In one implementation, the abort / exit command is submitted by privileged software (OS kernel or VMM) by writing to the abort / exit register. Upon receiving one of these commands, the DSA waits for the completion of certain descriptors (as described below). After the command completes, the software can be certain that there are no more descriptors in the specified pending category in the device.

[0622] In one implementation, there are three types of discharge commands: Discharge ALL, Discharge PASID, and Discharge WQ. Each command has an abort flag that tells the device it can discard any incomplete descriptors instead of processing them completely.

[0623] The all eject command awaits the completion of all descriptors submitted before it. Descriptors submitted after the all eject command may be in progress when the all eject is complete. While the all eject command awaits the completion of previous descriptors, the device can begin processing new descriptors.

[0624] The `Dump PASID` command waits for all descriptors associated with the specified PASID. When the `Dump PASID` command completes, there are no more descriptors for the specified PASID in the device. The software can ensure that no descriptors with the specified PASID are submitted to the device after the `Dump PASID` command is submitted and before its completion; otherwise, the behavior is undefined.

[0625] The WQ command excludes all descriptors awaiting submission to the specified work queue. The software ensures that no descriptors are submitted to the WQ after the WQ command is submitted and before it completes.

[0626] When an application or VM suspends its use of DSA, it may have incomplete descriptors submitted to DSA. This work must be completed so that the client is in a consistent state that can be recovered later. The OS or VMM uses the `exclude PASID` and `exclude all` commands to wait for any incomplete descriptors. The `exclude PASID` command is used for applications or VMs using a single PASID. The `exclude all` command is used for VMs using multiple PASIDs.

[0627] When an application using DSA exits or is terminated by the operating system (OS), the OS needs to ensure that there are no incomplete descriptors before it can release or reuse the address space, allocated memory, and PASID. To clear any incomplete descriptors, the OS uses the Expel PASID command to terminate the client's PASID and set the Abort flag to 1. Upon receiving this command, DSA will discard all descriptors belonging to the specified PASID without further processing.

[0628] One implementation of DSA provides a mechanism for specifying the Quality of Service (QoS) for dispatching work from multiple Work Queues (WQs). DSA allows software to divide the total WQ space into multiple WQs. Different priorities can be assigned to each WQ for work dispatch. In one implementation, the DSA scheduler / arbitrator 3513 dispatches work from WQs such that higher-priority WQs receive more service than lower-priority WQs. However, DSA ensures that higher-priority WQs do not deplete lower-priority WQs. As described above, various priority ordering schemes can be employed based on implementation requirements.

[0629] In one implementation, a WQ configuration register table is used to configure WQs. The software can configure the number of active WQs to match the desired number of QoS levels. The software configures each WQ by programming the WQ size in the WQ configuration register table and some additional parameters. This effectively divides the entire WQ space into the desired number of WQs. Unused WQs have a size of 0.

[0630] Errors can be broadly categorized into two types: 1) incidental errors that occur when processing descriptors specific to a PASID, and 2) non-incidental errors that are inherently global rather than PASID-specific. DSA attempts to minimize the impact of errors originating from one PASID on other PASIDs. PASID-specific errors are reported in the completion record of the corresponding descriptor, unless the error is on the completion record itself (e.g., a page fault at the completion record address).

[0631] Errors in descriptor commit or descriptor completion records can be reported to the host driver via the software error register (SWERROR). Hardware errors can be reported via the hardware error register (HWERROR).

[0632] One implementation of DSA performs the following check when the enable bit in the device enable register is set to 1:

[0633] • Bus master enabled is set to 1.

[0634] The combination of PASID, ATS, and PRS capabilities is effective. (See Table 6-3 in Section 6.1.3)

[0635] • The sum of the WQ size fields of all WQCFG registers is not greater than the total WQ size.

[0636] • For each GRPCFG register, the WQ and engine fields are either both 0 or neither is zero.

[0637] • Each WQ in the WQCFG register with a non-zero size field is in a group.

[0638] • Each WQ with a size field of zero in the WQCFG register is not in any group.

[0639] • Each engine is in no more than one group.

[0640] If any of these checks fail, the device is not enabled, and an error code is logged in the Error Code field of the Device Enable register. These checks can be performed in any order. Therefore, an indication of one type of error does not mean that other errors will not occur. The same configuration error may result in different error codes at different times or in different versions of the device. If no checks fail, the device is enabled, and the Enabled field is set to 1.

[0641] When the WQ enable bit in the WQCFG register is set to 1, the device performs the following checks:

[0642] • The device is enabled (i.e., the enabled field in the device enable register is 1).

[0643] • The WQ size field is not zero.

[0644] • The WQ threshold is not greater than the WQ size field.

[0645] The WQ mode field selects the supported modes. That is, if the shared mode support field in WQCAP is 0, then the WQ mode is 1; or if the dedicated mode support field in WQCAP is 0, then the WQ mode is 0. If both the shared mode support and dedicated mode support fields are 1, then any value for the WQ mode is allowed.

[0646] • If the error blocking support bit in GENCAP is 0, then the WQ error blocking enable field is 0.

[0647] If any of these checks fail, WQ is not enabled, and an error code is logged in the WQ Error Code field of the WQ configuration register 3500. These checks can be performed in any order. Therefore, an indication of one type of error does not mean that other errors will not occur. The same configuration error may result in different error codes at different times or in different versions of the device. If no checks fail, the device is enabled, and the WQ Enabled segment is set to 1.

[0648] In one implementation, the DSA performs the following checks when receiving a descriptor:

[0649] • The WQ identified by the register address used for the commit descriptor is the active WQ (the size field in the WQCFG register is non-zero). If this check fails, an error will be logged in the software error register (SWERROR).

[0650] • If the descriptor has already been committed to the shared WQ, then

[0651] It is submitted using ENQCMD or ENQCMDS. If this check fails, the error will be logged in SWERROR.

[0652] If the descriptor was submitted via a non-privileged or guest entry, the current queue occupancy is no greater than the WQ threshold. If this check fails, a retry response is returned.

[0653] If the descriptor was submitted via a privileged entry, the current queue occupancy is less than the WQ size. If this check fails, a retry response is returned.

[0654] • If the descriptor has been submitted to a dedicated WQ,

[0655] It was submitted using MOVDIR64B.

[0656] • The queue occupancy rate is less than the size of WQ.

[0657] If any of these checks fail, the error is logged in SWERROR.

[0658] In one implementation, the device performs the following checks on each descriptor during processing:

[0659] The value in the opcode field corresponds to the supported operations. This includes checking whether the operation is valid in the context of the operation submission. For example, a batch descriptor in a batch will be considered an invalid opcode.

[0660] • No reserved flags are set. This includes flags where the corresponding capability bit in the GENCAP register is 0.

[0661] • No unsupported flags are set. This includes flags reserved for certain operations. For example, guardrail bits are reserved in the direct queue descriptor instead of as part of a batch. It also includes flags disabled in the configuration, such as the error blocking flag, which is reserved when the error blocking enable field in the WQCFG register is 0.

[0662] • Required flags are set. For example, the request completion flag must be 1 in the descriptor of the comparison operation.

[0663] • Reserved fields are set to 0. This includes any field that has no explicit meaning for the specified operation. Some implementations may not check all reserved fields, but software should take care to clear all unused fields for maximum compatibility. In the batch descriptor, the descriptor count field is not greater than the maximum batch size field in the GENCAP register.

[0664] • The transfer size, source size, maximum incremental record size, incremental record size, and maximum destination size (applicable to descriptor types) are not greater than the maximum transfer size field in the GENCAP register.

[0665] • In memory copying with dual translation descriptors, bits 11:0 of ​​the two destination addresses are identical.

[0666] • If the interrupt message storage flag is set, the completion interrupt handle will be smaller than the interrupt message storage size.

[0667] In one implementation, if the completion record address 3804 cannot be translated, descriptor 3800 is discarded and the error is logged in the software error register. Otherwise, if any of these checks fail, a status field is written to the completion record indicating the type of failed check and that byte completion is set to 0. A completion interrupt is generated upon request.

[0668] These checks can be performed in any order. Therefore, the completion of an indication of one type of error in the log does not mean that there are no other errors. The same invalid descriptor may report different error codes at different times or with different versions of the device.

[0669] Reserved fields in a descriptor can be categorized into three types: fields that are always reserved; fields that are reserved under certain conditions (e.g., based on capabilities, configuration fields, the way the descriptor is submitted, or the values ​​of other fields in the descriptor itself); and fields that are reserved based on the operation type. The following table lists the conditions for reserving fields.

[0670] Table W (Conditional Retention Field Check)

[0671]

[0672]

[0673]

[0674] Table X (Operation-specific reserved field checks)

[0675] As mentioned above, DSA supports the use of either physical or virtual addresses. Using virtual addresses shared with processes running on the processor core is called Shared Virtual Memory (SVM). To support SVM, the device provides a PASID when performing address translation and handles page faults that occur when an address translation fails. However, the device itself does not distinguish between virtual and physical addresses; this distinction is controlled by the programming of the IOMMU 1710.

[0676] In one implementation, DSA supports Address Translation Service (ATS) and Page Request Service (PRS) PCI fast capabilities, such as Figure 28 As shown, Figure 28 The diagram illustrates PCIe logic 2820 communicating with PCIe logic 2808 using PCDI to utilize ATS. ATS describes the device behavior during address translation. When a descriptor enters the descriptor processing unit, device 2801 may request the translation of the address in the descriptor. If a hit is found in device TLB 2822, the device uses the corresponding Host Physical Address (HPA). If a miss or permission error occurs, one implementation of DSA 2801 sends an address translation request to IOMMU 2810 for translation (i.e., across multiprotocol link 2800). IOMMU 2810 can then locate the translation by traversing the corresponding page tables and returns an address translation response containing the translated address and valid permission. Device 2801 then stores the translation in device TLB 2822 and operates using the corresponding HPA. If IOMMU 2810 cannot locate the translation in the page tables, it can return an address translation response indicating that no translation is available. If the IOMMU 2810 response indicates that there is no translation or that a valid license does not include the license required for operation, it is considered a page fault.

[0677] DSA device 2801 may encounter a page fault in one of the following: 1) completion record address 3804; 2) descriptor list address in the batch descriptor; or 3) source buffer or destination buffer address. DSA device 2801 may block until the page fault is resolved, or complete the descriptor early and return partial completion to the client. In one implementation, DSA device 2801 always blocks when a page fault exists at completion record address 3804 or descriptor list address.

[0678] When the DSA encounters a page fault and is blocked, it reports the error as a Page Request Service (PRS) request to the IOMMU 2810, which is then serviced by the OS page fault handler. The IOMMU 2810 can notify the OS via an interrupt. The OS verifies the address and, upon a successful check, creates a mapping in the page table and returns a PRS response via the IOMMU 2810.

[0679] In one implementation, each descriptor 3800 has an error blocking flag indicating whether DSA 2801 should return to partial completion or block when a page fault occurs on the source or destination buffer address. When the error blocking flag is 1 and an error is encountered, the descriptor encountering the error will be blocked until a PRS response is received. Other operations following the descriptor with the error may also be blocked.

[0680] When the error blocking is 0 and a page fault is encountered at the source or destination buffer address, the device stops operating and writes the partial completion status, along with the error address and progress information, to the completion record. When the client software receives the completion record indicating partial completion, it can choose to correct the error on the processor (e.g., by contacting the page) and submit a new work descriptor describing the remaining work.

[0681] Alternatively, the software can perform the remaining work on the processor. The Error Blocking Support field in the General Capability Register (GENCAP) indicates that the device supports this feature, while the Error Blocking Enable field in the Work Queue Configuration Register allows the VMM or kernel driver to control whether an application is allowed to use this feature.

[0682] Device page faults can be relatively costly. In fact, maintaining device page faults can be more expensive than maintaining processor page faults. Even if the device performs partial work completion at the time of the fault rather than blocking the fault, it still incurs overhead because it requires software intervention to maintain the page fault and resubmit the work. Therefore, for optimal performance, it is desirable for software to minimize device page faults without incurring the overhead of pinning and unpinning.

[0683] The batch descriptor list and source data buffer are typically generated by software just before they are submitted to the device. Therefore, these addresses are unlikely to cause errors due to temporal locality. However, errors are more likely to occur if the completion descriptor and destination data buffer are not accessed by software before submission to the device. These errors can be minimized by the software explicitly "writing-accessing" these pages before submission.

[0684] During a device TLB invalidation request, if an invalidated address is being used in the descriptor processing unit, the device waits for the engine to complete the invalidation request at that address before completing the invalidation request.

[0685] Additional descriptor types

[0686] Some implementations may utilize one or more of the following additional descriptor types:

[0687] No operation

[0688] Figure 40 An exemplary no-operation descriptor 4000 and no-operation completion record 4001 are shown. No-operation operation 4005 does not perform a DMA operation. It can request a completion record and / or a completion interrupt. If it is in a batch, it can specify a guardrail flag to ensure that the no-operation descriptor completes after all previous descriptors in the batch have been completed.

[0689] batch

[0690] Figure 41 An exemplary batch descriptor 4100 and a no-operation-complete record 4101 are shown. Batch operation 4108 queues multiple descriptors at a time. Descriptor list address 4102 is the address of an array of consecutive working descriptors to be processed. In one implementation, each descriptor in the array is 64 bytes. Descriptor list address 4102 is 64-byte aligned. Descriptor count 4103 is the number of descriptors in the array. The set of descriptors in the array is called a "batch". The maximum number of descriptors allowed in a batch is given in the maximum batch size field of GENCAP.

[0691] The PASID 4104 and U / S flag 4105 in the batch descriptor are used for all descriptors in the batch. The PASID 4104 and U / S flag fields 4105 in the descriptors within the batch are ignored. If the completion queue enable flag in the batch descriptor 4100 is set, the completion record address valid flag must be 1, and the completion queue address field 4106 contains the address of the completion queue for all descriptors in the batch. In this case, the completion record address field 4106 in the descriptors within the batch is ignored. If the completion queue support field in the general capability register is 0, the completion queue enable flag is retained.

[0692] If the completion queue enable flag in the batch descriptor is 0, the completion record for each descriptor in the batch is written to the completion record address 4106 in each descriptor. In this case, if the request completion record flag in the batch descriptor is 1, the completion queue address field is used only as the completion record address 4106 of the batch descriptor.

[0693] If all descriptors in the batch complete successfully, the status field 4110 of the batch completion record 4101 indicates success; otherwise, it indicates that one or more descriptors completed with a status not equal to success. The completed descriptor field 4111 of the completion record contains the total number of descriptors processed in the batch, regardless of whether they were successful. If guardrails exist in the batch or a page fault occurs while reading the batch, the completed descriptor 4111 may be less than the descriptor count 4103.

[0694] discharge

[0695] Figure 42An exemplary discharge descriptor 4200 and discharge completion record 4201 are shown. Discharge operation 4208 waits for the completion of all pending descriptors associated with PASID 4202 in the work queue to which discharge descriptor 4200 has been submitted. This descriptor can be used during normal shutdown of a process that is already using the device. To wait for all descriptors associated with PASID 4202, the software should submit a separate discharge operation to each work queue using PASID 4202. The software should ensure that no descriptor with the specified PASID 4202 is submitted to the work queue after discharge descriptor 4201 is submitted and before its completion.

[0696] Exclusion descriptor 4201 may not be included in the batch; it is considered an unsupported operation type. Exclusion should specify either a request to complete a record or a request to interrupt completion. A completion notification will be sent after other descriptors are completed.

[0697] Memory movement

[0698] Figure 43 An exemplary memory move descriptor 4300 and memory move completion record 4301 are shown. Memory move operation 4308 copies memory from source address 4302 to destination address 4303. The number of bytes copied is given by transfer size 4304. There are no alignment requirements for the memory address or transfer size. If the source and destination regions overlap, the memory copy is completed as if the entire source buffer were copied to temporary space and then to the destination buffer. When the beginning of the destination buffer overlaps with the end of the source buffer, this can be achieved by reversing the direction of copying.

[0699] If the operation is partially completed due to a page fault, the direction field 4310 of the completed record is 0 if the copying started from the beginning of the source and destination buffers, and is 1 if the copying direction is reversed.

[0700] To resume operation after partial completion, if the direction is 0, the source and destination address fields 4302-4303 in the contiguous descriptor should be incremented by the completed byte, and the transfer size should be reduced by the completed byte 4311. If the direction is 1, the transfer size 4304 should be reduced by the completed byte 4311, but the source and destination address fields 4302-4303 should remain the same as in the original descriptor. Note that if subsequent partial completions occur, the direction field 4310 may differ from the one used for the first partial completion.

[0701] filling

[0702] Figure 44An exemplary fill descriptor 4400 is shown. The memory fill operation 4408 fills the memory at the destination address 4406 with the value in the pattern field 4405. The pattern size can be 8 bytes. To use a smaller pattern, the software must copy the pattern from the descriptor. The number of bytes written is given by the transfer size 4407. The transfer size does not need to be a multiple of the pattern size. There are no alignment requirements for the destination address or transfer size. If the operation is partially completed due to a page fault, the completed bytes field of the completed record contains the number of bytes written to the destination before the error occurred.

[0703] Compare

[0704] Figure 45 An exemplary comparison descriptor 4500 and comparison completion record 4501 are shown. Comparison operation 4508 compares the memory at source 1 address 4504 with the memory at source 2 address 4505. The number of bytes compared is given by transfer size 4506. There are no alignment requirements for the memory address or transfer size 4506. The completion record address must be valid and the request completion record flag must be 1; the completion record address must be valid. The comparison result is written to the result field 4510 of the completion record 4501: a value of 0 indicates that the two memory regions match, and a value of 1 indicates that they do not match. If result 4510 is 1, the completed byte 4511 field of the completion record indicates the byte offset of the first difference. If the operation is partially completed due to a page fault, the result is 0. If a difference is detected, a difference is reported instead of a page fault.

[0705] If the operation is successful and the result check flag is 1, the status field 4512 of the completion record is set according to the result and the expected result, as shown in the table below. This allows subsequent descriptors in the same batch with guardrail flags to continue or stop batch execution based on the comparison result.

[0706] Inspection result mark Expected result: 0 result state 0 X X success 1 0 0 success 1 0 1 Success with erroneous predicates 1 1 0 Success with erroneous predicates 1 1 1 success

[0707] Table Y

[0708] Compare immediate values.

[0709] Figure 46An exemplary compare immediate descriptor 4600 is shown. The compare immediate operation 4608 compares the memory at source address 4601 with the value in the pattern field 4602. The pattern size is 8 bytes. To use a smaller pattern, the software must copy the pattern from the descriptor. The number of bytes compared is given by the transfer size 4603. The transfer size does not need to be a multiple of the pattern size. The completion record address must be valid and the request completion record flag must be 1; the completion record address 4604 must be valid. The comparison result is written to the result field of the completion record: a value of 0 indicates a memory region matches the pattern, and a value of 1 indicates a mismatch. If the result is 1, the completed byte field of the completion record indicates the location of the first difference. It may not be the exact byte location, but it is guaranteed not to exceed the first difference. If the operation is partially completed due to a page fault, the result is 0. If a difference is detected, the difference is reported instead of a page fault. In one implementation, the completion record format of the compare immediate descriptor and the behavior of checking results and expected results are the same as those of the compare descriptor.

[0710] Create incremental record

[0711] Figure 47 An exemplary incremental record creation descriptor 4700 and incremental record creation completion record 4701 are illustrated. The incremental record creation operation 4708 compares the memory at source 1 address 4705 with the memory at source 2 address 4702 and generates an incremental record containing the information needed to update source 1 to match source 2. The number of bytes compared is given by the transfer size 4703. The transfer size is limited by the maximum offset that can be stored in the incremental record, as described below. There are no alignment requirements for the memory address or transfer size. The completion record address must be valid and the request completion record flag must be 1; the completion record address 4704 must be valid.

[0712] The maximum size of the incremental record is given by the maximum incremental record size 4709. The maximum incremental record size 4709 should be a multiple of the incremental size (10 bytes) and must not exceed the maximum transfer size in GENCAP. The actual size of the incremental record depends on the number of differences detected between source 1 and source 2; it is written to the incremental record size field 4710 of the completed record. If the space required in the incremental record exceeds the maximum incremental record size 4709 specified in the descriptor, the operation will be completed with a partial incremental record.

[0713] The comparison result is written to the result field 4711 of the completion record 4701. If the two regions match exactly, the result is 0, the increment record size is 0, and the completed bytes are 0. If the two regions do not match, and the complete set of increments is written to the increment record, the result is 1, the increment record size contains the total size of all differences found, and the completed bytes are 0. If the two regions do not match, and the space required to record all increments exceeds the maximum increment record size, the result is 2, the increment record size 4710 contains the size of the set of increments written to the increment record (usually equal to or nearly equal to the increment record size specified in the descriptor), and the completed bytes 4712 contain the number of bytes compared before exceeding the space in the increment record.

[0714] If the operation was partially completed due to a page fault, the result 4711 is 0 or 1. As described in the previous paragraph, the completed byte 4712 contains the number of bytes compared before the page fault occurred, and the increment record size contains the space used in the increment record before the page fault occurred.

[0715] The format of incremental records is in Figure 48 As shown in the diagram, the increment record contains an array of increments. Each increment contains a 2-byte offset 4801 from source 2 and an 8-byte data block 4802, which differs from the corresponding 8 bytes in source 1. The total size of the increment record is a multiple of 10. Since offset 4801 is a 16-bit field representing multiples of 8 bytes, the maximum offset that can be represented is 0x7FFF8, and therefore the maximum transfer size is 0x80000 bytes (512KB).

[0716] If the operation is successful and the result check flag is 1, the status field of the completed record is set according to the result and the expected result, as shown in the table below. This allows subsequent descriptors in the same batch with guardrail flags to continue or stop batch execution based on the result of incremental record creation. Bits 7:2 of the expected result are ignored.

[0717]

[0718] Table Z

[0719] Application Incremental Records

[0720] Figure 49An exemplary application increment record descriptor 4901 is shown. The application increment record operation 4902 applies an increment record to the memory contents at destination address 4903. Increment record address 4904 is the address of the increment record created by the create increment record operation 4902, which completes with a result equal to 1. Increment record size 4905 is the size of the increment record, as reported in the completion record of the create increment record operation 4902. Destination address 4903 is the address of a buffer containing the same contents as the memory at source address 1 when the increment record was created. Transfer size 4906 is the same as the transfer size used when the increment record was created. After the application increment record operation 4902 completes, the memory at destination address 4903 will match the contents of the memory at source address 2 when the increment record was created. There are no alignment requirements for the memory address or transfer size.

[0721] If a page fault is encountered during incremental record operation 4902, the Completed Bytes field of the completed record contains the number of bytes of the incremental record successfully applied to the destination. If the software chooses to commit another descriptor to resume the operation, the Continuation descriptor should contain the same destination address 4903 as the original. The incremental record address 4904 should be incremented by the Completed Bytes (so it points to the first unapplied incremental), and the incremental record size 4905 should be decremented by the Completed Bytes.

[0722] Figure 50 One implementation using the create incremental record and apply incremental record operations is illustrated. First, a create incremental record operation 5001 is performed. It reads two source buffers—source 1 and 2—and writes them to an incremental record 5010, recording the actual incremental record size 5004 in its completion record 5003. The apply incremental record operation 5005 retrieves the contents of the incremental record written by the create incremental record operation 5001, its size, and a copy of the source 1 data, and updates the destination buffer 5015 to a copy of the original source 2 buffer. The create incremental record operation includes a maximum incremental record size 5002.

[0723] Memory copying with dualcast

[0724] Figure 51 An exemplary memory copy with a double translation descriptor 5100 and a memory copy with a double translation completion record 5102 are shown. A memory copy 5104 with a double translation operation copies memory from a source address 5105 to destination 1 address 5106 and destination 2 address 5107. The number of bytes copied is given by the transfer size 5108. There are no alignment requirements for the source address or transfer size. Bits 11:0 of ​​the two destination addresses 5106-5107 should be identical.

[0725] If the source region overlaps with either destination region, the memory copy is completed, as if the entire source buffer were copied to temporary space and then to the destination buffer. When the beginning of the destination buffer overlaps with the end of the source buffer, this can be achieved by reversing the copy direction. An error occurs if the source region overlaps with two destination regions, or if two destination regions overlap. If the operation is partially completed due to a page fault, the copy operation stops after writing the same number of bytes to both destination regions, and the direction field 5110 is 0 if the copy started from the beginning of both the source and destination buffers, and 1 if the copy direction is reversed.

[0726] To resume operation after partial completion, if direction 5110 is 0, the source address field 5105 and the two destination address fields 5106-5107 in the continuation descriptor should be incremented by the completed byte 5111, and the transfer size should be reduced by the completed byte 5111. If direction is 1, the transfer size 5108 should be reduced by the completed byte 5111, but the source 5105 and destination 5106-5107 address fields should remain the same as in the original descriptor. Note that if subsequent partial completions occur, the direction field 5110 may differ from the one used for the first partial completion.

[0727] Cyclic Redundancy Check (CRC) generation

[0728] Figure 52 An exemplary CRC generation descriptor 5200 and CRC generation completion record 5201 are shown. CRC generation operation 5204 calculates the CRC in memory at the source address. The number of bytes used for the CRC calculation is given by the transfer size 5205. There are no alignment requirements for the memory address or transfer size 5205. The completion record address must be valid and the request completion record flag must be 1; the completion record address 5206 must be valid. The calculated CRC value is written to the completion record.

[0729] If the operation is partially completed due to a page fault, the partial CRC result is written to the completion record along with the page fault information. If the software corrects the error and resumes the operation, this partial result must be copied to the CRC seed field of the continuation descriptor. Otherwise, the CRC seed field should be 0.

[0730] Copy with CRC generation

[0731] Figure 53An exemplary copy operation with CRC generation descriptor 5300 is shown. Copy operation 5305 with CRC generation copies memory from source address 5302 to destination address 5303 and calculates the CRC on the copied data. The number of bytes copied is given by transfer size 5304. There are no alignment requirements for the memory address or transfer size. An error occurs if the source and destination regions overlap. The completion record address must be valid and the request completion record flag must be 1; the completion record address must be valid. The calculated CRC value is written to the completion record.

[0732] If the operation is partially completed due to a page fault, a portion of the CRC result is written to the completion record along with the page fault information. If the software corrects the error and resumes the operation, this portion of the result must be copied to the CRC seed field of the continuation descriptor. Otherwise, the CRC seed field should be 0. In one implementation, the format of the completion record with the CRC-generated copy is the same as the format of the CRC-generated copy.

[0733] Data Integrity Field (DIF) Insertion

[0734] Figure 54 An exemplary DIF insert descriptor 5400 and DIF insert completion record 5401 are shown. A DIF insert operation 5405 copies memory from a source address 5402 to a destination address 5403, calculates the Data Integrity Field (DIF) on the source data, and inserts the DIF into the output data. The number of source bytes copied is given by the transfer size 5406. The DIF calculation is performed for each source data block, such as 512, 520, 4096, or 4104 bytes. The transfer size should be a multiple of the source block size. The number of bytes written to the destination is the transfer size plus 8 bytes for each source block. There are no alignment requirements for memory addresses. If the source and destination regions overlap, an error occurs. If the operation is partially completed due to a page fault, the updated values ​​of the reference tag and application tag are written to the completion record along with the page fault information. If the software corrects the error and resumes the operation, these fields can be copied to a continuation descriptor.

[0735] DIF strip

[0736] Figure 55An exemplary DIF striping descriptor 5500 and a DIF striping completion record 5501 are shown. DIF striping operation 5505 copies memory from source address 5502 to destination address 5503, calculates the Data Integrity Field (DIF) on the source data, and compares the calculated DIF with the DIF contained in the data. The number of source bytes read is given by the transfer size 5506. The DIF calculation is performed on each source data block, which can be 512, 520, 4096, or 4104 bytes. The transfer size should be a multiple of the source block size plus 8 bytes for each source block. The number of bytes written to the destination is the transfer size minus the 8 bytes for each source block. There are no alignment requirements for memory addresses. If the source and destination regions overlap, an error occurs. If the operation is partially completed due to a page fault, the updated values ​​of the reference tag and application tag are written to the completion record along with the page fault information. If the software corrects the error and recovers the operation, these fields can be copied to a continuation descriptor.

[0737] DIF Update

[0738] Figure 56 An exemplary DIF update descriptor 5600 and DIF update completion record 5601 are shown. A memory move operation 5605 with a DIF update copies memory from a source address 5602 to a destination address 5603, calculates the Data Integrity Field (DIF) on the source data, and compares the calculated DIF with the DIF contained in the data. It simultaneously calculates the DIF on the source data using the destination DIF field in the descriptor and inserts the calculated DIF into the output data. The number of source bytes read is given by the transfer size 5606. The DIF calculation is performed on each source data block, which can be 512, 520, 4096, or 4104 bytes. The transfer size 5606 should be a multiple of the source block size plus 8 bytes for each source block. The number of bytes written to the destination is the same as the transfer size 5606. There are no alignment requirements for the memory addresses. If the source and destination regions overlap, an error occurs. If the operation is partially completed due to a page fault, the updated values ​​of the source and destination reference tags and the application tag are written to the completion record along with the page fault information. If the software corrects the error and restores operation, these fields can be copied to the continuation descriptor.

[0739] Table AA below shows the DIF flags used in one embodiment. Table BB shows the source DIF flags used in one embodiment, and Table CC shows the destination DIF flags used in one embodiment.

[0740]

[0741] Table AA (DIF mark)

[0742] Source DIF Mark

[0743]

[0744]

[0745] Table BB (Source DIF Mark) Destination DIF Mark

[0746]

[0747]

[0748] Table CC (Destination DIF Mark)

[0749] In one implementation, the DIF result field reports the status of the DIF operation. This field can be defined only for DIF striping and DIF update operations, and is defined only if the status field of the completion record is success or success with an error predicate. Table DD below shows exemplary DIF result field codes.

[0750]

[0751] Table DD (DIF result field code)

[0752] The F detection condition is detected when one of the following, as shown in Table EE, is true:

[0753]

[0754] Table EE

[0755] If the operation is successful and the result check flag is 1, the status field of the completion record is set according to the DIF result, as shown in Table FF below. This allows subsequent descriptors in the same batch with guardrail flags to continue or stop batch execution based on the result of the operation.

[0756] Inspection result mark DIF results state 0 X success 1 ＝0x01 success 1 ≠0x01 Success with erroneous predicates

[0757] Table FF

[0758] Cache refresh

[0759] Figure 57 An exemplary cache refresh descriptor 5700 is shown. Cache refresh operation 5705 refreshes the processor cache at the destination address. The number of bytes refreshed is given by the transfer size 5702. The transfer size does not need to be a multiple of the cache line size. There are no alignment requirements for the destination address or transfer size. Any cache lines partially covered by the destination region are refreshed.

[0760] If the destination cache fill flag is 0, affected cache lines can be invalidated from each level of the cache hierarchy. If the cache line contains modified data from any level of the cache hierarchy, the data is written back to memory. This is similar to the behavior of the CLFLUSH instruction implemented in some processors.

[0761] If the destination cache fill flag is 1, the modified cache line is written to main memory but not evicted from the cache. This is similar to the behavior of the CLWB instruction in some processors.

[0762] This article sometimes uses the term accelerator to refer to a loosely coupled agent that can be used by software running on a host processor to offload or perform any type of computation or I / O task. Depending on the type and use model of the accelerator, these can be any combination of tasks that perform data movement to memory or storage devices, computation, communication, or these tasks.

[0763] "Loose coupling" refers to how these accelerators are exposed and accessed by the host software. Specifically, they are not exposed as processor ISA extensions, but rather as PCI-Express enumerable endpoint devices on the platform. Loose coupling allows these agents to accept work requests from the host software and operate asynchronously with the main processor.

[0764] "Accelerators" can be programmable agents (such as GPUs / GPGPUs), fixed-function agents (such as compression or encryption engines), or reconfigurable agents such as field-programmable gate arrays (FPGAs). Some of these can be used for compute offloading, while others (such as RDMA or host infrastructure interfaces) can be used for packet processing, communication, storage, or messaging operations.

[0765] Accelerator devices can be physically integrated at different levels, including on the die (i.e., the same die as the processor), on the package, on the chipset, or on the motherboard; or they can be discrete PCIe attachment devices. For integrated accelerators, even those listed as PCI-Express endpoint devices, some of these accelerators can be uniformly attached (to an on-die uniform structure or an external uniform interface), while others can be attached to an internal non-uniform interface or an external PCI-Express interface.

[0766] At the conceptual level, "accelerators" and high-performance I / O device controllers are similar. Their differences lie in capabilities such as unified / shared virtual memory, the ability to operate on pageable memory, user-mode job submission, task scheduling / preemption, and support for low-latency synchronization. Therefore, accelerators can be considered a new and improved category of high-performance I / O devices.

[0767] Unloading processing model

[0768] Accelerator unloading processing models can be broadly categorized into three usage types:

[0769] 1. Streaming In the streaming offloading model, small units of work are streamed to the accelerator at high rates. A typical example of this usage is the network data plane performing various types of packet processing at high rates.

[0770] 2. low latency For some offloading uses, the latency of the offloading operation (dispatch of tasks to accelerators and the accelerator's operation on those tasks) is critical. An example of this use case is low-latency messaging structures that include remote fetch, place, and atomic operations across host architectures.

[0771] 3. Scalable Scalable offloading refers to the use of compute accelerator services (e.g., from the highest ring in a hierarchical protection domain, such as ring-3) that are directly accessible to a large (unlimited) number of client applications (within or across virtual machines) without limitations imposed by the accelerator device, such as the number of work queues or the number of doorbells supported on the device. Several accelerator devices and processor interconnects described in this paper fall into this category. This scalability is suitable for compute offloading devices that support time-sharing / scheduling of work, such as GPUs, GPGPUs, FPGAs, or compression accelerators, or for messaging purposes, such as for enterprise databases with high scalability requirements for lock-free operations.

[0772] Work dispatch across unloading models

[0773] Each of the above unloading processing models presents its own work assignment challenges, as described below.

[0774] 1. Work dispatch used for stream offloading

[0775] For streaming applications, a typical work dispatch model uses a memory-resident work queue. Specifically, the device is configured with the location and size of the work queue in memory. A hardware-implemented doorbell (tail pointer) register is updated by software when a new work element is added to the work queue. The hardware reports the current head pointer to the software to enforce producer-consumer flow control on work queue elements. For streaming applications, a typical model is that software checks for space in the work queue by querying the head pointer (typically maintained by hardware in host memory to avoid the overhead of software reads from UC MMIO) and the tail pointer, adds the new work element to the memory-resident work queue, and updates the tail pointer by writing to the device using the doorbell register.

[0776] Doorbell writes are typically 4-byte or 8-byte unbuffered (UC) writes to MMIO. On some processors, UC writes are serialized operations that ensure global observation of older storage before the UC write (required for producer-consumer use) is published, but also prevent all newer storage publications in the processor pipeline until the platform publishes the UC write. Typical latency for UC write operations on Xeon server processors is on the order of 80-100 ns, during which time all newer storage operations are blocked by the core, thus limiting streaming offload performance.

[0777] While one approach to address serialization of newer memory following a UC doorbell write is to use Write Combination (WC) memory operations for doorbell writes (due to WC's weak ordering), using WC memory for doorbell writes presents several challenges: the doorbell write size (typically a DWORD or QWORD) is smaller than the cache line size. These partial writes introduce additional latency as the processor holds them in its Write Combination Buffer (WCB) for potential write combination opportunities, resulting in delays in publishing the doorbell write from the processor. Software can force them to be published via explicit memory fencing, generating the same serialization for newer memory as with UC doorbells.

[0778] Another issue with WC-mapped MMIO is the exposure of unpredictable and speculative reads (using MOVNTDQA) to WC-mapped MMIO (with registers that may have read side effects). Addressing this is cumbersome for the device because it requires the device to host the WC-mapped doorbell registers on a different page than other UC-mapped MMIO registers. This also presents challenges for virtualization use, where VMM software can no longer ignore guest memory types and enforce UC mapping for any device MMIO exposed to guests using EPT page tables.

[0779] The MOVDIRI instruction described in this article addresses the aforementioned limitations of using UC or WC storage for doorbell writes with these streaming offloads.

[0780] 2. Work assignment for low-latency unloading

[0781] Certain types of accelerator devices are highly optimized to complete requested operations with minimal latency. Unlike streaming accelerators (optimized for throughput), these accelerators typically implement device-managed work queues (exposed via device MMIO) to avoid DMA read latency from memory-managed work queues (and in some cases, even data buffers). Instead, host software submits work by directly writing work descriptors (and in some cases, data) to the device-managed work queue exposed via device MMIO. Examples of such devices include host infrastructure controllers, remote DMA (RDMA) devices, and newer storage controllers such as Non-Volatile Memory (NVM)-Express. The use of device-managed work queues poses little challenge to existing ISAs.

[0782] To avoid the serialization overhead of UC writes, the MMIO address of the device-managed work queue is typically mapped to WC. This exposes the same challenges as WC mapping doorbells used in streaming accelerators.

[0783] Furthermore, using WC storage to a device-managed work queue requires the device to prevent certain processor write atomicity behaviors. For example, some processors only guarantee write atomicity for write operations up to 8 bytes in size within cache line boundaries (and for locking operations), and do not define any guaranteed write completion atomicity. Write operation atomicity is the granularity at which other agents observe processor memory operations and is a property of the processor instruction set architecture and coherence protocols. Write completion atomicity is the granularity at which the receiver (in the case of memory, the memory controller; in the case of MMIO, the device) observes non-cacheable memory operations. Write completion atomicity is stronger than write operation atomicity and is not only a feature of the processor instruction set architecture but also a platform feature. Without write completion atomicity, processor instructions performing N-byte non-cacheable memory operations can be received by the device-managed work queue as multiple (fragmented) write transactions. Currently, device hardware needs to prevent such fragmented writes by tracking each word of the work descriptor or the data written to the device-managed work queue.

[0784] The MOVDIR64B instruction described in this article addresses the aforementioned limitation by supporting 64-byte writes with guaranteed atomicity. MOVDIR64B can also be used for other purposes, such as writing to persistent memory (NVM attached to the memory controller) and copying data across systems via a non-transparent bridge (NTB).

[0785] 3. Work dispatch used by scalable unloading

[0786] The traditional approach to submitting work from an application to an I / O device involves making a system call to the kernel I / O stack, which routes the request to the I / O controller device via the kernel device driver. While this approach is scalable (any number of applications can share the device's services), it introduces latency and overhead by serializing the kernel I / O stack, which is often a performance bottleneck for high-performance devices and accelerators.

[0787] To support low-overhead job dispatch, some high-performance devices support direct Ring-3 access, allowing direct job dispatch to the device and job completion checks. In this model, some device resources (doorbell, work queue, completion queue, etc.) are allocated and mapped to the application's virtual address space. Once mapped, Ring-3 software (e.g., user-mode drivers or libraries) can directly dispatch jobs to the accelerator. For devices supporting Shared Virtual Memory (SVM) capabilities, the kernel-mode driver sets up the doorbell and work queues to identify the process address space identifier (PASID) of the application process to which the doorbell and work queue are mapped. When processing job items dispatched through a specific work queue, the device uses the corresponding PASID configured for that work queue to perform virtual-to-physical address translation via the I / O Memory Management Unit (IOMMU).

[0788] One of the challenges of direct ring-3 job submission is scalability. The number of application clients that can submit work directly to the accelerator device depends on the number of queues / doorbells (or device-hosted work queues) supported by the accelerator device. This is because doorbells or device-hosted work queues are statically assigned / mapped to application clients, and accelerator devices are designed to support a fixed number of these resources. Some accelerator devices attempt to “solve” this scalability challenge by oversubmitting the doorbell resources they have (by dynamically detaching and reconnecting doorbells as needed by the application), but this is often cumbersome and difficult to scale. For devices that support I / O virtualization (e.g., single root I / O virtualization (SR-IOV)), the limited doorbell / work queue resources are further constrained because these resources need to be partitioned among different virtual functions (VFs) allocated to different virtual machines.

[0789] Scaling is critical for high-performance messaging accelerators used in enterprise applications such as databases for lock-free operations (some of which support 64K to 1M queue pairs) and for compute accelerators that support sharing accelerator resources among tasks submitted from a large number of clients.

[0790] The ENQCMD / S directive described in this paper addresses the aforementioned scaling limitations, enabling an unlimited number of clients to subscribe to and share work queue resources on the accelerator.

[0791] One implementation includes a new type of memory operation for the processor core, including direct memory and queued memory.

[0792] In one implementation, direct memory is generated using the MOVDIRI and MOVDIR64B instructions described herein.

[0793] Cacheability Similar to UC and WC storage, direct storage is not cacheable. If a direct storage is issued to a cached address, the line is written back from the cache (if modified) before being directly stored and is invalid.

[0794] Memory sorting Similar to WC storage, direct storage is weakly ordered. Specifically, it is not ordered against older WB / WC / NT storage, CLFLUSHOPT, and CLWB to different addresses. Newer WB / WC / NT storage, CLFLUSHOPT, or CLWB to different addresses can allow older direct storage to pass through. Direct storage to the same address is always ordered together with older storage (including direct storage) to the same address. Any operation that enforces storage protection (e.g., SFENCE, MFENCE, UC / WP / WT storage, LOCK, IN / OUT instructions, etc.) can protect direct storage.

[0795] Write combination Direct storage exhibits different write combination behavior than regular WC storage. Specifically, direct storage is eligible for immediate evicting from the write combination buffer and therefore cannot be combined to the same address as newer storage (including direct storage). Older WC / NT storage held in the write combination buffer can be combined to the same address as newer direct storage, and to avoid such combination, guardrail WC / NT storage must be explicitly stored before performing direct storage to the same address.

[0796] atomicityDirect memory supports write completion atomicity for the write size of the instruction that issued the direct memory operation. In the case of MOVDIR, the write completion atomicity is 4 bytes (or 8 bytes) when the destination is 4-byte aligned (or 8-byte aligned). For MOVDIR64B, the destination is forced to be 64-byte aligned and the write completion atomicity is 64 bytes. Write completion atomicity guarantees that direct memory will not be broken into multiple write transactions processed by the memory controller or root compound. Root compound implementations on processors that support direct memory guarantee that direct memory is forwarded as a single, non-decomposed write transaction on external PCI-Express structures (and internal I / O structures within the SoC that follow PCI-Express ordering). Read operations from any agent (processor or non-processor agent) to the memory location will see all the data written by the instruction that issued the direct memory operation or none at all.

[0797] Ignore destination storage type Direct storage ignores the destination address memory type (including UC / WP types) and always follows a weak ordering. This allows software to map the device MMIO to a UC and access specific registers (such as doorbell or device-managed work queue registers) using direct storage instructions (MOVDIRI or MOVDIR64B), while continuing to access other registers that may have strict serialization requirements using normal MOV operations that follow UC ordering based on the mapped UC memory type. This also allows direct storage instructions to run from within the guest software, while the virtual machine monitor (VMM) software (without device-specific knowledge) maps the guest-exposed MMIO to a UC in the processor extended page table (EPT), ignoring the guest memory type.

[0798] SoCs that support direct memory need to ensure the atomicity of direct memory writes as follows:

[0799] Direct storage to main memoryFor direct storage to main memory, the consistency architecture and system agents should ensure that all data bytes in direct storage are published to the home agent or other globally observable (GO) point for use as a single (non-decomposed) write transaction to memory. For platforms supporting persistent memory, the home agent, memory controller, memory-side cache, inline memory cryptographic engine, memory bus connecting to persistent memory (e.g., DDR-T), and the persistent memory controller itself must support the same or higher granularity of write completion atomicity for direct storage. Therefore, software can use MOVDIR64B to directly store 64 bytes to memory (volatile or persistent) and guarantee that all 64-bit bytes written will be atomically processed by all agents. As with normal persistent memory writes, if the software requires an explicit commit to persistence, it follows a direct storage sequence with guardrails / commits / guardrails.

[0800] Direct storage to memory-mapped I / O For direct storage with Memory-Mapped I / O (MMIO), the consistency architecture and system agent must ensure that all data bytes in the direct storage are published to the root complex (the globally observable point for MMIO requests) as a single (non-decompositional) write transaction. The root complex implementation must ensure that each direct storage instance is processed and forwarded as a single (non-decompositional) published write transaction on the internal I / O structure connecting the PCI-Express Root Complex Integrated Endpoint (RCIEP) and the root port (RP). The PCI-Express root port and switch port must forward each direct storage instance as a single published write transaction. For direct storage on or after a secondary bridge (or legacy PCI, PCI-X bridge) or secondary bus (e.g., USB, LPC, etc.), write completion atomicity is undefined or not guaranteed.

[0801] Note that some SoC implementations already guarantee the atomicity of write completion for WC write requests. Specifically, partial row WC writes (WCiL) and full row WC writes (WCiLF) are handled by the system agent, memory controller, root complex, and I / O structures with write completion atomicity. For such implementations, the processor does not need to distinguish between direct writes and WC writes, and the behavioral differences between direct memory and WC memory reside within the processor core. Therefore, no changes are proposed to the internal or external structural specifications for direct writes.

[0802] The processing of direct writes received by PCI-Express endpoints or RCIEPs is implementation-specific. Depending on the device's programming interface, the device and its drivers may always require the use of direct storage instructions (such as MOVDIR64B) to write to certain registers (e.g., doorbell registers or device-managed work queue registers) and to perform atomic processing on them within the device. Writes to other registers on the device can be processed by the device without any atomicity considerations or expectations. For RCIEPs, if registers with write atomicity requirements are implemented for access via sideband or dedicated line interfaces, such implementations must ensure write atomicity properties in an implementation-specific manner.

[0803] In one implementation, the queued storage is generated by the ENQCMD and ENQCMDS instructions described herein. The intended target of the queued storage is a shared work queue (SWQ) on an accelerator device. In one implementation, the queued storage has the following properties.

[0804] Non-posted Queuing stores generate 64-byte non-published write transactions to the target address and receives a completion response indicating a success or retry status. The success / retry status returned in the completion response can be returned to the software via the ENQCMD / S instruction (e.g., in a zero flag).

[0805] Cacheability In one implementation, queued storage is not cacheable. Platforms that support queued storage enforce that queued non-published writes are routed only to address (MMIO) ranges that are explicitly enabled to accept these storages.

[0806] Memory sorting Queued stores can update the architectural state (e.g., zero-marking) with non-published write completion status. Therefore, from a given logical processor, at most one queued store can be incomplete. In this sense, a queued store from a logical processor cannot allow another queued store originating from the same logical processor to pass through. Queued stores are not ordered against older WB / WC / NT stores, CLFLUSHOPT, and CLWB to different addresses. Software that needs to enforce such ordering can use explicit storage protection after such stores and before queued stores. Queued stores are always ordered together with older stores to the same address.

[0807] Alignment The ENQCMD / S instruction forces the queued storage destination address to be 64-byte aligned.

[0808] atomicityQueued stores generated by the ENQCMD / S instruction support 64-byte write completion atomicity. Write completion atomicity guarantees that queued stores will not be broken into multiple transactions when processed by the root compound. The root compound implementation on processors that support queued stores guarantees that each queued store is forwarded to the endpoint device as a single (non-decomposed) 64-byte non-published write transaction.

[0809] Ignore destination storage type Similar to direct memory, queued memory ignores the destination address memory type (including UC / WP types) and always follows the ordering described above. This allows software to continue mapping the device MMIO to UC and accessing shared work queue (SWQ) registers using ENQCMD / S instructions, while continuing to access other registers using regular MOV instructions or via direct memory (MOVDIRI or MOVDIR64B) instructions. It also allows queued memory instructions to run from within the guest software, while the VMM software (lacking device-specific knowledge) maps the guest-exposed MMIO to UC in the processor extended page table (EPT), ignoring the guest memory type.

[0810] Platform considerations for queuing storage

[0811] In some implementations, a specific set of platform integration devices supports shared work queue (SWQ) capabilities. These devices can be attached to the root complex via internal I / O structures. These devices may be exposed to the host software as PCI Express Root Complex Integration Endpoint (RCIEP) or PCI Express endpoint devices behind Virtual Root Ports (VRPs).

[0812] Platforms supporting integrated devices with SWQ should restrict the routing of queued non-published write requests on the internal I / O infrastructure to such devices. This is to ensure that queued unknown endpoint devices do not treat new transaction types (queued non-published writes) as anomalous Transaction Layer Packets (TLPs).

[0813] The platform terminates queued stores to all other addresses (including main memory address ranges and address ranges of all other memory mappings) and returns a normal (non-error) response to the publishing processor with a retry completion status. No platform error is generated on such queued store terminations because non-privileged software (Ring-3 software or Ring-0 software in VMX non-root mode) can generate queued non-publishing write transactions by executing the ENQCMD / S instruction.

[0814] The root complex implementation should ensure that queued storage is processed and forwarded to the SWQ-enabled integrated device as a single (non-decomposed) non-published write transaction on the internal I / O structure.

[0815] Platform performance considerations

[0816] This section describes some performance considerations when system agents and multiple system agents handle queued storage.

[0817] Loose ordering for queuing and storing system agent tracker (TOR) entries:

[0818] To maintain memory consistency, system agent implementations typically enforce strict ordering for requests to cache line addresses for coherent memory and MMIO (when allocating TOR entries). While this requires supporting overall ordering of coherent memory accesses, this strict ordering of queued stores introduces performance issues. This is because queued stores target a shared work queue (SWQ) on the device, and therefore often multiple logical processors with the same destination SWQ address issue queued store requests. Furthermore, unlike regular stores published to the system agent, queued stores are not published and introduce latency similar to reads. To avoid situations where only one queued store is allowed to remain incomplete for a shared work queue, system agent implementations need to relax the strict ordering of queued store requests to the same address, instead allowing TOR allocation of queued stores to multiple processes at the same address. Since a logical processor can issue at most one queued store at a time, the system agent / platform can process each queued store independently without ordering concerns.

[0819] Support multiple pending queued non-release writes in the I / O bridge broker:

[0820] I / O bridge implementations typically limit the number of non-publishing (read) requests supported in the downstream path to a small number (usually a single request). This is because reads from the processor to MMIO (primarily UC reads) are not performance-critical for most use cases, and a large queue depth supporting reads requires buffers for the returned data, increasing hardware costs. Since queued storage is expected to be used normally for job dispatching to accelerator devices, this limited queue can be detrimental to performance for queued non-publishing write applications. It is recommended that I / O bridge implementations support increased queue depths (a practical ratio of the number of logical processors, since a logical processor can only have one incomplete queued storage request at a time) to improve queued non-publishing write bandwidth. Unlike read requests, queued storage does not incur the hardware cost of data buffers because queued non-publishing writes only return a completion status (success vs. retries) without returning the data.

[0821] Virtual channel support for queued non-published writes

[0822] Unlike typical memory read and write requests on I / O buses with producer-consumer ordering requirements (e.g., specified by PCI-Express transaction ordering), queued non-published writes have no ordering requirements on the I / O bus. This allows the use of non-VC0 virtual channels to issue queued non-published writes and return the corresponding completions. The advantage of using non-VC0 channels is that queued non-published write completions can have better latency (fewer cycles to keep the core running) by avoiding being ordered behind upstream publications from the device to the host on the VCO. The recommended implementation carefully considers the use case of integrated devices and minimizes queued non-published completion latency.

[0823] Intermediate termination of queued non-published writes

[0824] To handle specific flow control scenarios under high latency conditions (such as waking up internal links or power management on locked flows), intermediate agents (system agents, I / O bridges, etc.) are allowed to discard legitimate queued store requests and return completion with a retry response to the release core. Software releasing queued stores cannot directly see whether the retry response comes from the intermediate agent or the target, and typically retryes within the software (possibly with some backoff).

[0825] Implementing this intermediate termination implementation must be done with great care to ensure that such behavior does not expose any denial of service attacks between software clients sharing the SWQ.

[0826] Shared work queue support on endpoint devices

[0827] Figure 34 The concept of a shared work queue (SWQ) is illustrated, which allows multiple non-cooperative software agents (applications 3410-3412) to submit work via a shared work queue 3401 using the ENQCMD / S commands described herein.

[0828] The following considerations apply to endpoint devices that implement a shared work queue (SWQ).

[0829] SWQ and its enumerationA device physical function (PF) can support one or more SWQs. Each SWQ can be accessed for queued non-published writes via a 64-byte aligned and sized register (here referred to as SWQ_REG) within the device's MMIO address range. It is recommended that each such SWQ_REG on the device reside on a unique system page-size (4KB) region. The device driver is responsible for reporting / enumerating SWQ capabilities, the number of supported SWQs, and the corresponding SWQ_REG addresses to the software via the appropriate software interface. The driver may also optionally report the depth of supported SWQs for software tuning or informational purposes (although this is not necessary for functional correctness). For devices supporting multiple physical functions, it is recommended to support independent SWQs for each physical function.

[0830] SWQ support on a single root I / O virtualization (SR-IOV) device Devices supporting SR-IOV can support independent SWQs for each Virtual Function (VF) exposed via SWQ_REG in the respective VF Base Address Register (BAR). This design point allows for maximum performance isolation of job submissions across VFs and is applicable to small to medium numbers of VFs. For devices supporting a large number of VFs (where independent SWQs for each VF are not feasible), a single SWQ can be shared among multiple VFs. Even in this case, each VF has its own private SWQ_REG in its VF BAR, only they are backed by a public SWQ on the VFs sharing the SWQ. For such device designs, which VFs share SWQs can be determined statically through hardware design or dynamically by setting / removing the mapping between SWQ_REG and SWQ instances for a given VF through physical functions and their drivers. Device designs that share SWQs among VFs require special attention to QoS and protection against denial-of-service attacks, as described later in this section. When sharing SWQs among VFs, care must be taken in the device design to identify which VF received the queuing request for SWQ acceptance. When dispatching a work request from the SWQ, the device should ensure that the upstream request is correctly tagged with the requester ID (bus / device / function #) of the appropriate VF (in addition to the PASID transmitted in the queued request payload).

[0831] Queued non-published write addressEndpoint devices that support SWQ need to accept queued non-published writes to any address routed through their PF or VF memory BAR. For any queued non-published write request received by the endpoint device to an address that is not an SWQ_REG address, the device may need to not treat it as an error (e.g., an abnormal TLP, etc.) but instead return a completion status with a retry (MRS). Doing so ensures that unprivileged (Ring-3 or Ring-0 VMX guest) software erroneously or maliciously issuing queued stores to non-SWQ_REG addresses on SWQ-enabled devices using the ENQCMD / S command will not result in non-fatal or fatal error reports with platform-specific error handling consequences.

[0832] Non-queued request processing for SWQ_REG Endpoint devices supporting SWQ can silently discard non-queued requests (regular memory writes and reads) to the SWQ_REG address without treating them as fatal or non-fatal errors. Read requests to the SWQ_REG address may return a successful completion response (as opposed to UR or CA), with all requested data bytes set to 1. Normal memory (posted) write requests to the SWQ_REG address are simply discarded without any action from the endpoint device. This can be done to ensure that unprivileged software cannot generate regular read and write requests to the SWQ_REG address, thereby incorrectly or maliciously causing non-fatal or fatal errors reported as platform-specific error handling results.

[0833] SWQ queue depth and storage SWQ queue depth and storage are device-specific implementation details. Device designs should ensure that SWQs support sufficient queue depth to achieve maximum device utilization. SWQ storage can be implemented on-device. Integrated devices on a SoC can utilize stolen main memory (non-OS-visible private memory reserved for device use) as an overflow buffer for SWQs, allowing for a larger SWQ queue depth than is possible using on-device storage. For such designs, the use of the overflow buffer is transparent to software; the device hardware decides when to overflow (without discarding queued requests and sending a retry completion status), retrieves the execution command from the overflow buffer, and maintains any command-specific ordering requirements. For all purposes, this use of the overflow buffer is equivalent to a discrete device using locally connected DRAM for SWQ storage. Device designs using overflow buffers in stolen memory must be handled with extreme care to ensure that this stolen memory is protected from any access other than the overflow buffer reads and writes allocated to it by the device.

[0834] Non-blocking SWQ behaviorFor performance reasons, the device implementation should respond quickly to queued non-published write requests with a successful or retry completion status, and should not block the use of queued completion to free up SWQ capacity to accept the request. The decision to accept or reject a queued request for the SWQ can be based on capacity, QoS / occupancy, or any other policy. Some exemplary QoS considerations are described below.

[0835] SWQ QoS considerations For queued non-published writes to the SWQ_REG address, the endpoint device can apply admission control to decide whether to accept the request for the corresponding SWQ (and send a successful completion status) or discard it (and send a retry completion status). Admission control can be device- and usage-specific, and hardware-supported / enforced specific policies can be exposed to the software through the Physical Function (PF) driver interface. Since SWQs are shared resources with multiple producer clients, device implementations must ensure adequate protection against denial-of-service attacks between producers. The QoS of an SWQ refers only to accepting work requests for the SWQ (via queuing requests) and is orthogonal to any QoS applied by the device hardware in terms of how QoS is applied to share the device's execution resources when processing work requests submitted by different producers. Some example methods are described below for configuring endpoint devices to enforce admission policies to accept queuing requests for SWQs. These documents are for illustrative purposes only, and the specific implementation choice will be device-specific.

[0836] In one implementation, the MOVDIRI instruction uses a direct store operation to move a double-word integer from a source operand (the second operand) to a destination operand (the first operand). The source operand can be a general-purpose register. The destination operand can be a 32-bit memory location. In 64-bit mode, the default operation size of the instruction is 32 bits. MOVDIRI defines the destination as either double-word or quad-word aligned.

[0837] Direct storage can be implemented by writing data using the Write Combination (WC) memory type protocol. Using this protocol, the processor does not write data to the cache hierarchy, nor does it fetch the corresponding cache line from memory into the cache hierarchy. If the destination address is cached, the line is written back from the cache (if modified) and invalidated before direct storage. Unlike storage with non-temporal hints (which allow uncached (UC) and write-protected (WP) memory types of the destination to override non-temporal hints), direct storage always follows the WC memory type protocol regardless of the destination address's memory type (including UC and WP types).

[0838] Unlike WC stores and storage with non-time-sensitive features, direct storage is eligible for immediate eviction from the write combination buffer and therefore cannot be combined to the same address as newer stores (including direct storage). Older WC and non-temporary stores that remain in the write combination buffer can be combined to the same address as newer direct storage.

[0839] Because the WC protocol used by direct storage follows a weakly ordered memory consistency model, protection operations should follow the MOVDIRI instructions to enforce ordering when necessary.

[0840] Direct storage published by MOVDIRI to the destination is aligned to 4-byte boundaries and guarantees atomicity for 4-byte writes. This means that data arrives at the destination in a single, non-fragmented 4-byte (or 8-byte) write transaction. If the destination is not aligned to the write size, the direct storage published by MOVDIRI is split and arrives at the destination in two parts. Each part of this split direct storage is not merged with the newer storage but can arrive at the destination in any order.

[0841] Figure 59 An embodiment of a processor-executed method for processing MOVDIRI instructions is shown. For example, the hardware detailed herein is used.

[0842] At 5901, an instruction is fetched. For example, MOVDIRI is fetched. The MOVDIRI instruction includes an opcode (and in some embodiments a prefix), a destination field representing the destination operand, and a source field representing the source register operand.

[0843] The extracted instructions are decoded at 5903. For example, the MOVDIRI instruction is decoded by a decoding circuit such as the one detailed in this article.

[0844] At position 5905, the data value associated with the source operand of the decoded instruction is retrieved. Additionally, in some embodiments, the instruction is scheduled.

[0845] At 5907, the decoded instruction is executed by, for example, the execution circuitry (hardware) detailed herein, to move double-word-sized data from the source register operand to the destination register operand without buffering the data.

[0846] In some embodiments, a submit or exit instruction is given at 5909.

[0847] Moves 64 bytes as direct storage from the source memory address to the destination memory address with atomic 64-byte writes. The source operand is a regular memory operand. The destination operand is the memory location specified in a general-purpose register. The register contents are interpreted as offsets within an ES segment, without any segment overwriting. In 64-bit mode, the register operand width is 64 bits (or 32 bits). Outside of 64-bit mode, the register width is 32 bits or 16 bits. MOVDIR64B requires the destination address to be 64-byte aligned. Alignment constraints are not enforced on the source operand.

[0848] The MOVDIR64B reads 64 bytes from the source memory address and performs a 64-byte direct store operation on the destination address. The load operation follows the regular read ordering based on the source address's memory type. Direct store is implemented by writing data using the Write Combination (WC) memory type protocol. Using this protocol, the processor may not write data to the cache hierarchy and may not fetch the corresponding cache line from memory into the cache hierarchy. If the destination address is cached, the line is written back from the cache (if modified) and invalidated before direct store.

[0849] Unlike storage with non-time indication (which allows the destination's UC / WP memory type to override non-time indication), direct storage can follow the WC memory type protocol regardless of the destination address's memory type (including UC / WP types).

[0850] Unlike WC storage and storage with non-time-sensitive information, direct storage is eligible for immediate eviction from the write combination buffer and therefore cannot be combined to the same address as newer storage (including direct storage). Older WC and non-temporary storage that remain in the write combination buffer can be combined to the same address as newer direct storage.

[0851] Because the WC protocol used by direct memory follows a weakly ordered memory consistency model, protection operations should follow the MOVDIR64B instructions to enforce ordering when necessary.

[0852] No atomicity guarantee is provided for 64-byte load operations from the source address, and processor implementations may use multiple load operations to read 64 bytes. The 64-byte direct store issued by the MOVDIR64B guarantees the atomicity of the 64-byte write completion. This means that data arrives at its destination in a single, non-decomposed 64-byte write transaction.

[0853] Figure 60 An embodiment of a processor-executed method for processing MOVDIRI64B instructions is shown. For example, the hardware detailed herein is used.

[0854] At 6001, fetch the instruction. For example, fetch MOVDIRI64B. The MOVDIRI64B instruction includes an opcode (and in some embodiments a prefix), a destination field representing the destination operand, and a source field representing the source register operand.

[0855] The extracted instructions are decoded at 6003. For example, the MOVDIRI64B instruction is decoded by a decoding circuit such as the one detailed in this article.

[0856] At 6005, the data value associated with the source operand of the decoded instruction is retrieved. Additionally, in some embodiments, the instruction is scheduled.

[0857] At 6007, the decoded instructions are executed by, for example, the execution circuitry (hardware) detailed herein, to move 64 bytes of data from the source register operand to the destination register operand without buffering the data.

[0858] In some embodiments, a submit or exit instruction is given at 6009.

[0859] In one implementation, the ENQCMD command queues a 64-byte command using a non-published write with atomicity from the source memory address (second operand) to the device shared work queue (SWQ) memory address in the destination operand. T...

Claims

1. A computing system comprising: A decoder is used to decode instructions having an opcode, a field for a first packaged data source operand, one or more fields for a second to Nth packaged data source operand, and a field for a packaged data destination operand; and The execution circuitry is used to execute the decoded instructions for each position of a packed data element of the second to Nth packed data source operands: Multiply the data element at the specified position of the packed data element in the first packed data source operand by the corresponding data element at the specified position of the packed data element in the first packed data source operand to generate a temporary result. The first sum of the temporary results is generated. The first sum of the temporary results is added to the data element at the corresponding position of the packed data element in the destination operand to generate a second sum, and The second sum, obtained by adding the first sum of the temporary results to the data element at the corresponding packaged data element position of the packaged data destination operand, is stored in the corresponding packaged data element position of the packaged data destination operand.

2. The computing system according to claim 1, wherein, The second to Nth packaged data source operands contain data elements of the matrix.

3. The computing system according to claim 1, wherein, The destination operand of the packaged data contains data elements of a matrix.

4. The computing system according to claim 1, wherein, The one or more fields used for the second to Nth packed data source operands specify a packed data register for storing one of the second to Nth packed data source operands, and the other packed data source operands of the second to Nth packed data source operands are stored in a packed data register contiguous with the specified packed data register.

5. The computing system according to claim 1, wherein, The second to Nth packaged data source operands contain more than two packaged data source operands.

6. A calculation method, comprising: The instruction is decoded, and the instruction has an opcode, a field for a first packaged data source operand, one or more fields for a second to Nth packaged data source operand, and a field for a packaged data destination operand; and Execute the decoded instructions for each packaged data element position of the second to Nth packaged data source operands: Multiply the data element at the specified position of the packed data element in the first packed data source operand by the corresponding data element at the specified position of the packed data element in the first packed data source operand to generate a temporary result. The first sum of the temporary results is generated. The first sum of the temporary results is added to the data element at the corresponding position of the packed data element in the destination operand to generate a second sum, and The second sum, obtained by adding the first sum of the temporary results to the data element at the corresponding packaged data element position of the packaged data destination operand, is stored in the corresponding packaged data element position of the packaged data destination operand.

7. The calculation method according to claim 6, wherein, The second to Nth packaged data source operands contain data elements of the matrix.

8. The calculation method according to claim 6, wherein, The destination operand of the packaged data contains data elements of a matrix.

9. The calculation method according to claim 6, wherein, The one or more fields used for the second to Nth packed data source operands specify a packed data register for storing one of the second to Nth packed data source operands, and the other packed data source operands of the second to Nth packed data source operands are stored in a packed data register contiguous with the specified packed data register.

10. The calculation method according to claim 6, wherein, The second to Nth packaged data source operands contain more than two packaged data source operands.

11. One or more computer-readable media, the computer-readable medium comprising instructions that, in response to being executed by one or more processors, cause the one or more processors to perform the method of any one of claims 6 to 10.

12. A computing device comprising means for performing the method of any one of claims 6 to 10.

13. A computer program product comprising instructions that, in response to being executed on a computing device, cause the computing device to perform the method of any one of claims 6 to 10.

14. A memory system comprising: One or more cache units within the core; One or more shared cache units coupled to one or more cache units within the core; An integrated memory controller unit coupled to at least one external memory; as well as A ring-based interconnect unit configured to interconnect the following: one or more cache units within the core, one or more shared cache units, and the integrated memory controller unit; One or more shared cache units include at least one L2 cache and at least one L3 cache; Among these, consistency is maintained between the one or more shared cache units and the core.

Images