Method and apparatus for high-performance page fault handling of multitenant scalable accelerators

Abstract

The present invention provides a method and apparatus for high-performance page fault processing. [Solution] The device comprises one or more accelerator engines for processing work descriptors submitted to a plurality of work queues by a client, and fault handling hardware logic associated with one or more accelerator engines, which implements a specified page fault handling mode for each of the plurality of work queues, the page fault handling mode including a first page fault handling mode and a second page fault handling mode.

Description

Technical Field

[0001] The present invention generally relates to the field of computer processors. More specifically, the present invention relates to methods and apparatus for high performance page fault handling in a multi-tenant scalable accelerator.

Background Art

[0002] An instruction set or instruction set architecture (ISA) is a part of a computer architecture related to programming, including native data types, instructions, register architecture, addressing modes, memory architecture, interrupt and exception handling, and external input and output (I / O). Note that in this specification, the term "instruction" generally refers to a macro instruction. A macro instruction is an instruction supplied to the processor for execution, whereas a micro instruction or micro-op is the result of the processor's decoder decoding a macro instruction. A micro instruction or micro-op can be configured to instruct the execution unit of the processor to execute operations for implementing the logic associated with the macro instruction.

[0003] An ISA is distinct from a microarchitecture, which is a set of processor design techniques used to implement an instruction set. Processors with different microarchitectures can share a common instruction set. For example, Intel® Pentium® 4 processors, Intel® Core® processors, and processors from Advanced Micro Devices, Inc. of Sunnyvale, California, implement nearly the same version of the x86 instruction set (with some extensions added to newer versions), but have different internal designs. For example, the same register architecture of an ISA may be implemented differently on different microarchitectures using well-known techniques, including one or more dynamically allocated physical registers using dedicated physical registers and register renaming mechanisms (e.g., the use of a Register Alias ​​Table (RAT), Reorder Buffer (ROB), and Retirement Register File). Unless otherwise noted, in this specification, the terms register architecture, register file, and register are used to refer to what is visible to the software / programmer and how instructions specify registers. Where distinction is necessary, the adjectives "logical," "architectural," or "software visible" are used to refer to registers / files in a register architecture, while different adjectives are used to specify registers in a given microarchitecture (e.g., physical register, reorder buffer, retirement register, register pool). [Brief explanation of the drawing]

[0004] The present invention can be fully understood from the following detailed description, in conjunction with the following drawings.

[0005] A better understanding of the present invention can be obtained from the following detailed description in conjunction with the following drawings.

[0006] [Figure 1]An example of a computer system architecture is shown.

[0007] [Figure 2] This indicates a processor that includes multiple cores.

[0008] [Figure 3A] This shows multiple stages in the processing pipeline.

[0009] [Figure 3B] Details of one embodiment of the core are shown below.

[0010] [Figure 4] An execution circuit according to one embodiment is shown.

[0011] [Figure 5] This shows one embodiment of the register architecture.

[0012] [Figure 6] An example of the instruction format is shown below.

[0013] [Figure 7] An addressing technique according to one embodiment is shown.

[0014] [Figure 8] One embodiment of the instruction prefix is ​​shown.

[0015] [Figure 9A] This document illustrates an embodiment of how the R, X, and B fields of the prefix are used. [Figure 9B] This document illustrates an embodiment of how the R, X, and B fields of the prefix are used. [Figure 9C] This document illustrates an embodiment of how the R, X, and B fields of the prefix are used. [Figure 9D]Embodiments showing how the prefix R, X, and B fields are used are presented.

[0016] [Figure 10A] An example of a second instruction prefix is shown. [Figure 10B] An example of a second instruction prefix is shown.

[0017] [Figure 11] The payload bytes of one embodiment of an instruction prefix are shown.

[0018] [Figure 12] Instruction conversion and binary conversion implementations are shown.

[0019] [Figure 13] A scalable device according to one embodiment of the invention is shown.

[0020] [Figure 14] An example of a method for handling a page fault is shown.

[0021] [Figure 15] Another example of a method for handling a page fault according to one embodiment of the invention is shown.

[0022] [Figure 16] An accelerator having a work acceptance unit, a work dispatcher, and a work execution unit is shown.

[0023] [Figure 17] A transaction between an accelerator and a host root complex is shown.

[0024] [Figure 18] An accelerator having fault handling logic according to an embodiment of the present invention is shown.

[0025] [Figure 19A] The descriptors according to embodiments of the present invention are shown.

[0026] [Figure 19B] A batch descriptor according to an embodiment of the present invention is shown.

[0027] [Figure 20] This shows the completion record generated by the accelerator.

[0028] [Figure 21] This shows an event log entry generated by an accelerator according to an embodiment of the present invention.

[0029] [Figure 22] This shows an accelerator equipped with multiple control registers.

[0030] [Figure 23] An example of a batch completion record is shown below.

[0031] [Figure 24] An example of a drain descriptor and drain completion record is shown.

[0032] [Figure 25] This describes an accelerator with multiple engines and work queues, at least some of which use a page request service.

[0033] [Figure 26] The method according to embodiments of the present invention is shown. [Modes for carrying out the invention]

[0034] In the following description, numerous specific details are provided for illustrative purposes to provide a complete understanding of the embodiments of the invention described below. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without some of these specific details. In other examples, well-known structures and apparatus are shown in block diagram form to avoid obscuring the basic principles of the embodiments of the invention. Exemplary computer architecture

[0035] Exemplary computer architectures are described in detail below. Other system designs and configurations known in the art are also suitable for laptops, desktops, handheld PCs, personal digital assistants, engineering workstations, servers, network devices, network hubs, switches, embedded processors, digital signal processors (DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, mobile phones, portable media players, handheld devices, and various other electronic devices. Generally, a variety of systems or electronic devices capable of incorporating the processors and / or other execution logic disclosed herein are generally suitable.

[0036] Figure 1 shows an exemplary system embodiment. The multiprocessor system 100 is a point-to-point interconnect system and includes multiple processors, including a first processor 170 and a second processor 180, coupled via a point-to-point interconnect 150. In some embodiments, the first processor 170 and the second processor 180 are homogeneous. In some embodiments, the first processor 170 and the second processor 180 are heterogeneous.

[0037] Processors 170 and 180 are shown, each including an Integrated Memory Controller (IMC) unit circuit 172 and 182, respectively. Processor 170 also includes point-to-point (PP) interfaces 176 and 178 as part of its multiple interconnect controller units. Similarly, the second processor 180 includes PP interfaces 186 and 188. Processors 170 and 180 may exchange information using the PP interface circuits 178 and 188 via the point-to-point (PP) interconnect 150. The IMCs 172 and 182 connect processors 170 and 180 to their respective memories, i.e., memories 132 and 134, which may be part of the main memory locally attached to each processor.

[0038] Processors 170 and 180 may exchange information with chipset 190 via individual P-P interconnects 152 and 154 using point-to-point interface circuits 176, 194, 186, and 198, respectively. Chipset 190 may optionally exchange information with coprocessor 138 via high-performance interface 192. In some embodiments, coprocessor 138 is a dedicated processor such as a high-throughput MIC processor, a network or communications processor, a compression engine, a graphics processor, a GPGPU, or an embedded processor.

[0039] A shared cache (not shown) may be located within or outside of either processor 170, 180, and may also be connected to the processors via a P-P interconnect. As a result, when the processors are in low-power mode, the local cache information of one or both processors may be stored in the shared cache.

[0040] The chipset 190 may be coupled to a first interconnect 116 via interface 196. In some embodiments, the first interconnect 116 may be an interconnect such as a Peripheral Component Interconnection (PCI) interconnect, a PCI Express interconnect, or another I / O interconnect. In some embodiments, one of the interconnects may be coupled to a power control unit (PCU) 117, which may include circuitry, software, and / or firmware for performing power management operations related to the processors 170, 180, and / or coprocessor 138. The PCU 117 provides control information to a voltage regulator so that it causes the voltage regulator to generate an appropriate regulated voltage. The PCU 117 also provides control information to control the generated operating voltage. In various embodiments, the PCU 117 may include various power management logic units (circuitry) to perform hardware-based power management. Such power management may be fully processor-controlled (e.g., triggered by various processor hardware and workload and / or power, thermal, or other processor constraints), and / or power management may be performed in response to external sources (such as the platform or power management source or system software).

[0041] The PCU117 is shown as existing as logic separate from the processor 170 and / or processor 180. In other cases, the PCU117 may run on one or more cores (not shown) of the processor 170 or 180. In some cases, the PCU117 may be implemented as a microcontroller (dedicated or general purpose), or as other control logic configured to run its own dedicated power management code, which may be referred to as P-code. In yet another embodiment, the power management operations to be performed by the PCU117 may be implemented externally on the processor, such as in a separate power management integrated circuit (PMIC) or other component outside the processor. In yet another embodiment, the power management operations to be performed by the PCU117 may be implemented within the BIOS or other system software.

[0042] Various I / O devices 114 may be coupled to the first interconnect 116 along an interconnect (bus) bridge 118 that connects the first interconnect 116 to the second interconnect 120. In some embodiments, one or more additional processors 115, such as multiple coprocessors, multiple high-throughput MIC processors, multiple accelerators for a GPGPU (e.g., multiple graphics accelerators or multiple digital signal processing (DSP) units), multiple field-programmable gate arrays (FPGAs), or any other processor, may be coupled to the first interconnect 116. In some embodiments, the second interconnect 120 may be a low-pin-count (LPC) interconnect. Various devices may be coupled to the second interconnect 120, including, for example, a keyboard and / or mouse 122, a communication device 127, and a storage unit circuit 128. In some embodiments, the storage unit circuit 128 may be a disk drive or other mass storage device that may contain instructions / code and data 130. Furthermore, the audio I / O 124 may be coupled to the second interconnect 120. It should be noted that architectures other than the point-to-point architecture described above are possible. For example, instead of the point-to-point architecture, a system such as the multiprocessor system 100 may implement a multidrop interconnect or other such architectures. Exemplary core architectures, processors, and computer architectures

[0043] Processor cores may be implemented in different ways and for different purposes in different processors. For example, such core implementations may include 1) general-purpose in-order cores for general-purpose computing, 2) high-performance general-purpose out-of-order cores for general-purpose computing, and 3) dedicated cores primarily for graphics and / or scientific (throughput) computing. Different processor implementations may include 1) a CPU containing one or more general-purpose in-order cores and / or one or more general-purpose out-of-order cores for general-purpose computing, and 2) a coprocessor containing one or more dedicated cores primarily for graphics and / or scientific (throughput) computing. Such different processors result in different computer system architectures, which may include: 1) a coprocessor on a separate chip of the CPU; 2) a coprocessor on a separate die in the same package as the CPU; 3) a coprocessor on the same die as the CPU (in this case, such a coprocessor may be referred to as dedicated logic, or dedicated core, such as integrated graphics and / or scientific (throughput) logic); and 4) a system-on-a-chip that may include the aforementioned coprocessor and additional functionality on the same die as the described CPU (which may be referred to as an application core or application processor). Next, an exemplary core architecture will be described, followed by a description of exemplary processor and computer architectures.

[0044] Figure 2 shows a block diagram of an embodiment of processor 200 that may have more than one core, may have an integrated memory controller, and may have integrated graphics. The solid box shows processor 200 having a single core 202A, a system agent 210, and a set of one or more interconnect controller unit circuits 216, while optional additional dashed boxes show alternative example processor 200 having multiple multicores 202(A)-(N), a set of one or more integrated memory controller unit circuits 214 within the system agent unit circuit 210, and dedicated logic 208, and a set of one or more interconnect controller unit circuits 216. Note that processor 200 may be one of the processors 170 or 180 in Figure 1, or coprocessors 138 or 115.

[0045] Therefore, different implementations of processor 200 may include: 1) a CPU having dedicated logic 208 (which may include one or more cores, but are not shown) that is integrated graphics and / or scientific (throughput) logic, and cores 202(A)-(N) (e.g., general-purpose in-order cores, general-purpose out-of-order cores, or a combination of the two), 2) a coprocessor having a large number of dedicated cores 202(A)-(N) primarily for graphics and / or scientific (throughput), and 3) a coprocessor having a large number of general-purpose in-order cores 202(A)-(N). Therefore, processor 200 may be a general-purpose processor, coprocessor, or dedicated processor, such as a network or communications processor, a compression engine, a graphics processor, a GPGPU (general-purpose graphics processing unit circuit), a high-throughput multi-integrated core (MIC) coprocessor (which may include 30 or more cores), or an embedded processor. The processor may be implemented on one or more chips. The processor 200 may be part of one or more substrates and / or may be mounted on them using any of many process technologies, such as BiCMOS, CMOS, or NMOS.

[0046] The memory hierarchy includes one or more levels of cache unit circuits 204(A)-(N) within the cores 202(A)-(N), a set of one or more shared cache unit circuits 206, and external memory (not shown) coupled to a set of integrated memory controller unit circuits 214. The set of one or more shared cache unit circuits 206 may include one or more intermediate-level caches such as Level 2 (L2), Level 3 (L3), Level 4 (L4), etc., or other levels of caches such as the Last Level Cache (LLC), and / or combinations thereof. In some embodiments, a ring-based interconnect network circuit 212 interconnects dedicated logic 208 (e.g., integrated graphics logic), the set of shared cache unit circuits 206, and the system agent unit circuit 210, while alternative embodiments employ any number of known techniques for interconnecting such units. In some embodiments, coherency is maintained between the one or more shared cache unit circuits 206 and the cores 202(A)-(N).

[0047] In some embodiments, one or more of the cores 202(A)-(N) are capable of multithreading. The system agent unit circuit 210 includes the components that coordinate and operate the cores 202(A)-(N). The system agent unit circuit 210 may include, for example, a power control unit (PCU) circuit and / or a display unit circuit (not shown). The PCU may be, or include, the logic and / or components necessary to coordinate the power state of the cores 202(A)-(N) and dedicated logic 208 (e.g., integrated graphics logic). The display unit circuit is for driving one or more externally connected displays.

[0048] Cores 202(A)-(N) may be homogeneous or heterogeneous in terms of their architectural instruction sets. That is, two or more cores 202(A)-(N) may be capable of executing the same instruction set, while other cores may be capable of executing only that instruction set or a subset of a different instruction set. Exemplary Core Architecture Block diagrams of in-order and out-of-order cores

[0049] Figure 3A is a block diagram showing both an exemplary in-order pipeline and an exemplary register renaming out-of-order issue / execution pipeline according to an embodiment of the present invention. Figure 3B is a block diagram showing both an exemplary embodiment of an in-order architecture core and an exemplary register renaming out-of-order issue / execution architecture core included in a processor according to an embodiment of the present invention. In Figures 3A to 3B, solid boxes indicate the in-order pipeline and in-order core, while optional dashed boxes indicate the register renaming, out-of-order issue / execution pipeline and core. The out-of-order embodiments will be described assuming that the in-order embodiments are a subset of the out-of-order embodiments.

[0050] In Figure 3A, the processor pipeline 300 includes a fetch stage 302, a decode stage of optional length 304, a decode stage 306, an optional allocation stage 308, an optional rename stage 310, a scheduling (also known as dispatch or issue) stage 312, an optional register read / memory read stage 314, an execution stage 316, a write-back / memory write stage 318, an optional exception handling stage 322, and an optional commit stage 324. One or more operations may be performed in each of those processor pipeline stages. For example, during the fetch stage 302, one or more instructions may be fetched from instruction memory; during the decode stage 306, one or more fetched instructions may be decoded, an address using a transferred register port (e.g., a load-store unit (LSU) address) may be generated, and a branch transfer (e.g., an immediate offset or link register (LR)) may be performed. In one embodiment, the decode stage 306 and the register read / memory read stage 314 may be combined into a single pipeline stage. In one embodiment, during the execution stage 316, the decoded instruction may be executed, LSU addresses / data piped to an Advanced Microcontroller Bus (AHB) interface may be executed, multiplication and addition operations may be performed, arithmetic operations with branch results may be performed, and so on.

[0051] As an example, an exemplary register rename, out-of-order issue / execution core architecture may implement pipeline 300 as follows: 1) Instruction fetch 338 executes fetch and length decode stages 302 and 304; 2) Decode unit circuit 340 executes decode stage 306; 3) Rename / allocate unit circuit 352 executes allocation stage 308 and rename stage 310; 4) Scheduler unit circuit 356 executes schedule stage 312; 5) Physical register file unit circuit 358 and memory unit circuit 370 execute register read / memory read stage 314, and execution cluster 360 executes execution stage 316; 6) Memory unit circuit 370 and physical register file unit circuit 358 execute write-back / memory write stage 318; 7) Various units (unit circuits) may be associated with exception handling stage 322; 8) Retirement unit circuit 354 and physical register file unit circuit 358 execute commit stage 324.

[0052] Figure 3B shows a processor core 390 including a front-end unit circuit 330 coupled to an execution engine unit circuit 350, both of which are coupled to a memory unit circuit 370. The core 390 may be a reduced instruction set computing (RISC) core, a composite instruction set computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. Another option is that the core 390 may be a dedicated core, such as a network or communications core, a compression engine, a coprocessor core, a general-purpose computing graphics processing unit (GPGPU) core, or a graphics core.

[0053] The front-end unit circuit 330 may include a branch prediction unit circuit 332 coupled to an instruction cache unit circuit 334, which is coupled to an instruction translation lock-aside buffer (TLB) 336, which is coupled to an instruction fetch unit circuit 338, which is coupled to a decode unit circuit 340. In one embodiment, the instruction cache unit circuit 334 is included in the memory unit circuit 370 rather than the front-end unit circuit 330. The decode unit circuit 340 (or decoder) decodes an instruction and may produce as output one or more microoperations, microcode entry points, microinstructions, or other instructions, or other control signals that are decoded from the original instruction, or that reflect or are derived from them. The decode unit circuit 340 may further include an address generation unit circuit (AGU, not shown). In one embodiment, the AGU may generate an LSU address using a transferred register port and may further perform branch transfers (e.g., immediate offset branch transfer, LR register branch transfer, etc.). The decode unit circuit 340 may be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), and microcode read-only memory (ROM). In one embodiment, the core 390 includes a microcode ROM (not shown) or other medium for storing microcode for specific macro instructions (e.g., in the decode unit circuit 340, or otherwise in the front-end unit circuit 330). In one embodiment, the decode unit circuit 340 includes a micro-operation (micro-op) or arithmetic cache (not shown) to hold / cache decoded arithms, microtags, or microoperations generated during decoding or other stages of the processor pipeline 300. The decode unit circuit 340 may be coupled with a rename / assign unit circuit 352 in the execution engine unit circuit 350.

[0054] The execution engine unit circuit 350 includes a rename / assign unit circuit 352 coupled to a retirement unit circuit 354, and a set of one or more scheduler circuits 356. The scheduler circuits 356 represent any number of different schedulers, including multiple reservation stations, a central instruction window, etc. In some embodiments, the scheduler circuits 356 include arithmetic logic unit (ALU) scheduler / scheduling circuits, ALU queues, arithmetic generation unit (AGU) scheduler / scheduling circuits, AGU queues, etc. The scheduler circuits 356 are coupled to a physical register file circuit 358. Each of the multiple physical register file circuits 358 represents one or more physical register files, each storing one or more different data types, such as scalar integers, scalar floating-point numbers, packed integers, packed floating-point numbers, vector integers, vector floating-point numbers, and status (e.g., an instruction pointer, which is the address of the next instruction to be executed). In one embodiment, the physical register file unit circuit 358 includes a vector register unit circuit, a write mask register unit circuit, and a scalar register unit circuit. These register units may provide architecture vector registers, vector mask registers, general-purpose registers, etc. The physical register file unit circuit 358 is superimposed by a retirement unit circuit 354 (also known as a retirement queue or retirement queue) to show various ways in which register renaming and out-of-order execution may be implemented (e.g., using a reorder buffer (ROB) and a retirement register file, using a future file, a history buffer, and a retirement register file, using a register map and a pool of multiple registers). The retirement unit circuit 354 and the physical register file unit circuit 358 are coupled to an execution cluster 360. The execution cluster 360 includes one or more sets of execution unit circuits 362 and one or more sets of memory access circuits 364.The execution unit circuit 362 may perform various arithmetic, logic, floating-point, or other types of operations (e.g., shift, addition, subtraction, multiplication) on various types of data (e.g., scalar floating-point, packed integer, packed floating-point, vector integer, vector floating-point). Some embodiments may include many execution units or execution unit circuits dedicated to multiple specific functions or multiple sets of multiple functions, while other embodiments may include only one execution unit circuit or multiple execution units / execution unit circuits that perform all functions. Since certain embodiments form separate pipelines for certain types of data / operations (e.g., scalar integer pipelines, scalar floating-point / packed integer / packed floating-point / vector integer / vector floating-point pipelines, and / or memory access pipelines, each having its own scheduler circuit, physical register file unit circuit, and / or execution cluster; in the case of a separate memory access pipeline, certain embodiments are implemented such that only the execution cluster of this pipeline has the memory access unit circuit 364), the scheduler circuit 356, physical register file unit circuit 358, and execution cluster 360 may be shown as being multiple in some cases. It should also be understood that when separate pipelines are used, one or more of these pipelines may be out-of-order issue / execution, while the rest may be in-order.

[0055] In some embodiments, the execution engine unit circuit 350 may perform load / store unit (LSU) address / data pipeline processing, address phase and write-back, data phase load, store and branch to an advanced microcontroller bus (AHB) interface (not shown).

[0056] A set of memory access circuits 364 are coupled to a memory unit circuit 370 which includes a data TLB unit circuit 372 coupled to a data cache circuit 374 coupled to a level 2 (L2) cache circuit 376. In one exemplary embodiment, the memory access unit circuit 364 may include a load unit circuit, a store address unit circuit, and a store data unit circuit, each of which is coupled to a data TLB circuit 372 in the memory unit circuit 370. The instruction cache circuit 334 is further coupled to a level 2 (L2) cache unit circuit 376 in the memory unit circuit 370. In one embodiment, the instruction cache unit 334 and the data cache unit 374 are combined with an L2 cache unit circuit 376, a level 3 (L3) cache unit circuit (not shown), and / or a single instruction and data cache (not shown) in main memory. The L2 cache unit circuit 376 is coupled to one or more other levels of caches and ultimately to main memory.

[0057] The Core 390 may support one or more instruction sets (e.g., the x86 instruction set (with several extensions added in newer versions), the MIPS instruction set, the ARM instruction set (with optional additional extensions such as NEON)) including the instructions described herein. In one embodiment, the Core 390 includes logic to support packed data instruction set extensions (e.g., AVX1, AVX2), thereby enabling arithmetic operations used by many multimedia applications to be performed using packed data. Exemplary execution unit circuit

[0058] Figure 4 shows an embodiment of an execution unit circuit, such as the execution unit circuit 362 in Figure 3B. As shown, the execution unit circuit 362 may include one or more ALU circuits 401, a vector / SIMD unit circuit 403, a load / storage unit circuit 405, and / or a branch / jump unit circuit 407. The ALU circuit 401 performs integer and / or Boolean operations. The vector / SIMD unit circuit 403 performs vector / SIMD operations on packed data (such as SIMD / vector registers). The load / storage unit circuit 405 executes load and store instructions, loading data from memory into registers or storing data from registers into memory. The load / storage unit circuit 405 may also generate addresses. The branch / jump unit circuit 407, depending on the instruction, causes a branch or jump to a memory address. The floating-point unit (FPU) circuit 409 performs floating-point operations. The width of the execution unit circuit 362 varies depending on the embodiment and can range from 16 bits to 1,024 bits. In some embodiments, two or more smaller execution units are logically combined to form a larger execution unit (for example, two 128-bit execution units are logically combined to form a 256-bit execution unit). Exemplary Register Architecture

[0059] Figure 5 is a block diagram of the register architecture 500 according to several embodiments. As shown, there are vector / SIMD registers 510 ranging in width from 128 bits to 1,024 bits. In some embodiments, the vector / SIMD register 510 is physically 512 bits, and depending on the mapping, only some of the lower bits are used. For example, in some embodiments, the vector / SIMD register 510 is a 512-bit ZMM register, with the lower 256 bits used by the YMM register and the lower 128 bits used by the XMM register. Thus, there is a register overlay. In some embodiments, the vector length field is selected from the maximum length and one or more other shorter lengths, each of which is half the length of the aforementioned length. Multiple scalar operations are operations performed at the lowest data element positions in the ZMM / YMM / XMM registers. The higher data element positions are either left in the same state as those prior to the instruction, or zeroed out depending on the embodiment.

[0060] In some embodiments, the register architecture 500 includes write mask / predicate registers 515. For example, in some embodiments, there are eight write mask / predicate registers (sometimes referred to as k0 through k7) of sizes 16 bits, 32 bits, 64 bits, or 128 bits, respectively. The write mask / predicate registers 515 may enable merging (e.g., enabling any set of elements of a destination to be protected from updating during the execution of any operation) and / or zeroing (e.g., a zeroing vector mask allows any set of elements of a destination to be zeroed out during the execution of any operation). In some embodiments, each data element position in a given write mask / predicate register 515 corresponds to a data element position in a destination. In other embodiments, the write mask / predicate registers 515 are scalable and consist of a set number of enable bits for a given vector element (e.g., 8 enable bits for every 64-bit vector element).

[0061] The register architecture 500 includes several general-purpose registers 525. These registers may be 16-bit, 32-bit, 64-bit, etc., and may be used for scalar operations. In some embodiments, these registers are referred to by the names RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, and R8 through R15.

[0062] In some embodiments, the register architecture 500 includes scalar floating-point registers 545 used for scalar floating-point operations on 32 / 64 / 80-bit floating-point data using x87 instruction set extensions, or for performing operations on 64-bit packed integer data, and for holding operands for some operations performed between MMX and XMM registers.

[0063] One or more flag registers 540 (e.g., EFLAGS, RFLAGS, etc.) store status and control information for arithmetic, comparison, and system operation. For example, one or more flag registers 540 may store condition code information such as carry, parity, auxiliary carry, zero, sign, and overflow. In some embodiments, one or more flag registers 540 are called program status and control registers.

[0064] The segment register 520 contains segment points for use in accessing memory. In some embodiments, these registers are referred to by the names CS, DS, SS, ES, FS, and GS.

[0065] Machine-specific registers (MSRs) 535 control and report on processor performance. Most MSRs 535 handle system-related functions but are not accessible to application programs. Machine check registers 560 consist of error reporting MSRs used to detect and report control, status, and hardware errors.

[0066] One or more instruction pointer registers 530 store instruction pointer values. Control registers 555 (e.g., CR0-CR4) determine the processor's operating mode (e.g., processors 170, 180, 138, 115, and / or 200) and the characteristics of the task currently being performed. Debug registers 550 control and enable monitoring of the debugging operation of the processor or core.

[0067] The memory management register 565 specifies the location of data structures used in protected mode memory management. These registers may include the GDTR, IDRT, task register, and LDTR register.

[0068] Alternative embodiments of the present invention may use wider or narrower registers. In addition, alternative embodiments of the present invention may use more, fewer, or different register files and registers. Instruction set

[0069] An instruction set architecture (ISA) may include one or more instruction formats. A given instruction format may define various fields to specify (e.g., number of bits, bit positions), in particular the operation to be performed (e.g., opcode) and the operands and / or other data fields (e.g., masks) on which that operation is performed. Some instruction formats can be further broken down through the definition of multiple instruction templates (or multiple subformats). For example, an instruction template of a given instruction format may be defined to have different subsets of the fields of the instruction format (the fields included are usually in the same order, but at least some have different bit positions because they include fewer fields), and / or may be defined to have certain fields that are interpretably different. Thus, each instruction in an ISA is expressed using a given instruction format (and, if defined, a specific one of the multiple instruction templates of that instruction format) and includes fields for specifying the operation and operands. For example, the exemplary ADD instruction has an instruction format that includes a specific opcode, an opcode field for specifying that opcode, and operand fields (source 1 / destination and source 2) for selecting operands. The occurrence of this ADD instruction in the instruction stream will have specific content in the operand field for selecting a particular operand. Example instruction format

[0070] Embodiments of the instructions described herein can be embodied in different formats. In addition, exemplary systems, architectures, and pipelines are described below in detail. Embodiments of the instructions can be executed on, but are not limited to, such systems, architectures, and pipelines.

[0071] Figure 6 shows an embodiment of the instruction format. As shown, an instruction may include, but is not limited to, multiple components, including one or more fields for one or more prefixes 601, an opcode 603, addressing information 605 (e.g., register identifier, memory addressing information), a displacement value 607, and / or an immediate value 609. Note that some instructions may utilize some or all of the fields in the format, while others may utilize only the field for the opcode 603. In some embodiments, the order shown is the order in which those fields should be encoded; however, it should be understood that in other embodiments, those fields may be encoded in a different order, combination, etc.

[0072] The prefix field 601 qualifies an instruction when used. In some embodiments, one or more prefixes are used to iterate string instructions (e.g., 0xF0, 0xF2, 0xF3, etc.), provide section overrides (e.g., 0x2E, 0x36, 0x3E, 0x26, 0x64, 0x65, 0x2E, 0x3E, etc.), perform bus lock operations, and / or modify operands (e.g., 0x66) and address sizes (e.g., 0x67). Certain instructions require mandatory prefixes (e.g., 0x66, 0xF2, 0xF3, etc.). Some of these prefixes may be considered “legacy” prefixes. Other prefixes, one or more of which are detailed herein, exhibit and / or provide further capabilities, such as specifying certain registers, etc. Other prefixes typically follow the “legacy” prefixes.

[0073] The opcode field 603 is used to define, at least partially, the operation to be performed when the instruction is decoded. In some embodiments, the primary opcode encoded in the opcode field 603 is 1, 2, or 3 bytes long. In other embodiments, the primary opcode may be of a different length. An additional 3-bit opcode field may be encoded in another field, if applicable.

[0074] The addressing field 605 is used to address one or more operands of an instruction, such as locations in memory or one or more registers. Figure 7 shows an embodiment of the addressing field 605. In this explanatory diagram, the optional ModR / M byte 702 and the optional Scale, Index, Base (SIB) byte 704 are shown. The ModR / M byte 702 and the SIB byte 704 are used to encode instructions with up to two operands, each of which is a direct register or valid memory address. Note that each of these fields is optional, and not all instructions will contain one or more of these fields. The MOD R / M byte 702 includes the MOD field 742, the register field 744, and the R / M field 746.

[0075] The contents of the MOD field 742 distinguish between memory access and non-memory access modes. In some embodiments, if the MOD field 742 has the value of b11, register direct addressing mode is used; otherwise, register indirect addressing is used.

[0076] Register field 744 may encode either a destination register operand or a source register operand, or an opcode extension, and is not used to encode any instruction operand. The contents of register index field 744 specify the location of the source or destination operand (either in multiple registers or in memory), either directly or through address generation. In some embodiments, register field 744 is supplemented with additional bits from a prefix (e.g., prefix 601) to enable larger addressing.

[0077] The R / M field 746 may be used to encode an instruction operand that references a memory address, or it may be used to encode either a destination register operand or a source register operand. Note that the R / M field 746 may be combined with the MOD field 742 to define an addressing mode in some embodiments.

[0078] SIB byte 704 contains a scale field 752, an index field 754, and a base field 756 used in address generation. The scale field 752 indicates the scaling factor. The index field 754 specifies the index register to be used. In some embodiments, the index field 754 is supplemented with extra bits from a prefix (e.g., prefix 601) to enable larger addressing. The base field 756 specifies the base register to be used. In some embodiments, the base field 756 is supplemented with extra bits from a prefix (e.g., prefix 601) to enable larger addressing. In practice, the contents of the scale field 752 allow scaling of the contents of the index field 754 for memory address generation (e.g., using 2 to the power of scale × index + base for address generation).

[0079] Some addressing schemes utilize displacement values ​​to generate memory addresses. For example, memory addresses may be generated according to (2 to the power of scale) × index + base + displacement, index × scale + displacement, r / m + displacement, instruction pointer (RIP / EIP) + displacement, register + displacement, etc. The displacement may be a value of 1 byte, 2 bytes, 4 bytes, etc. In some embodiments, the displacement field 607 provides this value. In addition, in some embodiments, the displacement coefficient utilization is encoded in the MOD field of the addressing field 605, which indicates a compressed displacement scheme for calculating the displacement value by multiplying disp8 with a scaling coefficient N determined based on the instruction vector length, the value of b bits, and the input element size. The displacement value is stored in the displacement field 607.

[0080] In some embodiments, the immediate value field 609 specifies the immediate value of the instruction. The immediate value may be encoded as a 1-byte value, a 2-byte value, a 4-byte value, etc.

[0081] Figure 8 shows an embodiment of the first prefix 601(A). In some embodiments, the first prefix 601(A) is an embodiment of the REX prefix. Instructions using this prefix may specify general-purpose registers, 64-bit packed data registers (e.g., single instruction, multiple data (SIMD) registers, or vector registers), and / or control registers and debug registers (e.g., CR8-CR15 and DR8-DR15).

[0082] Instructions using the first prefix 601(A) may specify up to three registers using 3-bit fields, depending on the format, including 1) using the reg field 744 and R / M field 746 of the MOD R / M byte 702, 2) using the MOD R / M byte 702 together with the SIB byte 704, including using the reg field 744, the base field 756, and the index field 754, or 3) using the register fields of the opcode.

[0083] In the first prefix 601(A), bit positions 7:4 are set to 0100. Bit position 3(W) may be used to determine the operand size, but not only the operand width. Therefore, if W=0, the operand size is determined by the code segment descriptor (CS.D), and if W=1, the operand size is 64 bits.

[0084] By adding another bit, 16(2 4 Note that this allows the registers to be addressed, while the Mod R / M reg field 744 and MOD R / MR / M field 746 can each address only 8 registers individually.

[0085] In the first prefix 601(A), bit position 2(R) may be an extension of the Mod R / M reg field 744, which may be used to qualify the Mod R / M reg field 744 when encoding a general-purpose register, a 64-bit packed data register (e.g., an SSE register), or a control or debug register. R is ignored if the Mod R / M byte 702 specifies another register or defines an extended opcode.

[0086] Bit position 1(X)X bits may modify the SIB byte index field 754.

[0087] Bit position B(B)B may modify the base in the MOD R / MR / M field 746 or the SIB byte base field 756, or it may modify the opcode register field used to access a general-purpose register (e.g., general-purpose register 525).

[0088] Figures 9A to 9D illustrate embodiments of how the R, X, and B fields of the first prefix 601(A) are used. Figure 9A shows that when the SIB byte 704 is not used for memory addressing, the R and B fields of the first prefix 601(A) are used to extend the reg field 744 and R / M field 746 of the MOD R / M byte 702. Figure 9B shows that when the SIB byte 704 is not used (register-to-register addressing), the R and B from the first prefix 601(A) are used to extend the reg field 744 and R / M field 746 of the MOD R / M byte 702. Figure 9C shows that when the SIB byte 704 is used for memory addressing, the R, X, and B from the first prefix 601(A) are used to extend the reg field 744, as well as the index field 754 and base field 756 of the MOD R / M byte 702. Figure 9D shows the first prefix 601(A) to B used to extend the reg field 744 of MOD R / M byte 702 when the register is encoded in opcode 603.

[0089] Figures 10A and 10B illustrate embodiments of the second prefix 601(B). In some embodiments, the second prefix 601(B) is an embodiment of the VEX prefix. The encoding of the second prefix 601(B) allows instructions to have more than two operands and allows SIMD vector registers (e.g., vector / SIMD register 510) to be longer than 64 bits (e.g., 128 bits and 256 bits). The use of the second prefix 601(B) provides a three-operand (or more) syntax. For example, previous two-operand instructions performed operations such as A = A + B, which overwrite the source operand. The use of the second prefix 601(B) allows operands to perform non-destructive operations such as A = B + C.

[0090] In some embodiments, the second prefix 601(B) has two forms: a 2-byte form and a 3-byte form. The 2-byte second prefix 601(B) is mainly used for 128-bit, scalar, and some 256-bit instructions, while the 3-byte second prefix 601(B) provides a compact alternative to the first prefix 601(A) and 3-byte opcode instructions.

[0091] Figure 10A shows an embodiment of the 2-byte format of the second prefix 601(B). In one example, the format field 1001 (byte 0 1003) contains the value C5H. In one example, byte 1 1005 contains the value "R" of bit [7]. This value is the complement of the same value for the first prefix 601(A). Bit [2] is used to specify the length (L) of the vector (a value of 0 is a scalar or a 128-bit vector, and a value of 1 is a 256-bit vector). Bits [1:0] provide opcodes that are extensional equivalents to several legacy prefixes (e.g., 00=no prefix, 01=66H, 10=F3H, and 11=F2H). Bits [6:3], shown as vvvv, may be used as follows: 1) Encodes the first source register operand, specified in inverted (one's complement) form and valid for instructions with two or more source operands. 2) Encodes the destination register operand, specified in one's complement form for a particular vector shift. Or, 3) Does not encode any operands, and the field is reserved and should contain some value such as 1111b.

[0092] Instructions using this prefix may use the MOD R / MR / M field 746 to encode an instruction operand that references a memory address, or it may encode either a destination register operand or a source register operand.

[0093] Instructions using this prefix may encode either a destination register operand or a source register operand that is not used to encode any instruction operand, which is treated as an opcode extension, using the Mod R / M reg field 744.

[0094] For instruction syntax that supports four operands, vvvv, MOD R / MR / M field 746, and Mod R / M reg field 744 encode three of the four operands. Bits [7:4] of the immediate value 609 are then used to encode the third source register operand.

[0095] Figure 10B shows an embodiment of the 3-byte format of the second prefix 601(B). In one example, the format field 1011 (byte 0 1013) contains the value C4H. Byte 1 1015 contains, in bits [7:5], the complements of the same value of the first prefix 601(A): "R", "X", and "B". Bits [4:0] of Byte 1 1015 (shown as mmmmm) contain content that optionally encodes one or more suggested leading opcode bytes. For example, 00001 suggests the 0FH leading opcode, 00010 suggests the 0F38H leading opcode, 00011 suggests the leading 0F3AH opcode, and so on.

[0096] Bits [7] of byte 2 1017 are used in the same way as W for the first prefix 601(A). This includes helping to determine the size of the operands that can be promoted. Bit [2] is used to specify the length (L) of the vector (a value of 0 is a scalar or a 128-bit vector, and a value of 1 is a 256-bit vector). Bits [1:0] provide opcodes for several legacy prefixes and their extensional equivalents (e.g., 00=no prefix, 01=66H, 10=F3H, and 11=F2H). Bits [6:3], shown as vvvv, may be used as follows: 1) to encode the first source register operand, specified in inverted (one's complement) form and valid for instructions with two or more source operands; 2) to encode the destination register operand, specified in one's complement form for a particular vector shift. Alternatively, 3) Do not encode any operands, and the field should be reserved and contain some value such as 1111b.

[0097] Instructions using this prefix may use the MOD R / MR / M field 746 to encode an instruction operand that references a memory address, or it may encode either a destination register operand or a source register operand.

[0098] Instructions using this prefix may encode either a destination register operand or a source register operand that is not used to encode any instruction operand, which is treated as an opcode extension, using the Mod R / M reg field 744.

[0099] For instruction syntax that supports four operands, vvvv, MOD R / MR / M field 746, and Mod R / M reg field 744 encode three of the four operands. Bits [7:4] of the immediate value 609 are then used to encode the third source register operand.

[0100] Figure 11 shows an embodiment of the third prefix 601(C). In some embodiments, the first prefix 601(A) is an embodiment of the EVEX prefix. The third prefix 601(C) is a 4-byte prefix.

[0101] The third prefix 601(C) can encode 32 vector registers (e.g., 128-bit, 256-bit, and 512-bit registers) in 64-bit mode. In some embodiments, instructions that utilize write masks / opmasks (see the register descriptions in previous figures such as Figure 5) or predications utilize this prefix. Opmask registers enable conditional processing or selection control. An opmask instruction has an opmask register as its source / destination operand, processes the contents of the opmask register as a single value, and the opmask instruction is encoded using the second prefix 601(B).

[0102] The third prefix 601(C) can encode features specific to an instruction class (for example, packed instructions with "load + op" semantics may support built-in broadcast features, floating-point instructions with rounding semantics may support static rounding features, floating-point instructions with non-rounding arithmetic semantics may support the "suppress all exception features" feature, etc.).

[0103] The first byte of the third prefix 601(C) is, in one example, a format field 1111 having a value of 62H. Subsequent bytes are referred to as payload bytes 1115-1119 and collectively form 24-bit values ​​of P[23:0] that provide specific functionality in the form of one or more fields (as detailed herein).

[0104] In some embodiments, P[1:0] of payload byte 1119 is identical to the two lower mmmm bits. P[3:2] is reserved in some embodiments. Bit P[4](R') enables access to the upper 16 vector register set when combined with P[7] and the ModR / M reg field 744. P[6] may also provide access to the upper 16 vector registers when SIB type addressing is not required. P[7:5] consists of R, X, and B, which are operand designation qualifier bits for vector registers, general-purpose registers, and memory addressing, and when combined with the ModR / M register field 744 and the ModR / MR / M field 746, enables access to the next set of 8 registers beyond the lower 8 registers. P[9:8] provides several legacy prefixes and extensional equivalent opcodes (e.g., 00=no prefix, 01=66H, 10=F3H, and 11=F2H). P

[10] is a fixed value of 1 in some embodiments. P[14:11], denoted as vvvv, may be used as follows: 1) Encodes a first source register operand, specified in inverted (one's complement) form and valid for instructions with two or more source operands; 2) Encodes a destination register operand, specified in one's complement form for a particular vector shift; or 3) Does not encode any operand, and the field is reserved and should contain some value such as 1111b.

[0105] P

[15] is similar to W in the first prefix 601(A) and the second prefix 611(B), and can function as an opcode extension bit or operand size promotion.

[0106] P[18:16] specifies the index of the register in the opmask (write mask) register (e.g., write mask / predicate register 515). In one embodiment, the particular value aaa=000 has special behavior that suggests there is no opmask used for a particular instruction (this can be implemented in various ways, including the use of opmasks that bypass masking hardware or are built into the hardware). When merging, the vector mask allows any set of elements in a destination to be protected from updates during the execution of any operation (specified by basic and extended operations). In another embodiment, the old value of each element of the destination is retained if the corresponding mask bit is 0. In contrast, when zeroing, the vector mask allows any set of elements in a destination to be zeroed out during the execution of any operation (specified by basic and extended operations). In one embodiment, if the corresponding mask bit has a value of 0, the elements of the destination are set to 0. A subset of this functionality is the ability to control the vector length of the operation being performed (i.e., the range of elements being modified from the first to the last element), although the elements being modified do not need to be contiguous. Thus, the opmask field enables partial vector operations, including load, store, arithmetic, and logical operations. Embodiments of the present invention have been described in which the contents of the opmask field select one of many opmask registers containing the opmask to be used (thus indirectly identifying the masking to be performed), but alternative embodiments may, instead of or in addition to this, allow the contents of the mask write field to directly specify the masking to be performed.

[0107] P

[19] , in combination with P[14:11], can encode a second source vector register in a non-destructive source syntax that allows access to the upper 16 vector registers using P

[19] . P

[20] encodes several functions that vary across different instruction classes and can affect the meaning of the vector length / rounding control specifier field (P[22:21]). P

[23] indicates support for merge / write masking (e.g., when set to 0) or support for zeroing and merge / write masking (e.g., when set to 1).

[0108] Exemplary embodiments of encoding registers in an instruction using the third prefix 601(C) are described in detail in the following table. [Table 1] Table 1: 32 Register Support in 64-bit Mode [Table 2] Table 2: Encoded register specifiers in 32-bit mode [Table 3] Table 3: Operative Mask Register Specifier Encoding

[0109] Program code may be applied to input instructions to perform the functions described herein and generate output information. The output information may be applied to one or more output devices in known manner. For the purposes of this application, the processing system includes any system having a processor such as a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0110] The program code may be implemented in a high-level procedural or object-oriented programming language to communicate with the processing system. The program code may also be implemented in assembly language or machine code, as needed. In practice, the mechanisms described herein are not limited to any particular programming language. In any case, the language may be a compiled language or an interpreted language.

[0111] Embodiments of the mechanisms disclosed herein may be implemented as hardware, software, firmware, or a combination of such implementation methods. Embodiments of the present invention may be implemented as a computer program or program code that runs on a programmable system comprising at least one processor, a storage system (including volatile and non-volatile memory and / or memory elements), at least one input device, and at least one output device.

[0112] One or more aspects of at least one embodiment may be implemented by representative instructions stored in a machine-readable medium, which, when read by a machine, represents various logics within a processor that cause a machine to produce logic for performing the techniques described herein. Such representations, known as "IP cores," may be stored in tangible machine-readable medium and may be supplied to various customers or manufacturing facilities for loading into manufacturing machines that actually produce the logic or processor.

[0113] Such machine-readable storage media may include, but are not limited to, non-temporary tangible configurations of articles manufactured or formed by machines or devices, including storage media such as hard disks, floppy disks, optical disks, compact disk read-only memory (CD-ROM), compact disk rewritable (CD-RW), and any other type of disk including magneto-optical disks; read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), semiconductor devices such as erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM), and phase-change memory (PCM); magnetic cards or optical cards; or any other type of medium suitable for storing electronic instructions.

[0114] Accordingly, embodiments of the present invention also include non-temporary tangible machine-readable media, such as instructions including a hardware description language (HDL) or design data, that define the structures, circuits, devices, processors, and / or system functions described herein. Such embodiments may also be referred to as program products. Emulation (including binary conversion, code morphing, etc.)

[0115] In some cases, an instruction converter may be used to translate instructions from a source instruction set to a target instruction set. For example, an instruction converter may translate, morph, emulate, or otherwise translate an instruction to one or more other instructions to be processed by the core (e.g., using static binary translation, dynamic binary translation including dynamic compilation). An instruction converter may be implemented in software, hardware, firmware, or a combination thereof. An instruction converter may be on-processor, off-processor, or partially on-processor and partially off-processor.

[0116] Figure 12 is a block diagram illustrating the use of a software instruction converter, in a particular implementation, to translate binary instructions in a source instruction set to binary instructions in a target instruction set. In the embodiment shown, the instruction converter is a software instruction converter, but alternatively, the instruction converter may be implemented in software, firmware, hardware, or various combinations thereof. Figure 12 shows that a high-level language program 1202 may be compiled using a first ISA compiler 1204 to produce first ISA binary code 1206 that can be natively executed by a processor having at least one first instruction set core 1216. A processor having at least one first ISA instruction set core 1216 represents any processor that can substantially achieve the same functionality as an Intel processor having at least one first ISA instruction set core by processing (1) a substantial portion of the instruction set of the first ISA instruction set core, or (2) a version of the object code of an application or other software targeted to run on the processor using at least one first ISA instruction set core, so as to run compatible, or otherwise achieve substantially the same results as an Intel® processor having at least one first ISA instruction set core. The first ISA compiler 1204 represents a compiler capable of generating first ISA binary code 1206 (e.g., object code) that can be executed on a processor having at least one first ISA instruction set core 1216, with or without additional linking.

[0117] Similarly, Figure 12 shows that a high-level language program 1202 can be compiled using an alternative instruction set compiler 1208 to generate an alternative instruction set binary code 1210 that can be executed natively by the processor without the first ISA instruction set core 1214. An instruction converter 1212 is used to convert the first ISA binary code 1206 into code that can be executed natively by the processor without the first ISA instruction set core 1214. This converted code is unlikely to be identical to the alternative instruction set binary code 1210 because an instruction converter capable of this conversion would be difficult to manufacture. However, the converted code performs common operations and consists of instructions from the alternative instruction set. Thus, the instruction converter 1212 represents software, firmware, hardware, or a combination thereof that enables a processor or other electronic device without a first ISA instruction set processor or core to execute the first ISA binary code 1206 through emulation, simulation, or any other process.

[0118] In different embodiments, a device may be responsible for or otherwise support multiple clients. However, the scalability of a device may be limited by its ability to maintain information about each of the different clients it is responsible for, and this ability may be limited by the silicon die area or other space available for storing such information. For example, a single-root input / output virtualization (SR-IOV) device, as defined by the Peripheral Component Interconnect Express (PCIe) specification, can implement a larger number of virtual functions (VFs) to support a larger number of clients. The device can also implement a larger number of queues and associated client-specific states to support a larger number of clients.

[0119] Figure 13 shows one embodiment of a scalable device 100 that can be implemented with logic gates, storage elements, and / or any other type of circuitry. All or part of these may be included in discrete components and / or incorporated into the circuitry of a processing device or any other device in a computer or other information processing system. For example, device 1300 in Figure 13 may correspond to the coprocessor 138 in Figure 1, or the dedicated logic 208 in Figure 2.

[0120] A scalable device according to one embodiment may use any number of work queues 1350, where a work queue is a data structure used to receive work from clients. A work queue may be a single client work queue (SWQ) that can receive work from multiple clients, or a dedicated work queue (DWQ) that can receive work from a shared work queue (SWQ). For example, device 1300 is shown as including DWQs 1310 and 1314 and SWQ 1312 that can receive work from clients 1320, 1322, 1324, 1326, and 1328.

[0121] A work queue can be implemented using memory within a device (e.g., device 1300) and / or using memory not within the device (e.g., host memory, system memory, registers, or other storage locations not within the device). Work queue 1350 may be implemented using host memory, for example, with base registers, head registers, and tail registers that are writable by software to notify the device about work submissions. Work queue 1350 may also be implemented using device memory, for example, with addresses that software can write to to submit work.

[0122] Work queue 1350 may be used to store work descriptors containing client identifiers (IDs) and authorizations. The client ID is used to identify the client (for example, using a processing address space identifier (PASID)), and the authorizations indicate the authorizations that the device can use to identify the client's address domain and authorizations. Work descriptors may be entered by trusted hardware (e.g., a CPU in a secure environment) and / or trusted software (an operating system (OS) running on a CPU in a secure environment) to ensure that they cannot be spoofed by unauthorized clients.

[0123] In various implementations, the work descriptor is fully self-descriptive. As a result, the device does not need to maintain arbitrary client-specific state and therefore does not require additional memory to support additional clients. A fully self-descriptive work descriptor contains all the information necessary to perform the requested work.

[0124] For example, in one embodiment, a work descriptor may include a pointer to a completion record in host memory, on which the completion status (including any error status) is written. The completion record may also include any partial results that can be used by subsequent work descriptors, and as a result, data can be carried over through operations (for example, a cyclic redundancy check calculation may use the results from each step as input to the next step). The completion record may also include flag bits and / or any other information that can be used to perform operations. Therefore, no memory in the device is used to store client-specific status regarding completion status, partial results, flag bits, etc.

[0125] In various implementations, work descriptors can also be used to circumvent other limitations on device scalability. For example, the limitation of limiting the number of clients to one per message signal interrupt in the PCI MSI-X table on a device can be avoided by any of the following techniques. With either technique, any client can be set up with multiple interrupts, and as a result, the client can choose to specify different interrupts for different descriptors, thus allowing the client to perform interrupt rebalancing without involving the OS or other system software.

[0126] In one embodiment, a work descriptor may be designed to include interrupt message information (e.g., MSI address and data) passed by the client. Since the interrupt message is provided by the client, it is not trusted. Therefore, interrupt remapping hardware (e.g., an I / O memory management unit) may be responsible for ensuring that the client cannot request an interrupt that has not been allocated. For example, a PASID may be used to discover an interrupt remapping table entry for an interrupt message, along with the remappingable interrupt message (PASID-granular interrupt remapping). PASID-granular interrupt remapping allows the same message to have different meanings when used in conjunction with different PASIDs, and also allows system software to control which interrupt messages can be used by each client (identified by the client's PASID). The device can then generate an interrupt using the interrupt message information from the work descriptor when it has finished an operation, while avoiding storing the interrupt message in the device.

[0127] In one embodiment, a work descriptor may be designed to include interrupt handlers (instead of the full MSI address and data value). The interrupt handler in the descriptor specifies an entry in the interrupt message table. The device may implement the interrupt message table in host memory, with each interrupt table entry containing an MSI address and data. Because the interrupt message table resides in host memory rather than device memory, it can be large enough to support any number of clients. The interrupt handler in the work descriptor may be used by the device to index the interrupt table and identify the MSI address and data value to generate an interrupt when the device has finished its operation. The device first uses the PASID to verify the interrupt handler and ensure that a client is able to use a particular interrupt handler. The device may verify the interrupt handler by using a PASID-granular interrupt message table, or by including the PASID in the interrupt table entry during interrupt setup and generation, and matching the PASID of the entry with the client's PASID. To avoid reading interrupt table entries from host memory in every descriptor, an interrupt message cache within the device may cache frequently used interrupt entries.

[0128] In addition to reducing or eliminating per-client state on the device, certain embodiments offer efficient processing of work from many clients. Embodiments may enable prioritizing work from time-constrained clients while ensuring forward progress on work from other clients. Embodiments may prevent errors or performance issues originating from some clients from adversely impacting other clients.

[0129] A work submission portal is a means by which a client can submit work requests to a device. In one embodiment, instead of a single work submission portal being mapped to all clients using an SWQ, the SWQ may have one or more work submission portals from which clients can submit work. By using one work submission portal per SWQ, a client may repeatedly retry work submissions because they will experience a full SWQ. This can make the use of the device difficult and unpredictable for other clients. In contrast, using multiple work submission queues per SWQ, as in certain embodiments, can provide prioritization of work requests and prevent starvation for some clients. In embodiments, a work submission portal may be implemented as a memory-mapped I / O (MMIO) address to which a client can write to submit a work request. This allows for multiple work submission portals to be provided by allocating one or more MMIO addresses to the SWQ and providing different MMIO addresses to different clients. This does not require additional logic circuitry or storage in the device.

[0130] In various embodiments, different work submission portals for SWQ may have different characteristics, such as different work acceptance priorities for different clients. For example, SWQ may have a first submission portal designated as a restricted portal and a second submission portal designated as an unrestricted portal. SWQ may have configurable thresholds that can be used to exclusively reserve certain entries for work submitted through the unrestricted portal. For example, SWQ may be configured to use only up to 80% (threshold) of the SWQ space for work submitted through the restricted portal, while work submitted through the unrestricted portal may use 100% of the SWQ space. Authoritative software (e.g., a device driver) can then map the restricted portal to a user-space client and retain the unrestricted portal for itself. When a user-space client experiences an SWQ full condition (for example, when the SWQ is 80% full, the SWQ returns "Retry" to the user-space client), instead of continuing to retry the work submission itself, it can request the device driver to submit the work descriptor on its behalf. The device driver can serialize the request from the user-space client and submit the work request using an unrestricted portal. Since 20% of the SWQ space is reserved for the unrestricted portal, the device driver's work submission has a high probability of success.

[0131] Embodiments may also provide handling for situations where the SWQ is 100% full (e.g., even an unspecified portal returning a “Retry”). In one embodiment, a device driver may respond to a client with a “Retry” as if it were received directly from the device. In one embodiment, a device driver may block a client until it can submit work, and in the meantime, the device driver may schedule another client. In one embodiment, a device driver may place a work descriptor in a software-defined work queue until it can be submitted to the device, but then resume the client as if the work had been successfully submitted to the device. With the latter approach, the client can continue operations while waiting for the device to catch up. Embodiments may provide use of this latter approach by ensuring that all descriptors to a device are independent of each other in terms of order, or by preventing a client from directly submitting any descriptor to a device that may be processed before descriptors queued in software by the device driver. This can be prevented by removing the device submission portal from the client's memory map so that clients cannot submit work directly to the device, or by locking the SWQ so that no clients can submit work. In this locking technique, all client work submissions return a "retry," and clients must request the device driver to submit their work. This provides fairness but may also have high overhead due to the bottleneck of work submissions going through the device driver (which may be acceptable as it only occurs when the shared work queue is completely full).

[0132] Embodiments may also offer configurability of the work queues themselves. For example, any or each of multiple work queues may be configured as dedicated or shared at runtime based on client requests. Work queues may be assigned different priorities and configured by software. A device may dispatch commands from higher-priority work queues with higher priority than commands from lower-priority work queues, without exhausting lower-priority work queues. Some clients may have access to multiple work queues and therefore have the ability to prioritize their own work, while other clients may have access to only a single work queue and therefore fix the priority of all work they submit.

[0133] Embodiments may also offer to mitigate problems that occur when a device is blocked due to waiting for a response from another part of the system (e.g., completion of a memory read, address translation, or handling of a page fault) while performing an operation. This problem is called head-of-line blocking because the operation being performed on the device prevents other operations running in the background from progressing.

[0134] In various embodiments, to mitigate head-of-line blocking, the device may include multiple operational components (engines), such as engines 1330, 1332, 1334, and 1336, which can process individual work descriptors in parallel. In one embodiment, one or more work queues may be grouped together with one or more engines. The device may support multiple groups, such as groups 1340 and 1342. Each work queue and each engine may be configured by software to be part of any one of the groups. Work descriptors from one or more work queues in a group may be dispatched to any of the engines in that group. Thus, if one engine in a group is processing descriptors from a work queue while waiting for an external response, other engines in the group may continue processing other work descriptors from the same or other work queues.

[0135] In an alternative embodiment, the device may implement an out-of-order processing engine. It may hold work descriptors awaiting an external response while processing other work descriptors from the same or other work queues.

[0136] Furthermore, in various embodiments, the device may have internal resources (e.g., device internal memory) that the engine uses to process work descriptors from various clients. If these resources are limited, the device may prioritize (or provide QoS for) the use of these resources by different clients to ensure that work descriptors from one or more clients (e.g., relatively large or time-consuming) do not consume most or all of the internal resources, thereby impacting the processing of other clients and the overall performance of the device. Prioritization may be performed using a credit-based system in which credits represent internal resources. Credits may be assigned to groups, individual work queues, or individual engines, and each group, work queue, or engine controls the number of resources available to process work descriptors. In one embodiment, the device may define two credit thresholds for each group, work queue, or engine: a first threshold specifying the minimum number of credits reserved for the group, work queue, or engine (minimum guaranteed or reserved credits), and a second threshold specifying the maximum number of credits allowed for the group, work queue, or engine (maximum allowed credits).

[0137] Embodiments may also provide efficient handling of page faults. If a device supports virtual memory (e.g., shared virtual memory (SVM) or I / O virtual addresses (IOVA)), addresses assigned to the device in a work descriptor are not guaranteed to map in physical memory. Before accessing host memory (e.g., through direct memory access (DMA)), the device may request an address translation from the I / O memory management unit (IOMMU), for example, using the address translation service described in the PCIe specification. The IOMMU walks through the address translation table and, if a translation exists, returns the translated physical address to the device, and as a result the device can access the data in memory. However, if the virtual address does not currently exist in main memory, the result is a translation error (I / O page fault). A page fault can also occur when a page exists, but the processor or device does not have the right to perform the type of access requested (e.g., the device attempts to write to a read-only page).

[0138] When a device encounters a page fault, the memory management system software is not running on the device and therefore cannot process the page fault itself. Thus, according to the method shown in Figure 14 (1400), after the application submits a command to the device (1410), the device attempts to access the page (1420), the IOMMU responds with a page fault (1430), the device sends a page fault notification to the OS via the IOMMU (1440), and blocks descriptor processing until the page fault is resolved. After the page fault is resolved (1450), the OS responds to the device via the IOMMU that the page is available (1460). The device then attempts to access the page again, and this time the access is successful (1470).

[0139] Devices typically implement a limited number of simultaneous notifications to the OS for unprocessed I / O page faults. Therefore, when a device supports a large number of clients, it may encounter many page faults and may have to wait until at least one page fault is resolved, which significantly reduces the device's performance.

[0140] Therefore, instead of blocking on an I / O page fault, some implementations, such as method 1500 in Figure 15, may include the device notifying the client directly about the page fault (1540) and terminating processing of the work descriptor that caused the page fault (after the application submits a command to the device (1510), the device attempts to access the page (1520), and the IOMMU responds with a page fault (1530)). The device can then continue processing other work descriptors without blocking while the application can resolve the page fault or, if not, respond to it (as described below) (1550). After the page fault is resolved, the application may resubmit a command to the device (1560), and the device may attempt to access the page again (1570).

[0141] According to embodiments such as Method 1500, page fault handling may be delegated to the client. For example, a client application may request the OS to resolve the page fault (e.g., by accessing the fault page), resubmit the work descriptor, and resume work after the page fault has been resolved. Alternatively, the client may decide to complete the remaining work using some other method instead of using the device, such as by using the CPU to perform operations.

[0142] The mechanism used by a device to notify an application that it has encountered a page fault is called partial completion. In embodiments that include partial completion, the device may report the completion of an operation to the client in the usual way (e.g., by setting a device register, qualifying a queue or ring buffer tail pointer, writing to a shared memory location, generating an interrupt, or any combination thereof), but the completion log information may include the following additional elements: a status field indicating that a page fault has been encountered; an indication of how much of the operation was completed before the page fault; the untranslated virtual address; an indication of whether the device intends to read or write to the untranslated address; and any other information required by the software to resume the operation (see the example below).

[0143] As the name suggests, partial completion means that the device may have performed part of the requested operation before encountering a page fault. By reporting partial completion to the client, the client (e.g., application software) can begin using the completed result even while the page fault has been resolved and the remainder of the operation is being performed by the device.

[0144] In one embodiment, for example, if the device encounters a page fault on the first page required to initiate an operation, partial completion may report that no operation was completed.

[0145] In some embodiments, depending on the type, length, and complexity of the operation, the device may restart the operation from the beginning rather than resuming from the point where the page fault was encountered. In this case, even if the page fault is not on the first page, the device may report that none of the operations were completed (assuming that the partially completed operations did not overwrite any inputs).

[0146] In an embodiment, partial completion information includes all the information necessary to resume the operation. The client may resume the operation by submitting a new command that starts where the previous operation stopped. For operations that carry data over throughout the operation (e.g., CRC calculation), if a page fault is encountered in the middle of the operation, intermediate results are retained for use when the operation is resumed. Intermediate results may be stored in the completion record along with page fault information. When the application resumes the operation after satisfying the page fault, it passes the intermediate results along with the command to resume the operation. Method and apparatus for high-performance page fault handling of multitenant scalable accelerators

[0147] Accelerators are frequently configured to use shared virtual memory (SVM), which provides a ubiquitous programming model between the host / CPU and accelerator offloading. At the same time, there has been a rapid increase in highly multi-tenant (VM or container) environments, making it difficult to support SVM and I / O page faults in these environments.

[0148] Accelerator modes such as SVM Block-on-Fault and SVM Fault-and-Stream were used to address these limitations. In SVM Block-on-Fault, the accelerator / engine blocks until the I / O page fault is resolved by the OS / VMM software and a response is returned to continue the remaining work. In SVM Fault-and-Stream, the accelerator / engine context switches to a different offload context while the I / O page fault is resolved for a given context.

[0149] These existing models are subject to various limitations. SVM Block-on-Fault has scaling and multi-tenancy challenges. If the accelerator has only four engines, I / O page faults cannot scale beyond four concurrent clients or have noisy neighbor-related challenges in multi-tenancy (e.g., a page fault from one tenant delays offload execution for other tenants). SVM Fault-and-Stream has challenges related to the complexity of added devices and overhead associated with saved / restored states. Furthermore, the previous techniques simply support page faults on data buffers rather than on control structures.

[0150] Embodiments of the present invention include techniques for supporting high-performance I / O page fault handling for accelerator devices in highly multitenant and scalable environments, while maintaining accelerator complexity and check costs. In particular, the implementations described herein provide page fault handling for accelerators with per-work queue capability, direct notification of accelerator page faults to application / offload clients, and an event logging mechanism to enable the accelerator driver to proxy page fault handling / notification to offload clients (when direct notification is not feasible / preferred). In addition, some embodiments enable early completion of the fault context and immediate switching to the next context, allowing software to handle the page fault and complete the remaining work on the host processor (e.g., CPU) or by offloading it back to the accelerator. Furthermore, extensions for handling batch descriptors are described.

[0151] These embodiments address scaling and complexity challenges, enable VM live migration using allocated accelerators, and improve accelerator page fault handling performance. Therefore, the embodiments described herein make the use of SVMs in virtualized configurations more feasible and practical.

[0152] Referring to Figure 16, one embodiment of the accelerator 1600 includes a work receiving unit 1610 containing multiple work queues (WQs) 1601A-D, which may be organized into groups based on the characteristics of the workload described above. Multiple execution units, or "engines" 1621A-D, are assigned to process work descriptors from one or more of the work queues 1601A-D. One or more work dispatchers 1630 dispatch the work specified in the descriptor from the work queues 1601A-D to the execution engines 1621A-D.

[0153] The execution engines 1621A-D are typically shared among multiple clients, which may be drivers, applications, virtual machines (VMs), containers, or any other functional software components. Resource sharing reduces the cost / burden of building an independent engine for each client. Work queues 1601A-D send work to these engines 1621A-D, which is then assigned to individual clients (e.g., dedicated work queues 1310, 1314 in Figure 13) or shared among them (e.g., shared work queue 1312 in Figure 13).

[0154] Referring to Figure 17, in the shared virtual memory programming model, as part of processing the work descriptor, accelerator 1600 attempts to obtain an address translation associated with the address specified in the work descriptor, for example by using the PCIe address translation service. Accelerator 1600 may include a translation lookaside buffer 1705 for caching virtual-physical address translations. If a particular translation is not found in TLB 1705, a translation request 1750 is sent to IOMMU 1725 within the root complex 1720 of host processor 1710. The IOMMU obtains the translation from TLB or from memory 1715 and then sends a translation response 1751 along with the translation to accelerator 1600.

[0155] To ensure that the TLB 1705 of accelerator 1600 remains in a state consistent with the TLBs of IOMMU 1725 and host processor 1710, IOMMU 1725 and accelerator 1600 must communicate. Therefore, in response to certain events, such as the modification of a translation entry or context switch, IOMMU 1725 sends an invalidation request 1753 to accelerator 1600 to invalidate the relevant TLB entry, and sends back an invalidation acknowledgment 1754 when completed.

[0156] If the page does not exist in memory 1715, or if there are insufficient permissions to access the page, the translation request 1750 generates a page request 1755, for example by using the PCIe page request service. Such a page request is received by the IOMMU 1725 and passed to the system software (e.g., OS / VMM) to handle the I / O page fault. The system software fixes the page fault and returns a successful page response 1756 to the accelerator device via the IOMMU 1725. If access to the page is denied, the page response is returned to the accelerator 1600 along with the corresponding error.

[0157] During a series of operations, the accelerator 1600 (or the accelerator's engine) can halt engine / execution until an I / O page fault is handled (similar to how application execution on a core is halted until a CPU page fault is handled), a mode referred to as "SVM Block-on-Fault mode". Alternatively, the accelerator / engine can generate a page request, save the current work context, transition to the next work descriptor, and resume work later once the page fault is handled, a mode referred to as "SVM Fault-and-Stream mode".

[0158] As mentioned, Block-on-Fault mode presents scaling and noisy neighbor-related challenges. For example, in a scenario where four work queues related to four different clients are mapped to an engine, a page fault from one client can cause the engine to stop, resulting in delays for the other clients. This becomes an even greater problem in virtualization configurations where page fault handling latency is higher and the behavior of one VM / tenant affects the quality of service for other VMs / tenants sharing the same engine.

[0159] Fault-and-Stream mode attempts to address this outage issue. However, it requires complex hardware capable of supporting the saving and restoring of workload contexts and presents challenges regarding the costs and latency associated with context switching and save / restore operations.

[0160] Furthermore, these traditional virtualization configurations introduce additional latency and overhead, as page faults result in virtual machine (VM) exits and require another VM exit to queue page responses. Additional challenges arise when supporting live migration of VMs using allocated accelerator resources. For example, some VMMs first suspend the virtual processor and then the virtual device for the VM, which can result in deadlocks due to cyclic dependencies.

[0161] Figure 18 shows an example of an accelerator 1800 in which embodiments of the invention may be implemented. Multiple engines 1821A-C process descriptors from multiple work queues 1801A-C. Each accelerator engine 1821A-C includes fault handling logic 1805A-C to respond to page fault conditions described herein (however, for brevity only, only engine 1821A is shown). Engines 1821A-C may be any type of processing engine of the accelerator and include, but are not limited to, a data copy engine, a data compression engine, a security engine, a communication engine, and a graphics engine.

[0162] In one implementation, instead of blocking on an I / O page fault until it is resolved, the fault handling logic 1805A directly notifies the client 1801 and terminates work descriptor processing. This allows the engine 1821A to continue processing other work descriptors without blocking, and the client 1801 then performs page fault handling, possibly via a driver, OS, or other authorized software. For example, the client application can then resolve the page fault (e.g., by accessing the fault page) and resubmit the work descriptor to the work queue 1801A, requesting the OS to resume work after the page fault is resolved. Alternatively, the client application 1801 may decide to complete the remaining work using some other method instead of using the accelerator 1800, such as by performing operations using the host CPU 1710.

[0163] In various embodiments, I / O page faults are reported via a designated mechanism, such as an event log 1812, and proxied to a client application 1801. System software (e.g., a device driver or OS) can then pin the page fault based on the information received in the event log entry and pass a completion record to the client 1801 (acting as a proxy). In some embodiments, a completion record buffer 1811 is used to directly notify the client 1801 about the page fault and the corresponding information.

[0164] A single accelerator device 1800 may use any of the above techniques at different times or for different purposes.

[0165] The following are typical sources of page faults in an accelerator SVM environment. 1. Source / Destination Buffers: Most accelerator devices have source / input buffers from which data is read, processed / operated, and then written to destination / output buffers. 2. Completion Record / Response Queue Buffer: Many high-performance and scalable devices utilize completion record or response queue slots, where memory writes are generated, to notify software of work completion. 3. Descriptor List Buffer / Ring: Many high-performance devices utilize descriptor lists or rings to communicate / execute multiple descriptors in a single work submission / doorbell hit. 4. Scatter-gather list buffer: Some accelerator devices communicate scatter-gather list addresses instead of communicating source / destination addresses directly. 5. Operation-Specific Buffers: Some accelerator devices allow software to specify additional buffer addresses depending on the operation (e.g., CRC seed address, drain readback address, differential record address, etc.). 6. Configuration and Control Structures: Some accelerator devices may also have configuration and control structures that can encounter page faults during workload execution (e.g., offload / work context, permission table, etc.).

[0166] Figure 19A shows an example of a work descriptor 1900 that may be submitted to work queue 1801A and interpreted by accelerator engine 1821A to execute a work. The work descriptor 1900 includes an operation field 1901 that specifies the operation to be performed, several flags 1902 (e.g., completion record address valid, request completion record, request completion interrupt), a processing address space identifier (PASID) field 1903 that associates the work descriptor with the process / application that generates the work, a completion record address field 1904 that indicates the memory location for the completion record (e.g., to store metadata about the work to be performed), a source address field 1905 that indicates the memory location for the source data, a destination address field 1906 that indicates the memory location for the result data, a completion interrupt handler 1907, a transfer size field 1908, and one or more operation-specific fields 1909 that (optionally) specify additional details about the operation to be performed.

[0167] An example of a batch descriptor 1950 shown in Figure 19B contains many of the same fields as the work descriptor 1900, with the exception that the descriptor list address 1951 identifies the memory location of the list (or batch) of descriptors, and the descriptor count field 1952 for indicating the number of descriptors in the list.

[0168] Figure 20 shows an example of a completion record 2000. In one embodiment, the accelerator engine 1821A, which executes the work identified by the descriptor, generates a completion record which is stored in the completion record buffer 1811. The client 1801 that submitted the work can then access the completion record from the buffer.

[0169] As shown in Figure 20, in some implementations, the completion record 2000 is a 32-byte structure that is written to buffer 1811 when the requested work is completed or when an error is encountered. In certain embodiments, engine 1821A writes sufficient information to the completion record 2000, allowing engine 1821A to continue the operation if it is only partially completed due to a page fault.

[0170] The Completion Status field 2004 indicates whether the operation was completed. If the operation completed successfully, the Completion Record 2000 may include the results of the operation, depending on the type of operation (for example, in the Operation Specific Field 2006 or the Result Field 2003), if applicable. The results may also be stored in a different location in memory and may be identified by the address in the Completion Record 2000. If the operation did not complete successfully, the Completion Record includes fault or error information in the Fault Information Field 2003.

[0171] In one implementation, if this completion record 2000 is for a descriptor submitted as part of a batch, the index field (not shown) contains the batch index of the descriptor that generated this completion record. For batch descriptors, this field may be 0xff. For any other descriptor that is not part of a batch, this field may be reserved.

[0172] In one implementation, if an operation is partially completed due to a page fault, the byte completion field 2001 indicates the number of source bytes processed before the fault occurred. All source bytes represented by this count are fully processed, and the results are written to the destination address according to the operation type, as necessary. For some operation types, this field may be used if the operation is stopped before completion for some reason other than a fault. If the operation is fully completed, this field may be set to 0. For operation types where the output size cannot be easily determined from this value, the completion record 2000 also includes the number of bytes written to the destination address. If an operation is partially completed due to a page fault, the fault address field 2005 contains the address that caused the fault.

[0173] As mentioned, embodiments of the present invention use a completion record buffer 1811 to directly notify client 1801 of a page fault and provide corresponding information. For example, a fault address field 2005 indicates the address associated with the page fault, a fault information field 2002 captures information related to the page fault, such as the buffer type and whether the fault address field 2005 is valid, and a status field 2004 communicates relevant status information.

[0174] However, when a page fault is observed at the completion log address itself, it may not be feasible to use the completion log buffer 1811 to transmit the information to the client 1801. In this situation, a memory-based event log 1812 may be used to log such events. This type of event log 1812 may be implemented per accelerator device 1800, per engine 1821A-C, and / or per work queue 1801A-C.

[0175] Figure 21 shows an example of an event log entry 2100 having multiple fields for providing information about the relevant descriptor and / or fault conditions. Errors on the descriptor's completion record 2000, or errors during the processing of a descriptor, that do not have a valid completion record address are typically reported in a register (software error register). Multiple occurrences of such errors prior to the software processing of the register result in an overflow condition.

[0176] In contrast, embodiments of the present invention support logging such events of event log 1812 into memory. As shown in Figure 22, the event log configuration register (EVLCFG) 2201 is configured to store the address and size of the event log memory area. Event log 1812 is activated when the event log enable bit in the accelerator control register is set to 1, as shown in Figure 22 as the general configuration register (GENCFG) 2202.

[0177] In contrast to writing to the completion record buffer 1811, writing to the event log 1812 can be issued as either a translated or untranslated memory access. In some implementations, the software pins the memory page corresponding to the event log and performs the event log write with a traffic class (TC) value of 0.

[0178] In at least some embodiments, if the PASID enabled field 2201C in EVRCFG2201 is 1, writes to the event log 1812 are issued as writes using the PASID and the corresponding privilege level (which is also stored in EVRCFG2201 in some implementations). When the event log 1812 is enabled, the accelerator device 1800 initializes the head field 2203A and tail field 2203B in the event log status register (EVLSTATUS) 2203 (when device 1800 is enabled). When enabled, the hardware writes each event to the offset specified by the event log tail field 2203B, incrementing the tail value 2203B. When the tail reaches the end of the log, it wraps to a value of 0. The next event to be processed by the software is specified by the event log head field 2203A. The software updates the head field 2203A after processing the first one or more events in the event log. The log is full when tail value + 1 mod log size = head value.

[0179] When writing an event to event log 1812, if the event log interruptable field 2204A in the accelerator control register (GENCTRL) 2204 is 1 and the interrupt wait bit 2203C in EVLSTATUS is 0, the hardware sets the interrupt wait bit 2203C to 1, sets the event log field 2205A in the interrupt cause register (INTCAUSE) 2205 to 1, and generates an interrupt (for example, using MSI-X entry 0 if accelerator 1800 includes MSI-X support). No further interrupts are generated for additional log entries until the software clears the interrupt wait bit 2203C. If event log 1812 is full when the hardware attempts to append an event, the hardware will block until the software updates the event log head field 2203A after processing one or more events from the beginning of the log. Therefore, the software must ensure that the event log area in memory is of an appropriate size and that event log entries are processed in a timely manner.

[0180] In various embodiments, if the hardware encounters a page fault on a completion record address while the Page Request Service (PRS) is disabled, it is reported as an error. If the event log 1812 is enabled, the hardware writes an entry to the event log with an appropriate error code indicating the cause of the page fault. Otherwise, it is reported via field 2206A in the software error register (SWERROR) 2206. In the former case, the hardware also writes a completion record 2000 for that descriptor to the event log entry. The completion record 2000 written to the event log 1812 has the same format as described above. The software responsible for processing the event log 1812 is expected to take the necessary actions to send the completion record 2000, if applicable, to the software entity (e.g., client 1801) that submits the fault descriptor. The software may also generate a completion interrupt if requested.

[0181] Referring to Figure 23, if a completion record for any descriptor in a batch is written to event log 1812 due to a page fault on the completion address, then if either a completion record or a completion interrupt is required for the batch descriptor, a batch completion record 2300 for the corresponding batch descriptor 1950 is also written to event log 1812. In this case, the error code in the event log entry for batch descriptor 1950 indicates that one or more descriptors in the batch have associated the event log entry with a completion record 2000 that needs to be processed by the software. In various embodiments, the accelerator 1800 generates a batch identifier value that enables the software to associate the event log entries 1812 for descriptors in a batch and for the corresponding batch descriptor 1950, and this is reported in the batch identifier field of the event log 1812 entry. Once an event log entry for a batch descriptor is written, the batch identifier can be reused by the hardware. An event log entry with a first error in the batch flag, which is 1, identifies the first entry for that batch. This allows the software to identify any state event log entry that has the same batch identifier. If the software encounters an entry with this flag set to 1, any previously recorded unhandled page faults for the same batch identifier may be discarded.

[0182] An example of a drain descriptor 2400 and a drain completion record 2401 is shown in Figure 24. The drain descriptor waits for the completion of a specific preceding descriptor in the work queue 1801A to which the drain descriptor 2400 is submitted. If the drain descriptor 2400 is submitted to a dedicated work queue, it waits for the completion of all descriptors in the work queue. If the drain descriptor is submitted to a shared work queue 1801A, it waits for descriptors in work queues submitted with the same PASID as the drain descriptor. To wait for all descriptors with a particular PASID, the software submits a separate drain descriptor to all work queues where the PASID is used.

[0183] In some embodiments, completion for drain descriptor 2400 is always written to event log 1812 if enabled. If event log 1812 is not enabled, drain descriptor completion record 2401 is written to completion record address.

[0184] As shown in Figure 25, at least some embodiments of the present invention implement per-WQ page request services (PRS) 2501-2502 that can be dynamically enabled and disabled for each WQ. In Figure 25, for example, PRS 2501 and PRS 2502 are dynamically enabled for work queues 1801B and 1801C, and disabled for work queue 1801C, respectively. This uses high-performance techniques described herein rather than PRS (e.g., terminating work and submitting entries to event log 1812 and / or completion record buffer 1811).

[0185] In one embodiment, a capability bit is provided in a control register to indicate whether per-WQ PRS can be disabled on a given scalable accelerator 1800. The capability bit may be provided in any of the registers described above. When supported, even when enabled at the accelerator device 1800 level, the per-WQ control bit indicates whether PRS is enabled or disabled for a particular WQ, thereby providing granular control over page fault handling. In some implementations, for example, when the per-WQ control bit is set for a particular work queue (i.e., set to 1), one or more of the scalable page fault handling techniques described herein are used. When it is erased (i.e., binary 0), the legacy page fault handling technique is used.

[0186] Figure 26 shows a method according to an embodiment of the present invention. In 2601, a work descriptor is received in the accelerator's work queue, and in 2602, a page fault is detected for the work associated with the descriptor in a specific work queue.

[0187] Depending on the page fault mode configured for a particular work queue, as determined in 2603, page faults are handled in different ways. For example, when a page fault occurs and PRS2501-2502 is enabled for the associated work queues 1801A-B (mode 2 in Figure 26), the work associated with the descriptor is paused, and the page fault is reported to IOMMU1725 as a PRS request for service by the OS page fault handler in 2611. IOMMU1725 notifies the OS via an interrupt. The OS checks the address, and if the check is successful, creates a mapping in the page table and returns a PRS response through IOMMU1725-2612. The descriptor that encountered the fault is blocked until a PRS response is received. Other operations behind the descriptor with the fault may also be blocked. If the OS is unable to create the mapping, it returns an error response, and the descriptor is completed with an error. When PRS is disabled, as described in the next section, the error report is identical to the page fault report.

[0188] As described above, for work queue 1801A with PRS disabled (mode 1 in Figure 26), the work associated with the descriptor is terminated at 2604. When a page fault occurs on the completion record address, as determined at 2605, if enabled, the error is reported in the event log at 2606 (or, in some embodiments, in the SWERROR register 2206). At 2607, the client resolves the page fault and determines the next steps based on the event log. A completion record fault reported in event log 1812 is recoverable. The descriptor is completed, and the event log entry includes the completion record address and the contents of the completion record. After handling the page fault, the system software can copy the completion record to the completion record address. A completion record fault reported in SWERROR may not be recoverable. Faults may be lost, as only a single error can be logged. Since partial completion information is not logged, the descriptor is not completed and must be restarted.

[0189] As determined in 2605, when a page fault occurs for an address in a descriptor other than the completion record address, engine 1821A stops the operation, writes a partial completion status along with the fault address, and in 2608 advances the information into the completion record in the completion record buffer 1911 (described above). When the client software receives the completion record indicating partial completion, it resolves the page fault and in 2609 determines the next step based on the completion record. For example, the client has the option of fixing the fault on the host processor (e.g., by touching the page) and submitting a new work descriptor with the remaining work. Alternatively, the software can complete the remaining work on the host processor.

[0190] As explained above, operating with PRS disabled and event logging enabled reduces the impact on other applications in the presence of page faults, thereby improving performance.

[0191] In some implementations, if a descriptor in batch descriptor 1950 encounters a fault due to a completion address, the entry is pushed to event log 1812. Before the event log entry can be processed (e.g., by device driver software), client 1801 may, in some cases, observe an update to the entire batch descriptor completion record 2200, thereby causing inconsistent conditions or order / timing issues with the completion record update. A similar issue applies to drain descriptor 2300.

[0192] The following embodiment addresses this problem. If a completion record for any descriptor in a batch is written to event log 1812 (for example, due to a page fault related to the completion record), then if either a completion record or a completion interrupt is required for batch descriptor 1950, the completion for the corresponding batch descriptor 1950 is also written to the event log.

[0193] Each event log entry for a descriptor within a batch, or for a batch descriptor with an error code related to a page fault, includes a batch identifier used (for example, by a kernel-mode driver (KMD)) to associate the relevant completion record in the event log with the same batch.

[0194] If there is a page fault at completion record address 2200 for a batch descriptor, and there is no event log entry with error code ABC written for any descriptor in the batch, the event log entry for the batch descriptor will be written with error ABC. If there is a page fault at completion record address for a batch descriptor, and there is an event log entry with error code ABC written for any descriptor in the batch, the event log entry for the batch descriptor will be written with error XYZ. By using different error codes ABC / XYZ, the software can distinguish between these two situations.

[0195] In some embodiments, page faults are handled by disabling PRS2501-2502, stopping operations, and reporting a partial completion status in the completion log. In addition, page faults on the completion log address are reported through event log 1812 while enabled. In at least one implementation, the kernel-mode driver (KMD) is responsible for configuring and enabling event log 1812. In these implementations, the KMD must ensure that event log 1812 is sufficiently large and process entries in a timely manner to avoid subsequent event log writes from devices that would be blocked due to the event log being completely consumed.

[0196] If client 1801 is notified of a page fault directly through the completion log, the client can fix the (valid) page fault by accessing the page. Once the page fault is corrected, client 1801 has the option of completing the remaining work on the CPU or resuming / continuing the work by offloading it again to accelerator 1800 (for example, by storing a new work descriptor in work queue 1801A).

[0197] In various embodiments, when processing entries in event log 1812 for page faults on the completion record, KMD or other authorization software components perform the following operations: 1. If the first error in the batch flags is 1, discard any previously recorded errors associated with the batch identifier. This can occur when batch completion is lost due to an abort command or internal hardware error. Normally, no errors are recorded and no action is required. 2. Attempt to fix the page fault corresponding to the fault address and PASID reported in event log entry 1812. If successful, write a completion record to the fault address and generate a completion interrupt if the event log entry indicates that a completion interrupt should be generated. 3. If there is an error in writing a completion record, and the completion record is for a descriptor within a batch, KMD (or other software component) associates the error with the batch identifier in the event log entry and tracks it when an event log entry for the corresponding batch descriptor is observed. KMD does not need to track completion records that were written successfully.

[0198] If a batch descriptor has an error code indicating that one or more descriptors in a batch have an event log entry, and the event log entry indicates that a completion record should be written for the batch descriptor, KMD or other authorization software component will do the following before writing the batch completion record to memory, provided that the error is logged along with the matching batch identifier: 1. If the completion record status in the event log entry indicates that all descriptors were successful, KMD (or other component) will change it to indicate that one or more descriptors failed. 2. KMD or other components then clear the recorded errors in preparation for the next batch with the same batch identifier.

[0199] If no errors are logged for a batch (all completion records have been written successfully), KMD or other components write the batch completion record as is. If the event log entry indicates that an interrupt should be generated, the software should then generate a completion interrupt.

[0200] When an application, user-mode driver (UMD), or other client 1801 receives a completion record indicating partial completion, it may choose to fix the page fault and resubmit the descriptor to the accelerator device 1800 to complete the remaining operation. In most cases, the original descriptor may need to be updated to adjust the transfer size field based on the amount of work already completed. Certain operations may require additional updates to the original descriptor. When resubmitting a prematurely completed batch descriptor, the software may set a batch error flag in the descriptor based on the status of the partially completed batch, ensuring that the status of the final completion record for the batch reflects the accurate status across all descriptors in the batch.

[0201] In a virtualized configuration, the host software may expose virtual event logs to guests, inject fault information into guests (e.g., via guest KMD), and manage page faults as described above.

[0202] Embodiments of the invention described herein include one or more of the following: • Selection of page fault handling capability per WQ page, including the choice between PRS-based page fault handling and high-performance page fault handling as described above. • The ability to directly notify application / offload clients about accelerator page faults (when possible), and the use of event logs to allow accelerator drivers to proxy page fault handling / notification to offload clients (when direct notification is not feasible / notified). • Early completion of fault contexts with immediate switching to the next context. • Handling page faults in software and completing the remaining work on the host processor or by offloading it again to the accelerator device. • Techniques for processing batch descriptors.

[0203] Embodiments of the present invention may include various steps as described above. These steps may be embodied in machine-executable instructions that can be used to cause a general-purpose or dedicated processor to perform the steps. Alternatively, these steps may be performed by specific hardware components including hardwired logic for performing the steps, or by any combination of programmed computer components and custom hardware components.

[0204] example

[0205] The following are exemplary implementations of different embodiments of the present invention.

[0206] Example 1 It is a device, One or more accelerator engines for processing work descriptors submitted by a client to multiple work queues, Fault handling hardware logic relating to the one or more accelerator engines, which implements a specified page fault handling mode for each of the work queues, wherein the page fault handling mode includes a first page fault handling mode and a second page fault handling mode, and A device equipped with the following features.

[0207] Example 2 In the first page fault processing mode, the fault processing logic notifies the client to terminate processing of the work descriptor associated with the page fault, as described in Example 1.

[0208] Example 3 The apparatus according to Example 2, wherein the fault handling hardware logic submits information to a completion log buffer and / or event log in response to the page fault, and the information is related to the page fault and is available to the client and / or system software to resolve the page fault.

[0209] Example 4 The apparatus according to Example 3, wherein the information allows the client to continue processing or resubmit the work if, due to the page fault, the work related to the descriptor is only partially completed.

[0210] Example 5 The device according to Example 4, wherein the information includes a fault address and a byte completion field indicating the number of source bytes processed before the page fault occurred.

[0211] Example 6 The device as in Example 3, wherein if the page fault occurs on an address associated with the completion record buffer, the information is submitted to the event log but not to the completion record buffer.

[0212] Example 7 The device according to Example 6, wherein the event log is associated with a specific accelerator engine of the one or more accelerator engines, or a specific work queue of the multiple work queues.

[0213] Example 8 The apparatus according to Example 1, wherein in the second page fault handling mode, the processing of the work descriptor is suspended, the fault handling hardware logic generates a page request to the memory management hardware logic, and the processing of the work descriptor is resumed only after a response is received from the memory management hardware logic.

[0214] Example 9 It is a method, The process involves an accelerator engine processing a work descriptor, wherein the work descriptor is submitted by a client to multiple work queues. In response to detecting page faults related to work descriptors in a specific work queue, The steps include determining the page fault processing mode associated with the work queue, If the page fault processing mode includes a first page fault processing mode, the client is notified to terminate the processing of the work descriptor associated with the page fault. A method for providing this.

[0215] Example 10 The method according to Example 9, further comprising the steps of, if the page fault processing mode includes a second page fault processing mode, an operation to pause the work associated with the work descriptor and send a page fault request to the memory management hardware logic, and an operation to resume the work associated with the work descriptor after receiving a response from the memory management hardware logic.

[0216] Example 11 The method according to Example 9, further comprising the step of submitting information to a completion log buffer and / or event log in response to the page fault, wherein the information is related to the page fault.

[0217] Example 12 Based on the information, if the work associated with the work descriptor is only partially completed due to a page fault, the client continues to process the work or resubmits it, as described in Example 11.

[0218] Example 13 The information according to Example 12, wherein the information includes a fault address and a byte completion field indicating the number of source bytes processed before the page fault occurred.

[0219] Example 14 The method according to Example 11, wherein if the page fault occurs on an address associated with the completion log buffer, the information is submitted to the event log but not to the completion log buffer.

[0220] Example 15 The method according to Example 11, wherein when the page fault information for the first work descriptor is submitted to the event log, the page fault information for the second work descriptor is also submitted to the event log, but is not submitted to the completion record buffer.

[0221] Example 16 The method according to Example 15, wherein at least one work descriptor includes a batch descriptor containing a descriptor list address that identifies a memory location for a batch of descriptors, or a drain descriptor that causes the accelerator engine to wait for the completion of a specified preceding descriptor in the work queue before processing the drain descriptor.

[0222] Example 17 A machine-readable medium in which program code is stored, wherein when the program code is executed by a machine, the machine... An operation in which an accelerator engine processes a work descriptor, wherein the work descriptor is submitted to multiple work queues by a client. In response to detecting page faults related to work descriptors in a specific work queue, An operation to determine the page fault processing mode associated with the aforementioned work queue, If the page fault processing mode includes a first page fault processing mode, the client is notified of the operation to terminate the processing of the work descriptor associated with the page fault. A machine-readable medium that enables execution.

[0223] Example 18 The method according to Example 17, further comprising the steps of, if the page fault processing mode includes a second page fault processing mode, an operation to suspend the work associated with the work descriptor and send a page fault request to the memory management hardware logic, and an operation to resume the work associated with the work descriptor after receiving a response from the memory management hardware logic.

[0224] Example 19 The method according to Example 17, further comprising the step of submitting information to a completion log buffer and / or event log in response to the page fault, wherein the information is related to the page fault.

[0225] Example 20 Based on the information, if the work associated with the descriptor is only partially completed due to a page fault, the client continues to process the work or resubmits it, as described in Example 19.

[0226] Example 21 The information according to Example 20, wherein the information includes a fault address and a byte completion field indicating the number of source bytes processed before the page fault occurred.

[0227] Example 22 The method as in Example 19, wherein if the page fault occurs on an address associated with the completion log buffer, the information is submitted to the event log but not to the completion log buffer.

[0228] Example 23 The method according to Example 19, wherein when the page fault information for the first work descriptor is submitted to the event log, the page fault information for the second work descriptor is also submitted to the event log, but is not submitted to the completion log buffer.

[0229] Example 24 The method according to Example 23, wherein at least one work descriptor includes a batch descriptor containing a descriptor list address that identifies a memory location for a batch of descriptors, or a drain descriptor that causes the accelerator engine to wait for the completion of a specified preceding descriptor in the work queue before processing the drain descriptor.

[0230] As described herein, instructions may refer to a specific configuration of hardware, such as an application-specific integrated circuit (ASIC) configured to perform a specific operation or having a predetermined function, or to a software instruction stored in memory embodied in a non-temporary computer-readable medium. Thus, the techniques shown in the figures may be implemented using stored code and data and executed on one or more electronic devices (e.g., end stations, network elements, etc.). Such electronic devices store and communicate code and data (internally and / or with other electronic devices via a network) using computer-readable media such as non-temporary computer-readable storage media (e.g., magnetic disks, optical disks, random-access memory, read-only memory, flash memory devices, phase-shift memory) and temporary computer-readable communication media (e.g., electrical, optical, acoustic, or other forms of propagating signals (carrier waves, infrared signals, digital signals, etc.)). In addition, such electronic devices typically include a set of one or more processors coupled with one or more other components such as one or more storage devices (non-transient machine-readable storage media), user input / output devices (e.g., keyboards, touchscreens, and / or displays), and network connectivity. The coupling of the processors and the set of other components is typically done through one or more buses and bridges (also called bus controllers). The storage devices and the signals carrying network traffic represent one or more machine-readable storage media and machine-readable communication media, respectively. Thus, the storage devices of a given electronic device typically store code and / or data for execution on the set of one or more processors of that electronic device. Of course, one or more parts of one embodiment of the present invention may be implemented using different combinations of software, firmware, and / or hardware. Throughout this detailed description, for illustrative purposes, numerous specific details have been provided to provide a complete understanding of the invention.However, it will be apparent to those skilled in the art that the present invention can be carried out without some of these specific details. In certain examples, well-known structures and functions have not been described in the detailed description in order to avoid obscuring the subject matter of the present invention. Therefore, the scope and spirit of the present invention should be judged in terms of the following claims. [Other adjacent items] (Item 1) It is a device, One or more accelerator engines for processing work descriptors submitted by a client to multiple work queues, Fault handling hardware logic relating to the one or more accelerator engines, which implements a specified page fault handling mode for each of the work queues, wherein the page fault handling mode includes a first page fault handling mode and a second page fault handling mode, and A device equipped with the following features. (Item 2) In the first page fault processing mode, the fault processing logic notifies the client and terminates processing of the work descriptor associated with the page fault, as described in item 1. (Item 3) The fault handling hardware logic, in response to the page fault, submits information to a completion log buffer and / or event log, the information relating to the page fault and available to the client and / or system software for resolving the page fault, as described in item 2. (Item 4) The apparatus according to item 3, which allows the client to continue processing or resubmit the work if, due to the page fault, the work related to the descriptor is only partially completed. (Item 5) The device described in item 4, which includes the fault address and a byte completion field indicating the number of source bytes processed before the page fault occurred. (Item 6) The device described in item 3, wherein if the page fault occurs on an address associated with the completion record buffer, the information is submitted to the event log but not to the completion record buffer. (Item 7) The event log is associated with a specific accelerator engine of the one or more accelerator engines, or a specific work queue of the multiple work queues, as described in item 6. (Item 8) The apparatus according to item 1, wherein in the second page fault handling mode, the processing of the work descriptor is suspended, the fault handling hardware logic generates a page request to the memory management hardware logic, and the processing of the work descriptor is resumed only after a response is received from the memory management hardware logic. (Item 9) It is a method, The process involves an accelerator engine processing a work descriptor, wherein the work descriptor is submitted by a client to multiple work queues. In response to detecting page faults related to work descriptors in a specific work queue, The steps include determining the page fault processing mode associated with the work queue, If the page fault processing mode includes a first page fault processing mode, the client is notified to terminate the processing of the work descriptor associated with the page fault. A method for providing this. (Item 10) The method according to item 9, further comprising the steps of, if the page fault processing mode includes a second page fault processing mode, an operation to suspend the work associated with the work descriptor and send a page fault request to the memory management hardware logic, and an operation to resume the work associated with the work descriptor after receiving a response from the memory management hardware logic. (Item 11) The method of item 9, further comprising the step of submitting information to a completion log buffer and / or event log in response to the page fault, wherein the information is related to the page fault. (Item 12) Based on the information, if the work associated with the work descriptor is only partially completed due to a page fault, the client continues to process the work or resubmits it, as described in item 11. (Item 13) The information as described in item 12, wherein the information includes a fault address and a byte completion field indicating the number of source bytes processed before the page fault occurred. (Item 14) The method according to item 11, wherein if the page fault occurs on an address associated with the completion log buffer, the information is submitted to the event log but not to the completion log buffer. (Item 15) The method according to item 11, wherein when the page fault information for the first work descriptor is submitted to the event log, the page fault information for the second work descriptor is also submitted to the event log, but is not submitted to the completion record buffer. (Item 16) The method according to item 15, wherein at least one work descriptor includes a batch descriptor containing a descriptor list address that identifies a memory location for a batch of descriptors, or a drain descriptor that causes the accelerator engine to wait for the completion of a specified preceding descriptor in the work queue before processing the drain descriptor. (Item 17) A machine-readable medium in which program code is stored, wherein when the program code is executed by a machine, the machine... An operation in which an accelerator engine processes a work descriptor, wherein the work descriptor is submitted to multiple work queues by a client. In response to detecting page faults related to work descriptors in a specific work queue, An operation to determine the page fault processing mode associated with the aforementioned work queue, If the page fault processing mode includes a first page fault processing mode, the client is notified of the operation to terminate the processing of the work descriptor associated with the page fault. A machine-readable medium that enables execution. (Item 18) The method according to item 17, further comprising the steps of, if the page fault processing mode includes a second page fault processing mode, an operation to suspend the work associated with the work descriptor and send a page fault request to the memory management hardware logic, and an operation to resume the work associated with the work descriptor after receiving a response from the memory management hardware logic. (Item 19) The method of item 17, further comprising the step of submitting information to a completion log buffer and / or event log in response to the page fault, wherein the information is related to the page fault. (Item 20) The method according to item 19, wherein, based on the information, if the work related to the descriptor is only partially completed due to a page fault, the client continues to process the work or resubmits it. (Item 21) The method according to item 20, wherein the information includes a fault address and a byte completion field for indicating the number of source bytes processed before the page fault occurred. (Item 22) The method according to item 19, wherein when the page fault occurs at an address related to the completion record buffer, the information is submitted to the event log but not to the completion record buffer. (Item 23) The method according to item 19, wherein when the page fault information about the first work descriptor is submitted to the event log, the page fault information about the second work descriptor is also submitted to the event log but not to the completion record buffer. (Item 24) The method according to item 23, wherein at least one work descriptor includes a batch descriptor including a descriptor list address identifying a memory location of a batch of descriptors, or a drain descriptor for causing the accelerator engine to wait for completion of a specified preceding descriptor in the work queue before processing the drain descriptor.

Claims

1. It is a device, An accelerator having multiple data copy engines that process work descriptors submitted to a work queue, wherein the work descriptor identifies a memory location of a list of work to be processed by the multiple data copy engines, and status information associated with the work descriptor is stored in both a completion record buffer and an event log associated with the work queue, and the status information indicates whether the work was not completed due to an error, and the accelerator Fault handling hardware logic related to the plurality of data copy engines, which implements a specified page fault handling mode from among a plurality of page fault handling modes for the work queue, wherein the page fault handling mode includes a first page fault handling mode and a second page fault handling mode, and when a page fault occurs on an address related to the completion record buffer, information related to the page fault is submitted to the event log but not to the completion record buffer, and A device equipped with the following features.

2. The apparatus according to claim 1, wherein in the first page fault processing mode, the fault processing hardware logic generates a notification and terminates processing of the work descriptor.

3. The apparatus according to claim 2, wherein in the second page fault processing mode, the processing of the work descriptor is temporarily suspended, and then the processing of the work descriptor is resumed.

4. The apparatus according to claim 3, wherein the status information includes partially completed information from the list indicating completed work.

5. The apparatus according to claim 4, wherein the partially completed information is used to resume processing of workpieces that are indicated as incomplete in the list.

6. The apparatus according to claim 1, wherein, in order to resolve the page fault, the fault handling hardware logic generates a page request to the memory management hardware logic for the page on which the page fault occurred and receives a response including an indication of the page.

7. The apparatus according to claim 6, wherein the memory management hardware logic includes an input / output memory management unit (IOMMU) that translates the virtual address associated with the page to the physical address of the page.

Images