A technique for efficiently allowing simultaneous access to resources in a computing system

Abstract

The present application discloses a technique for effectively allowing simultaneous access to resources in a computing system, where memory page table failures of multiple execution contexts (client or guest machines) of a memory system are typically queued in a single physical command queue. Multiple execution contexts compete for access to the queue, resulting in poor performance. A virtual interface and one or more virtual command queues are assigned to each guest, rather than competing with other execution contexts to insert failure instructions into a single physical command queue. The execution contexts can simultaneously send failure instructions of the memory system through their respective virtual interfaces. In addition, each execution context can also transmit other (less frequently issued) commands through a hypervisor. Error handling and / or illegal access checks for failure instructions previously performed by the hypervisor are now performed by the respective virtual interfaces.

Description

Technical Field

[0001] The present invention relates to command queues, and more particularly to virtualization of command queues accessed by multiple execution contexts. Background Art

[0002] Storage page table invalidations for multiple execution contexts (guests, clients, processes, and / or CPUs) of a memory system are typically queued in a single physical command queue. Multiple execution contexts contend for access to the queue, resulting in poor performance. Furthermore, in a virtualized environment implemented using a hypervisor, invalidation requests generated by the client must be captured by the hypervisor, resulting in wasted cycles and / or delays on the CPU, which negatively impacts performance. There is a need to address these and / or other issues associated with the prior art. Summary of the invention

[0003] Memory page table invalidations for multiple execution contexts (client or guest) of a memory system are typically queued in a single physical command queue. Multiple execution contexts compete for access to the queue, resulting in poor performance. In a virtualized environment implemented using a hypervisor, since the single physical command queue is owned by the hypervisor, invalidation requests generated by the guest must be trapped by the hypervisor, resulting in delays that negatively impact performance. Rather than competing with other execution contexts to insert invalidation instructions into a single physical command queue or generate requests trapped by the hypervisor, a virtual interface and one or more virtual command queues are assigned to each client. Execution contexts can send invalidation commands for the memory system simultaneously through their respective virtual interfaces. In addition, each execution context can also transmit other (less frequently issued) commands through the hypervisor. Error handling and / or illegal access checks for invalidation instructions previously performed by the hypervisor are now performed by the respective virtual interfaces.

[0004] A method, computer readable medium, and system are disclosed for providing a virtual command queue through which an execution context accesses a shared physical command queue. The method provides a first virtual command queue for issuing failure commands to a first execution context of a memory system, each failure instruction corresponding to a portion of memory in the memory system. In one embodiment, the portion of memory is allocated to the first execution context. A first failure instruction is received from the first execution context on the first virtual command queue and stored in the first virtual command queue. A command is received from the first execution context through a hypervisor and stored in a shadow command queue associated with the hypervisor. A scheduler selects a command for execution by the memory system and selects a first failure instruction for execution by the memory system. The command and the first failure instruction are stored in a physical command queue. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1A A block diagram of a virtual command queue system according to one embodiment is illustrated.

[0006] Figure 1B According to an embodiment, Figure 1A Block diagram of the command queue virtualizer.

[0007] Figure 1C A flow chart of a method for implementing a command queue virtualizer according to one embodiment is illustrated.

[0008] Figure 2 A flow chart of a method of receiving a command on a virtual interface according to one embodiment is illustrated.

[0009] Figure 3 An exemplary system is described in which the various architecture and / or functionality of the various previous embodiments may be implemented. Detailed Description

[0010] The memory page table invalidations of multiple execution contexts of the memory system are typically queued in a single physical command queue. In the context described below, the term execution context may refer to a guest, a client application, a client, an operating system (OS), a thread, a process, a processor, an interrupt context, a central processing unit (CPU), or a logical CPU. In the context described below, a thread is a virtual CPU created by an operating system abstraction. In the context described below, a logical CPU is viewed as a CPU by software that can be a guest OS or a native OS. The memory system may be managed by a memory management unit (MMU) included in the processor or a system MMU shared by multiple processors. Multiple clients compete for access to the queue, resulting in poor performance. In a virtualized environment implemented using a conventional hypervisor, invalid requests generated by the client must be captured by the conventional hypervisor, resulting in delays that negatively affect performance.

[0011] Rather than competing with other clients to insert retired instructions into a single physical command queue through the hypervisor, a virtual interface and one or more virtual command queues are assigned to each client. Rather than generating requests that are trapped by the hypervisor, a virtual interface and one or more virtual command queues are assigned to each client. Execution contexts (client or client) can send retired instructions to the memory system simultaneously through their respective virtual interfaces. In addition, each execution context can also transmit other (low-frequency or infrequently issued) commands through the hypervisor, just as is done for all commands in a traditional system. Error handling and / or illegal access checks for retired instructions previously performed by the hypervisor are now performed by the respective virtual interfaces. The hypervisor performs error handling for other commands that are not transmitted through the virtual interface.

[0012] Figure 1A A block diagram of a virtual command queue system 100 according to one embodiment is illustrated. The virtual command queue system 100 includes N execution contexts 110-1 through 110-N, a hypervisor 115, a command queue virtualizer 120, and a physical command queue 125.

[0013] In traditional MMU systems, for example (Advanced Reduced Instruction Set Computing (RISC) Machine) SMMU architecture, the conventional hypervisor communicates directly with the physical command queue 125, and the execution context issues commands only through the conventional hypervisor. In the conventional command system, there is only a single communication path between the execution context and the physical command queue 125; in particular, all invalidation instructions issued by the guest (e.g., commands to invalidate page table entries of memory allocated to the guest) are passed through the conventional hypervisor before being input to the physical command queue 125. In one embodiment, the memory is such as Figure 3 Memory 304 is shown.

[0014] In a virtualized environment running multiple guests managed by a hypervisor, invalidation commands for memory in the virtual space need to be sent to the MMU (SMMU). Conventional hypervisors capture invalidation commands and send the captured invalidation commands to the physical command queue 125 after performing the required illegal access checks. The capture operation is expensive and may require thousands of CPU cycles. In a bare metal system, when different processes / threads are running on different CPUs, each CPU attempts to issue invalidation commands at the same time, and the first CPU will have to wait (e.g., spin lock) to acquire the lock on the physical command queue 125 while the second CPU completes the invalidation operation and releases the lock on the physical command queue 125. The first CPU may spend many CPU cycles waiting for the lock to be released, thereby reducing performance when other processing may be performed.

[0015] In the virtual command queue system 100, one or more virtual command queues are assigned to each execution context 110. Each execution context 110 can issue retire instructions to one or more virtual command queues instead of issuing retire instructions through the hypervisor 115. In one embodiment, the virtual command queues are implemented in a combination of registers in memory mapped input / output (MMIO) space and physical queues in memory, specifically guest dynamic random access memory (DRAM) space.

[0016] Error handling and / or illegal access checks for invalid commands executed by conventional hypervisors are performed by each virtual interface implemented by the command queue virtualizer 120. The communication path between the execution context 110 and the hypervisor 115 is maintained and can be used for the execution context 110 to indirectly issue commands to the command queue virtualizer 120. In one embodiment, the execution context 110 issues low-frequency or low-priority commands to the hypervisor 115. In one embodiment, the execution context 110 issues low-frequency or low-priority commands to the hypervisor 115. In one embodiment, multiple execution contexts 110 can simultaneously issue invalid commands to the command queue virtualizer. In one embodiment, the communication path from the execution context 110 via the command queue virtualizer to the physical command queue has a lower latency than the communication path from the execution context 110 via the hypervisor 115 and the command queue virtualizer to the physical command queue.

[0017] Arbitration is performed between invalidation commands received by one or more virtual queues and commands received by the hypervisor to insert commands into the physical command queue 125. The physical command queue 125 does not simply receive commands from a conventional hypervisor, but rather receives commands from the command queue virtual machine 120. However, in one embodiment, the hypervisor 115 interacts with the command queue virtual machine 120 in a manner that is compatible with conventional interfaces between conventional hypervisors and the physical command queue 125.

[0018] Compared to conventional command systems, when the number of virtual command queues is greater than the number of active execution contexts 110 in the virtual command queue system 100, the execution context 110 does not need to obtain the lock of the physical command queue, but can simply issue commands to the command queue virtualizer 120. When the number of virtual command queues is not greater than the number of active execution contexts 110, the latency of obtaining the lock is significantly reduced. In addition, in the virtual command queue system 100, the invalidation command sent directly by the guest is received by the command queue virtualizer 120, thereby reducing the need for the hypervisor 115 to capture each invalidation instruction.

[0019] Although the virtual command queue system 100 is described in the context of a processing unit, one or more units (including execution contexts 110-1 to 110-N), hypervisor 115, command queue virtualizer 120, and physical command queue 125 may be implemented as a program, custom circuitry, or a combination of custom circuitry and programs. For example, command queue virtualizer 120 may be implemented by a GPU (graphics processing unit), a CPU (central processing unit), or any processor capable of performing the operations of command queue virtualizer 120. In addition, it will be understood by those of ordinary skill in the art that any system that performs the operations of virtual command queue system 100 is within the scope and spirit of embodiments of the present invention.

[0020] Now, more illustrative information about various optional architectures and features will be described, depending on the needs of the user, with which the above-mentioned architecture can implement the above-mentioned framework. It should be particularly noted that the following information is provided for illustrative purposes and should not be interpreted as limiting in any way. Any of the following features can be optionally combined with the other features described or do not exclude other features.

[0021] Figure 1B According to an embodiment, Figure 1A 1 is a block diagram of a command queue virtualizer 120. The command queue virtualizer 120 includes N virtual interfaces 105, one for each client 110. Each virtual interface 105 is assigned one or more virtual command queues 140. In one embodiment, the virtual command queue 140 is a circular buffer. Figure 1B As shown, the execution context 110-1 issues a command to the virtual interface 110-1, and the virtual interface 110-1 is configured to access M virtual command queues 140-1. Similarly, the execution context 110-N issues a command to the virtual interface 105-N, and the virtual interface 105-N is configured to access K virtual command queues 140-N, where M and K may be equal or unequal. In other words, the number of virtual command queues 140 assigned to each virtual interface 105 and / or each execution context 110 may be specified, and the two may be different. In one embodiment, the virtual command queues 140 are not directly visible to the respective execution contexts 110, but are accessed through the virtual interfaces 105 assigned to the respective execution contexts 110. The command queue virtualizer 120 also includes a command queue scheduler 130 and a shadow command queue 135. In one embodiment, the hypervisor 115 is configured to communicate with the shadow command queue 135 through the hypervisor interface 105.

[0022] In one embodiment, each execution context 110 is a virtual machine in a virtual environment, which is assigned a virtual interface 105. In the context of the following description, the execution context 110-1, the virtual interface 105-1, and the virtual command queue 140-1 are used as examples to describe the functions performed by any execution context 110, the virtual interface 105, and the virtual command queue 140, respectively. In one embodiment, the hypervisor 115 programs the virtual interface 105-1 with the virtual machine identifier (VMID) of the execution context (virtual machine) and the execution context ID of the physical and / or virtual device assigned to the execution context. Then, the hypervisor 115 assigns M virtual command queues 140 from the pool of available command queues 140 to the virtual interface 105-1.

[0023] When the virtual interface 105-1 receives a command, the virtual interface 105-1 checks whether the virtual command queue 140 accessed by the execution context 110-1 is assigned to the execution context 110-1, so that a malicious execution context does not pollute the virtual command queue 140 assigned to a different execution context 110. When the virtual interface 105-1 receives an invalidation command, the virtual interface 105-1 checks whether the invalidation command specifies the correct VMID or stream ID for the execution context 110, and whether the invalidation command is a legitimate command. The stream ID can point to a physical device assigned to a virtual machine or a virtual device assigned to a virtual machine. The execution context is only allowed to send invalidation instructions to a physical or virtual device (indicated by the stream ID) assigned to the execution context. The virtual interface 105-1 only allows commands that pass the check to be scheduled to the command queue scheduler 130 for processing. In one embodiment, if the hypervisor 115 requires, the virtual stream ID can be mapped to the physical stream ID using the stream ID check / mapping control.

[0024] When a command fails the checks performed by the virtual interface 105, the virtual interface 105 stops processing any further commands and sets an error status flag. In one embodiment, when the hypervisor 115 defines the virtual interface 105 for its own use, the virtual interface 105 is configured with the checks disabled because the checks are performed by the hypervisor 115 before forwarding invalid requests to the virtual interface 105.

[0025] Each virtual interface 105 is configured by the hypervisor 115 to interface between the execution context 110 and one or more virtual command queues 140 assigned to the execution context 110 in memory. The virtual interface 105 for the execution context 110 provides a consumer index and a producer index for each virtual command queue 140 assigned to the execution context 110. In one embodiment, the producer index and the consumer index are located in the address space assigned to the virtual command queue 140. In one embodiment, the producer index is provided in the register space, the consumer index is provided in DRAM (cacheable), and the queue to which the actual command is written is provided in memory. When the consumer index is updated, the CPU execution context 110 can be woken up, so there is no need to poll the consumer index.

[0026] When commands are queued, the producer index of the virtual command queue 140 will be incremented; when commands are executed, the consumer index will be incremented. For example, when the execution context 110-1 writes a retired instruction to the virtual command queue 140-1, the execution context 110-1 increases the producer index of the virtual command queue 140-1 through the virtual interface 105-1. When the MMU executes the retired instruction, the MMU updates the consumer index of the physical command queue 125, and the command queue scheduler 130 increases the consumer index of the virtual command queue 140-1. In one embodiment, when the consumer index is stored in its own cache line in DRAM, the execution context 110-1 will use a "wait for event" (WFE) to poll for retired instruction execution.

[0027] Whenever one or more commands are queued into a virtual command queue 140 assigned to an execution context 110, the virtual interface 105 fetches the commands from the DRAM and dispatches the commands to the command queue scheduler 130. In one embodiment, regardless of the size of the commands, each command resides in a separate cache line in the DRAM, and the virtual interface 105 fetches the commands from the virtual command queue in the DRAM.

[0028] The command queue scheduler 130 arbitrates between the virtual command queues 140 and the shadow command queues 135. The shadow command queues 135 receive commands from the hypervisor 115, providing a path between the hypervisor 115 and the physical command queues 125. The command queue scheduler 130 transfers selected retired commands from the virtual command queues 140 and selected commands from the shadow command queues 135 to a single shared physical command queue 125. In one embodiment, the command queue scheduler 130 uses a hierarchical weighted round-robin (WRR) arbitration policy to provide fairness to each virtual interface 105 and virtual command queue 140. In one embodiment, the command queue scheduler 130 ensures that no more than a predetermined number of commands are processed from a single virtual interface 105 at a given time to ensure that denial of service (DOS) attacks from a given virtual machine execution context 110 are prevented. In one embodiment, the command queue scheduler 130 checks which virtual interfaces 105 have commands ready to be scheduled (e.g., stored in at least one virtual command queue) and selects a particular virtual interface 105 for operation only if the number of running commands for the particular virtual interface 105 does not exceed a predetermined number of processed commands.

[0029] In one embodiment, the shadow command queue 135 is a circular buffer allocated in memory. The shadow command queue 135 is associated with a dedicated shadow producer index and a dedicated shadow consumer index for tracking command execution and queue integrity. In one embodiment, the shadow producer index is provided in register space and the queue to which the actual commands are written is provided in memory.

[0030] The command queue scheduler 130 arbitrates between the different virtual command queues 140 and shadow command queues 135 to insert commands into the physical command queue 125 for execution by the MMU. The command queue scheduler 130 updates the dedicated physical producer index when inserting retired instructions into the physical command queue 125, and receives updates of the dedicated physical consumer index from the MMU when executing commands. The command queue scheduler 130 passes updates of the physical consumer index to the corresponding virtual interface 105 or shadow command queue 135 for each retired instruction executed by the MMU. Unlike conventional systems where only the conventional hypervisor can write to the physical command queue 125, in the virtual command queue system 100, any execution context 110 can also write to the physical command queue without communicating with the hypervisor 115.

[0031] Figure 1C A flow chart of a method 150 for implementing a command queue virtualizer 120 according to one embodiment is illustrated. Although the method 150 is described in the context of a processing unit, the method 150 may also be performed by a program, a custom circuit, or a combination of a custom circuit and a program. For example, the method 150 may be performed by a GPU (graphics processing unit), a CPU (central processing unit), or any processor capable of performing the operations of the command queue virtualizer 120. In addition, it will be appreciated by those of ordinary skill in the art that any system that performs the method 150 is within the scope and spirit of the embodiments of the present invention.

[0032] In step 155, a first execution context 110-1 of a memory system has a first virtual command queue 140-1 for issuing retire instructions, each retire instruction corresponding to a portion of memory in the memory system. In one embodiment, the portion of memory is assigned to the first execution context 110-1. In one embodiment, the retire instruction corresponding to the portion of memory is configured to invalidate a page table entry of the portion of memory.

[0033] At step 160, a first retire instruction is received from the first execution context 110-1 at the first virtual command queue 140-1. In one embodiment, the second execution context 110-2 of the memory system has a second virtual command queue 140-2 for issuing retire instructions to a portion of memory in the memory system assigned to the second execution context 110-2. In one embodiment, a second retire instruction is received from the second execution context 110-2 at the second virtual command queue 140-2 while the first retire instruction is received from the first execution context 110-1.

[0034] At step 165, the first retired instruction is stored in the first virtual command queue 140-1. In one embodiment, the first retired instruction is stored in a separate cache line in the first virtual command queue 140-1 compared to other retired instructions stored in the first virtual command queue 140-1. When fewer virtual command queues 105 are supported than the number of CPU execution contexts 110, storing each command in a separate cache line in the first virtual command queue 140-1 may enable multiple CPU execution contexts 110 to access the command queue 140-1.

[0035] In one embodiment, after storing the first retired instruction to the first virtual command queue 140-1, the first execution context 110-1 updates the producer index of the first virtual command queue 140-1. In one embodiment, the producer index is stored in a cache line, separate from the producer indexes corresponding to other virtual command queues 140. In one embodiment, the producer index is stored in a register accessible by the command queue scheduler 130. In one embodiment, the register is in the MMIO address space and is written by an uncached write. In one embodiment, in response to the execution of the first retired instruction, the consumer index corresponding to the first virtual command queue 140-1 is updated. In one embodiment, the consumer index is stored in a cache line, separate from the consumer indexes corresponding to other virtual command queues 140. In one embodiment, the first execution context 110-1 waits for the consumer index to be updated before continuing processing. The command queue scheduler 130 tracks commands dispatched to the physical command queue 125, and when the MMU updates the consumer index, the command queue scheduler 130 updates the consumer index of the virtual command queue 140 that provided the command.

[0036] At step 170, a command of the first execution context 110-1 is received by the hypervisor 115. At step 175, the command is stored in the shadow command queue 135 associated with the hypervisor 115. At step 180, the command queue scheduler 130 selects the command for execution by the memory system and stores the command in the physical command queue 125. At step 185, the command queue scheduler 130 selects the first retired instruction for execution by the memory system and stores the first retired instruction in the physical command queue 125.

[0037] The first execution context 110-1 may issue commands, which may be retired instructions, using a hardware accelerated communication path through the first virtual command queue 140-1, or a non-accelerated path through the hypervisor 115. In addition, multiple execution contexts 110 may issue commands simultaneously through corresponding virtual command queues 140, and a single execution context 110 may issue commands to one or more virtual command queues 140 assigned to the single execution context 110 simultaneously.

[0038] The virtual interface 105 can be configured to perform checks to ensure that a malicious virtual machine does not send illegal commands or attempt a DOS attack. These checks can also ensure that a malicious virtual machine cannot invalidate contexts owned by other virtual machines. In one embodiment, the virtual interface 105 is implemented in circuits and the checks are performed by the circuits. Each virtual interface 105 provides a control window through which access to the virtual command queue 140 and the physical command queue 125 by the execution context 110 can be restricted.

[0039] Figure 2 A flow chart of a method 200 for receiving a command on a virtual interface 105 according to one embodiment is illustrated. Although the method 200 is described in the context of a processing unit, the method 200 may also be performed by a program, a custom circuit, or a combination of a custom circuit and a program. For example, the method 200 may be performed by a GPU (graphics processing unit), a CPU (central processing unit), or any processor capable of performing virtual interface 105 operations. In addition, a person of ordinary skill in the art will appreciate that any system that performs the method 200 is within the scope and spirit of the embodiments of the present invention.

[0040] At step 210, a command is received at virtual interface 105-1. In one embodiment, the command is a fail instruction. At step 215, virtual interface 105-1 determines whether an error flag is set for virtual interface 105-1. In one embodiment, when an error flag is set for a particular virtual interface 105, virtual interface 105 will be unset and any further processing of commands issued by virtual interface 105 will be stopped.

[0041] At step 220, virtual interface 105-1 determines whether the command is legal. In one embodiment, hypervisor 115 configures virtual interface 105 to process a first set of commands issued by execution context 110, while hypervisor 115 processes a second set of commands issued by execution context 110, wherein a particular command is in either the first set or the second set, but not in both sets. Defining the first and second sets of commands can ensure that minimal command workload is offloaded from hypervisor 115 to command queue virtualizer 120.

[0042] In one embodiment, the first group of commands includes invalid instructions, and in step 220, when the command is an invalid instruction, the virtual interface 105-1 determines that the command is legal and continues to execute step 225. In one embodiment, the second group of commands includes invalid flow table entry (STE) commands because the command affects the execution context 110 to which the command is directed. In one embodiment, the second group of commands includes invalid flow table input commands, and in step 220, when the command is an invalid flow table input command, the virtual interface 105-1 determines that the command is illegal, continues to execute step 245 and ends processing of the command. In one embodiment, when the error flag is set, the command is discarded. In one embodiment, when the error flag is set, the first virtual command queue 140-1 is flushed. In one embodiment, when the error flag is set for a particular virtual interface 105, the virtual interface 105 is released and any further processing of commands issued by the virtual interface 105 is stopped.

[0043] In step 225, virtual interface 105-1 determines that the command is a retired instruction in a memory portion that is not allocated to execution context 110-1, and continues to execute step 240. Otherwise, when virtual interface 105-1 determines that the command is a retired instruction in a memory portion that is allocated to execution context 110-1, virtual interface 105-1 continues to execute step 230. In one embodiment, hypervisor 115 defines a memory mapping to ensure isolation between execution contexts 110. In one embodiment, the configuration of virtual command queue 120 is part of a separate page of memory that is only mapped on hypervisor 115, and the registers allocated for virtual command queue 140 are part of a different page for each virtual interface 105.

[0044] In step 230, the virtual interface 105-1 stores the command in the virtual command queue 140-1. In step 235, the virtual interface 105-1 updates the producer index of the virtual command queue 140-1. In one embodiment, after updating the producer index, the execution context 110-1 waits for the command to be completed by polling the consumer index until the consumer index is updated. During the polling, the execution context 110-1 can continue to write commands to other virtual command queues 140 or the same virtual command queue 140. In one embodiment, the thread executing on the CPU execution context 110-1 obtains exclusive access to (e.g., lock) the consumer index location and executes a WFE on the CPU execution context 110-1. The CPU execution context 110-1 enters a sleep state and wakes up when the consumer index location is updated, for example, when the MMU updates the consumer index.

[0045] In summary, each execution context 110 can issue page table invalidation instructions through a dedicated virtual interface 105 and one or more virtual command queues 140, and can also issue non-accelerated / infrequent commands to the hypervisor 115. The virtual interface 105 allows multiple client execution contexts to queue invalidation instructions in parallel without being captured by the hypervisor 115. As with traditional systems, the hypervisor can queue requests on the physical command queue 125. However, instead of directly accessing the physical command queue 125, the hypervisor 115 routes the request through the shadow command queue 135 to access the physical command queue 125. Importantly, the virtual interface 105 of each execution context 110 performs error checking and / or illegal access checking on each command queued through the virtual command queue 140 assigned to the execution context 110, and when an error is found, the virtual interface 105 associated with the virtual command queue 140 containing the error is released. The command queue scheduler 130 selects commands from the shadow command queue 135 and the virtual command queue 140 to output to the physical command queue, and performs queue management operations on the physical command queue 125 and the virtual command queue (producer and consumer index updates).

[0046] Exemplary Computing System

[0047] As developers expose and exploit more parallelism in applications such as artificial intelligence computing, systems with multiple GPUs and CPUs are used in a variety of industries. To solve larger problems, data centers, research institutions, and supercomputers have deployed high-performance GPU-accelerated systems with tens to thousands of computing nodes. As the number of processing devices in high-performance systems increases, communication and data transmission mechanisms need to scale to support the increased bandwidth.

[0048] Figure 3An exemplary system 565 is illustrated in which various architectures and / or functions of various previous embodiments may be implemented. The exemplary system 565 may be configured to implement Figure 1C The method 150 and Figure 2 The method 200 shown in . The exemplary system 565 can be implemented using one or more single semiconductor platforms. In the context of this specification, a single semiconductor platform can refer to an integrated circuit based on a single semiconductor manufactured on a wafer or chip. It should be noted that the term single semiconductor platform can also refer to multi-chip modules with enhanced connections that simulate on-chip operations and provide substantial improvements over traditional bus implementations. Of course, various circuits or devices can also be placed separately or in various semiconductor platform combinations, depending on the needs of the user. Alternatively, the exemplary system 565 can be implemented as a circuit board substrate, and each processor 530 and / or memory 540 can be a packaged device. As shown, the system 565 includes at least one processor (CPU or GPU) 530 connected to a communication bus 575. The communication bus 575 can be implemented using any suitable protocol, such as high-speed NVLink, PCI (Peripheral Component Interconnect), PCI Express, AGP (Accelerated Graphics Port), HyperTransport, or any other bus or point-to-point communication protocol. The system 565 also includes a main memory 540. As Figure 3 As shown, the memory 540 and the processor 530 are implemented as a system on a chip (SOC).

[0049] In one embodiment, processor 530 is configured to implement a graphics rendering pipeline for processing three-dimensional (3D) graphics data to generate two-dimensional (2D) image data for display on a display device such as a liquid crystal display (LCD) device. In other embodiments, processor 530 may be used to perform general purpose computations.

[0050] One or more processors 530 can be configured to accelerate thousands of high-performance computing (HPC), data center and machine learning applications. Processor 530 can be configured to accelerate many deep learning systems and applications, including autonomous driving platforms, deep learning, high-precision speech, image and text recognition systems, intelligent video analysis, molecular simulation, drug discovery, disease diagnosis, weather forecasting, big data analysis, astronomy. Molecular dynamics simulation, financial modeling, robotics, factory automation, real-time language translation, online search optimization and personalized user recommendations, etc.

[0051] The processor 530 may include one or more MMUs, each of which may include one or more command queue virtualizers 120. The memory 540 may include several memory devices. Control logic (software) and data are stored in the main memory 540, which may be in the form of a random access memory (RAM). In one embodiment, the memory 540 may include multiple dynamic random access memory (DRAM) devices. The DRAM device may be configured as a high bandwidth memory (HBM) subsystem, with multiple DRAM chips stacked in each device.

[0052] Memory 540 may be implemented using a memory device such as a high bandwidth memory stack or a graphics double data rate, version 5, synchronous dynamic random access memory, or other type of persistent memory. In one embodiment, memory 540 supports single error correction double error detection error correction double error detection (SECDED) error correction code (ECC) to protect data. ECC provides higher reliability for computing applications that are sensitive to data corruption. Reliability is particularly important in large cluster computing environments where processor 530 processes very large data sets and / or long-running applications.

[0053] The system 565 also includes an input device 560 and a display device 545, such as a conventional CRT (cathode ray tube), LCD (liquid crystal display), LED (light emitting diode), plasma display, etc. User input can be received from the input device 560, such as a keyboard, mouse, touch pad, microphone, etc. Each of the above modules and / or devices can even be located on a single semiconductor platform to form the system 565. Alternatively, various modules can also be placed separately or in various semiconductor platform combinations according to user needs.

[0054] Furthermore, for communication purposes, system 565 can be coupled to a network (e.g., a telecommunications network, a local area network (LAN), a wireless network, a wide area network (WAN) such as the Internet, a peer-to-peer network, a cable network, etc.) through network interface 535.

[0055] System 565 may also include auxiliary storage (not shown). Auxiliary storage includes, for example, a hard disk drive and / or a removable storage drive, representing a floppy disk drive, a tape drive, an optical disk drive, a digital versatile disk (DVD) drive, a recording device, a universal serial bus (USB) flash memory. The removable storage drive reads and / or writes to a removable storage unit in a well-known manner.

[0056] Computer programs or computer control logic algorithms may be stored in the main memory 540 and / or the secondary memory. When such computer programs are executed, the system 565 is enabled to perform various functions. The memory 540, memory and / or any other memory are possible examples of computer readable media.

[0057] The architecture and / or functionality of the various previous figures may be implemented in the context of a general purpose computer system, a circuit board system, an entertainment specific gaming system, an application specific system, and / or any other desired system. For example, the system 565 may be implemented in the context of a desktop computer, a laptop computer, a tablet computer, a server, a supercomputer, a smartphone (e.g., a wireless handheld device), a personal digital assistant (PDA), a digital camera, a vehicle, a head mounted display, a handheld electronic device, a mobile phone device, a television, a workstation, a gaming console, an embedded system, and / or any other type of logic.

[0058] Although various embodiments have been described above, it should be understood that they have been presented by way of example only and not limitation. Thus, the breadth and scope of a preferred embodiment should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

[0059] It should be noted that the techniques described herein may be implemented in executable instructions stored in a computer-readable medium for use with or in connection with a processor-based instruction execution machine, system, device, or apparatus. It should be appreciated by those skilled in the art that for some embodiments, various types of computer-readable media for storing data may be included. As used herein, "computer-readable media" includes one or more any suitable media for storing executable instructions of a computer program so that an instruction execution machine, system, device, or apparatus can read (or obtain) instructions from a computer-readable medium and execute the instructions to implement the described embodiments. Suitable storage formats include one or more of electronic, magnetic, optical, and electromagnetic formats. A non-exhaustive list of conventional exemplary computer-readable media includes: portable computer floppy disks; random access memory (RAM); read-only memory (ROM); erasable programmable read-only memory (EPROM); flash memory devices; and optical storage devices, including portable compact disks (CDs), portable digital video disks (DVDs), and the like.

[0060] It should be understood that the arrangement of the elements shown in the attached drawings is for illustrative purposes, and other arrangements are also possible. For example, one or more elements described herein may be implemented in whole or in part as electronic hardware components. Other elements may be implemented in software, hardware, or a combination of software and hardware. In addition, some or all of these other elements may be combined, some may be completely omitted, and other components may be added while implementing the functions described herein. Therefore, the subject matter described herein may be embodied in many different variations, and all of these variations are considered to be within the scope of the claims.

[0061] In order to facilitate understanding of the subject matter described herein, many aspects are described in terms of the order of actions. Those skilled in the art will recognize that various actions can be performed by dedicated circuits or circuits, program instructions executed by one or more processors, or a combination of the two. The description of any sequence of actions herein does not mean that the specific order described in the sequence must be followed. Unless otherwise specified herein or the context clearly contradicts otherwise, all methods described herein can be performed in any appropriate order.

[0062] The use of the terms "a" and "an" and "the" and similar references in the context of describing subject matter (particularly in the context of the following claims) should be interpreted to cover both the singular and the plural, unless otherwise stated herein or clearly contradicted by the context. The use of the term "at least one" followed by a list of one or more items (e.g., "at least one of A and B") should be understood to refer to one item selected from the listed items (A or B) or any combination of two or more of the listed items (A and B), unless otherwise stated herein or clearly contradicted by the context. In addition, the above description is intended to be illustrative only and not limiting, as the scope of protection sought is defined by the claims set forth below and any equivalents thereof. Unless otherwise required, the use of any and all examples or exemplary language (e.g., "such as") provided herein is intended solely to better illustrate the subject matter and does not limit the scope of the subject matter. In the claims and written description, the use of the term "based on" and other similar phrases indicates a condition that produces a result, and is not intended to cancel any other condition that produces that result. No language in the specification should be construed to indicate that any undeclared element is essential to the practice of the claimed invention.

Claims

1. A method comprising: providing a first virtual command queue for issuing retire commands for a first execution context of a memory system, each of the retire commands corresponding to a portion of memory in the memory system allocated to the first execution context; receiving, at the first virtual command queue, a first invalidation command from the first execution context; storing the first invalidation command in the first virtual command queue; receiving, by the hypervisor, a command from the first execution context; storing the command in a command queue associated with the hypervisor; selecting, by a scheduler, the command from the command queue for execution by the memory system, wherein the command is stored to a physical command queue shared by a plurality of execution contexts including the first execution context; as well as The first retire command is selected by the scheduler from the first virtual command queue for execution by the memory system, wherein the first retire command is stored to the physical command queue.

2. The method of claim 1, wherein the invalidate command corresponding to the memory portion is configured to invalidate page table entries of the memory portion.

3. The method of claim 1, further comprising: providing a second virtual command queue for a second execution context of the memory system, the second virtual command queue being used to issue invalidation commands for a portion of the memory in the memory system allocated to the second execution context; as well as A second invalidate command is received from the second execution context at the second virtual command queue concurrently with receiving the first invalidate command from the first execution context.

4. The method of claim 1, further comprising: receiving a second command from the first execution context; determining that the second command is an illegal command; as well as The second command is discarded.

5. The method of claim 1, further comprising: receiving a second invalidate command from the first execution context; determining that the second invalidate command specifies a second portion of the memory in the memory system that is allocated to a second execution context; as well as If the error flag is set, the second invalidation command is discarded.

6. The method of claim 1, further comprising: After storing the first invalidation command to the first virtual command queue, the first execution context indicates to the scheduler that the first invalidation command is ready for selection by updating a producer index of the first virtual command queue.

7. The method of claim 6, wherein the producer index is stored in a cache line separate from producer indexes corresponding to other virtual command queues.

8. The method of claim 7, wherein the producer index is also stored in a register accessible by the scheduler.

9. The method of claim 1, further comprising: In response to updating the consumer index corresponding to the physical command queue to indicate that the first invalidation command is executed, updating, by the scheduler, a second consumer index corresponding to the first virtual command queue to indicate to the first execution context that the first invalidation command is executed.

10. The method of claim 9, wherein the second consumer index is stored in a cache line separate from consumer indexes corresponding to other virtual command queues.

11. The method of claim 9, wherein the first execution context waits for the second consumer index to be updated before continuing processing.

12. The method of claim 1, wherein: The first invalidate command is stored in a cache line in the first virtual command queue that is separate from other invalidate commands stored in the first virtual command queue.

13. A system comprising: a memory including a first virtual command queue, a command queue associated with a hypervisor, and a physical command queue, the physical command queue being shared by a plurality of execution contexts including the first execution context; A processor configured to execute instructions, wherein the instructions implement at least a portion of a command queue virtualizer, wherein the command queue virtualizer is configured to: providing a first virtual command queue for issuing invalidation commands for said first execution context of a memory system, each of said invalidation commands corresponding to a portion of memory storage allocated to said first execution context; receiving, at the first virtual command queue, a first invalidation command from the first execution context; storing the first invalidation command in the first virtual command queue; receiving, by the hypervisor, a command from the first execution context; storing the command in a command queue associated with the hypervisor; selecting, by a scheduler, the command from the command queue for execution by the memory system, wherein the command is stored to the physical command queue; as well as The first retire command is selected by the scheduler from the first virtual command queue for execution by the memory system, wherein the first retire command is stored to the physical command queue.

14. The system of claim 13, wherein the invalidate command corresponding to the portion of memory is configured to invalidate page table entries for the portion of memory.

15. The system of claim 13, wherein the command queue virtualizer is further configured to: providing a second virtual command queue for a second execution context of the memory system, the second virtual command queue being used to issue invalidation commands to a portion of the memory in the memory system allocated to the second execution context; and A second invalidate command is received from the second execution context at the second virtual command queue concurrently with receiving the first invalidate command from the first execution context.

16. The system of claim 13, wherein the command queue virtualizer is further configured to: receiving a second command from the first execution context; determining that the second command is an illegal command; and The second command is discarded.

17. The system of claim 13, wherein the command queue virtualizer is further configured to: receiving a second invalidate command from the first execution context; determining that the second invalidate command specifies a second portion of the memory in the memory system that is allocated to a second execution context; and If the error flag is set, the second invalidation command is discarded.

18. The system of claim 13, wherein the command queue virtualizer is further configured to indicate to the scheduler that the first invalidation command is ready for selection by updating a producer index of the first virtual command queue after storing the first invalidation command in the first virtual command queue.

19. A non-transitory computer readable medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the following steps: providing a first virtual command queue for issuing retire commands for a first execution context of a memory system, each of the retire commands corresponding to a portion of memory in the memory system allocated to the first execution context; receiving, at the first virtual command queue, a first invalidation command from the first execution context; storing the first invalidation command in a first virtual command queue; receiving, by the hypervisor, a command from the first execution context; storing the command in a command queue associated with the hypervisor; selecting, by a scheduler, the command from the command queue for execution by the memory system, wherein the command is stored to a physical command queue shared by a plurality of execution contexts including the first execution context; as well as The first retire command is selected by the scheduler from the first virtual command queue for execution by the memory system, wherein the first retire command is stored to the physical command queue.

Images