Command isolation for memory-based communication queues

Abstract

The command manager may be configured to enforce a respective command execution policy for each of the plurality of queues in accordance with a respective command fence instruction. In an example, the command manager may be configured to: receive a first packet including a first fence instruction and a first command for a first queue; in response to the fact that the first fence instruction indicates fence participation, the fence counter is increased progressively; and providing the first command to a command execution unit, such as a memory controller of a memory device. The command manager may receive a first response message from the command execution unit based on the first command and may decrement the fence counter. In an example, the command manager includes a portion of an accelerator device that communicates with a host device using an out-of-order interconnect, such as a compute fast link (CXL) interconnect.

Description

[0001] Priority application

[0002] This application claims priority to U.S. Provisional Application No. 63 / 348,327, filed on June 2, 2022, the entirety of which is incorporated herein by reference.

[0003] STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0004] This invention was made with Government support under Agreement No. DE-AC05-76RL01830 awarded by the U.S. Department of Energy. The Government has certain rights in this invention. Technical Field

[0005] Embodiments relate to memory devices. Some embodiments relate to command isolation in memory devices. Background Art

[0006] Memory devices used in computers or other electronic devices can be classified as volatile and non-volatile memory. Volatile memory requires power to maintain its data, and includes random access memory (RAM), dynamic random access memory (DRAM), or synchronous dynamic random access memory (SDRAM), etc. Non-volatile memory can retain stored data when not powered, and includes flash memory, read-only memory (ROM), electrically erasable programmable ROM (EEPROM), static RAM (SRAM), erasable programmable ROM (EPROM), resistance variable memory, phase change memory, storage class memory, resistive random access memory (RRAM) and magnetoresistive random access memory (MRAM), etc. Persistent memory is an architectural property of the system, where the data stored in the media is still available after a system reset or power cycle. In some examples, non-volatile memory media can be used to build a system with a persistent memory model.

[0007] The memory device may be coupled to a host (e.g., a host computing device) to store data, commands, and / or instructions for use by the host in operation of a computer or electronic system. For example, data, commands, and / or instructions may be transferred between the host and the memory device(s) during operation of a computing or other electronic system.

[0008] Various protocols or standards may be applied to facilitate communication between a host and one or more other devices, such as a memory buffer, accelerator, or other input / output device. In an example, an out-of-order protocol, such as Compute Express Link (CXL), may be used to provide high bandwidth and low latency connectivity. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element is first introduced.

[0010] Figure 1 A block diagram of an example computing system including a host and a memory device is generally illustrated.

[0011] Figure 2 An example of a Compute Express Link (CXL) system is generally described.

[0012] Figure 3 An example of a command isolation protocol diagram is generally described.

[0013] Figure 4 Describes in general terms a portion of the command isolation protocol.

[0014] Figure 5 Describes in general terms a portion of the command isolation protocol.

[0015] Figure 6 A block diagram illustrating an example machine in which any one or more of the techniques discussed herein may be utilized, implemented, or performed. DETAILED DESCRIPTION

[0016] Compute Express Link (CXL) is an open standard interconnect configured for high-bandwidth, low-latency connectivity between host devices and other devices, such as accelerators, memory buffers, and other I / O devices. CXL is designed to facilitate high-performance computing workloads by supporting heterogeneous processing and memory systems. CXL implements coherency and memory semantics on top of PCI Express (PCIe)-based I / O semantics to optimize performance.

[0017] In some examples, CXL is used in applications such as artificial intelligence, machine learning, analytics, cloud infrastructure, edge computing devices, communication systems, etc. Data processing in such applications can use various scalar, vector, matrix, and spatial architectures that can be deployed in CPUs, GPUs, FPGAs, smart NICs, or other accelerators that can be coupled using CXL links.

[0018] CXL supports dynamic multiplexing using a set of protocols that include input / output (CXL.io, based on PCIe), cache (CXL.cache), and memory (CXL.memory) semantics. In an example, CXL can be used to maintain a unified consistent memory space between a CPU (e.g., a host device or host processor) and any memory on an attached CXL device. This configuration allows the CPU and CXL devices to share resources and operate on the same memory region for higher performance, reduced data movement, and reduced software stack complexity. In an example, the CPU is primarily responsible for maintaining or managing consistency in a CXL environment. Therefore, CXL can be utilized to help reduce device cost and complexity, as well as the overhead traditionally associated with consistency across I / O links.

[0019] CXL runs on top of the PCIe PHY and provides full interoperability with PCIe. In an example, a CXL device starts link training at the PCIe Gen 1 data rate and negotiates CXL as its operating protocol if its link partner supports CXL (e.g., using the alternative protocol negotiation mechanism defined in the PCIe 5.0 specification). Thus, by leveraging the PCIe infrastructure, devices and platforms can more easily adopt CXL without having to design and validate PCIe's PHY, channels, channel extension devices, or other upper layers.

[0020] In an example, CXL supports single-stage switching to achieve fan-out to multiple devices. This enables multiple devices in a platform to be migrated to CXL while maintaining CXL's backward compatibility and low latency characteristics. In an example, CXL can provide standardized computing fabrics that support multiple logical devices (MLDs) and pooling of a single logical device, for example, using CXL switches connected to several host devices or nodes (e.g., root ports). This feature enables servers to pool resources that can be assigned based on workloads, such as accelerators and / or memory. For example, CXL can help facilitate resource allocation or dedication and release. In an example, CXL can help allocate and de-allocate memory to various host devices as needed. This flexibility helps designers avoid over-provisioning while ensuring optimal performance.

[0021] Some of the compute-intensive applications and operations mentioned herein may require or use large data sets. The memory devices storing such data sets may be configured for low latency, high bandwidth, and persistence. One problem with load-store interconnect architectures includes ensuring persistence. CXL can help solve this problem using software architecture flows and standard memory management interfaces, for example, enabling the move of persistent memory from a controller-based approach to direct memory management.

[0022] The inventors have recognized that problems to be solved include maintaining ordering in command execution. For example, operation execution and timing issues may arise in out-of-order systems, such as systems including those using CXL. In an example, a host device may create a series of commands that are queued for execution by an accelerator (e.g., a CXL device). The accelerator may retrieve or receive commands from the queue and execute the commands in order (e.g., in a first-in, first-out manner). In some applications, maintaining order may be critical to optimizing performance and ensuring valid results. For example, operation order enforcement may be important for performing nested loops or matrix calculations, where results from earlier operations are used in later operations.

[0023] The inventors have recognized that a solution to the computational ordering problem may include or use command fencing or memory barriers to enforce a particular ordering of commands, such as reads and writes in a memory system. In an example, a host device may be configured to specify whether a particular command participates in a fencing operation, or whether a particular command needs to wait until an earlier fencing command completes. In an example, the solution may include one or more status bits appended to or comprising a portion of a command message to indicate whether a particular command participates in or initiates a command fence. A command execution unit (e.g., a processor or other downstream operator) may be prohibited from reordering instructions or operations across an established fence. In some examples, a command message may have its status bits configured so that a command may pass through an established fence.

[0024] The isolation protocol or barrier protocol discussed herein may be implemented in or using a command manager for an accelerator or CXL device. The command manager may be configured to implement a respective command execution policy for each of a plurality of queues according to respective command barrier instructions. The command manager may further be configured to receive a first packet including a first barrier instruction for a first queue and a first command, and in response to the first barrier instruction indicating barrier participation, the command manager may increment a barrier counter and provide the first command to a command execution unit. The command manager may later receive a first response based on the first command from the command execution unit, and may decrement the barrier counter. In an example, the command execution unit includes or uses a general or special purpose processor or processor group, such as a thread engine, a stream engine, a memory controller of a memory device, a data mover, or other functional unit. The command and response pairs may be tracked by one or more command execution units using transaction identifiers or other techniques.

[0025] Figure 1A block diagram generally illustrates an example of a computing system 100 including a host device 102 and a memory system 104. Host device 102 includes a central processing unit (CPU) or processor 110 and host memory 108. In an example, host device 102 may include a host system such as a personal computer, desktop computer, digital camera, smart phone, memory card reader, and / or IoT-enabled device, among various other types of hosts, and may include a memory access device such as processor 110. Processor 110 may include one or more processor cores, a parallel processor system, or other CPU arrangement.

[0026] The memory system 104 includes a controller 112, a buffer 114, a cache 116, and a first memory device 118. The first memory device 118 may include, for example, one or more memory modules (e.g., a single in-line memory module, a dual in-line memory module, etc.). The first memory device 118 may include volatile memory and / or non-volatile memory, and may include a multi-chip device including one or more different memory types or modules. In an example, the computing system 100 includes a second memory device 120 that interfaces with the memory system 104 and the host device 102.

[0027] Host device 102 may include a system backplane and may include a number of processing resources (e.g., one or more processors, microprocessors, or some other type of control circuitry). Computing system 100 may optionally include separate integrated circuits for host device 102, memory system 104, controller 112, buffer 114, cache 116, first memory device 118, second memory device 120, any one or more of which may include respective chiplets that may be connected and used together. In an example, computing system 100 includes a server system and / or a high performance computing (HPC) system and / or a portion thereof. Although Figure 1 The examples shown in illustrate a system having a Von Neumann architecture, but embodiments of the present disclosure may be implemented in a non-Von Neumann architecture, which may not include one or more components (e.g., CPU, ALU, etc.) typically associated with a Von Neumann architecture.

[0028] In an example, the first memory device 118 may provide main memory for the computing system 100, or the first memory device 118 may include auxiliary memory or storage for use by the computing system 100. In an example, the first memory device 118 or the second memory device 120 includes one or more arrays of memory cells (e.g., volatile and / or non-volatile memory cells). For example, the arrays may be flash arrays with a NAND architecture. Embodiments are not limited to a particular type of memory device. For example, the memory device may include RAM, ROM, DRAM, SDRAM, PCRAM, RRAM, and flash memory, among others.

[0029] In embodiments where the first memory device 118 includes persistent or non-volatile memory, the first memory device 118 may include a flash memory device, such as a NAND or NOR flash memory device. The first memory device 118 may include other non-volatile memory devices, such as non-volatile random access memory devices (e.g., NVRAM, ReRAM, FeRAM, MRAM, PCM), memory devices including ferroelectric capacitors that may exhibit hysteresis characteristics (e.g., ferroelectric RAM devices), 3D cross point (3D XP) memory devices, etc., or combinations thereof.

[0030] In an example, the controller 112 includes a media controller, such as a non-volatile memory express (NVMe) controller. The controller 112 may be configured to perform operations such as copying, writing, reading, error correction, etc. for the first memory device 118. In an example, the controller 112 may include dedicated circuitry and / or instructions to perform various operations. That is, in some embodiments, the controller 112 may include circuitry and / or may be configured to execute instructions to control the movement of data and / or addresses associated with the data, such as among the buffer 114, the cache 116, and / or the first memory device 118 or the second memory device 120.

[0031] In an example, at least one of the processor 110 and the controller 112 includes a command manager (CM) for the memory system 104. The CM may receive, for example, a read command for a particular logical row address in the first memory device 118 or the second memory device 120 from the host device 102. In some examples, the CM may determine that the logical row address is associated with the first row based at least in part on a pointer stored in a register of the controller 112. In an example, the CM may receive a write command for the logical row address from the host device 102, and the write command may be associated with the second data. In some examples, the CM may be configured to issue an access command associated with the first memory device 118 or the second memory device 120 to the non-volatile memory and between issuing the read command and the write command. In some examples, the CM may issue an access command associated with the first memory device 118 or the second memory device 120 to the non-volatile memory and between issuing the read command and the write command.

[0032] In an example, buffer 114 includes data buffering circuitry that includes a physical memory area used to temporarily store data, such as when moving data from one place to another. Buffer 114 may include a first-in, first-out (FIFO) buffer in which the oldest (e.g., first-in) data is processed first. In some embodiments, buffer 114 includes a hardware shift register, a circular buffer, or a list.

[0033] In an example, the cache 116 includes a physical memory area used to temporarily store specific data that may be used again. The cache 116 may include a pool of data entries. In some examples, the cache 116 may be configured to operate according to a write-back policy, in which data is written to the cache without being concurrently written to the first memory device 118. Therefore, in some embodiments, data written to the cache 116 may not have a corresponding data entry in the first memory device 118.

[0034] In an example, the controller 112 may receive write requests related to the cache 116 (e.g., from the host device 102) and cause data associated with each of the write requests to be written to the cache 116. In some examples, the controller 112 may receive write requests at a rate of thirty-two (32) gigatransfers (GTs) per second, e.g., according to or using the CXL protocol. The controller 112 may similarly receive read requests and cause data stored in, for example, the first memory device 118 or the second memory device 120 to be retrieved and written to, for example, the host device 102 via the interface 106.

[0035] In an example, the interface 106 may include any type of communication path, bus, or the like that allows information to be transferred between the host device 102 and the memory system 104. Non-limiting examples of the interface may include a peripheral component interconnect (PCI) interface, a peripheral component interconnect express (PCIe) interface, a serial advanced technology attachment (SATA) interface, and / or a micro serial advanced technology attachment (mSATA) interface, etc. In an example, the interface 106 includes a PCIe5.0 interface that complies with the Compute Express Link (CXL) protocol standard. Therefore, in some embodiments, the interface 106 supports a transfer speed of at least 32GT / s.

[0036] As similarly described elsewhere herein, CXL is a high-speed central processing unit (CPU)-to-device or CPU-to-memory interconnect designed to enhance computing performance. CXL technology maintains memory coherency between the CPU memory space (e.g., host memory 108) and the memory on the attached device or accelerator (e.g., first memory device 118 or second memory device 120), which allows resource sharing for higher performance, reduced software stack complexity, and lower overall system cost. CXL is designed as an industry open standard interface for high-speed communications as accelerators are increasingly used to complement CPUs to support emerging data-rich and compute-intensive applications, such as artificial intelligence and machine learning.

[0037] Figure 2 An example of a CXL system 200 is generally described that uses a CXL link 206 to connect a host device 202 and a CXL device 204. In the example, the host device 202 includes or corresponds to a CXL device 204 from Figure 1 The host device 102 of the computing system 100 in the embodiment of the present invention and the CXL device 204 includes or corresponds to the host device 102 of the computing system 100 in the embodiment of the present invention. Figure 1 104 of an instance of computing system 100 in the example. A memory system command manager (CM) may include a portion of host device 202 or CXL device 204. In an example, CXL link 206 (e.g., corresponding to a Figure 1106) may support communications using multiplexed protocols for caching (e.g., CXL.cache), memory access (e.g., CXL.mem or CXL.memory), and data input / output transactions (e.g., CXL.io). CXL.io may include a PCIe-based protocol used for functions such as device discovery, configuration, initialization, I / O virtualization, and direct memory access (DMA) using non-uniform load-store, producer-consumer semantics. CXL.cache may enable a device to cache data from host memory (e.g., from host memory 212) using a request and response protocol. CXL.memory may enable a host device 202 to use memory attached to a CXL device 204, such as in a virtualized memory space or using a virtualized memory space. In an example, a CXL.memory transaction may be a memory load and store operation that is run downstream or external to the host device 202.

[0038] exist Figure 2 In an example of , host device 202 includes a host processor 214 (e.g., including one or more CPUs or cores) and IO device(s) 228. Host device 202 may include or may be coupled to host memory 212. Host device 202 may include various circuitry or logic configured to facilitate CXL-based communications and transactions with CXL device 204. For example, host device 202 may include coherence and memory logic 218 configured to implement transactions according to CXL.cache and CXL.memory semantics, and host device 202 may include PCIe logic 220 configured to implement transactions according to CXL.io semantics. In an example, host device 202 may be configured to manage the coherence of data cached at CXL device 204 using, for example, its coherence and memory logic 218.

[0039] Host device 202 may further include a host multiplexer 216 configured to modulate communications over CXL link 206 (e.g., using a PCIe PHY layer). Multiplexing of protocols ensures that latency-sensitive protocols (e.g., CXL.cache and CXL.memory) have the same or similar latency as native processor-to-processor links. In an example, CXL defines an upper bound on the response time of latency-sensitive protocols to help ensure that device performance is not adversely affected by latency variations between different devices implementing coherency and memory semantics.

[0040] In an example, a symmetric cache coherence protocol may be difficult to implement between host processors because different architectures may use different solutions, which in turn may compromise backward compatibility. CXL can address this problem by merging the coherence functionality at the host device 202, for example using the coherence and memory logic 218.

[0041] The CXL device 204 may include an accelerator device including various accelerator logic 222. In an example, the CXL device 204 may include or may be coupled to a CXL device memory 226. The CXL device 204 may include various circuitry or logic configured to facilitate CXL-based communications and transactions with the host device 202 using the CXL link 206. For example, the accelerator logic 222 may be configured to implement transactions according to CXL.cache, CXL.memory, and CXL.io semantics. The CXL device 204 may include a CXL device multiplexer 224 configured to control communications via the CXL link 206.

[0042] In an example, one or more of the coherence and memory logic 218 and the accelerator logic 222 include a unified assist engine (UAE) or compute fabric having various functional units such as a command manager (CM), a thread engine (TE), a stream engine (SE), a data manager or data mover (DM), or other units. The compute fabric may be reconfigurable and may include separate synchronous and asynchronous flows.

[0043] The accelerator logic 222 or portions thereof may be configured to operate in the application space of the CXL system 200 and, in some examples, may launch its own threads or sub-threads that may operate in parallel and may optionally use resources or units on other CXL devices 204. Queue and transaction control through the system may be coordinated by the CM, TE, SE, or DM components of the UAE. In an example, each queue or thread may be mapped to a different loop iteration to thereby support multi-dimensional loops. With the ability to launch such nested loops, among other capabilities, the system may achieve significant time savings and latency improvements for compute-intensive operations.

[0044] In an example, command isolation may be used to help maintain order throughout such operations, which may be performed locally or throughout the computational space of the accelerator logic 222. In some examples, CM may be used to route commands to a specific command execution unit (e.g., the accelerator logic 222 including a specific instance of the CXL device 204) using an out-of-order interconnect that provides corresponding transaction identifiers (TIDs) to command and response message pairs.

[0045] In CXL system 200, host device 202 may provide a series of commands to CXL device 204, for example, using a memory resident queue. CXL device 204 may retrieve or receive the commands from the memory resident queue and execute the commands in an ordered manner. In an example, host device 202 may specify that a particular command does not begin execution until one or more previous commands have completed execution. This execution ordering may be implemented in CXL system 200 using a command isolation protocol. In an example, a host or device-side application may be used to impose an ordering on a series of commands belonging to a particular queue.

[0046] Figure 3 An example of a command isolation protocol diagram 300 is generally illustrated. The isolation protocol diagram 300 includes a host command queue 306 that can be queued by a host device 202 or a CXL device 204 for execution using one or more different command execution units 318. Figure 3 In the example of , there may be up to thirty-two host command queues 306 designated as q0 through q31. The various queues may be sent for execution using command execution unit(s) 318 (which may include accelerator logic 222, for example). Results or responses from command execution unit(s) 318 may be queued using corresponding host response queues 312.

[0047] In an example, the isolation protocol supports separate respective command fences for up to thirty-two queues used by the host application process, for example, to send commands to the attached CXL devices. The isolation protocol includes or uses a per-queue fence counter 314 and status bits to control the isolation protocol. The status bits may include a fence participant (Fp) bit and a fence initiator (Fi) bit.

[0048] A fence participant (Fp) bit comprises a portion of a command packet that specifies whether a corresponding command participates or does not participate in a particular isolated operation or series of isolated operations. If the Fp bit is set (e.g., set to 1), it indicates that the corresponding command is executed before a different isolated command can be executed. As commands whose corresponding Fp bits are set are retrieved from the host command queue 306, the fence counter 314 of the corresponding host queue can be incremented.

[0049] In an example, the Fp bit may be passed through a processing engine, such as the command execution unit(s) 318, and may be returned in a response message with a command response after the command completes execution. The response message or series of response messages may be processed and returned to the host application using the host response queue 312. As responses are processed, responses with the Fp bit set may decrement the fence counter 314. In other words, the fence counter 314 may decrement its count for each response returned from the command execution unit(s) 318 that includes a set Fp bit.

[0050] The Fence Initiator (Fi) bit includes a portion of a command packet that specifies whether the corresponding command will initiate a new fence for a series of operations. If the Fi bit is set (e.g., set to 1), then the corresponding command suspends or waits for execution until all previous packets with the Fp bit set have completed execution. As packets with the Fi bit set are processed, the value of the fence counter 314 of the corresponding host queue can be checked. If the fence counter 314 is equal to 0, then it can be known that all previous Fp packets have completed execution and the packet with the Fi bit set can continue.

[0051] In an example, if the fence counter 314 is not equal to 0, it can be known or assumed that at least one of the previous packets whose Fp bit is set is executing, and the packet with the Fi bit set can be suspended. In addition, any packets after the packet with the Fi bit set can be suspended. As response messages with the Fi bit set are returned, the fence counter 314 of the queue can be decremented. When the fence counter 314 reaches 0, then the packet with the Fi bit set can proceed and command traffic processing from the queue(s) can be resumed.

[0052] In an example, after receiving a particular packet whose Fi bit is set, the isolation protocol may allow some command packets to flow. In this example, subsequent packets from the queue whose Fi and Fp bits are not set (e.g., the Fi and Fp bits are each set to 0) may be allowed to bypass any pending packets whose Fi bits are set. Receipt of further subsequent packets whose Fp or Fi bits are set may then cause a hold, where packets from the queue may not advance until the fence counter 314 is decremented to zero.

[0053] Figure 3 The example of FIG. 3 shows a specific command flow of an example first command q0_cmd of the first command queue 302 (q0). Whether the first command includes a set Fp or Fi bit determines whether or when the first command is provided to the command execution unit(s) 318. In some examples, the first command q0_cmd may include a transaction identifier (TID) that uniquely identifies the command and its corresponding response.

[0054] In a first example, if the Fp bit of a first command is set and its Fi bit is not set, the first command may be provided to the command execution unit(s) 318 and the barrier counter 314 of the first queue may be incremented (e.g., from zero or from a non-zero value). The barrier counter 314 will then have a non-zero value. If a later received command arrives whose Fp bit is set, this command may be allowed to proceed and the barrier counter 314 may be further incremented. If a later received command arrives whose Fi bit is set, this command may be suspended or queued because the non-zero value of the barrier counter 314 indicates that an isolation operation is ongoing. In response to the first command, the command execution unit(s) 318 may provide a first response message q0_rsp with the corresponding Fp bit set. The first response message may then be received at the host response queue 312 and the barrier counter 314 may be decremented.

[0055] In a second example, if the Fi bit of the first command is set, the first command may be suspended unless or until the fence counter 314 is zero. For example, the isolation protocol may include a suspension check 316. The suspension check 316 may determine whether a particular command is suspended, for example, due to an ongoing isolation operation. Figure 3 In the example of , a command may be suspended when the Fi bit of the command is set and the value of the fence counter 314 is non-zero. If the Fi bit of the command is set and the value of the fence counter 314 is zero, it can be known or assumed that no isolation operation is in progress and the corresponding command can be allowed to proceed.

[0056] In a third example, if the Fp and Fi bits of the first command are not set, then the first command may be allowed to proceed regardless of the value or state of the fence counter 314 or any other isolation operations that may be in progress. In this third example, the first command may be allowed to proceed independently because the command does not participate in isolation operations and does not initiate new fences. That is, commands that have no relationship to existing queues or threads may be allowed to pass to the command execution unit without being stopped or blocked. Therefore, this mechanism allows tuning how to use the isolation protocol to selectively stop command traffic. In a specific example, a data movement operation may not require isolation and therefore may be allowed to flow freely through the command execution unit because its results or responses may not affect other operations in the queue or other results.

[0057] Figure 4A first isolation protocol instance 400 is generally described. The first isolation protocol instance 400 may include or follow the various isolation rules shown in the isolation protocol diagram 300. At block 402, the first isolation protocol instance 400 includes using a command manager to receive a first packet including a first barrier instruction and a first command for a first queue. The first barrier instruction may include a barrier participant bit (Fp), a barrier initiator bit (Fi), or both the barrier participant bit and the barrier initiator bit.

[0058] After block 402, the instance may continue to conditionally increment the fence counter depending on the contents of the Fp and Fi bits. At decision block 404, the first isolation protocol instance 400 may include determining whether the fence initiator (Fi) bit is set (e.g., set to 1) or not set. If the Fi bit is set, indicating the initiation of a new fence, the instance may proceed to decision block 408 to determine the value of the fence counter. If the fence counter is zero, no isolation operation is in progress and the first command may be allowed to proceed and establish a new fence. Therefore, at block 412, the fence counter may be incremented. If the fence counter is non-zero at decision block 408, the first command may be suspended at 410.

[0059] Returning to decision block 404, if the Fi bit is not set, the example may continue at decision block 406. At decision block 406, the first isolation protocol instance 400 may include determining whether the fence participant (Fp) bit is set (e.g., set to 1) or not set. If the Fp bit is set, indicating participation in the fence, the example may proceed to block 412 and increment the fence counter. If the Fp bit is not set, the first command may be provided to the command execution unit at block 416.

[0060] In an example, the barrier counter includes a different counter for each of the plurality of different queues. That is, a barrier counter for a first command of a first queue may be a first barrier counter that is different from a second barrier counter for a different second command of a second queue, and so on. When a barrier initiator bit or a barrier participant bit is set in a corresponding command (e.g., in a first barrier instruction), the queue-specific barrier counter may be incremented at block 412.

[0061] At block 414, the first isolation protocol instance 400 may include providing the first command to the command execution unit. Block 414 may include passing the fence participant bit to or through the command execution unit along with other operands or instructions of the first command. In an example, the CM may hold other commands belonging to the same first queue or other queues while the command execution unit operates on or executes the first command.

[0062] At block 416, the first isolation protocol instance 400 may include receiving a first response message from the command execution unit. In an example, the first response message may include a fence participant bit that was passed to the command execution unit (e.g., at block 414) along with the first command. At block 418, based on the fence participant bit in the first response message, a queue-specific fence counter may be decremented. If the counter value is zero after block 418, the protocol may allow a new command whose corresponding fence initiator bit is set to be received or processed to establish a new command fence. If the counter value is not zero after block 418, the protocol may receive or process other commands belonging to the same queue.

[0063] Figure 5 The second isolation protocol instance 500 is generally described. The second isolation protocol instance 500 may include or follow the various isolation rules shown in the isolation protocol diagram 300. At block 502, the second isolation protocol instance 500 includes receiving a second command at a command manager. In the example, the second command packet includes a second command from Figure 4 The second command may include packets of the same first queue of an instance of or it may include packets belonging to different queues. The second command may include a fence instruction with a fence initiator (Fi) bit set. The Fi bit may indicate that the second command initiates a new fence to implement sequencing of multiple different commands.

[0064] At decision block 504, the second isolation protocol instance 500 may determine whether the barrier counter value is zero or non-zero. If the barrier counter value is non-zero, the command execution unit resource may not be available, for example because it may be occupied by a different isolation command or operation, and the second isolation protocol instance 500 may continue at block 506. At block 506, the second command may be suspended. For example, the second command may be requeued in a host queue or buffer or may be delayed in other ways. The suspended second command may be stored (e.g., locally) with the barrier counter visible so that when the barrier counter value reaches zero, the second command may be retrieved and provided to the command execution unit.

[0065] If, after decision block 504, the barrier counter value is zero, then the command execution unit resources are available to process the second command. Thus, at block 508, the second isolation protocol instance 500 may include providing the second command to the command execution unit.

[0066] exist Figure 5In an example of , the fence participant (Fp) bit of the second command may or may not be set. If the Fp bit is set, then at block 508 the fence counter value may be incremented in coordination with providing the second command to the command execution unit. A response or result may be returned from the command execution unit in response to the second command, and the fence counter value may be decremented accordingly. The counter increment and decrement operations may be performed before, during, or after any operation of the command execution unit.

[0067] Figure 6 Block diagram illustrating an example machine 600, any one or more techniques (e.g., methodology) discussed herein may utilize, be implemented in, or be implemented by the machine 600. As described herein, the example may include logic or several components or mechanisms in the machine 600, or may be operated by it. A circuit system (e.g., a processing circuit system) is a collection of circuits implemented in a tangible entity of the machine 600 including hardware (e.g., simple circuits, gates, logic, etc.). Circuit system membership (e.g., as belonging to a host-side device or process, or to an accelerator-side device or process) may change over time. The circuit system includes components that can perform specified operations alone or in combination when operating. In an example, the hardware of the circuit system may be designed invariably to perform specific operations (e.g., hardwired), such as using the accelerator logic 222 or using its specific command execution unit. In an example, the hardware of the circuit system may include variably connected physical components (e.g., command execution units, transistors, simple circuits, etc.), which include machine-readable media that are physically modified (e.g., magnetically, electrically, removably placed invariant aggregate particles, etc.) to encode instructions for specific operations. When the physical components are connected, the basic electrical properties of the hardware composition change, such as from an insulator to a conductor or vice versa. The instructions enable embedded hardware (e.g., an execution unit or a loading mechanism) to create a component of the circuit system in hardware via a variable connection to perform a portion of a specific operation when operating. Therefore, in an example, the machine-readable media element is part of the circuit system or is communicatively coupled to other components of the circuit system when the device is operating. In an example, any of the physical components may be used in more than one component of more than one circuit system. For example, in operation, an execution unit may be used in a first circuit of a first circuit system at one point in time and reused by a second circuit in the first circuit system or by a third circuit in the second circuit system at a different time.

[0068] In alternative embodiments, the machine 600 may operate as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine 600 may operate as a server machine, a client machine, or both in a server-client network environment. In an example, the machine 600 may act as a peer machine in a peer-to-peer (P2P) (or other distributed) network environment. The machine 600 may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a network appliance, a network router, a switch or a bridge, or any machine capable of executing instructions (sequentially or otherwise) that specify actions to be taken by that machine. In addition, although only a single machine is described, the term "machine" should also be interpreted as including any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more methodologies discussed herein (e.g., cloud computing, software as a service (SaaS), other computer cluster configurations).

[0069] Any one or more of the components of machine 600 may include or use one or more instances of host device 202 or CXL device 204 or other components in or attached to computing system 100. Machine 600 (e.g., a computer system) may include hardware processor 602 (e.g., host processor 214, accelerator logic 222, central processing unit (CPU), graphics processing unit (GPU), hardware processor core, or any combination thereof), main memory 604, static memory 606 (e.g., memory or storage for firmware, microcode, basic input output (BIOS), unified extensible firmware interface (UEFI), etc.), and mass storage device 608 or a stack of memory dies, hard drive, tape drive, flash storage device, or other block device), some or all of which may communicate with each other via interconnection link 630 (e.g., a bus). Machine 600 may further include display device 610, alphanumeric input device 612 (e.g., keyboard), and user interface (UI) navigation device 614 (e.g., mouse). In an example, the display device 610, the input device 612, and the UI navigation device 614 may be a touch screen display. The machine 600 may additionally include a mass storage device 608 (e.g., a drive unit), a signal generating device 618 (e.g., a speaker), a network interface device 620, and one or more sensors 616, such as a global positioning system (GPS) sensor, a compass, an accelerometer, or other sensors. The machine 600 may include an output controller 628 (e.g., a serial (e.g., a universal serial bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection) to communicate with or control one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0070] The hardware processor 602, main memory 604, static memory 606, or a register of a mass storage device 608 may be or include a machine-readable medium 622 on which is stored one or more sets of data structures or instructions 624 (e.g., software) embodying or used by any one or more techniques or functions described herein. The instructions 624 may also reside, completely or at least partially, within any of the registers of the hardware processor 602, main memory 604, static memory 606, or a mass storage device 608 during execution by the machine 600. In an example, one or any combination of the hardware processor 602, main memory 604, static memory 606, or a mass storage device 608 may constitute the machine-readable medium 622. Although the machine-readable medium 622 is illustrated as a single medium, the term "machine-readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) configured to store one or more instructions 624.

[0071] The term "machine-readable medium" may include any medium capable of storing, encoding, or carrying instructions for execution by the machine 600 and causing the machine 600 to perform any one or more of the techniques of the present disclosure, or capable of storing, encoding, or carrying data structures used by or associated with such instructions. Non-limiting machine-readable medium examples may include solid-state memory, optical media, magnetic media, and signals (e.g., radio frequency signals, other photon-based signals, sound signals, etc.). In an example, a non-transitory machine-readable medium includes a machine-readable medium with a plurality of particles having a constant (e.g., stationary) mass, and is therefore a composition of matter. Thus, a non-transitory machine-readable medium is a machine-readable medium that does not include a transitory propagating signal. Specific examples of non-transitory machine-readable media may include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable magnetic disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0072] In an example, the information stored or otherwise provided on the machine-readable medium 622 may represent instructions 624, such as the instructions 624 themselves or a format from which the instructions 624 may be derived. Such a format from which the instructions 624 may be derived may include source code, encoded instructions (e.g., in compressed or encrypted form), packaged instructions (e.g., divided into multiple packets), or the like. The information in the machine-readable medium 622 representing the instructions 624 may be processed by the processing circuitry into instructions to implement any of the operations discussed herein. For example, deriving the instructions 624 from the information (e.g., processed by the processing circuitry) may include: compiling (e.g., from source code, object code, etc.), interpreting, loading, organizing (e.g., dynamically or statically linking), encoding, decoding, encrypting, decrypting, packaging, unpacking the instructions 624, or otherwise manipulating the information into instructions 624.

[0073] In an example, the export of instructions 624 may include assembling, compiling, or interpreting (e.g., by processing circuitry) information to create instructions 624 from some intermediate or pre-processed format provided by machine-readable medium 622. Information, when provided in multiple parts, may be combined, unpacked, and modified to create instructions 624. For example, the information may be located on one or more remote servers in the form of multiple compressed source code packages (or object code, or binary executable code, etc.). The source code packages may be encrypted when transmitted over the network and decrypted, decompressed, assembled (e.g., linked) as necessary, and compiled or interpreted (e.g., into a library, a stand-alone executable, etc.) at a local machine and executed by the local machine.

[0074] The instructions 624 may be further transmitted or received through the communication network 626 using a transmission medium via the network interface device 620 using any of a number of transmission protocols (e.g., frame relay, Internet Protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Example communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), a mobile telephone network (e.g., a cellular network), a plain old telephone (POTS) network, and a wireless data network (e.g., a wireless network called The Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards, known as 6 family of standards), IEEE 802.15.4 family of standards, peer-to-peer (P2P) networks, and the like. In an example, network interface device 620 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas to connect to network 626. In an example, network interface device 620 may include multiple antennas to communicate wirelessly using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technology. The term "transmission media" should be interpreted to include any intangible media that can store, encode, or carry instructions for execution by machine 600, and includes digital or analog communication signals or other intangible media to facilitate communication of such software. Transmission media is machine-readable media.

[0075] To better illustrate the methods and apparatus described herein, which may be used, for example, to invoke and implement command isolation protocols to ensure the orderly execution of specific commands, a set of non-limiting example embodiments are set forth below in the form of numerically identified examples.

[0076] Example 1 is a method, comprising receiving a first packet at a command manager, the first packet comprising a first barrier instruction and a first command for a first queue, wherein the command manager is configured to implement a respective command execution policy for each of a plurality of queues according to the respective barrier instruction. In Example 1, in response to the first barrier instruction indicating barrier participation, the method comprises: incrementing a barrier counter and providing the first command to a command execution unit; and receiving a first response message from the command execution unit based on the first command at a response queue. Example 1 may further comprise decrementing the barrier counter in coordination with receiving the first response message.

[0077] In Example 2, the subject matter of Example 1 includes providing the first command to the command execution unit includes providing the first command to a processor in the command manager, to a data mover, or to a processor external to the command manager.

[0078] In example 3, the subject matter of examples 1-2 includes, in response to the first barrier instruction indicating barrier initiation: determining a counter value of the barrier counter; and in response to the counter value indicating that a different isolation operation is in progress, suspending the first command. In response to the counter value indicating that a new isolation operation can proceed, example 3 may include incrementing the barrier counter and providing the first command to the command execution unit.

[0079] In example 4, the subject matter of example 3 includes the barrier instruction comprising a message field including a first bit indicating barrier participation and a second bit indicating barrier initiation.

[0080] In Example 5, the subject matter described in Examples 1 to 4 includes, in response to the first fence instruction indicating a fence initiation: determining a counter value of the fence counter, the fence counter corresponding to the first queue; and in response to the counter value indicating that a different isolation operation is in progress, suspending the first command; and in response to the counter value indicating that a new isolation operation can continue, incrementing the fence counter corresponding to the first queue and providing the first command to a designated one of a plurality of available command execution units.

[0081] In Example 6, the subject matter of Examples 1-5 includes receiving the first packet at the command manager, including receiving the first packet at a memory-based queue established by a host application.

[0082] In Example 7, the subject matter of Examples 1-6 includes providing the first command to the command execution unit, including communicating the first command from a host device to an accelerator device using a compute express link (CXL) interconnect.

[0083] In Example 8, the subject matter of Examples 1-7 includes providing the first command to the command execution unit includes using an out-of-order interconnect that provides corresponding transaction identifiers (TIDs) to command and response message pairs, including a first TID of a first pair including the first command and the first response message.

[0084] In Example 9, the subject matter of Examples 1-8 includes receiving a second packet at the command manager, the second packet including a second barrier instruction for the first queue and a second command. In response to information in the second barrier instruction, Example 9 may include selectively providing the second command to the command execution unit or suspending the second command.

[0085] In Example 10, the subject matter of Example 9 includes providing the second command to the command execution unit when the second barrier instruction indicates that a barrier is not participated in and the second barrier instruction indicates that a barrier is not initiated, regardless of a status of a previously suspended command.

[0086] In Example 11, the subject matter of Examples 1-10 includes receiving a second packet at the command manager, the second packet including a second fence instruction for a second queue and a first command, and the command manager is configurable to maintain separate fence policies for transactions from the first queue and the second queue.

[0087] Example 12 is a system comprising: a memory device; and a command manager configured to implement a respective command execution policy for each of a plurality of queues according to a respective command barrier instruction. In Example 12, the command manager is further configured to: receive a first packet including a first barrier instruction and a first command for a first queue; and in response to the first barrier instruction indicating barrier participation, increment a barrier counter and provide the first command to a memory controller of the memory device. In Example 12, the command manager is configured to receive a first response message from the memory controller based on the first command and decrement the barrier counter accordingly.

[0088] In Example 13, the subject matter of Example 12 includes the memory device comprising the command manager.

[0089] In Example 14, the subject matter of Example 13 includes the command manager being coupled to the memory controller using a compute express link (CXL) interconnect.

[0090] In Example 15, the subject matter of Examples 12-14 includes the command manager being further configured to: in response to the first barrier instruction indicating a barrier initiation: determine a value of the barrier counter, the value indicating whether the memory controller is occupied by a previous command; and in response to the value indicating that the memory controller is occupied, suspend the first command until the value indicates that the memory controller is not occupied; and in response to the value indicating that the memory controller is not occupied, increment the barrier counter and provide the first command to the memory device.

[0091] Example 16 is at least one non-transitory machine-readable storage medium comprising instructions that, when executed by a processor circuit of a memory system, cause the processor circuit to perform operations comprising: retrieving command packets from a memory-based command queue, wherein each command packet comprises a respective instruction for a per-queue command column; and in response to a barrier initiation instruction in a first packet of the command packets: determining a counter value of a barrier counter, the counter value indicating whether an isolation operation is in progress; and in response to the counter value indicating that a new isolation operation may proceed, incrementing the counter value of the barrier counter and providing commands from the first packet to the command execution unit. In response to a barrier initiation instruction in a second packet of the command packets, example 16 may include determining that the counter value of the barrier counter indicates that a different isolation operation is in progress and withholding commands from the second packet from the command execution unit. In example 16, the first packet and the second packet may comprise portions of the same queue.

[0092] In Example 17, the subject matter of Example 16 includes instructions that, when executed by the processor circuit, cause the processor circuit to perform operations comprising: receiving, at a response queue, a first response message from the command execution unit based on the command from the first packet; and decrementing the counter value of the fence counter; and releasing one or more commands from the second packet to the command execution unit. In an example, the first response message is associated with the command from the first packet using a transaction identifier (TID).

[0093] In Example 18, the subject matter described in Examples 16 to 17 includes instructions that, when executed by the processor circuit, cause the processor circuit to perform operations including: in response to a fence engagement instruction in a third group of the command groups and while continuing to withhold the commands from the second group, incrementing the counter value of the fence counter and providing the commands from the third group to the command execution unit.

[0094] In Example 19, the subject matter described in Example 18 includes instructions that, when executed by the processor circuit, cause the processor circuit to perform operations including: decrementing the counter value of the fence counter when a response message is received from the command execution unit based on the command from the first group or the third group.

[0095] In Example 20, the subject matter described in Examples 16 to 19 includes instructions that, when executed by the processor circuit, cause the processor circuit to perform operations including: retrieving a third group of the command groups, the third group including fence instructions indicating no participation in a fence and no initiation of a new fence; and providing commands from the third group to the command execution unit.

[0096] Example 21 is at least one machine-readable medium comprising instructions that, when executed by processing circuitry, cause the processing circuitry to perform operations to implement any of Examples 1-15.

[0097] Example 22 is an apparatus including means for implementing any one of Examples 1-20.

[0098] Example 23 is a system for implementing any one of Examples 1 to 20.

[0099] Example 24 is a method for implementing any one of examples 12 to 15.

[0100] Each of these non-limiting examples may stand alone or may be combined with one or more of the other examples in various permutations or combinations.

[0101] The above detailed description includes reference to the accompanying drawings, which form a part of the detailed description. The accompanying drawings show specific embodiments in which the present invention can be practiced by way of illustration. These embodiments are also referred to as "examples" in this article. Such examples may include elements other than the elements shown or described. However, the inventors also consider examples in which only those elements shown or described are provided. In addition, for a specific example (or one or more aspects thereof), or for other examples (or one or more aspects thereof) shown or described herein, the inventors also consider examples using any combination or arrangement of those elements shown or described (or one or more aspects thereof).

[0102] In this document, as is common in patent literature, the term "a or an" is used to include one or more than one, regardless of any other examples or usages of "at least one" or "one or more". In this document, the term "or" is used to refer to a non-exclusive "or", such that "A or B" may include "A but not B", "B but not A", and "A and B", unless otherwise indicated. In the appended claims, the terms "including" and "in which" are used as the plain English equivalents of the corresponding terms "including" and "wherein". Moreover, in the appended claims, the terms "including" and "comprising" are open-ended, that is, systems, devices, articles, or processes that include elements other than those listed after the term in a claim are still considered to fall within the scope of that claim. In addition, in the appended claims, the terms "first", "second", and "third", etc. are used merely as labels and are not intended to impose numerical requirements on their objects.

[0103] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more aspects thereof) may be used in combination with each other. For example, a person of ordinary skill in the art may use other embodiments after reviewing the above description. It should be understood at the time of submission that the above description will not be used to interpret or limit the scope or meaning of the claims. Moreover, in the above detailed description, various features may be grouped together to simplify the disclosure. This should not be interpreted as meaning that the disclosed features that are not claimed are essential to any claim. On the contrary, the subject matter of the invention may be less than all the features of a specific disclosed embodiment. Therefore, the attached claims are thus incorporated into the detailed description, wherein each claim itself requires a separate embodiment, and it is considered that such embodiments can be combined with each other in various combinations or arrangements. The scope of the present invention should be determined with reference to the attached claims together with the full scope of equivalents granted to this claim.

Claims

1. A method comprising: receiving, at a command manager, a first packet including a first barrier instruction and a first command for a first queue, wherein the command manager is configured to implement a respective command execution policy for each of a plurality of queues according to the respective barrier instruction; In response to the first barrier instruction indicating barrier engagement, incrementing a barrier counter and providing the first command to a command execution unit; and receiving, at a response queue, a first response message from the command execution unit based on the first command; and The fence counter is decremented in coordination with receiving the first response message.

2. The method of claim 1, wherein providing the first command to the command execution unit comprises providing the first command to a processor in the command manager, to a data mover, or to a processor external to the command manager.

3. The method according to claim 1, comprising: In response to the first barrier instruction indicating the barrier is initiated: determining a counter value of the barrier counter; In response to the counter value indicating that a different isolation operation is in progress, suspending the first command; and In response to the counter value indicating that a new isolate operation can proceed, the barrier counter is incremented and the first command is provided to the command execution unit. 4 . The method of claim 3 , wherein the barrier instruction comprises a message field including a first bit indicating barrier participation and a second bit indicating barrier initiation.

5. The method according to claim 1, comprising: In response to the first barrier instruction indicating the barrier is initiated: determining a counter value of the barrier counter, the barrier counter corresponding to the first queue; In response to the counter value indicating that a different isolation operation is in progress, suspending the first command; and In response to the counter value indicating that a new isolate operation can proceed, the fence counter corresponding to the first queue is incremented and the first command is provided to a designated one of a plurality of available command execution units.

6. The method of claim 1, wherein receiving the first packet at the command manager comprises receiving the first packet at a memory-based queue established by a host application.

7. The method of claim 1, wherein providing the first command to the command execution unit comprises communicating the first command from a host device to an accelerator device using a compute express link (CXL) interconnect.

8. The method of claim 1 , wherein providing the first command to the command execution unit comprises using an out-of-order interconnect that provides respective transaction identifiers (TIDs) to pairs of command and response messages, including a first TID of a first pair comprising the first command and the first response message.

9. The method according to claim 1, comprising: receiving, at the command manager, a second packet including a second fence instruction and a second command for the first queue; and The second command is selectively provided to the command execution unit or the second command is suspended in response to information in the second fence instruction.

10. The method of claim 9, comprising providing the second command to the command execution unit when the second barrier instruction indicates that a barrier is not participated in and the second barrier instruction indicates that a barrier is not initiated, regardless of a status of a previously suspended command.

11. The method of claim 1 , comprising receiving, at the command manager, a second packet comprising a second fence instruction for a second queue and a first command, wherein the command manager is configured to maintain separate fence policies for transactions from the first queue and the second queue.

12. A system comprising: Memory device; and A command manager configured to implement a respective command execution policy for each of a plurality of queues according to a respective command fence instruction, wherein the command manager is further configured to: receiving a first packet including a first barrier instruction and a first command for a first queue; responsive to the first barrier instruction indicating barrier engagement, incrementing a barrier counter and providing the first command to a memory controller of the memory device; and A first response message is received from the memory controller based on the first command and the fence counter is decremented.

13. The system of claim 12, wherein the memory device comprises the command manager.

14. The system of claim 13, wherein the command manager is coupled to the memory controller using a compute express link (CXL) interconnect.

15. The system of claim 12, wherein the command manager is further configured to: In response to the first barrier instruction indicating the barrier is initiated: determining a value of the fence counter, the value indicating whether the memory controller is occupied by a previous command; In response to the value indicating that the memory controller is busy, suspending the first command until the value indicates that the memory controller is not busy; and In response to the value indicating that the memory controller is not busy, the fence counter is incremented and the first command is provided to the memory device.

16. At least one non-transitory machine-readable storage medium comprising instructions that, when executed by a processor circuit of a memory system, cause the processor circuit to perform operations comprising: retrieving command packets from a memory-based command queue, wherein each command packet includes a corresponding instruction for a per-queue command column; In response to a barrier initiation instruction in a first group of the command groups: determining a counter value of a barrier counter, the counter value indicating whether an isolation operation is in progress; and In response to the counter value indicating that a new isolation operation can proceed, incrementing the counter value of the fence counter and providing commands from the first group to the command execution unit; and In response to a fence initiation instruction in a second group of the command groups: determining that the counter value of the barrier counter indicates that a different isolation operation is in progress; and withholding commands from the second group from the command execution unit; The first group and the second group include parts of the same queue.

17. The non-transitory machine-readable storage medium of claim 16, wherein the instructions, when executed by the processor circuit, cause the processor circuit to perform operations comprising: receiving, at a response queue, a first response message from the command execution unit based on the command from the first packet; decrementing the counter value of the barrier counter; and One or more commands from the second group are released to the command execution unit.

18. The non-transitory machine-readable storage medium of claim 16, wherein the instructions, when executed by the processor circuit, cause the processor circuit to perform operations comprising: In response to a fence engage instruction in a third of the command packets and while continuing to withhold the commands from the second packet: incrementing the counter value of the barrier counter; and The commands from the third packet are provided to the command execution unit.

19. The non-transitory machine-readable storage medium of claim 18, wherein the instructions, when executed by the processor circuit, cause the processor circuit to perform operations comprising: The counter value of the fence counter is decremented when a response message is received from the command execution unit based on the command from the first packet or the third packet.

20. The non-transitory machine-readable storage medium of claim 16, wherein the instructions, when executed by the processor circuit, cause the processor circuit to perform operations comprising: Retrieving a third group of the command groups, the third group including a fence instruction indicating not participating in a fence and not initiating a new fence; and Commands from the third group are provided to the command execution unit.

Images