Systems, methods, and apparatus, including computer programs encoded on a computer storage medium, for queue management with a consistency interface

Abstract

A method may include: accessing a queue via a first device, wherein the queue may be accessible by a second device, and the first device may be connected to the second device via a consistency interface; and instructing the second device to access the queue via the consistency interface. Instruction may include instruction via a monitoring mechanism. Instruction may include generating a monitoring request. Instruction may include generating an alarm based on the monitoring request. The queue may include a submission queue. The queue may include a completion queue. Access may include reading an entry from the queue. Access may include writing an entry to the queue. The entry may include a command. The entry may include completion. The first device may include a host, and the second device may include a device. The queue may be located at the host.

Description

[0001] Cross-references to related applications

[0002] This application claims priority and benefit to U.S. Provisional Patent Application Serial No. 63 / 303,985, filed January 27, 2022, entitled “Systems, Methods, and Devices For Queue Handling For a Coherent Interface”, and U.S. Patent Application Serial No. 17 / 712,066, filed April 1, 2022, entitled “Systems, Methods, and Devices For Queue Management With a Coherent Interface”, which are incorporated herein by reference. Technical Field

[0003] This disclosure relates generally to queue management, and more specifically to systems, methods, and apparatus for queue management utilizing a coherent interface. Background Technology

[0004] The host and device can communicate using a protocol that can utilize one or more queues. The host can write entries, such as commands, to the commit queue. The host can also provide an indication to the device that an entry has been written to the commit queue. Based on this indication, the device can retrieve the entry from the commit queue and process it. The device can also write entries, such as completed entries, to the completion queue. The host can retrieve the entry from the completion queue and process it.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of the inventive principles, and therefore may contain information that does not constitute prior art. Summary of the Invention

[0006] A method may include: accessing a queue by a first device, wherein the queue is accessible by a second device, and the first device is connected to the second device via a consistency interface; and instructing the second device to access the queue via the consistency interface. Instruction may include instruction via a monitoring mechanism. Instruction may include generating a monitoring request. Instruction may include generating an alarm based on the monitoring request. The queue may include a submission queue. The queue may include a completion queue. Access may include reading an entry from the queue. Access may include writing an entry to the queue. The entry may include a command. The entry may include completion. The first device may include a host, and the second device may include a device. The queue may be located at the host. The queue may be located at the device. The queue may be associated with a communication protocol. The communication protocol may include a storage protocol. The storage protocol may include a Nonvolatile Memory Express (NVMe) protocol. The consistency interface may include a Compute Express Link (CXL) interface.

[0007] An apparatus may include a consistency interface and a controller, the controller being configured to communicate using a queue via the consistency interface and to receive indications of access to the queue through the consistency interface. The apparatus may include a cache, and the consistency interface may be configured to maintain the consistency of at least a portion of the cache. The queue may be at least partially located in the cache. The apparatus may include shared local memory, and the consistency interface may be configured to maintain the consistency of at least a portion of the shared local memory. The queue may be at least partially located in at least a portion of the shared local memory. The consistency interface may be configured to receive monitoring requests based on access. The consistency interface may be configured to generate alarms based on monitoring requests. The controller may be configured to receive alarms as indications from the consistency interface. The controller may be configured to process entries in the queue based on the indications.

[0008] A system may include a host and a device coupled to the host using a consistency interface. The device may include a controller configured to communicate with the host using a queue via the consistency interface and to receive, through the consistency interface, an indication from the host to access the queue. The device may include shared local memory, the consistency interface may be configured to maintain consistency of at least a portion of the shared local memory, and the queue may be located at least partially within at least a portion of the shared local memory. The consistency interface may be a first consistency interface, the queue may be a first queue, the indication may be a first indication, the access may be a first access, and the host may further include a second consistency interface, wherein the host may be configured to communicate with the device using a second queue via the second consistency interface and to receive, through the second consistency interface, a second indication from the device to access the second queue. The device may include shared local memory, and the first and second consistency interfaces may be configured to maintain consistency of at least a portion of the shared local memory. The second queue may be located at least partially within at least a portion of the shared local memory. Attached Figure Description

[0009] The accompanying drawings are not necessarily drawn to scale, and in all drawings, elements with similar structures or functions are often indicated by the same reference numerals or portions thereof for illustrative purposes. The drawings are merely intended to facilitate the description of the various embodiments described herein. The drawings do not depict every aspect of the teachings disclosed herein and do not limit the scope of the claims. To prevent obscurity, not all components, connections, etc., are shown, and not all components have reference numerals. However, this can be readily apparent from the pattern of component configuration in the drawings. The accompanying drawings, together with the specification, illustrate exemplary embodiments of this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0010] Figure 1 An embodiment of a queuing scheme for a communication protocol according to an example embodiment of this disclosure is shown.

[0011] Figure 2 An embodiment of a scheme for communication using one or more queues and a consistency interface, according to an example embodiment of this disclosure, is shown.

[0012] Figure 3 An example embodiment of a system that communicates using one or more queues and a consistency interface, according to an example embodiment of this disclosure, is shown.

[0013] Figure 4 An example embodiment of a method for communicating using one or more queues and a consistency interface, according to an example embodiment of this disclosure, is shown.

[0014] Figure 5An example embodiment of a host device according to an example embodiment of this disclosure is shown.

[0015] Figure 6 An example embodiment of a device according to an example embodiment of this disclosure is shown.

[0016] Figure 7 An embodiment of a communication method according to an example embodiment of this disclosure is shown. Detailed Implementation

[0017] Communication protocols can use one or more queues to exchange commands and / or completions between components. For example, to read data from a device, a host can place a read command in a commit queue accessible to the device. The host can also send an indication to the device to notify it that the command exists in the commit queue. The device can retrieve the read command from the commit queue and process it (e.g., by sending the requested data to the host). The device can also place a completion in a completion queue to notify the host that the read command has been completed.

[0018] In some respects, communication protocols can use indication mechanisms to indicate that an entry has been added to a queue. For example, in addition to placing a command in a submission queue, the host can write an entry to a register (which might be called a doorbell register). A write to the register can generate an interrupt that notifies the device that a command has been placed in the submission queue. However, this indication mechanism can consume host and / or device resources, for example, due to writing the entry to the register (which may be located at the device), and / or service interruptions.

[0019] The communication scheme according to the exemplary embodiments of this disclosure can develop a consistency mechanism for a consistency interface to perform one or more aspects of an indication function for queue access. For example, a memory consistency interface may include a consistency mechanism that can notify one or more devices of changes in a coherent memory location. By implementing a queue with a coherent memory location, the consistency mechanism can notify one or more devices (e.g., a host, and / or a device, etc.) that the queue has been accessed (e.g., written to and / or read from). Depending on the implementation details, this can reduce or eliminate the amount of resources required to indicate that a queue has been accessed. For example, depending on the implementation details, this can reduce or eliminate the use of doorbell registers, one or more processing operations, and / or one or more interrupts, etc.

[0020] In some embodiments, the consistency features of the consistency interface can be used to implement a queue indication mechanism. For example, the consistency interface can implement a cache consistency protocol, which can be used, for example, to provide a device with access to a portion of host memory. A copy of this portion of main memory can be stored in a cache memory at the device, and the cache consistency mechanism of the consistency interface can maintain consistency between the cache memory at the device and this portion of main memory. In some embodiments, the consistency mechanism can use a monitoring mechanism (e.g., a snooping mechanism) to determine that a memory location has been updated (e.g., at the host or in the corresponding cache at the device). In such embodiments, if the device cache memory is used to implement the queue, the monitoring mechanism can be used to indicate to the host and / or device that the queue has been accessed.

[0021] As another example, the consistency interface can implement a memory consistency protocol, which can be used, for example, to provide a host with shared access to local memory at a device. The memory consistency mechanism of the consistency interface can maintain the consistency of shared device memory, for example, so that if a shared memory location is updated by one of the hosts or devices, the other host or device can be notified to prevent the use of expired data. In such an embodiment, if shared device memory is used to implement queues, the memory consistency mechanism of the consistency interface can be used to indicate to the hosts and / or devices that a queue has been accessed.

[0022] Alternatively or additionally, the cache coherency mechanism and memory coherency mechanism of the coherency interface can be used in combination to implement a queue indication mechanism. For example, the memory coherency mechanism of the coherency interface can be used to implement queues in a shared memory location of the devices. The cache coherency mechanism of the coherency interface can be used (e.g., by eavesdropping) to monitor queues in a shared memory location of the devices to determine that a queue has been updated by the host and to send an indication (e.g., an alarm) to the devices.

[0023] Figure 1 An embodiment of a queuing scheme for a communication protocol according to an example embodiment of this disclosure is shown. For example, Figure 1 The illustrated embodiments can be used with storage protocols such as Non-Volatile Fast Memory (NVMe), which can use interconnects such as Peripheral Component Interconnect Express (PCIe) and / or CXL non-consistent input and / or output (I / O or IO) protocols, which may be referred to as CXL.IO. As another example, Figure 1The embodiments shown can be used with mesh-based NVMe (NVMe-over-Fabric, NVMeoF) protocols that can use networks such as Ethernet, but the principles are not limited to these implementation details or any other implementation details. In some embodiments that support CXL implementations, the CXL.IO protocol may substantially correspond to PCIe, while the CXL.mem and / or CXL.cache protocols may implement one or more additional features that provide consistency.

[0024] Figure 1 The illustrated embodiment may include a host 102 and a device 104. The host 102 may include a Submission Queue (SQ) 106 and a Completion Queue (CQ) 108 located, for example, in host memory 110. In other embodiments, the Submission Queue 106 and / or the Completion Queue 108 may be located at the device 104 and / or at any other location. The device 104 may include a controller 112, which may include a Submission Queue tail bell register 114 and / or a Completion Queue head bell register 116.

[0025] In some embodiments, submit queue 106 and / or complete queue 108 can be used, for example, to enable device 104 to receive and / or process one or more commands from host 102. Submit queue 106 and / or complete queue 108 can be implemented as, for example, a circular first-in-first-out (FIFO) queue, wherein one end of the queue can logically wrap around to the other end of the queue so that entries can be added to and removed from the queue indefinitely (limited to a maximum number of entries in some embodiments), even if the queue can be implemented with a finite linear address space. Referring to submit queue 106 or complete queue 108, the slot with the oldest unretrieved entry (e.g., command or complete) can be referred to as the head, and the next available unoccupied slot can be referred to as the tail. The head of submit queue 106 can be pointed to by a head entry pointer 103 (also referred to as the head pointer), and the tail of submit queue 106 can be pointed to by a tail entry pointer 105 (also referred to as the tail pointer). The head of the completion queue 108 can be pointed to by the head entry pointer 107, and the tail of the completion queue 108 can be pointed to by the tail entry pointer 109. When the queue is empty, the head entry pointer and the tail entry pointer can point to the same position.

[0026] An example embodiment of a method for receiving and / or processing one or more commands from host 102 may be as follows.

[0027] In operation (1), host 102 may place one or more commands (e.g., writes) in commit queue 106 starting at an empty slot pointed to by tail entry pointer 105 (e.g., one command per slot). Tail entry pointer 105 may then be updated to point to the next available slot in commit queue 106.

[0028] In operation (2), host 102 may also update (e.g., write) the Submission Queue Tail Doorbell Register (SQ-TDB) 114 to initiate a process that can notify controller 112 that one or more new commands have been placed in the submission queue 106. For example, host 102 may write a new value of the submission queue tail entry pointer 105 to the submission queue tail doorbell register 114.

[0029] In some embodiments, the submit queue tail bell register 114 can be monitored using hardware, software, or a combination thereof to provide the controller 112 with an indication that one or more new commands have been placed in the submit queue 106. For example, in some embodiments, the submit queue tail bell register 114 can be implemented as a hardware-monitored register or memory location (e.g., a location such as a PCIe location at the controller 112 and / or the device 104) that can generate an interrupt for the device 104 based on an update to register 114. In some embodiments, the interrupt can serve as an indication that one or more new commands have been placed in the submit queue 106 at the controller 112 and / or the device 104.

[0030] In some embodiments, receiving an indication that one or more new commands have been placed in the submission queue 106 enables the controller 112 to track the number of unfetched and / or unprocessed commands that may exist in the submission queue 106. In some embodiments, this information may be used in conjunction with, for example, a command arbitration process that enables the controller 112 to determine from which submission queue (if there are multiple submission queues) the controller 112 may fetch one or more commands.

[0031] In operation (3), controller 112 may retrieve one or more commands from submission queue 106 (e.g., by reading) starting from the position pointed to by header entry pointer 103. For example, header entry pointer 103 may be implemented using a memory location accessible to device 104 (e.g., a PCIe memory location). In some embodiments, header entry pointer 103 may be established as part of the initialization process of submission queue 106. Header entry pointer 103 may then be updated to point to the next (e.g., oldest) unretrieved command in submission queue 106.

[0032] exist Figure 1 In the illustrated embodiment, the submission queue 106 may be a circular queue located at the host 102. Therefore, it may be advantageous for the host to track the head position of the submission queue 106, for example, to wrap the submission queue 106, thereby preventing collisions between the head and tail of the submission queue 106. In some embodiments, the head entry pointer 103 may be accessible to the host 102, so the host 102 can read the head entry pointer 103 to track the head position of the submission queue 106. However, in some other embodiments, the host 102 may not have direct access to the head entry pointer 103 of the submission queue 106. Therefore, Figure 1 The illustrated scheme can implement a mechanism that enables host 102 to track the head position of submit queue 106. For example, in some embodiments, such a mechanism can be implemented essentially, for example, using a PCIe base address register, as a submit queue head bell register. As another example, in some embodiments, such a mechanism can be implemented by device 104 sending the current value of head entry pointer 103 to host 102 according to the entries in completion queue 108 (e.g., each entry) as described below.

[0033] In operation (4), controller 112 may process one or more commands that it has already extracted from submission queue 106. In some embodiments, controller 112 may process one or more commands out of order. In some embodiments, extraction and / or processing may be referred to as consumption.

[0034] In operation (5), controller 112 may place one or more completions corresponding to one or more processed commands in completion queue 108, starting from, for example, the next available slot that may be pointed to by completion queue tail entry pointer 109. Completion queue tail entry pointer 109 may be updated to point to the next available slot in completion queue 108. In some embodiments, a completion may include a stage label that can be reversed from a previous entry, for example, to indicate to host 102 that a completion queue entry (e.g., a new completion) is a new entry available for processing.

[0035] During operation (6), controller 112 may generate interrupts (e.g., pin-based interrupts, message signaled interrupts (MSI), and / or extended MSI interrupts (MSI-X), etc.) to indicate to host 102 that one or more completions have been added to completion queue 108. In some embodiments, one or more interrupts for one or more completions may be combined into a smaller number of interrupts.

[0036] In operation (7), host 102 may retrieve one or more completions from completion queue 108 (e.g., by reading), starting from, for example, the head that may be pointed to by completion queue head entry pointer 107. In some embodiments, host 102 may process one or more of the completions it has retrieved from completion queue 108. In some embodiments, host 102 may continue to retrieve and / or process completions, for example, until it encounters a completion with a stage label that is reversed from the stage label of a previously retrieved completion.

[0037] In operation (8), host 102 may update (e.g., write) the Completion Queue Head Bell Register (CQ-HDB) 116 to initiate a process that may notify controller 112 (e.g., by reading one or more completions from completion queue 108) that one or more completions have been removed (e.g., released) from completion queue 108. For example, host 102 may write a new value to the Completion Queue Head Entry Pointer 107 to the Completion Queue Head Bell Register 116. In some embodiments, host 102 may retrieve and / or process one or more completions before updating the associated Completion Queue Head Bell Register 116.

[0038] against Figure 1 The described methods and / or apparatus enable communication protocols using one or more queues to coordinate the activities of host 102 and device 104 (e.g., maintaining a lock-step). However, depending on the implementation details, the use of schemes involving one or more doorbell registers may consume host and / or device resources, for example, for host 102 to write one or more entries to one or more registers 114 and / or 116 (which may be located at device 104), and / or servicing one or more interrupts, etc.

[0039] For illustrative purposes, some embodiments may be described in the context of specific implementation details. However, the principles are not limited to these or any other implementation details.

[0040] Figure 2 An embodiment of a scheme for communicating using one or more queues and a consistency interface, according to an example embodiment of this disclosure, is shown. Figure 2The illustrated embodiments may include a first device 222 and a second device 224. The first device 222 may include a first consistency interface 226. The second device 224 may include a second consistency interface 228 and a controller 230. The first consistency interface 226 and the second consistency interface 228 may communicate via a communication connection 232. In some embodiments, the first device 222 and / or the second device 224 may be configured as a host, client, and / or device, or any combination thereof. In some embodiments, the controller 230 may be a host controller, client controller, and / or device controller, etc. In some embodiments, in addition to coherent messages, the first consistency interface 226 and / or the second consistency interface 228 may also transmit non-coherent messages. For example, in embodiments implementing support for CXL, the CXL.IO protocol may be used to transmit non-coherent messages. Alternatively or additionally, some embodiments may, for example, use one or more interfaces other than the first consistency interface 226 and / or the second consistency interface 228 (e.g., one or more interfaces with non-coherent message transmission capabilities) to transmit non-coherent messages.

[0041] The controller 230 can be used by the second device 224 to communicate with the first device 222 via a second consistency interface 228 using a communication protocol. This communication protocol can use one or more queues 234 to exchange requests, commands, and / or completions between the first device 222 and the second device 224. In some embodiments, one or more of the one or more queues 234 can be used, for example, as described above. Figure 1 The described circular first-in-first-out (FIFO) arrangement is implemented. In some embodiments, one or more of the one or more queues 234 can be implemented using any other type of queue arrangement. For example, in some embodiments, the first device 222 can be implemented as a host, the second device 224 can be implemented as a device, and the queue 234 can be implemented as a memory location into which the host can write entries commanding the device to perform one or more operations. The device can (e.g., via indication 236) detect writes made by the host and perform one or more operations commanded by the entries. The device can write back to the memory location to indicate to the host that at least a portion of one or more operations has been completed. In some embodiments, the memory location can be implemented in an atomic memory access environment, which can maintain synchronization, for example, between a first central processing unit (CPU) and a second CPU, between a CPU and a graphics processing unit (GPU), and / or between a CPU and an accelerator. In some embodiments, and depending on the implementation details, such a memory location can or can not be implemented using models such as FIFO, SQ, and / or CQ.

[0042] One or more queues 234 may be located at the second device 224, such as Figure 2 The dashed outline is shown in the diagram. Additionally or alternatively, one or more queues 234 may be located at the first device 222, may be partially located at the first device 222 and partially located at the second device 224, or may be located at any other one or more locations. In some embodiments, for example, if the queues 234 are implemented using a coherent cache at the second device 224, which is configured to cache one or more locations of memory at the first device 222, then one or more of the one or more queues 234 may be substantially located at both the first device 222 and the second device 224.

[0043] The second consistency interface 228 can provide the controller 230 with an indication 236 of the first device 222 accessing one or more of the one or more queues 234. In some embodiments, accessing a queue may refer to an entry in the queue, and / or a pointer to a location (e.g., an entry) in the queue. For example, in some embodiments, the indication 236 may indicate access (e.g., read and / or write) to one or more memory locations, which may be at least partially contained in one or more of the one or more queues 234, and / or point to a location in one or more of the one or more queues 234. In some embodiments, and depending on the implementation details, Figure 2 The illustrated embodiment can utilize the consistency mechanisms of the first consistency interface 226 and / or the second consistency interface 228 to provide indication 236. For example, the first consistency interface 226 and / or the second consistency interface 228 can implement a consistency mechanism that notifies the first device 222 and / or the second device 224 of changes in one or more cache locations and / or shared consistent memory locations at the first device 222 and / or the second device 224. By implementing one or more queues 234 and / or one or more pointers to one or more locations in the queues using consistent caches and / or consistent memory locations, the consistency mechanism can provide indication 236 that one or more queues 234 and / or one or more pointers to one or more locations in the queues have been accessed by the first device 222. Depending on the implementation details, this can reduce or eliminate the amount of resources required to provide indication 236. For example, depending on the implementation details, this can reduce or eliminate the use of doorbell registers, one or more processing operations, and / or one or more interrupts, etc.

[0044] The first device 222 and / or the second device 224 may be implemented using any type of device, including servers (such as computing servers, storage servers, and / or network servers), computers (such as workstations, personal computers, tablets, and / or smartphones), devices (such as accelerator devices, storage devices, network devices, memory expansion devices and / or memory buffer devices, GPUs, neural processing units (NPUs), and / or tensor processing units (TPUs), or any combination thereof.

[0045] The first consistency interface 226 and / or the second consistency interface 228 can be implemented using any type of consistency interface (such as Gen-Z, Open Coherent Accelerator Processor Interface (OpenCAPI), and / or Cache Coherent Interconnect for Accelerators (CCIX)) and / or consistency protocols such as Compute High Speed ​​Link (CXL) and / or consistency protocols such as CXL.mem, CXL.cache, and / or CXL.IO, or any combination thereof.

[0046] The controller 230 can implement any type of communication protocol that can use one or more queues, including, for example, NVMe, NVMe-oF, PCIe, CXL.IO, Serial ATA (SATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), Direct Memory Access (DMA), Remote DMA (RDMA), RDMA over Converged Ethernet (ROCE), and / or Advanced Message Queuing Protocol (AMQP), and / or any combination thereof.

[0047] The communication connection 232 can be implemented using any type of wired and / or wireless communication medium, interface, and / or protocol, including PCIe, CXL, Advanced Extensible Interface (AXI), Ethernet, Transmission Control Protocol / Internet Protocol (TCP / IP), Fibre Channel, InfiniBand, SATA, SCSI, SAS, iWARP, any generation of wireless networks (including 2G, 3G, 4G, and / or 5G), any generation of Wi-Fi, Bluetooth, and / or Near Field Communication (NFC), or any combination thereof. In some embodiments, the communication connection 232 may include a communication mesh structure, which includes one or more links, buses, switches, hubs, nodes, routers, converters, and / or repeaters. In some embodiments, Figure 2The illustrated scheme may include one or more additional devices having one or more additional consistency interfaces connected using communication connection 232 and / or its extensions. In some embodiments, communication connection 232 may include a switch having memory accessible to one or more hosts, and / or devices, etc. In some embodiments, communication connection 232 may include one or more additional devices that may be configured as peers (e.g., peer-to-peer), and in some embodiments, the additional devices may include one or more memories accessible to other devices.

[0048] In embodiments where one or more of the first device 222 and / or the second device 224 are implemented as storage devices, the storage device may include any type of non-volatile storage medium, such as solid-state based non-volatile storage media (e.g., solid-state drives (SSDs)), magnetic-based non-volatile storage media (e.g., hard disk drives (HDDs)), and / or optical-based non-volatile storage media, or any combination thereof. For example, in some embodiments, the storage device may be implemented as an SSD based on NAND flash memory, permanent memory (such as cross-grid non-volatile memory), memory with varying bulk resistance, and / or phase-change memory (PCM), or any combination thereof. Any such storage device may be implemented using any connector configuration (such as SATA, SCSI, SAS, U.2, and / or M.2, etc.) in any form factor (such as 3.5-inch, 2.5-inch, 1.8-inch, M.2, enterprise and data center SSD form factor (EDSFF), and / or NF1, etc.). Any such storage device may be implemented, or used in combination with, server chassis, server racks, data rooms, data centers, edge data centers, mobile edge data centers and / or any combination thereof, in whole or in part.

[0049] Figure 3 An example embodiment of a system for communicating using one or more queues and a consistency interface, according to an example embodiment of this disclosure, is shown. For example, Figure 3 The illustrated embodiments can be used for implementation. Figure 2 The scheme shown. Figure 3 The illustrated embodiment may include a host 322 and a device 324 that can communicate using communication connection 332.

[0050] For example, host 322 can be implemented using a server or any other device. Host 322 may include a consistency interface 326 and / or protocol logic 342. Host 322 can use the consistency interface 326 to communicate with device 324 via communication connection 332.

[0051] Consistency interface 326 may include consistency engine 344 and / or snooping logic 346 (also referred to as monitoring logic). In some embodiments, snooping logic 346 may be able to detect queue accesses, such as one or more modifications (e.g., writes) and / or one or more reads of one or more data memory locations. Consistency interface 326 may be implemented, for example, with CXL and / or one or more CXL protocols or any other consistency interface, and / or consistency protocol. For example, in an embodiment where consistency interface 326 is implemented with CXL, consistency engine 344 may be implemented with a CXL consistency engine (also referred to as consistency logic). In some embodiments, snooping logic 346 may be separate from and / or partially integrated with consistency engine 344.

[0052] For example, protocol logic 342 may implement a storage protocol such as NVMe or any other communication protocol that may use one or more queues. For example, in an embodiment where protocol logic 342 implements the NVMe protocol, protocol logic 342 may configure submit queue 334 and / or completion queue 335 as described below. In some embodiments, protocol logic 342 and / or protocol controller 330 may implement any other type of communication protocol that may involve the use of one or more queues. For example, a host may use one or more queues to assign one or more work lists to one or more graphics processing units (GPUs), field-programmable gate arrays (FPGAs), and / or accelerators, etc. As another example, a host may use one or more queues to provide one or more data transfer lists to one or more networking devices, and / or communication devices, etc.

[0053] Device 324 may include a coherence interface 328, device local memory 350, protocol controller 330, and / or device functional circuitry 352. Device 324 may be implemented using any type of device, including, for example, accelerator devices, storage devices, network devices, memory expansion devices and / or memory buffer devices, GPUs, neural processing units (NPUs), tensor processing units (TPUs), and / or FPGAs, or any combination thereof (such as compute storage devices).

[0054] Device functional circuitry 352 may include any means that enable device 324 to perform its intended functions. For example, if device 324 is implemented as an accelerator, device functional circuitry 352 may include one or more computing resources, such as one or more processors (e.g., CPUs, such as Complex Instruction Set Computer (CISC) processors (e.g., x86 processors) and / or Reduced Instruction Set Computer (RISC) processors (e.g., RISC-V and / or ARM processors)), application-specific circuitry (ASICs), and / or FPGAs, or any combination thereof. As another example, if device 324 is implemented as a storage device, device functional circuitry 352 may include one or more storage media, a storage device controller, and / or a media conversion layer (e.g., a flash conversion layer (FTL)), or any combination thereof.

[0055] The consistency interface 328 may include a consistency engine 358 and / or snooping logic 360 (also referred to as monitoring logic). The consistency interface 328 may be implemented, for example, using CXL and / or one or more CXL protocols or any other consistency interface, and / or consistency protocol. For example, in an embodiment where the consistency interface 328 is implemented using CXL, the consistency engine 358 may be implemented using a CXL consistency engine (also referred to as consistency logic). In some embodiments, the snooping logic 360 may be separate from and / or partially integrated with the consistency engine 358.

[0056] In some embodiments, all or a portion of the device local memory 350 may be consistently maintained by a memory consistency protocol. For example, in an embodiment where the consistency interface 328 is implemented using CXL, the consistency interface 328 may use the CXL.mem protocol to maintain consistency between the device local memory 350 and the device 324 and / or the host 322.

[0057] Alternatively or additionally, in some embodiments, all or a portion of the device local memory 350 may be consistently maintained by a cache coherence protocol. For example, in an embodiment where the coherence interface 328 is implemented using CXL, the coherence interface 328 may use the CXL.cache protocol to maintain all or a portion of the device local memory 350 consistent with the device 324 and / or the host 322.

[0058] Alternatively or additionally, in some embodiments, the cache coherency mechanism and memory coherency mechanism of the coherency interface may be used in combination to consistently maintain all or a portion of the device local memory 350. For example, in an embodiment where the coherency interface 328 is implemented using CXL, the coherency interface 328 may use the CXL.mem protocol to configure all or a portion of the device local memory 350 as a coherent memory, and use the CXL.cache protocol to implement an indication mechanism for queues that may reside in the coherent portion of the device local memory 350.

[0059] In some embodiments, the device local memory 350 can be configured with any number of queues. For example, in Figure 3 In the illustrated embodiment, the device local memory 350 may be configured with a submit queue (SQ) 334 and / or a completion queue (CQ) 335.

[0060] For example, protocol controller 330 may implement a storage protocol such as NVMe or any other communication protocol that may use one or more queues. For example, in an embodiment where protocol controller 330 implements NVMe, submit queue 334 and / or completion queue 335 may be configured as an NVMe submit queue and / or an NVMe completion queue, respectively.

[0061] In some embodiments, the consistency interface 328 may provide the protocol controller 330 with an indication 336 that the host 322 accesses the commit queue 334 and / or the completion queue 335. For example... Figure 3 The illustrated embodiment may utilize the consistency engine 358 and / or the snooping logic 360 to provide the protocol controller 330 with indications of changes to the submission queue 334 and / or the completion queue 335, as described in more detail below.

[0062] In some embodiments, the consistency interface 326 may provide the protocol logic 342 at the host 322 with an indication 337 of the device 324's access to the submission queue 334 and / or the completion queue 335. For example, Figure 3 The illustrated embodiment may utilize the consistency engine 344 and / or the snooping logic 346 to provide the protocol logic 342 with indications of changes to the submission queue 334 and / or the completion queue 335, as described in more detail below.

[0063] In some embodiments, access to the submit queue 334 and / or the complete queue 335 may involve entries (e.g., commands) that may involve data transfer, such as reading data and / or writing data. In some embodiments, such data may reside at least partially in device local memory 350 (at least a portion of which may be consistently maintained as shared memory), in other device memory, in system memory (e.g., host memory at host 322), or at any other location.

[0064] Figure 4 An example embodiment of a method for communicating using one or more queues and a consistency interface, according to an example embodiment of this disclosure, is shown. Figure 4 The illustrated embodiments can be used with Figure 3 Use the system shown.

[0065] For illustrative purposes, it can be compared with Figure 4 In combination and within the context of a system in which consistency interfaces 326 and 328 can be implemented using CXL and protocol logic 342 and protocol controller 330 can be implemented using NVMe. Figure 3 The example embodiments shown are shown. However, the principles are not limited to these or any other implementation details.

[0066] refer to Figure 3 and Figure 4 In operation 462, one or both of the consistency engines 344 and / or 358 may (e.g., using the CXL.mem protocol) configure at least a portion of the device local memory 350 at device 324 as shared coherent memory for access by host 322. In some embodiments, one or both of the consistency engines 344 and / or 358 may, for example, use the CXL.cache protocol to configure some or all of the shared memory to be monitored, thereby providing indications to host 322 and / or device 324 when commit queue 334 and / or completion queue 335 are updated. Thus, in some embodiments, the CXL.mem protocol may be used to configure commit queue 334 and / or completion queue 335 in a shared portion of the device local memory 350 of device 324, and the consistency mechanisms of the CXL.cache protocol (e.g., consistency rules) may be applied to some or all of the memory locations where commit queue 334 and / or completion queue 335 can be configured.

[0067] In some embodiments implemented with CXL, consistency engines 344 and / or 358 may, for example, use CXL.cache host-to-device (H2D) and device-to-host (D2H) request, response, and data channels to implement hardware-based consistency management to maintain consistency between host 322 and device 324. In other embodiments, cache consistency (e.g., cache consistency) may be maintained using hardware, software, or any combination thereof.

[0068] In operation 464, submit queue 334 and / or completion queue 335 can be configured to reside at least partially in a shared portion of device local memory 350. For example, in embodiments where protocol logic 342 and / or protocol controller 330 are implemented using NVMe, submit queue 334 and / or completion queue 335 can be configured as NVMe submit queues and / or NVMe completion queues, respectively. In some embodiments, this arrangement allows host 322 to consistently access submit queue 334 and / or completion queue 335 as if it were part of host memory, but the consistency mechanism of the CXL.cache protocol is configured to provide snooping, thereby providing an alert (e.g., as indication 336) to protocol controller 330 when host 322 updates one or more entries in submit queue 334 and / or completion queue 335, as described below. In some embodiments, submit queue 334 and / or completion queue 335 can be configured directly using CXL.cache.

[0069] In operation 466, snooping logic 346 and / or 360 may be configured to monitor some or all of the commit queue 334 and / or completion queue 335, for example, in a shared portion of the device's local memory 350. For example, in some embodiments, snooping logic 360 may be configured to monitor commit queue 334 and provide an alert (e.g., as indication 336) to protocol controller 330 when host 322 updates (e.g., writes) one or more entries in commit queue 334. In some embodiments, an update by host 322 to one or more entries in commit queue 334 and / or completion queue 335 may generate a snooping request from host 322 to device 324. In some embodiments, consistency engine 358 and / or snooping logic 360 may be configured to generate an alert to protocol controller 330 based on the snooping request.

[0070] In operation 468, the listening logic 360 can determine that the host 322 has updated one or more entries in the submission queue 334 and / or completion queue 335, and can send an alert to the protocol controller 330, for example, as an instruction 336.

[0071] In operation 470, protocol controller 330 can process updates to submission queue 334 and / or completion queue 335 based on received alarms. The method can then return to operation 468 to wait for listening logic 360 to detect another update to submission queue 334 and / or completion queue 335.

[0072] Figure 4 The illustrated embodiment can be described in the context of protocol controller 330 receiving an alarm (e.g., as indication 336) from consistency interface 328 based on host 322's access to the queue. However, Figure 4 The method shown can be configured to operate in a similar manner to configure one or more snoops on the submission queue 334 and / or the completion queue 335 to provide an alert (e.g., as an indication 337) from the consistency interface 326 to the protocol logic 342 of the host 322 based on the device 324's access to the submission queue 334 and / or the completion queue 335.

[0073] Depending on the implementation details, refer to Figure 4 and / or Figure 3 The described system and / or method can be used to implement and Figure 1 A similar queue-based communication protocol is shown, but in which doorbell registers 114 and / or 116 and / or the accompanying interrupts and / or other resources are partially or completely replaced by the consistency mechanism of consistency interfaces 326 and / or 328. Therefore, depending on the implementation details, regarding... Figure 4 and / or Figure 3 The described system and / or method can utilize the consistency mechanisms of consistency interfaces 326 and / or 328 to reduce [the impact of] [the lack of] consistency mechanisms. Figure 1 The doorbell architecture shown is associated with time, power consumption, operation, memory space, and / or overhead.

[0074] For example, refer to Figure 1 In some embodiments using the consistency mechanism according to this disclosure, operations (1) and / or (2) may be at least partially replaced by operations in which a host can write commit queue entries to a commit queue, which may be located in, for example... Figure 3 The consistency mechanism is located in the device cache and / or device memory location, and can (e.g., using snooping and / or alarms) provide the device with an indication that the commit queue has been updated.

[0075] As another example, see Figure 1 In some embodiments using the consistency mechanism according to this disclosure, operations (5) and / or (6) may be at least partially replaced by operations in which the device is writable to a completion queue, which may be located, for example... Figure 3The consistency mechanism is located in the device cache and / or device memory location, and can (e.g., using snooping and / or alarms) provide the host with an indication that the completion queue has been updated.

[0076] As another example, see Figure 1 In some embodiments using the consistency mechanism according to this disclosure, any access (e.g., update) by a first device (e.g., host 102 and / or device 104) to any of pointers 103, 105, 107, and / or 109 can be directed to another device using the consistency mechanism, for example, as... Figure 2 , Figure 3 and / or Figure 4 As shown.

[0077] Refer again Figure 3 In some embodiments, the consistency mechanism of consistency interfaces 326 and / or 328 can provide indication 336 to enable protocol controller 330 to determine that a queue has been updated when or near the time when the queue is updated. In some embodiments, the consistency mechanism of consistency interfaces 326 and / or 328 can maintain the order of updates for one or more queues for protocol controller 330.

[0078] In some embodiments, when the protocol controller 330 retrieves an entry from the queue, the consistency mechanism of the consistency interfaces 326 and / or 328 can provide an alert to the host 322, for example, to enable the host 322 to track the head of the queue (e.g., to reduce or prevent queue overrun). Alternatively or additionally, in some embodiments, Figure 3 The system shown can be implemented in accordance with the above. Figure 1 The similar submission queue head bell mechanism notifies host 322 when protocol controller 330 should retrieve an entry from the queue.

[0079] In some embodiments, memory space for one or more queues can be configured at the device, and a consensus protocol can be configured to monitor (e.g., eavesdrop) some or all of the memory space to provide indication to the device based on host updates of one or more entries in one or more queues using the consensus mechanism of the consensus protocol.

[0080] In some embodiments, a communication protocol (e.g., NVMe) can be configured to reduce or prevent partial updates to a queue. In some embodiments, a communication protocol (e.g., NVMe) can be configured to implement atomic updates. For example, in some embodiments, one or more entries in a queue can be updated simultaneously. Depending on the implementation details, this can reduce or prevent potential errors, for example, if a partial update to a queue triggers snooping on consistency mechanisms and / or related alerts, and the protocol controller reads the partial update before one or more entries are fully updated.

[0081] In some embodiments, a communication protocol (e.g., NVMe) may utilize wraparound to implement (e.g., define) queue sizes (e.g., commit queues and / or completion queues), thereby implementing update notifications using the consistency mechanism of the consistency interface.

[0082] In some embodiments, a communication protocol (e.g., NVMe) may implement (e.g., define) one or more consistency settings (e.g., snooping settings, and / or alarm settings, etc.) on one or more queue memory addresses.

[0083] In some embodiments, updating a queue entry (e.g., adding a submission queue entry by a host) can trigger activity in a device (e.g., a storage device such as an SSD).

[0084] In some embodiments, the host may be interrupted based on eavesdropping and / or alarms.

[0085] In some embodiments, according to example embodiments of this disclosure, when booting a device (e.g., a host, and / or a device, etc.), the device may declare one or more capabilities. For example, if the device is implemented as an NVMe device, it may (e.g., using configuration bits) declare that it is capable of using a consistency mechanism to indicate queue updates.

[0086] Any functionality described herein, including any of the host functionality and / or device functionality (e.g., protocol logic 342, controller 230, protocol controller 330, snooping logic 346 and / or 360, and / or consistency engine 344 and / or 358, etc.), may be implemented in hardware, software, firmware, or any combination thereof, including, for example, hardware and / or software combinational logic, sequential logic, timers, counters, registers, state machines, volatile memory (such as dynamic random access memory (DRAM) and / or static random access memory (SRAM)), non-volatile memory (including flash memory, permanent memory (such as...) This includes cross-grid nonvolatile memory, memory with varying bulk resistance, and / or phase-change memory (PCM) and / or any combination thereof, complex programmable logic devices (CPLDs) that execute instructions stored in any type of memory, field-programmable gate arrays (FPGAs), application-specific integrated circuit (ASIC) CPUs (including complex instruction set computer (CISC) processors such as x86 processors and / or reduced instruction set computer (RISC) processors such as ARM processors), graphics processing units (GPUs), neural processing units (NPUs), and / or tensor processing units (TPUs), etc. In some embodiments, one or more components may be implemented as a system-on-a-chip (SOC).

[0087] Figure 5 An example embodiment of a host device according to an exemplary embodiment of this disclosure is shown. For example, Figure 5 The host device 500 shown can be used to implement any host disclosed herein. Figure 5 The host device 500 shown may include a processor 502 including a memory controller 504, a system memory 506, a host logic 508, and / or a coherence interface 510. Figure 5 Any or all of the components shown can communicate via one or more system buses 512. In some embodiments, Figure 5 One or more of the components shown can be implemented using other components. For example, in some embodiments, host control logic 508 can be implemented by processor 502 executing instructions stored in system memory 506 or other memory. In some embodiments, host logic 508 can implement any of the host functions disclosed herein, including, for example, protocol logic 342, and / or reading and / or writing to one or more queues, etc.

[0088] Figure 6 Example embodiments of a device according to exemplary embodiments of this disclosure are shown. For example, Figure 6The illustrated embodiment 600 can be used in any of the devices disclosed herein. Device 600 may include device controller 602, communication logic 608, device function circuitry 606, and / or conformance interface 610. Figure 6 The components shown can communicate via one or more device buses 612. Communication logic 608 can be used, for example, to implement controller 230 and / or protocol controller 330, etc.

[0089] Device functional circuitry 606 may include any hardware that implements the main functions of device 600. For example, if device 600 is implemented as a storage device, device functional circuitry 606 may include storage media such as one or more flash memory devices, and / or FTL, etc. As another example, if device 600 is implemented as a network interface card (NIC), device functional circuitry 606 may include one or more modems, network interfaces, physical layer (PHY), and / or media access control layer (MAC), etc. As another example, if device 600 is implemented as an accelerator, device functional circuitry 606 may include one or more accelerator circuits, and / or memory circuits, etc.

[0090] Figure 7 An embodiment of a method for communication according to an exemplary embodiment of this disclosure is shown. The method may begin at operation 702. At operation 704, the method may access a queue via a first device, wherein the queue is accessible by a second device, and the first device is connected to the second device via a consistency interface. For example, in some embodiments, the first device may be implemented as a host, the second device may be implemented as a device, and the queue may be implemented as a submission queue of a communication protocol. At operation 706, the method may indicate the access to the second device via the consistency interface. For example, in some embodiments, the access may be implemented by the host placing an entry in the submission queue, and the indication may be implemented by configuring the consistency interface to monitor (e.g., eavesdrop) the submission queue and send an alert to the device. The method may end at operation 708.

[0091] Figure 7 The embodiments shown, as well as all other embodiments described herein, are example operations and / or components. In some embodiments, some operations and / or components may be omitted, and / or other operations and / or components may be included. Furthermore, in some embodiments, the temporal and / or spatial order of operations and / or components may vary. Although some components and / or operations may be shown as separate components, in some embodiments, some components and / or operations shown separately may be integrated into a single component and / or operation, and / or some components and / or operations shown as single components and / or operations may be implemented using multiple components and / or operations.

[0092] Some embodiments disclosed above have been described in the context of various implementation details, but the principles of this disclosure are not limited to these or any other specific details. For example, a function has been described as being implemented by certain components, but in other embodiments, the function may be distributed among different systems and components in different locations and with various user interfaces. Some embodiments have been described as having specific processes, operations, etc., but these terms also cover embodiments in which specific processes, operations, etc. may be implemented by multiple processes, operations, etc., or in which multiple processes, operations, etc. may be integrated into a single process, step, etc. References to components or elements may refer to only a portion of a component or element. For example, a reference to a block may refer to the entire block or one or more sub-blocks. The use of terms such as “first” and “second” in this disclosure and claims may be merely for the purpose of distinguishing the elements they modify and may not indicate any spatial or temporal order unless obvious from the context. In some embodiments, a reference to an element may refer to at least a portion of an element; for example, “based on” may mean “at least partially based on”, etc. A reference to a first element does not imply the existence of a second element. The principles disclosed herein have independent utility and can be embodied individually, and not every embodiment can utilize every principle. However, the principles can also be embodied in various combinations, some of which can synergistically amplify the benefits of the individual principles.

[0093] Based on the inventive principles disclosed in this patent, the various details and embodiments described above can be combined to produce other embodiments. Since the inventive principles disclosed in this patent can be modified in arrangement and detail without departing from the inventive concept, such changes and modifications are considered to fall within the scope of the following claims.

Claims

1. A method for queue management, comprising: Access to a queue via a first device using a first protocol, wherein the queue is accessible by a second device including shared local memory, and the first device is connected to the second device via a consistency interface; and Through the consistency interface, the second protocol is used to provide the second device with an indication of access to the queue via the first device using the first protocol; in: The first device is configured to access the queue using the first protocol; The second device is configured to access the queue using the first protocol; The consistency interface is configured to maintain the consistency of at least a portion of the shared local memory through a memory consistency mechanism, and to maintain the consistency of the queue through a cache consistency mechanism; and The queue is located at least partially in the at least part of the shared local memory.

2. The method according to claim 1, wherein, Providing the instruction includes generating a monitoring request.

3. The method according to claim 2, wherein, Providing the instruction includes generating an alert based on the monitoring request.

4. The method according to claim 1, wherein, The queue includes a submission queue.

5. The method according to claim 1, wherein, The access includes reading an entry from the queue.

6. The method according to claim 1, wherein, The access includes writing an entry into the queue.

7. The method according to claim 6, wherein, The entries include commands.

8. The method according to claim 6, wherein, The entry includes completion.

9. The method according to claim 1, wherein: The first device includes a host computer; The second device includes equipment; and The queue is located at the device.

10. The method according to claim 1, wherein, The first protocol includes a storage protocol.

11. The method according to claim 1, wherein, The consistency interface includes monitoring logic, which is configured to operate as follows: The access is detected by monitoring the queue; and The indication is generated based on the detection.

12. A device for queue management, comprising: Consistent interface; as well as At least one control circuit is configured as follows: The first protocol uses queues for communication; as well as Through the consistency interface, an instruction to access the queue is received using a second protocol, wherein the access to the queue is performed using the first protocol; in: The device includes shared local memory; and The consistency interface is configured to maintain the consistency of at least a portion of the shared local memory through a memory consistency mechanism, and to maintain the consistency of the queue through a cache consistency mechanism; and The queue is located at least partially in the at least part of the shared local memory.

13. The device according to claim 12, wherein, The consistency interface is configured to receive monitoring requests based on the access.

14. The device according to claim 13, wherein, The consistency interface is configured to generate alerts based on the monitoring request.

15. The device according to claim 14, wherein, The at least one control circuit is configured to receive the alarm as the indication from the consistency interface.

16. The device according to claim 12, wherein, The consistency interface includes monitoring logic, which is configured to operate as follows: The access is detected by monitoring the queue; and The indication is generated based on the detection.

17. A system for queue management, comprising: Host; as well as A device coupled to the host using a consistency interface, the device including at least one control circuit, the at least one control circuit being configured to: Using the first protocol, a queue is used to communicate with the host; as well as Through the consistency interface, the host receives an instruction to access the queue using the first protocol via the second protocol; in: The device includes a shared local memory; The consistency interface is configured to maintain the consistency of at least a portion of the shared local memory through a memory consistency mechanism, and to maintain the consistency of the queue through a cache consistency mechanism. as well as The queue is located at least partially in the at least part of the shared local memory.

18. The system according to claim 17, wherein, The consistency interface includes monitoring logic, which is configured to operate as follows: The access is detected by monitoring the queue; and The indication is generated based on the detection.

Images