Inter-driver communication for co-operation

Abstract

The invention relates to inter-driver communication for cooperative operation, and discloses a system. The system may include two devices, and a processor in communication with the two devices. The buffer may store entries including communications from one device to another.

Description

Technical Field

[0001] The present disclosure relates generally to devices, and more particularly to inter-device communications. Background Art

[0002] Devices such as storage devices or other devices connected via a memory structure are usually under the control of a host. The host processor issues requests and the devices execute these commands. For the devices to communicate, one device sends a request to the host processor, and the host processor sends a request to the other device.

[0003] There still remains a need to allow devices to communicate without involving the host. Summary of the invention

[0004] The system may include a processor and two devices. The processor may communicate with each device, and each device may communicate with the other device. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The drawings described below are examples of how embodiments of the present disclosure may be implemented and are not intended to limit the embodiments of the present disclosure. Various embodiments of the present disclosure may include elements not shown in a particular drawing and / or may omit elements shown in a particular drawing. The drawings are intended to provide illustrations and may not be to scale.

[0006] Figure 1 A machine including a storage device that can support inter-device communication according to an embodiment of the present disclosure is shown.

[0007] Figure 2 The embodiment according to the present disclosure is shown Figure 1 Details of the machine.

[0008] Figure 3 FIG. 1 shows a method for communicating with each other using buffers according to an embodiment of the present disclosure. Figure 1 of three devices.

[0009] Figure 4 The embodiment according to the present disclosure is shown Figure 3 Details of a buffer in the buffers.

[0010] Figure 5 The use of an embodiment according to the present disclosure is shown Figure 3 The buffer Figure 1 communication between two devices.

[0011] Figure 6 The invention shows the establishment of an embodiment according to the present disclosure Figure 3 Buffer for inter-device communication Figure 1 processor and Figure 1 equipment.

[0012] Figure 7 FIG. 1 shows a method of communicating through a switch according to an embodiment of the present disclosure. Figure 1 equipment.

[0013] Figure 8 1 shows a method for communicating to create a processing pipeline according to an embodiment of the present disclosure. Figure 1 equipment.

[0014] Fig. 9 The embodiment according to the present disclosure is shown Figure 1 Device usage Figure 3 A flowchart of an example process for communicating with buffers.

[0015] Fig.10 Continuing with the embodiment of the present disclosure Figure 1 Device usage Figure 3 The buffer communicates Fig. 9 A flowchart of an example process.

[0016] Fig.11 The embodiment according to the present disclosure is shown Figure 1 The device places the entry in Figure 3 Flowchart of an example process in a buffer.

[0017] Fig.12 The embodiment according to the present disclosure is shown Figure 1 The device from Figure 3 Flowchart of an example process for retrieving entries from a buffer.

[0018] Fig.13 The embodiment according to the present disclosure is shown Figure 1 The device is created to be placed Figure 3 Flowchart of an example process for storing entries in a buffer.

[0019] Fig.14 The embodiment according to the present disclosure is shown Figure 1 Device identification Figure 3 A flowchart of an example process for storing entries in a buffer of Figure 3 The device is the intended recipient of this entry.

[0020] Fig.15 The embodiment according to the present disclosure is shown Figure 1 The device will be able to Figure 1 Entries for reading or writing from more than one device are placed in Figure 3 Flowchart of an example process in a buffer.

[0021] Fig.16The embodiment according to the present disclosure is shown Figure 1 processor or Figure 1 Equipment establishment Figure 3 A flowchart of an example process for a buffer. DETAILED DESCRIPTION

[0022] Reference will now be made in detail to embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. In the following detailed description, many specific details are set forth to enable a thorough understanding of the present disclosure. However, it should be understood that one of ordinary skill in the art may practice the present disclosure without these specific details. In other cases, well-known methods, processes, components, circuits, and networks are not described in detail to avoid unnecessarily obscuring aspects of the embodiments.

[0023] It should be understood that although the terms first, second, etc. can be used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the scope of the present disclosure, the first module can be referred to as the second module, and similarly, the second module can be referred to as the first module.

[0024] The terms used in the description of the present disclosure are only used for the purpose of describing specific embodiments and are not intended to limit the present disclosure. As used in the description of the present disclosure and the appended claims, the singular forms "one", "an" and "the" are intended to also include plural forms, unless the context clearly states otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that when used in this specification, the terms "include" and / or "comprising" specify the presence of stated features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. The components and features of the accompanying drawings are not necessarily drawn to scale.

[0025] Devices such as storage devices are typically under host control. The host processor can issue requests to the device, which can then execute the request and return the results to the host processor.

[0026] Such devices are typically connected via a memory structure. The memory structure enables the device and the host to exchange data. For example, the device can read data from or write data to the system's main memory. Similarly, to the extent that their internal storage devices (such as the device's local memory) are exposed to the host processor via the memory structure, the host processor can write data to or read data from the device's exposed storage devices.

[0027] Because the devices can all be connected to the same memory structure, the devices can all read and write the exposed memory of other devices. Requests are different from the host, however, in that they can be received at queues designated for this purpose. Typically, queues come in pairs: a submission queue, where devices can receive requests, and a completion queue, where devices can return request results.

[0028] A queue (or queue pair) can be specific to a sender and a receiver. Because a device may not know in advance how many other devices may be connected, typically the device manages queues (or queue pairs) only for the host processor. This fact means that for one device to send a request to another device, the first device can send a request to the host, which can then send a request to the second device (both requests are sent along the appropriate queue or queue pair).

[0029] Without adding the requirement for inter-device messaging, the host processor already has processes to manage. In addition, sending messages from one device to another via the host processor may increase latency due to the message traveling via the host processor rather than directly between the devices.

[0030] Embodiments of the present disclosure solve these problems by providing a mechanism for direct device-to-device communication. A buffer, such as a ring buffer, may be provided. A device may write a message to a ring buffer, and another device may read a message from the ring buffer. Although a ring buffer may be shared (so that more than one device may be able to write to a single read buffer, or more than one device may be able to read from a single read buffer), by establishing a ring buffer that may be written by only one device and may be read by only one device, a locking mechanism (for avoiding conflicting devices from reading from or writing to a single ring buffer) may be avoided. Therefore, for each pair of devices, two ring buffers may be used: one ring buffer per device to send a message to another device. These ring buffers may be located anywhere: in main memory, in a device that sends a message, in a device that receives a message, or in a third device.

[0031] Devices may use any desired logic to check their ring buffers. For example, a device may use a round robin technique to check the ring buffers for messages from other devices to process next. Alternatively, a device may look for the ring buffer with the oldest message to process next.

[0032] Such inter-device communication can be used for a variety of purposes. For example, a computational storage device may use such inter-device communication to create a pipeline to process data sequentially as directed by a host processor. Or, devices may use inter-device communication to share analytics and perform system-wide wear leveling or coordinated garbage collection. Or, devices may use inter-device communication to advertise their capabilities, including their available exposed storage or compute functions. Or, devices may use inter-device communication to distribute processing requests from a host. Or, devices may use inter-device communication to allocate and manage each other's available exposed storage. Or, devices may use inter-device communication to perform data migration: for example, because one device is beginning to fail.

[0033] Figure 1 A machine including a storage device that can support inter-device communication according to an embodiment of the present disclosure is shown. Figure 1 In the present invention, machine 105 (which may also be referred to as a host or system) may include processor 110, memory 115, and storage devices 120-1 and 120-2 (which may be collectively referred to as storage devices 120). Processor 110 may be any kind of processor. Processor 110 may also be referred to as a host processor. (For ease of illustration, processor 110 and other components discussed below are shown outside the machine: embodiments of the present disclosure may include these components within the machine.) Although Figure 1 A single processor 110 is shown, but the machine 105 may include any number of processors, each of which may be a single-core or multi-core processor, each of which may implement a reduced instruction set computer (RISC) architecture or a complex instruction set computer (CISC) architecture (among other possibilities), and which may be mixed in any desired combination.

[0034] Processor 110 may be coupled to memory 115. Memory 115 may be any kind of memory, such as dynamic random access memory (DRAM), static random access memory (SRAM), persistent random access memory, ferroelectric random access memory (FRAM), or non-volatile random access memory (NVRAM) (such as magnetoresistive random access memory (MRAM), flash memory, etc.). Memory 115 may also be any desired combination of different memory types and may be managed by memory controller 125. Memory 115 may be used to store what may be referred to as "short-term" data: that is, data that is not expected to be stored for an extended period of time. Examples of short-term data may include temporary files, data used locally by an application (which may have been copied from other storage locations), etc.

[0035] The processor 110 and the memory 115 may also support an operating system under which various applications may run. These applications may issue requests (which may also be referred to as commands) to read data from the memory 115 or the storage device 120 or write data to the memory 115 or the storage device 120.

[0036] Storage device 120 may be used to store what may be referred to as “long-term” data: that is, data that is intended to be stored for a longer period of time, or that does not need to be stored in memory 115. Storage device 120 may be accessed using device driver 130. Although Figure 1 Two storage devices 120 are shown, but any number (two or more) of storage devices may be present in machine 105. Figure 1 Two storage devices 120 are shown as being accessed using the same device driver 130, but embodiments of the present disclosure may make each storage device 120 accessible using a different device driver 130, the same device driver 130, or any combination thereof. The storage devices 120 may also be used to store data in a persistent or non-volatile manner: that is, in a manner that retains the data even if the storage devices 120 are no longer provided with power. This may be contrasted with, for example, the memory 115, which is typically implemented using volatile memory (such as DRAM) that may lose data if the memory 115 is no longer provided with power.

[0037] The storage device 120 may include a storage medium (media) on which data may be stored. For example, the storage medium may include NAND flash memory, such as may be included in a solid state drive (SSD) or a locally magnetized disk in a hard disk drive. The storage device 120 may also include a controller to manage writing data to the storage medium and reading data from the storage medium. The storage device 120 may also include a mechanism for performing the writing of data to the storage medium or reading data from the storage medium. For example, such a mechanism may include a circuit for storing or reading a voltage from one or more cells in the NAND flash memory, or an actuator for applying or reading magnetization of a location on the disk.

[0038] The storage devices 120 may be connected to each other and to the processor 110 using any desired method or mechanism. For example, the storage devices 120 may be connected via one or more buses in the machine 105. Alternatively, the storage devices 120 may include direct connections to other storage devices 120. Embodiments of the present disclosure may include any type of connection (or any combination of connections) between and / or among the storage devices 120. Note that a given storage device 120 may communicate with the processor 110 using one bus, path, and / or protocol, and communicate with another storage device 120 using a different bus, path, and / or protocol (and indeed, may communicate with each other storage device 120 using a different bus, path, and / or protocol).

[0039] Depending on the manner in which the processor 110 and the storage device 120 can communicate, the storage device 120 may include one or more interfaces to communicate with the processor 110 and / or other storage devices 120. For example, if the processor 110 and all storage devices 120 are connected via a single peripheral component interconnect express (PCIe) bus, a single interface on the storage device 120 may be sufficient to communicate with the processor 110 and with the other storage devices 120. If the storage device 120 uses a different mechanism to communicate with the processor 110 than with the other storage devices 120, the storage device 120 may include an interface appropriate for the communication bus, path, and / or protocol used, and thus may include more than one interface. The storage device 120 may potentially include up to n interfaces, where the storage device 120 communicates with n-1 other storage devices and the processor 110 (i.e., the machine 105 includes a total of n storage devices 120).

[0040] More generally, the storage device 120 may be an example of various types of devices that may benefit from inter-device communication. An example of another type of device that may benefit from inter-device communication is a computational storage unit (which may also be referred to as a computational storage device or a computing device). The computational storage unit may be a form of processing that may be performed closer to where the data is stored (on the storage device 120). To this end, the computational storage unit may include some form of processing circuitry that may be implemented using, for example, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a central processing unit (CPU) or other type of processor, a graphics processing unit (GPU), a general purpose GPU (GPGPU), a tensor processing unit (TPU), a neural processing unit (NPU), or any other desired form of implementation. Such an implementation may include one or more pre-programmed functions: for example, functions of filtering data, encrypting or decrypting data, compressing or decompressing data, etc. Such an implementation may also support custom functions that may be downloaded into the processing circuitry of the computational storage unit, so that the functions of the computational storage unit may be modified according to the needs of the application. In some embodiments of the present disclosure, the computational storage unit and the storage device may be combined into a single unit, thereby providing storage and computing capabilities.

[0041] Embodiments of the present disclosure may include any desired mechanism for communicating with device 120. For example, device 120 may be connected to one or more buses, such as a PCIe bus, or device 120 may include an Ethernet interface or some other network interface. Other potential interfaces and / or protocols to device 120 may include non-volatile memory express (NVMe), NVMe over Fabrics (NVMe-oF), remote direct memory access (RDMA), transmission control protocol / Internet protocol (TCP / IP), universal flash storage (UFS), embedded multimedia card (eMMC), InfiniBand, serial attached small computer system interface (SCSI) (SAS), Internet SCSI (iSCSI), serial AT attachment (SATA), and cache coherent interconnect protocols such as compute express link (Compute Express ) protocols, etc. (Compute Express Link and CXL are registered trademarks of Compute Express Link Consortium, Inc.) As described above, storage device 120 may include a required number of interfaces to support communications using a desired number of interfaces, protocols, and / or paths.

[0042] Although Figure 1The generic term "storage device" is used, but embodiments of the present disclosure may include any storage device format (or more generally, device format) that may benefit from inter-device communication, examples of which may include a hard drive, an SSD, or a computational storage unit. Any reference to any particular type of device 120 (such as a reference to an "SSD") should be understood to include such other embodiments of the present disclosure.

[0043] Figure 2 The embodiment according to the present disclosure is shown Figure 1 Details of the machine 105. Figure 2 In the embodiment of the present invention, generally, the machine 105 includes one or more processors 110, which may include a memory controller 120 and a clock 205, which may be used to coordinate the operation of the components of the machine. The processor 110 may also be coupled to a memory 115, which may include a random access memory (RAM), a read-only memory (ROM), or other state preservation medium as an example. The processor 110 may also be coupled to a storage device 125 and to a network connector 210, which may be, for example, an Ethernet connector or a wireless connector. The processor 110 may also be connected to a bus 215, to which a user interface 220 and input / output (I / O) interface ports that may be managed using an I / O engine 225 and other components may be attached.

[0044] Figure 3 FIG. 1 shows a method for communicating with each other using buffers according to an embodiment of the present disclosure. Figure 1 The three devices 120. Figure 3 , three devices 120-1, 120-2, and 120-3 are shown. Device 120, as illustrated by device 120-1, may include a peripheral component interconnect express (PCIe) layer 305, which may communicate with the PCIe bus using the PCIe protocol. Figure 1 10 (and other devices 120). In embodiments of the present disclosure where other forms of communication besides across a PCIe bus are used, PCI layer 305 may be replaced with another appropriate layer. Figure 1 In embodiments of the present disclosure where devices 120-2 and 120-3 and / or processor 110 communicate, PCIe layer 305 and another layer suitable for another bus, protocol and / or path may be used. That is, device 120-1 may include multiple interfaces to support communications using different buses, protocols and / or paths.

[0045] The device 120 may also include a non-volatile memory express (NVMe) layer 310, which may enable communication with the NVMe protocol over the PCIe bus. Figure 1 The device 120 may also include a media management interface 315, which may be used to access data from the storage media. The device 120 may also include a processor 320 (which may be referred to as a computational storage processor to communicate with the processor 110 (and other devices 120). Figure 1 The PCIe layer 305, NVMe layer 310, and media management interface 315 may be implemented individually, collectively, or in any desired combination using any desired circuit, controller, FPGA, ASIC, or other desired element. The PCIe layer 305, NVMe layer 310, and media management interface 315 may also be implemented individually or collectively as software executed on the processor 320.

[0046] Finally, the device 120 may also include a subsystem local memory (SLM) 325, which may serve as local memory that may be used by the computational storage processor 320 when performing its operations. The SLM 325 may be implemented using any desired form of volatile and / or non-volatile memory, such as DRAM, SRAM, flash memory, etc.

[0047] Within the SLM 325, each device 120 may include a buffer. For example, device 120-1 may include buffers 330-1 and 330-5, device 120-2 may include buffers 330-2 and 330-3, and device 120-3 may include buffers 330-4 and 330-6. Buffers 330-1 through 330-6 may be collectively referred to as buffers 330. Since in the SLM 325 of a device 120, each device 120 may read data from or write data to the SLM 325 of other devices 120 via an underlying bus (such as a PCIe bus). Any desired mechanism for accessing the SLM 325 of other devices 120 may be used: for example, including PCIe, CXL, remote direct memory access (RDMA), and The memory structure of NVLink can support access to the memory of another device 120 along the structure (NVLink is a registered trademark of NVIDIA Corporation). Buffer 330 can thus be used for inter-device communication.

[0048] Buffer 330 may take any desired form. In some embodiments of the present disclosure, buffer 330 may be a first-in-first-out (FIFO) queue. A FIFO queue may help ensure that the oldest entry in the FIFO queue is retrieved first. Any other desired queue structure may also be used. Figure 4 Let's discuss FIFO queues further.

[0049] In some embodiments of the present disclosure, each buffer 330 is dedicated to communication between a particular pair of devices 120 and in a particular direction. Thus, for example, buffer 330-1 of device 120-1 may be used to receive a message or request from device 120-2 and buffer 330-5 of device 120-1 may be used to receive a message or request from device 120-3, buffer 330-2 of device 120-2 may be used to receive a message or request from device 120-1 and buffer 330-3 of device 120-2 may be used to receive a message or request from device 120-3, and buffer 330-4 of device 120-3 may be used to receive a message or request from device 120-2 and buffer 330-6 of device 120-3 may be used to receive a message or request from device 120-1. By using a unique buffer 330 for communication from one specific device 120 to another specific device 120 (where communication in the other direction is handled by another buffer 330), embodiments of the present invention may avoid critical sections (where two devices 120 may attempt to use a buffer 330 at the same time). A consequence of using a single buffer 330 for unidirectional communication between a specific pair of devices 120 is that the number of buffers 330 may grow quadratically with the number of devices 120. For example, two devices 120 may require two buffers 330 to communicate, three devices 120 may require six buffers 330 to communicate, four devices 120 may require 12 buffers 330 to communicate, and so on. As a general rule, the number of buffers n required to support all communication methods between d devices may be expressed as (The reason why Because there is The device 120 may be coupled to a pair of devices 120, and one buffer 330 may be used for communication in each direction, resulting in two buffers 330 per pair of devices 120).

[0050] exist Figure 3 , each device 120 is shown as including a buffer 330 for receiving messages or requests from other devices 120. For example, buffer 330-1 is for messages or requests received from device 120-2, buffer 330-2 is for messages or requests received from device 120-1, and so on. Having each device 120 include a buffer 330 for receiving messages or requests from other devices 120 is one way to organize the buffers 330. However, other embodiments of the present disclosure may organize the buffers 330 in other ways. For example, some embodiments of the present disclosure may have the buffers 330 stored in the SLM 325 of the device 120 that is sending messages or requests (rather than receiving them). Other embodiments of the present disclosure may have the buffers 330 stored in the SLM 325 of the device 120 that is sending messages or requests (rather than receiving them). Figure 1Other embodiments of the present disclosure may have all buffers 330 stored in one device 120 (where each device 120 accesses the SLM 325 of one device 120 storing the buffers 330). Other combinations are possible, and embodiments of the present disclosure may include organizing and storing the buffers 330 in Figure 1 any desired location within the machine 105.

[0051] As described above, buffer 330 may be used to issue requests from one device 120 to another device 120. In some embodiments of the present disclosure, such requests may be placed in an entry in buffer 330, which may take the form of a submission queue entry (SQE), such as may be provided by Figure 1 The processor 110 is used to issue a request to the device 120. In addition, when a device 120 completes such a request, the device 120 executing the request may issue a response to the requesting device 120 to let the requesting device 120 know that the request has been completed. Such a response may be placed in an entry in the buffer 330, which may take the form of a completion queue entry (CQE), such as the one that can be placed in the buffer 330 after completing the request from the device 120. Figure 1 The request of the processor 110 is returned from the device 120 to Figure 1 The processor 110 of FIG.

[0052] Note that buffer 330 may include two types of entries: buffer 330 may include requests from another device 120 to be executed by receiving device 120, and responses from another device 120 to requests issued by device 120. Thus, for example, buffer 330-1 may include both a request sent from device 120-2 to device 120-1 and a response by device 120-1 to a request made by device 120-2 (such a request would have been placed in buffer 330-2 by device 120-1).

[0053] As described above, by having a unique buffer 330 for communication from one device 120 to another device 120, critical sections can be avoided. A critical section can be a mechanism by which a buffer 330 can be locked for use by a particular device 120: any other device 120 attempting to access that buffer 330 can wait until the lock is released. Thus, for example, a single buffer 330 can be used for all communications between two devices 120, regardless of the direction of the communication. But to prevent two devices 120 from trying to write to a single buffer 330 at the same time, a locking mechanism can be used. Whichever device 120 holds the lock can write to the single buffer 330, and the other device 120 can block until the lock is released. Any desired form of locking mechanism can be used to ensure that only one device 120 can write to a single buffer 330 at a time. In the case where multiple devices place requests in a single buffer 330, each device 120 can include an identifier of the device 120 that sent the message or request in the entry in the buffer 330, so that the receiving device 120 can know which device 120 issued the request. Thus, embodiments of the present disclosure may include locks if appropriate or desired for implementation.

[0054] In a similar manner, multiple devices 120 can read entries from a single buffer 330. That is, a single device 120 can place messages or requests to multiple devices 120, where all such messages or requests are placed in entries in a single buffer 330. The shared buffer 330 can be located anywhere desired: for example, in Figure 1 330 or in the SLM 325 of any device 120. The various devices 120 to which such messages or requests are directed may then read those entries from the buffer 330. Again, to prevent multiple devices 120 from accessing the buffer 330 simultaneously, a locking mechanism may be used to ensure that two different devices 120 may not attempt to read and execute the same message from an entry in the buffer 330.

[0055] In embodiments of the present disclosure where multiple devices 120 can write to a single buffer 330, entries in the buffer 330 can be modified to reflect which device 120 wrote the entry. In embodiments of the present disclosure where multiple devices 120 can read from a single buffer 330, entries in the buffer 330 can be modified to reflect which device 120 the entry is intended for. Note that these embodiments can also be combined: in the most general case, there may be only one buffer 330 that all devices 120 can both write to and read from. In the remainder of this document, the discussion will focus on embodiments of the present disclosure where each device 120 can write to a unique buffer 330 for communicating with another device 120 (e.g., Figure 3), but other embodiments of the present disclosure may be understood to be covered as well.

[0056] Even in embodiments of the present disclosure where each storage device 120 uses a unique buffer 330 to send messages to another storage device (such as Figure 3 330-2, and storage device 120-2 may be waiting for storage device 120-1 to execute a request from buffer 330-1. With each storage device 120 waiting for the other, neither can do any processing. The answer to this question would be "no" for several reasons. First, processor 320 may be able to process several requests from buffer 330 at once, much like how storage device 120 may simultaneously process requests from buffer 330. Figure 1 The machine 105 reads and executes multiple SQEs. For example, as shown below Figure 4 As discussed, device 120 may be able to process a request in buffer 330 other than the oldest request in buffer 330 (i.e., not the head of a line in buffer 330). Second, processor 320 may track the status of requests issued to other storage devices 120. If a request effectively times out due to a lack of response, processor 320 may conclude that the other storage device 120 is not functional, may treat the request issued to that storage device 120 as incomplete, and may proceed in any desired manner (e.g., potentially notifying the other storage device 120). Figure 1 Machine 105 causes the original request of the inter-device request to fail to complete successfully).

[0057] It is worth noting that buffer 330 serves as a communication path between storage devices 120. Storage devices 120 may continue to communicate with each other in any desired manner. Figure 1 For example, the storage device 120 may use queues, such as a submission queue / completion queue pair, which may be stored in, for example, Figure 1 In the memory 115, as Figure 1 The communication path of the processor 110 to Figure 1 The processor 110 receives the request and sends Figure 1 The processor 110 sends a response.

[0058] Figure 4 The embodiment according to the present disclosure is shown Figure 3 Details of the buffer 330. Figure 4, buffer 330 is shown as a circular buffer. A circular buffer is a circular buffer: the "last" entry in the circular buffer is followed by the "first" entry in the circular buffer. With a circular buffer, you don't have to worry about passing the "end" of the buffer. Buffer 330 is shown as containing eight entries 405-1 to 405-8 (which may be collectively referred to as entries 405).

[0059] The ring buffer may be implemented as, for example, an array of addresses in a memory. If the base address of the ring buffer is b, each entry 405 has a size e, and the total number of entries 405 in the ring buffer is n, then the address of the i-th entry 405 in the ring buffer may be calculated as a i = b + ((i-1) * e) (this assumes that the first entry 405 is referred to as entry 405 1: if entries 405 start counting from zero, then (i-1) can be replaced by i in the above equation. More importantly, given the address of entry 405i (which can be referred to as a i ), the address of the next entry 405 in the ring buffer (which would be the first entry if the current entry is the last entry in the ring buffer) can be calculated as a=((a i +eb) % (n*e)) + b, where % is the modulo operator. Using this equation, the sending device 120 does not need to determine which entry 405 was last used and check to see if that entry 405 was the "last" entry 405 in the circular buffer: the sending device 120 can calculate the address of the next entry 405, where wrapping from the "last" entry 405 to the "first" entry 405 occurs automatically.

[0060] The address of the entry 405 in the ring buffer may also be calculated for the kth packet written to the ring buffer. Again, if the base address of the ring buffer is b, each entry 405 has a size e, and the total number of entries 405 in the ring buffer is n, then the address where the kth packet may be written to the ring buffer may be calculated as a. k =b+((k-1)%n)*e.

[0061] Note that these last two equations assume that packets are always added to and removed from the circular buffer in the same order, without creating any gaps in the sequence. In embodiments of the present disclosure where packets may be removed from the circular buffer out of order, the entry 405 determined by these equations may not be "open" for storing a new packet, in which case the circular buffer may be searched to find the next open entry 405.

[0062] Buffer 330 may also include two additional pointers. Head pointer 410 may track the entry at the "head" of the row in buffer 330 (i.e., the entry 405 that was first placed in buffer 330). Tail pointer 415 may track the entry at the "tail" of the row in buffer 330 (i.e., the entry 405 that was last placed in buffer 330). The sending device 120 may use tail pointer 415 to determine which entry 405 may be used next to send a message to the receiving device 120, and the receiving device 120 may use head pointer 410 to determine which entry 405 may be retrieved next. Note that the terms may be interchangeable depending on the implementation. The names of pointers 410 and 415 are not as important as how they are used.

[0063] Both the sending device 120 and the receiving device 120 have access to the head pointer 410 and the tail pointer 415. The sending device 120 may update the tail pointer 415 after adding a new entry 405 to the buffer 330, and the receiving device 120 may update the head pointer 410 after retrieving an entry 405 from the buffer 330. Note, however, that the sending device 120 may also have access to the head pointer 410, and the receiving device 120 may also have access to the tail pointer 415. For example, the sending device 120 may check to see that there is room in the buffer 330 for the new entry 405. If the sending device 120 does not perform such a check, then the sending device 120 may overwrite an existing entry 405 that has not yet been processed by the receiving device 120. The sending device 120 can determine whether there is room for a new entry 405 in the buffer 330 by comparing the head pointer 410 and the tail pointer 415: if the head pointer 410 points to the entry 405 immediately following the entry 405 pointed to by the tail pointer 415, then the buffer 330 is full, and the sending device 120 should wait until the receiving device 120 has removed the entry from the buffer 330. Once there is room for the new entry 405 in the buffer 330, the sending device 120 can store the appropriate information in the entry 405 of the buffer 330. Similarly, the receiving device 120 can check to see if there are any entries 405 in the buffer 330 that have not yet been processed. If the receiving device 120 does not perform this check, then the receiving device 120 may read data from an entry 405 that was previously processed (and should not be processed again). Receiving device 120 may determine whether buffer 330 is empty by comparing head pointer 410 and tail pointer 415 : if both head pointer 410 and tail pointer 415 point to the same entry 405 , then buffer 330 is currently empty and receiving device 120 should not attempt to retrieve entry 405 from buffer 330 .

[0064] Note that the above description of buffer 330 indicates that receiving device 120 can remove only the entry 405 pointed to by head pointer 410. In some embodiments of the present disclosure, this implementation is intentional. However, other embodiments of the present disclosure may support removing any entry 405 from buffer 330, not just the entry 405 pointed to by head pointer 410. For example, an auxiliary array may be included that may include an indication Figure 1 The storage device 120 may store a value indicating whether the corresponding entry 405 has been processed by the receiving device 120. This auxiliary array may identify whether a particular entry 405 in the buffer 330 contains a message or request waiting to be processed. The head pointer 410 may be arranged to always point to the oldest entry 405 in the buffer 330 that contains a message or request waiting to be processed. Thus, when an entry 405 in the buffer 330 pointed to by the head pointer 410 is removed, the head pointer 410 may be moved to the next entry 405 in the buffer 330 that is waiting to be processed: this entry 405 may be any number of entries 405 below the buffer 330, rather than the next entry 405 in the buffer 330. The receiving device 120 may repeatedly increment the head pointer 410 to point to the next entry 405 in the buffer 330 until the head pointer 410 points to an entry 405 that the auxiliary list indicates contains a message or request waiting to be processed. Because an entry in the auxiliary list may only need to indicate whether the corresponding entry 405 in the buffer 330 contains a message or request waiting to be processed, the auxiliary list may only require one bit for each entry 405 in the buffer 330, which keeps the size of the auxiliary list small.

[0065] In case this is unclear, consider the following. The companion buffer 330 may be a bit array ( Figure 4 (not shown): one bit for each entry 405 in the buffer 330. This bit may be set to, for example, one to indicate that the corresponding entry 405 in the buffer 330 contains a request (or response) waiting to be processed. Figure 1 When the storage device 120 removes a request (or response) from the entry 405 in the buffer 330 other than the entry pointed to by the head pointer 410, Figure 1 The storage device 120 may change the bit in the auxiliary array corresponding to the entry 405 in the buffer 330 to zero to indicate that the corresponding entry 405 in the buffer 330 has been processed. Then, when the storage device 120 is ready to change the head pointer 410 (because the entry 405 in the buffer 330 pointed to by the head pointer 410 has been processed), Figure 1Storage device 120 may set the bit in the auxiliary array corresponding to entry 405 in buffer 330 to zero, locate the next bit in the auxiliary array that is set to one, and adjust head pointer 410 to point to entry 405 in buffer 330 corresponding to the bit in the auxiliary array so identified as being set to one. (Obviously, the significance of the values ​​zero and one may be interchanged without loss of applicability).

[0066] Although using a memory address array is one way to implement the buffer 330, embodiments of the present disclosure may also use other implementations. For example, the buffer 330 may be implemented as a linked list, where the head pointer 410 points to the first entry 405 in the list, the tail pointer 415 points to the last entry 405 in the list, and each entry 405 in the list (except the last entry 405) points to its successor entry 405. The linked list can avoid using modulo operations to determine the address of the next entry, and can also be unbounded (except for the last entry 405). Figure 3 The capacity of the subsystem local memory 325 is outside the capacity of the subsystem local memory 325). When the sending device 120 needs to place a new entry 405 in the linked list, the sending device 120 can allocate a memory block for the new entry 405, place the appropriate data in the new entry 405, and then add the new entry to the linked list (by making the entry 405 pointed to by the tail pointer 415 and the tail pointer 415 itself both point to the new entry 405, and making the new entry 405 include a null pointer for its successor). When the receiving device 120 wants to read data from the entry 405 in the linked list, the receiving device 120 can verify that the entry exists by checking that the head pointer 410 is not a null pointer: if so, the receiving device 120 can read data from the entry 405 pointed to by the head pointer 410, and can then update the head pointer 410 to point to the successor entry 405 of the entry 405 pointed to by the head pointer 410, and deallocate the memory used by the (now processed) entry 405.

[0067] A benefit of a linked list structure (or any queue structure that does not dictate a particular order in which elements are retrieved, such as a modified buffer 330 that uses an auxiliary list) is that multiple devices 120 can consume entries 405 from the buffer 330. That is, one (or more) sending devices 120 can place entries 405 intended for different recipient devices 120 in the buffer 330. The recipient device 120 can then determine which entries 405 in the buffer 330 contain messages or requests for that device, and can remove that entry 405 (or those entries 405) by changing a bit in the auxiliary list. The head pointer 410 can only be updated when the entry 405 pointed to by the head pointer 410 in the buffer 330 is being consumed.

[0068] Although the above description suggests that the auxiliary list can be used only to indicate whether a particular entry 405 in the buffer 330 is awaiting processing, the auxiliary list can also include additional information. Since the auxiliary list can act as metadata, other metadata for the entries 405 in the buffer 330 can also be included in the auxiliary list. For example, instead of including an identifier of the sending device 120 and / or the receiving device 120 in the entry 405 in the buffer 330, such identifiers (and other metadata) can be placed in the auxiliary list instead. Using the auxiliary list in this way can simplify the scanning of the buffer 330 because it is not necessary to directly access the individual entries 405 in the buffer 330 to determine who sent the message or request or who is intended to receive the message or request.

[0069] The above discussion shows that each entry in the buffer 330 has the same size, and each buffer 330 has the same number of entries. In some embodiments of the present disclosure, this condition is true. However, in other embodiments of the present disclosure, this condition can be relaxed. For example, based on the different computing functions provided by the device 120, larger-sized entries may be required, or smaller-sized entries may be sufficient, and different buffers may use entries of different sizes. Further generalized, each entry in a single buffer 330 may have a different size (the size of each entry may be stored in, for example, an auxiliary list so that the receiving device 120 can know how much data to read from the buffer 330). In addition, different buffers 330 may have different numbers of entries, depending on how busy a particular device 120 can be expected to be when processing inter-device communications. For example, a device 120 that is used only as a storage device may be satisfied with a buffer 330 with fewer entries than a device 120 that provides multiple computing functions that can be used in parallel. The size of the entries in the buffer 330 (assuming that each entry in the buffer 330 is the same size) and the number of entries in the buffer 330 can be communicated to other devices that need such information after the buffer 330 is established.

[0070] Figure 5 The use of an embodiment according to the present disclosure is shown Figure 3 The buffer 330 Figure 1 The communication between the two devices 120. Figure 5 In the example, device 120-1 may send message 505-1 to device 120-2. Message 505-1 may be, for example, a request to perform a certain process. Message 505-1 may be placed in request packet structure 510, which may be placed in Figure 3 in buffer 330-2.

[0071] Device 120-2 may also send message 505-2 to device 120-1. Message 505-2 may be placed in a request packet structure ( Figure 5 ), the request packet structure can be placed in Figure 3 330-1 of the device 120-2. Note that in this example, message 505-2 may be a separate request for device 120-1 to perform some processing on behalf of device 120-2. Thus, as shown, two devices may each issue a request to the other device.

[0072] This may lead to the question: does each device 120 block until the other device 120 completes the request made by the other device 120? The answer to this question is no for several reasons. First, there is no reason to think that one device 120 needs a response from another device 120 to perform any substantial processing. For example, as shown below with reference to Figure 8 As discussed, one device 120 may send a request to another device 120 as part of a processing pipeline, and a response is only required to be sent to the device 120 (or Figure 1 Second, the device 120 is able to process multiple requests (from Figure 1 1 or from other devices 120), and thus do not block until responses to messages 505-1 and 505-2 (which may be collectively referred to as messages 505) are received. Thus, the only situation where each device 120 might even partially block is when each device 120 requests that the other device 120 process something related to the same data. That is, device 120-1 needs data A from device 120-2 to process data B, but device 120-2 needs data B from device 120-1 to process data A. This is an unlikely situation, and even if it were, would only affect the processing of the data in question (and the return of that data to the requesting application(s)).

[0073] exist Figure 5 5, assume that device 120-2 first completes processing message 505-1. Device 120-2 may then send response 515-1. Response 515-1 may be placed in response packet structure 520, which may be placed in Figure 3 330-1 for return to device 120-1. Device 120-1 may then finish processing message 505-2, which may then send response 515-2 (which may be placed in Figure 5 In the response packet structure not shown in FIG, the response packet structure can be placed Figure 3 Responses 515-1 and 515-2 may be collectively referred to as responses 515.

[0074] Notice, Figure 5 Only an example of two messages 505 and two responses 515 involving two devices 120 is shown. As described above, embodiments of the present disclosure may include any number of devices 120, each of which may send any number of messages 505. Figure 5 It is suggested that each message 505 may include a corresponding response 515, and this implementation has its benefits (for example, if response 515 is not sent, there will be no concern as to whether a particular request was completed), but embodiments of the present disclosure may send response 515 only where any data or information needs to be returned to requesting device 120: if no such data or information needs to be returned, response 515 may be omitted.

[0075] One point that has not been discussed is how the device 120 can be alerted that the message 505 or response 515 has been placed on the Figure 3 In some embodiments of the present disclosure, device 120 may periodically check the buffer 330 used by device 120 to receive message 505 or response 515. Figure 3 buffer 330. If the message 505 or response 515 is Figure 3 If a message 505 or response 515 is waiting in one of the buffers 330, the device 120 can dequeue the message 505 or response 515 to process it. Hardware interrupts or doorbells can also be used to minimize Figure 3 The buffer 330 is polled to check for new messages.

[0076] Since each device 120 may have more than one buffer 330 for receiving communications, the device 120 may use any desired strategy to determine which buffer 330 a communication should be dequeued from. For example, the device 120 may use a round-robin scheduling strategy to access each buffer 330 in a particular order and dequeue the next communication from that buffer 330. Alternatively, the device 120 may check each buffer 330, determine which buffer 330 has the oldest pending communication, and dequeue the communication from the appropriate buffer 330. Embodiments of the present disclosure may also use any other desired mechanism to select a buffer 330 for dequeuing a communication.

[0077] Figure 6 The invention shows the establishment of an embodiment according to the present disclosure Figure 3 The buffer 330 is used for communication between devices. Figure 1 The processor 110 and Figure 1 Device 120. Figure 6 In the establishment Figure 3At operation 605 of the buffer 330 of the device 120, the device 120 is identified. The device 120 can be identified by the device 120 announcing its presence, or by a component broadcasting a discovery message, requiring all devices 120 to respond. Once all devices are identified, at operation 610, a discovery message can be established. Figure 3 As described above, in some embodiments of the present disclosure, a buffer 330 is established for each pair of devices 120. Figure 3 Two buffers 330 (so that each device can send communications to the other device); in other embodiments of the present disclosure, in which the device 120 can share Figure 3 In the case where there are more buffers 330 for sending or receiving communications, fewer buffers 330 may be established. Operation 610 may include, for example, Figure 3 The buffer 330 and Figure 3 Each buffer 330 Figure 4 The head pointer 410 and Figure 4 The tail pointer 415 allocates one, some or all of the devices 120 Figure 1 Memory 115 or Figure 3 The SLM 325 space (and initialization Figure 3 Each buffer 330 Figure 4 The head pointer 410 and Figure 4 Finally, at operation 615, the device 120 may be notified of the Figure 3 Buffer 330: Device 120 can use Figure 3 Which buffers 330 to use to send communications to other devices 120 (and to which other devices 120 to send communications), and the device 120 can use Figure 3 Operation 615 may include, for example, notifying each device 120 of which buffers 330 to receive communications from other devices 120 (and from which other devices 120). Figure 3 The base address of the buffer 330 and / or Figure 4 The head pointer 410 and Figure 4 The address of the tail pointer 415. (Note that Figure 3 The buffer 330 Figure 4 The size of the entry 405 and Figure 4 The number of entries 405 is known to the applicable device 120 in embodiments of the present disclosure, Figure 4 Head pointer 410, Figure 4 The address of tail pointer 415 may be used by device 120 Figure 3 The buffer 330 has enough information because Figure 4The address of entry 405 of can be determined based on this information). The notification can be performed, for example, by using a base address register (BAR), a controller memory buffer (CMB), or a persistent memory region (PMR), among other possibilities.

[0078] Careful readers will note that the above discussion is agnostic as to which element(s) perform operations 605, 610, and 615. This choice is intentional, as different implementations may have different elements that perform these operations. For example, in some embodiments of the present disclosure, processor 110 may be responsible for performing operations 605, 610, and 615: that is, processor 110 may be responsible for identifying device 120, establishing Figure 3 The buffer 330 and notifies the device 120 Figure 3 In other embodiments of the present disclosure, the processor 110 may be responsible for performing operation 605, and then notify the device 120 to perform operations 610 and 615 as needed. In other embodiments of the present disclosure, the device 120 may be responsible for performing operations 605, 610, and 615 independently (but operations 610 and 615 may be limited to Figure 3 For example, each device 120 may be responsible only for establishing a specific buffer 330 for each device 120 to receive communications from other devices 120. Figure 3 In other embodiments of the present disclosure, one device 120 may be responsible for all operations 605, 610, and 615. Figure 3 SLM 325 builds on Figure 3 All buffers 330 (i.e., one device 120 stores Figure 3 Embodiments of the present disclosure are intended to cover all such variations in which elements perform operations 605, 610, and 615 individually.

[0079] Figure 7 FIG. 1 shows a method of communicating through a switch according to an embodiment of the present disclosure. Figure 1 Device 120. Figure 7 , devices 120-1 and 120-2 may be connected to each other and to machine 105 via switch 705. In such an embodiment, device 120 may be remote from machine 105 (in the sense that device 120 is not physically within machine 105). Device 120 may be part of the same local space as machine 105 (e.g., in the same server room as machine 105, or even within the same rack as machine 105), or device 120 may be physically (geographically) remote from machine 105. Switch 705 may be any kind of switch: e.g., a PCIe switch, a CXL switch, an Ethernet switch, etc. All that is required is that switch 705 supports device 120 access to the network. Figure 3 The buffer 330 may be Figure 1 memory 115 or other device Figure 3 SLM 325.

[0080] Note that device 120-1 may also include inter-process communication (IPC) 710-1, and device 120-2 may include IPC 710-2. (IPCs 710-1 and 710-2 may be collectively referred to as IPC 710). IPC 710 may manage communications with other devices 120: using Figure 3 The buffer 330 sends communication to other devices 120, writing data to other devices Figure 3 SLM 325 (making data available for processing), etc.

[0081] Figure 8 It shows how IPC 710 may function (as part of device 120), particularly when implementing a pipeline for data processing. Figure 8 The communication according to an embodiment of the present disclosure is shown to create a processing pipeline Figure 1 Device 120. Figure 8 middle, Figure 1 Processor 110 may issue a request to device 120-1 via an appropriate submission queue to begin processing data. Figure 1 The request to the processor 110 may include, among other data, the address and size of the data to be processed, the function to be performed when processing the data in the pipeline, and the address where any results of the processing may be stored (e.g., Figure 1 In some embodiments of the present disclosure, Figure 1 The processor 110 may specify which devices 120 are to perform various functions to be performed when processing data in the pipeline; in other embodiments of the present disclosure, Figure 1 The processor 110 may only specify the functions to be performed, and leave it to the device 120-1 to determine which other devices 120 can provide the functions required in performing the functions of the pipeline. In some embodiments of the present disclosure, Figure 1 The processor 110 may allocate space in the computational memory 325 of each device 120 in the pipeline where data may be stored (as the data traverses the pipeline); in other embodiments of the present disclosure, Figure 1 The processor 110 may leave it to the devices 120 to allocate space in the compute memory 325 of each device 120 as needed to deliver data along the pipeline.

[0082] Once the device 120-1 has received the request from the processor 110, the device 120-1 may load the data 805-1 from the persistent storage 810-1 into the compute memory 325-1 (e.g., SLM 325-1) (as shown in operation 1). Once the data 805-1 has been loaded into the compute memory 325-1, the device 120-1 may perform a function 815-1 on the data 805-1 (as shown in operation 2). For example, the function 815-1 may be to filter the data 805-1 according to some criteria, allowing only the data that passes the filter to continue along the pipeline. The result of performing the function 815-1 on the data 805-1 may be written as data 805-2 into the compute memory 325-2 of the device 120-1 (as shown in operation 3). The IPC 710-1 of the device 120-1 may then notify the IPC 710-2 of the device 120-2 to begin performing the function 815-2 on the data 805-2 (as shown in operation 4). The communication can be e.g. Figure 5 The device 120-2 may then perform a function 815-2 on the data 805-2 (as shown in operation 5). For example, the function 815-2 may be to encrypt the data according to some encryption protocol. The result of performing the function 815-2 on the data 805-2 may then be written somewhere else (as shown in operation 6): the result may be written, for example, to the computation memory 325 of another device 120 for further execution of the pipeline, or the result may be written to a Figure 1 The location specified by processor 110 in its original request to device 120 - 1 .

[0083] Now, assume that device 120-2 is the last device 120 to process data in the pipeline, and after writing the data (as shown in operation 6), device 120-2 may Figure 5 The result 515 is sent back to device 120-1 (as shown in operation 7). At this point, device 120-2 has completed its operation.

[0084] Device 120-1 receives the Figure 5 The result is 515, now we can know that Figure 1 The processing of the request issued by processor 110 to device 120-1 is now complete. Device 120-1 may then send Figure 1 The processor 110 sends a response to notify Figure 1 The processor 110 completes the processing and the data exists in the specified location.

[0085] Note that device 120-2 is also shown as including persistent storage 810-2. In some embodiments of the present disclosure, device 120-2 may be a computational storage unit that is part of a storage device: the storage device itself may be represented as persistent storage 810-2. However, in other embodiments of the present disclosure, device 120-2 may include only processing capabilities and no persistent storage, which is why persistent storage 810-2 is shown with dashed lines. (In some embodiments of the present disclosure, device 120-1 may not include persistent storage 810-1, but may read data 805-1 from some other source).

[0086] Although Figure 8 A pipeline of only two devices 120 is shown, but embodiments of the present disclosure may include pipelines of any desired length. For example, a pipeline may include 50 devices, each performing a variety of tasks. In addition, devices 120 may be used multiple times along the pipeline. For example, one device 120-1 may include two functions 815 that may be used at different points in the pipeline. Device 120-1 may perform a first function and deliver the resulting data to the next device, ultimately receiving the data that has been further processed, and performing a second function on the data at that point. (Device 120-1 may be used more than twice: a device 120-1 including two such functions 815 used in a pipeline is merely an example). Data may be moved between devices 120 as needed to support processing by the functions used in the pipeline.

[0087] In some embodiments, the last device 120 in the pipeline may send a response back to Figure 1 The processor 110 notifies Figure 1 The processing of data in the processor 110 pipeline has been completed. However, in some embodiments, such a response must come from Figure 1 The processor 110 of the device 120 to which the request is issued. For example, the NVMe protocol specifies that devices have their completion queues and will need to be sent by the slave Figure 1 The processor 110 of the device 120 that receives the request has a completion queue to send a response. In this case, the device 120-1 (which receives the request from Figure 1 processor 110 receives the original request) may wait until it receives the Figure 5 The response is then sent back to Figure 1 The processor 110 of FIG.

[0088] However, if the pipeline is long, it may take some time for response 515 to percolate back through the pipeline to allow device 120-1 to notify Figure 1The processor 110 completes the processing of the data. In order to speed up the transmission from the device 120-1 to Figure 1 In response to the processor 110, the last device 120 in the pipeline can be connected via Figure 3 A suitable buffer 330 will Figure 5 The message 505 is sent to the device 120-1, notifying the device 120-1 that the processing of the data in the pipeline is completed. Figure 5 The message 505 can then be sent to Figure 1 The processor 110 sends a response without having to wait Figure 5 All responses 515 permeate through the pipeline (and in particular are received from device 120-2 Figure 5 In this way, device 120-1 can more quickly send a response to the appropriate completion queue. Figure 1 The processor 110 sends a response.

[0089] Since each device 120 may be responsible for delivering processed data to the next device 120 in the pipeline, each device 120 may need to know both what function it will perform and what device 120 (and the function 815 of that device 120) will process next for that device. Thus, the communications between the IPCs 710 may include information about where the processing is in the pipeline and what processing remains in the pipeline, so that the next device 120 can not only perform its function, but also inform the device 120 after it what function 815 that device will perform. (Of course, each device 120 can remove the processing it has already performed from the communication, so that devices 120 further down the pipeline only see the functions 815 that they or other later devices 120 need to perform, rather than any functions 815 that have already been performed).

[0090] As described above, in some embodiments of the present disclosure, each device 120 may be left to allocate space in the computation memory 325 of the next device 120. In such embodiments of the present disclosure, allocating space in the computation memory 325 of the next device 120 may be performed before executing the function 815 on the data 805 (so that there is a location to write the result of executing the function 815).

[0091] exist Figure 8 In the example of FIG. 1 , each device is shown as writing the result of its function 815 to the computing memory 325 of the next device 120. In some embodiments of the present disclosure, this may be considered advantageous: one device 120 may access data 805 from its own computing memory 325 more quickly than from the computing memory 325 of another device 120 (or from the computing memory 325 of another device 120). Figure 1However, in other embodiments of the present disclosure, data 805 may be stored in any desired location, even if not in the computational memory 325 of the device 120 that is used to execute the next function 815 in the pipeline: each device 120 may access data 805 from anywhere that data 805 may be stored.

[0092] exist Figure 8 , it is implied that each function 815 completely processes data 805 and writes the resulting data 805 to the appropriate location before the IPC 710 triggers the next device 120 to begin executing its function 815. However, in some embodiments of the present disclosure, the data 805 to be processed may be large: large enough that each device 120 waits until the previous device 120 has completely processed all of the data 805 before beginning its own processing, so that the inherent delay may become significant. Therefore, in some embodiments of the present disclosure, each device 120 may stream data 805 to the next device 120 as the data 805 is processed, rather than waiting for all of the data 805 to be processed before triggering the next device 120 to begin executing its function 815. For example, device 120-1 may process a portion of data 805-1 (e.g., four kilobytes (KB)) and write the result as data 805-2 to the computational memory 325 of device 120-2. IPC 710-1 may then send to device 120-2 Figure 5 505 to start processing data 805-2. Meanwhile, function 815-1 may continue to process another data block 805-1: once the data block 805-1 is processed, the result of processing the data block 805-1 (i.e., the other data block 805-2) may be written to the computing memory 325 of device 120-2, and IPC 710-1 may send another data block 805-2 to device 120-2. Figure 5 message 505 to continue processing data 805-2. In such an embodiment of the present disclosure, Figure 5 The message 505 may include a flag indicating whether further data 805-2 may be written to the computing memory 325 of the device 120-2. If the flag is set, the device 120-2 may know that additional data 805-2 may be forthcoming; if the flag is not set, the device 120-2 may know that this is the last of the data 805-2 to be processed. Note that the flag may only need to distinguish between two cases, and thus may only need to be one bit in size.

[0093] Figure 8One aspect of the present invention that has not been discussed is orchestrators 820-1 and 820-2 (which may be collectively referred to as orchestrators 820). While IPC 710 may issue communications to other devices 120, orchestrator 820 may identify which device 120 IPC 710 should communicate with. For example, orchestrator 820 may examine what processing is yet to be performed in the pipeline and may identify which device 120 may include the next function 815 to be performed on data 805. Knowing what device 120 may be used next in the pipeline, orchestrator 820 may inform IPC 710 which device it should communicate with next. Orchestrator 820 may also be responsible for determining which device 120 should use IPC 710 to receive data from the current device 120. Figure 5 Response 515.

[0094] In some embodiments of the present disclosure, each device 120 may be responsible for determining what function 815 it must perform and what device 120 is next in the pipeline. However, in other embodiments of the present disclosure, a single device 120 may be responsible for orchestrating the entire pipeline. For example, in Figure 1 When the processor 110 receives the request, the device 120-1 may perform its function 815-1 on the data 805-1 and then Figure 5 Device 120-2 may then send a message 505 to device 120-2 to perform its function 815-2 on data 805-2 (using orchestrator 820-1 and IPC 710-1). Device 120-2 may then send a message 505 to device 120-1 to perform its function 815-2 on data 805-2 (using orchestrator 820-1 and IPC 710-1). Figure 5 Response 515, after which device 120-2 may send a Figure 5 message 505 to perform its function 815 on data 805 (again, using orchestrator 820-1 and IPC 710-1). This process can be repeated as needed until the pipeline is complete, with orchestrator 820-1 taking turns managing notifications for each device 120 to perform its portion of the pipeline.

[0095] In some embodiments of the present disclosure, Figure 1 Processor 110 may issue a separate request for each operation that device 120-1 is expected to perform. For example, rather than a single request instructing device 120-1 to load data 805-1 from persistent storage 810-1 into compute memory 325-1 and then process data 805-1 along the pipeline, Figure 1 Processor 110 may issue one request to cause device 120-1 to load data 805-1 from persistent storage 810-1 into compute memory 325, and then issue a separate request to begin processing data 805-1 in the pipeline.

[0096] As described above, in some embodiments of the present disclosure, Figure 1 The processor 110 of the pipeline can simply specify which functions 815 will be executed in the pipeline, rather than which devices 120 can perform such functions 815. In such an embodiment of the present disclosure, the orchestrator 820-1 can determine which device 120 can perform the next function 815 in the pipeline (and possibly other functions 815 in the pipeline thereafter, so that all devices 120 thereafter know what device 120 will perform what function 815 in the pipeline). Each device 120 can advertise what functions 815 it provides so that other devices 120 can know this information. The orchestrator 820-1 can then use this information to select the device(s) 120 that include the required function(s) in the pipeline to correctly orchestrate which devices 120 will perform which functions 815 when executing the pipeline.

[0097] In addition to using IPC 710 for pipelined execution of commands between devices 120, IPC 710 may also be used for other tasks. These other tasks may include write leveling, distributed garbage collection, memory sharing, compute sharing, discovery, data migration, and autonomic analysis, among other possibilities.

[0098] Write leveling can occur when the drive is currently busy and would otherwise be delayed in completing the write request. For example, consider Figure 1 1 requires device 120-1 to write some data. If device 120-1 is currently involved in a lengthy task (e.g., reading 100 gigabytes (GB) of data), then executing the write request as specified may be significantly delayed until the read request is completed. To avoid delaying the write request, device 120-1 may use IPC 710-1 to send message 505 to device 120-2, forwarding the write request to device 120-2 and specifying the write request as specified by device 120-1. Figure 1 The processor 110 specifies in the original write request Figure 1 Device 120-2 may use direct memory access (DMA) transfer to transfer data from Figure 1 Device 120-2 can then copy the memory 115 of Figure 5 Response 515 is sent back to device 120-1, and device 120-1 may then respond to the original write request, but notify Figure 1 The processor 110 device 120-2 actually stores the data (because Figure 1 processor 110 will expect device 120-1 to store data as the recipient of the original write request).

[0099] If data is moved from one device 120 to another device 120 as part of wear leveling or for any other reason as described below, the device 120 may send Figure 1 The processor 110 issues an asynchronous event notification (AEN) to notify Figure 1 The processor 110 is where the data now resides.

[0100] In some cases, there may be some conflict for storing data on device 120-2. The conflict may include a conflicting logical block address (LBA) (device 120-2 may have stored data associated with the LBA specified in the write request), a namespace conflict (device 120-2 may have stored data associated with the namespace specified in the write request), an endurance group conflict (device 120-2 may not have an endurance group as specified in the write request), or other conflicts. Such conflicts may be resolved in any desired manner, wherein device 120-2 reports to device 120-1 how the conflict was resolved so that device 120-1 may notify processor 110-1 of the conflict and its resolution. For example, in the event of an LBA conflict, device 120-2 may select another LBA that does not conflict with device 120-2 and notify device 120-1 that the new LBA is used instead.

[0101] In some embodiments of the present disclosure, devices 120 may share an analysis (described below) that may include their average wear (i.e., how many program / erase cycles each cell in device 120 has on average). Device 120-1 may then use this information to select device 120-2 that optimizes the overall wear of devices 120.

[0102] In some embodiments of the present disclosure, the devices 120 may share analytics (described below) that may include a ratio of hot / cold data on each device 120. The ratio of hot / cold data may reflect how much data is currently on any individual device that is hot data (data that is accessed or updated relatively frequently) or cold data (data that is accessed or updated relatively infrequently). The devices 120 may attempt to balance their relative hot / cold ratios to ensure that the device loads are balanced. For example, if one device 120 has primarily hot data and another device 120 has primarily cold data, it may be expected that the device 120 that has primarily hot data will be relatively busy, while the device 120 that has primarily cold data will be relatively idle. Balancing their respective ratios of hot and cold data may result in both devices 120 being relatively equally busy, rather than one device 120 being potentially idle and the other device 120 being potentially overloaded.

[0103] Inter-device communication can also be used to coordinate garbage collection across devices. For example, one device 120 can let other devices 120 know that the device 120 is starting garbage collection and that other devices 120 should not start garbage collection at this time (to maximize the availability of the devices 120). In addition, as part of performing garbage collection, one device 120 can perform wear leveling of its data, potentially distributing some of that data to other devices 120. In such a case, the other devices 120 may need to be available to receive write requests from the device 120 performing garbage collection, which the other devices 120 may not be able to do if they start garbage collection themselves. Alternatively, the hot / cold ratio of data on the device performing garbage collection can be considered out of balance and can be rebalanced as part of garbage collection.

[0104] Inter-device communication can also be used for memory sharing. Memory can be abstracted: it is possible for a device 120 to write data to an address without knowing which device 120 hosts the address. Ideally, the address would be somewhere where the data will be used next (e.g., in another device 120 if the data is moving through a processing pipeline, or for a resource to be used by another device 120). Figure 1 The processor 110 accesses the data, then Figure 1 However, there may be situations where the ideal location for storing data may not be able to store the data. For example, if the next device 120 in the pipeline Figure 3 If there is not enough space in the SLM 325, the data may need to be stored elsewhere (e.g., another device 120). The device 120 attempting to store the data may allocate memory elsewhere, write the data to the newly allocated memory, and notify the next device 120 in the pipeline where the data is actually stored so that the next device 120 in the pipeline can read the data from that address.

[0105] Inter-device communications may also be used for discovery purposes. A device 120 may send communications to other devices 120 to let them know the capabilities of the device 120. For example, a device 120 may advertise what computing functions the device 120 provides, or its storage capacity or the amount of free space or its available local memory. Other devices 120, particularly busy devices, may then use this information to pass messages or requests to less busy devices 120.

[0106] Inter-device communication can also be used for computing sharing. As described above, devices 120 can advertise their capabilities: for example, the computing functions they provide. A currently busy device 120 can identify another device 120 that can provide that particular computing function, and can redirect requests to use that computing function to another device 120.

[0107] Inter-device communication can also be used for data migration. For example, consider the case where a die in a device 120 begins to fail. Data can still be read from the die (using error correction techniques to correct errors that may reflect that the die has begun to fail). The data on that die can be migrated to another device 120 so that the data is not lost. (It may also be beneficial to transfer data from other dies on that device 120, as the failure of one die may indicate that the entire device 120 may soon fail). AEN can be used to Figure 1 The processor 110 notifies data migration.

[0108] Inter-device communication may also be used for autonomous analysis. Devices 120 may share information about their states so that data and / or processing functions may be shifted between or among devices 120 to best optimize operation.

[0109] Fig. 9 The embodiment according to the present disclosure is shown Figure 1 120 devices used Figure 3 Flowchart of an example process for communicating with the buffer 330. Fig. 9 In block 905, Figure 1 A device 120 may identify a device to which a communication is to be sent. Figure 1 The second device 120. At block 910, Figure 1 The first device 120 may establish Figure 4 405 of the entry. Figure 4 The entry 405 may include Figure 1 The first device 120 sends to Figure 1 Finally, at block 915, Figure 1 The first device 120 can Figure 4 The entry 405 is placed in Figure 3 The buffer 330 is provided Figure 1 The second device 120 retrieves it later.

[0110] Fig.10 Continuing with the embodiment of the present disclosure, Figure 1 120 devices used Figure 3 The buffer 330 communicates Fig. 9 A flowchart of an example process. Fig.10 In block 1005, Fig. 9 At a point after frame 915 of Figure 1 The second device 120 can be Figure 3 The buffer 330 retrieves Figure 4 405. At block 1010, Figure 1 The second device 120 may then process the Figure 4 The entry 405 is for communication.

[0111] Fig.11 The embodiment according to the present disclosure is shown Figure 1 The device 120 will Figure 4 The entry 405 is placed in Figure 3 A flowchart of an example process in the buffer 330 of FIG. Fig.11 In block 1105, Figure 1 The first device 120 can Figure 4 The entry 405 is placed in Figure 3 The buffer 330 is composed of Figure 4 At the location pointed to by the tail pointer 415 of . At block 1110, Figure 1 The first device 120 may update Figure 4 The tail pointer 415 points to Figure 3 The next position in the buffer 330 (to store Figure 4 Next entry 405).

[0112] Fig.12 The method for Figure 1 Device 120 from Figure 3 The buffer 330 retrieves Figure 4 405 of the example process of FIG. Fig.12 In block 1205, Figure 1 The second device 120 can be obtained from Figure 4 The location pointed to by the head pointer 410 is retrieved Figure 4 405. At block 1210, Figure 1 The second device 120 may update Figure 4 The head pointer 410 points to Figure 3 The next position in the buffer 330 (to retrieve Figure 4 Next entry 405).

[0113] Fig.13 The embodiment according to the present disclosure is shown Figure 1 The device 120 creates a device to be placed Figure 3 The buffer 330 Figure 4 405 of the example process of FIG. Fig.13 In block 1305, Figure 1 The first device 120 may be Figure 4 The grouping structure is established in entry 405, such as Figure 5 The SQE 510 or CQE 520. At block 1310, Figure 1 The first device 120 can Figure 1 The identifier of the first device 120 is stored in Figure 4 405. For example, if Figure 1 The plurality of devices 120 may be Figure 4 The entry 405 is placed in Figure 3 In the buffer 330, the identifier can be used. Block 1310 can be omitted, as shown by dashed line 1315. At block 1320, Figure 1 The first device 120 can Figure 1 The identifier of the second device 120 is stored in Figure 4 405. For example, if Figure 1 The plurality of devices 120 may be from Figure 3 The buffer 330 retrieves Figure 4 The identifier may be used if entry 405 is present. Block 1320 may be omitted, as indicated by dashed line 1325.

[0114] Fig.14 The method for Figure 1 120 identification of the device Figure 3 The buffer 330 Figure 4 405 of the example process flow chart, wherein Figure 3 Device 120 is the intended recipient of entry 405. Fig.14 In block 1405, Figure 1 The second device 120 may check Figure 4 The entry 405 is to determine Figure 4 Does the entry 405 include Figure 1 The identifier of the second device 120. Figure 1 This identifier of the second device 120 may indicate Figure 4 The entry 405 is intended for Figure 1 The second device 120 (rather than for Figure 1 Also available from Figure 3 The buffer 330 retrieves Figure 4 some other device 120 of entry 405).

[0115] Fig.15 The embodiment according to the present disclosure is shown Figure 1 The device 120 will Figure 4 The entry 405 is placed in Figure 3 Flowchart of an example process of buffer 330, the entry 405 can be made by Figure 1 More than one device 120 reads or writes. Fig.15 In block 1505, Figure 1 The device 120 can obtain Figure 3 The buffer 330 is locked. At block 1510, Figure 1 The device 120 can be obtained from Figure 3 Buffer 330 accesses (i.e., reads or writes as appropriate) Figure 4 405. Finally, at block 1515, Figure 1 The device 120 can release Figure 3 The buffer 330 is locked so that Figure 1 Another device 120 can access Figure 3 Buffer 330.

[0116] Fig.16 The embodiment according to the present disclosure is shown Figure 1 The processor 110 or Figure 1 120 devices are established Figure 3 A flowchart of an example process of the buffer 330. Fig.16 In block 1605, identify Figure 1 At block 1610, a Figure 3 buffer 330. Finally, Fig.13 At block 1320, you can Figure 1 120 notifications of the device and Figure 3 Device 330 related Figure 3 Buffer 330.

[0117] exist Figure 9-16 In the flowchart, some embodiments of the present disclosure are shown. However, those skilled in the art will recognize that other embodiments of the present disclosure are possible by changing the order of the blocks, by omitting blocks, or by including links not shown in the drawings. All such variations of the flowchart are considered embodiments of the present disclosure, whether or not explicitly described.

[0118] Some embodiments of the present disclosure may include a device-accessible buffer. One or more devices may write to the buffer, and one or more devices may read from the buffer. The technical advantage provided by including the buffer is that devices may communicate with each other without having to use a host processor as an intermediary.

[0119] Embodiments of the present disclosure include methods for allowing drivers, such as storage devices, and more specifically non-volatile memory express (NVMe) storage devices, to send commands / responses to each other. A host having information about all the drivers in a system can configure a ring buffer in a device peripheral component interconnect express (PCIe), such as a controller memory buffer (CMB), a persistent memory region (PMR), or other memory accessible through a base address register (BAR) or other memory accessible to the drivers. The ring buffer can use PCIe, a cache coherent interconnect protocol such as compute express link protocol), Remote Direct Memory Access (RDMA), or (Compute Express Link and CXL are registered trademarks of Compute Express Link Consortium, Inc., and NVLink is a registered trademark of NVIDIA Corporation.) The host can notify each driver of the memory to use for each ring buffer, and can notify the peer driver of the address where a message can be written to send to another driver. The ring buffer can use data structures such as first-in-first-out (FIFO) queues, circular queues, head / tail pointers, and / or buffers.

[0120] The peer drive may be identified using its NVMe Qualified Name (NQN).

[0121] For each pair of drives, there can be two ring buffers: one for each drive to write to. In this way, write conflicts can be avoided: each drive can write to a separate ring buffer (and each drive can read from a ring buffer written by the other drive). Thus, for a system containing n drives, the total number of ring buffers may be That is to say, there is For drivers, there are two ring buffers per pair of drivers. Each driver can read from the head of its ring buffer and can write to the tail of its peer's ring buffer. If the head and tail pointers of any ring buffer are the same, then the ring buffer is empty; if the tail pointer identifies the entry of the ring buffer after the head pointer (mathematically, if (Head+1)%Max=Tail), then the ring buffer is full. To avoid critical sections, the sender (driver writing to the ring buffer) can have the head pointer, and the receiver (driver reading from the ring buffer) can have the tail pointer. This structure can support communication using messages similar to the Submit Queue Entry / Complete Queue Entry grouping.

[0122] Embodiments of the present disclosure may include protocols for drive configuration and / or reset / shutdown.

[0123] In one embodiment of the present disclosure, computational storage drivers (CSDs) may perform portions of a processing task collaboratively. A first CSD may process data and write the results to a PCIe-accessible memory of a second CSD. Once the write is complete, the first CSD may use inter-driver communication to notify the second CSD with a message that the data is present in its memory and that the second CSD may be processing the data. Once the second CSD completes processing the data, the second CSD may send a message to the first CSD to complete the computational procedure and notify the host of the data availability (in the second CSD's memory space).

[0124] Inter-drive communication can also be used for other purposes, such as:

[0125] * Write leveling: For a drive selected by the host, writes may be blocked and may be redirected to another drive (using the inter-drive communication path) where completion identifies that the other drive completed the command. Similarly, if the drives communicate their wear increments, writes may be redirected to a less wearable drive.

[0126] * Distributed garbage collection: Drivers can communicate to employ a distributed garbage collection strategy.

[0127] * Memory Sharing: Drives can temporarily share their PCIe-accessible memory using inter-drive messages to allocate and deallocate space.

[0128] * Compute sharing: Drivers can use inter-driver messages to utilize features in other drivers to request data processing.

[0129] * Discovery: A driver can advertise its capabilities (memory and / or features) to peer drivers for their use (enabling memory and compute sharing).

[0130] * Data Migration: Drives can migrate data to other drives (for performance or protection) with subsequent host notification.

[0131] *Autonomous analysis: The drive can share its monitoring data for predictive analysis and failure prediction. This data can also be used for write leveling (autonomous data balancing).

[0132] The following discussion is intended to provide a brief general description of one or more suitable machines in which certain aspects of the present disclosure may be implemented. One or more machines may be controlled at least in part by input from a traditional input device (e.g., a keyboard, a mouse, etc.) and by instructions received from another machine, interaction with a virtual reality (VR) environment, biometric feedback, or other input signals. As used herein, the term "machine" is intended to broadly cover a single machine, a virtual machine, or a system of communicatively coupled machines, virtual machines, or devices operating together. Exemplary machines include computing devices such as personal computers, workstations, servers, portable computers, handheld devices, phones, tablets, etc., and transportation equipment such as private or public transportation, such as cars, trains, taxis, etc.

[0133] One or more machines may include embedded controllers, such as programmable or non-programmable logic devices or arrays, application specific integrated circuits (ASICs), embedded computers, smart cards, etc. One or more machines may utilize one or more connections to one or more remote machines, such as through a network interface, modem, or other communication coupling. The machines may be interconnected by physical and / or logical networks, such as an intranet, the Internet, a local area network, a wide area network, etc. Those skilled in the art will appreciate that network communications may utilize a variety of wired and / or wireless short-range or long-range carriers and protocols, including radio frequency (RF), satellite, microwave, Institute of Electrical and Electronics Engineers (IEEE) 802.11, Optical, infrared, cable, laser, etc.

[0134] Embodiments of the present disclosure may be described by reference or in conjunction with associated data including functions, procedures, data structures, applications, etc., which, when accessed by a machine, cause the machine to perform a task or define an abstract data type or low-level hardware context. The associated data may be stored, for example, in volatile and / or non-volatile memory, such as RAM, ROM, etc., or in other storage devices and their associated storage media, including hard drives, floppy disks, optical storage, magnetic tapes, flash memory, memory sticks, digital video disks, bio-storage, etc. The associated data may be transmitted in the form of packets, serial data, parallel data, propagated signals, etc. in a transmission environment, including a physical and / or logical network, and may be used in a compressed or encrypted format. The associated data may be used in a distributed environment and stored locally and / or remotely for machine access.

[0135] Embodiments of the present disclosure may include a tangible, non-transitory machine-readable medium including instructions executable by one or more processors, the instructions including instructions for performing elements of the present disclosure as described herein.

[0136] The various operations of the methods described above may be performed by any suitable unit capable of performing these operations, such as various (multiple) hardware and / or software components, circuits and / or (multiple) modules. Software may include an ordered list of executable instructions for implementing logical functions, and may be embodied in any "processor-readable medium" for use by or in conjunction with an instruction execution system, device or apparatus (such as a single-core or multi-core processor or a system containing a processor).

[0137] The blocks or steps of the methods or algorithms and functions described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. If implemented in software, the functions may be stored on or transmitted via a tangible, non-transitory computer-readable medium. The software module may reside in a random access memory (RAM), a flash memory, a read-only memory (ROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a register, a hard disk, a removable disk, a CD ROM, or any other form of storage medium known in the art.

[0138] Having described and illustrated the principles of the present disclosure with reference to the illustrated embodiments, it will be appreciated that the illustrated embodiments may be modified in arrangement and detail without departing from these principles, and may be combined in any desired manner. Furthermore, although the foregoing discussion has focused on specific embodiments, other configurations are contemplated. In particular, even when expressions such as "according to embodiments of the present disclosure" are used herein, these phrases are intended to refer generally to embodiment possibilities, and are not intended to limit the present disclosure to specific embodiment configurations. As used herein, these terms may reference the same or different embodiments that may be combined into other embodiments.

[0139] The foregoing illustrative embodiments should not be construed as limiting the disclosure thereof. Although some embodiments have been described, it will be readily appreciated by those skilled in the art that many modifications may be made to those embodiments without substantially departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure as defined in the claims.

[0140] The embodiments of the present disclosure may be extended to the following statements, but are not limited thereto:

[0141] Statement 1. An embodiment of the present disclosure includes a system comprising:

[0142] First device;

[0143] Second device;

[0144] a processor configured to communicate with the first device and the second device; and

[0145] A buffer is provided for storing entries comprising communications from a first device to a second device.

[0146] Statement 2. An embodiment of the present disclosure includes a system according to Statement 1, wherein the first device includes:

[0147] a first interface for communicating with a second device; and

[0148] A second interface for communicating with the processor.

[0149] Statement 3. An embodiment of the present disclosure includes a system according to Statement 1, wherein the system includes:

[0150] a first communication path between the first storage device and the second device; and

[0151] A second communication path between the first storage device and the processor.

[0152] Statement 4. Embodiments of the present disclosure include a system according to Statement 1, wherein the first device includes a storage device or a computational storage unit.

[0153] Statement 5. Embodiments of the present disclosure include a system according to Statement 1, wherein the second device comprises a storage device or a computational storage unit.

[0154] Statement 6. An embodiment of the present disclosure includes a system according to Statement 1, wherein the buffer comprises a ring buffer.

[0155] Statement 7. An embodiment of the present disclosure includes a system according to Statement 6, wherein the ring buffer includes:

[0156] the head pointer; and

[0157] Tail pointer.

[0158] Statement 8. An embodiment of the present disclosure includes a system according to Statement 7, wherein:

[0159] The first device is configured to write an entry into the ring buffer based at least in part on the tail pointer; and

[0160] The second device is configured to read an entry from the ring buffer based at least in part on the head pointer.

[0161] Statement 9. An embodiment of the present disclosure includes a system according to Statement 1, wherein the buffer is further configured to store a second entry, the second entry comprising a second communication from the second device to the first device.

[0162] Statement 10. An embodiment of the present disclosure comprises a system according to Statement 9, wherein:

[0163] The entry includes a first identifier of the first device; and

[0164] The second entry includes a second identifier of the second device.

[0165] Statement 11. An embodiment of the present disclosure includes a system according to Statement 9, wherein the buffer includes a locking mechanism to prevent a first device from writing an entry and a second device from simultaneously writing a second entry to the buffer.

[0166] Statement 12. An embodiment of the present disclosure includes a system according to Statement 1, further comprising a memory coupled to the processor, the memory of the system including a buffer.

[0167] Statement 13. An embodiment of the present disclosure includes a system according to Statement 1, wherein the first device includes a memory, and the memory of the first device includes a buffer.

[0168] Statement 14. An embodiment of the present disclosure includes a system according to Statement 1, wherein the second device includes a memory, and the memory of the second device includes a buffer.

[0169] Statement 15. An embodiment of the present disclosure comprises a system according to Statement 1, further comprising a third device.

[0170] Statement 16. An embodiment of the present disclosure includes a system according to Statement 15, wherein the third device includes a memory, and the memory of the third device includes a buffer.

[0171] Statement 17. An embodiment of the present disclosure includes a system according to Statement 16, wherein the buffer is configured to store a second entry including a second communication from the third device to the second device.

[0172] Statement 18. An embodiment of the present disclosure comprises a system according to Statement 17, wherein:

[0173] The entry includes a first identifier of the first device; and

[0174] The second entry includes a second identifier of the third device.

[0175] Statement 19. An embodiment of the present disclosure includes a system according to Statement 17, wherein the buffer includes a locking mechanism to prevent a first device from writing an entry and a third device from simultaneously writing a second entry to the buffer.

[0176] Statement 20. An embodiment of the present disclosure comprises a system according to Statement 1, further comprising a second buffer for storing a second entry comprising a second communication from the second device to the first device.

[0177] Statement 21. An embodiment of the present disclosure comprises a system according to Statement 20, wherein the second buffer comprises a ring buffer.

[0178] Statement 22. An embodiment of the present disclosure includes a system according to Statement 21, wherein the ring buffer includes:

[0179] The head pointer; and

[0180] Tail pointer.

[0181] Statement 23. An embodiment of the present disclosure comprises a system according to Statement 22, wherein:

[0182] The second device is configured to write an entry into the ring buffer based at least in part on the tail pointer; and

[0183] The first device is configured to read an entry from the ring buffer based at least in part on the head pointer.

[0184] Statement 24. An embodiment of the present disclosure includes a system according to Statement 20, further comprising a memory coupled to the processor, the memory of the system including a second buffer.

[0185] Statement 25. An embodiment of the present disclosure includes a system according to Statement 20, wherein the first device includes a memory, and the memory of the first device includes the second buffer.

[0186] Statement 26. An embodiment of the present disclosure includes a system according to Statement 20, wherein the second device includes a memory, and the memory of the second device includes the second buffer.

[0187] Statement 27. An embodiment of the present disclosure comprises a system according to Statement 20, further comprising a third device, the third device comprising a memory, the memory of the third device comprising a second buffer.

[0188] Statement 28. An embodiment of the present disclosure comprises a system according to Statement 1, wherein the entry comprises a grouping structure.

[0189] Statement 29. An embodiment of the present disclosure comprises a system according to Statement 28, wherein the grouping structure comprises a submitting grouping structure or a completing grouping structure.

[0190] Statement 30. An embodiment of the disclosure includes the system of Statement 1, wherein the processor is configured to establish a buffer in the memory based at least in part on the processor identifying the first device and the second device.

[0191] Statement 31. An embodiment of the present disclosure comprises a system according to Statement 30, wherein the processor is further configured to send a notification to the first device and the second device regarding the buffer in the memory of the system.

[0192] Statement 32. An embodiment of the present disclosure includes a system according to Statement 1, wherein the first device is configured to establish a buffer.

[0193] Statement 33. An embodiment of the present disclosure comprises a system according to Statement 32, wherein the first device is configured to establish a buffer based at least in part on the first device receiving a notification regarding the second device.

[0194] Statement 34. An embodiment of the present disclosure includes a system according to Statement 33, wherein the first device is configured to establish a buffer based at least in part on the processor sending a notification about the second device to the first device, wherein the processor sends the notification about the second device to the first device based at least in part on the processor identifying the second device.

[0195] Statement 35. An embodiment of the present disclosure comprises a system according to Statement 33, wherein the first device is configured to establish a buffer based at least in part on the first device receiving a notification from the second device.

[0196] Statement 36. An embodiment of the present disclosure comprises a system according to Statement 32, wherein the first device is further configured to send a notification about the buffer to the second device.

[0197] Statement 37. An embodiment of the present disclosure includes a system according to Statement 1, wherein the second device is configured to establish a buffer.

[0198] Statement 38. An embodiment of the present disclosure comprises a system according to Statement 37, wherein the second device is configured to establish a buffer based at least in part on the second device receiving a notification regarding the first device.

[0199] Statement 39. An embodiment of the present disclosure includes a system according to Statement 38, wherein the second device is configured to establish a buffer based at least in part on the processor sending a notification about the first device to the second device, wherein the processor sends the notification about the first device to the second device based at least in part on the processor identifying the first device.

[0200] Statement 40. An embodiment of the present disclosure comprises the system of Statement 38, wherein the second device is configured to establish the buffer based at least in part on the second device receiving the notification from the first device.

[0201] Statement 41. An embodiment of the present disclosure includes a system according to Statement 37, wherein the second device is further configured to send a notification about the buffer to the first device.

[0202] Statement 42. An embodiment of the present disclosure includes a system according to Statement 1, wherein:

[0203] The system further includes a third device; and

[0204] The third device is configured to establish a buffer.

[0205] Statement 43. An embodiment of the present disclosure comprises a system according to Statement 42, wherein the third device is configured to establish a buffer based at least in part on the third device receiving a notification regarding the first device and the second device.

[0206] Statement 44. An embodiment of the present disclosure comprises a system according to Statement 43, wherein the notification comprises a first notification regarding the first device and a second notification regarding the second device.

[0207] Statement 45. An embodiment of the present disclosure includes a system according to Statement 43, wherein the third device is configured to establish a buffer based at least in part on the processor sending a notification about the first device and the second device to the third device, wherein the processor sends the notification about the first device and the second device to the third device based at least in part on the processor identifying the first device and the second device.

[0208] Statement 46. An embodiment of the present disclosure includes a system according to Statement 43, wherein the third device is configured to establish a buffer based at least in part on the third device receiving notifications from the first device and the second device.

[0209] Statement 47. An embodiment of the present disclosure includes a system according to Statement 42, wherein the third device is further configured to send a notification about the buffer to the first device and the second device.

[0210] Statement 48. An embodiment of the present disclosure includes a system according to Statement 1, wherein the communication includes advertising capabilities of the first device, analysis of the first device, redirection of a write request received from a processor at the first device, garbage collection coordination, execution of a computing function by the second device, or data migration from the first device.

[0211] Statement 49. An embodiment of the present disclosure includes a method comprising:

[0212] identifying, at the first device, a second device to which the communication is to be directed;

[0213] establishing, by the first device, an entry, the entry comprising a communication; and

[0214] placing an entry in a buffer by the first device for delivery to the second device,

[0215] Wherein the second device retrieves the entry from the buffer.

[0216] Statement 50. Embodiments of the present disclosure include a method according to Statement 49, wherein the first device comprises:

[0217] a first interface for communicating with a second device; and

[0218] A second interface for communicating with the processor.

[0219] Statement 51. Embodiments of the present disclosure include a method according to Statement 49, wherein:

[0220] A first communication path enables communication between the first storage device and the second device; and

[0221] The second communication path enables communication between the first storage device and the processor.

[0222] Statement 52. Embodiments of the present disclosure include a method according to Statement 49, wherein the first device comprises a storage device or a computational storage unit.

[0223] Statement 53. Embodiments of the present disclosure include a method according to Statement 49, wherein the second device comprises a storage device or a computational storage unit.

[0224] Statement 54. An embodiment of the present disclosure includes a method according to Statement 49, wherein the first device and the second device are in communication with a processor.

[0225] Statement 55. An embodiment of the present disclosure comprises a method according to Statement 49, wherein the buffer comprises a ring buffer.

[0226] Statement 56. An embodiment of the present disclosure includes a method according to Statement 55, wherein the ring buffer comprises:

[0227] the head pointer; and

[0228] Tail pointer.

[0229] Statement 57. An embodiment of the present disclosure includes a method according to Statement 56, wherein placing, by the first device, the entry in the buffer for delivery to the second device comprises:

[0230] placing, by the first device, an entry in the buffer at a location identified by the tail pointer; and

[0231] Update the tail pointer.

[0232] Statement 58. An embodiment of the present disclosure includes a method according to Statement 56, further comprising:

[0233] retrieving, by the second device, an entry from the buffer based on a second location identified by the head pointer; and

[0234] Update the head pointer.

[0235] Statement 59. An embodiment of the disclosure includes the method of Statement 49, further comprising retrieving, by the first device, a second entry from the buffer, the second entry comprising a second communication from the second device to the first device.

[0236] Statement 60. Embodiments of the present disclosure include a method according to Statement 59, wherein:

[0237] Creating, by the first device, the entry includes storing a first identifier of the first device in the entry; and

[0238] Retrieving, by the first device, the second entry from the buffer includes reading a second identifier of the second device from the second entry.

[0239] Statement 61. An embodiment of the present disclosure includes a method according to Statement 59, wherein placing, by the first device, the entry in the buffer for delivery to the second device comprises:

[0240] locking a buffer for the first device; and

[0241] Unlock the buffer for the first device.

[0242] Statement 62. An embodiment of the present disclosure includes a method according to Statement 49, wherein placing, by the first device, the entry in a buffer for delivery to the second device comprises: placing, by the first device, the entry in a buffer of a memory coupled to the processor for delivery to the second device.

[0243] Statement 63. An embodiment of the present disclosure includes a method according to Statement 49, wherein placing, by the first device, the entry in a buffer for delivery to the second device comprises: placing, by the first device, the entry in a buffer of a memory of the first device for delivery to the second device.

[0244] Statement 64. An embodiment of the present disclosure includes a method according to Statement 49, wherein placing, by the first device, the entry in a buffer for delivery to the second device comprises: placing, by the first device, the entry in a buffer of a memory of the second device for delivery to the second device.

[0245] Statement 65. An embodiment of the present disclosure includes a method according to Statement 49, wherein placing, by the first device, the entry in a buffer for delivery to the second device comprises placing, by the first device, the entry in a buffer of a memory of a third device for delivery to the second device.

[0246] Statement 66. An embodiment of the present disclosure comprises a method according to Statement 49, further comprising retrieving, by the first device, a response from the second device.

[0247] Statement 67. An embodiment of the present disclosure comprises a method according to Statement 66, wherein retrieving, by the first device, the response from the second device comprises retrieving, by the first device, the response from the second device from a second entry in the second buffer.

[0248] Statement 68. An embodiment of the present disclosure includes a method according to Statement 67, wherein retrieving, by the first device, a response from the second device from a second entry in a second buffer includes retrieving, by the first device, a response from the second device from a second entry in a second buffer of a memory coupled to the processor.

[0249] Statement 69. An embodiment of the present disclosure includes a method according to Statement 67, wherein retrieving, by the first device, the response from the second device from the second entry in the second buffer comprises: retrieving, by the first device, the response from the second device from the second entry in the second buffer of a memory of the first device.

[0250] Statement 70. An embodiment of the present disclosure comprises a method according to Statement 67, wherein retrieving, by the first device, the response from the second device from the second entry in the second buffer comprises: retrieving, by the first device, the response from the second device from the second entry in the second buffer of a memory of the second device.

[0251] Statement 71. An embodiment of the present disclosure includes a method according to Statement 67, wherein retrieving, by the first device, the response from the second device from the second entry in the second buffer comprises: retrieving, by the first device, the response from the second device from the second entry in the second buffer of a memory of a third device.

[0252] Statement 72. An embodiment of the present disclosure comprises a method according to Statement 66, wherein retrieving, by the first device, a response from the second device comprises retrieving, by the first device, a response from the second device from a second entry in the buffer.

[0253] Statement 73. An embodiment of the present disclosure comprises a method according to Statement 49, wherein establishing, by the first device, the entry comprises establishing, by the first device, a grouping structure.

[0254] Statement 74. An embodiment of the present disclosure comprises a method according to Statement 73, wherein grouping the structure comprises submitting the grouping structure or completing the grouping structure.

[0255] Statement 75. An embodiment of the present disclosure comprises a method according to Statement 49, further comprising establishing a buffer.

[0256] Statement 76. An embodiment of the present disclosure includes a method according to Statement 75, wherein establishing the buffer includes establishing the buffer in a memory coupled to the processor.

[0257] Statement 77. An embodiment of the present disclosure includes a method according to Statement 75, wherein establishing the buffer includes establishing the buffer in a memory of the first device.

[0258] Statement 78. An embodiment of the present disclosure comprises a method according to Statement 75, wherein establishing the buffer comprises establishing the buffer in a memory of the second device.

[0259] Statement 79. An embodiment of the present disclosure includes a method according to Statement 75, wherein establishing the buffer includes establishing the buffer in a memory of the third device.

[0260] Statement 80. An embodiment of the disclosure includes the method of Statement 75, wherein establishing the buffer includes establishing the buffer by a processor in communication with the first device and the second device.

[0261] Statement 81. An embodiment of the present disclosure includes a method according to Statement 80, wherein establishing, by a processor in communication with a first device and a second device, a buffer includes:

[0262] identifying, by the processor, the first device;

[0263] identifying, by the processor, the second device; and

[0264] Establishing, by the processor in communication with the first device and the second device, the buffer includes establishing, by the processor in communication with the first device and the second device, the buffer based on the processor identifying the first device and the second device.

[0265] Statement 82. An embodiment of the present disclosure includes a method according to Statement 80, wherein establishing, by a processor in communication with a first device and a second device, a buffer includes:

[0266] notifying the first device, by the processor, of the buffer; and

[0267] The second device is notified of the buffer by the processor.

[0268] Statement 83. An embodiment of the present disclosure includes a method according to Statement 75, wherein establishing the buffer includes establishing the buffer by the first device.

[0269] Statement 84. An embodiment of the present disclosure includes a method according to Statement 83, wherein establishing, by the first device, a buffer comprises:

[0270] identifying, by the first device, the second device; and

[0271] Establishing, by the first device, the buffer includes establishing, by the first device, the buffer based on the first device identifying the second device.

[0272] Statement 85. An embodiment of the present disclosure includes a method according to Statement 84, wherein identifying, by the first device, the second device comprises receiving, by the first device, a notification regarding the second device from a processor in communication with the first device and the second device.

[0273] Statement 86. An embodiment of the present disclosure includes a method according to Statement 84, wherein identifying, by the first device, the second device includes receiving, by the first device, a notification from the second device regarding the second device.

[0274] Statement 87. An embodiment of the present disclosure comprises a method according to Statement 83, wherein establishing, by the first device, the buffer comprises notifying, by the first device, the second device of the buffer.

[0275] Statement 88. An embodiment of the present disclosure includes a method according to Statement 75, wherein establishing the buffer includes establishing the buffer by the second device.

[0276] Statement 89. An embodiment of the present disclosure includes a method according to Statement 88, wherein establishing, by the second device, a buffer comprises:

[0277] identifying, by the second device, the first device; and

[0278] Establishing, by the second device, the buffer includes establishing, by the second device, the buffer based on the second device identifying the first device.

[0279] Statement 90. An embodiment of the present disclosure includes the method of Statement 89, wherein identifying, by the second device, the first device comprises receiving, by the second device, a notification regarding the first device from a processor in communication with the first device and the second device.

[0280] Statement 91. An embodiment of the present disclosure includes a method according to Statement 89, wherein identifying, by the second device, the first device includes receiving, by the second device, a notification from the first device regarding the first device.

[0281] Statement 92. An embodiment of the present disclosure comprises a method according to Statement 88, wherein establishing, by the second device, the buffer comprises notifying, by the second device, the first device of the buffer.

[0282] Statement 93. An embodiment of the present disclosure includes a method according to Statement 75, wherein establishing the buffer includes establishing the buffer by a third device.

[0283] Statement 94. An embodiment of the present disclosure includes a method according to Statement 93, wherein establishing a buffer by the third device includes:

[0284] identifying the first device by the third device;

[0285] identifying, by the third device, the second device; and

[0286] Establishing the buffer by the third device includes: establishing the buffer by the third device based on the third device identifying the first device and the second device.

[0287] Statement 95. Embodiments of the present disclosure include a method according to Statement 94, wherein:

[0288] identifying, by the third device, the first device comprises receiving, by the third device, a first notification regarding the first device from a processor in communication with the first device, the second device, and the third device; and

[0289] Identifying, by the third device, the second device includes receiving, by the third device, a second notification regarding the second device from a processor in communication with the first device, the second device, and the third device.

[0290] Statement 96. Embodiments of the present disclosure include a method according to Statement 94, wherein:

[0291] Identifying, by the third device, the first device comprises receiving, by the third device, a first notification from the first device regarding the first device; and

[0292] Identifying, by the third device, the second device includes receiving, by the third device, a second notification regarding the second device from the second device.

[0293] Statement 97. An embodiment of the present disclosure comprises a method according to Statement 93, wherein establishing, by the third device, the buffer comprises notifying, by the third device, the first device and the second device of the buffer.

[0294] Statement 98. An embodiment of the present disclosure includes a method according to Statement 49, wherein the communication includes advertising capabilities of the first device, analysis of the first device, redirection of a write request received from a processor at the first device, garbage collection coordination, execution of a computing function by the second device, or data migration from the first device.

[0295] Statement 99. Embodiments of the present disclosure include an article of manufacture comprising a non-transitory storage medium having stored thereon instructions that, when executed by a machine, result in:

[0296] identifying, at the first device, a second device to which the communication is to be directed;

[0297] establishing, by the first device, an entry, the entry comprising a communication; and

[0298] placing an entry in a buffer by the first device for delivery to the second device,

[0299] Therein, the second device retrieves the entry from the buffer.

[0300] Statement 100. Embodiments of the present disclosure include an article according to Statement 99, wherein the first device comprises:

[0301] a first interface for communicating with a second device; and

[0302] A second interface for communicating with the processor.

[0303] Statement 101. Embodiments of the present disclosure include an article according to Statement 99, wherein:

[0304] A first communication path enables communication between the first storage device and the second device; and

[0305] The second communication path enables communication between the first storage device and the processor.

[0306] Statement 102. Embodiments of the present disclosure include an article of manufacture according to Statement 99, wherein the first device comprises a storage device or a computational storage unit.

[0307] Statement 103. Embodiments of the present disclosure include an article of manufacture according to Statement 99, wherein the second device comprises a storage device or a computational storage unit.

[0308] Statement 104. An embodiment of the present disclosure includes an article of manufacture according to Statement 99, wherein the first device and the second device are in communication with a processor.

[0309] Statement 105. An embodiment of the present disclosure comprises an article of manufacture according to Statement 99, wherein the buffer comprises a ring buffer.

[0310] Statement 106. Embodiments of the present disclosure include an article of manufacture according to Statement 105, wherein the ring buffer comprises:

[0311] the head pointer; and

[0312] Tail pointer.

[0313] Statement 107. An embodiment of the present disclosure includes an article of manufacture according to statement 106, wherein placing, by the first device, the entry in the buffer for delivery to the second device comprises:

[0314] placing, by the first device, an entry in the buffer at a location identified by the tail pointer; and

[0315] Update the tail pointer.

[0316] Statement 108. Embodiments of the present disclosure include an article according to statement 106, a non-transitory storage medium having further instructions stored thereon, the further instructions, when executed by a machine, resulting in:

[0317] retrieving, by the second device, an entry from the buffer based on a second location identified by the head pointer; and

[0318] Update the head pointer.

[0319] Statement 109. An embodiment of the present disclosure comprises an article according to statement 99, a non-transitory storage medium having further instructions stored thereon, the further instructions, when executed by a machine, result in: retrieving, by the first device, a second entry from the buffer, the second entry comprising a second communication from the second device to the first device.

[0320] Statement 110. Embodiments of the present disclosure include an article according to Statement 109, wherein:

[0321] Creating, by the first device, the entry includes storing a first identifier of the first device in the entry; and

[0322] Retrieving, by the first device, the second entry from the buffer includes reading a second identifier of the second device from the second entry.

[0323] Statement 111. An embodiment of the present disclosure includes an article of manufacture according to statement 109, wherein placing, by the first device, the entry in the buffer for delivery to the second device comprises:

[0324] locking a buffer for the first device; and

[0325] Unlock the buffer for the first device.

[0326] Statement 112. An embodiment of the disclosure includes an article of manufacture according to Statement 99, wherein placing, by the first device, the entry in the buffer for delivery to the second device comprises placing, by the first device, the entry in the buffer of a memory coupled to the processor for delivery to the second device.

[0327] Statement 113. An embodiment of the present disclosure includes an article of manufacture according to Statement 99, wherein placing, by the first device, the entry in the buffer for delivery to the second device comprises placing, by the first device, the entry in a buffer of a memory of the first device for delivery to the second device.

[0328] Statement 114. An embodiment of the present disclosure includes an article of manufacture according to Statement 99, wherein placing, by the first device, the entry in a buffer for delivery to the second device comprises placing, by the first device, the entry in a buffer of a memory of the second device for delivery to the second device.

[0329] Statement 115. An embodiment of the disclosure includes an article of manufacture according to Statement 99, wherein placing, by the first device, the entry in a buffer for delivery to the second device comprises placing, by the first device, the entry in a buffer of a memory of a third device for delivery to the second device.

[0330] Statement 116. An embodiment of the present disclosure comprises an article according to statement 99, a non-transitory storage medium having further instructions stored thereon, the further instructions, when executed by a machine, causing the first device to retrieve a response from the second device.

[0331] Statement 117. An embodiment of the present disclosure comprises an article of manufacture according to statement 116, wherein retrieving, by the first device, the response from the second device comprises retrieving, by the first device, the response from the second device from a second entry in the second buffer.

[0332] Statement 118. An embodiment of the present disclosure includes an article of manufacture according to Statement 117, wherein retrieving, by the first device, the response from the second device from the second entry in the second buffer comprises: retrieving, by the first device, the response from the second device from the second entry in the second buffer of a memory coupled to the processor.

[0333] Statement 119. An embodiment of the present disclosure comprises an article of manufacture according to Statement 117, wherein retrieving, by the first device, the response from the second device from the second entry in the second buffer comprises: retrieving, by the first device, the response from the second device from the second entry in the second buffer of a memory of the first device.

[0334] Statement 120. An embodiment of the disclosure includes an article of manufacture according to Statement 117, wherein retrieving, by the first device, the response from the second device from the second entry in the second buffer comprises: retrieving, by the first device, the response from the second device from the second entry in the second buffer of a memory of the second device.

[0335] Statement 121. An embodiment of the disclosure includes an article of manufacture according to Statement 117, wherein retrieving, by the first device, the response from the second device from the second entry in the second buffer comprises: retrieving, by the first device, the response from the second device from the second entry in the second buffer of a memory of a third device.

[0336] Statement 122. An embodiment of the present disclosure comprises an article of manufacture according to Statement 116, wherein retrieving, by the first device, the response from the second device comprises retrieving, by the first device, the response from the second device from the second entry in the buffer.

[0337] Statement 123. An embodiment of the present disclosure comprises an article of manufacture according to Statement 99, wherein establishing, by the first device, the entry comprises establishing, by the first device, the grouping structure.

[0338] Statement 124. An embodiment of the present disclosure comprises an article of manufacture according to statement 123, wherein the grouping structure comprises a submission grouping structure or a completion grouping structure.

[0339] Statement 125. An embodiment of the present disclosure comprises an article according to Statement 99, a non-transitory storage medium having further instructions stored thereon, the further instructions when executed by a machine causing a buffer to be established.

[0340] Statement 126. An embodiment of the present disclosure includes an article of manufacture according to Statement 125, wherein establishing the buffer comprises establishing the buffer in a memory coupled to the processor.

[0341] Statement 127. Embodiments of the present disclosure include an article of manufacture according to Statement 125, wherein establishing the buffer comprises establishing the buffer in a memory of the first device.

[0342] Statement 128. An embodiment of the present disclosure comprises an article of manufacture according to Statement 125, wherein establishing the buffer comprises establishing the buffer in a memory of the second device.

[0343] Statement 129. An embodiment of the present disclosure includes an article of manufacture according to Statement 125, wherein establishing the buffer comprises establishing the buffer in a memory of the third device.

[0344] Statement 130. An embodiment of the disclosure includes an article of manufacture according to Statement 125, wherein establishing the buffer comprises establishing the buffer by a processor in communication with the first device and the second device.

[0345] Statement 131. An embodiment of the disclosure includes an article according to statement 130, wherein establishing, by a processor in communication with a first device and a second device, a buffer comprises:

[0346] identifying, by the processor, the first device;

[0347] identifying, by the processor, the second device; and

[0348] Establishing, by the processor in communication with the first device and the second device, the buffer includes establishing, by the processor in communication with the first device and the second device, the buffer based on the processor identifying the first device and the second device.

[0349] Statement 132. An embodiment of the disclosure includes an article according to statement 130, wherein establishing, by a processor in communication with a first device and a second device, a buffer comprises:

[0350] notifying the first device, by the processor, of the buffer; and

[0351] The second device is notified of the buffer by the processor.

[0352] Statement 133. An embodiment of the present disclosure includes an article of manufacture according to Statement 125, wherein establishing the buffer comprises establishing the buffer by the first device.

[0353] Statement 134. Embodiments of the present disclosure include an article according to Statement 133, wherein establishing, by the first device, the buffer comprises:

[0354] identifying, by the first device, the second device; and

[0355] Establishing, by the first device, the buffer includes establishing, by the first device, the buffer based on the first device identifying the second device.

[0356] Statement 135. An embodiment of the disclosure comprises an article of manufacture according to statement 134, wherein identifying, by the first device, the second device comprises receiving, by the first device, a notification regarding the second device from a processor in communication with the first device and the second device.

[0357] Statement 136. An embodiment of the present disclosure comprises an article of manufacture according to statement 134, wherein identifying, by the first device, the second device comprises receiving, by the first device, a notification from the second device regarding the second device.

[0358] Statement 137. An embodiment of the disclosure comprises an article of manufacture according to Statement 133, wherein establishing, by the first device, the buffer comprises notifying, by the first device, the second device of the buffer.

[0359] Statement 138. An embodiment of the present disclosure includes an article of manufacture according to Statement 125, wherein establishing the buffer comprises establishing the buffer by the second device.

[0360] Statement 139. Embodiments of the present disclosure include an article of manufacture according to Statement 138, wherein establishing, by the second device, the buffer comprises:

[0361] identifying, by the second device, the first device; and

[0362] Establishing, by the second device, the buffer includes establishing, by the second device, the buffer based on the second device identifying the first device.

[0363] Statement 140. An embodiment of the disclosure comprises an article of manufacture according to statement 139, wherein identifying, by the second device, the first device comprises receiving, by the second device, a notification regarding the first device from a processor in communication with the first device and the second device.

[0364] Statement 141. An embodiment of the present disclosure comprises an article of manufacture according to statement 139, wherein identifying, by the second device, the first device comprises receiving, by the second device, a notification from the first device regarding the first device.

[0365] Statement 142. An embodiment of the present disclosure comprises an article of manufacture according to Statement 138, wherein establishing, by the second device, the buffer comprises notifying, by the second device, the first device of the buffer.

[0366] Statement 143. An embodiment of the present disclosure includes an article of manufacture according to Statement 125, wherein establishing the buffer comprises establishing the buffer by a third device.

[0367] Statement 144. An embodiment of the present disclosure includes an article according to Statement 143, wherein establishing, by the third device, a buffer comprises:

[0368] identifying the first device by the third device;

[0369] identifying, by the third device, the second device; and

[0370] Establishing the buffer by the third device includes: establishing the buffer by the third device based on the third device identifying the first device and the second device.

[0371] Statement 145. Embodiments of the present disclosure include an article according to Statement 144, wherein:

[0372] identifying, by the third device, the first device comprises receiving, by the third device, a first notification regarding the first device from a processor in communication with the first device, the second device, and the third device; and

[0373] Identifying, by the third device, the second device includes receiving, by the third device, a second notification regarding the second device from a processor in communication with the first device, the second device, and the third device.

[0374] Statement 146. Embodiments of the present disclosure include an article according to Statement 144, wherein:

[0375] Identifying, by the third device, the first device comprises receiving, by the third device, a first notification from the first device regarding the first device; and

[0376] Identifying, by the third device, the second device includes receiving, by the third device, a second notification regarding the second device from the second device.

[0377] Statement 147. An embodiment of the disclosure comprises an article of manufacture according to Statement 143, wherein establishing, by the third device, the buffer comprises notifying, by the third device, the first device and the second device of the buffer.

[0378] Statement 148. Embodiments of the present disclosure include an article according to statement 99, wherein the communication includes advertising capabilities of the first device, analysis of the first device, redirection of a write request received from a processor at the first device, garbage collection coordination, execution of a computing function by the second device, or data migration from the first device.

[0379] Therefore, in view of the various arrangements of the embodiments described herein, this detailed description and the accompanying material are intended to be illustrative only and should not be taken as limiting the scope of the disclosure. Therefore, the disclosure claims all such modifications that may fall within the scope and spirit of the appended claims and their equivalents.

Claims

1. A system for communication between devices, comprising: First device; Second device; a processor, configured to communicate with the first device and the second device; and A buffer is provided for storing entries comprising communications from the first device to the second device.

2. The system according to claim 1, wherein: The buffer comprises a ring buffer.

3. The system according to claim 1, wherein: The second device comprises a memory, and the memory of the second device comprises the buffer. 4 . The system of claim 1 , further comprising a second buffer for storing a second entry comprising a second communication from the second device to the first device.

5. The system according to claim 4, wherein: The first device comprises a memory, the memory of the first device comprises the second buffer.

6. The system according to claim 1, wherein: The processor is configured to establish the buffer in a memory based at least in part on the processor identifying the first device and the second device.

7. The system according to claim 6, wherein: The processor is also configured to send a notification to the first device and the second device regarding the buffer in the memory of the system.

8. The system according to claim 1, wherein: The communication includes advertising capabilities of the first device, analysis of the first device, redirection of a write request received from the processor at the first device, garbage collection coordination, execution of a computing function by the second device, or data migration from the first device.

9. A method for inter-device communication, comprising: identifying, at the first device, a second device to which the communication is to be directed; establishing, by the first device, an entry, the entry including the communication; and placing, by the first device, the entry in a buffer for delivery to the second device, wherein the second device retrieves the entry from the buffer.

10. The method according to claim 9, wherein: Placing, by the first device, the entry in the buffer for delivery to the second device includes placing, by the first device, the entry in the buffer of a memory of the second device for delivery to the second device. The method of claim 9 , further comprising retrieving, by the first device, a response from the second device.

12. The method according to claim 11, wherein: Retrieving, by the first device, the response from the second device includes retrieving, by the first device, the response from the second device from a second entry in a second buffer.

13. The method according to claim 12, wherein: Retrieving, by the first device, the response from the second device from the second entry in the second buffer includes retrieving, by the first device, the response from the second device from the second entry in the second buffer of a memory of the first device. The method of claim 9 , further comprising establishing the buffer.

15. The method according to claim 14, wherein: Establishing the buffer includes establishing the buffer in a memory of the second device.

16. The method according to claim 14, wherein: Establishing the buffer includes establishing the buffer by a processor in communication with the first device and the second device.

17. The method according to claim 9, wherein: The communication includes advertising capabilities of the first device, analysis of the first device, redirection of a write request received from a processor at the first device, garbage collection coordination, execution of a computing function by the second device, or data migration from the first device.

18. An article of manufacture for inter-device communication, comprising a non-transitory storage medium having stored thereon instructions that, when executed by a machine, result in: identifying, at the first device, a second device to which the communication is to be directed; establishing, by the first device, an entry, the entry including the communication; and placing, by the first device, the entry in a buffer for delivery to the second device, wherein the second device retrieves the entry from the buffer.

19. The article of claim 18, the non-transitory storage medium having further instructions stored thereon that when executed by the machine result in establishing the buffer.

20. The article of claim 18, wherein: The communication includes advertising capabilities of the first device, analysis of the first device, redirection of a write request received from a processor at the first device, garbage collection coordination, execution of a computing function by the second device, or data migration from the first device.

Images