System for providing multiple processes in a host and asynchronously accessing multiple portions of memory of another host

Abstract

The present disclosure relates to a system providing a plurality of processes in a host and asynchronous access to a plurality of portions of a memory of another host. An API is combined with a bridge chip and a first host and a second host each having a first memory and a second memory. The bridge chip connects the memories. The API comprises: a key identifier registration function registering a key identifier for each of a plurality of computer processes executed by the first host, thereby defining a plurality of key identifiers; and / or an access control function providing access, typically via the bridge chip, to at least a local memory buffer M2 residing in the second memory, typically at least for a computer process P1 executed by the first host, after the access control function first verifies that the process P1 has a registered (e.g. via the key identifier registration function) key identifier. Typically, the access control function also at least prevents a computer process P2 executed by the first host, which has not registered a key identifier, from accessing the local memory buffer M2, e.g. via the bridge chip.

Description

Technical Field

[0001] This invention generally relates to bridging chips, and more specifically, to non-transparent bridges (NTBs). Background Technology

[0002] Transparent bridges are described in the following online link: etutorials.org / Networking / Lan+switching+fundamentals / Chapter+1.+LAN+Switching+Foundation+Technologies / Transparent+Bridging / .

[0003] Missinglinkelectronics.com explains (at this link:

[0004] (missinglinkelectronics.com / devzone / index.php / fpga-debug-ntb), using RC to represent the "root complex", that is:

[0005] "NTB stands for Non-Transparent Bridge. Unlike a PCI-Express bridge, also known as a PCIe (Transparent) bridge, where the RC 'sees' all the PCIe buses all the way to all endpoints, the NTB acts like a bridge, forwarding PCIe traffic between individual PCIe buses. Each RC treats the NTB as an endpoint device but cannot see other RCs or devices on the other side. This means that everything behind the NTB is not directly visible to a particular RC, hence it is 'non-transparent'."

[0006] Eetimes.com describes (at the following online link: eetimes.com / non-transparent-bridging-makes-pci-express-ha-friendly / #) that "a non-transparent bridge is functionally similar to a transparent bridge, except that there are intelligent devices or processors on both sides of the bridge, each with its own independent address domain. The host on one side of the bridge will not be able to see the full memory or I / O space on the other side of the bridge."

[0007] Mellanox multi-host technology supports connecting multiple compute / storage hosts to a single host. "Network adapter", for example, as described in the following online link: docs.mellanox.com / display / BlueFieldSWv31011424 / Multi-host.

[0008] Broadband.com describes (in the following online publication: docs.broadcom.com / doc / 12353427) that “developers have been using non-transparent bridging for years… to design multi-host systems and intelligent I / O… The non-transparent bridging (NTB) feature can isolate two host or memory domains while allowing state and data exchange between the two host or subsystems.”

[0009] An example of a non-transparent bridge (NTB) for sharing memory between two hosts is provided in the following online publication: kernel.org / doc / html / latest / driver-api / ntb.html, and includes the following operations:

[0010] Local devices:

[0011] 1. Allocate memory for shared windows

[0012] 2. Initialize the memory window using the translation address of the allocated region (this may fail if local memory window initialization is not supported).

[0013] 3. Send the converted address and memory window index to the peer device.

[0014] Peer-to-peer equipment:

[0015] 1. Initialize the memory window with the address retrieved from the memory region of another device (this may fail if peer memory window initialization is not supported).

[0016] 2. Map outbound storage window.

[0017] The online publication is available at the following link:

[0018] The document docs.broadcom.com / doc / 12353428#:~:text=A%20transparent%20bridge%20does%20not,opposite%20(other%20side)%20interface provides a detailed technical comparison between transparent and non-transparent bridges. It is understood that non-transparent bridges do not need to possess every technical feature mentioned in this publication.

[0019] Asynchronous I / O is described in the following online publication: en.wikipedia.org / wiki / Asynchronous_I / O. Summary of the Invention

[0020] Some embodiments seek to provide asynchronous and / or non-privileged NTB control and an input-output application programming interface (IO API). Typically, an IO API provides inter-host I / O (input-output) communication that can be interconnected via NTBs and also provides NTB control for asynchronous read-write transactions between hosts or for I / O. Typically, a given process (which may be running on one of the hosts) does not need to stop its processing until a transaction completes; rather, the process can typically continue its processing even if the transaction has not yet completed.

[0021] Some embodiments seek to provide nonprivileged, nontransparent bridged control and / or nonprivileged, nontransparent bridged I / O, which are typically nonprivileged in the sense that the control and / or I / O can be performed by (typically any) nonprivileged process rather than only by privileged processes.

[0022] At least the following embodiments are provided:

[0023] Example 1. An API (Application Programming Interface) device operating in conjunction with a bridging chip, a first host, and a second host, wherein the first host and the second host each have a first memory and a second memory, and wherein the bridging chip connects the first memory and the second memory, the device comprising: a key identifier registration function for registering a key identifier for each of a plurality of computer processes executed by the first host, thereby defining a plurality of key identifiers; and / or an access control function for, typically after the access control function first verifies that at least one computer process P1 executed by the first host has a key identifier that has been registered, for example, via the key identifier registration function, granting the computer process P1 access, typically via the bridging chip, to at least one local memory buffer M2 residing in the second memory, and wherein the access control function also prevents at least one computer process P2, for example via the bridging chip, from accessing the local memory buffer M2, the at least one computer process P2 being executed by the first host and / or not having registered a key identifier via the key identifier registration function.

[0024] Example 2. An apparatus according to any embodiment described herein, wherein the first host having a local memory buffer M1 registers the local memory buffer in the bridge chip and, in response, receives a key identifier from the bridge chip.

[0025] Example 3. An apparatus according to any embodiment described herein, wherein the second host registers the local memory buffer M2 in the bridge chip and, in response, receives a key identifier from the bridge chip.

[0026] Example 4. An apparatus according to any embodiment described herein, wherein at least one command for performing a read transaction is issued by the first host, and wherein the command includes a target address and a local address, and the bridging chip writes the required read response at the target address at the local address.

[0027] Example 5. An apparatus according to any embodiment described herein, wherein at least one command for performing a write transaction is issued by the first host, and wherein the command includes a target address and a local address, and the bridging chip obtains write data from the local address.

[0028] Example 6. The apparatus according to any embodiment described herein, wherein the command further includes a transaction size parameter indicating the amount of data to be read.

[0029] Example 7. The apparatus according to any embodiment described herein, wherein the command further includes a transaction size parameter indicating the amount of data to be written.

[0030] Example 8. An apparatus according to any embodiment described herein, wherein the bridging chip receives the command via the API and, in response, performs a verification attempt, the verification attempt including attempting to verify that the command has a valid key (aka a valid local key) from the first host and a valid key (aka a valid remote key) from the second host, and wherein the read transaction is executed only if the bridging chip succeeds in the verification attempt.

[0031] Example 9. An apparatus according to any embodiment described herein, wherein the bridging chip receives the command via the API and, in response, performs a verification attempt, the verification attempt including attempting to verify that the command has a valid key (aka a valid local key) from the first host and a valid key (aka a valid remote key) from the second host, and wherein the write transaction is executed only if the bridging chip succeeds in the verification attempt.

[0032] Example 10. The apparatus according to any embodiment described herein, wherein a command counter is incremented to indicate that the read transaction has been performed.

[0033] Example 11. An apparatus according to any embodiment described herein, wherein a command counter is incremented to indicate that the write transaction has been performed.

[0034] Example 12. An apparatus according to any embodiment described herein, wherein the bridging chip includes a non-transparent bridge (NTB) that connects multiple memories residing on multiple computers to a single PCI-Express structure.

[0035] Example 13. An apparatus according to any embodiment described herein, wherein the second host includes a smart NIC.

[0036] Example 14. An apparatus according to any embodiment described herein, wherein the second host includes a DPU.

[0037] Example 15. An apparatus according to any embodiment described herein, wherein an error message is generated whenever the bridging chip determines that the first host does not yet have a key identifier registered via the key identifier registration function.

[0038] Example 16. An apparatus according to any embodiment described herein, which provides an interface to the bridge chip and provides memory isolation between at least one computer process A executed by a first host and the at least one local memory buffer M2.

[0039] Example 17. An apparatus according to any embodiment described herein, wherein the API apparatus further comprises:

[0040] Additional key identifier registration functionality; and

[0041] Additional access control functions, which, after verifying that at least one computer process P3 executed by the second host has a key identifier that has been registered via the additional key identifier registration function, grant the computer process P3 access via the bridge chip to at least one local memory buffer M1 residing in the first memory.

[0042] Furthermore, the additional access control function also prevents at least one computer process P4 from accessing the local memory buffer M1 via the bridge chip. The at least one computer process P4 is executed by the second host and has not yet registered a key identifier via the additional key identifier registration function.

[0043] Example 18. An apparatus according to any embodiment described herein, wherein the API (Application Programming Interface) apparatus includes the bridge chip, the first host, and the second host.

[0044] Example 19. An apparatus according to any embodiment described herein, wherein a completion bit is provided to indicate that the read transaction has been performed.

[0045] Example 20. An apparatus according to any embodiment described herein, wherein a completion bit is provided to indicate that the write transaction has been performed.

[0046] Example 21. A method for an application programming interface (API) that operates in conjunction with a bridging chip, a first host, and a second host, wherein the first host and the second host each have a first memory and a second memory, and wherein the bridging chip connects the first memory and the second memory, the method comprising:

[0047] For each of the multiple computer processes executed by the first host, a plurality of key identifiers are defined; and

[0048] After first verifying that at least one computer process P1 executed by the first host has a registered key identifier, the computer process P1 is provided with access via the bridge chip to at least one local memory buffer M2 residing in the second memory, and

[0049] At least one computer process P2 is prevented from accessing the local memory buffer M2 via the bridging chip. The at least one computer process P2 is executed by the first host and has not yet registered a key identifier.

[0050] Example 22. An apparatus according to any embodiment described herein, wherein the key identifier identifies the local memory buffer M2.

[0051] Example 23. An apparatus according to any embodiment described herein, wherein the key identifier uniquely identifies the local memory buffer M2. Attached Figure Description

[0052] Figure 1 This is a simplified block diagram of a system for non-privileged, non-transparent bridge control and I / O constructed and operated according to embodiments of the present invention.

[0053] Figure 2 This is a simplified flowchart illustrating a method for providing non-privileged, non-transparent bridge control and I / O, whereby multiple processes in a host (e.g., a DPU host) can asynchronously access various portions of the main host memory at various granularities, all according to embodiments of the invention; this can be done individually or in combination. Figure 1 The method can be performed on all or any suitable subset of the boxes shown, or on all or any suitable subset of the operations shown. This method can be executed each time the first host issues a memory request to the memory of the second host via a bridge such as an NTB.

[0054] Figures 3a-3b This is a simplified block diagram that enables the multi-host environment described in the embodiments of this article. Detailed Implementation

[0055] Known multi-host technologies allow multiple compute or storage hosts to connect to a single interconnect adapter, for example, by separating the adapter (e.g., PCIe) bus into several independent interfaces. For example, Mellanox Multi-Host... TM Technology (first time and) (Introduced together with -4) Enabled in Mellanox Socket Direct cards. This technology allows multiple hosts to be connected to a single adapter by separating the PCIe (e.g.) interface into multiple independent interfaces. For example, as described in the following https: / / nvidia.com / en-us / networking / multi-host / , each interface typically connects to a separate host CPU. Typically, in a multi-host architecture, multiple hosts connect directly to a single network controller, allowing direct access to data with lower capital and operating costs. A multi-host architecture can include different CPU architectures (e.g., x86 and / or Power and / or ARM central processing units), where each host is independent of the others, but all hosts can share the same network interface, saving switch ports, cabling, space, and power.

[0056] PCI (Peripheral Component Interconnect) or PCIe devices are typically enabled during addressing, for example, by mapping to the system's I / O port address space or memory-mapped address space. The system's firmware, device drivers, or operating system can program a so-called "base address" register (also known as BAR) to inform the device of the PCIe device's address mapping, for example, by writing configuration commands to the PCIe controller.

[0057] InfiniBand (IB) is a computer networking communication standard used for data interconnection between and / or within computers (such as supercomputers), and / or as a direct or exchanged interconnection between servers and storage systems or between storage systems.

[0058] The published InfiniBand specifications are available from the InfiniBand Trade Association. InfiniBand provides Remote Direct Memory Access (RDMA) functionality to reduce CPU overhead. InfiniBand uses a switched network topology, also known as a switching architecture, where nodes are typically interconnected via multiple network switches, such as crossbar switches. For example, Mellanox's InfiniBand host bus adapters and network switches are used in many commercial computer systems and databases.

[0059] The following terms may be interpreted according to any appropriate definition appearing in the literature of the relevant technical field, or according to this specification, or may be included within their respective scopes:

[0060] As a non-limiting example, the term "fabric" is intended to include the fabric shown at the following https link: etherealmind.com / wp-content / uploads / 2011 / 07 / what-switch-fabric-pt2-6.jpg / .

[0061] The term "privilege" refers to an individual attribute (e.g., of a computer process) that allows a "privileged" process to perform security-related operations (such as memory access operations, like the DPU accessing host memory from the ARM side) and disallows other ("non-privileged") processes to perform the same operations. A process running with one or more privileges is called a privileged process, and the program that process is running is called a privileged or trusted program. A privileged computer process is authorized and / or trusted to perform at least one given security-related function, while other ("non-privileged") processes are not trusted to perform that function and are therefore not authorized to do so.

[0062] Host: A networked computer, for example, as a non-limiting example, a DPU.

[0063] The terms “compute host” and “storage host” are defined at the following https: / / www.nvidia.com / en-us / networking / multi-host / .

[0064] A “key identifier” is an identifier, such as a number or other string, that identifies (and is usually uniquely associated with) a given memory region and is used as a key (e.g., the key identifier must be provided to the hardware in order to access the given memory region).

[0065] "Memory isolation" occurs when process A cannot access the memory of process B (e.g., cannot access any memory associated with any process other than process A itself), unless process A is granted privileges or explicit permissions to access the memory associated with a process other than process A itself. For example, "logical" memory isolation can be implemented through an NTB that provides processor domain partitioning and address translation between memory mapping spaces of processor domains, making devices on each side of the bridge invisible from the other side, and, despite this, data transfer and state exchange between processor domains are possible.

[0066] The term "process" is intended to include, for example, as described in the following link: en.wikipedia.org / wiki / Process_(computing), an instance of a computer program being executed by one or more threads. A process typically contains program code and its activities. Depending on the operating system (OS), a process may include multiple threads of execution, with all instructions executing concurrently.

[0067] Bridge chip: A device that connects one bus (e.g., a PCIe bus) (on which a first host may be located) to another bus (e.g., a PCIe bus on which a second host may be located). Bridge chips may have onboard read and write queues, may have prefetch functionality, and may have caching functionality; one or both of these can be configured by the user.

[0068] Local / Remote: Given a computer process executed by a first host that seeks access to at least one local memory buffer residing on a second host via a bridging chip, the "local" key is a key from the first host, and the "remote" key is a key from the second host.

[0069] Read operations include reading data from remote host memory and writing that data to local host memory.

[0070] Write operations include reading data from local host memory and writing that data to remote host memory.

[0071] The host's "offset" is a term that defines the starting position of a memory transaction within a given memory area or region pointed to by a given key. For example, particularly when the memory area comprises a very large memory region, the offset (e.g., from the beginning of the region) and length parameters can be used to define the sub-region (within the given region) where a given transaction will take action.

[0072] Registers: are intended to include storage, for example, for key identifiers; if this is the case according to the PCIe specification, each register contains a valid address in the device address space range or configuration space from which the host can read and / or write.

[0073] A Non-Transparent Bridge (NTB) is a bridging chip (e.g., described in the online link below: kernel.org / doc / html / latest / driver-api / ntb.html) that connects multiple memory systems or memory residing on multiple computers to a single fabric (e.g., to a PCI-Express (aka PCIe) fabric, in which case the NTB can be called a PCIe NTB chip). NTB hardware supports read-write registers accessible from both sides of the device, allowing peers to exchange a certain amount of data at fixed addresses. The NTB client driver registers with the NTB core driver; registration can, for example, use a Linux device framework. NTBs are useful when it is necessary to share some memory between multiple systems deployed on two separate "sides" of the bridge. Typically, NTBs have an API that supports two types of memory window interfaces—inbound translation and outbound translation. The former is typically configured on the local NTB port, while the latter (outbound translation) is typically configured by the peer or remote end of the bridge on the peer's NTB port.

[0074] For users in multi-host environments, especially those including SmartNICs or DPUs, offloading some processing and computation to the DPU host itself (e.g.) An ARM core on a 2 could be useful. However, those offloaded processes might need to access data residing on the main host itself (e.g., x86). The problem to be solved is how multiple processes in the DPU host can access different portions of the main host's memory asynchronously with variable or disparate granularity, while granting each process different access permissions and ensuring memory isolation (as defined above) between the processes. It's understandable that if multiple processes are to be downloaded to the DPU host, each such process might need to access host memory.

[0075] Possible solutions to the problem of supporting asynchronous access by multiple processes on a host to different parts of the host's memory with varying granularities include:

[0076] 1. Traditional NTB – The standard API for a conventional NTB may include a fixed mapping from host memory to the NTB address space. Here, accessing different parts or regions of memory typically involves reconfiguring the window each time – which can only be done by a privileged process. Furthermore, according to the PCIe specification, reconfiguring the window typically cannot be done asynchronously. If you want or need to support access to different address spaces, you might need to provide a window for each address space – this could translate to an unattainable BAR size.

[0077] 2. RDMA – This problem can be overcome by using RDMA read and RDMA write. However, this solution requires an RDMA-enabled device and multiple connections to the host for each offloaded process. Understandably, this unnecessarily consumes network resources when an RDMA-enabled device opens multiple connections between multiple offloaded processes and the host.

[0078] 3. A client-server model using RPC (Remote Procedure Call) can be used. This involves the DPU host having a client daemon to send transactions to read / write memory on the primary host and having an API exposed to other processes. The primary host itself may have a server daemon to accept these transactions, execute them, and send responses when needed. This solution requires additional CPU utilization—for both the DPU host and the primary host. Furthermore, this approach has high overall latency and requires complex software.

[0079] However, in contrast, some embodiments of the present invention include an NTB device API or an IO API, such as... Figure 1 As shown, it typically supports both reading and writing asynchronously, typically provides memory isolation, and typically offers privileged memory access.

[0080] Typically, the term "asynchronous" is used to distinguish it from "synchronous" operations in a system, where tasks, I / O operations, or transactions are executed one at a time, and the next task is only unblocked when one completes, thus requiring waiting for a task to finish before moving on to the next. In asynchronous operations, the next task can begin before the previous task completes. Therefore, asynchronous programming allows multiple requests to be processed concurrently, enabling more tasks to be completed within a given time period, thereby promoting parallelism. For example, conventional computer processes typically need to wait to acquire memory because conventional or traditional NTBs usually use regular PCIe transactions that only generate responses after a certain period of time; this is not the case for the asynchronous implementation described here.

[0081] The asynchronicity of tasks, such as NTB IO operations (e.g., read-write transactions between two hosts interconnected via NTB) and asynchronous I / O, is defined in the following online entry: en.wikipedia.org / wiki / Asynchronous_I / O.

[0082] therefore, Figure 1 An API (Application Programming Interface) device is shown that operates in conjunction with a bridge chip, a first host, and a second host, and provides multiple processes in the host to asynchronously access multiple portions of the memory of the other host.

[0083] Figure 1Such devices typically provide non-privileged, non-transparent bridge control and I / O, allowing multiple processes in the host (e.g., the DPU host) to asynchronously access different portions of the main host's memory, for example, according to Figure 2 The method is as follows. Typically, a privileged process is not required to control the NTB, because instead, pre-registration is provided, where a key identifier for a memory region can be registered. After this, NTB control can be provided by at least one (usually any) process that has previously registered the key identifier for a given memory region that is being used in a given transaction or I / O session between two hosts (e.g., another host accessing the local host via the NTB).

[0084] In some embodiments, the first host is external to the API (Application Programming Interface) device, but in other embodiments, the first host may be included therein. Similarly, in some embodiments, the second host is external to the API (Application Programming Interface) device, but in other embodiments, the second host may be included therein. In some embodiments, the bridging chip is external to the API (Application Programming Interface) device, but in other embodiments, the bridging chip may be included therein.

[0085] Figure 1 The device typically includes two APIs, located on either side of the bridge chip, such as an NTB; each API belongs to a different address space, as the first API belongs to the address space of host1 (host 1) and the second API belongs to the address space of host2 (host 2). According to some embodiments, a host is connected to the bus on either side of the NTB, and when one of the host's CPUs uses an API, that host can typically access the memory of the other host via the bridge chip (e.g., the NTB). A non-transparent bridge or NTB may have, but does not necessarily need to have, temporary registers and / or doorbell registers and / or heartbeat messages. The NTB device API can be via PCIe or any other suitable standard for device connectors, and transactions between the two hosts may or may not be PCIe transactions.

[0086] The second host may include a DPU (e.g., 2DPU), in which case the second host includes an ARM processor, and the DPU host may also be referred to as the "ARM side" or "DPU side / host" of the bridge, which is opposite to the other side of the bridge, which may be referred to as the "x86 side" of the bridge (assuming the first host or the primary host includes an x86 processor, which is the case in most cases) or "primary host side".

[0087] exist Figure 1In the diagram, dotted and dashed lines are used to distinguish commands on either side of the bridge. NTBs are typically symmetrical because each NTB API generates READ / WRITE commands for both hosts.

[0088] It is understandable that the DPU can obtain non-privileged random access to host memory from the ARM side asynchronously (e.g., via RDMA). However, Figure 1 The API shown and described herein may be a less complex solution; for example, creating a valid RDMA connection for use may include various commands and / or data structures that do not need to be provided when using the API shown and described herein.

[0089] Figure 2 It is a kind of thing that can be made by Figure 1 The system executes and provides a method for non-privileged, non-transparent bridge control and I / O, allowing multiple processes in the host (e.g., the DPU host) to asynchronously access different parts of the main host's memory at various granularities. It is understood that processes are typically asynchronous because the NTB is responsible for executing all transactions, unlike traditional NTB architectures where the process itself (not the bridge) performs read / write transactions.

[0090] Whenever a process on the first host (host1) attempts to access the memory of a process on the second host (host2), it can execute... Figure 2 Alternatively, or additionally, a process on host2 may attempt to access the memory of a process on the second host.

[0091] Figure 2 This may include the following operations or Figure 2 The operation shown here properly sorts all or any suitable subset of d, for example as shown in the figure or as follows:

[0092] Operation 210: host1 (or a process on it) has a memory request to make to host2 memory (e.g., to write or read a given size of data from host2 memory). Typically, host1 includes... Figure 1 The DPU host shown is shown, while host2 includes... Figure 1 The primary host shown is (e.g., x86). However, this is not intended to be limiting, as other alternatives are possible. For example, host2 could include... Figure 1 The DPU host is shown, and host1 can include the primary host. Host1 can be a compute host or a storage host, as can host2.

[0093] If host1's memory request is approved, host1 can appropriately use the memory body requested by host1. Example use case: host1 might be a DPU, and host2 offloads certain processes to that DPU, for example, to use the DPU as an accelerator. However, as a result, the offloaded processes might need to access host2's memory (e.g., a database used by host2), so in this use case, host1 (e.g., processes offloaded to it) might need to access host2's memory. Furthermore, some devices can connect to the DPU but may not be directly connected to the host, for example... -2x SSD. If host1's request is approved, this enables the device to access host memory and / or enables the host (e.g., the primary host) to access the DPU device buffer. It is understood that the above are just examples of many use cases where the DPU may need to perform non-privileged random access to host memory from the ARM side (e.g., asynchronously).

[0094] In some embodiments, each process has a command gateway that it can control (one gateway per process), for example, as described herein.

[0095] Suppose that in operation 210, "Process 1" on host1 attempts to issue a memory request to "Process 2" on host2. Subsequently, the following operations may be performed:

[0096] Operation 220: Process 1 on host1 registers its local memory buffer in the NTB and obtains its key identifier (typically using...). Figure 1 (NTB API).

[0097] Operation 230: Process 2 on host2 registers its local memory buffer in the NTB and obtains its key identifier (typically using...). Figure 1 (NTB API).

[0098] Typically, host2 provides host1 with the host2 key identifier through a method outside the NTB API. Any suitable technology can be used to exchange key identifiers; for example, as a non-limiting example, via a network connection (e.g., TCP), via a shared file that can be read and / or written by both hosts, or even by manual copying.

[0099] It is understandable that operation 220 is usually omitted if the memory buffer being used is already registered. Furthermore, operation 230 is usually omitted if the memory buffer being used is already registered.

[0100] Operation 240: Process 1 on host1 issues a secure NTB access command (which may include all or any suitable subset of <host1_key, host2_key, host1_offset, host2_offset, size, read / write>). The command may typically include a local address where the NTB writes the read response required at the target address or from which write data is obtained, and / or a remote address that serves as the target address for the read response or write data, and / or a transaction size, and / or an indicator bit that indicates whether the transaction is a read transaction or a write transaction (usually using Figure 1 the NTB API).

[0101] It is appreciated that the forward operation 240 may be performed multiple times (depending on the number of accesses that host1 needs to perform).

[0102] Operation 250: The NTB obtains the command to execute the transaction

[0103] Operation 260: The NTB checks whether the keys are valid and whether there is permission

[0104] Operation 270: Determine validity, e.g., yes / no

[0105] Operation 280: Report an error

[0106] Operation 290: Issue the transaction

[0107] Operation 300: Increment the command counter, or write a completion bit indicating that the command has been completed, e.g., at a local address which may be in host1's own local memory. The term "completion bit" is intended to include a bit whose value indicates whether a certain transaction has been issued and completed. Because when the bridge chip completes the transaction, the "completion" bit is flipped (e.g., if the bit is 0, the bridge chip writes 1, or if the bit is 1, the NTB writes 0), and at least one (usually any) process can determine whether the transaction has completed by tracking the value of the completion bit. However, it should be understood that implementation via a command counter or completion bit is described herein only as a non-limiting example. Alternatively, any other appropriate method may be used to track when a transaction ends or completes.

[0108] Typically, the command counter is physically stored on the internal memory of the NTB. The command counter may logically form part of the address space range indicated by the BAR of the NTB.

[0109] Operation 310: Complete.

[0110] According to some embodiments, each time a process in host1 wants to make a memory request to a portion of host2's memory, execution is performed. Figure 2 The process. For example, if Figure 2 The method is executed n times by n processes in host1, where each of the n processes attempts to make a memory request to the corresponding part of the memory of host2. Each of the n processes usually obtains asynchronous access to the corresponding part of the memory of host2.

[0111] In response to an error reported in operation 280, no appropriate action may be taken or not. Typically, whichever entity, such as a process in host1, commands the NTB to execute a transaction, is the entity that receives the error report. If an error has been reported indicating that a previously issued command by an entity (e.g., a process in host1) is not valid, one action that entity may take is to issue a command to execute a valid transaction. For example, if the length field of a command issued by a process is larger than the size of a memory region—the NTB might fail to execute the command but will not enter a fatal error state; instead, the method returns to operation 250 after operation 280, and the process that issued the failed or invalid command can issue another command to use a valid transaction. In this case, no error is reported in operation 80, and the transaction can be issued (operation 290). A “fatal” error state, if any, can be defined using any suitable criteria, for example, it could be a result of the internal NTB hardware implementation. For example, the NTB hardware implementation might produce fatal errors.

[0112] It should be understood that incrementing or flipping the completion bit of the command counter is one of two possible methods, mentioned in the non-restrictive example, to indicate to the process that issued the command that the asynchronous transaction (the object of operation 280) has been completed.

[0113] According to some embodiments, memory isolation is achieved because only process A with a given key identifier can access the given memory, while at least one (typically any) process without the given key identifier cannot access the given memory. Therefore, the given memory is isolated from all processes A.

[0114] The NTB API can be characterized by a command structure comprising all or any suitable subset of the following command structure components: <local key, remote key, local offset, remote offset, transaction size (e.g., in bytes), and binary read / write bits>. The local and / or remote keys typically uniquely identify the range of memory addresses or memory regions that the NTB can access; this allows for the implementation of security and / or isolation. Typically, certain portions of each memory region or range, such as the first few bytes (e.g., 4 bytes) of the memory region, are reserved for NTB use, for example, to provide a command gateway as described herein. The stored local / remote offsets can indicate the offset from the beginning of a memory space that is typically uniquely identified by the key. The read / write bits indicate whether a given transaction is a read or write operation.

[0115] NTB API commands can be written by at least one process to its assigned "command gateway," which is typically unique; that is, a command gateway is usually assigned to a given process and not to other processes. The NTB can then begin processing the command and can write new commands to the gateway. Typically, each command gateway has a corresponding "command counter," which increments when a command is completed; the command gateway and its command counter usually require no more than a few bytes of storage space.

[0116] Multi-host environment

[0117] NTB's API can include commands in the following formats:

[0118] <Local address, remote address, size, read / write>

[0119] The "local address" is the space to which the bridging chip can write read responses or to which the bridging chip can retrieve write data; the remote address, also known as the "target" address in the host memory, is the transaction size in bytes, and read / write is a transaction type.

[0120] Each time a process attempts to read S bytes from host memory at address X and tries to store the result at address Y of the process, the process can issue the following command:<Y,X,S,read> .

[0121] The command for writing transactions can be similar. For example, if a process attempts to write S bytes to host memory at address X, and those S bytes are currently located at address Y of the process, then the process might issue the following command:<Y,X,S,write> After this command is issued, the process does not need to wait because the NTB will process the command asynchronously. The NTB typically notifies the process of command completion either by incrementing a command counter corresponding to the command gateway through which the command was issued, or by notifying the process of command completion by storing a completion bit in the first byte of the local address. In response to being notified of command completion, the process can then proceed accordingly; for example, the program can take some actions and / or not take some actions based on its own logic. For example, if a process on one side of the bridge host is configured to read data from another host on the other side of the bridge, the process may only begin using the data upon receiving notification of command completion. In contrast, it is understood that conventional computer processes typically need to wait to acquire memory because conventional NTBs use conventional PCIe transactions, which only generate responses after a certain period of time; this is not the case for the embodiments described here.

[0122] To allow multiple processes to access the NTB concurrently, each process can be provided with at least one "command gateway" where it can write its commands. It's understandable that multiple command gateways could be assigned to a single process; however, to ensure isolation, multiple processes are typically not assigned to a single command gateway.

[0123] Since each process's command gateway only needs to be a few bytes in size, for example, just the length of the command itself, it can support up to thousands of processes, and the "cost" is only a few KB of BAR, which may lead to dedicated command gateways.

[0124] A dedicated KB for the command gateway can include a suitable memory address space or memory region, which can be pointed to by the base address register (BAR) and is typically only a few KB in size, sufficient to include the address of the command gateway. The dedicated KB for the command gateway can be a portion of the device's BAR address space (which may or may not be resized), and therefore can simply be the memory address exposed by the device to the host, and may not be connected to any actual memory. The NTB can have an internal queue that monitors commands written to the gateway (i.e., the GW) (or can use any other suitable method or data structure to monitor commands). The NTB typically cannot distinguish between different processes, and only allows a process to use the NTB if and only if that process has the correct key. Therefore, typically, the NTB does not need to know how many commands are in the gateway, nor does it need to know which command belongs to which process.

[0125] It is understood that even a regular NTB can be configured to transfer transactions (e.g., reads or writes) between two hosts. A command gateway, provided according to certain embodiments, serves as an interface for commanding the NTB to transfer transactions (reads or writes) between two hosts. Typically, a key identifier, such as as described herein, indicates to the NTB on which memory region to operate on, and / or the size of the data to be read / written, and / or read / write bits indicate to the NTB the type of transaction to be initiated.

[0126] To support licensing and / or isolation, key identifiers can be used. For example, a secure command format may include all or any suitable subset of the following format components:

[0127] <Local key, remote key, local offset, remote offset, size, read / write>, where the local / remote key is an identifier, usually unique, used for the local / remote memory address ranges that the NTB can access, the local / remote offsets are the offsets from the beginning of the local / remote memory space, and the remaining parameters (size, read / write) characterize the transaction that the process is trying to issue by representing the amount of data to be read / written, and whether the transaction is a read or write type transaction.

[0128] Typically, key identifiers need to be pre-registered by relevant processes with all relevant permissions to ensure that only processes that possess the previously registered key identifiers and uniquely (usually) identify a given region in memory can access that region.

[0129] Typically, the key is not a one-time key. The key identifier is usually registered only once and can then be reused by the NTB from that moment on (e.g., unless and until explicitly not registered). Therefore, generally, even given a memory region to be used multiple times (read and / or write) with a given key, a single registration of that key identifier is sufficient, assuming that the same key is indeed used for all the various uses of the memory region. Any suitable method can be used to register the memory; for example, as a non-limiting example, conventional memory registration methods known in InfiniBand technology, such as those described in the following https: / / rdmamojo.com / 2012 / 09 / 07 / ibv_reg_mr / .

[0130] It is understandable that each process registers its key identifier, and this pre-registration typically occurs each time a memory region needs to be accessed by the NTB. This is because using the NTB with a memory region usually requires registering the memory region, and the key identifier, which typically uniquely identifies the registered memory region, is provided as a condition for accessing that region. The entity responsible for registering a particular memory region is usually the process that owns that memory region.

[0131] It is understandable that in the normal use of NTB to share memory between two hosts, NTB can create regions in its own address space that point to the memory regions of the hosts.

[0132] If host A wants to write to host B, it can perform all or any suitable subset of the following operations:

[0133] Operation 1a: Host A can configure the NTB window to point to the desired memory.

[0134] Operation 1b: Host A can notify host B that host B can write.

[0135] Operation 1c: Host B can write its actual data into the NTB address space.

[0136] The NTB can forward this write to the memory pointed to by the NTB window (to host A's memory). Typically, this window allocation on the NTB address space limits the size and / or number of memories that a single NTB can span. However, alternatively, the NTB provided according to embodiments of the invention does not include the type of window described above, but instead copies from a region on host A to a region on host B (and vice versa). For example (using the example above), if host A wants to write to host B, it can perform all or any suitable subset of the following operations:

[0137] Operation 2a: Host A can register a memory region with write permissions.

[0138] Operation 2b: Host B can register a memory region with read permissions.

[0139] Operation 2c: Host A can notify host B about the key of host A.

[0140] Operation 2d: Host B can write its data to its registered area.

[0141] Operation 2e: Host B can write a command, whose command NTB is copied from the host B memory pointed to by the host B key to the host A memory pointed to by the host A key.

[0142] Therefore, host B typically does not write actual data, as commands usually do not include actual data. Understandably, host B typically requires host A's key to proceed, which prevents other processes on host B from gaining access to memory. Generally, an NTB command is only effective if it contains two keys (a local key and a remote key).

[0143] It should be understood that the embodiments described herein have many advantages in practice (e.g., as non-limiting examples): processes in a first host accessing another host's memory can be non-privileged. Security and isolation between processes are maintained. Multiple processes are supported. Asynchronous access to host memory is supported, typically with more than one level of granularity. Granularity can define a relationship, such as the ratio between computational load and communication load. If parallelism is fine-grained, the task code size and execution time are small, and small amounts of data (e.g., a few memory words or less) are frequently transferred between processors. Conversely, if parallelism is coarse-grained, the task code size and execution time are large, and the amount of data transferred between processors is also large, and the frequency of transfer between processors is low. Therefore, granularity defines the frequency of data communication and the computational load (large or small). It is understood that the determination of granularity is generally subject to the following trade-offs: fine-grainedness increases parallelism and speed, but also increases the overhead of synchronization and communication. Conversely, coarse-grainedness reduces parallelism and speed, but also reduces the overhead of synchronization and communication.

[0144] According to certain embodiments, using Figure 1 Systems and / or Figure 2 This method allows multiple processes in a host (e.g., a DPU host) to asynchronously access different parts of the main host's memory at various granularities.

[0145] Example 1: Consider a single offloading process with a single granularity (e.g., table entry size). This offloading process may need to access some entries in a table in primary host memory. The table is large (e.g., tens of GB), and each entry is small (e.g., only a few bytes per entry). Because the table is so large, it cannot be mapped all at once via a traditional NTB, so accessing the first and last entries would require an expensive reconfiguration of the traditional NTB. In contrast, the API described here, for example, via... Figure 1 and / or Figure 2 In some implementations, such access is allowed to occur only once through registration (e.g., only in a local memory buffer within the NTB).

[0146] It is understandable that in Example 1 above, the use of Figure 2 Implementations typically reduce communication overhead and / or prevent load imbalance, where load indicates the number of processes using NTB and how NTB resources are allocated to each process.

[0147] Referring back to Example 1, it's clear that on a traditional NTB, each window reconfiguration requires notifying other hosts. Therefore, accessing unmapped entries on the access table necessitates reconfiguring the window and informing other hosts of the reconfiguration. Conversely, when using the API shown and described here (e.g., when using...),... Figure 2 In the example implementation, the entire table is mapped at once.

[0148] Understandably, if process A only needs to access a few table entries but maps the entire table, it will lead to a load imbalance. In this case, only one mapping is needed, and that mapping is coarse because it includes a large number of unused table entries / regions. Conversely, if the window only includes the memory required each time, the window usually needs to be configured on each access, so each window configuration usually requires communication to notify other hosts about window reconfiguration.

[0149] It is understandable that references to memory regions can be interchanged with references to buffers, and vice versa.

[0150] Example 2: The offloading process needs to access large amounts of data residing on the primary host. If a traditional NTB were used, the large amounts of data would likely be mapped; however, using the large amounts of data would require the process to transfer or read this large amount of data from the window itself, which would require the process to use its own resources to access the large amounts of data. Conversely, if the API described here is used, for example, according to... Figure 1 and / or Figure 2 In this embodiment, the NTB executes a transaction and copies the transaction result to any registered memory buffer, thereby freeing up the resources of the offloading process for other tasks. It is understood that the embodiments described here improve parallel performance by achieving a better balance between load and communication overhead. Due to the fact that different granularities are supported, performance is no longer strictly limited by the aforementioned trade-offs. Instead, according to some embodiments, the possibility of performance being affected by communication overhead due to overly fine granularity can be reduced or eliminated, as can the possibility of performance being affected by load imbalance due to overly coarse granularity can also be reduced or eliminated.

[0151] It is understandable that in Example 2, since the copying is performed by the NTB rather than the CPU, the CPU usage is less (more economical).

[0152] For example, DPU use cases may require non-privileged random access to host memory from the ARM side (which is asynchronous), or benefit from or improve upon it.

[0153] Figures 3a-3bAn example of a multi-host environment that can implement the embodiments of this document is shown; as illustrated, each host is associated with a memory (aka MEM). For simplicity, the term "all" is used herein to describe the example embodiments. However, it will be understood that, alternatively, whether the terms "all," "each," or "every" members of a set are true, or characterize or relate to them, it may also be true, or characterize or relate to, most but not all members of the set, or all members of the set except a few, or at least one (but not all) members of the set, in other embodiments.

[0154] It is understood that the software components of this invention may be implemented in ROM (Read-Only Memory) if desired. The software components may typically be implemented in firmware or hardware using conventional techniques if desired. It should also be understood that the software components may be instantiated, for example, as a computer program product or on a tangible medium. In some cases, the software components may be instantiated as signals interpretable by a suitable computer, although such instantiation may be excluded in certain embodiments of this invention.

[0155] It should be understood that the various features of the invention described in the context of individual embodiments for clarity may also be provided in combination in a single embodiment. Conversely, the various features of the invention described in the context of individual embodiments for brevity may also be provided individually or in any suitable sub-combination.

[0156] Those skilled in the art will understand that the present invention is not limited to what has been specifically shown and described above. Rather, the scope of the invention includes, in particular, the appended claims and their equivalents.

Claims

1. An apparatus for implementing an application programming interface (API) that operates in conjunction with a bridging chip, a first host, and a second host, wherein the first host and the second host each have a first memory and a second memory, and wherein the bridging chip connects the first memory and the second memory, the apparatus comprising: Processor, the processor providing: The key identifier registration function is used to register a key identifier for each of the multiple computer processes executed by the first host, thereby defining multiple key identifiers; An access control function is configured to, after the access control function first verifies that at least one computer process P1 executed by the first host has a key identifier that has been registered via the key identifier registration function, provide the at least one computer process P1 with access via the bridge chip to at least one local memory buffer M2 residing in the second memory, and wherein the access control function also prevents at least one computer process P2 executed by the first host from accessing the at least one local memory buffer M2 via the bridge chip, the at least one computer process P2 being executed by the first host and having not yet registered a key identifier via the key identifier registration function; Additional key identifier registration functionality; and An additional access control function, which first verifies that at least one computer process P3 executed by the second host has a key identifier that has been registered via the additional key identifier registration function, grants the at least one computer process P3 access via the bridge chip to at least one local memory buffer M1 residing in the first memory, and wherein the additional access control function also prevents at least one computer process P4 executed by the second host from accessing the at least one local memory buffer M1 via the bridge chip, the at least one computer process P4 being executed by the second host and not having registered a key identifier via the additional key identifier registration function.

2. The apparatus of claim 1, wherein the first host registers the at least one local memory buffer M1 in the bridge chip and, in response, receives a key identifier from the bridge chip.

3. The apparatus of claim 1, wherein the second host registers the at least one local memory buffer M2 in the bridge chip and, in response, receives a key identifier from the bridge chip.

4. The apparatus of claim 1, wherein at least one command for performing a read transaction is issued by the first host, and wherein the at least one command includes a target address and a local address, and the bridging chip writes the required read response at the target address at the local address.

5. The apparatus of claim 1, wherein at least one command for performing a write transaction is issued by the first host, and wherein the at least one command includes a target address and a local address, and the bridging chip obtains the write data from the local address.

6. The apparatus of claim 4, wherein the at least one command further includes a transaction size parameter indicating the amount of data to be read.

7. The apparatus of claim 5, wherein the at least one command further includes a transaction size parameter indicating the amount of data to be written.

8. The apparatus of claim 4, wherein the bridging chip receives the at least one command via the API and performs a verification attempt, the verification attempt including attempting to verify that the at least one command has a valid key from the first host and a valid key from the second host, and wherein, The read transaction is executed only after the bridging chip successfully completes the verification attempt.

9. The apparatus of claim 5, wherein the bridging chip receives the at least one command via the API and, in response, performs a verification attempt, the verification attempt comprising attempting to verify that the at least one command has a valid key from the first host and a valid key from the second host, and wherein, The write transaction is executed only after the bridging chip successfully completes the verification attempt.

10. The apparatus of claim 4, wherein the command counter is incremented to indicate that the read transaction has been performed.

11. The apparatus of claim 5, wherein the command counter is incremented to indicate that the write transaction has been performed.

12. The apparatus of claim 1, wherein the bridging chip includes a non-transparent bridge NTB that connects multiple memories residing on multiple computers to a single PCI-Express structure.

13. The apparatus of claim 1, wherein the second host comprises a smart network interface card (NIC).

14. The apparatus of claim 13, wherein the second host includes a data processing unit (DPU).

15. The apparatus of claim 8, wherein an error message is generated whenever the bridging chip determines that the first host does not yet have a key identifier registered via the key identifier registration function.

16. The apparatus of claim 1, wherein it provides an interface to the bridge chip and provides memory isolation between at least one computer process A executed by the first host and the at least one local memory buffer M2.

17. The apparatus of claim 1, further comprising the bridging chip, the first host, and the second host.

18. The apparatus of claim 4, wherein a completion bit is provided to indicate that the read transaction has been performed.

19. The apparatus of claim 5, wherein a completion bit is provided to indicate that the write transaction has been performed.

20. The apparatus of claim 1, wherein the key identifier identifies the at least one local memory buffer M2.

21. The apparatus of claim 1, wherein the key identifier uniquely identifies the at least one local memory buffer M2.

22. An application programming interface (API) method for operation in conjunction with a bridging chip, a first host, and a second host, wherein the first host and the second host respectively have a first memory and a second memory, and wherein, The bridging chip connects the first memory and the second memory, and the method includes: For each of the multiple computer processes executed by the first host, a number of key identifiers are defined; After first verifying that at least one computer process P1 executed by the first host has a registered key identifier, the at least one computer process P1 is provided with access to at least one local memory buffer M2 residing in the second memory via the bridge chip. Prevent at least one computer process P2 from accessing the local memory buffer M2 via the bridge chip, wherein the at least one computer process P2 is executed by the first host and has not yet registered a key identifier; After first verifying that at least one computer process P3 executed by the second host has a registered key identifier, the at least one computer process P3 is provided with access via the bridge chip to at least one local memory buffer M1 residing in the first memory; and At least one computer process P4 is prevented from accessing at least one local memory buffer M1 via the bridge chip. The at least one computer process P4 is executed by the second host and has not yet registered a key identifier.

Images