Network interface for data transmission in heterogeneous computing environments
By adopting a multi-head network controller in a heterogeneous computing environment, using the main header and auxiliary header to directly copy data to the accelerator memory, the problem of low data transmission efficiency caused by a single header network controller is solved, and efficient data transmission and memory is achieved optimization.
Patent Information
- Application Number
- CN202411081737.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2019-06-07
- Filing Date
- 2020-03-25
- Publication Date
- 2024-11-29
AI Technical Summary
In heterogeneous computing environments, the current network controller is single-header, resulting in increased waiting time and memory bandwidth usage when data passes through multiple architectures. The existing solutions require customized software stacks and multiple memory copies, affecting efficiency .
The multi-head network controller is adopted, and the host and accelerator architecture are connected by the main header and the auxiliary header respectively, so that data is directly copied to the accelerator memory, reducing the intermediate steps through the host memory, and optimizing data routing using DMA operations and address translation services.
It improves data transmission efficiency in heterogeneous computing environments, reduces latency and memory bandwidth usage, supports high bandwidth and low latency data transmission, and is suitable for computing-intensive tasks such as deep learning.
Smart Images

Figure CN119052167A_ABST
Abstract
Description
This application is a divisional application of a patent application filed on March 25, 2020, with application number 202010218842.X and invention name “Network interface for data transmission in heterogeneous computing environments”. Technical Field
[0001] Various examples described herein relate to routing packets to target memory devices in a heterogeneous computing environment. Background Art
[0002] In the context of artificial intelligence (AI) and machine learning (ML), deep neural networks (DNNs) are becoming increasingly popular for performing tasks such as object classification, object detection, speech recognition, and other inference tasks. DNNs find applications in both performance-driven servers and power-optimized edge devices. As the size of the model increases, the number of weights and activations increases as the amount of computation increases. Accelerator devices dedicated to performing tasks related to the computation of neural networks are common.
[0003] Deep learning (DL) algorithms for AI require memory, compute, and network scaling to efficiently complete computations. These algorithms can be implemented as data parallel, model parallel, or hybrid models. DL algorithm implementation requires processing large data sets distributed to many compute nodes and merging the results from the nodes for the final result. BRIEF DESCRIPTION OF THE DRAWINGS
[0004] Figure 1 An example of components used to run deep learning applications in a data center is shown.
[0005] Figure 2A Depicted is a high-level system diagram in which a network interface provides received packets directly to an accelerator fabric for transfer to a memory device.
[0006] Figure 2B An example system is depicted in which a network interface provides received packets directly to an accelerator architecture for transmission to a memory device.
[0007] Figure 3A An example setup operation is depicted.
[0008] Figure 3B Depicted is example processing of received packets.
[0009] Figure 3C An example of determining a destination address of a received packet is depicted.
[0010] Figure 3D An example of transmitting a packet to a destination memory address to an accelerator memory is depicted.
[0011] Figure 4 An example communication sequence is depicted.
[0012] Figures 5A-5C Describes the processes that can be performed by a network interface.
[0013] Figure 6 Describes the system.
[0014] Figure 7 Depicted are network interfaces in which embodiments may be used or utilized.
[0015] Figure 8 An example of a data center is depicted. DETAILED DESCRIPTION
[0016] Figure 1 Examples of components used to run deep learning or other machine learning (ML) or other artificial intelligence (AI) applications in a data center are shown. In order to distribute, calculate and synchronize large data sets between computing nodes, the following key components can be used in a data center running a large DL application. The computing node can be composed of multiple heterogeneous computing elements connected together using a scale-up fabric - a CPU scale-up fabric, an accelerator scale-up fabric. An optional host-to-device fabric can connect the host to the accelerator device. A network including a Top of Rack (ToR) or an End of Row (EoR) can be used to connect the computing nodes together.
[0017] For example, a variety of computing engines may include CPU, core, accelerator or graphics processing unit (GPU). In addition or alternatively, a computing node with high bandwidth memory (HBM) may be used to process large data sets. The scaled-up architecture of high bandwidth and low latency is used to provide the connection of computing elements within and across nodes. The host architecture (hostfabric) may be used to connect processors (such as core or central processing unit (CPU)) to each other (such as Intel ultrapath interconnect (UPI) or quick path interconnect (QPI)). The I / O architecture may be used to connect a network interface to a computing node (such as a fast peripheral component interconnect (PCIe)). The accelerator architecture may be used to connect a GPU or accelerator together (such as NvidiaNVLink, PCIe exchange, CCIX, GenZ, compute fast link (CXL)). These architectures may be connected to a top-of-rack (ToR) or end-of-row (EoR) switch.
[0018] However, current network controllers are single-headed and can be connected to only one host architecture. This limitation requires that data targeted to the accelerator pass through multiple architectures and incurs additional data copies, resulting in increased latency and higher memory bandwidth usage. For example, for data received at a network controller or architecture interface to be provided to a target accelerator, the data is provided to the host architecture by the network controller, stored in a memory associated with a processor connected to the host architecture, and then copied to a memory device or domain associated with the target accelerator via the accelerator architecture. Two copies are used in this example: one to the processor memory and the second to the accelerator memory.
[0019] Another known solution involves data received by the network interface being written directly to the accelerator memory (without an intermediate copy to the host memory). However, the path of the data traverses multiple architectures, first through the host-to-device architecture to the CPU memory, possibly through the CPU scale-up architecture across CPU sockets, and then through the host-to-device architecture to the accelerator memory. In addition, this solution requires a custom software stack to enable.
[0020] Various embodiments provide a network controller enhancement to connect a high bandwidth and low latency accelerator scaled-up architecture within a computing node in a large data center. Elements connected to the scaled-up architecture can be located in the same server rack, share the use of an operating system or share the same host central processing unit (CPU). Various embodiments provide a multi-head network controller for low latency and non-uniform memory access (NUMA) optimized remote direct memory access (RDMA) input / output operations on a platform with heterogeneous computing elements running various accelerator applications. The network controller can have a main head and an auxiliary head, wherein the main head is connected to the host to the device architecture and the auxiliary head is connected to the accelerator scaled-up architecture to provide a NUMA input / output solution. The multi-head network controller is configured to determine whether to use the accelerator scaled-up architecture to copy the received packet directly to the memory attached to the accelerator in which the CPU memory and the accelerator memory can be accessed by all computing elements. The accelerator memory can be accessed by the CPU or exposed to the CPU as a conventional write-back memory or as an MMIO aperture. The network controller may determine whether a translated address of a destination memory address (e.g., an untranslated address) is available at a network interface by means of a cache memory or through an address translation service. If the translated address is available, the translated address is decoded to determine a route to a destination memory that is to receive a portion of the received packet. If the route passes through an accelerator architecture, an auxiliary header may be used, whereby a direct memory access (DMA) operation is invoked to copy a portion of the received packet to the destination memory through the accelerator architecture. If the route is not specified to use an accelerator architecture, a portion of the received packet is copied to the destination accelerator memory using the host architecture.
[0021] Figure 2A A high-level system diagram is depicted in which a network interface provides received packets directly to an accelerator architecture for transmission to a destination memory accessible to the accelerator device. For example, a target platform 210 may include or use a network interface 216 that receives packets from a remote platform 200 via a data center network 220. If authorized, the network interface 216 copies the content (e.g., from memory 208-0 or memory 208-1) or packets directly to memory 218 through the accelerator architecture 212 via an auxiliary header. The network interface may support Ethernet packets or additionally or alternatively provide an architecture interface (e.g., non-volatile memory express over fabrics (NVMeOF) on an architecture, an interface to an interconnect or bus, etc.).
[0022] Figure 2BAn example system is depicted. Network interface 250 can couple heterogeneous computing elements of a platform to devices or nodes coupled to a data center (DC) network 240. Heterogeneous computing elements can include general-purpose central processing units (CPUs), field programmable gate arrays (FPGAs), graphics processing unit (GPU)-based accelerators, and other devices. DC network 202 can be any network or interconnect or bus such as PCI, PCIe, and others. Examples of devices or nodes coupled to data center (DC) network 202 include computing, memory, or storage element platforms; computing, memory, or storage element slides; and racks of computing, memory, or storage elements.
[0023] The network interface 250 can be an Ethernet compatible network interface that supports remote direct memory access (RDMA) to memory buffers via a host-to-device architecture 256-0 or 256-1 or an accelerator architecture 258. The network interface 250 can support other standards such as InfiniBand. The network interface can be multi-headed and support multiple main heads and auxiliary heads. The head consists of multiple high-bandwidth I / O buses or links (e.g., PCI, PCIe, etc.). In some examples, the head can be a link from a host-to-device architecture or an accelerator architecture. The multi-head network interface can have multiple links, some of which are coupled to the host-to-device architecture and some of which are coupled to the accelerator architecture.
[0024] The network interface 250 processes RDMA send / receive requests associated with various storage areas and uses the converted addresses to generate input / output (I / O) requests for the primary architecture or secondary architecture via the corresponding primary header or auxiliary header. The primary header of the network interface 250 communicates with the host to device architecture 206-0 or 206-1. The primary header of the network interface 250 can be managed by a driver running on a general-purpose processor (e.g., a CPU). In addition, multiple primary headers are used for I / O non-uniform memory access (NUMA) performance optimization. The primary header and auxiliary header of the network interface 250 are assigned unique identifiers in their corresponding architectures, such as bus: device: function in the case of a PCI architecture.
[0025] The network interface 250 can support multiple protocols for control plane communication via a host-to-device architecture 256 (e.g., PCIe, compute express link (CXL), etc.) and protocols for an accelerator architecture 258 (e.g., PCIe, compute express link (CXL), NVLink, coherent accelerator processor interface (CAPI), cache coherent interconnect for accelerators (CCIX), Gen-Z, etc.). For example, the host-to-device architecture 256-0 and 256-1 can be compatible with UPI, HyperTransport, QPI, etc. The host-to-device architecture 256-0 or 256-1 can couple the CPU to the accelerator and couple the accelerator to other accelerators. The accelerator scaled-up architecture 208 can couple accelerators to each other. Both the host architecture and the accelerator architecture can be logically and physically separate architectures or logically separate but physically identical architectures. These architectures can be composed of PCIe, PCI, or CXL interconnects. The network interface 250 can support RDMA (e.g., RDMA over Converged Ethernet (RoCE), Internet Wide Area RDMA Protocol (iWARP)) to allow remote access to storage areas and provide large data sets with reduced overhead to accelerator engines that are part of a heterogeneous computing platform. In some examples, RDMA is used for accelerator services that use (one or more) auxiliary heads for low latency and efficient data movement. However, RDMA can be used for direct copy or memory access by any device coupled to the host-to-device architecture 256-0 or 256-1. When using a coherent host-to-device architecture (e.g., NVLink or CXL), the accelerator memory can be addressed using either a host physical address (HPA), or when using an incoherent host-to-device architecture (e.g., PCI-E), a unified device-specific physical address (DPA) space can be used to address the accelerator memory.
[0026] The network interface 250 can be configured to decode the destination memory address in the HPA or DPA depending on the configuration information provided by the control plane application during the RDMA I / O memory configuration. For example, an application running on a CPU establishes a connection to a remote device coupled through a data center (DC) network 240. The application uses the main header to establish the control plane operation. The control plane application running on a general-purpose CPU or one or more cores can discover various heterogeneous computing elements and associated physical memories. The application or control plane application can use RDMA verbs to configure storage areas associated with various accelerators that collaborate with the network interface driver. The control plane application can configure various quality of service (QoS) attributes for RDMA services targeting the accelerator architecture 208.
[0027] When the network interface 250 receives a packet from a remote device via the network 240 and uses one or more of the flow identifier, 5-tuple, manipulation tag, and / or destination address to determine whether to copy the contents of the packet to a memory buffer via the accelerator scaled-up architecture 258. The network interface 250 can determine the memory target location based on the physical address. When the IOMMU is enabled, the address provided by the control plane is not a physical address and therefore cannot be directly decoded. As described later, the network interface 250 can use an address translation service (ATS) to translate the address to obtain a physical address.
[0028] In order to reduce the cost or latency of address translation, the device may implement an address translation cache (ATC) that stores a copy of the translated physical address. If a cached copy of the translated address is not available, the network interface 250 generates an address translation request via the primary header to obtain the translated address. The address translation request is serviced by the input-output memory management unit (IOMMU) to convert the virtual or logical address to a physical address. For example, a PCIe-compatible address translation unit (ATU) may return a host physical address (HPA) to the network interface 250. The network interface 250 receives the HPA from the IOMMU and decodes the physical memory target in which the data is to be written. However, if the address translation service is not configured with the translated address of the destination address, the network interface 250 will not copy the received packet to the destination memory buffer. Instead, other actions may occur, such as discarding or abandoning the packet, reporting an error to the application, or the IOMMU may also present an error to the platform software.
[0029] Trusted applications running on the CPU can configure address translation tables based on a global view of the platform memory.ATS support on the primary header enables function flow through the secondary header with required memory address isolation to prevent unauthorized copying to the accelerator memory.
[0030] For available translated addresses, the network interface 250 may use DMA operations to copy a portion of the received packet to the translated memory destination through the accelerator scaled-up architecture 258 rather than through the host-to-device architecture 256-0 or 256-1 to the destination memory. Input / output devices (e.g., network or fabric interfaces), solid-state drives (SSDs) are connected to the host-to-device architecture 256-0 or 256-1, so the host-to-device architecture 256-0 or 256-1 may become overloaded and become a bottleneck. Data destined for the accelerator may be sent directly to the accelerator scaled-up architecture 258 to avoid using the bandwidth of the host architecture 256.
[0031] In some examples, the network interface 250 routes the data portion of the packet to the destination memory through the scale-up fabric 258 and routes the control portion of the packet to the destination memory through the host-to-device fabric 256-0 or 256-1. Control plane software (not depicted) can configure the routing of portions of the received packet (e.g., data or control information) through the host-to-device fabric 206 or the accelerator scale-up fabric 208.
[0032] Depending on the choice of fabric interconnect between the host fabric and the scaled-up fabric, the accelerator memory may be viewed as a unified host physical address (HPA) space (e.g., using Intel Accelerator Link (IAL) as the interconnect) or a unified device-specific physical address (DPA) space (e.g., using PCI as the interconnect). For the HPA memory view, the multi-head network interface 250 supports the PCI Address Translation Services (ATS) capability and is used to request pre-translated addresses (HPA) on its auxiliary head for the targeted memory address. For the DPA memory view configuration, the control plane application configures the memory buffers with hints for the network controller to decode the I / O addresses accordingly.
[0033] The operating system (OS) uses address translation to protect all memory managed by it. For the network interface, the OS does this by providing IO addresses to the network interface. Before accessing the memory, the IO address will be converted to HPA. For some systems, the accelerator memory may not be part of the address space managed by the OS. The driver creates an address space for the accelerator memory on such a system and this is called a device-specific physical address (DPA). The device driver can implement DPA translation (optionally) to provide isolation of the accelerator memory. In this case, the driver can provide a client DPA to the network interface. Note that the network interface can be a physical network interface, a network interface controller, or a virtual network interface instance that shares the use of a physical network interface.
[0034] The accelerator memory may be coupled to the accelerator as locally attached memory using an interface or bus compatible with any variation of high bandwidth memory (HBM), double data rate (DDR) (e.g., DDR4, DDR5, etc.), or graphics double data rate (GDDR) (e.g., GDDR4, GDDR5, etc.). The locally attached memory may be connected to the accelerator using a memory bus or interface rather than through an architecture.
[0035] Figures 3A-3D An example sequence of an enhanced network controller for communicating data or other information in a heterogeneous computing platform is depicted. Figure 3AAn example setup operation is depicted. Application 302 may request to establish a remote direct memory access (RDMA) data transfer by a remote entity 304 to a storage area that can be accessed by an accelerator device. Protocols other than RDMA may be used, such as direct data placement (DDP), InfiniBand, Intel Omni-Path, implementations of RDMA such as RoCE and iWARP, and the like. Application 302 may use an InfiniBand-compatible verb application program interface (API) to establish a queue pair with a remote application to enable RDMA writes directly to the destination memory while bypassing the use of the OS to manage the copy operation at the receiver. For example, application 302 may use the OpenFabrics Alliance Verbs API to initiate an RDMA write to a storage area that can be accessed by an accelerator device. For example, application 302 may provide an oil or gas high performance computing (HPC) environment, an artificial intelligence (AI) training and reasoning engine, image recognition, speech recognition, augmented reality, virtual reality, and the like. For example, remote entity 304 may be a user application executed on a mobile or smart phone, an Internet-connected self-driving control system for a vehicle (e.g., a car, bus, truck, airplane, helicopter).
[0036] The host CPU control plane software 306 can perform pre-work to establish a descriptor ring and a memory buffer address in a host or accelerator memory space that can receive packets from a remote entity 304. The descriptor can specify a destination address of a packet in a memory or other memory locally attached to the accelerator. The descriptor can also provide an address to be written by the network interface to indicate that the packet has been copied to the destination. An address translation service (described later) is used to determine whether the destination address is in the host memory or in the accelerator memory. The network interface driver can program a routing table in the network interface that identifies whether to use an auxiliary header for writing to a destination translated address or to use an auxiliary header for reading from a target translated address.
[0037] In some embodiments, the control plane software 306 provides the destination address of the packet in the descriptor and the control plane software 306 provides an indication (e.g., (one or more) indicator bits) of whether to use the main header or the auxiliary header to write the packet to the destination address. When the use of the main header is indicated, the network interface uses the address of the IOMMU or cache to convert the destination address. When the use of the auxiliary header is indicated, the network interface uses the auxiliary header, DMAs the packet with its DPA (address) to the destination accelerator via the accelerator architecture and the accelerator can interpret the DPA. In some cases, the address has been converted, and the accelerator uses the converted address to place the packet into the assigned accelerator memory destination. In some other cases, the interface device can implement device-specific address conversion for the DPA (address). Similar operations described later can be used by the network interface to read data from a memory locally attached to the accelerator.
[0038] The control plane software 306 registers the destination address in the address translation service and provides address translation of the physical or virtual destination address for copying the contents of the received packet carrying the destination address. If the translated address is not registered for the destination address provided in the received packet, the direct copying of a portion of the received packet to the local memory of the accelerator by the accelerator architecture is not performed.
[0039] The control plane software 306 can manage a routing table that defines the transmission of received packets from the remote entity 304 to the appropriate memory buffer and the completion of the descriptor of the received packet indicating that the packet (or a portion thereof) has been copied to the destination address in the memory. The control plane 306 can manage the operation of the network interface controller 300 by performing system configuration, management, and exchange of routing table information. The control plane 306 can exchange topology information with other devices or entities and construct a routing table based on a routing protocol. The control plane 306 can guide business via routing rules so as to prioritize, de-prioritize, and block packets of a specific type of packet. The control plane packet can be transmitted to the receiver from another entity or can be caused locally by the network interface or router itself.
[0040] The network interface controller 300 may support transmission and reception based on at least Ethernet, RDMA (eg, RoCE or iWARP), a virtual interface architecture, InfiniBand, or Omni-Path.
[0041] Figure 3BDepicted is an example processing of a received packet. The network interface controller 300 receives a packet 308. The received packet 308 may represent one or more packets. The packet 308 may be an Ethernet frame, although other protocols such as TCP / IP may be used. The physical and transport layer processing 310 may perform physical and transport layer processing according to the applicable protocol, such as, but not limited to, encoding or decoding data packets according to the applicable physical layer specification or standard, assembling or disassembling data based on the destination address and source address together with network control information and error detection hash values. In this example, the network interface controller 300 identifies that the packet 308 is part of an RDMA connection by the characteristics of its packet header. Based on the received packet header (e.g., the tagged buffer, memory address, and length of the remote destination buffer), the network interface controller 300 may determine the destination address of a portion of the received packet. On a system where the IOMMU is disabled, the destination memory address may be a physical address. On a system where the IOMMU is enabled, the destination memory address is an IO virtual address (IOVA). The IOVA is to be converted into a physical address that can be decoded to make a routing decision.
[0042] Address translation is used to determine whether the received packet can be directly copied or placed in memory without calling the operating system or kernel-level processing. Address decoding logic 312 can provide the destination address from packet 308 to request address translation cache 314 to determine whether the translated address is available. PCIe address translation services can be used to convert IOVA to obtain physical addresses. Physical addresses can be decoded to make routing decisions (e.g., using auxiliary or primary headers). Some examples use address translation caches to locally save copies of translated addresses in cache 314. If the translated address is stored in cache 314, the previously received packet with the same destination address has been verified and the received packet 308 can be written directly to memory by the network interface controller 300 via the architecture of the accelerator scaled up. In other words, the same destination address of the previously received packet has been used by address translation service 316 and the translated address has been stored in cache 314. In some cases, the translated address can be filled into cache 314 and the availability of the cached translated address is due to pre-filling. Use of cache 314 may avoid delays and processing overhead associated with invoking use of address translation service 316. If a translated address is not available for the destination address in cache 314, decode logic 312 may request address translation service 316 for the translated address.
[0043] The ongoing protocol between the control plane and the cache 314 can be used to invalidate one or more addresses in the cache 314 to prevent unauthorized copying to the destination memory. The operating system can manage the address translation cache entries. The control plane application can register the buffer using the OS API and the OS can maintain the management page table and address translation cache. In addition, the control plane can update the address translation stored in the cache 314 to the connection that is most likely to be used, such as receiving the most packets.
[0044] Figure 3C An example of determining the destination address of a received packet is depicted. If the address is not in the address translation cache 314, the translation cache 314 transmits the destination memory address to the address translation service 316 via the control plane communication 315 through the host CPU scaled-up architecture 317. In some examples, the main header of the network interface controller 300 is used to communicate with the address translation service 316 via the host CPU scaled-up architecture 317. The address translation service 316 can be implemented as an IOMMU address translation. If the address translation service 316 has been provided with a translated address of the destination address, the translated address is provided in response for storage as a pair of destination addresses and the translated address is stored in the address translation cache 314.
[0045] If the address translation service 312 has not been provided with the translated address of the destination address, a null is provided in response and the received packet 308 is not copied directly to the memory address. Instead, the packet 308 may be processed using the operating system (OS) of the host system or may be assigned to the destination memory buffer according to an applicable scheme. In other examples, the packet 308 may be discarded. Thus, the presence or absence of the translated address of the destination address in the packet may provide a security feature to prevent direct copying at least where address translation is not available.
[0046] Figure 3D An example of a destination memory address to which a packet is transmitted to an accelerator memory is depicted. The network interface controller 300 may use a routing table 319 to determine whether to use an auxiliary header based on the converted destination memory address. In this example, the auxiliary header of the network interface controller 300 is used to provide a connection to an accelerator scaled-up architecture 322 and a destination memory buffer allocated for the accelerator. One or more DMA engines 318 may support direct copying of received data as shown in the accelerator payload 320 from the received packet 308 to the destination memory buffer 324. The destination memory buffer 324 may be a physical or logically addressable area of a memory device locally connected to the accelerator device.
[0047] Direct memory access (DMA) is a technology that allows input / output (I / O) devices to bypass the central processing unit (CPU) or core and directly send data to system memory or directly receive data from system memory. Because DMA allows the CPU or core to not manage copy operations when sending data to system memory or receiving data from system memory, the CPU or core can be used to perform other operations. Without DMA, when the CPU or core is using programmed input / output, the CPU or core is generally occupied for the entire duration of the read or write operation and cannot be used to perform other work. With DMA, the CPU or core can, for example, initiate a data transfer and then perform other operations while the data transfer is in progress. When the data transfer is completed, the CPU or core can receive an interrupt from the DMA controller.
[0048] In some examples, for multiple received packets having the same destination memory address, the contents of the multiple received packets may be merged and copied to the destination memory area in a single DMA operation rather than multiple DMA operations.
[0049] After the data is processed by the accelerator, the processed data may be sent to a memory of a remote entity using an auxiliary head to copy the data from the accelerator memory through the accelerator architecture to a network interface controller, may be sent to another accelerator memory for additional processing, may be provided to a CPU for access, may be retained in memory, may be copied to a storage device, or for other uses.
[0050] Figure 4 An example communication sequence is depicted. At 402, the target application may request the control plane of the RDMA connection of the remote application to write directly to a storage area accessible to the accelerator used by the application. For example, the target application may use the RDMA verb API to initiate an RDMA write operation to the storage area by the remote application.
[0051] At 404, the control plane creates an RDMA connection for the packet from the remote application from a source buffer (not depicted). At 406, if the destination address provided with the packet corresponds to a valid memory address, the control plane can program the network interface to provide an RDMA copy operation for the packet from the remote application to the assigned memory buffer.
[0052] At 408, the remote application can cause the packet to be transmitted to the network interface and the memory address associated with the accelerator or processor used by the application. At 410, the network interface receives the packet, and if the cached address conversion of the destination address is not available, the network interface requests the address conversion service to perform the address conversion of the destination address specified in the received packet. In some cases, instead of requesting address conversion on a per-packet basis, address conversion can be requested for a group of packets with the same specified destination address. At 412, the address conversion service provides the converted address to the network interface for one or more packets. The indication of the destination address of the specified packet in the routing table or descriptor in the network interface is used to identify whether the auxiliary header is to be used by the network interface. At 414, the network interface copies one or more packets from a buffer (not depicted) to the destination buffer specified by the converted address for access by the accelerator used by the application via the auxiliary header and by means of DMA through the accelerator architecture.
[0053] Note that the example provided enables the destination address provided in the received packet to be identified and converted. However, if an address conversion service is not provided to identify or convert the destination address, the copy operation using DMA copying will not occur and the packet can be discarded. In some cases, if the destination address of the received packet is already on the isolation list or is not identified, the packet can be discarded.
[0054] Figure 5A Depicted is a process that can be performed by a network interface to copy a received packet according to an embodiment. At 502, the network interface is programmed to support RDMA requests from a specific sender. For example, the network interface can be programmed to perform address translation that can be requested for a destination address provided with a packet received from a specific sender. The sender can be a user application executed on a mobile or smart phone, an Internet-connected self-driving control system for a vehicle (e.g., a car, bus, truck, airplane, helicopter). At 504, the network interface receives a packet as part of an RDMA connection. At 506, the network interface attempts an address translation of a destination address specified in the received packet. For example, action 508 can be performed, whereby the address translation can include checking whether an address translation cache or memory accessible to the network interface stores an association of a destination address and a translated address. At 510, if a cached address translation is not available, the network interface can request an address translation service (e.g., an IOMMU) to translate the destination address.
[0055] At 512, a determination is made as to whether the address translation was successful. If the translated address is available through either a cache memory or an address translation service, the address translation may be successful. If the address translation is successful, 514 may then occur. At 514, the network interface determines a header for copying a portion of the packet. An indication of the destination address of the specified packet in a routing table or descriptor in the network interface is used to identify whether an auxiliary header is to be used by the network interface. At 516, the network interface calls a DMA operation to copy a portion of the received packet to a destination memory specified by the translated address via a specified header (e.g., an auxiliary header) and an accelerator architecture. The host-to-device architecture may be avoided to prevent the host-to-device architecture from being used.
[0056] If the address translation is unsuccessful, then 520 may follow. At 520, the network interface does not cause DMA of the received packet to the destination memory buffer without invoking the use of the CPU, OS, or network interface driver. Instead, the packet may be discarded and routed to another destination address determined by the CPU, OS, or network interface driver.
[0057] Figure 5B Depicted is a process for packet transmission according to an embodiment that can be performed by a network interface. At 550, an application requests the transmission of data or packets from a memory locally attached to an accelerator. A destination device or memory for data or packets can be specified in a descriptor received at the network interface. At 552, the network interface can use a DMA engine to copy data from a memory locally attached to an accelerator via an accelerator architecture. In some examples, 552 includes 554-558 or 560-562. At 554, the network interface can access a descriptor specifying an IOVA (address). At 556, the network interface can use an IOMMU or ATC to convert the IOVA (address) and use a routing table to identify an auxiliary header used to access data. At 558, the network interface can use a DMA engine and an auxiliary header to use the accelerator architecture to read data from a memory locally attached to an accelerator.
[0058] Figure 5C Depicted are actions 560 and 562 that may be performed in relation to 552. At 560, the network interface may access a descriptor that specifies an IOVA (address) and also indicates that the network interface is to use an auxiliary header to transfer data or packets using the accelerator architecture. At 562, a DMA engine of the network interface uses the auxiliary header and reads data from memory locally attached to the accelerator using the accelerator architecture.
[0059] Thereafter, the network interface may transmit the data in the packet or the packet to a destination device or memory.
[0060] Figure 6 A system is depicted. The system can use the embodiments described herein to allocate accelerator traffic to accelerator memory via an accelerator architecture rather than using a host to device architecture. System 600 includes a processor 610 that provides processing, operation management, and execution of instructions for system 600. Processor 610 may include any type of microprocessor, central processing unit (CPU), graphics processing unit (GPU), processing core, or other processing hardware that provides processing for system 600, or a combination of processors. Processor 610 controls the overall operation of system 600 and may be or may include one or more programmable general or special microprocessors, digital signal processors (DSPs), programmable controllers, application specific integrated circuits (ASICs), programmable logic devices (PLDs), etc., or a combination of such devices.
[0061] In one example, the system 600 includes an interface 612 coupled to the processor 610, which may represent a higher speed interface or a high throughput interface for system components such as the memory subsystem 620 or the graphics interface component 640 or the accelerator 642 that require a higher bandwidth connection. The interface 612 represents an interface circuit, which may be a separate component or integrated onto the processor die. Where present, the graphics interface 640 is connected to the graphics component interface for providing a visual display to a user of the system 600. In one example, the graphics interface 640 may drive a high definition (HD) display that provides output to the user. High definition may refer to a display having a pixel density of about 100 PPI (pixels per inch) or greater and may include formats such as full HD (e.g., 1080p), a retina display, 4K (ultra high definition or UHD), or other things. In one example, the display may include a touch screen display. In one example, the graphics interface 640 generates a display based on data stored in the memory 630 or based on operations performed by the processor 610 or both. In one example, graphics interface 640 generates a display based on data stored in memory 630 or based on operations performed by processor 610 , or both.
[0062] The accelerator 642 may be a fixed function offload engine that may be accessed or used by the processor 610. For example, an accelerator among the accelerators 642 may provide: compression (DC) capabilities; cryptographic services such as public key encryption (PKE), cryptography, hashing / authentication capabilities, decryption; or other capabilities or services. In some embodiments, in addition or alternatively, an accelerator among the accelerators 642 provides a field selection controller capability as described herein. In some cases, the accelerator 642 may be integrated into a CPU socket (e.g., a connector to a motherboard or circuit board that includes a CPU and provides an electrical interface with the CPU). For example, the accelerator 642 may include a single-core or multi-core processor, a graphics processing unit, a logic execution unit, a single-level or multi-level cache memory, a functional unit that can be used to independently execute a program or thread, an application specific integrated circuit (ASIC), a neural network processor (NNP), programmable control logic, and a programmable processing element such as a field programmable gate array (FPGA). The accelerator 642 may provide multiple neural networks, CPUs, processor cores, general-purpose graphics processing units, or may enable the graphics processing unit to be utilized by an artificial intelligence (AI) or machine learning (ML) model. For example, the AI model may use or include any one or combination of the following: a reinforcement learning scheme, a Q learning scheme, a deep Q learning or an asynchronous dominant actor-evaluator (A3C), a combination neural network, a recurrent combination neural network, or other AI or ML models. Multiple neural networks, processor cores, or graphics processing units may be made available to the AI or ML model.
[0063] The memory subsystem 620 represents the main memory of the system 600 and provides storage for code to be executed by the processor 610 or data values to be used in executing routines. The memory subsystem 620 may include one or more memory devices 630, such as read-only memory (ROM), flash memory, one or more random access memories (RAM) such as DRAM, or other memory devices, or a combination of such devices. The memory 630 stores and hosts, among others, an operating system (OS) 632 to provide a software platform for the execution of instructions in the system 600. In addition, applications 634 can be executed from the memory 630 on the software platform of the OS 632. Applications 634 represent implemented programs with their own operating logic to perform one or more functions. Processes 636 represent agents or routines that provide auxiliary functions to the OS 632 or one or more applications 634 or a combination. The OS 632, applications 634, and processes 636 provide software logic to provide functions for the system 600. In one example, the memory subsystem 620 includes a memory controller 622, which is a memory controller used to generate commands and issue commands to the memory 630. It will be appreciated that the memory controller 622 may be a physical part of the processor 610 or a physical part of the interface 612. For example, the memory controller 622 may be an integrated memory controller integrated onto a circuit with the processor 610.
[0064] Although not specifically described, it will be understood that system 600 may include one or more buses or bus systems between devices, such as a memory bus, a graphics bus, an interface bus, or other things. The bus or other signal lines may couple components together in a communication mode or in an electrical mode, or couple components in a communication mode and in an electrical mode. The bus may include a physical communication line, a point-to-point connection, a bridge, an adapter, a controller, or other circuit systems or combinations. The bus may include, for example, a system bus, a peripheral component interconnect (PCI) bus, a hypertransport or industry standard architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal serial bus (USB), or one or more of the Institute of Electrical and Electronics Engineers (IEEE) standard 1394 bus (FireWire).
[0065] In one example, system 600 includes interface 614, which can be coupled to interface 612. In one example, interface 614 represents interface circuit, which can include independent components and integrated circuit systems. In one example, multiple user interface components or peripheral components or both are coupled to interface 614. Network interface 650 provides system 600 with the ability to communicate with remote devices (such as servers or other computing devices) through one or more networks. Network interface 650 can include Ethernet adapter, wireless interconnection component, cellular network interconnection component, USB (universal serial bus) or other wired or wireless standard or proprietary interface. Network interface 650 can transmit data to devices or remote devices in the same data center or rack, which can include sending data stored in memory. Network interface 650 can receive data from remote devices, which can include storing received data in memory. Various embodiments can be used in relation to network interface 650, processor 610 and memory subsystem 620.
[0066] In one example, the system 600 includes one or more input / output (I / O) interfaces 660. The I / O interfaces 660 may include one or more interface components through which a user interacts with the system 600 (e.g., audio, alphanumeric, tactile / touch, or other interface connections). The peripheral interfaces 670 may include any hardware interfaces not specifically mentioned above. Peripheral devices generally refer to devices that are relatedly connected to the system 600. Related connections are connections in which the system 600 provides a software platform or a hardware platform or both on which operations are executed and the user interacts with.
[0067] In one example, the system 600 includes a storage subsystem 680 for storing data in a non-volatile manner. In one example, in certain system implementations, at least some components of the storage device 680 may overlap with components of the memory subsystem 620. The storage subsystem 680 includes (one or more) storage devices 684, which may be or may include any conventional media for storing large amounts of data in a non-volatile manner, such as one or more magnetic, solid-state, or optical-based disks or a combination. The storage device 684 maintains code or instructions and data 686 in a persistent state (i.e., the values are retained despite interruptions to the power to the system 600). The storage device 684 may be generally considered to be "memory," although the memory 630 is generally used to provide execution or operating memory for instructions to the processor 610. Although the storage device 684 is non-volatile, the memory 630 may include volatile memory (i.e., if the power to the system 600 is interrupted, the value or state of the data is indeterminate). In one example, storage subsystem 680 includes controller 682 to interface with storage device 684. In one example, controller 682 is a physical part of interface 614 or processor 610 or may include circuitry or logic in both processor 610 and interface 614.
[0068] Volatile memory is memory whose state (and therefore the data stored therein) is uncertain if power to the device is interrupted. Dynamic volatile memory requires refreshing of the data stored in the device to maintain the state. An example of dynamic volatile memory includes DRAM (dynamic random access memory), or some variants such as synchronous DRAM (SDRAM). The memory subsystem as described herein is compatible with many memory technologies, such as DDR3 (Double Data Rate Version 3, originally released by JEDEC (Joint Electron Device Engineering Council) on June 27, 2007), DDR4 (DDR Version 4, initial specification published by JEDEC in September 2012), DDR4E (DDR Version 4), LPDDR3 (Low Power DDR Version 3, JESD209-3B, published by JEDEC in August 2013), LPDDR4 (LPDDR Version 4, JESD2 09-4, originally published by JEDEC in August 2014), WIO2 (Wide Input / Output Version 2, JESD229-2, originally published by JEDEC in August 2014), HBM (High Bandwidth Memory, JESD325, originally published by JEDEC in October 2013), LPDDR5 (currently under discussion by JEDEC), HBM2 (HBM Version 2, currently under discussion by JEDEC), or other things or combinations of memory technologies, and technologies based on derivatives or extensions of such specifications. JEDEC standards are available at www.jedec.org.
[0069] A non-volatile memory (NVM) device is a memory whose state is determined even if power to the device is interrupted. In one embodiment, the NVM device may include a block-addressable memory device such as NAND technology, or more specifically, may include a multi-threshold level NAND flash memory (e.g., single-level cell ("SLC"), multi-level cell ("MLC"), quad-level cell ("QLC"), triple-level cell ("TLC"), or other NAND). NVM devices may also include byte-addressable write-to-bit three-dimensional cross-point memory devices or other byte-addressable write-to-bit NVM devices (also referred to as permanent memory), such as single-level or multi-level phase change memory (PCM) or phase change memory with switches (PCMS), NVM devices using chalcogenide phase change materials (e.g., chalcogenide glass), resistive memories including metal oxide-based, oxygen vacancy-based and conductive bridge random access memories (CB-RAM), nanowire memories, ferroelectric random access memories (FeRAM, FRAM), magnetoresistive random access memories (MRAM) combined with memristor technology, spin transfer torque (STT)-MRAM, devices based on spin electronic magnetic junction memory, devices based on magnetic tunnel junction (MTJ), devices based on DW (domain wall) and SOT (spin-orbit transfer), thyristor-based memory devices, or any combination of the above, or other memories.
[0070] A power source (not depicted) provides power to the components of system 600. More specifically, the power source generally interfaces to one or more power supplies in system 600 to provide power to the components of system 600. In one example, the power supply includes an AC to DC (alternating current to direct current) adapter to plug into a wall outlet. Such an AC power source can be a renewable energy (e.g., solar) power source. In one example, the power source includes a DC power source such as an external AC to DC converter. In one example, the power source or power supply includes wireless charging hardware to charge via proximity to a charging field. In one example, the power source can include an internal battery, an alternating current supply, a motion-based power supply, a solar power supply, or a fuel cell source.
[0071] In an example, the system 600 may be implemented using an interconnected computing sled of processors, memory, storage devices, network interfaces, and other components. A high-speed interconnect such as PCIe, Ethernet, or optical interconnect (or a combination thereof) may be used.
[0072] The embodiments herein can be implemented in various types of computing and networking devices such as switches, routers, racks, and blade servers, such as those employed in data center and / or server farm environments. The servers used in data centers and server farms include server configurations such as rack-based servers or arrangements of blade servers. These servers are communicatively interconnected by means of various network supplies, such as dividing a collection of servers into local area networks (LANs), with appropriate switching and routing facilities between LANs to form a private intranet. For example, cloud hosting facilities generally employ large data centers with multiple servers. The blades include separate computing platforms, i.e., "servers on cards," configured to perform server-type functions. Therefore, each blade includes components common to conventional servers, including a main printed circuit board (motherboard) that provides internal wiring (i.e., a bus) for coupling appropriate integrated circuits (ICs) and other components mounted on the board.
[0073] Figure 7 A network interface that can be used or used by an embodiment is depicted. The network interface 700 may include a transceiver 702, a processor 704, a transmit queue 706, a receive queue 708, a memory 710, and a bus interface 712 and a DMA engine 752. The transceiver 702 may be capable of receiving and transmitting packets according to an applicable protocol such as Ethernet described in IEEE 802.3, although other protocols may be used. The transceiver 702 may receive packets from a network and transmit packets to a network via a network medium (not depicted). The transceiver 702 may include a PHY circuit system 714 and a media access control (MAC) circuit system 716. The PHY circuit system 714 may include an encoding and decoding circuit system (not shown) to encode and decode data packets according to an applicable physical layer specification or standard. The MAC circuit system 716 may be configured to assemble the data to be transmitted into packets, the packets including a destination address and a source address together with network control information and an error detection hash value. The processor 704 may be any combination of a processor, a core, a graphics processing unit (GPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or other programmable hardware device that allows programming of the network interface 700. For example, the processor 704 may provide identification of resources to be used to execute a workload and provide generation of a bitstream for execution on selected resources. For example, an "intelligent network interface" may use the processor 704 to provide packet processing capabilities in the network interface.
[0074] The packet distributor 724 can use the timeslot allocation or RSS described herein to provide distribution of received packets for processing by multiple CPUs or cores. When the packet distributor 724 uses RSS, the packet distributor 724 can calculate a hash or make another determination based on the content of the received packet to determine which CPU or core is going to process the packet.
[0075] Interrupt coalescing 722 may perform interrupt moderation, whereby the network interface interrupt coalescing 722 waits for multiple packets to arrive or for a timeout to expire before generating an interrupt to the host system to process (one or more) received packets. Receive segment coalescing (RSC) may be performed by the network interface 700, whereby portions of incoming packets are combined into segments of a packet. The network interface 700 provides this coalesced packet to the application.
[0076] Direct memory access (DMA) engine 752 can copy packet header, packet payload and / or descriptor directly from host memory to network interface or vice versa, instead of copying the packet to an intermediate buffer at the host and then using another copy operation from the intermediate buffer to the destination buffer. In some embodiments, multiple DMA engines can be used to transfer the content of the packet to a destination memory associated with a host device or a destination memory associated with an accelerator device. Address conversion 754 can be used to determine whether the conversion of the address provided with the received packet is available in the cache memory of the network interface 700.
[0077] The memory 710 may be any type of volatile or non-volatile memory device and may store any queue or instruction used to program the network interface 700. The transmit queue 706 may include data or a reference to data for transmission by the network interface. The receive queue 708 may include data or a reference to data received by the network interface from the network. The descriptor queue 720 may include a descriptor that references the data or packets in the transmit queue 706 or receive queue 708 and the corresponding destination storage area. The bus interface 712 may provide an interface with a host device (not depicted). For example, the bus interface 712 may be compatible with PCI, PCI Express, PCI-x, Serial ATA, and / or USB compatible interfaces (although other interconnect standards may be used).
[0078] Figure 8 An example of a data center is depicted. Various embodiments may be used in Figure 8 data center or can be connected with Figure 8 data centers to provide data transfer to memory via an accelerator architecture. Figure 8As shown, the data center 800 may include an optical architecture 812. The optical architecture 812 may generally include a combination of optical signal transmission media (such as optical cable wiring) and optical switching infrastructure, by means of which any particular rail in the data center 800 can send signals to other rails in the data center 800 (and receive signals from other rails in the data center 800). The signal transmission connectivity provided by the optical architecture 812 to any given rail may include connectivity to both other rails in the same rack and rails in other racks. The data center 800 includes four racks 802A to 802D and the racks 802A to 802D accommodate corresponding pairs of rails 804A-1 and 804A-2, 804B-1 and 804B-2, 804C-1 and 804C-2, and 804D-1 and 804D-2. Therefore, in this example, the data center 800 includes a total of eight rails. The optical architecture 812 may provide rail signal transmission connectivity with one or more of the seven other rails. For example, via the optical architecture 812, the rail 804A-1 in the rack 802A can have signal transmission connectivity with the rail 804A-2 in the rack 802A and six other rails 804B-1, 804B-2, 804C-1, 804C-2, 804D-1, and 804D-2 distributed to other racks 802B, 802C, and 802D in the data center 800. Embodiments are not limited to this example. For example, the architecture 812 can provide optical and / or electrical signal transmission.
[0079] Various examples can be realized using hardware elements, software elements or a combination of the two. In some examples, hardware elements can include equipment, parts, processors, microprocessors, circuits, circuit elements (such as transistors, resistors, capacitors, inductors, etc.), integrated circuits, ASICs, PLDs, DSPs, FPGAs, memory cells, logic gates, registers, semiconductor devices, chips, microchips, chipsets, etc. In some examples, software elements can include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, processes, software interfaces, APIs, instruction sets, computing codes, computer codes, code segments, computer code segments, words, values, symbols or any combination thereof. As desired for a given implementation, determining whether to use hardware elements and / or software elements to realize an example can vary according to factors such as desired computing rate, power level, heat resistance, processing cycle budget, input data rate, output data rate, memory resources, data bus speed and other design or performance constraints of any quantity. Note that hardware, firmware, and / or software elements may be collectively or individually referred to herein as “modules,” “logic,” “circuits,” or “circuitry.” A processor may be a hardware state machine, digital control logic, a central processing unit, or one or more combinations of any hardware, firmware, and / or software elements.
[0080] Some examples may be implemented using an article or at least one computer-readable medium or some examples may be implemented as an article or at least one computer-readable medium. Computer-readable media may include non-transitory storage media for storing logic. In some examples, non-transitory storage media may include one or more types of computer-readable storage media capable of storing electronic data, including volatile memory or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, etc. In some examples, logic may include various software elements, such as software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, processes, software interfaces, APIs, instruction sets, computing codes, computer codes, code segments, computer code segments, words, values, symbols, or any combination thereof.
[0081] According to some examples, computer-readable media may include non-transitory storage media for storing or maintaining instructions, which, when executed by a machine, computing device, or system, cause the machine, computing device, or system to perform methods and / or operations according to the described examples. Instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, etc. Instructions may be implemented according to a predefined computer language, manner, or syntax for commanding a machine, computing device, or system to perform a certain function. Instructions may be implemented using any suitable high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language.
[0082] One or more aspects of at least one example may be implemented by representative instructions stored on at least one machine-readable medium, the representative instructions representing various logic within a processor, which when read by a machine, computing device, or system causes the machine, computing device, or system to fabricate logic to perform the techniques described herein. Such representations, known as "IP cores," may be stored on a tangible machine-readable medium and provided to various customers or manufacturing facilities to load into a manufacturing machine that actually fabricates the logic or processor.
[0083] The appearance of the phrase "one example" or "example" does not necessarily refer to the same example or embodiment in all cases. Any aspect described herein may be combined with any other aspect or similar aspect described herein, regardless of whether the aspect is described with respect to the same figures or elements. The division, omission, or inclusion of block functions depicted in the figures does not infer that hardware components, circuits, software, and / or elements for implementing these functions will necessarily be divided, omitted, or included in the embodiments.
[0084] Some examples may be described using the expressions "coupled" and "connected," and their derivatives. These terms are not necessarily intended to be synonyms for each other. For example, descriptions using the terms "connected" and / or "coupled" may indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" may also refer to two or more elements that are not in direct contact with each other, but still cooperate or interact with each other.
[0085] The terms "first", "second", etc. do not represent any order, quantity or importance in this article, but are used to distinguish one element from another element. The terms "a" and "an" do not represent the limitation of quantity in this article, but instead represent the existence of at least one of the referenced items. The term "asserted" used in this article about a signal represents the state of a signal, wherein the signal is active, and this can be achieved by applying any logic level of either logic 0 or logic 1 to the signal. The term "following" or "after..." can refer to following or following another event or some other event. According to alternative embodiments, other sequences of steps can also be performed. In addition, additional steps can be added or removed according to specific applications. Any combination of changes can be used and those of ordinary skill in the art who benefit from the present disclosure will understand its many changes, modifications and alternative embodiments.
[0086] Unless expressly stated otherwise, as is commonly used, disjunctive language such as the phrase "at least one of X, Y, or Z" is otherwise understood within the context to indicate that an item, term, etc. can be either X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Thus, such disjunctive language is generally not intended to, and should not, imply that certain embodiments require that at least one of X, at least one of Y, or at least one of Z each be present. Additionally, conjunctive language such as the phrase "at least one of X, Y, and Z" should also be understood to refer to X, Y, Z, or any combination thereof, including "X, Y, and / or Z," unless expressly stated otherwise.
[0087] Illustrative examples of the devices, systems, and methods disclosed herein are provided below. Embodiments of the devices, systems, and methods may include any one or more and any combination of the examples described below.
[0088] Example 1 includes a network interface, the network interface being configured to include: an interface to a host-to-device architecture; an interface to an accelerator architecture; and at least one processor, the at least one processor being configured to: process a received packet to determine a destination memory location; request an address translation for the destination memory location; request decoding of the address translation to determine a header to be used from a primary header or one or more auxiliary headers; and based on the availability of the address translation for the destination memory location and the determined header indicating the auxiliary header, allow a portion of the received packet to be copied to a memory locally attached to the accelerator via the accelerator architecture.
[0089] Example 2 includes any example and includes at least one port to receive or send a packet and a direct memory access (DMA) engine to copy a received packet to a memory buffer via the host-to-device architecture or to copy a portion of a received packet to a memory locally attached to the accelerator via the accelerator architecture.
[0090] Example 3 includes any example and includes an address cache memory, and wherein the at least one processor is to: determine whether the address cache memory stores the destination memory location and the address translation and based on the address cache memory not storing the address translation of the destination memory location, cause a request for the address translation to be transmitted to an address translation service.
[0091] Example 4 includes any example wherein the address translation service comprises an input-output memory management unit (IOMMU).
[0092] Example 5 includes any example, wherein the at least one processor is to: discard the received packet based on the address translation service indicating that there is no address translation for the destination memory location.
[0093] Example 6 includes any example wherein the at least one processor is to: based on the address translation service providing an address translation of the destination memory location and the determined header identifying a header to be used, cause the DMA engine to copy a portion of the received packet to a memory buffer associated with the address translation.
[0094] Example 7 includes any of the examples and includes an accelerator architecture coupled to the network interface, an accelerator coupled to the accelerator architecture, and a memory locally attached to the accelerator, the memory locally attached to the accelerator to receive the portion of the received packet.
[0095] Example 8 includes any of the examples and includes a host-to-device architecture coupled to the network interface, a central processing unit (CPU) coupled to the host-to-device architecture, and an accelerator coupled to the host-to-device architecture.
[0096] Example 9 includes any of the examples and includes an interface to a data center network to receive a packet from a remote entity for copying to a memory buffer.
[0097] Example 10 includes any example and wherein the accelerator includes one or more of: a general purpose graphics processing unit (GPGPU), a graphics processing unit (GPU), compression, cryptography, hashing / authentication, decryption, image recognition, speech recognition, or neural network-based inference.
[0098] Example 11 includes any example and includes a rack, a server, a data center, or a slide.
[0099] Example 12 includes a computer-implemented method, the computer-implemented method comprising: configuring a network interface to determine whether to copy a portion of a received packet from a remote entity directly to a memory buffer associated with an accelerator via an accelerator architecture, wherein the accelerator architecture provides a communication coupling between one or more accelerators; receiving a packet from the remote entity at the network interface, the packet including a destination memory address; determining whether an address translation of the destination memory address of the packet is available; determining a header to be used to transmit the portion of the packet; based on the address translation being available and the determined header, performing a direct memory access (DMA) copy operation of the portion of the packet to a memory buffer associated with the address translation via the accelerator architecture; and discarding the packet based on the address translation being unavailable.
[0100] Example 13 includes any example and wherein configuring the network interface to determine whether to copy a portion of a received packet from a remote entity directly to a memory buffer via an accelerator architecture comprises using control plane software.
[0101] Example 14 includes any example and wherein determining whether an address translation for the destination memory address of the packet is available includes checking whether a cached copy of the address translation for the destination memory address is available at a network interface, and based on the cached copy of the address translation for the destination memory address being not available at the network interface, requesting an address translation service for the address translation for the destination memory address via a host-to-device architecture.
[0102] Example 15 includes any of the examples and wherein the address translation service comprises an input-output memory management unit (IOMMU).
[0103] Example 16 includes any example and wherein the accelerator architecture is coupled to at least one memory and at least one accelerator device, and wherein the memory is locally attached to the accelerator device.
[0104] Example 17 includes any example and includes: based on the received packet not being associated with a memory buffer associated with the accelerator, using direct memory access (DMA) to copy a portion of the received packet to a memory buffer associated with the core via a host-to-device architecture, wherein the host-to-device architecture couples the core to the accelerator.
[0105] Example 18 includes a system comprising: a network interface controller; a host-to-device architecture coupled to the network interface controller, the host-to-device architecture providing a communication coupling between a central processing unit and an accelerator; an accelerator architecture coupled to the network interface controller, the accelerator architecture providing a communication coupling between multiple accelerators; a memory coupled to the accelerator device, wherein the network interface controller includes at least one direct memory access (DMA) engine, and the network interface is configured to: determine whether to copy a portion of a received packet from a remote entity directly to a memory buffer associated with the accelerator via the accelerator architecture, wherein the accelerator architecture provides a communication coupling between one or more accelerators, process the received packet to determine an associated destination memory location, request an address translation of the destination memory location, determine a header to be used to transmit a portion of the received packet from a primary header and an auxiliary header, and based on the availability of the address translation of the destination memory location and the determination that the header is an auxiliary header, allow the use of the DMA engine to copy a portion of the received packet directly to a memory buffer associated with the address translation via the accelerator architecture.
[0106] Example 19 includes any example and wherein determining whether to copy a portion of a received packet from a remote entity directly to a memory buffer associated with the accelerator via an accelerator architecture includes determining whether a cached copy of an address translation for a destination memory address is available at the network interface, and based on the cached copy of the address translation for the destination memory address being not available at the network interface, requesting an address translation service for the address translation for the destination memory address via the host-to-device architecture.
[0107] Example 20 includes any example and wherein the address translation comprises an input-output memory management unit (IOMMU).
[0108] Example 21 includes any example and wherein the network interface is configured to: based on the received packet not being associated with a memory buffer associated with the accelerator, copy a portion of the received packet to a memory buffer associated with the core using a DMA engine via a host-to-device architecture.
[0109] Example 22 includes any example and wherein to process a received packet to determine an associated destination memory location, the network interface is to access a descriptor that specifies a destination address, and wherein if the network interface determines to use an auxiliary header, the network interface is to use a direct memory access (DMA) engine to copy a portion of the received packet to a memory buffer associated with the address translation via an accelerator architecture using the auxiliary header.
[0110] Example 23 includes any example and wherein, to process a received packet to determine an associated destination memory location, the network interface accesses a descriptor that specifies a destination address and whether to use an auxiliary header, and the network interface uses a direct memory access (DMA) engine to copy a portion of the received packet to a memory buffer associated with the address translation using the auxiliary header via an accelerator architecture based on the descriptor that specifies that the auxiliary header is to be used.
[0111] Example 24 includes any example and wherein the memory buffer is in a memory locally attached to the accelerator device.
Claims
1. A network interface controller circuit system, the network interface controller circuit system being configurable for use in a host node and associated with a device driver, the host node comprising at least one memory accessible to a graphics processing unit (GPU), at least one host memory, and at least one host fabric, the host node being communicatively coupled to a remote system via at least one multi-switch fabric, the remote system comprising at least one other GPU accessible memory, at least one other host memory, and at least one other host fabric, the network interface controller circuit system comprising: a network interface circuit system for initiating remote direct memory access (RoCE) packet data communications over a converged Ethernet network with the remote system via the at least one multi-switch fabric, the RoCE packet data communications being used to indicate at least one remote direct memory access (RDMA) write from the remote system to the host node and / or at least one RDMA read from the host node to the remote system, the RoCE packet data communications being initiated at least in part in response to at least one host application request; as well as Programmable circuitry for performing operations comprising: Where the RoCE packet data communication indicates the at least one RDMA write, directly writing the received packet data to a memory accessible to the at least one GPU via the at least one host architecture; where the RoCE packet data communication indicates the at least one RDMA read, directly reading, via the at least one host architecture, from memory accessible to the at least one GPU, other data to be provided to the remote system via the RoCE packet data communication; and Host central processing unit CPU offload operations related to encryption, decryption and compression; in: The writing and the reading are to be performed in a manner that bypasses both (1) a host CPU and / or a host operating system OS during the writing and the reading and (2) a copying of the received packet data and the other data to the at least one host memory of the host node; The writing and / or the reading can be configured to include the use of direct data placement (DDP); The writing and / or the reading can be configured to include the use of address translation; to implement said address translation at least in part using said device driver; to route said received packet data and / or portions of said other data to their destinations via corresponding architecture-associated routing; The respective architecture-associated routings are configurable to be at least partially different from one another; and The at least one multi-switch fabric is to communicatively couple a plurality of switches associated with the host node and the remote system.
2. The network interface controller circuit system of claim 1, wherein: Prior to being received by the programmable circuit system, the received packet data is directly read from a memory accessible to the at least one other GPU via the at least one other host architecture in a manner that bypasses both (1) a remote system CPU and / or a remote system OS in the remote system and (2) a copy of the received packet data to the at least one other host memory.
3. The network interface controller circuit system of claim 2, wherein: The at least one host architecture comprises a Peripheral Component Interconnect Express (PCIe) interconnect; and The network interface controller circuitry is to be included in a circuit board that is to be communicatively coupled to the PCIe interconnect.
4. The network interface controller circuit system of claim 3, wherein: The received packet data and / or the other data are used in conjunction with artificial intelligence and / or machine learning.
5. The network interface controller circuit system of claim 4, wherein: The host node and the remote system each include a plurality of respective graphics processing units; The at least one GPU-accessible memory is accessible to the plurality of corresponding graphics processing units of the host node; as well as The at least one other GPU-accessible memory is accessible to the plurality of corresponding graphics processing units of the remote system.
6. The network interface controller circuit system of claim 1, wherein: The programmable circuitry is also intended for use in conjunction with memory isolation.
7. The network interface controller circuit system of claim 1, wherein: At least one application specific integrated circuit ASIC comprises said programmable circuitry; and The at least one host architecture includes at least one accelerator architecture.
8. The network interface controller circuit system of any one of claims 1 to 7, wherein: The data center system includes the host node, the remote system, and the at least one multi-switch architecture.
9. A method to be implemented using a network interface controller circuit system, the network interface controller circuit system to be configured for use in a host node and associated with a device driver, the host node comprising at least one memory accessible to a graphics processing unit (GPU), at least one host memory, and at least one host fabric, the host node to be communicatively coupled to a remote system via at least one multi-switch fabric, the remote system comprising at least one other GPU accessible memory, at least one other host memory, and at least one other host fabric, the network interface controller circuit system comprising a network interface circuit system and a programmable circuit system, the method comprising: using the network interface circuitry in remote direct memory access (RoCE) packet data communications over a converged Ethernet network with the remote system via the at least one multi-switch fabric, the RoCE packet data communications being used to indicate at least one remote direct memory access (RDMA) write from the remote system to the host node and / or at least one RDMA read from the host node to the remote system, the RoCE packet data communications being initiated at least in part in response to at least one host application request; as well as Using the programmable circuit system to perform operations, the operations comprising: Where the RoCE packet data communication indicates the at least one RDMA write, directly writing the received packet data to a memory accessible to the at least one GPU via the at least one host architecture; where the RoCE packet data communication indicates the at least one RDMA read, directly reading, via the at least one host architecture, from memory accessible to the at least one GPU, other data to be provided to the remote system via the RoCE packet data communication; and Host central processing unit CPU offload operations related to encryption, decryption and compression; in: The writing and the reading are to be performed in a manner that bypasses both (1) a host CPU and / or a host operating system OS during the writing and the reading and (2) a copying of the received packet data and the other data to the at least one host memory of the host node; The writing and / or the reading can be configured to include the use of direct data placement (DDP); The writing and / or the reading can be configured to include the use of address translation; to implement said address translation at least in part using said device driver; to route said received packet data and / or portions of said other data to their destinations via corresponding architecture-associated routing; The respective architecture-associated routings are configurable to be at least partially different from one another; and The at least one multi-switch fabric is to communicatively couple a plurality of switches associated with the host node and the remote system.
10. The method of claim 9, wherein: Prior to being received by the programmable circuit system, the received packet data is directly read from a memory accessible to the at least one other GPU via the at least one other host architecture in a manner that bypasses both (1) a remote system CPU and / or a remote system OS in the remote system and (2) a copy of the received packet data to the at least one other host memory.
11. The method of claim 10, wherein: The at least one host architecture comprises a Peripheral Component Interconnect Express (PCIe) interconnect; and The network interface controller circuitry is to be included in a circuit board that is to be communicatively coupled to the PCIe interconnect.
12. The method of claim 11, wherein: The received packet data and / or the other data are used in conjunction with artificial intelligence and / or machine learning.
13. The method of claim 12, wherein: The host node and the remote system each include a plurality of respective graphics processing units; The at least one GPU-accessible memory is accessible to the plurality of corresponding graphics processing units of the host node; as well as The at least one other GPU-accessible memory is accessible to the plurality of corresponding graphics processing units of the remote system.
14. The method of claim 9, wherein: The programmable circuitry is also used in conjunction with memory isolation.
15. The method of claim 9, wherein: At least one application specific integrated circuit ASIC comprises said programmable circuitry; and The at least one host architecture includes at least one accelerator architecture.
16. The method of claim 9, wherein: The data center system includes the host node, the remote system, and the at least one multi-switch architecture.
17. At least one machine-readable storage medium storing instructions which, when executed by at least one machine, result in the execution of the method of any one of claims 9 to 16.
18. An apparatus comprising means for performing the method of any one of claims 9 to 16.
19. A computer program product comprising instructions which, when executed by at least one processor, cause the at least one processor to perform the method of any one of claims 9 to 16.
Citation Information
Cited By
Circuit, data processing method, equipment, medium and program product
CN121233494A
