Atomic memory operations

Abstract

Systems and methods for atomic memory operations in a remote direct memory access network are disclosed. A system includes a network interface card (NIC) comprising a first memory and one or more processors coupled to the first memory. The one or more processors receive an atomic memory operation (AMO) remote procedure call (RPC) comprising a memory address and an AMO type. The one or more processors are also to retrieve, from a second memory, a value corresponding to the memory address of the AMO RPC. The one or more processors are also to perform an AMO on the value from the second memory corresponding to the AMO type to obtain a modified value. The one or more processors are also to store the modified value in the first memory.

Description

Technical Field

[0001] At least one embodiment relates to performing atomic memory operations in a remote direct memory access (RDMA, direct memory access) network, and more specifically to performing atomic memory operations using a network interface controller (NIC). Background Technology

[0002] Processing devices in a Remote Direct Memory Access (RDMA) network can be connected (e.g., via one or more network connections) so that a first processing device can access (e.g., read, write, etc.) the memory of a second processing device. Memory access and / or modification can be performed as a unilateral operation using only the processing unit of the requesting device. For example, the first processing device can access the memory of the second processing device without the involvement of the main processing unit of the second processing device (e.g., a central processing unit (CPU)). This allows the main processing unit of the second processing device to freely perform operations independent of memory accesses from the first processing device. Attached Figure Description

[0003] Figure 1 This is a block diagram of an example system for improving AMO performance in an RDMA network according to at least one embodiment.

[0004] Figure 2 This is a flowchart of an example method for improving AMO performance in an RDMA network according to at least one embodiment.

[0005] Figure 3 This is a flowchart of an example method for improving AMO performance according to at least one embodiment.

[0006] Figure 4 This is a flowchart of an example method for performing memory operations within a host having a DPU, according to at least one embodiment.

[0007] Figure 5 A block diagram of an exemplary computer system according to at least one embodiment is illustrated.

[0008] Figure 6 A block diagram of an electronic device utilizing a processor is illustrated according to at least one embodiment.

[0009] Figure 7A It is a block diagram of an example network architecture according to at least one embodiment.

[0010] Figure 7B It is a block diagram of an example network architecture according to at least one embodiment.

[0011] Figure 8The illustration depicts a distributed system according to at least some embodiments.

[0012] Figure 9 An example data center according to at least one embodiment is illustrated.

[0013] Figure 10 The illustration depicts a client-server network consisting of multiple interconnected network server computers according to at least one embodiment.

[0014] Figure 11 The illustration depicts a computer network connecting one or more computing machines according to at least some embodiments.

[0015] Figure 12A The illustration depicts a networked computer system according to at least some embodiments.

[0016] Figure 12B The illustration depicts a networked computer system according to at least some embodiments.

[0017] Figure 12C The illustration depicts a networked computer system according to at least some embodiments.

[0018] Figure 13 It is a block diagram of a computing system having two processing devices coupled to each other and multiple networks according to at least one embodiment.

[0019] Figure 14 It is a block diagram of a computing system having a CPU and a GPU in a single integrated circuit according to at least one embodiment.

[0020] Figure 15 It is a block diagram of a computing system with a tensor core GPU according to at least one embodiment. Detailed Implementation

[0021] Processing devices in an RDMA network may include a network interface card (NIC) configured to perform atomic memory operations (AMOs) on the RDMA network. For example, the NIC may be configured to receive an RDMA packet, retrieve a value from memory associated with the host processor based on the RDMA packet, perform one or more atomic operations on that value, store the modified value in the memory associated with the host processor, and return the modified value to the requesting device. Once the modified value has been stored in the memory associated with the host processor and returned to the requesting device, the AMO can be completed. However, the host memory from the NIC to each AMO can potentially cause performance bottlenecks.

[0022] The aspects of this disclosure address the aforementioned and other shortcomings by improving the performance of AMO in RDMA networks. More specifically, the NIC of the processing device in the RDMA network can be replaced by a NIC referred to as a "smart NIC," which includes a data processing unit (DPU) comprising one or more processing units and memory. The DPU's memory can be used as a cache for values ​​from the AMO received from the RDMA network, and the AMO can be performed by one or more processing units of the DPU, thereby improving the performance of AMO in the RDMA network. For example, the AMO can be directed to host memory associated with a host processing device (e.g., CPU or GPU). Upon receiving the AMO, the NIC's DPU can retrieve the value from the host memory, modify the value, store the modified value in the DPU's local memory, and return the modified value to the remote requester. Notably, the AMO can be performed without first writing the updated value to the host memory. Additionally, if a subsequent AMO is received pointing to the same memory address, the DPU can retrieve the modified value from its local memory, further modify the value, store the further modified value in local memory, and return the further modified value to the remote requester. Thus, in some embodiments, the AMO can be performed entirely on the NIC without reading from or writing to host memory. In various embodiments, this can reduce the amount of time spent completing the AMO.

[0023] In various embodiments, the processing device of the RDMA network can be configured to send an AMO remote procedure call (RPC) in place of (or appended to) an RDMA packet. The AMO RPC may include a memory address to which one or more operations are performed and an AMO type. Upon receiving an AMO RPC, the DPU can retrieve a value corresponding to the memory address of the AMO RPC from the memory of the processing device hosting the DPU (assuming the value is not already cached in the DPU's memory). The AMO of the AMO RPC can be performed by one or more processing units of the DPU, and the resulting modified value can be stored in the DPU's memory. Therefore, the AMO can be performed without writing the updated value to the host processing device's memory. In various embodiments, the DPU's memory can act as a cache. If another AMO RPC is received and targets the same memory address (or another memory address still stored in the DPU's memory), the DPU can perform the AMO using one or more processing units of the DPU and the value stored in the DPU's memory (e.g., without loading the value from the memory of the processing device hosting the DPU).

[0024] The DPU can periodically refresh its memory to the memory of the processing device hosting the DPU. For example, the DPU can refresh its memory after each AMO. In some embodiments, the DPU can refresh its memory after receiving a refresh memory instruction (e.g., from a remote device in an RDMA network, a processing unit in a host processing device, etc.). In some embodiments, the DPU can refresh its memory after its memory occupancy meets an occupancy criterion. For example, the DPU might refresh its memory after 90% of its memory is filled (e.g., "occupied"). In some embodiments, the DPU can refresh its memory in response to receiving a synchronization instruction (e.g., from a remote device in an RDMA network). In some embodiments, the DPU can refresh its memory in response to a host processing device attempting to perform one or more operations on a memory address cached in the DPU's memory. For example, the host processing device (e.g., software running on the host processing device) can send a signal to the DPU indicating a memory address to be accessed. If the memory address is cached in the DPU's memory, the DPU can refresh its memory so that the host processing device can access the current value at the memory address. This ensures that the host processing device does not operate on expired data.

[0025] In some embodiments, one or more hardware circuits that can monitor memory access requests may be coupled between the DPU and the host processing device. If one or more hardware circuits detect a memory access request from the host processing device for a memory address cached in the DPU's memory, one or more hardware circuits may trigger a DPU memory refresh before the memory address is accessed by the host processing device.

[0026] Therefore, the DPU's memory can be used as a cache for AMO in an RDMA network, and one or more processing units of the DPU can perform AMO efficiently.

[0027] The advantages of the disclosed technology include, but are not limited to, improved AMO performance in RDMA networks.

[0028] Figure 1This is a block diagram of an example system 100 for improving AMO performance in an RDMA network according to at least one embodiment. System 100 may include a target node 102 and a remote node 122 connected via a network 128. Network 128 may be a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or a wide area network (WAN)), a wired network (e.g., Ethernet), a wireless network (e.g., an 802.11 network or a Wi-Fi network), a cellular network (e.g., a Long Term Evolution (LTE) network), a router, a hub, a switch, a server computer, and / or a combination thereof. In some embodiments, remote node 122 and target node 102 are nodes in a data center, and network 128 includes one or more switch layers connecting multiple nodes in the data center. For example, remote node 122 and target node 102 may each be a server device in a data center. In some embodiments, target node 102 and remote node 122 may be part of a Remote Direct Memory Access (RDMA) network. For example, remote node 122 can access the memory of target node 102 (e.g., host memory 104, memory 110) via network 128 without the host processor 106 performing any operation. In some embodiments, additional processing devices (e.g., additional nodes) may be included in the RDMA network and may be connected via network 128.

[0029] Target node 102 may include one or more host processors 106, host memory 104, and network interface cards (NICs) 108, as well as other components. In some embodiments, target node 102 may be a desktop computer, server, laptop computer, mobile device, data center processing equipment, etc. Host processor 106 may be used to perform one or more operations (e.g., execute one or more programs or applications). Host processor 106 may be connected to host memory 104 via memory access 116 and may perform operations on host memory 104, provided that the accessed memory address is not currently cached in memory 110. If the memory address is cached in memory 110, host processor 106 may trigger a memory refresh so that the value from memory 110 is written back to host memory 104.

[0030] Host memory 104 may include at least one of flash memory or random access memory (RAM), such as dynamic RAM (DRAM) or synchronous DRAM (SDRAM). In some embodiments, host memory 104 may be accessed by a remote device within an RDMA network.

[0031] NIC 108 may include memory 110 and one or more processors 112. In some embodiments, one or more processors 112 include a Data Processing Unit (DPU). A DPU is a dedicated processor designed to handle data center tasks, complementing the traditional central processing unit (CPU) and graphics processing unit (GPU) in modern computing systems. DPUs are optimized for offloading, accelerating, and managing data processing tasks typically associated with networking, storage, and security functions. DPUs are particularly valuable in cloud computing, data centers, and environments requiring high-performance data processing. Examples of operations that a DPU can perform include network offloading (e.g., handling network packet processing), storage acceleration (e.g., managing data movement between storage and computing resources), security processing, virtualization, and so on. In various embodiments, processor 112 may perform atomic memory operations (AMOs) received from remote devices within an RDMA network (e.g., from remote node 122). In some embodiments, the memory 110 of NIC 108 (e.g., the DPU) may act as a cache for AMOs. For example, a memory value accessed during an AMO can be retrieved from host memory 104 (e.g., external memory) and cached in memory 110 (e.g., local memory). The processor 112 can then perform the AMO based on the value in memory 110 without having to write the updated value to host memory 104 to complete the AMO, and in some cases, without having to read the value from host memory 104 to perform the AMO (for subsequent AMOs that are directed to the memory address of host memory 104 cached in memory 110). Periodically, the value cached in memory 110 can be flushed (e.g., written, stored, etc.) back to host memory 104.

[0032] In various embodiments, host processor 106 can perform AMO on memory addresses in host memory 104. In some instances, such as for memory addresses not currently cached in memory 110, such AMO can be performed without sending messages between NIC 108 and host processor 106. If host processor 106 attempts to perform AMO or other operations on memory addresses in host memory 104 that are cached in memory 110, this can trigger a refresh operation to refresh the cache (e.g., the value of the address in memory 110 corresponding to the memory address in host memory 104). For example, host processor 106 can query NIC 108 and / or memory 110 to determine whether a specific memory address is cached in memory 110. If the memory address is cached in memory 110, host processor 106 can signal to NIC 108 that memory 110 be refreshed to host memory 104. Once host memory 104 contains the latest values ​​of its memory addresses, host processor 106 can perform operations on these values ​​or use these values ​​to perform operations.

[0033] AMO can be executed atomically, meaning that once an operation begins, it cannot be interrupted by another process or thread. In some embodiments, to perform AMO, access to memory values ​​and / or addresses used for AMO can be temporarily restricted to ensure consistency of results. For example, one or more memory addresses can be "locked," and then the processor performing the AMO can read the value at the memory address, perform the AMO, write the new value back to the memory address, and "unlock" the memory address. In some embodiments, semaphores, memory barriers, etc., can be used to guarantee the atomic execution of AMO. The processor can ensure that the execution of AMO is not interrupted by other processors or processing threads.

[0034] NIC 108 can receive AMO remote procedure calls (RPCs) from remote NIC 124 of remote node 122. An AMO RPC may include an AMO type and a memory address. The AMO type identifies the AMO that should be performed on a value stored at a memory address in host memory 104. For example, the AMO corresponding to the AMO type may be a compare-and-swap operation, a fetch-and-add operation, a fetch-and-store operation, a fetch-and-XOR operation, an atomic increment operation, an atomic decrement operation, a swap operation, a software-defined operation, etc. In some embodiments, a software-defined operation may include a "load-link / conditional store" operation that can be performed by a processor (e.g., processor 112 of NIC 108).

[0035] In some cases, an AMO RPC may also include one or more AMO operators. For example, if the AMO type corresponds to "take-and-add" AMO, then operators can be included in the AMO RPC, which includes the value to be added to the value stored at the memory address. As another example, if the AMO type corresponds to "compare-and-swap" AMO, then the AMO RPC may include two operators: a first operator for conditional swapping and a second operator for potential swapping.

[0036] In some embodiments, the RPC used to send AMOs is protocol-based. In some embodiments, the protocol uses Transmission Control Protocol (TCP) / Internet Protocol (IP) packets. In some embodiments, the protocol uses RDMA to send and receive commands. In some embodiments, RDMA atoms are sent directly and intercepted by the NIC 108 (e.g., by the NIC 108's DPU) instead of using the RPC protocol.

[0037] Upon receiving an AMO RPC, NIC 108 can provide the AMO RPC to processor 112 for execution. Processor 112 can retrieve a value from memory based on the memory address in the AMO RPC. In some cases, the value at the memory address is stored in host memory 104 and can be copied to NIC 108's memory 110 (e.g., via memory access 118). In other cases, the value at the memory address is already in NIC 108's memory 110 (e.g., cached in memory 110). For example, a previous AMO may have targeted the same memory address, the memory value may have been copied to a cache, and the cache may not have been flushed back to host memory 104. In some embodiments, a given memory address is available in both host memory 104 and memory 110, and the value in memory 110 can be prioritized (e.g., used to replace the value in host memory 104).

[0038] Processor 112 can perform an AMO corresponding to the AMO type included in the AMO RPC on the cached value to obtain a modified value. The modified value can be stored back into the cache (e.g., memory 110). In some cases, the modified value is immediately stored (e.g., flushed to) host memory 104 (e.g., the modified value can be stored in memory 110, and memory 110 can be immediately flushed to host memory 104 via memory access 118).

[0039] In some embodiments, NIC 108 may receive one or more AMO RPCs 120 from host processor 106. Although host processor 106 has direct access to host memory 104, in some cases, performing one or more AMOs via RPC 120 may be advantageous. For example, the value for the AMO from host processor 106 may be stored in a cache (e.g., memory 110), allowing the AMO to be performed efficiently by processor 112 instead of first flushing the value from memory 110 to host memory 104. In some embodiments, host processor 106 may have direct access to host memory 104 (e.g., and perform AMO on the value at an address from host memory 104). Thus, the embodiments combine the advantages of performing AMO using host memory 104 with the advantages of performing AMO using the NIC's memory.

[0040] Values ​​in memory 110 can be periodically refreshed to host memory 104. In some embodiments, values ​​are refreshed in response to a memory refresh trigger. In some embodiments, a memory refresh trigger is a memory refresh instruction received from other devices in the RDMA network. For example, a remote device can send a “synchronize” instruction to all devices in the RDMA network, which can cause each device in the RDMA network to refresh its respective AMO cache.

[0041] In some embodiments, memory refresh triggering is based on one or more heuristics to determine whether the state of the RDMA network and / or the state of the target node 102 meets refresh criteria. For example, devices in the RDMA network may be collaboratively running an application using an AMO cache (e.g., memory 110 of NIC 108). During application execution, each device may enter an "execution phase" in which cache refresh is beneficial. The device may send an RPC to NIC 108 indicating that a cache refresh is needed. After a predetermined percentage of devices in the network (e.g., 10%, 50%, 90%, etc.) has reached this "execution phase," the heuristics can determine when it is time to refresh the cache.

[0042] As another example, in some embodiments, the memory refresh triggering heuristic can be based on the occupancy of memory 110. For example, if the occupancy of memory 110 meets occupancy criteria, such as occupancy exceeding an occupancy threshold (e.g., 20%, 60%, 80%, etc.), then memory 110 can be refreshed and the value stored in host memory 104.

[0043] In some embodiments, a memory refresh trigger can be invoked based on the host processor 106's attempt to access a memory address cached in memory 110. For example, in some embodiments, when the host processor 106 (e.g., software running on the host processor 106) attempts to access a memory address, the host processor 106 can query the NIC 108 and / or memory 110 to determine whether the memory address is cached in memory 110. If the memory address is cached, the host processor 106 can send a signal to the NIC 108 causing a refresh of memory 110 to host memory 104. The host processor 106 can then access the current value at the memory address from memory 104.

[0044] In some embodiments, memory addresses can be cached in memory 110 in response to a caching criterion being met. For example, memory 110 can cache memory addresses that have been accessed a threshold number of times within a threshold time period. In some embodiments, memory 110 can cache memory addresses if a specific memory region including a memory address has been accessed a threshold number of times within a threshold time period. This ensures that only memory addresses frequently accessed by remote nodes are cached, so that host processor 106 does not need to wait for the cache to be flushed before accessing most memory addresses in host memory 104.

[0045] In some embodiments, remote devices in an RDMA network may send RDMA atomic commands instead of RPC AMO. In some embodiments, NIC 108 may receive RDMA atomic commands and perform atomic operations. In some embodiments, NIC 108 may receive RDMA atomic commands and provide them to host processor 106 for execution.

[0046] Remote node 122 may include remote memory 134, remote processor 136, and remote NIC 124. Remote node 122 may be part of an RDMA network having target node 102. Remote node 122 may perform operations similar to those of target node 102. For example, remote processor 136 may access remote memory 134 to execute applications local to remote node 122. Remote NIC 124 may use memory 130 as an AMO cache (e.g., via processor 132) to perform AMO on remote memory 134. Remote NIC 124 may be configured to send AMO RPCs to NIC 108 of target node 102 (e.g., from remote NIC 124 to NIC 108 via network 128).

[0047] Figure 2 This is a flowchart of an example method 200 for improving AMO performance in an RDMA network according to at least one embodiment.

[0048] Method 200 can be executed using one or more processing units (e.g., CPU, GPU, accelerator, physical processing unit (PPU), data processing unit (DPU), etc.), which may include one or more memory devices (or communicate with one or more memory devices). In at least one embodiment, method 200 can be executed using the processing circuitry of a NIC. In at least one embodiment, method 200 can use... Figure 1 The processing unit of NIC 108 executes the method. In at least one embodiment, the processing unit executing method 200 may be executing instructions stored on a non-transitory computer-readable storage medium. In at least one embodiment, method 200 may be executed using multiple processing threads, each thread executing one or more independent functions, routines, subroutines, or operations of the method. In at least one embodiment, the processing threads implementing method 200 may be synchronized (e.g., using semaphores, critical sections, and / or other thread synchronization mechanisms). Alternatively, the processing threads implementing method 200 may be implemented asynchronously with respect to each other. Figure 2 Compared to the order shown, the various operations of method 200 can be performed in different orders. Some operations of method 200 can be performed in parallel with other operations. In at least one embodiment, Figure 2 One or more of the operations shown may not always be performed.

[0049] At block 202, the processing unit implementing method 200 may receive an Atomic Memory Operation (AMO) Remote Procedure Call (RPC) including a memory address and an AMO type. In some embodiments, the processing unit is coupled to a first memory. In some embodiments, the processing unit and the first memory are included within a data processing unit (DPU) (such as a DPU of a NIC). In some embodiments, the AMO RPC is received from a remote processing device. For example, the processing unit may be included within a DPU that is part of an RDMA network. The remote processing device may send the AMO RPC to the DPU to access (e.g., read, write, modify, etc.) memory present on the DPU or memory of a device hosting the DPU. In some embodiments, the AMO RPC is received from the processor of the device hosting the DPU.

[0050] At block 204, the processing unit can retrieve a value corresponding to the memory address of the AMO RPC from the second memory. In some embodiments, the second memory is associated with one or more host processors. For example, the DPU may be part of the NIC and may be hosted within a computing device that includes the second memory and one or more host processors (e.g., GPU, CPU, etc.).

[0051] At block 206, the processing unit can perform an AMO operation corresponding to an AMO type on the value from the second memory to obtain a modified value. In some embodiments, the AMO corresponding to the AMO type includes at least one of the following: compare-and-swap operation, take-and-add operation, take-and-store operation, take-and-XOR operation, atomic increment operation, atomic decrement operation, swap operation, or software-defined operation. For example, the AMO corresponding to the AMO type can be defined using the load-link / conditional store operation of the processing unit to define an atomic memory operation.

[0052] At block 208, the processing unit may store the modified value in a first memory. In some embodiments, the processing unit may also store the modified value in a second memory (e.g., refresh the value to the second memory). In some embodiments, storing the modified value in the second memory is in response to a memory refresh trigger. In some embodiments, a memory refresh trigger is at least one of the following: receiving a memory refresh instruction (e.g., from another device in the RDMA network) or determining, based on one or more heuristics, that the state of the device network (e.g., devices in the RDMA network) meets refresh criteria. For example, the processing unit and the first memory may be included within a first device that is part of the device network. Devices in the network may be collaboratively running an application using an atomic cache (e.g., the first memory of the DPU). During application execution, each device may enter an "execution phase" in which refreshing the cache may be beneficial. The device may send an RPC to the DPU indicating that the cache needs to be refreshed. Heuristics may determine when it is time to refresh the cache after a predetermined percentage of devices in the network (e.g., 10%, 50%, 90%, etc.) have reached the "execution phase".

[0053] As another example, in some embodiments, a memory refresh triggering heuristic can be based on the occupancy of a first memory (e.g., an AMO cache memory). For example, if the occupancy of the first memory meets occupancy criteria, such as occupancy exceeding an occupancy threshold (e.g., 20%, 60%, 80%, etc.), the first memory can be refreshed and the value can be stored in a second memory.

[0054] In some embodiments, at block 210, the processing unit may determine a target memory for the modified value between at least a first memory and a second memory. In some embodiments, this determination is based on one or more heuristics relating to the state of the RDMA network. For example, in some cases, it may be advantageous to store the modified value in the DPU's memory because the value will be accessed frequently and many AMOs will be performed on that value. In some cases, it may be advantageous to refresh the modified value from the DPU's memory and store the modified value in the host memory. At block 212, the processing unit may store the modified value in the target memory.

[0055] In some embodiments, one or more host computing devices are connected to the NIC. The host computing devices may include one or more host processors (e.g., additional processors) and a second memory (e.g., additional memory). The second memory may be associated with one or more host processors. The target memory address of the AMO RPC may be pointed to the second memory associated with one or more host processors.

[0056] Figure 3This is a flowchart of an example method 300 for improving AMO performance according to at least one embodiment. In some embodiments, method 300 may be executed by processing circuitry and / or processing units of a network interface card (NIC), as disclosed herein. The NIC may be hosted by a host device having one or more host processors and host memory. The NIC may include a DPU having one or more processors and DPU memory. At block 302, the processing unit may receive an Atomic Memory Operation (AMO) remote procedure call (RPC) including a memory address and an AMO type. At decision block 304, the processing unit may determine whether the value of the memory address is cached in the NIC's memory (e.g., in the memory of the NIC's DPU). If the value of the memory address is cached in the DPU's memory, then at block 306, the processing unit may retrieve the value of the memory address from the DPU's memory. If the value of the memory address is not cached in the DPU's memory, then at block 308, the processing unit may retrieve the value of the memory address from the host memory (e.g., the memory of the device hosting the NIC). At block 310, the processing unit may perform an AMO corresponding to the AMO type on the value of the retrieved memory address to obtain a modified value. In some embodiments, the AMO is performed by one or more processing units of the DPU. At block 312, the processing unit may store the modified value in the DPU memory. At decision block 314, the processing unit may determine whether a refresh criterion is met. In some embodiments, the refresh criterion may be receiving a refresh command or determining that the state of the device network (e.g., a device in an RDMA network) meets the refresh criterion. If the refresh criterion is met, at block 316, the processing unit may refresh the DPU memory to host memory. If the refresh criterion is not met, at block 318, the processing unit does not refresh the DPU memory to host memory.

[0057] Figure 4This is a flowchart of an example method 400 for performing memory operations within a host having a DPU, according to at least one embodiment. For example, the host device may include one or more processing units, memory, and a NIC including a DPU as discussed herein. The DPU may include one or more processing units and memory. At block 402, a processing unit of the host device may determine whether a memory address is cached in the DPU memory. For example, the host device may want to perform one or more operations on a specific memory address and may want to ensure that the value at the memory address is not expired. If the memory address is cached in the DPU memory, at block 404, the processing unit may flush the DPU memory to the host memory. At block 406, the processing unit may retrieve the value of the memory address from the host memory. At block 408, the processing unit may perform operations on the value of the memory address to obtain a modified value. At block 410, the processing unit may store the modified value in the host memory.

[0058] Computer Architecture

[0059] Figure 5 This is a block diagram illustrating an exemplary computer system 500 according to at least one embodiment. The exemplary computer system 500 may be a system having interconnect devices and components, a system-on-a-chip (SOC), or some combination thereof formed with a processor, which may include an execution unit for executing instructions. In at least one embodiment, according to embodiments of this disclosure, the computer system 500 may include, but is not limited to, components such as a processor 502 employing an execution unit including logic for executing algorithms for processing data. In one example, the computer system 500 and... Figure 1 The target node 102 and / or remote node 122 correspond to each other. In at least one embodiment, the computer system 500 may include a processor available from Intel Corporation in Santa Clara, California, such as... Processor series, Xeon™ XScale™ and / or StrongARM™ Core TM or Nervana TM The microprocessor may also be used, although other systems (including PCs, engineering workstations, set-top boxes, etc.) with other microprocessors may also be used. In at least one embodiment, the computer system 500 may execute a version of the Windows operating system available from Microsoft Corporation, Redmond, Washington, although other operating systems (e.g., UNIX and Linux), embedded software, and / or graphical user interfaces may also be used.

[0060] The embodiments can be used in other devices, such as handheld devices and embedded applications. Some examples of handheld devices include cellular phones, Internet Protocol devices, digital cameras, personal digital assistants (“PDAs”), and handheld PCs. In at least one embodiment, the embedded application may include a microcontroller, a digital signal processor (“DSP”), a system-on-a-chip, a network computer (“NetPC”), a set-top box, a network hub, a wide area network (“WAN”) switch, an edge device, an Internet of Things (“IoT”) device, or any other system that executes one or more instructions according to at least one embodiment.

[0061] In at least one embodiment, the computer system 500 may include, but is not limited to, a processor 502, which may include, but is not limited to, one or more execution units 508 for performing operations described herein, such as machine learning model training and / or inference operations. In at least one embodiment, the computer system 500 is a single-processor desktop or server system, but in another embodiment, the computer system 500 may be a multiprocessor system. In at least one embodiment, the processor 502 may include, but is not limited to, a Complex Instruction Set Computer (CISC) microprocessor, a Reduced Instruction Set Computing (RISC) microprocessor, a Very Long Instruction Word (VLIW) microprocessor, a processor implementing instruction set combinations, or any other processor device (such as, for example, a digital signal processor). In at least one embodiment, the processor 502 may be coupled to a processor bus 510, which may transmit data signals between the processor 502 and other components in the computer system 500.

[0062] In at least one embodiment, processor 502 may include, but is not limited to, an internal cache memory (“cache”) 504 at level 1 (“L1”). In at least one embodiment, processor 502 may have a single internal cache or multiple internal cache levels. In at least one embodiment, the cache memory may reside outside of processor 502. Depending on the specific implementation and requirements, other embodiments may also include a combination of both internal and external caches.

[0063] In at least one embodiment, processor 502 may include, but is not limited to, a tier 2 (“L2”) internal cache memory (“cache”) 504. Compared to the L1 cache, which is still faster than accessing main memory (e.g., via memory controller hub 516), the L2 cache can be used as a larger, slightly slower cache as a secondary cache. Therefore, the L2 cache can improve performance by reducing the time spent by the processor accessing main memory. In at least one embodiment, processor 502 may have a single internal L2 cache or a multi-tier internal cache memory. In embodiments where processor 502 is a multi-core processor, the L2 cache may be shared among multiple cores of processor 502, thereby providing a larger intermediate tier of cache memory for more than one processing core. In at least one embodiment, the L2 cache memory may reside outside of processor 502. In various embodiments, the L1 cache memory and / or L2 cache memory (e.g., cache 504) may be integrated with… Figure 1 The host memory 104 corresponds to this.

[0064] In at least one embodiment, processor 502 may include, but is not limited to, a level 3 (“L3”) internal cache memory (“cache”) 504. Compared to both L1 and L2 caches, the L3 cache can be used as a larger, slower cache at level three. The L3 cache can improve performance by reducing the time the processor spends accessing main memory. The L3 cache can be shared among multiple cores of processor 502, thereby providing a larger pool of fast-access memory for the processor cores' data. In at least one embodiment, processor 502 may have a single internal L3 cache or a multi-level internal cache. In at least one embodiment, the L3 cache memory and Figure 1 This corresponds to the host memory 104. In at least one embodiment, the L3 cache memory may reside outside the processor 502. Depending on the specific implementation and requirements, other embodiments may also include any combination of internal or external L1, L2, and / or L3 caches. In at least one embodiment, the register file 506 may store different types of data in various registers, including but not limited to integer registers, floating-point registers, status registers, and instruction pointer registers.

[0065] In at least one embodiment, execution unit 508 (including, but not limited to, logic for performing integer and floating-point operations) also resides in processor 502. In at least one embodiment, processor 502 may also include a microcode (“ucode”) read-only memory (“ROM”) storing the microcode of certain macro instructions. In at least one embodiment, execution unit 508 may include logic for disposing of packaged instruction set 509. In at least one embodiment, by including packaged instruction set 509 and associated circuitry for executing instructions in the instruction set of general-purpose processor 502, packaged data in general-purpose processor 502 can be used to perform operations used by many multimedia applications. In one or more embodiments, by using the full width of the processor data bus to perform operations on packaged data, many multimedia applications can be executed faster and more efficiently, eliminating the need to transfer smaller data units across the processor data bus to perform one or more operations on data elements at a time.

[0066] In at least one embodiment, execution unit 508 may also be used as a microcontroller, embedded processor, graphics device, DSP, and other types of logic circuitry. In at least one embodiment, computer system 500 may include, but is not limited to, memory 520. In at least one embodiment, memory 520 may be implemented as a dynamic random access memory (“DRAM”) device, a static random access memory (“SRAM”) device, a flash memory device, or other memory device. In at least one embodiment, memory 520 may store one or more instructions 519 and / or data 521 represented by data signals executable by processor 502.

[0067] In at least one embodiment, the system logic chip may be coupled to the processor bus 510 and the memory 520. In at least one embodiment, the system logic chip may include, but is not limited to, a memory controller hub (“MCH”) 516, and the processor 502 may communicate with the MCH 516 via the processor bus 510. In at least one embodiment, the MCH 516 may provide a high-bandwidth memory path 518 to the memory 520 for storing instructions and data, and for storing graphics commands, data, and textures. In at least one embodiment, the MCH 516 may direct data signals between the processor 502, the memory 520, and other components in the computer system 500, and bridge data signals between the processor bus 510, the memory 520, and the system I / O 522. In at least one embodiment, the system logic chip may provide a graphics port for coupling to a graphics controller. In at least one embodiment, the MCH 516 may be coupled to the memory 520 via the high-bandwidth memory path 518, and the graphics / video card 512 may be coupled to the MCH 516 via an Accelerated Graphics Port (“AGP”) interconnect 514.

[0068] In at least one embodiment, computer system 500 may use system I / O 522, which serves as a proprietary hub interface bus, to couple MCH 516 to I / O controller hub (“ICH”) 530. In at least one embodiment, ICH 530 may provide direct connectivity to some I / O devices via a local I / O bus. In at least one embodiment, the local I / O bus may include, but is not limited to, a high-speed I / O bus for connecting peripheral devices to memory 520, chipset, and processor 502. Examples may include, but are not limited to, an audio controller 529, a firmware hub (“Flash BIOS”) 528, a wireless transceiver 526, a data storage device 524, a conventional I / O controller 523 including a user input and keyboard interface 525, a serial expansion port 527 (such as Universal Serial Bus (“USB”)), and a network controller 532, which in some embodiments may include a data processing unit. Data storage device 524 may include a hard disk drive, floppy disk drive, CD-ROM device, flash memory device, or other mass storage device.

[0069] In at least one embodiment, Figure 5 The illustration depicts a system that includes interconnecting hardware devices or "chips," while in other embodiments, Figure 5 An exemplary system-on-a-chip (“SoC”) may be illustrated. In at least one embodiment, the device may be interconnected via proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment, one or more components of the computer system 500 are interconnected using a compute fast link (CXL) interconnect.

[0070] In some examples, processor 502 may include inference and / or training logic 515, which may be used to perform inference and / or training operations associated with one or more embodiments. In at least one embodiment, inference and / or training logic 515 may be used for Figure 5 The system is used to infer or predict operations based at least in part on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein. In some embodiments, such operations may include AMO and may benefit from the embodiments discussed herein.

[0071] Figure 6 This is a block diagram illustrating an electronic device 600 utilizing a processor 610 according to at least one embodiment. In at least one embodiment, the electronic device 600 may be, for example, but not limited to, a laptop computer, tower server, rack server, blade server, desktop computer, tablet computer, mobile device, telephone, embedded computer, edge device, IoT device, or any other suitable electronic device. In at least one embodiment, the computer system 600 and... Figure 1 The target node 102 and / or the remote node 122 correspond to each other.

[0072] In at least one embodiment, the electronic device 600 may, but is not limited to, a processor 610 communicatively coupled to any suitable number or type of components, peripherals, modules, or devices. In at least one embodiment, the processor 610 is coupled using a bus or interface such as an I2C bus, a System Management Bus (“SMBus”), a Low Pin Count (LPC) bus, a Serial Peripheral Interface (“SPI”), a High Definition Audio (“HDA”) bus, a Serial Advanced Technology Accessory (“SATA”) bus, a Universal Serial Bus (“USB”) (versions 1, 2, 3), or a Universal Asynchronous Receiver / Transmitter (“UART”) bus. In at least one embodiment, Figure 6 The illustration depicts a system comprising interconnected hardware devices or "chips," while in other embodiments, Figure 6 An exemplary system-on-a-chip (“SoC”) may be illustrated. In at least one embodiment, Figure 6 The devices illustrated herein can be interconnected via proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment, Figure 6 One or more components are interconnected using Computational Fast Link (CXL) interconnects.

[0073] In at least one embodiment, Figure 6This may include a display 624, a touchscreen 625, a touchpad 630, a near-field communication unit (“NFC”) 645, a sensor hub 640, a thermal sensor 646, a fast chipset (“EC”) 635, a trusted platform module (“TPM”) 638, a BIOS / firmware / flash memory (“BIOS, FW flash”) 622, a DSP 660, a drive 620 such as a solid-state drive (“SSD”) or a hard disk drive (“HDD”), a wireless local area network unit (“WLAN”) 650, a Bluetooth unit 652, a wireless wide area network unit (“WWAN”) 656, a global positioning system (GPS) 655, a camera such as a USB 3.0 camera (“USB 3.0 camera”) 654, and / or a low-power double data rate (“LPDDR”) memory unit (“LPDDR3”) 615 implemented in, for example, the LPDDR3 standard. Each of these components may be implemented in any suitable manner.

[0074] In at least one embodiment, other components may be communicatively coupled to processor 610 via the components discussed above. In at least one embodiment, accelerometer 641, ambient light sensor (“ALS”) 642, compass 643, and gyroscope 644 may be communicatively coupled to sensor hub 640. In at least one embodiment, thermal sensor 639, fan 637, keyboard 636, and touchpad 630 may be communicatively coupled to EC 635. In at least one embodiment, speaker 663, headphones 664, and microphone (“mic”) 665 may be communicatively coupled to audio unit (“audio codec and Class D amplifier”) 662, which in turn may be communicatively coupled to DSP 660. In at least one embodiment, audio unit 662 may include, for example, but not limited to, audio encoder / decoder (“codec”) and Class D amplifier. In at least one embodiment, SIM card (“SIM”) 657 may be communicatively coupled to WWAN unit 656. In at least one embodiment, components such as WLAN unit 650, Bluetooth unit 652, and WWAN unit 656 may be implemented in a next-generation size specification (“NGFF”).

[0075] In at least one embodiment, the inference and / or training logic 515 can be used for Figure 6 A system for reasoning or predicting operations based at least in part on weight parameters computed using neural network training operations, neural network functions and / or architectures or neural network use cases described herein.

[0076] These components can be used to generate synthetic data that simulates fault cases during network training, which can help improve network performance while limiting the amount of synthetic data to avoid overfitting.

[0077] Servers and data centers

[0078] The following figures illustrate, but are not limited to, exemplary network servers and data center-based systems that can be used to implement at least one embodiment.

[0079] Data centers can include multiple network switches within a specific topology (such as a fat tree topology, a thin fly topology, a dragonfly topology, etc.). The specifications and composition of the network switches within the topology affect the overall network performance of the data center (e.g., bandwidth capacity).

[0080] Example data center environment

[0081] Data centers, high-performance computing clusters, etc., are typically formed by various computing components or networked devices, and communication networks formed by electrical and / or optical equipment can be used to enable communication between these networked devices that form these implementations. For example, see reference... Figure 7A and Figure 7B Network architecture 700 may include data center 702, communication network 704, and one or more network devices 706. Network architecture 700 may illustrate a general computing architecture in which more specific systems and / or subsystems can run.

[0082] For example, data center 702 can be a centralized facility designed to house computing resources and related components. Data center 702 can operate to support the infrastructure required for advanced computing tasks to achieve efficient, secure, and reliable operation. Data center 702 may include building and structural components, including power, cooling systems, fire suppression systems, and physical safety measures configured to maintain optimal operating conditions and / or protect equipment from environmental hazards and unauthorized access. Example data center 702 may include components typically arranged in racks, such as... Figure 7B As shown, these are high-performance servers or computing nodes connected via high-speed networks as described herein. These servers may include processors (e.g., central processing units (CPUs), graphics processing units (GPUs), data processing units (DPUs), etc.), memory (e.g., RAM), and storage solutions (e.g., hard disk drives (HDDs), solid-state drives (SSDs), etc.). The hardware configuration can be designed for parallel processing and high throughput to meet the demands of high-performance computing (HPC) applications.

[0083] Data center 702 may include high-speed network devices (such as network switches, routers, firewalls, etc.) to facilitate fast and secure data transmission within data center 702 (e.g., between servers or compute nodes) and between external networks. Data center 702 can facilitate communication between servers or compute nodes through a network topology that ensures efficient data exchange, minimizes latency, and maximizes bandwidth. The network topology can define how various network devices (such as switches and routers) interconnect to enable data flow. By implementing an efficient network topology, data center 702 can support high-performance computing tasks. Examples of various network topologies can include hierarchical network topologies such as fat tree topology, thin fly topology, dragonfly topology, etc.

[0084] Communication network 704 can communicatively couple data center 702, as well as one or more network devices 706 and other external devices, for data exchange and connectivity. Examples of communication network 704 may include Internet Protocol (IP) networks, Ethernet, InfiniBand (IB) networks, Fibre Channel networks, the Internet, cellular communication networks, wireless communication networks and combinations thereof (e.g., Ethernet-based Fibre Channel), and their variations. The ability of communication network 704 to combine many network types and configurations allows data center 702 to adapt to a variety of application requirements, from general data communication to dedicated high-performance computing (HPC) tasks. As described herein, communication network 704 can utilize various optical components to establish communication links (e.g., communicatively couple between components in architecture 700) between components in architecture 700. Thus, communication network 704 may include various optical devices, transceivers, modules, etc., configured to generate optical signals (e.g., provide optical transmitter functionality) and / or receive optical signals (e.g., provide optical receiver functionality).

[0085] One or more network devices 706 may include various computing devices capable of transmitting and receiving signals over a communication network 704. The range of network devices 706 can be from personal computing devices to complex server configurations. Examples include personal computers (PCs), laptops, tablets, smartphones, and servers. One or more network devices 706 can facilitate user interaction with data center 702, allowing data to be entered, retrieved, and processed from remote locations. In addition to individual computing devices, one or more network devices 706 may also include a collection of servers or additional data centers. For example, these could be other data centers similar to or identical to data center 702. This interconnection can allow for the formation of distributed computing environments to improve redundancy, load balancing, and disaster recovery capabilities. By linking multiple data centers, network architecture 700 can leverage geographically dispersed resources to optimize performance and ensure high availability.

[0086] As described herein, data center 702 and / or one or more network devices 706 may include storage devices and processing circuitry for performing computational tasks, such as controlling the flow of data within and through communication network 704. The processing circuitry may include software, hardware, or a combination thereof. For example, the processing circuitry may include a memory containing executable instructions and a processor (e.g., a microprocessor) that executes those instructions. The memory may correspond to any suitable type of memory device or collection of memory devices configured to store instructions. Non-limiting examples of suitable memory devices include flash memory, random access memory (RAM), read-only memory (ROM), variations thereof, combinations thereof, or similar technologies. In specific embodiments, the memory and processor may be integrated into a general-purpose device, such as a microprocessor with integrated memory. Additionally or alternatively, the processing circuitry may include hardware components such as application-specific integrated circuits (ASICs). Other non-limiting examples of processing circuitry include integrated circuit (IC) chips, CPUs, GPUs, microprocessors, field-programmable gate arrays (FPGAs), collections of logic gates or transistors, resistors, capacitors, inductors, and diodes. Some or all of the processing circuitry can be housed on a printed circuit board (PCB) or an assembly of PCBs. It should be noted that any suitable type of electrical component or assembly of electrical components may be suitable for inclusion in the processing circuitry.

[0087] Additionally, although not explicitly shown, this disclosure contemplates that data center 702 and one or more network devices 706 may include one or more communication interfaces for facilitating wired and / or wireless communication between themselves and other components (not shown) within network architecture 700. These communication interfaces may include various technologies, including but not limited to Ethernet ports, fiber optic connections, etc., for integration and interoperability between various components within network architecture 700. transceiver Module and cellular communication module.

[0088] Furthermore, this disclosure envisions that network architecture 700 may include additional components and functions. For example, the network architecture may include, but is not limited to, additional processing units, dedicated accelerators (such as tensor processing units or TPUs), enhanced security modules, and redundant power supplies. Including these elements may be intended to ensure that network architecture 700 is robust, scalable, and capable of meeting a wide variety of operational requirements. Any variations, modifications, or adjustments to the elements falling within the spirit and scope of this disclosure are considered to be covered herein. This includes any combination, sub-combination, or enhancement of the various elements to achieve improved performance, reliability, and efficiency in network architecture 700.

[0089] Figure 8A distributed system 800 according to at least some embodiments is illustrated. In at least some embodiments, the distributed system 800 includes one or more client computing devices 802, 804, 806, and 808, which are configured to execute and operate client applications, such as web browsers, proprietary clients, and / or variations thereof, on one or more networks 810. In at least one embodiment, a server 812 may be communicatively coupled to remote client computing devices 802, 804, 806, and 808 via one or more networks 810. In at least one embodiment, client computing devices 802, 804, 806, 808, and / or server 812 may be connected to... Figure 1 The target node 102 and / or the remote node 122 correspond to each other.

[0090] In at least one embodiment, server 812 may be adapted to run one or more services or software applications, such as services and applications that manage session activity for single sign-on (SSO) access across multiple data centers. In at least one embodiment, server 812 may also provide other services or software applications, which may include non-virtual and virtual environments. In at least one embodiment, these services may be provided as web-based services or cloud services or under a Software as a Service (SaaS) model to users of client computing devices 802, 804, 806, and / or 808. In at least one embodiment, users operating client computing devices 802, 804, 806, and / or 808 may in turn use one or more client applications to interact with server 812 to utilize the services provided by these components.

[0091] In at least one embodiment, software components 818, 820, and 822 of the distributed system 800 are implemented on server 812. In at least one embodiment, one or more components of the distributed system 800 and / or the services provided by these components may also be implemented by one or more client computing devices 802, 804, 806, and / or 808. In at least one embodiment, a user operating a client computing device can then utilize one or more client applications to use the services provided by these components. In at least one embodiment, these components can be implemented using hardware, firmware, software, or a combination thereof. It should be understood that various different system configurations are possible and may differ from the distributed system 800. Therefore, Figure 8 The embodiments shown are examples of distributed systems for implementing the systems of the embodiments, and are not intended to be limiting.

[0092] In at least one embodiment, client computing devices 802, 804, 806, and / or 808 may include different types of computing systems. In at least one embodiment, the client computing device may include a portable handheld device (e.g., Cellular phone Computing tablets, personal digital assistants (PDAs), or wearable devices (e.g., Google) Head-mounted display), running software (such as Microsoft Windows) And / or various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 10, Palm OS, and / or variants thereof). In at least one embodiment, the device may support different applications, such as various Internet-related applications, email, short message service (SMS) applications, and may use various other communication protocols. In at least one embodiment, the client computing device may also include a general-purpose personal computer, for example, a general-purpose personal computer running various versions of Microsoft... Apple Personal computers and / or laptops running Linux operating systems. In at least one embodiment, the client computing device can be running various commercially available operating systems. The client computing device may be a workstation computer operating system similar to UNIX, including but not limited to various GNU / Linux operating systems such as Google Chrome OS. In at least one embodiment, the client computing device may further include electronic devices capable of communicating over one or more networks 810, such as thin client computers, internet-enabled gaming systems (e.g., with or without...). Gesture input devices include Microsoft Xbox game consoles and / or personal messaging devices. Despite Figure 8 The distributed system 800 is shown as having four client computing devices, but can support any number of client computing devices. Other devices (such as devices with sensors) can interact with the server 812.

[0093] In at least one embodiment, network 810 in distributed system 800 can be any type of network capable of supporting data communication using any of the various available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (System Network Architecture), IPX (Internet Packet Switching), AppleTalk, and / or variations thereof. In at least one embodiment, network 810 can be a local area network (LAN), an Ethernet-based network, Token Ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, or a wireless network (e.g., in the IEEE 802.11 protocol suite). Networks operating under any of the wireless protocols (and / or any other wireless protocols), and / or any combination of these and / or other networks.

[0094] In at least one embodiment, server 812 may consist of one or more general-purpose computers, dedicated server computers (e.g., including PC (personal computer) servers), Servers (including mid-range servers, mainframe computers, rack servers, etc.), server farms, server clusters, or any other suitable arrangement and / or combination thereof. In at least one embodiment, server 812 may include one or more virtual machines running a virtual operating system or other computing architectures involving virtualization. In at least one embodiment, one or more flexible pools of logical storage devices may be virtualized to maintain virtual storage devices for the server. In at least one embodiment, the virtual network may be controlled by server 812 using software-defined networking. In at least one embodiment, server 812 may be adapted to run one or more services or software applications.

[0095] In at least one embodiment, server 812 can run any operating system, and any commercially available server operating system. In at least one embodiment, server 812 can also run any of a variety of additional server applications and / or mid-level applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, etc. Servers, database servers, and / or variations thereof. In at least one embodiment, exemplary database servers include, but are not limited to, those commercially available from Oracle, Microsoft, Sybase, IBM (International Business Machines), and / or variations thereof.

[0096] In at least one embodiment, server 812 may include one or more applications for analyzing and merging data feeds and / or event updates received from users of client computing devices 802, 804, 806, and 808. In at least one embodiment, data feeds and / or event updates may include, but are not limited to, data received from one or more third-party information sources and continuous data streams. feed, Updates or real-time updates may include real-time events related to sensor data applications, financial quotes, network performance measurement tools (e.g., network monitoring and business management applications), clickstream analysis tools, vehicle traffic monitoring, and / or their changes. In at least one embodiment, server 812 may also include one or more applications for displaying data feeds and / or real-time events via one or more display devices of client computing devices 802, 804, 806, and 808.

[0097] In at least one embodiment, the distributed system 800 may further include one or more databases 814 and 816. In at least one embodiment, the databases may provide mechanisms for storing information such as user interaction information, usage pattern information, adaptation rule information, and other information. In at least one embodiment, databases 814 and 816 may reside in various locations. In at least one embodiment, one or more databases 814 and 816 may reside on a non-transitory storage medium local to server 812 (and / or within server 812). In at least one embodiment, databases 814 and 816 may be located remotely from server 812 and communicate with server 812 via a network-based connection or a dedicated connection. In at least one embodiment, databases 814 and 816 may reside in a storage area network (SAN). In at least one embodiment, any necessary files for performing functions belonging to server 812 may be appropriately stored locally on server 812 and / or remotely. In at least one embodiment, databases 814 and 816 may include relational databases, such as databases adapted to store, update, and retrieve data in response to SQL-formatted commands.

[0098] Figure 9 An exemplary data center 900 according to at least one embodiment is shown. In at least one embodiment, the data center 900 includes, but is not limited to, a data center infrastructure layer 920, a framework layer 910, a software layer 906, and an application layer 902.

[0099] In at least one embodiment, such as Figure 9As shown, the data center infrastructure layer 920 may include a resource coordinator 922, grouped computing resources 924, and node computing resources (“nodes CR”) 926a-926c, where “c” represents any complete positive integer. In at least one embodiment, nodes CR916(1)-916(N) may include, but are not limited to, any number of central processing units (“CPUs”) or other processors (including accelerators, field-programmable gate arrays (“FPGAs”), graphics processors, etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state drives or disk drives), network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power modules, and cooling modules, etc. In at least one embodiment, one or more nodes CR926a-926c (e.g., node CR926b) may be a server having one or more of the aforementioned computing resources. In some embodiments, at least one of nodes CRs 926a, 926b, and 926c may correspond to Figure 1 The target node 102 and / or the remote node 122.

[0100] In at least one embodiment, the grouped computing resources 924 may include individual groups (not shown) of node CRs housed in one or more racks, or a plurality of racks (also not shown) housed in data centers in various geographical locations. The individual groups of node CRs within the grouped computing resources 924 may include computing, networking, memory, or storage resources that can be configured or allocated to support groups of one or more workloads. In at least one embodiment, several node CRs, including CPUs or processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. In at least one embodiment, the one or more racks may also include any number of power modules, cooling modules, and network switches, in any combination.

[0101] In at least one embodiment, resource coordinator 922 may configure or otherwise control one or more nodes CR926a-926c and / or grouped computing resources 924. In at least one embodiment, resource coordinator 922 may include a Software Design Infrastructure (“SDI”) management entity for data center 900. In at least one embodiment, resource coordinator 922 may include hardware, software, or some combination thereof.

[0102] In at least one embodiment, such as Figure 9As shown, the framework layer 910 includes, but is not limited to, a job scheduler 912, a configuration manager 914, a resource manager 918, and a distributed file system 916. In at least one embodiment, the framework layer 910 may include a framework of software 908 supporting the software layer 906 and / or one or more applications 904 supporting the application layer 902. In at least one embodiment, the software 908 or application 904 may respectively include web-based service software or applications, such as services or applications provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, the framework layer 910 may be, but is not limited to, a free and open-source software web application framework, such as Apache Spark™ (hereinafter referred to as "Spark") which can leverage the distributed file system 916 for large-scale data processing (e.g., "big data"). In at least one embodiment, the job scheduler 912 may include a Spark driver to facilitate the scheduling of workloads supported by the various layers of the data center 900. In at least one embodiment, the configuration manager 914 may be able to configure different layers, such as the software layer 906 and the framework layer 910 including Spark and the distributed file system 906 for supporting large-scale data processing. In at least one embodiment, resource manager 916 is capable of managing cluster or group computing resources mapped to or allocated to support distributed file system 916 and job scheduler 912. In at least one embodiment, cluster or group computing resources may include group computing resources 924 on data center infrastructure layer 920. In at least one embodiment, resource manager 918 may coordinate with resource coordinator 922 to manage these mapped or allocated computing resources.

[0103] In at least one embodiment, the software 908 included in software layer 906 may include software used by at least a portion of nodes CR926a-926c, grouped computing resources 924, and / or the distributed file system 916 of framework layer 910. One or more types of software may include, but are not limited to, Internet webpage search software, email virus scanning software, database software, and streaming video content software.

[0104] In at least one embodiment, the application layer 902 may include one or more applications 904 that can be used by at least a portion of nodes CR926a-926c, grouped computing resources 924, and / or the distributed file system 916 of the framework layer 910. The one or more types of applications may include, but are not limited to, CUDA applications, 5G network applications, artificial intelligence applications, data center applications, and / or variations thereof.

[0105] In at least one embodiment, any of the configuration manager 914, resource manager 918, and resource coordinator 922 can implement any number and type of self-modification actions based on any amount and type of data acquired in any technically feasible manner. In at least one embodiment, self-modification actions can mitigate potentially poor configuration decisions by data center operators of data center 900 and can prevent underutilization and / or poor performance of the data center.

[0106] Figure 10 A client-server network 1004, formed by a plurality of interconnected network server computers 1002, is illustrated according to at least one embodiment. In at least one embodiment, each network server computer 1002 stores data accessible to other network server computers 1002 and client computers 1006 and remote networks 1008 linked to the wide-area client-server network 1004. In at least one embodiment, the configuration of the client-server network 1004 may change over time when client computers 1006 and one or more remote networks 1008 connect and disconnect from the client-server network 1004, and when one or more backbone server computers 1002 are added to or removed from the client-server network 1004. In at least one embodiment, the client-server network includes client computers 1006 and remote networks 1008 when connected to network server computers 1002. In at least one embodiment, the term "computer" includes any device or machine capable of accepting data, applying prescribed processes to the data, and providing the results of the processes.

[0107] In at least one embodiment, the network server computer 1002 may correspond to Figure 1 The target node 102, and the client computer 1006 can correspond to Figure 1 Remote node 122.

[0108] In at least one embodiment, the client-server network 1004 stores information accessible to the network server computer 1002, the remote network 1008, and the client computer 1006. In at least one embodiment, the network server computer 1002 is formed from a mainframe computer, a minicomputer, and / or a microcomputer, each having one or more processors. In at least one embodiment, the server computer 1002 is linked together via wired and / or wireless transmission media (such as wires, fiber optic cables) and / or microwave transmission media, satellite transmission media, or other conductive, optical, or electromagnetic wave transmission media. In at least one embodiment, the client computer 1006 accesses the network server computer 1002 via similar wired or wireless transmission media. In at least one embodiment, the client computer 1006 can be linked to the client-server network 1004 using a modem and standard telephone communication networks. In at least one embodiment, alternative carrier systems (such as cable and satellite communication systems) can also be used to link to the client-server network 1004. In at least one embodiment, other private or time-sharing carrier systems can be used. In at least one embodiment, the client-server network 1004 is a global information network, such as the Internet. In at least one embodiment, the network is a private intranet using protocols similar to the Internet but with added security measures and restricted access controls. In at least one embodiment, the client-server network 1004 is a private or semi-private network using proprietary communication protocols.

[0109] In at least one embodiment, the client computer 1006 is any end-user computer, and may also be a mainframe computer, minicomputer, or microcomputer with one or more microprocessors. In at least one embodiment, the server computer 1002 may sometimes be used as a client computer to access another server computer 1002. In at least one embodiment, the remote network 1008 may be a local area network (LAN), a network added to a wide area network via an independent service provider (ISP) for the Internet, or another group of computers interconnected via wired or wireless transmission media with fixed or time-varying configurations. In at least one embodiment, the client computer 1006 may be independently or via the remote network 1008 linked to and access the client-server network 1004.

[0110] Figure 11A computer network 1108 connecting one or more computing machines is illustrated according to at least one embodiment. In at least some embodiments, network 1108 can be any type of electrically connected group of computers, including, for example, the Internet, intranet, local area network (LAN), wide area network (WAN), or an interconnection combination of these network types. In at least one embodiment, the connection within network 1108 can be a remote modem, Ethernet (IEEE 802.3), Token Ring (IEEE 802.5), Fiber Distributed Data Link Interface (FDDI), Asynchronous Transfer Mode (ATM), or any other communication protocol. In at least one embodiment, the computing devices linked to the network can be desktop computers, servers, portable, handheld, set-top boxes, personal digital assistants (PDAs), terminals, or any other desired type or configuration. In at least one embodiment, network-connected devices can vary widely in terms of processing power, internal memory, and other performance characteristics, depending on their functionality. In at least one embodiment, communication within the network, as well as communication to or from computing devices connected to the network, can be wired or wireless. In at least one embodiment, network 1108 may at least partially comprise the global public Internet, which typically connects multiple users according to the Transmission Control Protocol / Internet Protocol (TCP / IP) specification in a client-server model. In at least one embodiment, the client-server network is the dominant model for communication between two computers. In at least one embodiment, a client computer (“client”) issues one or more commands to a server computer (“server”). In at least one embodiment, the server fulfills client commands by accessing available network resources and returning information to the client in accordance with the client commands. In at least one embodiment, client computer systems and network resources residing on the network server are assigned network addresses for identification during communication between network elements. In at least one embodiment, communication from other network-connected systems to the server will include the network address of the relevant server / network resource as part of the communication, such that the appropriate destination of the data / request is identified as the recipient. In at least one embodiment, when network 1108 comprises the global Internet, the network address is an IP address in TCP / IP format, which can at least partially route data to email accounts, websites, or other Internet tools residing on the server. In at least one embodiment, information and services residing on a web server can be made available to a client computer's web browser via a domain name (e.g., www.site.com) (which maps to the IP address of the web server).

[0111] In at least one embodiment, multiple clients 1102, 1104, and 1106 are connected to network 1108 via respective communication links. In at least one embodiment, each of these clients can access network 1108 via any desired form of communication, such as via dial-up modem connection, cable link, digital subscriber line (DSL), wireless or satellite link, or any other form of communication. In at least one embodiment, each client can communicate using any machine compatible with network 1108 (e.g., personal computer (PC), workstation, dedicated terminal, personal data assistant (PDA), or other similar device). In at least one embodiment, clients 1102, 1104, and 1106 may or may not be located in the same geographical area.

[0112] In at least one embodiment, at least one of the plurality of clients 1102, 1104, and 1106 may correspond to Figure 1 The target node 102 and / or the remote node 122.

[0113] In at least one embodiment, multiple servers 1110, 1112, and 1114 are connected to network 1118 to serve clients communicating with network 1118. In at least one embodiment, each server is typically a powerful computer or device that manages network resources and responds to client commands. In at least one embodiment, the server includes computer-readable data storage media, such as hard disk drives and RAM memory, that store program instructions and data. In at least one embodiment, servers 1110, 1112, and 1114 run applications that respond to client commands. In at least one embodiment, server 1110 may run a web server application for responding to client requests for HTML pages, and may also run a mail server application for receiving and routing emails. In at least one embodiment, other applications, such as an FTP server or media server for streaming audio / video data to clients, may also run on server 1110. In at least one embodiment, different servers may be dedicated to performing different tasks. In at least one embodiment, server 1110 may be a dedicated web server for managing website-related resources for different users, while server 1112 may be dedicated to providing email management. In at least one embodiment, the other servers may be dedicated to a combination of two or more services typically available or provided over a network, such as media (audio, video, etc.), File Transfer Protocol (FTP), or other services. In at least one embodiment, each server may be located in the same or different location as the other servers. In at least one embodiment, multiple servers may exist to perform mirroring tasks for users, thereby mitigating congestion or minimizing traffic directed to and from a single server. In at least one embodiment, servers 1110, 1112, and 1114 are under the control of a web hosting provider that maintains and delivers third-party content over network 1118.

[0114] In at least one embodiment, the web hosting provider delivers services to two different types of clients. In at least one embodiment, one type, which may be referred to as a browser, requests content, such as web pages, email messages, video clips, etc., from servers 1110, 1112, and 1114. In at least one embodiment, a second type (which may be referred to as a user) hires the web hosting provider to maintain network resources (such as websites) and make them available to the browser. In at least one embodiment, the user contracts with the web hosting provider to make memory space, processor capacity, and communication bandwidth available to the network resources they desire, according to the amount of server resources the user expects to utilize.

[0115] In at least one embodiment, in order for a web hosting provider to serve both clients, the application managing network resources hosted on the server must be properly configured. In at least one embodiment, the program configuration process involves defining a set of parameters that at least partially control the application's response to browser requests and also at least partially define the server resources available to a particular user.

[0116] In one embodiment, intranet server 1116 communicates with network 1108 via a communication link. In at least one embodiment, intranet server 1116 communicates with server manager 1118. In at least one embodiment, server manager 1118 includes a database of application configuration parameters used by servers 1110, 1112, and 1114. In at least one embodiment, a user modifies database 1120 via intranet server 1116, and server manager 1118 interacts with servers 1110, 1112, and 1114 to modify application parameters such that they match the contents of the database. In at least one embodiment, a user logs into intranet 1116 by connecting to intranet server 1116 via client 1102 and entering authentication information such as a username and password.

[0117] In at least one embodiment, when a user wishes to log in to a new service or modify an existing service, the intranet server 1116 authenticates the user and provides the user with an interactive screen display / control panel that allows the user to access configuration parameters for a specific application. In at least one embodiment, multiple modifiable text boxes describing aspects of the user's website or other network resources' configuration are presented to the user. In at least one embodiment, if the user desires to increase the storage space reserved for their website on the server, a field is provided where the user specifies the desired storage space. In at least one embodiment, in response to receiving this information, the intranet server 1116 updates the database 1120. In at least one embodiment, the server manager 1118 forwards the information to the appropriate server and uses the new parameters during application operation. In at least one embodiment, the intranet server 1116 is configured to provide the user with access to configuration parameters of network resources (e.g., web pages, email, FTP sites, media sites, etc.) that the user has contracted with a web hosting service provider.

[0118] Figure 12AA networked computer system 1200a according to at least one embodiment is illustrated. In at least some embodiments, the networked computer system 1200a includes a plurality of nodes or personal computers (“PCs”) 1202, 1218, 1220. In at least one embodiment, PC 1202 (e.g., a node) includes a processor 1214, memory 1216, a camera 1204, a microphone 1206, a mouse 1208, a speaker 1210, and a monitor 1212. In at least one embodiment, PCs 1202, 1218, 1220 may each run one or more desktop servers, such as those on an internal network within a given company, or may be servers on a general network not limited to a specific environment. In at least one embodiment, each PC node in the network has one server, such that each PC node in the network represents a specific network server with a specific network URL address. In at least one embodiment, each server defaults to a default webpage for the user of that server, and the default webpage itself may include embedded URLs pointing to further subpages for that user on that server, or to pages on other servers on the network or on other servers.

[0119] In at least one embodiment, PCs 1202, 1218, 1220 and other nodes of the network are interconnected via medium 1222. In at least one embodiment, medium 1222 may be a communication channel such as Integrated Services Digital Network (“ISDN”). In at least one embodiment, the individual nodes of the networked computer system may be connected via various communication media, including a local area network (“LAN”), a simple old-fashioned telephone line (“POTS”) (sometimes referred to as the Public Switched Telephone Network (“PSTN”)), and / or variations thereof. In at least one embodiment, the individual nodes of the network may also constitute users of computer systems interconnected via a network such as the Internet. In at least one embodiment, each server on the network (running from a specific node of the network at a given instance) has a unique address or identifier within the network, which may be specified according to a URL.

[0120] In at least one embodiment, at least one of PC 1202, 1218, and 1220 may correspond to Figure 1 The target node 102 and / or the remote node 122.

[0121] In at least one embodiment, multiple multipoint conferencing units (“MCUs”) can therefore be used to transmit data to and from various nodes or “endpoints” of the conferencing system. In at least one embodiment, in addition to various other communication media (such as nodes connected via the Internet), the nodes and / or MCUs may be interconnected via ISDN links or through a local area network (“LAN”). In at least one embodiment, the nodes of the conferencing system may typically be directly connected to a communication medium (such as a LAN) or connected via an MCU, and the conferencing system may include other nodes or components, such as routers, servers, and / or variations thereof.

[0122] In at least one embodiment, processor 1214 is a general-purpose programmable processor. In at least one embodiment, the processor of a node in the networked computer system 1200a may also be a dedicated video processor. In at least one embodiment, the different peripherals and components of the node (such as those of PC 1202) may be different from those of other nodes. In at least one embodiment, PC 1218 and PC 1220 may be configured to be the same as or different from PC 1202. In at least one embodiment, the node may be implemented on any suitable computer system other than a PC system.

[0123] Figure 12B A networked computer system 1200b according to at least one embodiment is illustrated. In at least some embodiments, the networked computer system 1200b illustrates a network (such as LAN 1224) that can be used to interconnect various nodes that can communicate with each other. In at least one embodiment, multiple nodes, such as PCs 1226, 1228, and 1230, are attached to LAN 1224. In at least one embodiment, the nodes (e.g., PCs) may also be connected to the LAN via a network server or other device. In at least one embodiment, the networked computer system 1200b includes other types of nodes or elements, including, for at least one embodiment, routers, servers, and nodes.

[0124] In at least one embodiment, at least one of PC 1226, 1228, and 1230 may correspond to Figure 1 The target node 102 and / or the remote node 122. For example, PC 1226 may correspond to the target node 102, while PC 1228 may correspond to the remote node 122.

[0125] Figure 12CA networked computer system 1200c is illustrated according to at least one embodiment. In at least some embodiments, the networked computer system 1200c illustrates a WWW system with communication across a backbone communication network (such as the Internet 1232), the backbone communication network being usable for various nodes interconnecting the network. In at least one embodiment, the WWW is a set of protocols operating on top of the Internet and allowing a graphical interface system to operate on it to access information via the Internet. In at least one embodiment, the Internet 1232 attached to the WWW consists of multiple nodes, such as PCs 1240, 1242, and 1244. In at least one embodiment, nodes interface with other nodes of the WWW via WWW HTTP servers (such as WWW HTTP servers 1234 and 1236). In at least one embodiment, PC 1244 may be a PC forming a node of the Internet 1232, and PC 1244 itself runs its WWW HTTP server 1236, although for illustrative purposes... Figure 12C PC 1244 and WWWHTTP server 1236 are shown separately.

[0126] In at least one embodiment, the WWW is a distributed type of application characterized by WWW HTTP, the WWW protocol, which runs on top of the Internet's Transmission Control Protocol / Internet Protocol (“TCP / IP”). In at least one embodiment, the WWW can therefore be characterized by a set of protocols running on the Internet (i.e., HTTP) as its “backbone”.

[0127] In at least one embodiment, the web browser is an application running on a node of a network in a WWW-compatible network system, allowing users of a particular server or node to view such information and thus allowing users to search for graphics and text-based documents linked together using hypertext links embedded in documents or files available from servers that understand HTTP. In at least one embodiment, when a user uses another server on a network such as the Internet to retrieve a given webpage from a first server associated with a first node, the retrieved document may have different hypertext links embedded therein, and a local copy of the page created locally by the user is also retrieved. In at least one embodiment, when a user clicks a hypertext link, locally stored information associated with the selected hypertext link is generally sufficient to allow the user's machine to open a connection over the Internet to the server indicated by the hypertext link.

[0128] In at least one embodiment, more than one user may be coupled to each HTTP server, for example, via a LAN (such as LAN 1238, as shown with respect to WWW HTTP server 1234). In at least one embodiment, the network computer system 1200c may also include other types of nodes or elements. In at least one embodiment, the WWW HTTP server is an application running on a machine such as a PC. In at least one embodiment, each user can be considered to have a unique "server," as shown with respect to PC 1244. In at least one embodiment, a server can be considered to be a server such as WWW HTTP server 1234 that provides access to the network for a LAN or multiple nodes or multiple LANs. In at least one embodiment, there are multiple users, each user having a desktop PC or a node on the network, and each desktop PC potentially establishing a server for its users. In at least one embodiment, each server is associated with a specific network address or URL that, when accessed, provides a default webpage for that user at that specific network address or URL. In at least one embodiment, the webpage may include further links (embedded URLs) pointing to further subpages for that user on that server, or to other servers on the network or to pages on other servers on the network.

[0129] In at least one embodiment, at least one of PCs 1240, 1242, and 1244 can be connected with... Figure 1 The target node 102 and / or the remote node 122 correspond to each other. For example, PC 1240 may correspond to the target node 102, and PC 1242 may correspond to the remote node 122.

[0130] Figure 13This is a block diagram of a computing system 1300 having two processing devices coupled to each other and multiple networks according to at least one embodiment. The computing system 1300 is designed with multiple integrated circuits (referred to as processing devices), each including a CPU and two GPUs, forming a powerful and flexible architecture. These processing devices are interconnected via NVLink (or other high-speed interconnects) to enable high-speed communication between the processing devices, and are also connected via network interface cards (NICs) or data processing units (DPUs) to ensure efficient data transfer across the computing system 1300. Coupling the processing devices via NVLink enables seamless data exchange and parallel processing, thereby improving overall computing performance. Additionally, these processing devices are connected to multiple networks via one or more network interface cards (NICs) or DPUs, enabling the system to handle complex multi-network tasks with high bandwidth and low latency. This configuration makes the computing system 1300 ideal for demanding applications requiring significant processing power, such as artificial intelligence (AI), machine learning (ML), and data-intensive computing, while ensuring robust connectivity and scalability across a variety of networked environments. The integrated circuits of the computing system 1300 may include one or more CPUs and one or more GPUs. Figure 13 The diagram illustrates an example architecture for a multi-GPU architecture.

[0131] like Figure 13 As illustrated, the computing system 1300 includes a processing device 1302 with a multi-GPU architecture. Specifically, the processing device 1302 includes a CPU 1306, a GPU 1308, and a GPU 1310. The CPU 1306 may be coupled to the GPU 1308 via a chip-to-chip (D2D) or chip-to-chip (C2C) interconnect 1312 (such as a Ground Reference Signalling Interconnect (GRS interconnect)). The CPU 1306 may be coupled to the GPU 1310 via a D2D or C2C interconnect 1314. The CPU 1306 may also be coupled to the GPU 1308 and GPU 1310 via a PCIe interconnect. The CPU 1306 may be coupled to one or more network interface cards (NICs) or data processing units (DPUs), which in turn are coupled to one or more networks. For example, as Figure 13 As illustrated, CPU 1306 is coupled to a first NIC / DPU 1326, which is coupled to network 1330. CPU 1306 is also coupled to a second NIC / DPU 1328, which is coupled to network 1330. NIC / DPU 1326 and NIC / DPU 1328 can be coupled to network 1330 via Ethernet (ETH) or InfiniBand (IB) connections.

[0132] The computing system 1300 also includes a processing device 1304 with a multi-GPU architecture. Specifically, the processing device 1304 includes a CPU 1316, a GPU 1318, and a GPU 1320. The CPU 1316 can be coupled to the GPU 1318 via a D2D or C2C interconnect 1322. The CPU 1316 can be coupled to the GPU 1320 via a D2D or C2C interconnect 1324. The CPU 1316 can also be coupled to the GPU 1318 and GPU 1320 via a PCIe interconnect. The CPU 1316 can be coupled to one or more NICs or DPUs, which in turn are coupled to one or more networks. For example, as... Figure 13 As illustrated, CPU 1316 is coupled to a first NIC / DPU 1332, which is coupled to network 1336. CPU 1316 is also coupled to a second NIC / DPU 1334, which is coupled to network 1336. NIC / DPU 1332 and NIC / DPU 1334 can be coupled to network 1336 via Ethernet (ETH) or InfiniBand (IB) connections.

[0133] In at least one embodiment, processing device 1302 and processing device 1304 can communicate with each other via NIC / DPU 1338 (such as via PCIe interconnect). Processing device 1302 and processing device 1304 can also communicate with each other via high-bandwidth communication interconnect 1340 (such as NVLink interconnect or other high-speed interconnect).

[0134] In at least one embodiment, the computing system 1300 is used for high-speed network communication and includes a processing unit (e.g., CPU 1306, GPU 1308, GPU 1310, CPU 1316, GPU 1318, GPU 1320, NIC / DPU 1326, NIC / DPU 1328, NIC / DPU 1332, NIC / DPU 1334, or NIC / DPU 1338) and a network interface coupled to the processing unit. The network interface includes receiver circuitry, forward error correction (FEC) circuitry operatively coupled to the receiver circuitry, and a controller operatively coupled to both the receiver circuitry and the FEC circuitry. The controller can receive equalization error data from the receiver circuitry. The controller can use the equalization error data and nominal signal power to determine a signal-to-noise ratio (SNR) deviation metric, which indicates the estimated post-FEC bit error rate (BER) of the FEC circuitry. The controller can adjust at least one of the FEC parameters of the FEC circuitry or the link parameters of the receiver circuitry based on the SNR deviation metric. In some embodiments, one or more of NIC / DPU 1326, NIC / DPU 1328, NIC / DPU 1332, NIC / DPU 1334 or NIC / DPU 1338 are “smart NICs” disclosed herein.

[0135] Figure 14This is a block diagram of a computing system 1400 having a CPU 1402 and a GPU 1404 on a single integrated circuit, according to at least one embodiment. The computing system 1400 can be a highly integrated design where the CPU 1402 and GPU 1404 are connected on a single integrated circuit, thereby enabling fast, low-latency communication between the two processing units using NVLink C2C (chip-to-chip) interconnect 1406. This tight integration allows for efficient data transfer and parallel processing between the CPU 1402 and GPU 1404, thereby optimizing the execution of complex computational tasks. The GPU elements within the computing system 1400 can be interconnected using an NVLink network, allowing for scalability of up to 256 GPU elements, creating a robust unified processing environment for large-scale AI, ML, and high-performance computing applications. The NVLink network can be a GPU configuration with a high-bandwidth communication interconnect 1410. Additionally, the computing system 1400 can be designed to interface with high-speed I / O via PCIe interconnect 1408, thereby ensuring rapid data transfer with external devices, further enhancing the system's ability to handle data-intensive tasks, and providing robust connectivity to peripheral components. It should be noted that since the CPU 1402 and GPU 1404 reside on the same integrated circuit, the C2C interconnect 1406 can be considered a D2D interconnect. The integrated circuit may include CPU memory (also referred to as main memory) and GPU memory, which the CPU 1402 and GPU 1404 can access respectively via a high-speed interconnect. The computing system 1400 can combine the performance of the GPU 1404 with the versatility of the CPU 1402. The CPU 1402 can be connected to a single integrated circuit via a high-bandwidth and memory-coherent C2C interconnect 1406. The computing system 1400 can support a link-switching system.

[0136] In at least one embodiment, the computing system 1400 is used for high-speed network communication and includes a processing unit (e.g., CPU 1402, GPU 1404, NVLink network) and a network interface coupled to the processing unit. In some embodiments, the computing system 1400 may include a “smart NIC” as disclosed herein. In some embodiments, the computing system 1400 may be part of an RDMA network, and a smart NIC may be used to achieve high performance for AMO RPC.

[0137] Figure 15This is a block diagram of a computing system 1500 having a Tensor Core GPU 1508 according to at least one embodiment. The computing system 1500 may be a DBX H100 system, which is a high-performance computing platform designed to meet the needs of AL, ML, and deep learning (DL) workloads. The computing system 1500 may include multiple Tensor Core GPUs 1508 (e.g., NVIDIA H100 Tensor Core GPUs). Each of the Tensor Core GPUs 1508 may be one of the above-described... Figure 14 One of the integrated circuits described is the Tensor Core GPU 1508, which is optimized for AI / ML / DL applications, delivering superior performance for deep learning training, inference, and high-performance computing tasks. The Tensor Core GPUs 1508 within the Computing System 1500 interconnect using high-speed communication interfaces such as NVLinks, enabling rapid data transfer between them, crucial for handling large-scale AI models and datasets with low latency. The Computing System 1500 is designed for scalability, allowing for the integration of additional GPUs as needed, providing sufficient diversity for research, development, and deployment in data centers used for production AI workloads. Each GPU is equipped with a Tensor Core, a dedicated processing unit that accelerates matrix operations and is a fundamental component of AI and deep learning algorithms. These Tensor Cores enable the system to perform mixed-precision computations efficiently, balancing speed and accuracy. Considering the power consumption and heat generation of multiple Tensor Core GPUs 1508, the Computing System 1500 may include advanced cooling solutions and power management features to ensure safe operation while maintaining peak performance. It is supported by a comprehensive software ecosystem that includes NVIDIA's CUDA programming model, AI frameworks such as TensorFlow and PyTorch, and other HPC and AI software tools that enable developers and researchers to fully leverage the TensorCore GPU 1108 for their specific applications. The Computing System 1500 is ideal for large-scale AI model training, real-time inference, scientific simulations, data analysis, and other compute-intensive tasks requiring significant parallel processing power.

[0138] The Tensor Core GPU 1508 can be coupled to multiple CPUs, such as CPU 1502 and CPU 1504, using a switch 1506 (e.g., a CX7 HCA / NIC with a PCIe switch). The Tensor Core GPU 1508 can be coupled to each other via a switch 1510 (e.g., an NVSwitch). Switches 1506 and 1510 can be coupled to a high-speed transceiver module 1512. The high-speed transceiver module 1512 can be an eight-channel small form factor pluggable (OSFP) module. OSFP modules are high-speed transceiver modules designed for fast data communication (specifically, in environments requiring significant bandwidth, such as data centers and high-performance computing systems). These modules support extremely high data rates, typically up to 400 Gbps per module, with future capabilities expanding to 800 Gbps or higher. OSFP modules interface with the system via a PCIe interface, enabling fast and efficient data transfer between the integrated CPU-GPU components and external networks or other connected systems. Their hot-swappable nature allows for easy insertion or removal without shutting down the system, providing flexibility and maintainability crucial in critical uptime environments. Additionally, OSFP modules are designed for high density, maximizing the number of high-speed connections within limited spaces, such as dense server racks. By complying with the latest networking standards, OSFP modules ensure that the Compute System 1500 can meet ever-increasing data demands and can be upgraded to support future advancements in network speeds, thus contributing to improved overall system performance and scalability.

[0139] In at least one embodiment, the computing system 1500 can be viewed as a data network configuration with full-bandwidth in-server NVLinks. In this example, all eight tensor core GPUs 1508 can simultaneously saturate eighteen NVLinks within the server leading to other GPUs. Bandwidth is limited by over-subscription from multiple other GPUs. In another embodiment, the data network configuration can be half-bandwidth in-server NVLinks. In this example, all eight tensor core GPUs 3008 can half-subscribe to eighteen NVLinks leading to GPUs in other servers. Four tensor core GPUs 1508 can saturate eighteen NVLinks leading to GPUs in other servers. This is equivalent to full bandwidth on AllReduce with Scalable Hierarchical Aggregation and Reduction Protocol (SHARP). The reduction in all-to-all (All2All) bandwidth is a trade-off between server complexity and cost. In at least one embodiment, all eight tensor core GPUs 1508 can independently transfer data in a multi-track InfiniBand / Ethernet configuration using the Remote Direct Memory Access (RDMA) protocol via their own dedicated switches (e.g., 400Gb / sHCA / NIC). In this example, for non-NVLink network devices, the total aggregated full-duplex bandwidth is 800 GBp.

[0140] In at least one embodiment, the computing system 1500 is used for high-speed network communication and includes a processing unit (e.g., CPU 1502, CPU 1504, switch 1506, Tensor Core GPU 1508, switch 1510, high-speed transceiver module 1512) and a network interface coupled to the processing unit. In some embodiments, the network interface may be a “smart NIC” as disclosed herein.

[0141] Other variations are within the spirit of this disclosure. Therefore, while the disclosed technology is readily adaptable to various modifications and alternative configurations, certain embodiments thereof are illustrated in the accompanying drawings and have been described in detail above. However, it should be understood that the disclosure is not intended to be limited to one or more specific forms disclosed, but rather, it is intended to cover all modifications, alternative configurations, and equivalents falling within the spirit and scope of this disclosure as defined in the appended claims.

[0142] Unless otherwise stated or obviously contradicted by the context, the terms “a,” “an,” and “the,” and similar references, used in the context of describing the disclosed embodiments (particularly in the context of the appended claims), should be interpreted as encompassing both singular and plural forms, rather than as definitions of terms. Unless otherwise stated, the terms “comprising,” “having,” “including,” and “containing” should be interpreted as open-ended terms (meaning “including, but not limited to”). “Connection” (referring to a physical connection where not modified) should be interpreted as partially or wholly included, attached to, or connected together, even with some intervention. Unless otherwise indicated herein, references to numerical ranges herein are intended only as a way of abbreviating each individual value falling within that range, and each individual value is incorporated into the specification as if it were separately described herein. In at least one embodiment, unless otherwise indicated or contradicted by the context, the use of the terms “set” (e.g., “item set”) or “subset” should be interpreted as a non-empty set comprising one or more members. Furthermore, unless otherwise indicated or contradicted by the context, the term “subset” of the corresponding set does not necessarily mean an appropriate subset of the corresponding set, but rather that the subset and the corresponding set can be equal.

[0143] Unless otherwise explicitly stated or clearly contradicted by the context, connective phrases such as “at least one of A, B, and C” or “at least one of A, B, and C” are understood in the context to generally refer to items, terms, etc., which can be A or B or C, or any non-empty subset of the set A, B, and C. For example, in an exemplary example of a set with three members, the connective phrases “at least one of A, B, and C” and “at least one of A, B, and C” refer to any of the following sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Therefore, such connective language is generally not intended to imply that some embodiments require the presence of at least one of A, at least one of B, and at least one of C. Additionally, unless otherwise stated or contradicted by the context, the term “multiple” indicates a plural state (e.g., “multiple items” means multiple items). In at least one embodiment, the number of items in the multiple items is at least two, but may be more if explicitly indicated or indicated by the context. Furthermore, unless otherwise stated or clearly understood from the context, the phrase “based on” means “at least partially based on” or “at least based on”, not “based on only”.

[0144] Unless otherwise indicated herein or clearly contradicted by the context, the operations of the processes described herein may be performed in any suitable order. In at least one embodiment, processes such as those described herein (or variations thereof and / or combinations thereof) are executed under the control of one or more computer systems configured with executable instructions and are implemented as code (e.g., executable instructions, one or more computer programs, or one or more application programs) that are executed jointly on one or more processors by hardware or a combination thereof, for example, the code is stored in the form of a computer-readable storage medium as a computer program, which in at least one embodiment includes a plurality of instructions executable by one or more processors. In at least one embodiment, the computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transient signals (e.g., propagating transient electrical or electromagnetic transmissions) but includes non-transitory data storage circuitry (e.g., buffers, caches, and queues). In at least one embodiment, code (e.g., executable code or source code) is stored on one or more non-transitory computer-readable storage media (or other memory for storing executable instructions) on which executable instructions are stored, which, when executed by one or more processors of a computer system (i.e., as a result of execution), cause the computer system to perform the operations described herein. In at least one embodiment, the set of non-transitory computer-readable storage media comprises multiple non-transitory computer-readable storage media, and one or more of the individual non-transitory storage media lack all the code, but the multiple non-transitory computer-readable storage media collectively store all the code. In at least one embodiment, the executable instructions are executed such that different instructions are executed by different processors; for example, the non-transitory computer-readable storage media store the instructions, and the main central processing unit (“CPU”) executes some instructions while the graphics processing unit (“GPU”) executes other instructions. In at least one embodiment, different components of the computer system have separate processors, and the different processors execute different subsets of the instructions.

[0145] Therefore, in at least one embodiment, the computer system is configured to implement one or more services that perform the operations of the processes described herein, either individually or collectively, and such a computer system is configured with suitable hardware and / or software to enable the implementation of the operations. Furthermore, the computer system implementing at least one embodiment of this disclosure is a single device, and in another embodiment it is a distributed computer system comprising multiple devices operating in different ways, such that the distributed computer system performs the operations described herein, and that a single device does not perform all the operations.

[0146] The use of any and all examples or exemplary language (e.g., “such as”) provided herein is intended only to better illustrate embodiments of this disclosure and does not constitute a limitation on the scope of the disclosure unless otherwise required. No language in the specification should be construed as indicating that any unclaimed element is essential to the practice of the disclosure.

[0147] All references cited in this article, including publications, patent applications and patents, are incorporated herein by reference as if each reference were individually and specifically indicated to be incorporated herein by reference and the entire contents of which are described herein.

[0148] The terms “coupled” and “connected”, and their derivatives, may be used in the specification and claims. It should be understood that these terms may not be intended to be synonyms with each other. Rather, in certain examples, “connected” or “coupled” may be used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. “Coupled” may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0149] Unless otherwise expressly stated, it will be understood that in some embodiments, throughout the specification, terms such as “processing,” “computing,” “determining,” etc., refer to the actions and / or processes of a computer or computing system or similar electronic computing device that process and / or convert data represented as physical quantities (e.g., electrons) in the registers and / or memory of the computing system into other data represented as physical quantities in the memory, registers, or other such information storage, transmission, or display devices of the computing system.

[0150] Similarly, the term "processor" can refer to any device or part of memory that processes electronic data from registers and / or memory and converts that electronic data into other electronic data that can be stored in registers and / or memory. As a non-limiting example, a "processor" can be a CPU or a GPU. A "computing platform" can include one or more processors. As used herein, in at least one embodiment, a "software" process can include, for example, software and / or hardware entities that perform work over time, such as tasks, threads, and intelligent agents. Likewise, each process can refer to multiple processes that execute instructions sequentially or intermittently, or in parallel. In at least one embodiment, the terms "system" and "method" are used interchangeably herein, provided that a system can embody one or more methods, and a method can be considered a system.

[0151] In this document, reference may be made to obtaining, acquiring, receiving, or inputting analog or digital data into a subsystem, computer system, or computer-implemented machine. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog and digital data can be accomplished in various ways, such as by receiving data as a parameter to a function call or a call to an application programming interface. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data can be accomplished by transmitting data via a serial or parallel interface. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data can be accomplished by transmitting data from a providing entity to an acquiring entity via a computer network. In at least one embodiment, reference may also be made to providing, outputting, transmitting, sending, or presenting analog or digital data. In various examples, the process of providing, outputting, transmitting, sending, or presenting analog or digital data can be implemented by transmitting data as an input or output parameter to a function call, an application programming interface, or an inter-process communication mechanism.

[0152] While exemplary embodiments of the described technologies are illustrated herein, other architectures may be used to implement the described functionality and are intended to fall within the scope of this disclosure. Furthermore, although specific assignments of responsibilities may be defined above for descriptive purposes, various functions and responsibilities may be assigned and divided in different ways depending on the circumstances.

[0153] Furthermore, although the subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter claimed in the appended claims is not necessarily limited to the specific features or actions described. Rather, specific features and actions are disclosed as exemplary forms for implementing the claims.

Claims

1. A system comprising: A network interface card (NIC), wherein the network interface card (NIC) includes a first memory; as well as One or more processors, said one or more processors coupled to the first memory, said one or more processors being used for: Receives AMO Remote Procedure Calls (RPCs) containing the memory address and AMO type atomic memory operation. Retrieve from the second memory the value corresponding to the memory address of the atomic memory operation AMO remote procedure call RPC; Perform an atomic memory operation (AMO) corresponding to the type of atomic memory operation (AMO) on the value from the second memory to obtain a modified value; as well as The modified value is stored in the first memory.

2. The system as claimed in claim 1, wherein, The first memory and the one or more processors are included within the data processing unit (DPU) of the network interface card (NIC), and wherein the second memory is associated with one or more host processors.

3. The system as described in claim 2, wherein, The atomic memory operation AMO remote procedure call (RPC) is received from one or more host processors.

4. The system as claimed in claim 1, wherein, The one or more processors are also used for: The target memory for the modified value is determined between at least the first memory and the second memory; as well as The modified value is stored in the target memory.

5. The system as claimed in claim 1, wherein, The one or more processors are also configured to store the modified value in the second memory.

6. The system of claim 5, wherein, The one or more processors are configured to store the modified value in the second memory in response to a memory refresh trigger.

7. The system of claim 6, wherein, The memory refresh trigger is at least one of the following: Receive memory refresh command; or Based on one or more heuristics, it is determined that the state of the system satisfies the refresh criterion.

8. The system of claim 1, wherein, The atomic memory operation AMO corresponding to the atomic memory operation AMO type includes at least one of the following: Compare and swap operations; Take, join, and add operations; Fetch and store operation; The AND-XOR operation is used to perform the AND-XOR operation. Atom increment operation; Atom decrement operation; Exchange operation; or Software-defined operations.

9. The system of claim 1, wherein, The atomic memory operation AMO remote procedure call (RPC) is received from a remote processing device.

10. The system of claim 1, further comprising: A host computing device connected to the network interface card (NIC), the host computing device including one or more host processors and a second memory, wherein the second memory is associated with the one or more host processors, and wherein the memory address of the atomic memory operation (AMO) remote procedure call (RPC) points to the second memory associated with the one or more host processors.

11. A method, the method comprising: The first processor, coupled to the first memory, receives an AMO remote procedure call (RPC) of type AMO, which includes the memory address and the atomic memory operation AMO. Retrieve from the second memory the value corresponding to the memory address of the atomic memory operation AMO remote procedure call RPC; The first processor performs an atomic memory operation (AMO) corresponding to the type of atomic memory operation (AMO) on the value from the second memory to obtain a modified value; as well as The modified value is stored in the first memory.

12. The method of claim 11, further comprising storing the modified value in the second memory.

13. The method of claim 12, wherein, The step of storing the modified value in the second memory is performed in response to a memory refresh trigger.

14. The method of claim 13, wherein, The memory refresh trigger is at least one of the following: Receive memory refresh command; or Based on one or more heuristics, it is determined that the state of a device network satisfies a refresh criterion, wherein the device network includes a first device, the first device including the first processor and the first memory.

15. The method of claim 11, wherein, The atomic memory operation AMO corresponding to the atomic memory operation AMO type includes at least one of the following: Compare and swap operations; Take, join, and add operations; Fetch and store operation; The AND-XOR operation is used to perform the AND-XOR operation. Atom increment operation; Atom decrement operation; Exchange operation; or Software-defined operations.

16. The method of claim 11, wherein, The atomic memory operation AMO remote procedure call (RPC) is received from a remote processing device.

17. A network interface card (NIC), the NIC including processing circuitry for performing operations, the operations including: Receives AMO Remote Procedure Calls (RPCs) containing the memory address and AMO type atomic memory operation. Retrieve from external memory the value corresponding to the memory address of the AMO remote procedure call RPC of the atomic memory operation; Perform an atomic memory operation (AMO) corresponding to the type of atomic memory operation (AMO) on the value from the external memory to obtain the modified value; as well as The modified value is stored in local memory.

18. The network interface card (NIC) of claim 17, wherein the operation further includes storing the modified value in the external memory.

19. The network interface card (NIC) as described in claim 18, wherein, The storage of the modified value in the external memory is in response to a memory refresh trigger.

20. The network interface card (NIC) as described in claim 19, wherein, The memory refresh trigger is at least one of the following: Receive memory refresh command; or Based on one or more heuristics, determine whether the state of the network interface card (NIC) and / or the local memory meets the refresh criteria.

21. A data center, the data center comprising: A plurality of host computing devices interconnected via a plurality of switches, wherein one or more of the plurality of host computing devices include: One or more host processors; Host memory associated with the one or more host processors; and A network interface card (NIC), the NIC including additional memory and one or more additional processors coupled to the additional memory, the one or more additional processors being used for: Receives an Atomic Memory Operation (AMO) Remote Procedure Call (RPC) of type AMO, which includes the memory address of the host memory. Retrieve from the host memory the value corresponding to the memory address of the AMO remote procedure call RPC; Perform an atomic memory operation (AMO) corresponding to the type of atomic memory operation (AMO) on the value from the host memory to obtain a modified value; and The modified value is stored in the additional memory.

22. The data center of claim 21, wherein, The additional memory and the one or more additional processors are included within the data processing unit (DPU) of the network interface card (NIC).

23. The data center of claim 21, wherein, The one or more additional processors are also used for: The target memory for the modified value is determined between at least the host memory and the additional memory; as well as The modified value is stored in the target memory.

24. The data center of claim 21, wherein, The one or more additional processors are also used to store the modified value in the host memory.

25. The data center as described in claim 24, wherein, The one or more additional processors are used to store the modified value in the host memory in response to a memory refresh trigger.

26. The data center of claim 25, wherein, The memory refresh trigger is at least one of the following: Receive memory refresh command; or Based on one or more heuristics, it is determined that the state of the data center satisfies the refresh criteria.

27. The data center of claim 21, wherein, The atomic memory operation AMO corresponding to the atomic memory operation AMO type includes at least one of the following: Compare and swap operations; Take, join, and add operations; Fetch and store operation; The AND-XOR operation is used to perform the AND-XOR operation. Atom increment operation; Atom decrement operation; Exchange operation; or Software-defined operations.

28. The data center as described in claim 21, wherein, The atomic memory operation AMO remote procedure call (RPC) is received from a remote processing device.

Images