Processing TCP packets with one DPU and one GPU

Abstract

Devices, systems, methods, and techniques for obtaining information transmitted in data packets and for storing data, based at least partially on that information, in the GPU's memory. In at least one embodiment, the information is obtained and stored in the GPU's memory without using a central processing unit (CPU). In at least one embodiment, a data processing unit (DPU) receives incoming data packets, stores information based at least partially on the data packets in the DPU's memory, and initiates a transfer directly to the GPU's memory. In at least one embodiment, the data packets are TCP data packets.

Description

CROSS-REFERENCE TO RELATED REGISTRATIONS

[0001] The present application claims priority over provisional US patent application No. 63 / 688,807, filed on August 29, 2024, entitled “PROCESSING TCP PACKETS WITH A DPU AND GPU”, the contents of which are deemed to be incorporated in full by reference. TECHNICAL AREA

[0002] At least one embodiment relates to the use of a data processing unit (DPU) operating in conjunction with a graphics processing unit (GPU) to receive and process data packets according to the Transmission Control Protocol (TCP). At least one embodiment relates to instructions configured to cause at least one DPU to receive data packets and store them in the GPU's memory. For example, at least one embodiment relates to processors or computer systems used to receive data packets and store information contained in the data packets in memory associated with at least one parallel processing unit (PPU), according to various novel techniques described herein. STATE OF THE ART

[0003] In a conventional computer network, connections according to the Transmission Control Protocol (TCP) are handled by a central processing unit (CPU) of a host. The host CPU must manage all aspects of communication using TCP data packets, including managing the TCP software (SW) stack, parsing TCP headers, and reordering packets to reconstruct the original payload (e.g., if the host CPU is a component of an HTTP server receiving files). Processing TCP data packets can consume CPU cycles, preventing the host CPU from performing other critical tasks. Additionally, moving data from CPU memory to GPU memory can take several milliseconds, contributing to increased overall latency. Improvements in TCP data packet processing are desirable. SUMMARY

[0004] The invention is defined by the claims. To illustrate the invention, aspects and embodiments are described here that may or may not fall within the scope of the claims.

[0005] Devices, systems, methods, and techniques for obtaining information transmitted in data packets and for storing data, based at least partially on that information, in the GPU's memory. In at least one embodiment, the information is obtained and stored in the GPU's memory without using a central processing unit (CPU). In at least one embodiment, a data processing unit (DPU) receives incoming data packets, stores information based at least partially on the data packets in the DPU's memory, and initiates a transfer directly to the GPU's memory. In at least one embodiment, the data packets are TCP data packets.

[0006] Each feature of an aspect or embodiment can be applied to other aspects or embodiments in any suitable combination. In particular, each feature of a process aspect or embodiment can be applied to a device aspect or embodiment, and vice versa. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 illustrates a functional block diagram of an exemplary processing system according to at least one embodiment; Fig. Figure 2 illustrates a functional block diagram depicting a data processing unit (DPU) that receives data packets and stores the data packets in the GPU memory, according to at least one embodiment; Fig. 3 is a flowchart illustrating a process described in Fig. 1 and Fig. 2 illustrated system can be implemented according to at least one embodiment; Fig. 4A illustrates an example of a system comprising a driver and / or a runtime environment that includes one or more libraries providing one or more application programming interfaces (APIs), according to at least one embodiment; Fig. 4B is a block diagram illustrating an example of a processor and modules according to at least one embodiment; Fig. 5A illustrates logic according to at least one embodiment; Fig. 5B illustrates logic according to at least one embodiment; Fig. Figure 6 illustrates an exemplary data center system according to at least one embodiment; and Fig. Figure 7 is a block diagram illustrating a computer system according to at least one embodiment. DETAILED DESCRIPTION

[0007] In at least one embodiment, a data processing unit (DPU) handles TCP processing and delivers data received in TCP packets (e.g., an HTTP file) directly to the GPU's memory (e.g., via a direct memory access transfer, or DMA). For example, an application executed by at least one processor of the DPU can obtain data packets from a network interface (e.g., a network adapter), retrieve the data transmitted with those packets, store the data in the host GPU's memory, and set a flag indicating that new data has been stored in the host GPU's memory. One or more functions (e.g.,(a persistent kernel), which are performed by the GPU, can query the flag and, after determining that new data has been stored in the GPU's memory, access the data, perform one or more processing operations on the data, and send an acknowledgment to the DPU indicating that the data was successfully accessed. In at least one embodiment, using the DPU to handle the TCP data allows the GPU to process TCP data in real time.

[0008] Fig. Figure 1 illustrates a functional block diagram of an exemplary processing system 100 according to at least one embodiment. The processing system 100 comprises at least one data processing unit (DPU) 130 configured to execute an application 132 that handles all TCP processing and directly outputs the data (e.g., an HTTP file) to the GPU's memory (e.g., the one in Fig. The DPU 130 provides GPU memory 214 (illustrated in Figure 2), which can be accessed by one or more graphics processing units (“GPU(s)”) 104 (e.g., via a direct memory access (DMA) transfer). The DPU 130 can receive data (e.g., TCP data packets) from a sending device (e.g., the device 134) connected to a network 128 (e.g., via a network adapter 126) and process this incoming data (e.g., store it in the GPU memory 214). The DPU 130 can receive data from the GPU(s) 104 and send this data (e.g., as TCP data packets via the network adapter 126) to a receiving device (e.g., the device 134) connected to the processing system 100 via the network 128.

[0009] In at least one embodiment, the processing system 100 comprises one or more processors 102, which may include the GPU(s) 104, and may be a single-processor desktop system, a multi-processor workstation system, or a server system with a large number of processors 102 and / or processor cores 106. For example, the processing system 100 may implement one or more computing devices, one or more network devices (e.g., computer devices, switches, routers, aggregators, telemetry servers, etc.), one or more sensors, a data center, a cloud computing system, and / or the like.

[0010] In at least one embodiment, the processor(s) 102 each comprise one or more processor cores 106 configured to process instructions (e.g., the instructions 122) which, when executed, perform operations for system and / or user software. In at least one embodiment, the processor core(s) 106 is configured to process an instruction set 108. In at least one embodiment, the instruction set 108 can enable Complex Instruction Set Computing (“CISC”), Reduced Instruction Set Computing (“RISC”), or Computing via a Very Long Instruction Word (“VLIW”). In at least one embodiment, the processor cores 106 can process different instruction sets 108, each instruction set 108 potentially including instructions to enable the emulation of other instruction sets.In at least one embodiment, the processor core(s) 106 may also include other processing devices, such as a digital signal processor (“DSP”).

[0011] In at least one embodiment, the processor / each of the processors 102 comprises a cache memory (“cache”) 110. In at least one embodiment, the processor / each of the processors 102 may have a single internal cache or multiple levels of internal cache. In at least one embodiment, the cache memory is shared by different components of the processor / each of the processors 102. In at least one embodiment, the processor / each of the processors 102 additionally comprises a register set 112, which may include different types of registers for storing different types of data (e.g., integer registers, floating-point registers, status registers, and an instruction pointer register). In at least one embodiment, the register set 112 may include general-purpose registers or other registers.

[0012] In at least one embodiment, the processor(s) 102 are coupled to one or more interface buses 114 to transmit communication signals, such as address, data, or control signals, between the processor(s) 102 and other components in the processing system 100. In at least one embodiment, the interface bus(s) 114 may comprise at least one processor bus, such as a version of a Direct Media Interface (“DMI”) bus. In at least one embodiment, the interface bus(s) 114 are not limited to one or more DMI buses, but may comprise one or more Peripheral Component Interconnect (“PCI”) buses (e.g., PCI Express (“PCIe”) buses), one or more memory buses, or other types of interface buses.In at least one embodiment, the processor(s) 102 comprise an integrated memory control device 116 and a platform controller hub (“PCH”) 118. In at least one embodiment, the memory control device 116 enables communication between a memory device 120 and other components of the processing system 100, while the PCH 118 provides connections to input / output (“I / O”) devices 121, such as a keyboard, mouse, data storage device and display unit (not shown), and the network adapter 126.

[0013] In at least one embodiment, the memory device 120 can be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, a phase-change memory device, or another memory device with performance suitable for processor memory. In at least one embodiment, the memory device 120 can function as system memory for the processing system 100 to store instructions 122 and data 124 for use when the processor(s) 102 executes an application or process. In at least one embodiment, the memory control device 116 is coupled to the GPU(s) 104 in the processor(s) 102 to perform graphics and media operations.

[0014] In at least one embodiment, the network adapter 126 (e.g., NVIDIA ConnectX) provides an interface with the network 128. In at least one embodiment, the network adapter 126 controls the interaction between the DPU 130 and the network 128, such as a data center network and / or the internet. In at least one embodiment, the interface bus(s) 114 can comprise one or more PCI buses (e.g., implemented according to PCIe Gen 5) which are connected to at least one connector providing an interface between the PCI bus(s) and the DPU 130.

[0015] A TCP stack refers to different layers of a communication protocol, similar to the layers defined by an Open Systems Interconnection (OSI) model. The TCP stack includes an application layer, a transport layer, a network access layer, a network interface layer, and a hardware layer. In at least one embodiment, the DPU 130 implements some or all of the different layers of the TCP stack.

[0016] In at least one embodiment, at least a part of the processing system 100 is implemented using at least a part of any system(s) that, with respect to the Fig. 5 to Fig. 7 is / are shown and / or described. In at least one embodiment, at least one section of the processing system 100 is used to implement at least one section of any system(s) that, with respect to the Fig. 5- Fig. 7 is / are shown and / or described.

[0017] Fig. Figure 2 illustrates a functional block diagram depicting the DPU 130 receiving data packets 204 (e.g., TCP packets) and storing the data packets 204 in the GPU memory 214, according to at least one embodiment. Fig. 2. The DPU 130 interacts with the GPU(s) 104. In at least one embodiment, the DPU 130 is implemented with an Advanced RISC Machine (ARM) architecture. The DPU 130 can have one or more processors (e.g., the one(s) in Fig. 4B illustrated processor(s) 422) which may be implemented using one or more ARM Cores 200. TCP batch management can be handled by the processor(s) 422 (e.g., one of the ARM Core(s) 200) of the DPU 130. The processor(s) of the DPU 130 can also perform other tasks normally handled by the host CPU (e.g., one or more of the processor(s) 102), such as parsing the TCP headers, reordering the packets to reassemble them and generate the original payload, and transferring the packets to GPU memory. Since the data packets 204 can be reassembled into TCP file segments or files by the DPU 130, these file segments or files can be transferred from the DPU 130 to the GPU memory 214 using DMA transfers.In at least one embodiment, at least one section of the processor / at least one of the processors of the DPU 130 is implemented using at least one part of any system / any systems that, with respect to the . Fig. 5 to Fig. 7 is / are shown and / or described. In at least one embodiment, at least one section of the processor / at least one of the processors of the DPU 130 is used to implement at least one section of any system / systems that, with respect to the Fig. 5 to Fig. 7 is / are shown and / or described.

[0018] The DPU 130 includes a working memory (e.g., one or more non-transitory processor-readable media) to hold processor-executable instructions (which are stored, for example, in modules 424 located in Fig. 4B are illustrated) to store, which, when accessed by the processor(s) (e.g., processor(s) 422, which is / are in Fig. Figure 4B illustrates the following: the DPU 130 can execute the application 132 and / or implement other functionality. As additional non-restrictive examples, the memory (e.g., one or more non-transient processor-readable media) of the DPU 130 can be implemented, for example, using volatile memory (e.g., dynamic random-access memory (“DRAM”)) and / or non-volatile memory (e.g., a hard disk, a solid-state device (“SSD”), and / or the like). The memory of the DPU 130 can include memory 208 capable of storing the data packets 204 and / or data obtained from the data packets 204 (e.g., a TCP frame and / or an HTTP file).In at least one embodiment, at least one section of the DPU 130's main memory is implemented using at least one section of any system(s) that, with respect to the . Fig. 5 to Fig. 7 is / are shown and / or described. In at least one embodiment, at least one section of the working memory 110 is used to implement at least one section of any system(s) that, with respect to the Fig. 5 to Fig. 7 is / are shown and / or described.

[0019] The processor(s) (e.g., one of the ARM Core(s) 200) of the DPU 130 may comprise one or more circuits that execute at least one section of the instructions stored in the DPU 130's main memory. The processor(s) of the DPU 130 may comprise one or more parallel processing units (“PPU(s)”) 130, such as one or more GPU(s) and / or the like. The processor(s) of the DPU 130 may be implemented, for example, using one of the ARM Core(s) 200, a main central processing unit (“CPU”) complex, one or more microprocessors, one or more microcontrollers, the PPU(s) 130 (e.g., GPU(s)), one or more DPU(s), one or more arithmetic logic units (“ALU(s)”), and / or the like.

[0020] The processor(s) and / or the main memory of the DPU 130 can communicate with each other via one or more connections or buses 240, such as a Peripheral Component Interconnect Express (“PCIe”) connection (or a bus) and / or the like. In at least one embodiment, at least one section of this bus(s) 240 is implemented using at least one section of any system(s) that, with respect to the Fig. 5 to Fig. 7 is / are shown and / or described. In at least one embodiment, at least one section of the connection(s) 118 is used to implement at least one section of any system(s) that, with respect to the Fig. 5 to Fig. 7 is / are shown and / or described.

[0021] In at least one embodiment, at least one section of DPU 130 is implemented using at least one section of any system(s) that, with respect to the Fig. 5 to Fig. 7 is / are shown and / or described. In at least one embodiment, at least one section of DPU 130 is used to implement at least one section of any system(s) that, with respect to the Fig. 5 to Fig. 7 is / are shown and / or described.

[0022] The application 132 can implement a communication interface (e.g., a TCP / HTTP server 206) and / or data transfer functionality (e.g., DMA functionality implemented by a Data Center Operations and Control Architecture (DOCA) DMA plug-in 210). During operation, the network adapter 126 communicates with the network 128 to receive the multiple incoming data packets 204 (e.g., TCP packets). In at least one embodiment, the network adapter 126 uses one or more application programming interfaces (APIs) to access one or more Linux sockets to interact with the network 128 and thus send and receive packets. In at least one embodiment, the communication interface (e.g., the TCP / HTTP server 206, such as...) processes...NGINX receives the incoming data packets (204) and, if necessary, reassembles them to reconstruct the original TCP frame and / or HTTP file, transferring the reconstructed packets to memory (208) for temporary storage. The reconstructed frames / file are stored in memory (208) and await transfer.

[0023] In at least one embodiment, the data transfer functionality (implemented, for example, by the DOCA DMA Plug-In 210) controls the transfer of data from the main memory 208 to the GPU main memory 214, which is associated with the GPU(s) 104. In at least one embodiment, the data transfer functionality (implemented, for example, by the DOCA DMA Plug-In 210) detects that a new HTTP packet / file is ready and then triggers a data transfer from the main memory 208 to the GPU main memory 214, which is associated with the GPU(s) 104. In at least one embodiment, the data transfer is handled by a DMA functional unit (not shown) in the DPU 130. The DMA functional unit can be a component of the data transfer functionality and / or be used by it. In at least one embodiment, the data transfer is a DMA data transfer that uses bus(s) 240 (e.g.one or more PCIe interface buses). Following the transfer, the DPU 130 receives an acknowledgment signal (“ACK” signal) 212 and / or a request to the DPU 130 to send further data from the GPU(s) 104.

[0024] In at least one embodiment, the data transfer functionality (implemented, for example, by the DOCA DMA Plug-In 210) stores the transferred data in the GPU memory 214, which is a dedicated memory or part of the memory device 120 in Fig. 1. In at least one embodiment, the data is stored in a memory buffer 216 within the GPU memory 214. Following the transfer, the data transfer functionality sends an ACK signal 218 from the DPU 130 to the GPU memory 214 to inform the GPU(s) 104 that new data has been received from the DPU 130.

[0025] Elements within the GPU(s) 104 detect the completion of the data transfer (which is implemented, for example, using DMA). In at least one embodiment, a first Compute Unified Device Architecture (CUDA) kernel 222, executed by the GPU(s) 104 on the host side, creates a queue pair (QP) dedicated to memory copy operations associated with the DMA functional unit (not shown) in the DPU 130, thus exposing the DPU functional unit for use by any application executed by the GPU(s) 104 (e.g., CUDA tasks 230). In at least one embodiment, a CUDA kernel for data polling (referred to as CUDA data polling kernel 224) directly detects the ACK signal 218 from the DPU 130, which indicates that new data is stored in the GPU memory 214.The CUDA data polling kernel 224 is provided with a specific location in the GPU memory 214 and the size of the received data (e.g., the number of bytes). In at least one embodiment, the GPU memory 214 is pre-allocated by the CPU 102, and the address is communicated to the remote peer before the CUDA kernel (e.g., the CUDA kernel 222) is started. The location and size of this allocation are communicated to the DMA functional unit and / or the data transfer functionality for use in performing data transfers. Similarly, the CPU 102 pre-allocates a memory location where the data transfer functionality (e.g., the DMA functional unit) can store the ACK signal 218.

[0026] In at least one embodiment, the CUDA kernel 222 acts as a proxy to receive the DMA data and performs an initial analysis phase before passing only relevant / filtered / interesting data to the next CUDA task. Thus, other CUDA tasks / kernels do not require the DMA QP data as a reference. In this way, the second CUDA kernel and the DMA functional unit can communicate with each other regarding memory copy operations. Alternatively, the CPU 102 can pre-start a CUDA kernel or CUDA tasks 230 in such a way that the CUDA kernel or tasks are in an active wait mode until the CUDA kernel 222 sends a notification that processing of the received data should begin.

[0027] In at least one embodiment, the first CUDA kernel 222 acts as a proxy, receiving the TCP data via DMA transfer from the DPU 130 and sending the ACK signal 212 to the DPU via DMA.

[0028] In at least one embodiment, a GPUNetIO library 226 provides functions that can be called by a CUDA kernel. In one embodiment, the GPUNetIO library 226 acknowledges receipt of the new data to the DPU 130 and provides feedback to the DPU 130 indicating that the GPU(s) 104 is / are ready to receive further data. For example, the GPUNetIO library 226 can send the ACK signal 212 to the DPU 130.

[0029] In at least one embodiment, the GPUNetIO library 226 is implemented as a DOCA GPUNetIO library, which provides the first CUDA kernel 222 with a set of functions that allow it to create a work queue entry (WQE) in the QP, create the WQE memory copy, and queue the WQE in the QP. The DOCA GPUNetIO library 226 can directly update a register in the DPU 130 to notify it that a new copy is to be executed. In response, the DMA functional unit "wakes up," reads the WQE memory copy, and executes it.

[0030] In at least one embodiment, the first CUDA kernel 222 and / or a CUDA kernel 228 perform tasks such as further analysis, processing, filtering (e.g., Snort rules, regular expressions, or the like), and / or selecting one or more sections of the data, files, and / or packets that may be relevant for the next CUDA task(s) (e.g., one or more of the CUDA tasks 230). In at least one embodiment, the CUDA tasks 230 can perform further operations on the received data. This can include, for example, security analysis for network security applications that process data for artificial intelligence to detect specific features in the data. In another example, the CUDA tasks 230 can generate retrieval-augmented generated (RAG) embeddings suitable for storage in a vector database.The GPU(s) 104 can perform other operations, and the processing system 100 is not limited to these few examples.

[0031] In at least one embodiment, the use of the DPU 130 to handle TCP data enables the GPU(s) 104 to process TCP data in real time. Each of the GPU(s) 104 can comprise multiple GPU cores 220. The GPU(s) 104 can execute the CUDA data polling kernel 224 to immediately detect new data stored in the GPU memory 214. This entire process bypasses the host CPU, allowing the host CPU to perform other tasks.

[0032] Thus, the application 132, which is executed by at least one processor of the DPU 130, can obtain the data packets 204 from the network adapter 126, obtain data transmitted with the data packets 204, and use the data transfer functionality (implemented, for example, by the DOCA DMA Plug-In 210) to store the data in the host GPU memory 214 and send the ACK signal 218 to set a flag indicating that new data has been stored in the host GPU memory 214. One or more functions (e.g., the CUDA data polling kernel 224) executed by the GPU(s) 104 can query the flag in the polling procedure and, after determining that new data has been stored in the GPU memory 214, access the data, perform one or more processing operations with respect to the data, and issue an acknowledgment (e.g.,send the ACK signal 212) to the DPU 130, which indicates that the data has been successfully accessed.

[0033] Fig. Figure 3 is a flowchart illustrating a method 300 according to at least one embodiment. The method 300 can be described in the Fig. 1 and Fig. The illustrated processing system 100 is carried out. At start 302 (see Fig. 3) The processing system 100 is configured to receive and process incoming data. At block 304, the network adapter 126, which is connected to the DPU 130, receives a series of packets (e.g., data packets 204). The network adapter 126 interacts with the network 128 to receive the series of packets (e.g., data packets 204) and interacts with the communication interface (e.g., the TCP / HTTP server 206) to transfer the series of packets for processing.

[0034] At block 306, the communication interface (e.g., the TCP / HTTP server 206) of the DPU 130 processes the incoming series of packets (e.g., the data packets 204) and, if necessary, reassembles them to reconstruct the original TCP frame and / or HTTP file. At block 308, the reconstructed data is transferred to the DPU 130's memory 208 for temporary storage.

[0035] At block 310, the data transfer functionality (e.g., implemented by the DOCA DMA Plug-In 210) of the DPU 130 determines that the newly assembled data is ready for transfer and initiates a transfer to the GPU memory 214. In at least one embodiment, the data transfer is a DMA transfer from the memory 208 to the GPU memory 214.

[0036] At block 312, the first CUDA kernel 222 (e.g., using the CUDA data polling kernel 224) detects the arrival of new data from the DPU 130 and begins processing the received data. In at least one embodiment, the CUDA data polling kernel 224 uses a polling method to detect the arrival of new data.

[0037] At block 314, the first CUDA kernel 222 causes the GPU(s) 104 to process the newly received data, for example by starting one or more CUDA kernels (e.g., one or more of the CUDA tasks 230). As noted above, the processing can encompass a wide range of applications, particularly security processing, AI applications, graphics processing applications, etc.

[0038] At block 316, the first CUDA kernel causes a library (e.g., the GPUNetIO library 226) to generate an acknowledgment signal (e.g., the ACK signal 212) and send it to DPU 130. DPU 130 receives the acknowledgment signal, indicating that GPU(s) 104 are ready for further data processing. Processing unit 100 then returns to block 304 to continue receiving and processing data packets.

[0039] Thus, the processing system 100 can offload all TCP processing from the host CPU (e.g., processor(s) 102) to the DPU 130, saving processing cycles that allow the host CPU to perform other important processing tasks. As a result, the system 100 saves PCIe bandwidth, and GPUs 104, acting as the host CPU, no longer need to trigger resource-intensive memory copies for each HTTP file / TCP payload from the CPU memory to the GPU memory 214.

[0040] Fig. Figure 4A illustrates an example of a System 400 comprising one or more drivers and / or one or more runtime environments (illustrated by reference numeral 404) that include one or more libraries 406 providing one or more application programming interfaces (“API(s)”) 410, according to at least one embodiment. In at least one embodiment, the System 400 comprises the driver(s) 404 and / or the runtime environment(s) 404 that include the library(s) 406 providing the API(s) 410. In at least one embodiment, the API(s) 410 are sets of software instructions that, when executed, access one or more processors (e.g., the processor(s) in the system). Fig. 4B illustrates the processor(s) 422) to perform one or more arithmetic operations. In at least one embodiment, one or more of the API(s) 410 are distributed or otherwise provided as part of one or more of the library(ies) 406, one or more of the runtime environment(s) 404, one or more of the driver(s) 404, and / or one or more components of any other grouping of software and / or executable code, which are further described herein. In at least one embodiment, one or more of the API(s) 410 perform one or more arithmetic operations in response to a call by one or more software programs 402.

[0041] In at least one embodiment, one or more of the software program(s) 402 is / are a software module and / or comprises one or more software modules. In at least one embodiment, a software module is as further described non-exclusively in Fig. Figure 4B illustrates one or more modules 424 and describes them in relation to this. In at least one embodiment, one or more of the software program(s) 402 is a collection of software code, commands, instructions, and / or other text sequences that instruct a computing device (e.g., the DPU 130) to perform one or more arithmetic operations and / or to call one or more other sets of instructions, such as the API(s) 410 or the API function(s) 412, which are provided for execution by the computing device. In at least one embodiment, functionality provided by one or more of the API(s) 410 includes the API function(s) 412, such as those that can be used to accelerate one or more sections of the software program(s) 402 using one or more parallel processing units (PPUs), such as graphics processing units (e.g., the GPU(s) 104).

[0042] In at least one embodiment, one or more of the API(s) 410 are one or more hardware interfaces to one or more circuits configured to perform one or more computational operations. In at least one embodiment, one or more of the API(s) 410 described herein are implemented as one or more circuits configured to perform one or more techniques associated with the Fig. 1 to Fig. 3 are described. In at least one embodiment, one or more of the software program(s) 402 comprise instructions which, when executed, cause one or more hardware devices and / or circuits to perform one or more techniques which are further described in connection with the Fig. 1 to Fig. 3 are described. In at least one embodiment, the system 400 comprises one or more or all components of the system described in relation to Fig. 1 described processing system 100, and system 400 can perform one or more or all of the processes and / or operations that the systems and components of processing system 100 perform.

[0043] In at least one embodiment, the software program(s) 402, such as user-implemented software programs, utilize one or more of the API(s) 410 to perform various computational operations, such as memory allocation, matrix multiplication, arithmetic operations, and / or any computational operation performed by PPUs, such as GPUs, as further described herein. In at least one embodiment, the function(s) 412 comprise a set of callable functions provided by one or more of the API(s) 410, referred to herein as APIs, API functions, software functions, and / or functions, each of which performs one or more computational operations, such as computational operations related to parallel computing. In at least one embodiment, one or more of the API(s) 410 perform the initial reception of data packets (e.g.,the data packets 204) and / or perform other actions present (e.g. in connection with the . Fig. 1 to Fig. 3) performed the described operations.

[0044] In at least one embodiment, one or more of the software program(s) 402 interact with or otherwise communicate with one or more of the API(s) 410 to perform one or more computational operations using one or more processors (e.g., the one in Fig. 4B illustrated processor(s) 422), such as one or more PPUs, such as GPUs, to perform. In at least one embodiment, one or more computational operations using one or more PPUs comprise at least one or more groups of computational operations that are to be accelerated by execution at least partially by the one or more PPUs. In at least one embodiment, one or more of the software program(s) 402 interact with one or more of the API(s) 410 to process the initial reception of data packets (e.g., the data packets 204) and / or other present (e.g., in conjunction with the Fig. 1- Fig. 3) to perform the described operations.

[0045] In at least one embodiment, an interface consists of software instructions that, when executed, provide access to one or more of the function(s) 412 provided by one or more of the API(s) 410. In at least one embodiment, one or more of the software program(s) 402 use a local interface when a software developer compiles one or more of the software program(s) 402 in conjunction with one or more of the library(ies) 406 that include or otherwise provide access to one or more of the API(s) 410. In at least one embodiment, one or more of the software program(s) 402 are statically compiled in conjunction with one or more precompiled libraries 406 and / or uncompiled source code containing instructions for performing one or more of the API(s) 410.In at least one embodiment, one or more of the software program(s) 402 are dynamically compiled and the dynamically compiled software program(s) use a linker to link with one or more precompiled library(ies) 406, which comprise one or more of the API(s) 410.

[0046] In at least one embodiment, the software program(s) 402 use a remote interface when a software developer executes a software program that uses or otherwise communicates with at least one of the libraries 406 comprising one or more of the API(s) 410 over a network or other remote communication medium. In at least one embodiment, one or more of the library(ies) 406 comprising one or more of the API(s) 410 are executed by a remote computing service, such as a computing resource service provider.In at least one embodiment, it is provided that one or more of the library(s) 406, which comprise one or more specific APIs (of the API(s) 410), are executed by any other computing host which provides the specific API(s) for one or more of the software programs 402.

[0047] In at least one embodiment, it is provided that a processor (e.g., the one(s) in Fig. Figure 4B illustrates a processor (422) that executes one or more of the software programs 402, calls, uses, executes, and / or otherwise implements one or more of the API(s) 410 to allocate and otherwise manage a memory 414 intended to be used by the specific software program(s). In at least one embodiment, one or more of the software programs 402 utilize one or more of the API(s) 410 to allocate and otherwise manage the memory 414 intended to be used by one or more sections of the specific software program(s) intended to be accelerated using one or more PPUs, such as GPUs, or any other accelerator or processor further described herein.In at least one embodiment, one or more software programs 402 request one or more neural networks to perform signal processing using one or more of the functions 412 provided by one or more of the APIs 410. In at least one embodiment, the working memory of the DPU 130 implements the working memory 414.

[0048] In at least one embodiment, one or more of the API(s) 410 are an API that enables parallel computing. In at least one embodiment, one or more of the API(s) 410 are any other API that is further described herein. In at least one embodiment, one or more of the API(s) 410 are provided by one or more of the drivers 404 and / or one or more of the runtime environment(s) 404. In at least one embodiment, one or more of the API(s) 410 are provided by a CUDA user-mode driver. In at least one embodiment, one or more of the API(s) 410 are provided by a CUDA runtime environment.In at least one embodiment, one or more of the drivers 404 are data values ​​and software instructions which, when executed, perform and / or otherwise enable the operation of one or more of the functions 412 of one or more of the APIs 410 during the loading and execution of one or more sections of at least one of the software program(s) 402. In at least one embodiment, one or more of the runtime environment(s) 404 are data values ​​and / or software instructions which, when executed, perform or otherwise enable the operation of one or more of the functions 412 of one or more of the APIs 410 during the execution of at least one of the software program(s) 402.In at least one embodiment, one or more of the software program(s) 402 utilize one or more of the API(s) 410, which are implemented and / or otherwise provided by one or more of the drivers 404 and / or one or more of the runtime environment(s) 404, to perform combined arithmetic operations of the specific software program(s) during execution by one or more PPUs, such as GPUs.

[0049] In at least one embodiment, one or more of the software program(s) 402 utilize one or more of the API(s) 410 provided by one or more of the drivers 404 and / or one or more of the runtime environment(s) 404 to perform combined arithmetic operations of one or more PPUs, such as GPUs. In at least one embodiment, one or more of the API(s) 410 provide combined arithmetic operations as described above through one or more of the drivers 404 and / or one or more of the runtime environment(s) 404. In at least one embodiment, one or more of the software program(s) 402 utilize one or more of the API(s) 410 provided by one or more of the drivers 404 and / or one or more of the runtime environment(s) 404 to allocate or otherwise reserve one or more blocks of the memory 414 of one or more PPUs, such as GPUs.In at least one embodiment, one or more of the software program(s) 402 utilize one or more of the API(s) 410 provided by one or more of the drivers 404 and / or one or more of the runtime environment(s) 404 to allocate or otherwise reserve blocks of the main memory 414.

[0050] In at least one embodiment, to improve the usability and / or performance of one or more of the software programs 402, one or more sections of the software programs are accelerated by one or more PPUs (such as GPUs). In at least one embodiment, one or more of the function(s) 412 receive one or more input parameters specifying one or more inputs to one or more neural networks and / or other data to be used by the neural network(s), such as one or more hyperparameters of the neural network(s). In at least one embodiment, the input parameter(s) comprise the one or more inputs and / or the other data.In at least one embodiment, the input parameter(s) comprise one or more pointers to one or more memory locations where the input(s) and / or the other data is / are stored.

[0051] In at least one embodiment, the system 400 comprises at least one processor (e.g., processor(s) 422, which are located in Fig. 4B illustrated), comprising one or more circuits for executing one or more software programs that combine two or more of the API(s) 410 into a single API. In at least one embodiment, the system 400 comprises at least one processor (e.g., processor(s) 422, which are illustrated in Fig. 4B illustrated), which uses one or more of the API(s) 410 to process the initial reception of data packets (e.g., data packets 204) and / or to perform other operations described herein. In at least one embodiment, the system 400 comprises at least one processor (e.g., processor(s) 422, which are illustrated in Fig. 4B are illustrated), which uses one or more of the API(s) 410 to perform one or more operations relating to one or more of the Fig. 1 to Fig. 3 are illustrated and / or described, such as one or more processes that are in the Fig. 1 to Fig. 2 are illustrated, or sections thereof. In at least one embodiment, the system 400 comprises at least one processor (e.g., processor(s) 422, which are shown in Fig. 4B are illustrated), to perform one or more of the function(s) 412, such as those associated with the Fig. 1 to Fig. 3 are described. In at least one embodiment, it is provided that one or more of the API(s) 410 are executed by hardware that is used in conjunction with the Fig. 5 to Fig. 7 is described.

[0052] Fig. Figure 4B is a block diagram 420 illustrating exemplary processor(s) 422 and module(s) 424 according to at least one embodiment. With reference to Fig. 4B The processor(s) 422 can be implemented in at least one embodiment by the GPU(s) 104, which operate in conjunction with the DPU 130. In at least one embodiment, the processor(s) 422 can perform one or more processes, such as those described herein with respect to the DPU 130 and the GPU(s) 104, and / or can perform operations otherwise described herein. In at least one embodiment, the processor(s) 422 execute one or more processes, such as those described herein with respect to the Fig. 1 to Fig. 3 are described.

[0053] In at least one embodiment, the processor(s) 422 comprise one or more processors, such as those associated with the Fig. 5 to Fig. 7 are described. In at least one embodiment, the processor(s) 422 can each be any suitable processing unit and / or combination of processing units, such as one or more CPUs, GPUs, DPUs, GPGPUs, PPUs, and / or variations thereof. The processor(s) 422 comprise the module(s) 424, each of which can comprise a software module 426 for controlling the network adapter 126, a software module 428 for implementing the TCP / HTTP server, and a software module 430 for implementing the DOCA DMA plug-in. The module(s) 424 can be distributed across multiple processors that communicate via a bus, a network, by writing to shared memory, and / or any suitable communication process, such as those described herein.In at least one embodiment, the module(s) 424 may comprise processor-executable instructions that implement the process of initially receiving the data packets (e.g., the data packets 204), processing the packets, and transferring the processed packets to the GPU(s) 104.

[0054] As used in any implementation described herein, the term "module" refers to any combination of software logic, firmware logic, hardware logic, and / or circuitry configured to provide functionality described herein, unless otherwise clearly or explicitly stated in the context. Software may be embodied as a software package, code, and / or instruction set or instructions, and "hardware," as used in any implementation described herein, may include, for example, hardwired circuitry, programmable circuitry, state machine circuitry, fixed-function circuitry, execution unit circuitry, and / or firmware storing instructions to be executed by programmable circuitry, individually or in any combination.Modules can be collectively or individually embodied as circuits that form part of a larger system, such as an integrated circuit (IC), a system-on-a-chip (SoC), and so on. A module, in conjunction with any suitable processing unit and / or combination of processing units, such as one or more CPUs, GPUs, DPUs, GPGPUs, PPUs, and / or variations thereof, executes one or more processes.

[0055] In at least one embodiment, terms used in any implementation described herein, such as "module" and nominalized verbs (e.g., image manager, image analyzer, analytics functional unit, control device or controller and / or other terms), unless otherwise clearly indicated from the context or expressly stated otherwise, refer in each case to any combination of software logic, firmware logic, hardware logic and / or circuitry configured to provide functionality described herein.In at least one embodiment, software can be embodied as a software package, code and / or instruction set or instructions, and "hardware," as used in any implementation described herein, can include, for example, individually or in any combination, hardwired circuits, programmable circuits, state machine circuits, fixed-function circuits, execution unit circuits, and / or firmware that stores instructions that are executed by programmable circuits. In at least one embodiment, modules can be embodied collectively or individually as circuits that form part of a larger system, for example, an integrated circuit (IC), a system-on-a-chip (SoC), and so forth. LOGIC

[0056] Fig. Figure 5A illustrates logic 515, which, as described elsewhere, can be used in one or more devices to perform operations such as those discussed herein, according to at least one embodiment. In at least one embodiment, logic 515 is used to perform inference and / or training operations associated with one or more embodiments. In at least one embodiment, logic 515 is inference and / or training logic. Details regarding logic 515 are given below in conjunction with Fig. 5A and / or Fig. 5B provided. In at least one embodiment, logic refers to any combination of software logic, hardware logic and / or firmware logic that provides functionality or operations described herein, wherein the logic may be embodied collectively or individually as circuits that form part of a larger system, for example, an integrated circuit (IC), a system-on-a-chip (SoC), or one or more processors (e.g., CPU, GPU).

[0057] In at least one embodiment, the logic 515 may, in particular, comprise code and / or data storage 501 for storing forward and / or output weights and / or input / output data and / or other parameters for configuring neurons or layers of a neural network that are trained in aspects of one or more embodiments and / or used for inference. In at least one embodiment, the logic 515 may comprise or be coupled to code and / or data storage 501 for storing graph code or other software that controls the timing and / or sequence in which weights and / or other parameter information are loaded to configure logic, including integer and / or floating-point units (collectively referred to as arithmetic logic units (ALUs)).In at least one embodiment, code, such as graph code, based on the architecture of a neural network to which that code corresponds, loads weights or other parameter information into processor ALUs. In at least one embodiment, the code and / or data memory 501 stores weight parameters and / or input / output data of each layer of a neural network that is trained or used in conjunction with one or more embodiments during the forward propagation of input / output data, and / or weight parameters during training and / or inference using aspects of one or more embodiments. In at least one embodiment, any portion of the code and / or data memory 501 may be included in another on-chip or off-chip data memory, including the L1, L2, or L3 cache memory or system memory of a processor.

[0058] In at least one embodiment, any section of the code and / or data memory 501 can be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or code and / or data memory 501 can be a cache memory, a dynamic directly addressable memory (“DRAM”), a static directly addressable memory (“SRAM”), a non-volatile memory (e.g., flash memory), or another type of memory.In at least one embodiment, the choice of whether the code memory and / or code and / or data memory 501 is, for example, internal or external to a processor, or comprises DRAM, SRAM, Flash, or another type of memory, may depend on available on-chip memory versus off-chip memory, latency requirements of training and / or inference functions being performed, batch size of data used in inferring and / or training a neural network, or a combination of these factors.

[0059] In at least one embodiment, the logic 515 may, in particular, comprise a code and / or data memory 505 for storing backward and / or output weights and / or input / output data corresponding to neurons or layers of a neural network that is trained and / or used for inference in aspects of one or more embodiments. In at least one embodiment, the code and / or data memory 505 stores weight parameters and / or input / output data of each layer of a neural network that is trained or used in conjunction with one or more embodiments during backward propagation of input / output data, and / or weight parameters during training and / or inference using aspects of one or more embodiments.In at least one embodiment, the logic 515 may comprise or be coupled to code and / or data storage 505 to store graph code or other software which controls the timing and / or sequence in which weights and / or other parameter information are to be loaded to configure logic, including integer and / or floating-point units (collectively referred to as arithmetic logic units (ALUs)).

[0060] In at least one embodiment, code, such as graph code, causes weights or other parameter information to be loaded into processor ALUs based on a neural network architecture to which that code corresponds. In at least one embodiment, any portion of the code and / or data memory 505 may be included in another on-chip or off-chip data memory, including the L1, L2, or L3 cache memory or system memory of a processor. In at least one embodiment, any portion of the code and / or data memory 505 may be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or data memory 505 may be a cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other memory.In at least one embodiment, the choice of whether the code and / or data memory 505 is, for example, internal or external to a processor, or comprises DRAM, SRAM, Flash, or another type of memory, may depend on available on-chip memory versus off-chip memory, latency requirements of training and / or inference functions being performed, batch size of data used in inferring and / or training a neural network, or a combination of these factors.

[0061] In at least one embodiment, the code and / or data memory 501 and the code and / or data memory 505 can be separate memory structures. In at least one embodiment, the code and / or data memory 501 and the code and / or data memory 505 can be a combined memory structure. In at least one embodiment, the code and / or data memory 501 and the code and / or data memory 505 can be partially combined and partially separate. In at least one embodiment, any portion of the code and / or data memory 501 and the code and / or data memory 505 can be included in another on-chip or off-chip data memory, including the L1, L2, or L3 cache memory or system memory of a processor.

[0062] In at least one embodiment, the logic 515 may in particular comprise one or more arithmetic logic units (“ALU(s)”) 510, including integer and / or floating-point units, to perform logical and / or mathematical operations that are at least partially based on or specified by training and / or inference code (e.g., graph code), wherein a result thereof may generate activations (e.g., output values ​​of layers or neurons within a neural network) that are stored in an activation memory 520 and that are functions of input / output and / or weight parameter data stored in code and / or data memory 501 and / or code and / or data memory 505.In at least one embodiment, activations stored in the activation memory 520 are generated according to linear algebraic and / or matrix-based mathematics, performed by the ALU(s) 510 in response to the execution of instructions or other code, wherein weight values ​​stored in the code and / or data memory 505 and / or data memory 501 are used as operands together with other values, such as deviation values, gradient information, pulse values, or other parameters or hyperparameters, one, several, or all of which may be stored in the code and / or data memory 505 or the code and / or data memory 501 or in another memory on or off the chip.

[0063] In at least one embodiment, the ALU(s) 510 are included in one or more processors or other hardware logic devices or circuits, whereas in another embodiment, the ALU(s) 510 may be external to a processor or other hardware logic device or circuit that uses them (e.g., a coprocessor). In at least one embodiment, the ALU(s) 510 may be included in the execution units of a processor or otherwise in a bank of ALUs that can be accessed by the execution units of a processor either within the same processor or that are distributed among different processors of different types (e.g., central processing units, graphics processing units, fixed-function units, etc.).In at least one embodiment, the code and / or data memory 501, the code and / or data memory 505, and the activation memory 520 can share a processor or other hardware logic device or circuit, whereas in another embodiment, they can be located in different processors or other hardware logic devices or circuits, or a combination of the same and different processors or other hardware logic devices or circuits. In at least one embodiment, any portion of the activation memory 520 can be included in another on-chip or off-chip data memory, including the L1, L2, or L3 cache memory or system memory of a processor.Furthermore, inference and / or training code can be stored together with other code that a processor or other hardware logic or circuitry can access and that is retrieved and / or processed using the fetch, decode, schedule, execute, retract and / or other logical circuitry of a processor.

[0064] In at least one embodiment, the activation memory 520 can be a cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or another type of memory. In at least one embodiment, the activation memory 520 can be located wholly or partially inside or external to one or more processors or other logic circuits. In at least one embodiment, the choice of whether the activation memory 520 is, for example, internal or external to a processor, or whether it comprises DRAM, SRAM, flash memory, or another type of memory, can depend on available on-chip versus off-chip memory, the latency requirements of training and / or inference functions being performed, the batch size of data used in inferring and / or training a neural network, or a combination of these factors.

[0065] In at least one embodiment, the in Fig. 5A illustrated logic 515 can be used in conjunction with an application-specific integrated circuit (“ASIC”), such as a Google TensorFlow® processing unit, a Graphcore™ inference processing unit (IPU), or an Intel Corp. Nervana® processor (e.g., “Lake Crest”). In at least one embodiment, the logic illustrated in Fig. 5A illustrated logic 515 in conjunction with central processing unit (“CPU”) hardware, graphics processing unit (“GPU”) hardware, or other hardware, such as field-programmable gate arrays (“FPGAs”).

[0066] Fig. Figure 5B illustrates logic 515 according to at least one embodiment. In at least one embodiment, the logic 515 is inference and / or training logic. In at least one embodiment, the logic 515 may, in particular, comprise hardware logic in which computing resources are dedicated or otherwise used exclusively in conjunction with weight values ​​or other information corresponding to one or more layers of neurons within a neural network. In at least one embodiment, the logic 515 may, in particular, comprise hardware logic in which computing resources are dedicated or otherwise used exclusively in conjunction with weight values ​​or other information corresponding to one or more layers of neurons within a neural network. Fig. 5B illustrates how Logic 515 can be used in conjunction with an application-specific integrated circuit (ASIC), such as a Google TensorFlow® processing unit, a Graphcore™ inference processing unit (IPU), or an Intel Corp. Nervana® processor (e.g., “Lake Crest”). In at least one embodiment, the logic illustrated in Figure 5B can be used in conjunction with an application-specific integrated circuit (ASIC), such as a Google TensorFlow® processing unit, a Graphcore™ inference processing unit (IPU), or an Intel Corp. Nervana® processor (e.g., “Lake Crest”). Fig. Figure 5B illustrates how logic 515 can be used in conjunction with central processing unit (CPU) hardware, graphics processing unit (GPU) hardware, or other hardware, such as field-programmable gate arrays (FPGAs). In at least one embodiment, the logic 515 specifically comprises data memory 501 and code and / or data memory 505, which can be used to store code (e.g., graph code), weight values, and / or other information, including deviation values, gradient information, pulse values, and / or other parameter or hyperparameter information. In at least one embodiment, Fig. In the embodiment illustrated in Figure 5B, the code and / or data memory 501 and the code and / or data memory 505 are each associated with a dedicated computing resource, such as compute hardware 502 and compute hardware 506, respectively. In at least one embodiment, the compute hardware 502 and the compute hardware 506 each comprise one or more ALUs that perform mathematical functions, such as linear algebraic functions, only on information stored in the code and / or data memory 501 and the code and / or data memory 505, respectively, with one result of this being stored in the activation memory 520.

[0067] In at least one embodiment, the code and / or data storage 501 and 505 and the corresponding computing hardware 502 and 506, respectively, each correspond to different layers of a neural network, such that the resulting activation from a storage / computing pair 501 / 502 consisting of code and / or data storage 501 and computing hardware 502 is provided as an input to a next storage / computing pair 505 / 506 consisting of code and / or data storage 505 and computing hardware 506, in order to reflect a conceptual organization of a neural network. In at least one embodiment, each of the storage / computing pairs 501 / 502 and 505 / 506 can correspond to more than one layer of a neural network. In at least one embodiment, additional memory / computing pairs (not shown) in logic 515 can be included following or in parallel to the memory / computing pairs 501 / 502 and 505 / 506.

[0068] In relation to Fig. 5A The data store 501 and the code and / or data store 505 can be used to implement the drivers for the network adapter 126, the software for the TCP / HTTP server 206, and / or the DOCA DMA plug-in 210. Similarly, with regard to Fig. 5B the data store 501 and the code and / or data store 505 are used to implement the drivers for the network adapter 126, the software for the TCP / HTTP server 206 and / or the DOCA DMA plug-in 210. DATA CENTER

[0069] Fig. Figure 6 illustrates an exemplary data center 600 in which at least one embodiment can be used. In at least one embodiment, the data center 600 comprises a data center infrastructure layer 610, a framework layer 620, a software layer 630, and an application layer 640.

[0070] In at least one embodiment, as in Fig. As shown in Figure 6, the data center infrastructure layer 610 can comprise a resource orchestrator 612, clustered computing resources 614, and node computing resources (“node CRs”) 616(1)-616(N), where “N” is a positive integer (which may be a different integer “N” than used in other figures). In at least one embodiment, the node CRs 616(1)-616(N) can, in particular, comprise any number of central processing units (“CPUs”) or other processors (including accelerators, field-programmable gate arrays (FPGAs), graphics processing units (e.g., the GPU(s) 104 of the Fig. 1-2) etc.), memory devices 618(1)-618(N) (e.g., dynamic read-only memory, solid-state memory, or disk drives), network input / output devices (“NW I / O”), network switches, virtual machines (“VMs”), power modules, and cooling modules, etc. In at least one embodiment, one or more Node CRs from Node CRs 616(1)-616(N) can be a server comprising one or more of the computing resources mentioned above, such as the GPU(s) 104, the GPU memory 214, the network adapter 126, the DPU 130, and the like.

[0071] In at least one embodiment, the grouped computing resources 614 can comprise separate groupings of node CRs located in one or more racks (not shown) or many racks located in data centers at different geographical locations (also not shown). In at least one embodiment, separate groupings of node CRs within the grouped computing resources 614 can comprise grouped computing, networking, memory, or storage resources that can be configured or allocated to support one or more workloads. In at least one embodiment, multiple node CRs, including CPUs or processors (e.g., the GPU(s) 104), can be grouped in one or more racks to provide computing resources that support one or more workloads.In at least one embodiment, one or more racks can also include any number of power modules, cooling modules and network switches in any combination.

[0072] In at least one embodiment, the resource orchestrator 612 can configure or otherwise control one or more node CRs 616(1)-616(N) and / or grouped computing resources 614. In at least one embodiment, the resource orchestrator 612 can include a software design infrastructure (“SDI”) management entity for the data center 600. In at least one embodiment, the resource orchestrator 612 can include hardware, software, or a combination thereof.

[0073] In at least one embodiment, as in Fig. As shown in Figure 6, the framework layer 620 comprises a job scheduler 622, a configuration manager 624, a resource manager 626, and a distributed file system 628. In at least one embodiment, the framework layer 620 may include a framework to support software 632 of software layer 630 and / or one or more application(s) 642 of application layer 640. In at least one embodiment, the software 632 or application(s) 642 may each comprise web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, the framework layer 620 may, in particular, be a type of free and open-source software web application framework, such as Apache Spark™ (hereinafter “Spark”), which can use the distributed file system 628 for large-scale data processing (e.g., “Big Data”).In at least one embodiment, the job scheduler 622 can include a Spark driver to enable the scheduling of workloads supported by different layers of the data center 600. In at least one embodiment, the configuration manager 624 can be able to configure different layers, such as the software layer 630 and the framework layer 620, including Spark and the distributed file system 628, to support data processing at scale. In at least one embodiment, the resource manager 626 can be able to manage clustered or grouped computing resources that are mapped to or allocated to support the distributed file system 628 and the job scheduler 622. In at least one embodiment, clustered or grouped computing resources can include grouped computing resources 614 in the data center infrastructure layer 610.In at least one embodiment, the resource manager 626 can coordinate with the resource orchestrator 612 to manage these mapped or allocated computing resources.

[0074] In at least one embodiment, the software 632 contained in software layer 630 may comprise software used by at least sections of the node CRs 616(1)-616(N), the grouped computing resources 614, and / or the distributed file system 628 of framework layer 620. In at least one embodiment, one or more types of software may include, in particular, internet website search software, email virus scanning software, database software, and streaming video content software.

[0075] In at least one embodiment, the application(s) 642 included in the application layer 640 may comprise one or more types of applications used by at least sections of the node CRs 616(1)-616(N), the grouped computing resources 614, and / or the distributed file system 628 of the framework layer 620. In at least one embodiment, one or more types of applications may, in particular, comprise any number of a genomics application, a cognitive computing application, and a machine learning application, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), or other machine learning applications used in conjunction with one or more embodiments.

[0076] In at least one embodiment, any configuration manager 624, resource manager 626, and resource orchestrator 612 can implement any number and type of self-modifying actions based on any set and type of data acquired in any technically feasible way. In at least one embodiment, self-modifying actions can relieve a data center operator of data center 600 of potentially making poor configuration decisions and thereby potentially avoiding underutilized and / or poorly performing sections of a data center.

[0077] In at least one embodiment, the data center 600 may include tools, services, software, or other resources for training one or more machine learning models or for predicting or inferring information using one or more machine learning models according to one or more embodiments described herein. For example, in at least one embodiment, a machine learning model may be trained by calculating weight parameters according to a neural network architecture using software and computing resources described above in relation to the data center 600.In at least one embodiment, trained machine learning models corresponding to one or more neural networks can be used to infer or predict information using resources described above in relation to the Computing Center 600, using weight parameters calculated by one or more training techniques described herein.

[0078] In at least one embodiment, the data center can use 600 CPUs, application-specific integrated circuits (ASICs), GPUs (e.g., the GPU(s) 104), FPGAs, or other hardware to perform training and / or inference using the resources described above. Furthermore, one or more of the software and / or hardware resources described above can be provided as a service to allow users to perform training or inference on information, such as image recognition, speech recognition, or other artificial intelligence services.

[0079] Logic 515 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding Logic 515 are provided here in conjunction with Fig. 5A and / or Fig. 5B provided. In at least one embodiment, the logic 515 in the data center 600 can be used for inferencing or predicting operations that are at least partially based on weight parameters computed using neural network training operations, neural network functions and / or architectures, or neural network use cases, as described herein.

[0080] In relation to Fig. 6. Data Center 600 uses the shared data processing capabilities of a large number of processors to perform network processing tasks. The network requires communication between the various node CRs 616(1)-616(N). In at least one embodiment, the communication between the node CRs can use TCP communication to transfer data. In at least one example, the GPU(s) 104 can receive data to perform a requested network processing task. In such a case, the DPU 130 is used as above with respect to the Fig. 1 to Fig. 3 described is used to process the TCP data and store the data in the GPU memory (e.g., GPU memory 214 in Fig. 2) to place. As described above, one or more DPU(s) 130 can be used to process incoming TCP data from other node CRs (e.g., node CRs 616(1)-616(N)) for delivery to the GPU(s) 104, so that the GPU(s) can perform the requested network processing task. After completion of the network processing task, the GPU(s) 104 can use the DPU(s) 130 to process outgoing TCP data for transmission to other node CRs. The DPU 130 can be considered a section of the processing system 100 of the Fig. 1 to 2 can be implemented, for example, the DPU 130 and the GPUs 104 can be a section of another processor 102 in the data center 600. Communication between the DPU 130 and the GPU(s) 104 can use internal communication links within the data center 600. COMPUTER SYSTEMS

[0081] Fig. Figure 7 is a block diagram illustrating an exemplary computer system, which may be a system with interconnected devices and components, a system-on-a-chip (SOC), or a combination thereof, configured with a processor that may include execution units for executing an instruction, according to at least one embodiment. In at least one embodiment, a computer system 700 may, in particular, include a component, such as a processor 702, configured to use execution units comprising logic to execute algorithms on process data according to the present disclosure, as in an embodiment described herein.In at least one embodiment, the Computer System 700 may include processors such as PENTIUM® processor family processors, Xeon™, Itanium®, XScale™ and / or StrongARM™, Intel® Core™ or Intel® Nervana™ microprocessors available from Intel Corporation in Santa Clara, California, although other systems (including PCs with other microprocessors, engineering workstations, set-top boxes, and the like) may also be used. In at least one embodiment, the Computer System 700 may run a version of the WINDOWS operating system available from Microsoft Corporation in Redmond, Washington, although other operating systems (for example, UNIX and Linux), embedded software, and / or graphical user interfaces may also be used.

[0082] Embodiments can be used in other devices, such as handheld devices and embedded applications. Some examples of handheld devices include mobile phones, Internet Protocol devices, digital cameras, personal digital assistants (PDAs), and handheld PCs. In at least one embodiment, embedded applications can include a microcontroller, a digital signal processor (DSP), a system-on-a-chip, network computers (NetPCs), set-top boxes, network hubs, wide area network (WAN) switches, or any other system capable of executing one or more instructions according to at least one embodiment.

[0083] In at least one embodiment, the computer system 700 may, in particular, comprise a processor 702, which may, in particular, have one or more execution units 708 configured to perform machine learning for training models and / or inference according to the technical teachings described herein. In at least one embodiment, the computer system 700 is a single-processor desktop or server system, but in another embodiment, the computer system 700 may be a multiprocessor system. In at least one embodiment, the processor 702 may, in particular, be a microprocessor of a computer with a complex instruction set (“CISC”), a microprocessor of a computer with a reduced instruction set (“RISC”), a microprocessor with very long instruction words (“VLIW”), a processor implementing a combination of instruction sets, a GPU (e.g.,The processor 702 may include a GPU (104), a DPU (e.g., the DPU 130), or any other processing device, such as a digital signal processor. In at least one embodiment, the processor 702 may be coupled to a processor bus 710, which can transmit data signals between the processor 702 and other components in the computer system 700.

[0084] In at least one embodiment, the processor 702 may, in particular, comprise an internal level 1 cache memory (“cache”) 704 (“L1”). In at least one embodiment, the processor 702 may have a single internal cache or multiple levels of internal cache. In at least one embodiment, cache memory may be located outside the processor 702. Other embodiments may also include a combination of both internal and external caches, depending on a specific implementation and requirements. In at least one embodiment, a register set 706 may store different data types in different registers, such as, in particular, integer registers, floating-point registers, status registers, and instruction pointer registers.

[0085] In at least one embodiment, the processor 702 also includes an execution unit 708, which in particular comprises logic for performing integer and floating-point operations. In at least one embodiment, the processor 702 may also include a read-only memory (“ROM”) containing microcode (“ucode”) in which microcode for specific macro instructions is stored. In at least one embodiment, the execution unit 708 may include logic configured to handle a packed instruction set 709. In at least one embodiment, by incorporating the packed instruction set 709 into an instruction set of a general-purpose processor, together with associated circuitry for executing the instructions, operations used by many multimedia applications can be performed using packed data in the processor 702.In at least one embodiment, many multimedia applications can be accelerated and run more efficiently by using a full width of a processor's data bus to perform operations on packed data, which can eliminate the need to transfer smaller data units over that processor's data bus to perform one or more operations sequentially on one data element at a time.

[0086] In at least one embodiment, the execution unit 708 can also be used in microcontrollers, embedded processors, graphics devices, DSPs, and other types of logic circuits. In at least one embodiment, the computer system 700 can, in particular, comprise a main memory 720. In at least one embodiment, the main memory 720 can be a storage device with dynamic random-access memory (“DRAM”), static random-access memory (“SRAM”), or flash memory, or another main memory device. In at least one embodiment, the main memory 720 can store instruction(s) 719 and / or data 721, which are represented by data signals that can be executed by a processor 702.

[0087] In at least one embodiment, a system logic chip can be coupled to the processor bus 710 and the main memory 720. In at least one embodiment, a system logic chip can, in particular, comprise a memory controller hub (MCH) 716, and the processor 702 can communicate with the MCH 716 via the processor bus 710. In at least one embodiment, the MCH 716 can provide a high-bandwidth memory path 718 to the main memory 720 for storing applications and data and for storing graphics instructions, data, and textures. In at least one embodiment, the MCH 716 can route data signals between the processor 702, the main memory 720, and other components in the computer system 700 and form a bridge for data signals between the processor bus 710, the main memory 720, and a system I / O interface 722.In at least one embodiment, a system logic chip can provide a graphics port for coupling with a graphics control device. In at least one embodiment, the MCH 716 can be coupled to the main memory 720 via a high-bandwidth memory path 718, and a graphics / video card 712 can be coupled to the MCH 716 via an Accelerated Graphics Port (“AGP”) Interconnect 714.

[0088] In at least one embodiment, the computer system 700 can use the system I / O interface 722 as a proprietary hub interface bus to couple the MCH 716 to an I / O controller hub (ICH) 730. In at least one embodiment, the ICH 730 can provide direct connections to some I / O devices via a local I / O bus. In at least one embodiment, a local I / O bus can, in particular, comprise a high-speed I / O bus for connecting peripheral devices to the main memory 720, a chipset, and the processor 702.Examples may include, in particular, an audio control device 729, a firmware hub (“Flash BIOS”) 728, a wireless transceiver 726, a data storage device 724, a legacy I / O control device 723 having a user input and a keyboard interface 725, a serial expansion port 727, such as a Universal Serial Bus (“USB”) port, and a network control device 734. In at least one embodiment, the data storage device 724 may comprise a hard disk drive, a floppy disk drive, a CD-ROM device, a flash memory device, or another mass storage device.

[0089] Illustrated in at least one embodiment Fig. 7 a system comprising interconnected hardware devices or “chips”, whereas Fig. 7 can illustrate an exemplary SoC in other embodiments. In at least one embodiment, in Fig. 7 illustrated devices are interconnected using proprietary interconnects, standardized interconnects (e.g., PCIe), or a combination thereof. In at least one embodiment, one or more components of the Computer System 700 are interconnected using Compute Express Link (CXL) interconnects.

[0090] Logic 515 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding Logic 515 are provided here in conjunction with Fig. 5A and / or Fig. 5B provided. In at least one embodiment, the logic 515 in the computer system 700 can be used for inferencing or predicting operations that are at least partially based on weight parameters computed using neural network training operations, neural network functions and / or architectures, or neural network use cases, as described herein.

[0091] In relation to Fig. The 702 processor can be used as the DPU 130 in the 7 Fig. 1 and Fig. 2 be implemented. The network control device 734 is used to control the network adapter 126 of the Fig. 1 and Fig. 2 to implement. Other elements of the Fig. 1 and Fig. 2, such as the TCP / HTTP server 206 and the DOCA DMA Plug-In 210, can be implemented as instructions 719 and data 721.

[0092] At least one embodiment of the disclosure can be described with regard to the following clauses: 1. Data processing unit (DPU), comprising one or more circuits configured to obtain information transmitted in multiple Transmission Control Protocol (TCP) data packets and to store data, based at least in part on the information, in a graphics processing unit (GPU) memory. 2. DPU according to clause 1, wherein the one or more circuits are configured to reorder the TCP data packets in order to assemble the information into a data file, and the data is intended to comprise the data file. 3. DPU according to clause 1 or 2, wherein the one or more circuits are configured to use a Direct Memory Access (DMA) transfer protocol to store the data in the GPU memory. 4. DPU according to any one of clauses 1 to 3, wherein the one or more circuits are configured to indicate to at least one GPU that the data has been stored in the GPU memory. 5. DPU according to any one of clauses 1 to 4, wherein the one or more circuits are configured to receive a signal from at least one GPU indicating that the at least one GPU has successfully accessed the data stored in the GPU memory. 6. DPU according to any one of clauses 1 to 5, wherein the one or more circuits are configured to receive a signal from at least one GPU indicating that the at least one GPU is ready to receive additional data. 7. System comprising main memory and at least one parallel processing unit (PPU) connected to the main memory and comprising one or more circuits configured to access data stored in the main memory by a data processing unit (DPU) obtained from a message transmitted in accordance with the Transmission Control Protocol (TCP); and wherein the one or more circuits are configured to transmit an acknowledgment to the DPU indicating that the data has been accessed. 8. System according to clause 7, wherein one or more circuits are configured to detect that the data has been stored in the main memory by the DPU before accessing the data. 9. Data center comprising multiple graphics processing units (GPUs) and a data processing unit (DPU), comprising: a network adapter configured to receive multiple data packets, at least one server configured to receive the multiple data packets and obtain processed data by processing the multiple data packets, and a transfer functional unit configured to transfer the processed data to a memory of at least one of the multiple GPUs. 10. Data center according to clause 9, wherein the multiple data packets comprise multiple Transmission Control Protocol (TCP) data packets. 11. Data center according to one of clauses 9 to 10, wherein at least one server is set up to assemble a data file by rearranging the TCP data packets, and it is intended that the processed data will comprise the data file. 12. Data center according to any of clauses 9 to 11, wherein the transfer machine is configured to use a direct memory access (DMA) transfer protocol to store the processed data in the memory of the at least one GPU. 13. Data center according to any of clauses 9 to 12, wherein the DPU is configured to indicate to the at least one GPU that the processed data has been stored in the memory of the at least one GPU. 14. Data center according to any of clauses 9 to 13, wherein the DPU is configured to receive a signal from the at least one GPU indicating that the at least one GPU is ready to receive additional data. 15. Data center according to any one of clauses 9 to 14, further comprising at least one computer system comprising the multiple GPUs, the DPU and at least one central processing unit (CPU), wherein the at least one server is configured to obtain the processed data without involving the CPU and the transfer functional unit is configured to transfer the processed data to main memory without involving the CPU. 16. Method comprising obtaining, by means of at least one data processing unit (DPU), information transmitted in multiple Transmission Control Protocol (TCP) data packets, and storage, by means of which at least one DPU obtains data, based at least partially on the information, in a working memory associated with a graphics processing unit (GPU). 17. Method according to clause 16, wherein the at least one DPU is configured to reorder the TCP data packets in order to compile the information into a data file, and the data is intended to comprise the data file. 18. A method according to any of clauses 16 to 17, wherein storing the data includes using a Direct Memory Access (DMA) transfer protocol to store the data in the main memory associated with the GPU. 19. A method according to any of clauses 16 to 18, wherein the at least one DPU is configured to indicate to the GPU that the data has been stored in the main memory associated with the GPU. 20. A method according to any of clauses 16 to 19, wherein the at least one DPU is configured to receive a signal from the GPU indicating that the GPU is ready to receive additional data.

[0093] In at least one embodiment, a single semiconductor platform can refer to a standalone, unified integrated circuit or a standalone, unified semiconductor-based integrated chip. In at least one embodiment, multi-chip modules with enhanced connectivity can be used, simulating on-chip operation and achieving significant improvements over the use of a conventional implementation with a central processing unit (CPU) and bus. In at least one embodiment, different modules can also be arranged separately or in various combinations of semiconductor platforms, depending on the user's requirements.

[0094] In at least one embodiment, computer programs are stored in the form of machine-readable executable code or computer control logic algorithms in main memory and / or secondary memory, such as those described herein. When computer programs are executed by one or more processors, these enable at least one system described herein to perform various functions according to at least one embodiment. In at least one embodiment, main memory, mass storage, and / or any other storage medium are possible examples of computer-readable media.In at least one embodiment, secondary storage can refer to any suitable mass storage device or storage system, such as a hard disk drive and / or a removable storage drive, which is a floppy disk drive, a magnetic tape drive, a compact disc drive, a digital versatile disc drive (“DVD”), a recording device, a universal serial bus flash memory (“USB”), etc. In at least one embodiment, an architecture and / or functionality of various figures above is used in the context of a CPU, such as the one described herein, a parallel processing system, such as the one described herein, an integrated circuit capable of at least some of the capabilities of both the CPU and the parallel processing system, or a chipset (e.g.,a group of integrated circuits designed to operate and be sold as a single unit to perform associated functions, etc.) and / or any suitable combination of integrated circuit(s).

[0095] In at least one embodiment, the architecture and / or functionality of various preceding figures are implemented in the context of a general-purpose computer system, a printed circuit board system, a gaming console system dedicated to entertainment purposes, an application-specific system, and more. In at least one embodiment, a computer system described herein may take the form of a desktop computer, a laptop computer, a tablet computer, servers, supercomputers, a smartphone (e.g., a wireless handheld device), a personal digital assistant (PDA), a digital camera, a vehicle, a head-mounted display, a handheld electronic device, a mobile phone device, a television, a workstation, gaming consoles, an embedded system, and / or any other type of logic.In at least one embodiment, a computer system comprises or refers to any devices illustrated in any of the drawings and / or described herein.

[0096] In at least one embodiment, a parallel processing system comprises, in particular, several parallel processing units (PPUs) and associated main memory. In at least one embodiment, PPUs are connected to a host processor or other peripheral devices via an interconnect and a switch or multiplexer. In at least one embodiment, a parallel processing system distributes computational tasks across the PPUs, which may be parallelizable—for example, as part of distributing computational tasks across multiple graphics processing unit (GPU) thread blocks. In at least one embodiment, main memory is shared and accessed across some or all of the PPUs (e.g.,for read and / or write access), although such shared memory may suffer performance losses compared to using local memory and registers located in a PPU. In at least one embodiment, the operation of the PPUs is synchronized by using an instruction such as _syncthreads(), whereby all threads in a block (which may be running across multiple PPUs) reach a specific point in code execution before proceeding.

[0097] In at least one embodiment, one or more of the techniques described herein utilize a oneAPI programming model. In at least one embodiment, a oneAPI programming model refers to a programming model for interacting with different compute accelerator architectures. In at least one embodiment, oneAPI refers to an application programming interface (API) configured to interact with different compute accelerator architectures. In at least one embodiment, a oneAPI programming model utilizes a DPC++ programming language. In at least one embodiment, a DPC++ programming language refers to a high-level language for productivity in programming parallel data processing. In at least one embodiment, a DPC++ programming language is based at least partially on the C and / or C++ programming languages.In at least one embodiment, a oneAPI programming model is a programming model such as the one developed by Intel Corporation in Santa Clara, CA.

[0098] In at least one embodiment, oneAPI and / or the oneAPI programming model is used to interact with various accelerator architectures, GPU architectures, processor architectures, and / or variations thereof. In at least one embodiment, oneAPI comprises a set of libraries that implement various functionalities. In at least one embodiment, oneAPI comprises at least one oneAPI DPC++ library, oneAPI math kernel library, oneAPI data analysis library, oneAPI deep neural network library, oneAPI collection communication library, oneAPI threading building blocks library, oneAPI video processing library, and / or variations thereof.

[0099] In at least one embodiment, a oneAPI-DPC++ library, also referred to as oneDPL, is a library that implements algorithms and functions to accelerate DPC++ kernel programming. In at least one embodiment, oneDPL implements one or more Standard Template Library (STL) functions. In at least one embodiment, oneDPL implements one or more parallel STL functions. In at least one embodiment, oneDPL provides a set of library classes and functions, such as parallel algorithms, iterators, function object classes, range-based APIs, and / or variations thereof. In at least one embodiment, oneDPL implements one or more classes and / or functions from a C++ standard library. In at least one embodiment, oneDPL implements one or more random number generator functions.

[0100] In at least one embodiment, a oneAPI math kernel library, also referred to as oneMKL, is a library that implements various optimized and parallelized routines for different mathematical functions and / or operations. In at least one embodiment, oneMKL implements one or more Basic Linear Algebra Subprograms (BLAS) and / or Linear Algebra Package (LAPACK) routines for dense linear algebra. In at least one embodiment, oneMKL implements one or more sparse BLAS routines for linear algebra. In at least one embodiment, oneMKL implements one or more random number generators (RNGs). In at least one embodiment, oneMKL implements one or more vector mathematics routines (VMs) for mathematical operations on vectors.In at least one embodiment, oneMKL implements one or more Fast Fourier Transform functions (FFT function).

[0101] In at least one embodiment, a oneAPI data analysis library, also referred to as oneDAL, is a library that implements various data analysis applications and distributed computations. In at least one embodiment, oneDAL implements various algorithms for preprocessing, transformation, analysis, modeling, validation, and decision-making for data analysis, in batch, online, and distributed computing modes. In at least one embodiment, oneDAL implements various C++ and / or Java APIs and various connectors to one or more data sources. In at least one embodiment, oneDAL implements DPC++ API extensions to a conventional C++ interface and enables GPU utilization for various algorithms.

[0102] In at least one embodiment, a oneAPI deep neural network library, also referred to as oneDNN, is a library that implements various deep learning functions. In at least one embodiment, oneDNN implements various neural network, machine learning, and deep learning functions, algorithms, and / or variations thereof.

[0103] In at least one embodiment, a oneAPI collective communication library, also referred to as oneCCL, is a library that implements various applications for deep learning and machine learning workloads. In at least one embodiment, oneCCL is built on top of lower-level communication middleware, such as Message Passing Interface (MPI) and libfabrics. In at least one embodiment, oneCCL enables a set of deep learning-specific optimizations, such as prioritization, persistent operations, out-of-order execution, and / or variations thereof. In at least one embodiment, oneCCL implements various CPU and GPU functions.

[0104] In at least one embodiment, a oneAPI Threading Building Blocks library, also known as oneTBB, is a library that implements various parallelized processes for different applications. In at least one embodiment, oneTBB is used for task-based, co-parallel programming on a host. In at least one embodiment, oneTBB implements generic parallel algorithms. In at least one embodiment, oneTBB implements concurrent containers. In at least one embodiment, oneTBB implements a scalable memory allocator. In at least one embodiment, oneTBB implements a work-stealing task scheduler. In at least one embodiment, oneTBB implements low-level synchronization primitives. In at least one embodiment, oneTBB is compiler-independent and usable on various processors, such as GPUs, PPUs, CPUs, and / or variations thereof.

[0105] In at least one embodiment, a oneAPI video processing library, also referred to as oneVPL, is a library used to accelerate video processing in one or more applications. In at least one embodiment, oneVPL implements various video decoding, video encoding, and video processing functions. In at least one embodiment, oneVPL implements various functions for media pipelines on CPUs, GPUs, and other accelerators. In at least one embodiment, oneVPL implements device discovery and selection for media-centric workloads and video analytics workloads. In at least one embodiment, oneVPL implements API primitives for zero-copy buffer sharing.

[0106] In at least one embodiment, a oneAPI programming model utilizes a DPC++ programming language. In at least one embodiment, a DPC++ programming language is a programming language that includes, in particular, functionally similar versions of CUDA mechanisms for defining device code and distinguishing between device code and host code. In at least one embodiment, a DPC++ programming language may include a subset of functionalities of a CUDA programming language. In at least one embodiment, one or more operations of the CUDA programming model are performed using a oneAPI programming model with a DPC++ programming language.

[0107] In at least one embodiment, each application programming interface (API) described herein is compiled by a compiler, interpreter, or other software tool into one or more instructions, operations, or other signals. In at least one embodiment, the compilation includes generating one or more machine-executable instructions, operations, or other signals from the source code. In at least one embodiment, an API compiled into one or more instructions, operations, or other signals, when executed, causes one or more processors, such as graphics processors, graphics cores, parallel processors, a CPU, or any other logic circuit further described herein, to perform one or more arithmetic operations.

[0108] It should be noted that, although the exemplary embodiments described herein may refer to a CUDA programming model, the techniques described herein can be used with any suitable programming model, such as HIP, oneAPI and / or variations thereof.

[0109] Other variations are inherent in the concept of the present disclosure. Disclosed techniques can be modified or alternatively constructed in various ways; however, only certain illustrated embodiments are shown in the drawings and described in detail above. It is understood, however, that the disclosure is not intended to be limited to a specific form or the disclosed forms, but rather, on the contrary, it is intended to cover all modifications, alternative constructions, and equivalents that fall within the concept and scope of the disclosure as defined in the appended claims.

[0110] The use of the terms "a," "an," and "the" and similar references in connection with the description of disclosed embodiments (particularly in connection with the appended claims) is to be interpreted as covering both the singular and the plural, unless otherwise specified or the context clearly contradicts this, and is not to be understood as a definition of terms. The terms "comprise," "have," "exhibit," "contain," "including," and "with" are to be interpreted as non-restrictive, open terms (meaning "including, but not limited to"), unless otherwise specified. "Connected," when not modified and referring to physical connections, is to be interpreted as meaning that the connected elements are partially or completely encompassed, attached to, or joined together, even if something is in between.The mention of ranges of values ​​is intended here merely as an efficient method of referring individually to each separate value falling within the range, unless otherwise specified, and each separate value is deemed to be included in the description as if it had been explicitly mentioned. In at least one embodiment, the use of the term "set" (e.g., "a set of elements") or "subset," unless otherwise specified or the context contradicts, is to be interpreted as a non-empty collection comprising one or more elements. Furthermore, unless otherwise specified or the context contradicts, the term "subset" of a corresponding set does not necessarily denote a proper subset of a corresponding set; rather, the subset and the corresponding set may be identical.

[0111] Language with conjunctions, such as phrases of the form "at least one of A, B, and C" or "at least one of A, B, and C," is, unless explicitly stated otherwise or the context clearly contradicts it, to be understood in the context that it is generally intended to show that an element, a concept, etc., can be either A, B, C, or any non-empty subset of a set containing A, B, and C. For example, in an illustrative example of a set with three elements, sentences with conjunctions "at least one of A, B, and C" and "at least one of A, B, and C" refer to each of the following sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Thus, such a linguistic construction with conjunctions is generally not intended to imply that certain embodiments require, for instance, the presence of at least one of A, at least one of B, and at least one of C.Additionally, unless otherwise stated or the context contradicts, the terms "several" or "a plurality" indicate a plural state (e.g., "a plurality of elements" indicates multiple elements). In at least one embodiment, the number of elements in a plurality (the number of multiple elements) is at least two, but may be greater if either explicitly stated or indicated by context. Furthermore, unless otherwise stated or the context clearly indicates, the phrase "based on" means "based at least in part on" and not "based exclusively on".

[0112] The operations of the processes described herein may be performed in any suitable order, unless otherwise specified herein or the context clearly contradicts this. In at least one embodiment, a process, such as one of the processes described herein (or variations and / or combinations thereof), is carried out under the control of one or more computer systems configured with executable instructions and is implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors by hardware or combinations thereof. In at least one embodiment, code is stored on a computer-readable storage medium, for example, in the form of a computer program comprising several instructions that are executable by one or more processors.In at least one embodiment, a computer-readable storage medium is a non-volatile computer-readable storage medium that excludes volatile signals (e.g., a propagating transient electrical or electromagnetic transmission) but includes non-volatile data storage circuitry (e.g., buffers, caches, and queues) within transceivers of volatile signals. In at least one embodiment, code (e.g., executable code or source code) is stored on a set of one or more non-volatile computer-readable storage media on which executable instructions are stored (or on other storage for executable instructions) which, when executed by one or more processors of a computer system (i.e., as a result of execution), cause the computer system to perform the operations described herein.In at least one embodiment, the set of non-volatile, computer-readable storage media comprises multiple non-volatile, computer-readable storage media, and one or more individual non-volatile storage media from among multiple non-volatile, computer-readable storage media lack all the code, whereas multiple non-volatile, computer-readable storage media collectively store all the code. In at least one embodiment, executable instructions are executed such that different instructions are executed by different processors—for example, a non-volatile, computer-readable storage medium stores instructions, and a central processing unit (“CPU”) executes some of the instructions, while a graphics processing unit (“GPU”) executes other instructions.In at least one embodiment, different components of a computer system have separate processors, and different processors execute different subsets of instructions.

[0113] In at least one embodiment, an arithmetic logic unit is a set of combinational logic circuits that take one or more inputs to produce a result. In at least one embodiment, an arithmetic logic unit is used by a processor to implement a mathematical operation, such as addition, subtraction, or multiplication. In at least one embodiment, an arithmetic logic unit is used to implement logical operations, such as logical AND / OR or XOR. In at least one embodiment, an arithmetic logic unit is stateless and consists of physical switching components, such as semiconductor transistors, arranged to form logic gates. In at least one embodiment, an arithmetic logic unit can operate internally as a stateful logic circuit with an associated clock.In at least one embodiment, an arithmetic logic unit can be constructed as an asynchronous logic circuit with an internal state that is not held in an associated register set. In at least one embodiment, an arithmetic logic unit is used by a processor to combine operands stored in one or more registers of the processor and to generate an output that can be stored by the processor in another register or memory location.

[0114] In at least one embodiment, as a result of processing an instruction called by the processor, the processor provides one or more inputs or operands to an arithmetic logic unit. This causes the arithmetic logic unit to generate a result that is based at least partially on an instruction code provided to the inputs of the arithmetic logic unit. In at least one embodiment, the instruction codes provided by the processor to the arithmetic logic unit are based at least partially on the instruction being executed by the processor. In at least one embodiment, combinational logic within the arithmetic logic unit processes the inputs and generates an output that is placed on a bus within the processor.In at least one embodiment, the processor selects a destination register, a memory location, an output device, or an output memory location on the output bus, such that the clocking of the processor causes the results generated by the arithmetic logic unit to be sent to the desired location.

[0115] Within the scope of this application, the term "arithmetic logic unit" or "ALU" (English: "Arithmetic Logic Unit") is used to refer to any computational logic circuit that processes operands to produce a result. For example, in this document, the term ALU may refer to a floating-point unit, a DSP, a tensor core, a shader core, a coprocessor, or a CPU.

[0116] In at least one embodiment, one or more components of systems and / or processors disclosed above can communicate with one or more CPUs, ASICs, GPUs, FPGAs or other hardware, other circuits or integrated circuit components, which may include, for example: an upscaler or upsampler for upscaling an image, an image mixer for blending, mixing or adding images together, a sampler for sampling an image (e.g. as part of a DSP), a neural network circuit configured to communicate with an upscaler for upscaling an image (e.g.from a low-resolution image to a high-resolution image), or other hardware for modifying or generating an image, frame, or video to adjust its resolution, size, or pixels; one or more components of the system and / or processors disclosed above may use the components described herein to perform procedures, operations, or instructions that generate or modify an image.

[0117] Accordingly, in at least one embodiment, computer systems are configured to implement one or more services that individually or jointly perform operations of the processes described herein, and such computer systems are configured with applicable hardware and / or software that enables the execution of the operations. Furthermore, a computer system implementing at least one embodiment of the present disclosure can be a single device, or, in another embodiment, it can be a distributed computer system comprising several devices that operate differently, such that the distributed computer system performs the operations described herein and a single device does not perform all operations.

[0118] The use of any and all examples or exemplary language provided herein (e.g., "such as") is intended solely to better illuminate embodiments of the disclosure and does not constitute a limitation of the scope of the disclosure unless claimed otherwise. No wording in the description should be interpreted as indicating that an unclaimed element is essential for the practical implementation of the disclosure.

[0119] All references, in particular publications, patent applications and patents, cited herein are hereby deemed to be included by reference to the same extent as if each reference had been individually and specifically stated to be included by reference and were reproduced here in its entirety.

[0120] In the description and claims, the terms "coupled" and "connected," along with their derivatives, may be used. It is understood that these terms are not necessarily intended as synonyms. Rather, in certain examples, "connected" or "coupled" may be used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. "Coupled" may also mean that two or more elements are not in direct contact with each other but nevertheless work together or interact with each other.

[0121] Unless expressly stated otherwise, terms such as "processing", "calculating", "calculating", "determining" or the like are understood to refer to actions and / or processes of a computer or computing system or similar electronic computing device that manipulate and / or convert data represented as physical or electronic quantities within the registers and / or memory of the computing system into other data represented in a similar manner as physical quantities within the memory, registers or other such information storage, transmission or display devices of the computer system.

[0122] Similarly, the term "processor" can refer to any device or section of a device that processes electronic data from registers and / or memory and converts that electronic data into other electronic data that can be stored in registers and / or memory. As non-restrictive examples, "processor" can be a CPU or a GPU. A "computing platform" can include one or more processors. As used here, "software" processes can include, for example, software and / or hardware entities that perform work over time, such as tasks, threads, and intelligent agents. Furthermore, a given process can also refer to multiple processes for executing instructions sequentially or in parallel, continuously or intermittently.In at least one embodiment, the terms “system” and “method” are used interchangeably in the present case, insofar as the system can embody one or more methods and insofar as methods can be considered as a system.

[0123] This document may refer to the receiving, acquiring, capturing, receiving, or inputting of analog or digital data into a subsystem, computer system, or computer-implemented machine. In at least one embodiment, the process of receiving, acquiring, capturing, receiving, or inputting analog and digital data can be achieved in various ways, such as receiving data as a parameter of a function call or a call to an application programming interface. In at least one embodiment, processes of receiving, acquiring, capturing, receiving, or inputting analog or digital data can be achieved by transmitting data over a serial or parallel interface.In at least one embodiment, processes of obtaining, acquiring, capturing, receiving, or inputting analog or digital data can be achieved by transmitting data over a computer network from a providing entity to a receiving entity. In at least one embodiment, reference can also be made to providing, outputting, transmitting, sending, or presenting analog or digital data. In various examples, processes of providing, outputting, transmitting, sending, or presenting analog or digital data can be achieved by transmitting data as an input or output parameter of a function call, a parameter of an application programming interface, or as an interprocess communication mechanism.

[0124] Although the descriptions presented here are exemplary implementations of the described techniques, other architectures may be used to implement the described functionality and are intended to be within the scope of this disclosure. While specific distributions of responsibilities may be defined above for the purpose of description, various functions and responsibilities may also be distributed and subdivided in different ways depending on the circumstances.

[0125] Furthermore, although the subject matter was described in a language specific to structural features and / or methodological actions, it is understood that the subject matter claimed in the attached claims is not necessarily limited to specific features or described actions. Rather, specific features and actions are disclosed as exemplary forms of implementing the claims.

[0126] It is understood that the aspects and embodiments described above are purely exemplary and that details may be modified within the scope of the claims.

[0127] The individual devices, methods and features disclosed in the description and (where appropriate) the claims and drawings may be provided independently of one another or in any suitable combination.

[0128] Reference numerals appearing in the claims are for illustrative purposes only and do not limit the scope of the claims. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 63 / 688,807

[0001]

Claims

[1] Data processing unit (DPU), comprising: one or more circuits that are set up for this purpose: to obtain information transmitted in multiple Transmission Control Protocol (TCP) data packets; and To store data, based at least in part on the information, in a graphics processing unit (GPU) memory. [2] DPU according to claim 1, wherein the one or more circuits are configured to rearrange the TCP data packets in order to assemble the information into a data file, and it is provided that the data comprise the data file. [3] DPU according to claim 1 or 2, wherein the one or more circuits are configured to use a Direct Memory Access (DMA) transfer protocol to store the data in the GPU memory. [4] DPU according to claim 1, 2 or 3, wherein the one or more circuits are configured to indicate to at least one GPU that the data has been stored in the GPU memory. [5] DPU according to one of the preceding claims, wherein the one or more circuits are configured to receive a signal from at least one GPU indicating that the at least one GPU has successfully accessed the data stored in the GPU memory. [6] DPU according to any of the preceding claims, wherein the one or more circuits are configured to receive a signal from at least one GPU indicating that the at least one GPU is ready to receive additional data. [7] System, comprehensive: RAM; and at least one parallel processing unit (PPU) connected to main memory and comprising one or more circuits configured to: to access data stored in main memory by a data processing unit (DPU) obtained from a message transmitted according to the Transmission Control Protocol (TCP); and to transmit a confirmation to the DPU indicating that the data has been accessed. [8] System according to claim 7, wherein one or more circuits are provided for this purpose: to detect that the data has been stored in memory by the DPU before accessing the data. [9] Data center, comprehensive: multiple graphics processing units (GPUs); and a data processing unit (DPU) that includes: a network adapter that is configured to receive multiple data packets; at least one server configured to receive the multiple data packets and to obtain processed data by processing the multiple data packets; and a transfer functional unit that is set up to transfer the processed data to the main memory of at least one of the multiple GPUs. [10] Data center according to claim 9, wherein the multiple data packets comprise multiple Transmission Control Protocol (TCP) data packets. [11] Data center according to claim 10, wherein the at least one server is configured to assemble a data file by rearranging the TCP data packets, and it is provided that the processed data comprises the data file. [12] Data center according to claim 9, 10 or 11, wherein the transfer machine is configured to use a direct memory access (DMA) transfer protocol to store the processed data in the main memory of the at least one GPU. [13] Data center according to any one of claims 9 to 12, wherein the DPU is configured to indicate to the at least one GPU that the processed data has been stored in the main memory of the at least one GPU. [14] Data center according to any one of claims 9 to 13, wherein the DPU is configured to receive a signal from the at least one GPU indicating that the at least one GPU is ready to receive additional data. [15] Data center according to any one of claims 9 to 14, further comprising: at least one computer system comprising the multiple GPUs, the DPU and at least one central processing unit (CPU), wherein the at least one server is configured to obtain the processed data without involving the CPU, and the transfer functional unit is configured to transfer the processed data to main memory without involving the CPU. [16] Procedures, including: Erlangen, through at least one Data Processing Unit (DPU), of information transmitted in multiple Transmission Control Protocol (TCP) data packets; and Storage by which at least one DPU stores data, based at least partially on the information, in a working memory associated with a graphics processing unit (GPU). [17] Method according to claim 16, wherein the at least one DPU is configured to reorder the TCP data packets in order to compile the information into a data file, and the data is provided to comprise the data file. [18] Method according to claim 16 or 17, wherein storing the data comprises using a Direct Memory Access (DMA) transfer protocol to store the data in the main memory associated with the GPU. [19] Method according to claim 16, 17 or 18, wherein the at least one DPU is configured to indicate to the GPU that the data has been stored in the main memory associated with the GPU. [20] Method according to any one of claims 16 to 19, wherein the at least one DPU is configured to receive a signal from the GPU indicating that the GPU is ready to receive additional data.

Images