Parallelization of network communications on graphics processing unit (GPU)

Abstract

The invention relates to parallelization of network communications on a graphics processing unit (GPU). Embodiments are directed to parallel processing of network communications on a device supporting high parallelism, such as a graphics processing unit (GPU). In general, embodiments are directed to inline packet processing pipelines to receive packets in GPU memory without having to temporarily store copies through central processing unit (CPU) memory, to process the received packets in parallel with one or more cores of the GPU, and then run reasoning, evaluation, or to send compute results over a network. In this manner, the highly parallel characteristics of the GPU may be utilized to process network communications without involving other elements of the system (e.g., CPU), as other elements may be consumed soon by processing the network communications, thereby impairing other processes.

Description

Technical Field

[0001] The present disclosure relates generally to processing of network communications and, more particularly, to parallel processing of network communications on devices that support a high degree of parallelization, such as graphics processing units (GPUs). Background Art

[0002] Real-time graphics processing unit (GPU) processing of network traffic packets is a technique suitable for applications involving signal processing, network security, information collection, and input reconstruction. These applications employ a central processing unit (CPU)-centric approach, where the CPU in the critical path coordinates with the network interface controller (NIC) to receive packets from GPU memory and notify the packet processing cores waiting on the GPU for a new set of packets. In low-power platforms, the CPU can easily become a bottleneck, overshadowing the value of the GPU. Therefore, there is a need in the art for improved methods and systems for processing network communications. Summary of the Invention

[0003] Embodiments of the present disclosure are directed to parallel processing of network communications on devices that support high levels of parallelism, such as graphics processing units (GPUs). Generally speaking, embodiments of the present disclosure are directed to an inline packet processing pipeline for receiving packets in GPU memory without requiring a temporary copy in central processing unit (CPU) memory, processing the received packets in parallel with one or more cores of the GPU, and then running inference, evaluation, or sending the results of the computations over a network. In this way, the highly parallel nature of the GPU can be exploited to process network communications without involving other elements of the system (e.g., a CPU), which could quickly be consumed by processing the network communications, thereby compromising other processes.

[0004] According to one embodiment, a central processing unit (CPU) may include control circuitry for controlling CPU operations. The control circuitry may cause the CPU to receive a plurality of data packets from a network interface card (NIC) via a communication network, such as an Ethernet network. The control circuitry may also cause the CPU to post a plurality of work queue entries (WQEs) in a receive queue (RQ) of a graphics processing unit (GPU), each of the plurality of WQEs corresponding to one of the plurality of received packets, and concurrently poll a plurality of completion queue entries (CQEs) from a completion queue (CQ) of the GPU, each of the plurality of CQEs corresponding to one of the plurality of WQEs.

[0005] The GPU's memory may include a pre-allocated memory portion mapped to the NIC. The pre-allocated memory portion mapped to the NIC may be divided into a plurality of strides of a fixed maximum transmission unit (MTU) size. Each of the plurality of WQEs may reference a different stride from the plurality of strides.

[0006] Issuing the plurality of WQEs into the RQ of the GPU may include creating a plurality of WQEs in the RQ of the GPU based on the received plurality of packets, issuing a memory barrier instruction for a doorbell record of the NIC, and updating the doorbell record of the NIC based on the created plurality of WQEs.

[0007] Polling the plurality of CQEs in parallel from the GPU's CQ may include polling the plurality of CQEs from each of the plurality of execution threads. More specifically, polling the plurality of CQEs in parallel from the GPU's CQ may include locking the GPU's CQ and storing data from each of the plurality of CQEs in a memory of the GPU. Storing the data from each of the plurality of CQEs in the GPU's memory may include reading an index of the plurality of CQEs, checking whether data of the CQE corresponding to the index in the plurality of CQEs is erroneous, and in response to the data of the CQE corresponding to the index in the plurality of CQEs being erroneous, storing the data of the CQE corresponding to the index in the GPU's memory, and incrementing the index of the plurality of CQEs. Polling the plurality of CQEs from the GPU's CQ may also include issuing a memory barrier instruction against a doorbell record of the NIC, updating the doorbell record of the NIC, and unlocking the GPU's CQ.

[0008] According to another embodiment, a system may include a communication network, a NIC coupled to the communication network, a GPU coupled to the network, and a CPU coupled to the communication network. For example, the communication network may include an Ethernet network. The CPU may include control circuitry for controlling the operation of the CPU. The control circuitry may cause the CPU to receive multiple data packets from the NIC via the communication network, issue multiple WQEs (WQEs) in an RQ of the GPU, each of the multiple WQEs corresponding to one of the multiple received packets, and concurrently poll multiple CQEs (CQEs) from a CQ of the GPU, each of the multiple CQEs corresponding to one of the multiple WQEs.

[0009] The GPU memory may include a pre-allocated memory portion mapped to the NIC. The pre-allocated memory portion mapped to the NIC may be divided into a plurality of strides of a fixed MTU size. Each of the plurality of WQEs may reference a different stride in the plurality of strides.

[0010] Issuing the plurality of WQEs into the RQ of the GPU may include creating a plurality of WQEs in the RQ of the GPU based on the received plurality of packets, issuing a memory barrier instruction for a doorbell record of the NIC, and updating the doorbell record of the NIC based on the created plurality of WQEs.

[0011] Polling the plurality of CQEs in parallel from the GPU's CQ may include polling the plurality of CQEs from each of the plurality of execution threads. More specifically, polling the plurality of CQEs in parallel from the GPU's CQ may include locking the GPU's CQ and storing data from each of the plurality of CQEs in a memory of the GPU. Storing the data from each of the plurality of CQEs in the GPU's memory may include reading an index of the plurality of CQEs, checking whether data of the CQE corresponding to the index in the plurality of CQEs is erroneous, and in response to the data of the CQE corresponding to the index in the plurality of CQEs being erroneous, storing the data of the CQE corresponding to the index in the GPU's memory, and incrementing the index of the plurality of CQEs. Polling the plurality of CQEs from the GPU's CQ may also include issuing a memory barrier instruction against a doorbell record of the NIC, updating the doorbell record of the NIC, and unlocking the GPU's CQ.

[0012] According to yet another embodiment, a method for processing network communications in parallel may include: receiving, by a CPU, a plurality of data packets from a NIC over an Ethernet network, and posting a plurality of WQEs in an RQ of a GPU. Each of the plurality of WQEs may correspond to one of the plurality of received packets. The plurality of CQEs may be polled in parallel from a CQ of the GPU. Each of the plurality of CQEs may correspond to one of the plurality of WQEs. Polling the plurality of CQEs in parallel from the CQ of the GPU may include polling the plurality of CQEs from each of a plurality of execution threads.

[0013] The memory of the GPU may include a pre-allocated memory portion mapped to the NIC. The pre-allocated memory portion mapped to the NIC may be divided into a plurality of strides of a fixed MTU size, and each of the plurality of WQEs may reference a different stride in the plurality of strides.

[0014] Issuing the plurality of WQEs into the RQ of the GPU may include locking the RQ of the GPU, creating the plurality of WQEs in the RQ of the GPU based on the received plurality of packets, issuing a memory barrier instruction for a doorbell record of the NIC, updating the doorbell record of the NIC based on the created plurality of WQEs, and unlocking the RQ of the GPU.

[0015] Polling the plurality of CQEs from the GPU's CQ in parallel may include locking the GPU's CQ and storing data from each of the plurality of CQEs in a memory of the GPU. Storing the data from each of the plurality of CQEs in the GPU's memory may include reading an index of the plurality of CQEs and checking whether data of the CQE in the plurality of CQEs corresponding to the index is erroneous. In response to the data of the CQE in the plurality of CQEs corresponding to the index being erroneous, the data of the CQE in the plurality of CQEs corresponding to the index may be stored in the GPU's memory, and the index of the plurality of CQEs may be incremented. Polling the plurality of CQEs from the GPU's CQ may further include issuing a memory barrier instruction to a doorbell record of a NIC, updating the doorbell record of the NIC, and unlocking the GPU's CQ. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present disclosure is described in conjunction with the accompanying drawings, which are not necessarily drawn to scale.

[0017] Figure 1 is a block diagram illustrating an exemplary environment in which embodiments of the present disclosure may be implemented.

[0018] Figure 2 is a block diagram illustrating a corresponding relationship among a request queue, a completion queue, and a memory according to one embodiment of the present disclosure.

[0019] Figure 3 is a flowchart illustrating an exemplary process of processing network communications in parallel according to one embodiment of the present disclosure.

[0020] Figure 4 is a flow chart illustrating additional details of an exemplary process for polling completion queue entries according to one embodiment of the present disclosure.

[0021] Figure 5 is a flow chart illustrating additional details of an exemplary process for storing data to memory according to one embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] The following description provides only examples and is not intended to limit the scope, applicability, or configuration of the claims. Instead, the following description will provide those skilled in the art with a feasible description of implementing the described embodiments. It should be understood that various changes may be made to the function and arrangement of the elements without departing from the spirit and scope of the appended claims.

[0023] As will be understood from the following description, for reasons of computational efficiency, the components of the system may be arranged at any suitable location in a distributed network of components without affecting the operation of the system.

[0024] Furthermore, it should be understood that the various links connecting the elements may be wired, traced, or wireless links, or any suitable combination thereof, or any other suitable known or later developed element capable of providing data to and / or transmitting data from the connected elements. For example, the transmission medium used as the link may be any suitable electrical signal carrier, including coaxial cable, copper wire and optical fiber, electrical traces on a printed circuit board (PCB), etc.

[0025] As used herein, the phrases "at least one," "one or more," "or," and "and / or" are open-ended expressions that are operationally both conjunctions and disjunctions. For example, each of the expressions "at least one of A, B, and C," "at least one of A, B, or C," "one or more of A, B, and C," "one or more of A, B, or C," "A, B and / or C," and "A, B, or C" means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together.

[0026] As used herein, the term "automatic" and its variations refer to any suitable process or operation that can be performed without substantial human input when the process or operation is performed. However, if input is received before the process or operation is performed, then the process or operation may be automatic, even if the process or operation is performed with substantial or immaterial human input. Human input is considered substantial if it affects the manner in which the process or operation is performed. Human input that consents to the performance of the process or operation shall not be considered "substantial."

[0027] As used herein, the terms "determine," "calculate," and "compute," and variations thereof, are used interchangeably and include any suitable type of methodology, process, operation, or technique.

[0028] Various aspects of the disclosure are described herein with reference to the accompanying drawings, which are schematic illustrations of idealized configurations.

[0029] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It should also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with that in the context of the relevant art and this disclosure.

[0030] As used herein, unless the context clearly indicates otherwise, the singular forms "a," "an," and "the" are intended to include the plural forms as well. It should also be understood that when the terms "include," "comprise," and / or "comprising" are used in this specification, they specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. The term "and / or" includes any and all combinations of one or more of the associated listed items.

[0031] Now refer to Figures 1 to 5 , various systems and methods for processing network communications in parallel on a device that supports high levels of parallelism, such as a graphics processing unit (GPU), will be described. Generally speaking, embodiments of the present disclosure are directed to an inline packet processing pipeline to receive packets in GPU memory without temporarily storing a copy in central processing unit (CPU) memory, process the received packets in parallel with one or more cores of the GPU, and then run inference, evaluation, or send the results of the computation over a network. In this way, the highly parallel nature of the GPU can be exploited to process network communications without involving other elements of the system, such as the CPU, which could quickly become consumed by processing the network communications to the detriment of other processes.

[0032] Figure 1 1 is a block diagram illustrating an exemplary environment in which embodiments of the present disclosure may be implemented. As shown in this example, environment 100 may include multiple processing threads 105 executing in parallel on one or more elements of environment 100 (not shown herein). These threads may include, for example, one or more CUDA threads executing on any of a variety of general-purpose endpoints within environment 100. Processing threads 105 may send multiple packets 110 over a communication network 115 (e.g., an Ethernet network). For example, packets 110 may be received by a network interface controller (NIC) for routing to other elements of environment 100, as known in the art. Environment 100 may also include a GPU 125. GPU 125 may include control circuitry 130 that controls the operation of GPU 125. Control circuitry 130 includes a central processing unit (CPU) 150, which also includes control circuitry 155, such as one or more microprocessors, or similar components known in the art.

[0033] As described above, embodiments are directed to utilizing a GPU to process data packets 110 in parallel without intervention by other components of the environment 100. To this end, the control circuitry 155 of the CPU 150 may cause the CPU 150 to receive a plurality of data packets 110 from the NIC 120 via the communication network 115. The control circuitry 155 may also cause the CPU 150 to post a plurality of work queue entries (WQEs) in a receive queue (RQ) 135 of the GPU 125, each of the plurality of WQEs corresponding to one of the plurality of received packets, and to poll a plurality of completion queue entries (CQEs) in parallel from a completion queue (CQ) 140 of the GPU 125, each of the plurality of CQEs corresponding to one of the plurality of WQEs.

[0034] For example, GPU 125 may utilize the NVIDIA DOCA GPUNetIO library, which has modified GPU receive operations capable of receiving several packets simultaneously in parallel within a given number of nanoseconds, exploring the possibility of multiple CUDA kernels collaborating on the same receive queue.

[0035] Typically, in the MLX5 protocol, the application (CPU or GPU) repeats the previous steps each time it wants to receive a new set of packets. Creating a receive WQE means creating a new 16B descriptor in the RQ memory with the following information about the memory area where the packet should be received: memory key (mkey), address, and number of bytes.

[0036] In Ethernet communications, applications are expected to receive packets up to the maximum transmission unit (MTU) set on the interface. Additionally, the memory key associated with each WQE can be the same if it references the same memory region allocated and mapped to receive multiple packets.

[0037] Figure 2 is a block diagram illustrating the correspondence between request queues, completion queues, and memory according to one embodiment of the present disclosure. As shown here, the memory of GPU 125 may include a pre-allocated memory portion 205 mapped to NIC 120. The pre-allocated memory portion 205 mapped to NIC 120 may be divided into a plurality of strides 210A-210D of a fixed MTU size. Each of a plurality of WQEs 215A-215C in RQ 135 may reference a different stride in the plurality of strides 210A-210C.

[0038] By pre-allocating a large portion of GPU memory and mapping it to the NIC 120, for example, by using a single mkey for the entire memory region and dividing the memory 205 into multiple strides 210A-210D of a fixed MTU size, all WQEs 215A-215C in the RQ 135 can be pre-issued once from the CPU at the beginning (setup phase), with each WQE mkey, address, and size connected to a different stride 210A-210C of the same GPU memory block.

[0039] This queue structure does not require any WQE updates at runtime when receiving packets from the CUDA kernel, as each WQE 215A-215C is posted and connected to the same GPU memory stride 210A-210C. The only operation that the GPU must perform at runtime is to update the NIC 120's doorbell record 145 to communicate from the application to the NIC 120 what the next available WQE is for receiving new packets.

[0040] The MLX5 protocol specifies that for X consecutive receive WQEs 215A-215C, if the NIC 120 receives X packets, then X consecutive CQEs 220A-220C are posted in the CQ 140. As an example, if the RQ 135 posts five WQEs (WQE0, ..., WQE4) and five packets containing these WQEs are received, then five CQEs (CQE0, ..., CQE4) will be created in the CQ without any "empty space" between the CQEs.

[0041] When executing this algorithm in a CUDA kernel, the operation can be parallelized, that is, multiple CUDA threads (at the CUDA block or CUDA warp level) can poll different CQEs at different locations in parallel. DOCA GPUNetIO provides a parallelized receive function that the CUDA kernel can call to poll multiple CQEs from different CUDA threads within a given number of nanoseconds. Specifically, the receive function can be called by all threads in a CUDA block or CUDA warp.

[0042] Combined with the assumption of continuous CQE for continuously received packets and the assumption that each packet is received in the next stride of the GPU memory receive buffer, the function may return the first stride id used to receive the first packet and the number of packets received during the receive function execution.

[0043] Figure 33 is a flow chart illustrating an exemplary process for parallel processing of network communications according to one embodiment of the present disclosure. As shown in this example, the parallel processing of network communications that may be performed by the CPU 150 as described above may include receiving 305 a plurality of data packets 110 from the NIC 120 via the network 115 by the CPU 150. As noted, the network 110 may include, for example, an Ethernet network. A plurality of WQEs 215A-215C may be issued 310 to the RQ 135 of the GPU 125. Each of the plurality of WQEs 215A-215C may correspond to one of the plurality of packets 110 received 305. Reference will now be made to FIG. Figure 4 Further details of an exemplary process for issuing 310 multiple WQEs 215A- 215C in RQ 135 of GPU 125 are described.

[0044] The plurality of CQEs 220A-220C may be polled 315 from the CQ 140 of the CPU 150. Each of the plurality of CQEs 220A-220C may correspond to one of the plurality of WQEs 215A-215C. Polling 315 the plurality of CQEs 220A-220C from the CQ 140 of the GPU 125 may include polling the plurality of CQEs in parallel from each of the plurality of execution threads 105. Figure 5 Further details of an exemplary process of polling 315 multiple CQEs 220A-220C from the CQ 140 of the GPU 125 are described.

[0045] Figure 4 4 is a flow chart illustrating further details of an exemplary process for polling completion queue entries according to one embodiment of the present disclosure. As shown in this example, as described above, the CPU 150 may execute a plurality of WQEs 215A-215C to the RQ 135 of the GPU 125, which may include optionally locking 405 the RQ 135 of the GPU 125. Based on the received plurality of packets 110, the plurality of WQEs 215A-215C may be created 410 in the RQ 135 of the GPU 125. A memory barrier instruction may be issued 415 for the doorbell record 145 of the NIC 120. The doorbell record 145 of the NIC 120 may then be updated 420 based on the plurality of WQEs 215A-215C created 410, and the RQ 135 of the GPU 125 may be unlocked 425 (if previously locked 405). It should be noted that while locking the RQ in each CUDA block or CUDA warp is logically correct, explicitly locking 405 and unlocking 425 the RQ does not need to be performed via lock / unlock instructions. Instead, the application can simply assign RQ0 to CUDA block 0, RQ1 to CUDA block 1, and so on.

[0046] Figure 5 is a flow chart illustrating additional details of an exemplary process for storing data in memory according to one embodiment of the present disclosure. As shown in this example, polling a plurality of CQEs 220A-220C in parallel from the CQ 140 of the GPU 125, as may be performed by the CPU 150 as described above, may include locking 605 the CQ 140 of the GPU 125 and storing data from each of the plurality of CQEs 220A-220C in the memory 205 of the GPU 125. Storing data from each of the plurality of CQEs 220A-220C in the memory 205 of the GPU 125 may include reading 510 an index 225 of the plurality of CQEs 220A-220C and checking 515 whether the data of the CQE corresponding to the index 225 in the plurality of CQEs 220A-220C is erroneous. In response to determining 515 that the data of the CQE corresponding to the index 225 in the plurality of CQEs 220A-220C is error-free, the data of the CQE corresponding to the index 225 in the plurality of CQEs 220A-220C may be stored 520 in the memory 205 of the GPU 125, and the index 225 of the plurality of CQEs 220A-220C may be incremented 525. Polling the plurality of CQEs 220A-220C from the CQ 140 of the GPU 125 may then further include issuing 530 a memory barrier instruction to the doorbell record 145 of the NIC 120, updating 535 the doorbell record 145 of the NIC 120, and unlocking 540 the CQ 140 of the GPU 125.

[0047] It should be noted that there may be many variations in the structure, functionality, order of operations, and / or other aspects of the various embodiments described herein. The operations of the exemplary process described above for synchronizing clocks between computing devices may be performed in a different order, and each operation need not be dependent on a previous event or operation. For example, the sending of a synchronization message may be initiated by any device at any time and need not occur in response to an event such as the receipt of a synchronization message or other event. Furthermore, the process of setting the clock need not be performed in response to the completion of a conversation. For example, the task of measuring a clock offset may be performed in one process, while the task of setting the clock based on the clock offset may be completed in a second process that runs asynchronously relative to the first process. Other such variations are also contemplated and are considered to be within the scope of this disclosure.

[0048] The present disclosure includes, in various aspects, embodiments, and / or configurations, components, methods, processes, systems, and / or apparatus substantially as described and illustrated herein, including various aspects, embodiments, configuration embodiments, subcombinations, and / or subsets. One skilled in the art will understand how to make and use the disclosed aspects, embodiments, and / or configurations after understanding the present disclosure. The present disclosure includes, in various aspects, embodiments, and / or configurations, providing devices and processes without items not described and / or not described herein, or providing devices and processes in various aspects, embodiments, and / or configurations herein, including providing devices and processes without items that may have been used in previous devices or processes, for example, to improve performance, achieve simplicity, and / or reduce implementation costs.

[0049] The foregoing discussion has been presented for purposes of illustration and description. The foregoing is not intended to limit the present disclosure to the forms disclosed herein. For example, in the foregoing detailed description, various features of the present disclosure are combined in one or more aspects, embodiments and / or configurations in order to simplify the disclosure. Features of aspects, embodiments and / or configurations of the present disclosure may be combined in alternative aspects, embodiments and / or configurations other than those described above. This method of disclosure should not be interpreted as reflecting an intention that the claims require more features than expressly recited in each claim. On the contrary, as reflected in the following claims, inventive aspects are present in all features of a single foregoing disclosed aspect, embodiment and / or configuration. Therefore, the following claims are hereby incorporated into this detailed description, with each claim independently serving as a separate preferred embodiment of the present disclosure.

[0050] In addition, although this specification includes descriptions of one or more aspects, embodiments and / or configurations and certain variations and modifications, other variations, combinations and modifications are also within the scope of this disclosure, for example, after understanding the present disclosure, the skills and knowledge of those skilled in the art may encompass such variations, combinations and modifications. It is intended to obtain the right to include alternative aspects, embodiments and / or configurations to the extent permitted, including structures, functions, ranges or steps that are substituted, interchangeable and / or equivalent to the claimed structures, functions, ranges or steps, regardless of whether such substituted, interchangeable and / or equivalent structures, functions, ranges or steps are disclosed herein, and it is not intended to disclose any contribution to patentable subject matter.

Claims

1. A central processing unit (CPU), comprising: a control circuit that controls the operation of the CPU, wherein the control circuit causes the CPU to: receiving a plurality of data packets from a network interface card (NIC) via a communication network; Posting a plurality of work queue entries WQE in a receive queue RQ of a graphics processing unit GPU, each WQE in the plurality of WQEs corresponding to one of the plurality of received packets; and A plurality of completion queue entries CQEs are polled in parallel from a completion queue CQ of the GPU, each CQE of the plurality of CQEs corresponding to one WQE of the plurality of WQEs.

2. The CPU of claim 1 , wherein issuing the plurality of WQEs in the RQ of the GPU comprises: creating the plurality of WQEs in the RQ of the GPU based on the received plurality of packets; issuing a memory barrier instruction for the doorbell record of the NIC; as well as The doorbell record of the NIC is updated based on the created plurality of WQEs.

3. The CPU of claim 2 , wherein the memory of the GPU comprises a pre-allocated memory portion mapped to the NIC, wherein the pre-allocated memory portion mapped to the NIC is divided into a plurality of strides of a fixed maximum transmission unit (MTU) size, and wherein each of the plurality of WQEs references a different stride among the plurality of strides.

4. The CPU of claim 1 , wherein polling the plurality of CQEs in parallel from the CQ of the GPU comprises: Locking the CQ of the GPU; storing data from each of the plurality of CQEs to a memory of the GPU; issuing a memory barrier instruction for the doorbell record of the NIC; Updating the doorbell record of the NIC; as well as Unlock the CQ of the GPU.

5. The CPU of claim 4 , wherein storing data from each of the plurality of CQEs to a memory of the GPU comprises: Reading indexes of the plurality of CQEs; Checking whether data of a CQE corresponding to the index among the plurality of CQEs is incorrect; In response to data of the CQE corresponding to the index among the plurality of CQEs being free of error, storing the data of the CQE corresponding to the index among the plurality of CQEs in the memory of the GPU; and The index of the plurality of CQEs is incremented. 6 . The CPU of claim 4 , wherein polling the plurality of CQEs in parallel from the CQ of the GPU comprises polling a plurality of CQEs from each of a plurality of execution threads.

7. The CPU according to claim 1, wherein the communication network comprises an Ethernet network.

8. A system comprising: Communication networks; A network interface card NIC, coupled to the communication network; A graphics processing unit (GPU) coupled to the communication network; as well as a central processing unit (CPU) coupled to the communication network, the CPU comprising a control circuit for controlling the operation of the CPU, wherein the control circuit causes the CPU to: receiving a plurality of data packets from the NIC via the communication network; Posting a plurality of work queue entries (WQEs) in a receive queue (RQ) of the GPU, each WQE in the plurality of WQEs corresponding to one of the plurality of received packets; and A plurality of completion queue entries CQEs are polled in parallel from a completion queue CQ of the GPU, each CQE of the plurality of CQEs corresponding to one WQE of the plurality of WQEs.

9. The system of claim 8, wherein publishing the plurality of WQEs in an RQ of the GPU comprises: creating the plurality of WQEs in the RQ of the GPU based on the received plurality of packets; issuing a memory barrier instruction for the doorbell record of the NIC; as well as The doorbell record of the NIC is updated based on the created plurality of WQEs.

10. The system of claim 9, wherein the memory of the GPU comprises a pre-allocated memory portion mapped to the NIC, wherein The pre-allocated memory portion mapped to the NIC is divided into a plurality of strides of a fixed maximum transmission unit (MTU) size, and wherein each WQE of the plurality of WQEs references a different stride of the plurality of strides.

11. The system of claim 8, wherein polling the plurality of CQEs in parallel from the CQ of the GPU comprises: Locking the CQ of the GPU; storing data from each of the plurality of CQEs into a memory of the GPU; issuing a memory barrier instruction for the doorbell record of the NIC; Updating the doorbell record of the NIC; as well as Unlock the CQ of the GPU.

12. The system of claim 11 , wherein storing data from each of the plurality of CQEs into a memory of the GPU comprises: Reading indexes of the plurality of CQEs; Checking whether data of a CQE corresponding to the index among the plurality of CQEs is incorrect; In response to data of the CQE corresponding to the index among the plurality of CQEs being free of error, storing the data of the CQE corresponding to the index among the plurality of CQEs in the memory of the GPU; and The index of the plurality of CQEs is incremented. 13 . The system of claim 11 , wherein polling the plurality of CQEs in parallel from the CQ of the GPU comprises polling a plurality of CQEs from each of a plurality of execution threads.

14. The system of claim 8, wherein the communication network comprises an Ethernet network.

15. A method for parallel processing of network communications, the method comprising: The control circuit of the central processing unit CPU receives a plurality of data packets from the network interface card NIC via the Ethernet network; Posting, by the control circuit of the CPU, a plurality of work queue entries (WQEs) in a receive queue (RQ) of a graphics processing unit (GPU), each of the plurality of WQEs corresponding to one of the plurality of received packets; and A plurality of completion queue entries CQEs are polled in parallel from a completion queue CQ of the GPU by the control circuit of the CPU, each CQE of the plurality of CQEs corresponding to one WQE of the plurality of WQEs.

16. The method of claim 15 , wherein publishing the plurality of WQEs in the RQ of the GPU comprises: creating the plurality of WQEs in the RQ of the GPU based on the received plurality of packets; issuing a memory barrier instruction for the doorbell record of the NIC; as well as The doorbell record of the NIC is updated based on the created plurality of WQEs.

17. The method of claim 16 , wherein the memory of the GPU comprises a pre-allocated memory portion mapped to the NIC, wherein the pre-allocated memory portion mapped to the NIC is divided into a plurality of strides of a fixed maximum transmission unit (MTU) size, and wherein each of the plurality of WQEs references a different stride from the plurality of strides.

18. The method of claim 15, wherein polling the plurality of CQEs in parallel from the CQ of the GPU comprises: Locking the CQ of the GPU; storing data from each of the plurality of CQEs to a memory of the GPU; issuing a memory barrier instruction for the doorbell record of the NIC; Updating the doorbell record of the NIC; as well as Unlock the CQ of the GPU.

19. The method of claim 18, wherein storing data from each of the plurality of CQEs to a memory of the GPU comprises: Reading indexes of the plurality of CQEs; Checking whether data of a CQE corresponding to the index among the plurality of CQEs is incorrect; In response to data of the CQE corresponding to the index among the plurality of CQEs being free of error, storing the data of the CQE corresponding to the index among the plurality of CQEs in the memory of the GPU; and The index of the plurality of CQEs is incremented.

20. The method of claim 18, wherein polling the plurality of CQEs in parallel from the CQ of the GPU comprises polling a plurality of CQEs from each of a plurality of execution threads.

Images