Data packet receiving method, data processing unit, system and network card

Abstract

The invention provides a data packet receiving and processing method, which comprises the following steps of: when a network data packet is received, caching the data packet in a cache unit of a data processing unit according to a queue; reading a notification from a host, analyzing the notification to obtain a receiving queue entry address, and obtaining address information of an empty cache region allocated by the host from the receiving queue entry; writing a plurality of data packets cached in the data processing unit into an empty cache region allocated by a host through a single PCIe write operation, and supplementing each packet of data according to a cache line alignment principle; generating a corresponding completion queue entry CQE for each data packet written into the host cache region; and uploading the plurality of CQEs to a host through a single PCIe write operation. The invention further discloses a corresponding data processing unit and system, a storage medium and a network card. By implementing the application, the efficiency of receiving the packet data can be improved, and the system performance is improved.

Description

Technical Field

[0001] The present application relates to the technical field of communication between a host and a data processing unit (DPU), and in particular to a data packet receiving and processing method, a data processing unit, a system, a storage medium, and a network card. Background Art

[0002] With the rapid development of the cloud computing industry, virtualization services have been fully developed. However, with the development of high-computing services such as AI (Artificial Intelligence), equipment has put forward higher requirements for performance, latency, etc. There is an urgent need for a more efficient solution to improve the processing performance of network cards. Therefore, Yunbao Intelligent Company proposed the Jnet Protocol. The Jnet protocol is adopted on the host driver side of a series of smart network interface cards (SmartNetwork Interface Card, SmartNIC) and data processing unit (Data Processing Unit, DPU) products. The Jnet protocol can provide higher performance and lower latency.

[0003] Smart NICs and DPUs are back-end devices based on the Jnet protocol, interacting with front-end drivers through Jnet queues. NICs play a crucial role in the entire data path, and a high-performance Smart NIC or DPU can significantly impact a product's market success.

[0004] The performance of the network card is an indicator that needs to be paid attention to. It mainly includes two indicators: the number of packets processed per second (Packet Per Second, pps) and the port rate (Bits Per Second, bps). Among them: pps is one of the important indicators to measure the processing capacity of the network card and the performance of the network. It indicates the number of data packets that the network card can process or forward within the time range of one second. The data packet is the basic unit in network communication and contains data transmission information, such as source address, destination address, protocol type, etc. The pps capability of the network card directly affects the throughput and response speed of the network. A higher pps value means that the server can process data packets faster and provide higher network performance. Therefore, pps is an important indicator when designing and selecting a server, especially in scenarios where a large number of data packets need to be processed, such as network routers, firewalls, load balancers, etc.

[0005] Therefore, the existing network RX direction (receiving direction) performance has the following main shortcomings:

[0006] First, the performance limit for receiving small packets is 250Mpps. Because the Jnet protocol, like large packets, uses multiple high-speed Peripheral Component Interconnect Express (PCIe) write operations for small I / O data transmission, and PCIe bandwidth is a significant factor affecting performance, performance cannot be increased. Limited by the Jnet protocol's small packet PPS performance limit, the BPS of small packets can only reach a maximum of 16Gbps, which is very low compared to the full bandwidth of 200Gbps, affecting the BPS performance of the network card.

[0007] Secondly, the Jnet driver software has the same processing flow for small and large packets, and needs to maintain a set of complex processes. When processing small packets (less than 64 bytes), the buffer utilization rate is not high, only 1.56% (64B / 4096B), which wastes host resources. Therefore, the driver or software is inefficient in processing small packets.

[0008] In summary, performance indicators are very important for network cards. However, the performance of receiving small packets is currently limited by the Jnet protocol standard and can only reach a maximum of 250Mpps. This affects the bps and pps performance indicators of small packets. At the same time, the efficiency of the driver software in processing small packets is also relatively low. Summary of the Invention

[0009] The technical problem to be solved by this application is to provide a data packet receiving and processing method, a data processing unit, a system, a storage medium and a network card, which can improve the efficiency of receiving small packet data and improve system performance.

[0010] To solve the above technical problems, as one aspect of the present application, a method for receiving and processing a data packet is provided, which is applied to a data processing unit and includes the following steps:

[0011] Receive network data packets and cache the data packets in the cache unit of the data processing unit according to the queue;

[0012] Reading a notification from the host, parsing the notification to obtain a receive queue entry address, and obtaining address information of an empty buffer area allocated by the host from the receive queue entry;

[0013] Multiple packets from the same queue in the cache unit are written to an empty cache area allocated by the host through a single PCIe write operation, where each packet is padded according to the cache line alignment principle;

[0014] Generate a corresponding completion queue entry CQE for each data packet written to the host buffer;

[0015] The plurality of CQEs are uploaded to the host through a single PCIe write operation.

[0016] Optionally, writing the multiple data packets in the same queue in the cache unit into an empty cache area allocated by the host through a single PCIe write operation includes:

[0017] When the length of the data packets in the same queue in the cache unit reaches the length of the PCIe bandwidth, multiple data packets in the same queue in the cache unit are written into the empty cache area allocated by the host through a single PCIe write operation;

[0018] Or when a preset timeout period is reached, the data packets in the same queue in the cache unit are written into an empty cache area allocated by the host through a single PCIe write operation.

[0019] Optionally, generating a corresponding completion queue entry CQE for each data packet written into the host buffer area further includes:

[0020] The data processing unit generates a CQE for each data packet and fills the enable bit information, start packet identifier and end packet identifier of the data packet in each CQE;

[0021] The enable bit information is used to indicate whether to enable the sequence recognition function of multiple data packets. When the enable bit information is valid, the start packet identifier and end packet identifier in the CQE are valid;

[0022] When the start packet identifier is valid, it indicates that the data packet corresponding to the current CQE is the start data packet written into the current empty buffer area;

[0023] When the end packet identifier is valid, it indicates that the data packet corresponding to the current CQE is the end data packet written into the current buffer area.

[0024] Optionally, uploading the plurality of CQEs to the host through a single PCIe write operation further includes:

[0025] Multiple CQEs are aggregated into one data block, and the data block is uploaded to the host through a single PCIe write operation.

[0026] A second aspect of the present application provides a method for receiving and processing a data packet, comprising the following steps:

[0027] The host allocates an empty buffer area for each receive queue entry, generates the receive queue entry, and sends a notification to the data processing unit;

[0028] The data processing unit receives network data packets and caches the data packets in a cache unit of the data processing unit according to the queue;

[0029] The data processing unit reads a notification from the host, parses the notification to obtain a receive queue entry address, and obtains address information of an empty buffer area allocated by the host from the receive queue entry;

[0030] The data processing unit writes multiple data packets of the same queue in the cache unit into the empty cache area allocated by the host through a single PCIe write operation, and each data packet is padded according to the cache line alignment principle;

[0031] The data processing unit generates a corresponding completion queue entry CQE for each data packet written into the empty buffer area;

[0032] The data processing unit uploads multiple CQEs to the host through a single PCIe write operation

[0033] The host obtains corresponding data packets according to the multiple CQEs.

[0034] Optionally, the data processing unit generating a corresponding completion queue entry CQE for each data packet written into the empty buffer area further includes:

[0035] The data processing unit generates a CQE for each data packet, and each CQE is filled with an enable bit, a start packet identifier, and an end packet identifier;

[0036] The enable bit signal is used to indicate whether to enable the sequence recognition function of multiple data packets. When the enable bit signal is valid, the start packet identifier and end packet identifier in the CQE are valid;

[0037] When the start packet identifier is valid, it indicates that the data packet corresponding to the current CQE is the start data packet written into the current empty buffer area;

[0038] When the end packet identifier is valid, it indicates that the data packet corresponding to the current CQE is the end data packet written into the current buffer area.

[0039] Optionally, the host receives the CQE and obtains valid data of each received data packet, further comprising:

[0040] The host determines the start data packet and the end data packet in the buffer area according to the start packet identifier and the end packet identifier in each CQE, and determines the order of each data packet;

[0041] The host reads the corresponding data block from the buffer area according to the length information of the data identified in each CQE to obtain the corresponding data packet.

[0042] A third aspect of the present application provides a data processing unit, comprising:

[0043] A cache unit, configured to cache received network data packets according to queues when receiving the data packets;

[0044] a notification parsing unit, configured to receive a notification sent by a host, parse the notification to obtain a receive queue entry address, and obtain address information of an empty buffer area allocated by the host from the receive queue entry;

[0045] A write data processing unit, configured to write multiple data packets of the same queue in the cache unit into an empty cache area allocated by the host through a single PCIe write operation, wherein each data packet is padded according to a cache line alignment principle;

[0046] A CQE allocation unit is configured to generate a corresponding completion queue entry CQE for each data packet written into the host buffer area;

[0047] The CQE upload unit is used to upload multiple CQEs to the host through a single PCIe write operation.

[0048] Optionally, the CQE allocation unit is further configured to generate a CQE for each data packet, and fill in the enable bit information, start packet identifier, and end packet identifier of the data packet in each CQE;

[0049] The enable bit information is used to indicate whether to enable the sequence recognition function of multiple data packets. When the enable bit information is valid, the start packet identifier and end packet identifier in the CQE are valid;

[0050] When the start packet identifier is valid, it indicates that the data packet corresponding to the current CQE is the start data packet written into the current empty buffer area;

[0051] When the end packet identifier is valid, it indicates that the data packet corresponding to the current CQE is the end data packet written into the current buffer area.

[0052] A fourth aspect of the present application provides a data packet receiving and processing system, comprising a host and the aforementioned data processing unit:

[0053] The host is configured to allocate an empty buffer area for each receive queue entry, generate the receive queue entry, and send a notification to the data processing unit DPU;

[0054] The host is further configured to receive multiple CQEs sent by the data processing unit, obtain the positions and sequences of multiple data packets in the buffer area, and obtain each data packet.

[0055] A fifth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the aforementioned method when executed by a processor.

[0056] A sixth aspect of the present application provides a network card in which the data processing unit as described above is deployed.

[0057] Implementing this embodiment has the following beneficial effects:

[0058] The present application provides a data packet receiving and processing method, a data processing unit, a system, a storage medium, and a network card. Through a single PCIe, multiple data packets of the same queue are written into an empty cache area on the host side. Based on the cache line alignment principle, the software and hardware collaborate to achieve the transmission of multiple small packets of data in a single cache area. In addition, the completion queue entry structure of the receiving queue is improved, and a chained CQE structure is adopted to improve the efficiency of a single PCIe write of valid data. This technology effectively reduces the number of PCIe writes to the cache area and CQE, increases the amount of valid data in a single PCIe write, and greatly improves the PCIe bandwidth utilization. It has been verified that the present application can significantly improve the performance of small packets in the receiving direction.

[0059] Without changing the core chip architecture, this application achieves efficient utilization of PCIe bandwidth and improves the driver software's utilization of empty buffers when processing small packets. This eliminates the need for software to maintain multiple buffer identifiers and fragmented address spaces, allowing multiple small packets to be transmitted using a single, continuous block of space. This further improves software efficiency and system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, without paying any creative labor, obtaining other drawings based on these drawings still falls within the scope of the present application.

[0061] Figure 1 A schematic diagram of the process of receiving data communication between the host and the data processing unit (DPU) involved in the operating environment of this application;

[0062] Figure 2 for Figure 1 A schematic diagram of the data processing flow in the receiving direction adopted by the receiving queue;

[0063] Figure 3 Schematic diagram of the delay model involved in the transmission process;

[0064] Figure 4 A schematic diagram of the main flow of an embodiment of a method for receiving and processing data packets provided by the present application;

[0065] Figure 5 for Figure 4 A schematic diagram of the principle of data packet reception and processing of the method involved;

[0066] Figure 6 A schematic diagram of the data structure of an existing completion queue entry;

[0067] Figure 7 This is a schematic diagram of the data structure of the improved completion queue entry in this application;

[0068] Figure 8 Schematic diagram comparing the aggregation of complete queue entries before and after the method of the present application is adopted;

[0069] Figure 9 A schematic structural diagram of an embodiment of a data packet receiving and processing system provided by the present application;

[0070] Figure 10 A schematic structural diagram of an embodiment of a data packet receiving and processing system provided by the present application;

[0071] Figure 11 for Figure 10 Schematic diagram of the structure of the host;

[0072] Figure 12 for Figure 10 Schematic diagram of the structure of the data processing unit. DETAILED DESCRIPTION

[0073] In order to make the objectives, technical solutions and advantages of this application clearer, this application will be described in further detail below with reference to the accompanying drawings.

[0074] Before describing the detailed method of this application, first Figures 1 to 4 Describe the implementation environment involved in this application method.

[0075] like Figure 1 As shown, a flow chart of data communication between the host (Host) and the data processing unit (DPU) involved in the operating environment of this application is shown; the data communication protocol adopted in this application is the Jnet protocol, and Jnet uses paired rings (Ring) to implement Jnet queues (Queue). Each Queue is a queue that carries a large amount of data. The specific number of queues used depends on business needs. In a specific example, the Jnet network driver generally uses multiple Tx Queues (multiple pairs of Tx-Rings) to implement network data transmission.

[0076] The host and DPU receive remote network data through the Jnet receive queue ring (Jnet-Rx Rq-Ring) and the Jnet receive completion queue ring (Jnet-Rx Cq-Ring). The process is as follows: Figure 1 As shown, the main steps are as follows:

[0077] Step 1: The host driver prepares an empty data buffer and updates the receive queue index (Rq ring index) corresponding to this I / O.

[0078] Step 2: The host sends a notification to the DPU, which includes the corresponding virtual function ID (vf_id), queue ID (queue_id), and the request index.

[0079] Step 3: The DPU parses the notification and initiates a request to read the receive queue ring entry (RQE) to obtain the address and length information of the empty buffer allocated by the host.

[0080] Step 4: When the remote device sends data, the DPU will determine whether there is an empty buffer area in the current queue. If so, it will proceed to the next step.

[0081] Step 5: The DPU initiates a PCIe write data request to the Host and sends the received data to the Host's empty buffer.

[0082] Step 6: The DPU updates the host-side receive queue completion queue entry (CQE), including the buffer ID, length, and some network associated information, for the host to process.

[0083] Step 7: The DPU sends an MSI-X interrupt to the Host to inform the Host that the data transmission is completed;

[0084] Step 8: The host updates its locally maintained receive queue entry and wrap counter based on the uploaded entry information. It then parses the receive queue entry information, obtains the buffer information, processes the data in the buffer, and releases the buffer information in the entry for next use.

[0085] Figure 2 The following is a simplified diagram of the data processing flow in the RX direction of the Jnet protocol. Figure 1 From the above process analysis, we can see that when the Host sends the receive queue entry, the RQE will carry information such as the address of the empty buffer area reserved for the receive direction, the length of the buffer area, and the buffer area identifier (buf_id). Figure 2This is shown in the process circled in 1. When the host injects an empty buffer, it does not know the actual size of the received message. Therefore, to ensure that it can fully receive the next message, it generally allocates a larger empty buffer (for example, length = 4KB). Furthermore, the host does not know the number of received messages, so it allocates more empty buffers to ensure that there is sufficient empty buffer space to receive data in the RX direction. The Rq Ring has a total queue size (Queue_size) of entries, and generally allocates the largest queue size of empty buffers, which is fixed at 4KB.

[0086] The Jnet protocol stipulates that when the Host allocates RQE, it will allocate 4 times the number of completion queue entries CQE for receiving the buf_id, data length and other information sent, such as Figure 2 As shown in the middle circle 2. When a frame data packet (frame_packet) is sent to the Host, the DPU will write the data into the buffer in byte order, as shown in the following example: Figure 2 As shown in the middle circle 3. Finally, the relevant information of this buffer area will be written into CQE, which is provided to the host to process the received data.

[0087] The above analysis is correct. Generally, the host allocates empty buffers of the same size for each queue size. This presents a problem: when receiving small packets, the buffer utilization is low. For example, the host allocates an empty buffer of 4KB for the RX queue's I / O, but the majority of received packets are small (less than 64 bytes) or short (64 to 1518 bytes). In this case, buffer utilization is extremely low. Jnet requires a complete process to receive a small amount of data (64 bytes), resulting in very low PCIe write bandwidth utilization for each transaction. As will be explained in the following analysis, multiple PCIe write operations occur during this process, and PCIe physical bandwidth is a key factor in performance. Therefore, RX performance can only reach a maximum of 250Mpps.

[0088] By combing through the DPU's RX data processing process, we found that regardless of whether it receives large or small packets, both the software and hardware need to go through a complete set of Jnet protocol processes: updating the producer index, sending the doorbell message, processing RQE data, filling empty buffer data, sending CQE data, sending interrupt data, parsing CQE data, releasing and reusing the buffer, CQE, and RQE, etc. Looking at the entire transmission process, the data transmission delay model in the receiving direction is as follows: Figure 3 As shown:

[0089] T1: Host notification latency, which is a single PCIe write latency and typically 1000ns.

[0090] T2: DPU single read cache descriptor (CQE) latency, the latency is a single PCIe read request and read return latency of 2 * 1000ns;

[0091] T3: DPU data receiving delay, which is the internal path delay;

[0092] T4: DPU writes back to host memory. The latency is the one-way latency of the PCIe link, 1000ns.

[0093] T5: DPU writes back to Host CQE. The latency is the one-way latency of the PCIe link, 1000ns.

[0094] T6: DPU writes back to the Host MSIX interrupt. The latency is the one-way latency of the PCIe link, 1000ns.

[0095] From the above analysis, we know that the total processing time for a received data is: T1+T2+T3+T4+T5+T6. The idea of ​​this design is to reduce the total processing time and thus improve data transmission performance.

[0096] The T1 time can be achieved by triggering a notification through software, thereby reducing the number of doorbell transmission times.

[0097] T2 is the time to read CQE. You can also read multiple CQEs at once to reduce the number of PCIe reads and thus reduce T2.

[0098] T3 is the processing delay inside the DPU. The delay of a high-performance DPU can already be very low, so the improvement is limited.

[0099] T4 represents the latency of writing to host memory. When processing large packets, the PCIe write bandwidth is already saturated and cannot be increased. However, when processing small packets, the PCIe write bandwidth can be increased, combining multiple PCIe writes into a single one. This reduces the number of PCIe writes and thus reduces T4.

[0100] T5 is the Host CQE write latency. Similarly, CQE aggregation can be used to reduce the number of PCIe writes and shorten T4.

[0101] T6 can reduce the number of PCIe writes and thus the latency by using interrupt merging.

[0102] Analysis shows that T4 and T5 account for a large proportion of the time in the I / O processing of small packets, especially when processing multiple data records. If the time of T4 and T5 can be effectively reduced, the performance of small packet reception can be greatly improved.

[0103] like Figure 4 FIG. 1 shows a schematic diagram of the main flow of an embodiment of a method for receiving and processing a data packet provided by the present application; and FIG. Figures 5 to 8 As shown, in this embodiment, the method includes at least the following steps:

[0104] In step S10 , the host allocates an empty buffer area for each receive queue entry, generates the receive queue entry, and sends a notification to the data processing unit.

[0105] Specifically, the host allocates an empty buffer area for each receive queue entry, and fills the starting address and length information of the buffer area into the receive queue entry. The host sends a notification to the DPU, which includes a queue identifier and an entry index so that the DPU can obtain the address information of the empty buffer area from the receive queue entry.

[0106] Further integration Figure 5 As shown, taking the empty buffer area allocated by the host side as 4Kbyte as an example, the host writes the relevant information of the empty buffer0 into RQE0, and then the DPU obtains it by sending a notification, such as Figure 5 Shown in the middle circle 1.

[0107] Step S11 : The data processing unit DPU receives network data packets and caches the data packets in a cache unit of the data processing unit according to a queue.

[0108] Step S12: The DPU receives the notification, parses the notification to obtain a receive queue entry address, and obtains address information of an empty buffer area allocated by the host from the receive queue entry.

[0109] In step S13, the DPU writes multiple data packets of the same queue in the cache unit into the empty cache area allocated by the host through a single PCIe write operation. Each data packet is padded according to the cache line alignment principle to ensure that the written data block is an integer multiple of the cache line size.

[0110] Further Figure 5As shown, when the DPU receives small packets, it writes frame_packet_0, frame_packet_1, frame_packet_2, ...frame_packet_n-1 into an empty buffer in sequence. To improve the performance of software reading and writing memory, the written data is generally aligned with the cache line (cache_line). That is, when the packet length is not an integer multiple of 64 bytes, the tail portion less than 64 bytes will be written with 0, and the entire message will be padded to an integer multiple of 64 bytes. Therefore, a 4Kbyte empty buffer can store a maximum of 64 messages less than 64 bytes (4096B / 64B), and a maximum of 32 messages between 64 bytes and 128 bytes (4096B / 128B). And so on. It can be found that the smaller the message, the more buffer space is saved.

[0111] like Figure 5 As shown in the process of circle 2, the first message frame_packet0, whose length pkt_len0 is 46 bytes in hexadecimal, is written to buffer0, and the starting address is the starting address of buffer0 + offset address. Since it is the first message, the offset is 0x0000h, and its length is not an integer multiple of 64 bytes, so it will be padded to 64 bytes. Then write frame_packet1, this message length pkt_len1 = 0x32B, start writing from the offset address 0x0080h of buffer0, write to 0x00B2h, and padded cache_line to 0x00C0h. Write the subsequent frame_packet2, ..., frame_packetn-1 in sequence. The present application implements the aggregation of data packets (messages) in the same queue in the above manner. When the data packets in the same queue in the cache unit reach the number of PCIe bandwidth, multiple data packets in the same queue in the cache unit are written to the empty buffer area allocated by the host through a single PCIe write operation. Or when a preset timeout period is reached, the data packets in the same queue in the cache unit are written into an empty cache area allocated by the host through a single PCIe write operation.

[0112] In step S14 , the DPU generates a corresponding completion queue entry (CQE) for each data packet written into the empty buffer area.

[0113] Specifically, the DPU generates a CQE for each data packet written into an empty buffer, and populates each CQE with the packet's length, buffer identifier, enable bit, start packet identifier, and end packet identifier. The enable bit indicates whether to enable the sequence recognition function for multiple data packets. When this bit is valid, the start packet identifier and end packet identifier in the CQE are valid.

[0114] When the start packet flag is valid, it indicates that the data packet corresponding to the current CQE is the start data packet written to the current empty buffer area, and when the end packet flag is valid, it indicates that the data packet corresponding to the current CQE is the end data packet written to the current buffer area. In one specific embodiment, when the start packet flag is 1, the start packet flag is valid, and when the end packet flag is 1, the end packet flag is valid.

[0115] Further Figure 5 As shown, the DPU will send the valid information such as the address and length of each buffer area to the corresponding CQE, such as Figure 5 The process is shown in circle 3. To implement this process, the structure of CQE needs to be modified.

[0116] Specifically, the original CQE structure is as follows Figure 6 As shown in the figure, the buf_id field is the ID of the empty buffer allocated by the software, identifying a unique buffer area. In CQE, it represents the buf_id used by the DPU, that is, the buf_id of the backfilled data.

[0117] The pkt_len field indicates the length of the host buffer actually written by the DPU, that is, the effective length of the data sent.

[0118] The improved CQE structure is as follows Figure 7 Among them, the rx_sp_perf_en, frm_sop, and frm_eop fields are added to the original CQE to allow software to parse the chain relationship of the chained CQE. The meanings of the newly added fields are as follows:

[0119] rx_sp_perf_en: Considering protocol compatibility, this is an enable bit (feature bit), named rx_small_packet_perform_enable (rx_sp_perf_en for short). This enable bit is used to indicate whether the multi-packet sequence recognition function is enabled. It can accurately determine whether the corresponding function is enabled for each buffer, which is very flexible. Specifically, if this bit is 1, it indicates that the recognition function is enabled, and the start packet identifier and end packet identifier in the CQE are valid, which can be used to identify the order of each message information stored in the buffer. If this bit is 0, it indicates that the feature is disabled, and the start packet identifier and end packet identifier in the CQE are invalid.

[0120] frm_sop: The identifier of the first packet to start aggregation, valid when rx_sp_perf_en = 1. If this bit is 1, it means that the data packet corresponding to the CQE is the first packet information in the buffer. If this bit is 0, it means that the CQE is not the first packet information in the buffer.

[0121] frm_eop: The identifier of the last message in the aggregation, valid when rx_sp_perf_en = 1. If this bit is 1, it means that the CQE is the last message information of the buffer. If this bit is 0, it means that the CQE is not the last message information of the buffer.

[0122] Compared with the original CQE, the optimized CQE structure can realize a chain structure, that is, CQEs are interrelated, and their relationship represents the arrangement relationship of each small packet in buffer0. The chain structure can be realized through the above three new data structures.

[0123] The three fields (rx_sp_perf_en, frm_sop, frm_eop), each field occupies 1 bit, use Figure 6 The reserved fields in the 0th data word (DW) are used as the identifier for rx_sp_perf_en, bit 5 of the 3rd DW for frm_sop, and bit 21 of the 3rd DW for frm_eop. Figure 7 The bold part in .

[0124] In step S15 , the DPU uploads multiple CQEs to the host through a single PCIe write operation.

[0125] Specifically, the DPU aggregates multiple CQEs into a data block, and uploads the data block to the host through a single PCIe write operation.

[0126] Step S16: The host receives the CQE and obtains data of each packet according to the CQE.

[0127] Specifically, the host determines the start data packet and the end data packet in the buffer area according to the start packet identifier and the end packet identifier in each CQE, determines the order of each data packet, and reads the corresponding data block from the buffer area according to the length information of each small packet data to obtain the valid data of each data packet.

[0128] It can be understood that steps S14-S16 implement the process of software decomposing the data packet in the buffer area into multiple small packets according to the CQE information. Before optimizing the Jnet protocol, the corresponding relationship between the buf_id sent by the hardware and the CQE can only be one-to-one. After optimization, multiple CQEs can correspond to one buf_id, that is, a chained CQE structure, such as Figure 5 In the CQE content on the left, all the information carried in CQE_0 to CQE_N-1 is related to buffer_id0, so the backfilled buf_id is buffer_id0. The hardware fills the packet length information of each small packet into the pkt_len field of each CQE, such as Figure 5 The CQE content on the left side of the middle packet is displayed. In this way, the software can parse the length of each packet, and then calculate the starting address of the next packet in buffer0 according to the cache_line alignment principle. Then, according to each pkt_len, the length of the sent message is known, and the data of each packet can be accurately obtained in buffer0.

[0129] The software can read the frm_sop and frm_eop fields to parse the chained CQE structure. The main function of the chained structure is to facilitate the software to parse the number of small packets carried by buffer0, such as Figure 5 As shown in the CQE contents on the left, frm_sop = 1 and frm_eop = 0 in CQE_0 indicate that this is the CQE information for the first packet in the buffer with buf_id = 0. frm_sop = 0 and frm_eop = 0 in CQE_1, CQE_2, ..., and CQE_N-2 indicate that this is the CQE information for the middle packet in the buffer with buf_id = 0. As long as frm_eop = 1 is not read, it is necessary to continue reading. frm_sop = 0 and frm_eop = 1 in CQE_N-1 indicate that this is the CQE information for the last packet in the buffer with buf_id = 0. This completes the CQE chain for buf_id = 0, and subsequent CQEs belong to other buffer IDs.

[0130] The above functions require the hardware to set CQE rx_sp_perf_en to 1 to be effective, such as Figure 5 As shown in the CQE content on the left, this application can achieve backward compatibility without changing the main architecture of the chip. At the same time, in order to improve flexibility, the CQE of each uploaded transaction can be used to determine whether to enable this feature.

[0131] This application increases the effective data bandwidth of each PCIe by aggregating multiple small messages into a cache area according to the cache line alignment rule, and writing the aggregated messages to the empty cache area on the host side through a single PCIe write, thereby reducing the number of PCIe writes and improving the performance of small packet transmission.

[0132] As can be understood, this application effectively addresses the current Jnet protocol's limited performance when receiving small packets. It effectively reduces the number of PCIe write operations to host memory when processing small packets, improving PCIe bandwidth utilization and significantly boosting performance. The maximum PPS performance for small packets can reach over 600Mpps, a 2.5x performance improvement. The BPS also significantly increases to over 100Gbps, a 4x performance improvement. This improves the driver software's utilization of the spatiotemporal buffer for processing small packets, freeing up CPU resources for other tasks and boosting software performance by over 20% for small packet processing.

[0133] The solution of this application also adopts a chain CQE structure, adding 3 fields to the original CQE to identify the chain information of CQE. Based on the cache line alignment principle, the software and hardware can realize the transmission of multiple small packets of data in a single cache area; the second is to improve the efficiency of a single PCIe write of valid data, which is mainly achieved through the aggregation of CQE. This solution is backward compatible without changing the main architecture of the chip. While greatly reducing the number of PCIe write caches and CQEs, it can increase the amount of valid data in a single PCIe write, greatly improving the PCIe bandwidth utilization, and it has been proven that it can increase the performance of small packets in the receiving direction exponentially. At the same time, it is very friendly to software, greatly improving the cache utilization and saving host resources.

[0134] like Figure 9 FIG. 1 is a schematic diagram showing the main flow of an embodiment of a method for receiving and processing a data packet provided by the present application. In this embodiment, the method is applied to a data processing unit and includes the following steps:

[0135] Step S20: receiving network data packets, and buffering the data packets in a buffer unit of the data processing unit according to the queue.

[0136] Step S21: receiving a notification sent by the host, parsing the notification to obtain a receive queue entry address, and obtaining address information of an empty buffer area allocated by the host from the receive queue entry.

[0137] Step S22 , writing multiple data packets in the same queue in the cache unit into an empty cache area allocated by the host through a single PCIe write operation, wherein each data packet is padded according to the cache line alignment principle.

[0138] Specifically, when the number of data packets in the same queue in the cache unit reaches the amount of PCIe bandwidth, multiple data packets in the same queue in the cache unit are written into an empty cache area allocated by the host through a single PCIe write operation.

[0139] Furthermore, to prevent data packets from being constantly aggregated but unable to meet the PCIe bandwidth requirement, when a preset timeout is reached, a single PCIe write operation is used to write the aggregated data packets from the same queue in the cache unit to an empty buffer allocated by the host. At this point, the number of aggregated data packets is less than the PCIe bandwidth requirement.

[0140] Step S23 : Generate a corresponding completion queue entry CQE for each data packet written into the empty buffer area of ​​the host, and upload multiple CQEs to the host through a single PCIe write operation.

[0141] For more details, please refer to the aforementioned Figures 1 to 8 The description is not repeated here.

[0142] like Figure 12 , a schematic structural diagram of a data processing unit is shown, which includes a cache unit 20 , a notification parsing unit 21 , a write data processing unit 22 , a CQE allocation unit 23 and a CQE upload unit 24 .

[0143] Specifically, the cache unit 20 is used to cache the received network data packets according to the queue. The notification parsing unit 21 is used to read the notification from the host, parse the notification to obtain the address of the receiving queue entry, and obtain the address information of the empty cache area allocated by the host from the receiving queue entry. The write data processing unit 22 is used to write multiple data packets cached in the data processing unit into the empty cache area allocated by the host through a single PCIe write operation, and each data packet is padded according to the cache line alignment principle. The CQE allocation unit 23 is used to generate a corresponding completion queue entry CQE for each written data packet, and the CQE is used to identify the position and order of multiple data packets in the cache area. The CQE upload unit 24 is used to upload multiple CQEs to the host through a single PCIe write operation.

[0144] Specifically, the CQE allocation unit 23 generates a corresponding CQE for each small packet data written, and fills the length information of the small packet data, the buffer area identifier, the enable bit, the start packet identifier and the end packet identifier in each CQE; wherein, the enable bit is used to indicate whether the sequential recognition function of multiple data packets is turned on. When this bit is valid (1), the start packet identifier and the end packet identifier in the CQE are valid; when the start packet identifier is valid (1), it indicates that the data packet written by the current CQE is the start data packet of the current empty buffer area; when the end packet identifier is valid (1), it indicates that the data packet corresponding to the current CQE is the end data packet written to the current buffer area.

[0145] Specifically, the CQE uploading unit 24 is configured to aggregate multiple CQEs into a data block, and upload the data block to the host through a single PCIe write operation.

[0146] For more details, please refer to and combine the above Figures 1 to 8 The description is not repeated here.

[0147] like Figure 10 FIG. 1 shows a schematic diagram of a structure of an embodiment of a data packet receiving and processing system provided by the present application; and FIG. Figure 11 and Figure 12 As shown, in this embodiment, the system at least includes:

[0148] Host 1 is configured to run driver software, allocate an empty buffer area for each receive queue entry, generate the receive queue entry, and send a notification to the data processing unit DPU.

[0149] The data processing unit DPU 2 is used to receive network data packets and cache the data packets in the cache unit of the data processing unit according to the queue; receive a notification sent by the host, parse the notification to obtain the address of the receiving queue entry, and obtain the address information of the empty buffer area allocated by the host from the receiving queue entry; write multiple data packets of the same queue in the cache unit to the empty buffer area allocated by the host through a single PCIe write operation, wherein each data packet is padded according to the cache line alignment principle; generate a corresponding completion queue entry CQE for each data packet written to the host buffer area; and upload multiple CQEs to the host through a single PCIe write operation.

[0150] The host 1 is further configured to receive a CQE sent by the data processing unit, obtain the position and order of multiple small packet data in the buffer area according to the CQE, and obtain the data of each small packet.

[0151] In a specific example, such as Figure 11 As shown, the host 1 further includes:

[0152] The receiving queue entry generating unit 10 is configured to allocate an empty buffer area to each receiving queue entry and fill the starting address and length information of the buffer area into the receiving queue entry;

[0153] a notification sending unit 11, configured to send a notification to the DPU, wherein the notification includes a queue identifier and an entry index;

[0154] The data splitting and reading unit 12 is used to receive the CQE and buffer area identifier uploaded by the DPU, determine the starting data packet and the ending data packet in the buffer area according to the starting packet identifier and the ending packet identifier in each CQE, and determine the order of each data packet; read the corresponding data block from the buffer area according to the length information of the data identified in each CQE, and obtain the valid data of each data packet.

[0155] Accordingly, as another aspect of the present application, a computer-readable storage medium is provided, which stores a computer program, and when the program is executed by a processor, the following is achieved: Figures 1 to 9 For more details, please refer to and combine the above Figures 1 to 9 The description is not repeated here.

[0156] Accordingly, as another aspect of the present application, a network card is provided, wherein a network card is deployed as follows Figure 10 For more details, please refer to and combine the above Figure 10 The description is not repeated here.

[0157] Implementing this embodiment has the following beneficial effects:

[0158] The present application provides a data packet receiving and processing method, a data processing unit, a system, a storage medium, and a network card. The present application aggregates multiple small packets into a cache area according to cache line alignment rules, and writes the aggregated packets to an empty cache area on the host side through a single PCIe write, thereby increasing the effective data bandwidth of each PCIe transmission, thereby reducing the number of PCIe writes and improving the performance of small packet transmission.

[0159] In addition, by improving the completion queue entry structure of the receive queue and adopting a chained CQE structure, three fields are added to the original CQE to identify chained information. The chained CQE structure enables the software to more efficiently parse and process multiple small packets in the buffer area. CQE aggregation technology is used to improve the efficiency of a single PCIe write of valid data. This technology effectively reduces the number of PCIe writes to the buffer area and CQE, increases the amount of valid data in a single PCIe write, and greatly improves PCIe bandwidth utilization. It has been verified that this application can significantly improve the performance of small packets in the receive direction.

[0160] Without changing the core chip architecture, this application achieves efficient utilization of PCIe bandwidth and improves the driver software's utilization of empty buffers when processing small packets. Software no longer needs to maintain multiple buffer identifiers and fragmented address spaces; instead, it can transmit multiple small packets using a single, continuous block of space, further improving software efficiency and reducing communication costs.

[0161] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0162] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0163] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data packet receiving and processing method, applied to a data processing unit, characterized in that: The following steps are involved: Receive network data packets and cache the data packets in the cache unit of the data processing unit according to the queue; receiving a notification sent by the host, parsing the notification to obtain a receive queue entry address, and obtaining address information of an empty buffer area allocated by the host from the receive queue entry; Multiple packets from the same queue in the cache unit are written to an empty cache area allocated by the host through a single PCIe write operation, where each packet is padded according to the cache line alignment principle; Generate a corresponding completion queue entry CQE for each data packet written into the empty buffer area of ​​the host; The plurality of CQEs are uploaded to the host through a single PCIe write operation.

2. The method according to claim 1, wherein The method of writing multiple data packets in the same queue in the cache unit into an empty cache area allocated by the host through a single PCIe write operation includes: When the length of the data packets in the same queue in the cache unit reaches the length of the PCIe bandwidth, multiple data packets in the same queue in the cache unit are written into the empty cache area allocated by the host through a single PCIe write operation; Or when a preset timeout period is reached, the data packets in the same queue in the cache unit are written into an empty cache area allocated by the host through a single PCIe write operation.

3. The method according to claim 1, wherein Generating a corresponding completion queue entry CQE for each data packet written into an empty buffer area of ​​the host further includes: The data processing unit generates a CQE for each data packet and fills the enable bit information, start packet identifier and end packet identifier of the data packet in each CQE; The enable bit information is used to indicate whether to enable the sequence recognition function of multiple data packets. When the enable bit information is valid, the start packet identifier and end packet identifier in the CQE are valid; When the start packet identifier is valid, it indicates that the data packet corresponding to the current CQE is the start data packet written into the current empty buffer area; When the end packet identifier is valid, it indicates that the data packet corresponding to the current CQE is the end data packet written into the current buffer area.

4. The method according to claim 3, wherein The method further includes uploading the plurality of CQEs to the host through a single PCIe write operation: Multiple CQEs are aggregated into one data block, and the data block is uploaded to the host through a single PCIe write operation.

5. A method for receiving and processing a data packet, characterized in that: The following steps are involved: The host allocates an empty buffer area for each receive queue entry, generates the receive queue entry, and sends a notification to the data processing unit; The data processing unit receives network data packets and caches the data packets in a cache unit of the data processing unit according to the queue; The data processing unit reads a notification from the host, parses the notification to obtain a receive queue entry address, and obtains address information of an empty buffer area allocated by the host from the receive queue entry; The data processing unit writes multiple data packets of the same queue in the cache unit into the empty cache area allocated by the host through a single PCIe write operation, and each data packet is padded according to the cache line alignment principle; The data processing unit generates a corresponding completion queue entry CQE for each data packet written into the empty buffer area; The data processing unit uploads multiple CQEs to the host through a single PCIe write operation The host receives the multiple CQEs and obtains corresponding data packets according to the multiple CQEs.

6. The method according to claim 5, wherein The data processing unit generates a corresponding completion queue entry CQE for each data packet written into the empty buffer area, further comprising: The data processing unit generates a CQE for each data packet, and each CQE is filled with an enable bit, a start packet identifier, and an end packet identifier; The enable bit information is used to indicate whether to enable the sequence recognition function of multiple data packets. When the enable bit information is valid, the start packet identifier and end packet identifier in the CQE are valid; When the start packet identifier is valid, it indicates that the data packet corresponding to the current CQE is the start data packet written into the current empty buffer area; When the end packet identifier is valid, it indicates that the data packet corresponding to the current CQE is the end data packet written into the current buffer area.

7. The method according to claim 6, wherein The host receives the multiple CQEs and obtains corresponding data packets according to the multiple CQEs, further comprising: The host determines the start data packet and the end data packet in the buffer area according to the start packet identifier and the end packet identifier in each CQE, and determines the order of each data packet; The host reads the corresponding data block from the buffer area according to the length information of the data identified in each CQE to obtain the corresponding data packet.

8. A data processing unit, characterized in that: include: A cache unit, configured to cache received network data packets according to queues when receiving the data packets; a notification parsing unit, configured to receive a notification sent by a host, parse the notification to obtain a receive queue entry address, and obtain address information of an empty buffer area allocated by the host from the receive queue entry; A write data processing unit, configured to write multiple data packets of the same queue in the cache unit into an empty cache area allocated by the host through a single PCIe write operation, wherein each data packet is padded according to a cache line alignment principle; A CQE allocation unit is configured to generate a corresponding completion queue entry CQE for each data packet written into the host buffer area; The CQE upload unit is used to upload multiple CQEs to the host through a single PCIe write operation.

9. The data processing unit according to claim 8, characterized in that The CQE allocation unit is further used to generate a CQE for each data packet, and fill the enable bit information, start packet identifier and end packet identifier of the data packet in each CQE; The enable bit information is used to indicate whether to enable the sequence recognition function of multiple data packets. When the enable bit information is valid, the start packet identifier and end packet identifier in the CQE are valid; When the start packet identifier is valid, it indicates that the data packet corresponding to the current CQE is the start data packet written into the current empty buffer area; When the end packet identifier is valid, it indicates that the data packet corresponding to the current CQE is the end data packet written into the current buffer area.

10. A data packet receiving and processing system, characterized in that: comprising a host and a data processing unit as claimed in claim 8 or 9: The host is configured to allocate an empty buffer area to each receive queue entry, generate the receive queue entry, and send a notification to the data processing unit; The host is further configured to receive a plurality of CQEs sent by the data processing unit, and obtain corresponding data packets according to the plurality of CQEs.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

12. A network card, characterized in that: A data processing unit as claimed in claim 8 is deployed therein.

Images