Data transmission process processing method and apparatus, device, and storage medium
By allocating a contiguous chain of memory descriptors for the Virtio driver of the smart network card and optimizing DMA requests, the problem of low PCIe bandwidth utilization caused by the Virtio descriptor length limitation is solved, thereby improving the network performance of the smart network card.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- CHINA TELECOM CLOUD TECH CO LTD
- Filing Date
- 2025-11-25
- Publication Date
- 2026-06-11
AI Technical Summary
The short length of the Virtio descriptor results in low PCIe bandwidth utilization, which affects the performance of smart network cards.
During the sending or receiving of messages, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, including multiple memory descriptors in contiguous memory. After obtaining the descriptor with the smallest memory address through a DMA request, the number of contiguous descriptors is determined for batch acquisition.
It significantly improves the bandwidth utilization of PCIe and enhances the efficiency and performance of smart network interface cards in sending and receiving packets on the network.
Smart Images
Figure CN2025137513_11062026_PF_FP_ABST
Abstract
Description
A method, apparatus, device, and storage medium for data transmission.
[0001] Cross-reference to related applications
[0002] This application claims priority to Chinese Patent Application No. 202411782574.9, filed on December 5, 2024, entitled "A method, apparatus, device and storage medium for processing a data transmission process", the entire contents of which are incorporated herein by reference. Technical Field
[0003] This application relates to the technical field of data processing, and more specifically, to a method, apparatus, device, and storage medium for data transmission. Background Technology
[0004] A Smart NIC is a high-performance network interface card (NIC) specifically designed for network data processing. It employs a custom chip, a high-speed network interface, and robust software support, providing faster, more secure, and more reliable network connectivity and data transmission services for data centers and enterprise networks. Furthermore, Smart NICs (SNICs) possess rich hardware offloading capabilities and can provide Virtio (device abstraction layer in a paravirtualization hypervisor) hardware acceleration.
[0005] The lowest level of a smart network card is a PCIe (peripheral component interconnect express, high-speed serial computer expansion bus standard) device, which uses the standard PCIe protocol for communication. Above the PCIe protocol is the Virtio protocol, and the smart network card sends and receives network packets according to the Virtio protocol format.
[0006] Virtio is a semi-virtualization technology that uses shared memory to enable data transfer between the virtual machine and the host machine. The hardware of the smart network card uses the Virtio protocol for hardware-software communication. During the sending and receiving of data packets, the hardware of the smart network card first reads memory through DMA (Direct Memory Access) to obtain the memory descriptor, then parses the memory information represented in the memory descriptor, and finally reads / writes data packets to the memory address specified by the memory descriptor through DMA.
[0007] Because a descriptor in a Virtio-based smart NIC is relatively short, typically 16 bytes, while the maximum payload size (Max_Payload_Size) for PCIe is usually 256 bytes, the length of a DMA descriptor read request (e.g., 16 bytes) is much shorter than the 256-byte Max_Payload_Size of a PCIe TLP (transaction layer packet). This results in a lower payload for the PCIe TLP. In the case of descriptor chaining, only one short descriptor is retrieved by DMA at a time, leading to reduced PCIe bandwidth utilization and impacting the performance of the smart NIC. Summary of the Invention
[0008] This application provides a data transmission processing method, apparatus, device, and storage medium, aiming to improve the efficiency of smart network interface cards (NICs) in sending and receiving messages on the network, thereby improving the performance of smart NICs.
[0009] In a first aspect, embodiments of this application provide a method for processing data transmission, applied to a smart network interface card (NIC), the method comprising:
[0010] During the process of sending or receiving a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message. The target memory descriptor chain includes multiple memory descriptors that are contiguous in memory. The memory descriptor with the smallest memory address among the multiple contiguous memory descriptors includes a number of contiguous memory descriptors.
[0011] When the smart network card hardware obtains the memory descriptor with the smallest memory address among multiple consecutive memory descriptors through the first DMA request, it determines the number of multiple consecutive memory descriptors based on the memory descriptor and determines the second DMA request. The second DMA request is used to obtain the remaining multiple consecutive memory descriptors.
[0012] In some embodiments of this application, the Virtio driver of the smart network interface card allocates a target memory descriptor chain for any packet, including:
[0013] When the Virtio driver of a smart network card allocates a target memory descriptor chain containing multiple contiguous memory descriptors for any packet, the target byte of the memory descriptor with the smallest memory address among the contiguous memory descriptors is written to the remaining contiguous memory descriptors.
[0014] In some embodiments of this application, after the smart network card hardware obtains the memory descriptor with the smallest memory address among a plurality of contiguous memory descriptors through a first DMA request, it determines the number of the plurality of contiguous memory descriptors based on the memory descriptor and determines a second DMA request, including:
[0015] When the smart network card hardware obtains the memory descriptor with the smallest memory address among multiple consecutive memory descriptors through the first DMA request, it parses the target byte in the memory descriptor to obtain the number of the remaining multiple consecutive memory descriptors.
[0016] The second DMA request is determined based on the number of remaining contiguous memory descriptors.
[0017] In some embodiments of this application, the number of times the target byte of the memory descriptor with the smallest memory address among a plurality of contiguous memory descriptors is written to the remaining plurality of contiguous memory descriptors includes:
[0018] Take the high 12 bits of the flag field of the memory descriptor with the smallest memory address among multiple contiguous memory descriptors as the target byte, and write the number of the remaining contiguous memory descriptors into the high 12 bits of the flag field of that memory descriptor.
[0019] In some embodiments of this application, the Virtio driver of the smart network interface card allocates a target memory descriptor chain for any packet, including:
[0020] During the message transmission process, the Virtio driver of the smart network card determines the number of memory descriptors corresponding to the message to be transmitted based on the size of the message to be transmitted.
[0021] According to the descriptor allocation strategy, a target memory descriptor chain is allocated in the sending descriptor pool. The number of memory descriptors included in the target memory descriptor chain is the number of memory descriptors corresponding to the message to be sent.
[0022] The descriptor allocation strategy is used to maximize the number of contiguous memory descriptors when allocating multiple memory descriptors.
[0023] In some embodiments of this application, the Virtio driver of the smart network interface card allocates a target memory descriptor chain for any packet, including:
[0024] Before receiving a message, the Virtio driver of the smart NIC determines the maximum memory descriptor chain in the receive descriptor pool based on the depth of the receive queue and the descriptor allocation strategy.
[0025] Descriptor allocation strategies are used to maximize the number of contiguous memory descriptors when allocating multiple memory descriptors.
[0026] In some embodiments of this application, the data transmission process further includes the following steps during message reception:
[0027] Based on the size of the message to be received, the target memory descriptor chain corresponding to the message to be received is determined in the largest memory descriptor chain. The target memory descriptor chain corresponding to the message to be received includes multiple memory descriptors that are contiguous in memory.
[0028] Secondly, embodiments of this application provide a data transmission processing apparatus applied to a smart network interface card (NIC), the apparatus comprising:
[0029] The descriptor determination module is used by the Virtio driver of the smart network card to allocate a target memory descriptor chain for any message during the sending or receiving of messages. The target memory descriptor chain includes multiple memory descriptors that are contiguous in memory. The memory descriptor with the smallest memory address among the multiple memory descriptors that are contiguous in memory includes the number of multiple memory descriptors that are contiguous in memory.
[0030] The request module is used to determine the number of consecutive memory descriptors based on the memory descriptor after the smart network card hardware obtains the memory descriptor with the smallest memory address among multiple consecutive memory descriptors through the first DMA request, and then determine the second DMA request. The second DMA request is used to obtain the remaining consecutive memory descriptors.
[0031] Thirdly, embodiments of this application provide a computer device, including: at least one processor and a memory, the memory storing a computer program executable on the processor, wherein the processor executes the computer program to perform the data transmission process processing method of the first aspect of the embodiment.
[0032] Fourthly, embodiments of this application provide a non-volatile readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it executes the data transmission process processing method of the first aspect of the embodiment. Beneficial effects:
[0033] During the sending or receiving of a message, the Virtio driver of the smart network interface card (NIC) allocates a target memory descriptor chain for any message. The target memory descriptor chain includes multiple contiguous memory descriptors. The memory descriptor with the smallest memory address among the contiguous memory descriptors contains the number of contiguous memory descriptors. When the smart NIC hardware obtains the memory descriptor with the smallest memory address among the contiguous memory descriptors through a first DMA request, it determines the number of contiguous memory descriptors based on that memory descriptor and determines a second DMA request. The second DMA request is used to obtain the remaining contiguous memory descriptors.
[0034] In the data transmission process processing method provided in this application embodiment, during the process of sending or receiving a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message. The target memory descriptor chain includes multiple memory descriptors that are contiguous in memory, and the memory descriptor with the smallest memory address among the multiple memory descriptors that are contiguous in memory includes the number of memory descriptors that are contiguous in memory.
[0035] Furthermore, when the smart network card's hardware acquires memory descriptors via DMA, for multiple contiguous memory descriptors, after acquiring the memory descriptor with the smallest memory address through the first DMA request, the number of contiguous memory descriptors can be determined based on this memory descriptor. This allows the second DMA request to acquire the remaining contiguous memory descriptors at once, significantly reducing the number of DMA requests. This fully utilizes the Max_payload_size feature of PCIe TLP, improving PCIe bandwidth utilization and thus enhancing the overall efficiency of the smart network card in network packet transmission and reception, thereby optimizing the smart network card's performance. Attached Figure Description
[0036] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0037] Figure 1 is a flowchart of the data transmission method according to an embodiment of this application;
[0038] Figure 2 is a schematic diagram of memory descriptor allocation during data transmission according to an embodiment of this application;
[0039] Figure 3 is a schematic diagram of memory descriptor allocation during data reception according to an embodiment of this application;
[0040] Figure 4 is a schematic diagram of obtaining a memory descriptor according to an embodiment of this application;
[0041] Figure 5 is a functional block diagram of a data transmission processing apparatus according to an embodiment of this application;
[0042] Figure 6 is a schematic diagram of a computer device provided in an embodiment of this application;
[0043] Figure 7 is a schematic diagram of a non-volatile readable storage medium proposed in an embodiment of this application. Detailed Implementation
[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0045] DMA: Direct Memory Access, which allows direct access to memory without CPU intervention;
[0046] PCIe: Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard;
[0047] Tlp: Transaction Layer Packets, plays a very important role in PCIe communication. Tlp is used to transmit data and control information between devices. They are the basic information transmission unit of PCIe.
[0048] Virtio: An abstraction layer above the device in a semi-virtualized hypervisor.
[0049] Tx: Sending direction;
[0050] Rx: Receive direction;
[0051] Descriptor Pool: In Vulkan (a cross-platform graphics and computing application programming interface), the descriptor pool is a resource pool that manages descriptor sets for efficient allocation and recycling. A descriptor set is a data structure in Vulkan used to bind resources (such as buffers, textures, etc.) to shaders. The descriptor pool ensures that the allocation and recycling process of descriptor sets is efficient and controllable.
[0052] A Smart NIC is a high-performance network interface card (NIC) specifically designed for network data processing. It employs custom chips, high-speed network interfaces, and robust software support to provide faster, more secure, and more reliable network connectivity and data transmission services for data centers and enterprise networks. It offers many performance advantages and functional enhancements that traditional NICs cannot achieve, and can also cope with increasingly complex network applications and business needs.
[0053] High performance: Smart network interface cards (NICs) typically use dedicated chips and high-speed network interfaces, enabling ultra-high data processing rates, throughput, and flow control efficiency, thereby improving network performance and response speed.
[0054] Safe and reliable: Smart network interface cards (NICs) can not only accelerate data transmission, but also provide better security features and protection mechanisms, such as data encryption, access control, DDoS (Distributed Denial of Service) defense, firewalls, etc., thereby enhancing network security and stability.
[0055] Supports distributed storage and computing: Smart NICs can also be combined with distributed storage, computing and other technologies to achieve more efficient data processing and analysis, while reducing network latency and load pressure, thereby improving overall performance and efficiency.
[0056] Smart NICs (Smart Network Interface Cards) possess rich hardware offloading capabilities, providing Virtio hardware acceleration and security-related data plane offloading. Unlike traditional NICs, smart NICs offload cloud-based network forwarding functions, providing high-performance network forwarding capabilities and freeing up CPU (Central Processing Unit) general-purpose computing resources, including transmission and storage protocols. At the same time, they provide network programmability, enabling customization of network forwarding logic and ensuring functional flexibility.
[0057] Descriptors play a crucial role in memory management. They not only identify the location and size of memory regions but also provide access control and permission management for these regions. Through descriptors, the operating system can effectively manage the address spaces of processes, ensuring that each process has its own independent memory space, thereby preventing interference between processes. Descriptors play a key role in memory management, ensuring system stability and security by identifying and managing memory regions.
[0058] Virtio is a paravirtualization technology that uses shared memory to enable data transfer between the virtual machine and the host machine. The smart network interface card (NIC) hardware uses the virtio protocol for hardware-software communication. During the sending and receiving of data packets, the smart NIC hardware reads memory through DMA to obtain memory descriptors, then parses the memory information represented in the memory descriptors, and finally reads / writes data packets to the memory address specified by the memory descriptors through DMA.
[0059] A memory descriptor corresponds to a memory region. During the sending and receiving of data packets, large data packets usually need to be fragmented, such as splitting a large packet into multiple smaller packets. Therefore, multiple contiguous or non-contiguous memory addresses may be involved. Thus, multiple descriptors are usually needed to form a chain, that is, one descriptor can point to the next descriptor. In addition, due to the scatter aggregation characteristics of SGL (Scatter Gather List), multiple descriptors are also needed to form a chain.
[0060] Because a Virtio descriptor is relatively short, typically 16 bytes, while the PCIe Max_Payload_Size is usually 256 bytes, the DMA read descriptor request length (e.g., 16 bytes) is less than the PCIe TLP message's Max_Payload_Size length of 256 bytes. This results in a lower effective payload for the PCIe TLP. In the case of descriptor chaining, only one descriptor is DMA'd at a time, and the DMA request length is small, leading to reduced PCIe bandwidth utilization and impacting performance.
[0061] Therefore, in order to improve the bandwidth utilization of PCIe and thus improve the performance of Virtio-based network cards in receiving or sending packets on the network, this application provides a data transmission method.
[0062] Referring to Figure 1, a flowchart of a data transmission method according to an embodiment of this application is shown. The method can be applied to a smart network interface card (NIC), and the method may specifically include the following steps:
[0063] S101: During the sending or receiving of a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message. The target memory descriptor chain includes multiple memory descriptors that are contiguous in memory. The memory descriptor with the smallest memory address among the multiple contiguous memory descriptors includes the number of contiguous memory descriptors.
[0064] Memory refers to the memory of the host where the smart network interface card (NIC) is deployed. In the Virtio framework, a memory descriptor is used to describe the location of data in memory. The memory descriptor of the Virtio driver of the smart NIC is stored in a memory region, and the memory descriptors are generally of the same size.
[0065] The memory region that stores the memory descriptors of the Virtio driver is called the Descriptor Pool, which includes the transmit descriptor pool (Tx) and the receive descriptor pool (Rx). The size of the Descriptor Pool is related to the specifications of the smart network interface card.
[0066] For example, smart network interface card A can be allocated X1 memory descriptors, and smart network interface card B can be allocated X2 memory descriptors.
[0067] Memory descriptors are used to describe memory information of memory regions. Specifically, memory descriptors use multiple fields to describe the memory information of a memory region. For example, the addr (address) field of a memory descriptor indicates the starting location of the physical address space of a memory region on the host; the len (length) field usually describes the read / write attributes and the length of data read / written for that memory region; and the flag field is used to indicate the characteristics of the descriptor itself. For example, VRING_DESC_F_NEXT indicates whether the next (next) field of the memory descriptor is valid, and VRING_DESC_F_INDIRECT indicates whether the descriptor points to an indirect descriptor table.
[0068] In actual implementation, the data packets being read and written may be located in different locations in memory; that is, a single piece of data may typically involve multiple consecutive or non-consecutive memory addresses.
[0069] For example, when data A needs to be acquired for transmission, if part of data A is in memory address ab and another part of data A is in memory address bc, then the data is not in contiguous memory addresses, but rather in non-contiguous memory addresses.
[0070] The driver for a smart network card needs to allocate multiple memory descriptors to correspond to the location of data packets in memory. Then, the hardware of the smart network card first obtains the memory descriptor corresponding to the data packet in memory through a DMA request, and then retrieves the data content of the data packet from the corresponding memory address based on the memory descriptor; while a typical DMA request reads only one memory descriptor at a time.
[0071] All memory descriptors for data A are organized into a memory descriptor chain. For example, the next field in memory descriptor 1 points to the next memory descriptor 2. Generally, after DMA memory descriptor 1, the next field in memory descriptor 1 is resolved, and then the next memory descriptor 2 pointed to by DMA memory descriptor 1 is accessed.
[0072] In the method provided in this application embodiment, during the process of sending or receiving a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message. The target memory descriptor chain includes multiple memory descriptors. For example, during the process of the smart network card sending a message, the multiple memory descriptors comprehensively describe the location of the message to be sent in memory, etc. After obtaining the multiple memory descriptors, the message data can be obtained from the corresponding memory and then sent.
[0073] Specifically, when the Virtio driver of the smart network card allocates the target memory descriptor chain, the target memory descriptor chain includes multiple memory descriptors that are contiguous in memory. That is, the Virtio driver of the smart network card can allocate multiple memory descriptors with contiguous memory addresses as much as possible, and the memory descriptor with the smallest memory address among the multiple contiguous memory descriptors includes the number of multiple contiguous memory descriptors.
[0074] In one feasible implementation, the process by which the Virtio driver of the smart network interface card allocates a target memory descriptor chain for any packet during packet transmission includes the following steps:
[0075] A1: The Virtio driver for the smart network card determines the number of memory descriptors corresponding to the packet to be sent based on the size of the packet to be sent.
[0076] First, when sending a message, the Virtio driver of the smart network card determines the number of memory descriptors that need to be allocated based on the size of the message to be sent. For example, the maximum data length that each memory descriptor can describe is limited. When the message to be sent is large, multiple memory descriptors need to be allocated to describe the location of the message in memory.
[0077] In actual implementation, the number of memory descriptors can be determined based on the size of the message to be sent and its location in memory. For example, if the maximum data length described by a memory descriptor is A, and the size of a message to be sent is less than A, one memory descriptor is sufficient. However, if the first part of the message to be sent is located in the first memory location and the second part is located in the second memory location, then two memory descriptors can be allocated. The first memory descriptor is used to describe the first memory location, and the second memory descriptor is used to describe the second memory location.
[0078] A2: According to the descriptor allocation strategy, allocate a target memory descriptor chain in the sending descriptor pool. The number of memory descriptors included in the target memory descriptor chain is the number of memory descriptors corresponding to the message to be sent.
[0079] In this method, when the message to be sent corresponds to multiple memory descriptors, the Virtio driver of the smart network card executes a descriptor allocation strategy in the transmit descriptor pool, i.e., the Tx descriptor pool. The descriptor allocation strategy is configured to maximize the number of contiguous memory descriptors when allocating multiple memory descriptors.
[0080] Referring to Figure 2, a schematic diagram of memory descriptor allocation during data transmission provided in this application embodiment is shown. Scenario 1: Assuming that the message to be sent requires 4 memory descriptors, all memory descriptors in the Tx descriptor pool are in an allocable state, that is, when the descriptor is not currently in use, the Virtio driver of the smart network card executes the descriptor allocation strategy, maximizes the number of multiple consecutive memory descriptors, and uses memory descriptor 1, memory descriptor 2, memory descriptor 3 and memory descriptor 4 as the target memory descriptor chain of the message to be sent. The memory descriptors 1-4 are stored in consecutive memory locations, that is, these 4 memory descriptors are memory descriptors with consecutive memory addresses.
[0081] Scenario 2: Assuming the packet to be sent requires 4 memory descriptors, memory descriptors 1, 3, and 7-n in the Tx descriptor pool are all already allocated, meaning these descriptors are currently in use. In this case, the Virtio driver of the smart NIC executes a descriptor allocation strategy, maximizing the number of contiguous memory descriptors. Then, memory descriptors 4, 5, and 6 are allocated as 3 contiguous memory descriptors, while memory descriptor 2 is allocated as a separate memory address. Ultimately, the target memory descriptor chain for the packet to be sent includes memory descriptor 2 and memory descriptors 4-6.
[0082] A3: When the Virtio driver of a smart network card allocates multiple contiguous memory descriptors in the target memory descriptor chain for any packet, the number of the remaining contiguous memory descriptors whose target bytes are written to the memory descriptor with the smallest memory address among the multiple contiguous memory descriptors.
[0083] Specifically, the high 12 bits of the flag field of the memory descriptor with the smallest memory address among multiple contiguous memory descriptors are used as the target byte, and the number of the remaining contiguous memory descriptors is written into the high 12 bits of the flag field of that memory descriptor.
[0084] In actual implementation, when the hardware of a smart network card performs DMA on multiple memory descriptors, it usually performs DMA sequentially according to the memory address order of the memory descriptors. Therefore, for multiple contiguous memory descriptors, only the number of the remaining contiguous memory descriptors can be written to the target byte of the target memory descriptor. The target memory descriptor refers to the memory descriptor that is DMAped first among the multiple contiguous memory descriptors.
[0085] For example, taking scenario one in Figure 2 as an example, the target memory descriptor chain corresponding to the message to be sent includes multiple memory descriptors with consecutive memory addresses, namely memory descriptors 1-4. At this time, the number of the remaining consecutive memory descriptors is written into the target byte of memory descriptor 1, that is, 3 is written into the high 12 bits of the flag field of memory descriptor 1. The value 3 in the high 12 bits of the flag field represents the number of subsequent consecutive memory descriptors, that is, after memory descriptor 1, there are 3 consecutive memory addresses including memory descriptors 2, 3, and 4.
[0086] For example, taking scenario two in Figure 2 as an example, the target memory descriptor chain corresponding to the message to be sent includes three memory descriptors with consecutive memory addresses, namely memory descriptors 4, 5, and 6, and a memory descriptor 2 with a separate memory address. Since there are no other memory descriptors with consecutive addresses to memory descriptor 2, the high 12 bits of the flag field in memory descriptor 2 do not contain the number of the remaining consecutive memory descriptors. For example, 0 can be written to the high 12 bits of the flag field. Then, among the consecutive memory descriptors 4, 5, and 6, the number of the remaining consecutive memory descriptors is written to the high 12 bits of the flag field of memory descriptor 4, which is the first memory descriptor to be DMAped. That is, 2 is written to the high 12 bits of the flag field of memory descriptor 4. The value 2 in the high 12 bits of the flag field represents the number of subsequent consecutive memory descriptors, that is, after memory descriptor 4, there are two consecutive memory descriptors (memory descriptors 5 and 6).
[0087] In one feasible implementation, the process by which the Virtio driver of the smart network interface card allocates a target memory descriptor chain for any packet during packet transmission includes the following steps:
[0088] B1: Before receiving packets, the Virtio driver of the smart NIC determines the maximum memory descriptor chain in the receive descriptor pool based on the depth of the receive queue and the descriptor allocation strategy.
[0089] Before receiving a packet, the size of the subsequent received packets is unknown. However, the smart network card may receive multiple packets at once. Therefore, based on the depth of the receive queue, the Virtio driver of the smart network card executes a descriptor allocation strategy in the receive descriptor pool, i.e., the Rx descriptor pool, to allocate enough memory descriptors to form the maximum memory descriptor chain, which is the number of memory descriptors required when the receive queue is full of received packets, and is used when a packet to be received is received.
[0090] The descriptor allocation strategy is used to maximize the number of contiguous memory descriptors when allocating multiple memory descriptors. Therefore, the Virtio driver of the smart network card allocates as many contiguous memory descriptors as possible in the Rx descriptor pool as possible as the target memory descriptor chain when receiving packets.
[0091] B2: Based on the size of the message to be received, determine the target memory descriptor chain corresponding to the message to be received in the largest memory descriptor chain. The target memory descriptor chain corresponding to the message to be received includes multiple memory descriptors that are contiguous in memory.
[0092] Upon receiving a message to be received, the target memory descriptor chain corresponding to the message is determined in the target memory descriptor chain based on the size of the message to be received. For example, although the Virtio driver of the smart network card has allocated the maximum memory descriptor chain for the maximum number of messages to be received based on the smart network card's receive queue before the message is received, the actual size of the message to be received may only require a portion of the memory descriptor chain to be used as the target memory descriptor chain.
[0093] In determining the target memory descriptor chain within the maximum memory descriptor chain, the number of multiple memory descriptors with consecutive memory addresses is still maximized. The number of the remaining multiple consecutive memory descriptors is written into the target byte of the memory descriptor that is the first to be DMAped among the multiple memory descriptors with consecutive memory addresses. For example, the high 12 bits of the flag field of the memory descriptor can be used as the target byte.
[0094] Referring to Figure 3, a schematic diagram of memory descriptor allocation during data reception provided in an embodiment of this application is shown. Before data reception, the unallocated memory descriptors in the receive descriptor pool Rx descriptor pool include memory descriptors 2-3, memory descriptors 5-9 and memory descriptor 11. Memory descriptors 2-3, memory descriptors 5-9 and memory descriptor 11 are determined as the maximum memory descriptor chain.
[0095] Assuming that after receiving the message A to be received, based on the size of the message A, it is determined that the message A requires 4 memory descriptors. Then, the maximum number of memory descriptors with consecutive memory addresses is selected from the largest memory descriptor chain, that is, 4 memory descriptors with consecutive memory addresses are selected. That is, memory descriptors 5-8 are selected as the target memory descriptor chain, and 3 is written into the high 12 bits of the flag field of memory descriptor 5, indicating that after memory descriptor 5, there are 3 memory descriptors (memory descriptors 6, 7, and 8) with consecutive memory addresses.
[0096] S102: After the smart network card hardware obtains the memory descriptor with the smallest memory address among multiple consecutive memory descriptors through the first DMA request, it determines the number of multiple consecutive memory descriptors based on the memory descriptor and determines the second DMA request. The second DMA request is used to obtain the remaining multiple consecutive memory descriptors.
[0097] Specifically, for multiple memory descriptors with consecutive memory addresses, after the smart network card hardware obtains the memory descriptor with the smallest memory address among the multiple consecutive memory descriptors through the first DMA request, it parses the target byte in the memory descriptor to obtain the number of the remaining multiple consecutive memory descriptors; based on the number of the remaining multiple consecutive memory descriptors, it determines the second DMA request.
[0098] When the smart network interface card (NIC) hardware requests memory descriptors in memory via DMA, after obtaining any memory descriptor, it parses the target byte in the memory descriptor to determine whether the memory descriptor contains the number of consecutive remaining memory descriptors. If the target byte is 0, it indicates that the memory descriptor adjacent to the memory address of the memory descriptor is not in the target memory descriptor chain. If the target byte is a non-zero value, it indicates that the memory descriptor adjacent to the memory address of the memory descriptor is in the target memory descriptor chain. The DMA request can be combined according to the number of consecutive remaining memory descriptors to obtain the consecutive remaining memory descriptors at once, instead of performing DMA on each consecutive remaining memory descriptor individually.
[0099] Referring to Figure 4, a schematic diagram of obtaining memory descriptors provided in an embodiment of this application is shown. It is assumed that the target memory descriptor chain corresponding to the message to be sent includes 8 memory descriptors, namely memory descriptor 1, memory descriptors 3-8 and memory descriptor 10. Among them, memory descriptors 3-8 are 6 memory descriptors with consecutive memory addresses. Based on this method, the number of the remaining multiple memory descriptors with consecutive memory addresses written in the high 12 bits of the flag field of memory descriptor 3 is 5.
[0100] When the hardware of a smart network interface card is arranged according to memory address order, it includes:
[0101] First, the first DMA acquires memory descriptor 1. The high 12 bits of the flag field of memory descriptor 1 are parsed and found to be 0. Based on the memory descriptor 3 pointed to by the next field of memory descriptor 1, the memory descriptor for the next DMA is determined to be 3.
[0102] Then, the second DMA memory descriptor 3 is retrieved, and the high 12 bits of the flag field of memory descriptor 3 are resolved to be 5, generating a third DMA request. The length of the generated DMA request increases, and memory descriptors 4, 5, 6, 7, and 8 can be obtained at once, that is, 5 memory descriptors can be obtained in one DMA request.
[0103] Next, the third DMA acquires memory descriptors 4, 5, 6, 7, and 8 at a time. It parses the high 12 bits of the flag field of memory descriptor 8, which is 0. Based on the memory descriptor 10 pointed to by the next field of memory descriptor 8, it determines the memory descriptor 10 for the next DMA.
[0104] Finally, for the fourth DMA memory descriptor 10, the high 12 bits of the flag field of memory descriptor 10 are parsed to be 0, and the DMA is stopped based on the next field of memory descriptor 10 indicating that memory descriptor 10 is the last memory descriptor in the target memory descriptor chain.
[0105] If the current method of acquiring memory descriptors based on the hardware of the smart network card requires performing DMA on memory descriptor 1, memory descriptors 3-8 and memory descriptor 10 respectively, a total of 8 DMA acquisition processes are required. However, the method provided by the embodiments of this application only requires 4 DMA acquisition processes, which can significantly reduce the number of memory descriptors acquired by DMA when there are many memory descriptors.
[0106] In some embodiments of this application, after a message is sent or received, the Virtio driver of the smart network card releases the memory descriptors allocated for receiving or sending the message, and sorts the descriptors in the send descriptor pool (Tx descriptor pool) and the receive descriptor pool (Rx descriptor pool) according to their memory addresses for subsequent allocation.
[0107] The processing method provided in this application embodiment can allocate a target memory descriptor chain for any message during the sending or receiving of a message by the Virtio driver of the smart network card. The target memory descriptor chain includes multiple memory descriptors that are contiguous in memory, and the memory descriptor with the smallest memory address among the multiple memory descriptors that are contiguous in memory includes the number of memory descriptors that are contiguous in memory.
[0108] Furthermore, when the smart network card hardware acquires memory descriptors via DMA, for multiple consecutive memory descriptors, after acquiring the memory descriptor with the smallest memory address through the first DMA request, the number of consecutive memory descriptors can be determined based on this memory descriptor, and the remaining consecutive memory descriptors can be acquired in one second DMA request.
[0109] By combining multiple contiguous memory descriptions into DMA requests, the number of DMA requests is significantly reduced. This fully utilizes the Max_payload_size feature of PCIe TLP, improves PCIe bandwidth utilization, and thus enhances the overall efficiency of the smart NIC in sending and receiving packets over the network, optimizing the performance of the smart NIC.
[0110] Referring to Figure 5, a functional block diagram of a data transmission processing apparatus provided in an embodiment of this application is shown. This apparatus is applied to a smart network interface card (NIC) and includes:
[0111] The descriptor determination module 100 is used by the Virtio driver of the smart network card to allocate a target memory descriptor chain for any message during the sending or receiving of a message. The target memory descriptor chain includes multiple memory descriptors that are contiguous in memory. The memory descriptor with the smallest memory address among the multiple memory descriptors that are contiguous in memory includes the number of multiple memory descriptors that are contiguous in memory.
[0112] The request module 200 is used to determine the number of consecutive memory descriptors and make a second DMA request after the hardware of the smart network card obtains the memory descriptor with the smallest memory address among multiple consecutive memory descriptors through the first DMA request. The second DMA request is used to obtain the remaining consecutive memory descriptors.
[0113] In some embodiments of this application, the descriptor determination module includes:
[0114] The target byte write unit is used to write the target byte of the memory descriptor with the smallest memory address among the multiple memory descriptors in the memory contiguous memory chain when the Virtio driver of the smart network card allocates multiple memory descriptors in the target memory descriptor chain for any packet.
[0115] In some embodiments of this application, the request module includes:
[0116] The parsing unit is used to parse the target byte in the memory descriptor after the smart network card hardware obtains the memory descriptor with the smallest memory address among multiple consecutive memory descriptors through the first DMA request, and obtain the number of the remaining multiple consecutive memory descriptors.
[0117] The request generation unit is used to determine the second DMA request based on the number of remaining contiguous memory descriptors.
[0118] In some embodiments of this application, the target byte writing unit includes:
[0119] The target byte write subunit is used to take the high 12 bits of the flag field of the memory descriptor with the smallest memory address among multiple contiguous memory descriptors as the target byte, and write the number of the remaining multiple contiguous memory descriptors in the high 12 bits of the flag field of that memory descriptor.
[0120] In some embodiments of this application, the descriptor determination module includes a first descriptor determination unit, used for:
[0121] During the message transmission process, the Virtio driver of the smart network card determines the number of memory descriptors corresponding to the message to be transmitted based on the size of the message to be transmitted.
[0122] According to the descriptor allocation strategy, a target memory descriptor chain is allocated in the sending descriptor pool. The number of memory descriptors included in the target memory descriptor chain is the number of memory descriptors corresponding to the message to be sent.
[0123] The descriptor allocation strategy is used to maximize the number of contiguous memory descriptors when allocating multiple memory descriptors.
[0124] In some embodiments of this application, the descriptor determination module includes a second descriptor determination unit, used for:
[0125] Before receiving a message, the Virtio driver of the smart NIC determines the maximum memory descriptor chain in the receive descriptor pool based on the depth of the receive queue and the descriptor allocation strategy.
[0126] Descriptor allocation strategies are used to maximize the number of contiguous memory descriptors when allocating multiple memory descriptors.
[0127] In some embodiments of this application, the second descriptor determining unit is further configured to:
[0128] During the message receiving process, the target memory descriptor chain corresponding to the message to be received is determined in the largest memory descriptor chain according to the size of the message to be received. The target memory descriptor chain corresponding to the message to be received includes multiple memory descriptors that are contiguous in memory.
[0129] Referring to FIG6, a schematic diagram of a computer device provided in an embodiment of the present application is shown, including: at least one processor 601 and a memory 602, wherein the memory 602 stores a computer program that can run on the processor 601, wherein when the processor 601 executes the computer program, it executes a data transmission process processing method of any one of the embodiments.
[0130] When the processor 601 executes a computer program and performs a data transmission process processing method according to any one of the embodiments, the specific steps include: during the process of sending or receiving a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, the target memory descriptor chain includes a plurality of memory descriptors in contiguous memory, and the memory descriptor with the smallest memory address among the plurality of memory descriptors in contiguous memory includes the number of the plurality of memory descriptors in contiguous memory; after the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the plurality of memory descriptors in contiguous memory through a first DMA request, it determines the number of the plurality of memory descriptors in contiguous memory based on the memory descriptor, and determines a second DMA request, the second DMA request being used to obtain the remaining plurality of memory descriptors in contiguous memory.
[0131] Referring to FIG7, a schematic diagram of a non-volatile readable storage medium provided in an embodiment of the present application is shown. The non-volatile readable storage medium 700 stores a computer program 701, wherein when the computer program 701 is executed by a processor, it executes a data transmission process processing method of any one of the embodiments.
[0132] When computer program 701 is executed by a processor, it performs a data transmission process processing method according to any one of the embodiments, specifically including: during the process of sending or receiving a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, the target memory descriptor chain includes multiple memory descriptors with contiguous memory, and the memory descriptor with the smallest memory address among the multiple memory descriptors with contiguous memory includes the number of the multiple memory descriptors with contiguous memory; after the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple memory descriptors with contiguous memory through a first DMA request, it determines the number of the multiple memory descriptors with contiguous memory based on the memory descriptor, and determines a second DMA request, the second DMA request being used to obtain the remaining multiple memory descriptors with contiguous memory.
[0133] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0134] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM (Compact Disc Read-Only Memory), optical storage, etc.) containing computer-usable program code.
[0135] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.
[0136] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.
[0137] These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable terminal equipment, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.
[0138] Although some embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including some embodiments of this application as well as all changes and modifications falling within the scope of the embodiments of this application.
[0139] Finally, throughout this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element.
[0140] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for processing data transmission, characterized in that, Applied to smart network interface cards, the method includes: During the process of sending or receiving a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message. The target memory descriptor chain includes multiple memory descriptors that are contiguous in memory. The memory descriptor with the smallest memory address among the multiple memory descriptors that are contiguous in memory includes the number of memory descriptors that are contiguous in memory. When the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple memory descriptors in contiguous memory through the first DMA request, it determines the number of multiple memory descriptors in contiguous memory based on the memory descriptor and determines the second DMA request. The second DMA request is used to obtain the remaining multiple memory descriptors in contiguous memory.
2. The method according to claim 1, characterized in that, The Virtio driver of the smart network interface card allocates a target memory descriptor chain for any packet, including: When the Virtio driver of the smart network card allocates a target memory descriptor chain that includes multiple memory descriptors in contiguous memory for any packet, the target byte of the memory descriptor with the smallest memory address among the multiple memory descriptors in contiguous memory is written to the remaining multiple memory descriptors in contiguous memory.
3. The method according to claim 2, characterized in that, When the smart network card hardware obtains the memory descriptor with the smallest memory address among a plurality of contiguous memory descriptors through a first DMA request, it determines the number of contiguous memory descriptors based on this memory descriptor and determines a second DMA request, including: When the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple memory descriptors in contiguous memory through the first DMA request, it parses the target byte in the memory descriptor to obtain the number of the remaining multiple memory descriptors in contiguous memory. The second DMA request is determined based on the number of remaining contiguous memory descriptors.
4. The method according to claim 3, characterized in that, The number of times the target byte of the memory descriptor with the smallest memory address among the multiple contiguous memory descriptors is written to the remaining multiple contiguous memory descriptors includes: Take the high 12 bits of the flag field of the memory descriptor with the smallest memory address among the multiple contiguous memory descriptors as the target byte, and write the number of the remaining multiple contiguous memory descriptors into the high 12 bits of the flag field of that memory descriptor.
5. The method according to claim 1, characterized in that, The Virtio driver of the smart network interface card allocates a target memory descriptor chain for any packet, including: During the message transmission process, the Virtio driver of the smart network card determines the number of memory descriptors corresponding to the message to be transmitted based on the size of the message to be transmitted. According to the descriptor allocation strategy, a target memory descriptor chain is allocated in the sending descriptor pool, and the number of memory descriptors included in the target memory descriptor chain is the number of memory descriptors corresponding to the message to be sent; The descriptor allocation strategy is used to maximize the number of contiguous memory descriptors when allocating multiple memory descriptors.
6. The method according to claim 1, characterized in that, The Virtio driver of the smart network interface card allocates a target memory descriptor chain for any packet, including: Before receiving a message, the Virtio driver of the smart network card determines the maximum memory descriptor chain in the receive descriptor pool based on the depth of the receive queue and the descriptor allocation strategy. The descriptor allocation strategy is used to maximize the number of contiguous memory descriptors when allocating multiple memory descriptors.
7. The method according to claim 6, characterized in that, During the process of receiving messages, the method further includes: Based on the size of the message to be received, the target memory descriptor chain corresponding to the message to be received is determined in the maximum memory descriptor chain, and the target memory descriptor chain corresponding to the message to be received includes multiple memory descriptors that are contiguous in memory.
8. A data transmission processing apparatus, characterized in that, The device, applied to a smart network interface card, includes: The descriptor determination module is used, during the process of sending or receiving a message, by the Virtio driver of the smart network card to allocate a target memory descriptor chain for any message. The target memory descriptor chain includes multiple memory descriptors that are contiguous in memory. The memory descriptor with the smallest memory address among the multiple memory descriptors that are contiguous in memory includes the number of memory descriptors that are contiguous in memory. The request module is configured to, after the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple memory descriptors in contiguous memory through a first DMA request, determine the number of the multiple memory descriptors in contiguous memory based on the memory descriptor, and determine a second DMA request, the second DMA request being used to obtain the remaining multiple memory descriptors in contiguous memory.
9. A computer device, characterized in that, include: At least one processor, and a memory storing a computer program executable on the processor, wherein the processor executes the computer program to perform a processing method of the data transmission process according to any one of claims 1-7.
10. A non-volatile readable storage medium, characterized in that, The non-volatile readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it performs the processing method of the data transmission process according to any one of claims 1-7.