Processing method and device for data transmission process, equipment and storage medium

Abstract

The invention provides a data transmission process processing method and device, equipment and a storage medium, and belongs to the technical field of data processing. The method comprises the steps that in the process of sending a message or receiving the message, a Virtio driver of an intelligent network card allocates a target memory descriptor chain for any message, and the target memory descriptor chain comprises a plurality of memory descriptors with continuous memories; the memory descriptor with the minimum memory address in the plurality of memory descriptors with continuous memories comprises the number of the plurality of memory descriptors with continuous memories; and when the hardware of the intelligent network card obtains the memory descriptor with the minimum memory address in the plurality of memory descriptors with the continuous memories through the first DMA request, determining the number of the plurality of memory descriptors with the continuous memories according to the memory descriptor, and determining that the second DMA request obtains the residual plurality of memory descriptors with the continuous memories. The invention aims to improve the efficiency of receiving and transmitting messages by the intelligent network card in a network, and further improve the performance of the intelligent network card.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of data processing, and specifically, to a method, apparatus, device and storage medium for processing a data transmission process. Background Art

[0002] Smart NIC is a high-performance network card dedicated to network data processing. It uses customized chips, high-speed network interfaces and powerful software support. It can provide faster, safer and more reliable network connection and data transmission services for data centers and enterprise networks. Smart NIC SNIC has rich hardware offload capabilities and can provide Virtio hardware acceleration.

[0003] The bottom layer of the smart network card is a PCIe device, which uses the standard PCIe protocol for communication. The Virtio protocol runs above the PCIe protocol, and the smart network card sends and receives network messages in the format of the Virtio protocol.

[0004] Virtio is a semi-virtualization technology that uses shared memory to achieve data transmission between virtual machines and host machines. The Virtio protocol is used in the hardware of the smart network card for software and hardware communication. In the process of sending and receiving data packets, the hardware of the smart network card first reads the memory through DMA to obtain the memory descriptor, then parses the memory information represented in the memory descriptor, and finally reads / writes the data packet to the memory address specified by the memory descriptor through DMA.

[0005] In Virtio-based smart NICs, the length of a descriptor is relatively small, such as 16 bytes, while the length of the Max_Payload_Size of PCIe is usually 256 bytes; the request length of DMA read descriptors, such as 16 bytes, is much smaller than the length of the Max_Payload_Size of the PCIe tlp message, which is 256 bytes. This makes the payload of PCIe tlp relatively low. When the descriptors are chained, only one descriptor of a smaller length is obtained by DMA each time, resulting in reduced PCIe bandwidth utilization and affecting the performance of the smart NIC. Summary of the invention

[0006] The embodiments of the present application provide a method, apparatus, device and storage medium for processing a data transmission process, aiming to improve the efficiency of a smart network card in sending and receiving messages on a network, thereby improving the performance of the smart network card.

[0007] In a first aspect, an embodiment of the present application provides a method for processing a data transmission process, which is applied to a smart network card. The method includes: In the process of sending or receiving a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, wherein the target memory descriptor chain includes a plurality of memory descriptors that are continuous in memory, and the memory descriptor with the smallest memory address among the plurality of memory descriptors that are continuous in memory includes the number of the plurality of memory descriptors that are continuous in memory; After the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors of the memory through the first DMA request, the number of the multiple continuous memory descriptors of the memory is determined according to the memory descriptor, and a second DMA request is determined, where the second DMA request is used to obtain the remaining multiple continuous memory descriptors of the memory.

[0008] Optionally, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message including: When the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, including multiple memory descriptors that are continuous in memory, the target byte of the memory descriptor with the smallest memory address among the multiple memory descriptors that are continuous in memory is written to the number of the remaining multiple memory descriptors that are continuous in memory.

[0009] Optionally, after the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors of the memory through the first DMA request, the number of the multiple continuous memory descriptors of the memory is determined according to the memory descriptor, and the second DMA request is determined, including: After the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors of the memory through the first DMA request, the target byte in the memory descriptor is parsed to obtain the number of the remaining multiple continuous memory descriptors of the memory; A second DMA request is determined according to the number of the plurality of memory descriptors remaining continuously in the memory.

[0010] Optionally, writing the target byte of the memory descriptor with the smallest memory address among the multiple memory descriptors in the continuous memory to the number of the remaining multiple memory descriptors in the continuous memory includes: The upper 12 bits of the flag field of the memory descriptor with the smallest memory address among the multiple continuous memory descriptors are used as the target byte, and the number of the remaining multiple continuous memory descriptors is written in the upper 12 bits of the flag field of the memory descriptor.

[0011] Optionally, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, including: In the process of sending the message, the Virtio driver of the smart network card determines the number of multiple memory descriptors corresponding to the message to be sent according to the size of the message to be sent; Allocating a target memory descriptor chain in a sending descriptor pool according to a descriptor allocation strategy, wherein the number of memory descriptors included in the target memory descriptor chain is the number of multiple memory descriptors corresponding to the message to be sent; The descriptor allocation strategy is used to maximize the number of multiple consecutive memory descriptors when allocating multiple memory descriptors.

[0012] Optionally, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, including: Before receiving the message, the Virtio driver of the smart network card determines the maximum memory descriptor chain in the receiving descriptor pool according to the depth of the receiving queue and the descriptor allocation policy; The descriptor allocation strategy is used to maximize the number of multiple consecutive memory descriptors when allocating multiple memory descriptors.

[0013] Optionally, during the process of receiving the message, the method further includes: According to the size of the message to be received, a target memory descriptor chain corresponding to the message to be received is determined in the maximum memory descriptor chain, and the target memory descriptor chain corresponding to the message to be received includes multiple memory descriptors that are continuous in memory.

[0014] In a second aspect, an embodiment of the present application provides a device for processing a data transmission process, which is applied to a smart network card, and the device includes: A descriptor determination module, used for allocating a target memory descriptor chain for any message by the Virtio driver of the intelligent network card during the process of sending or receiving a message, wherein the target memory descriptor chain includes a plurality of memory descriptors of continuous memory, and the memory descriptor with the smallest memory address among the plurality of memory descriptors of continuous memory includes the number of the plurality of memory descriptors of continuous memory; A request module is used to determine the number of multiple continuous memory descriptors according to the memory descriptor after the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors of the memory through a first DMA request, and determine a second DMA request, wherein the second DMA request is used to obtain the remaining multiple continuous memory descriptors of the memory.

[0015] In a third aspect, an embodiment of the present application provides a computer device, comprising: at least one processor, and a memory, wherein the memory stores a computer program that can be run on the processor, wherein when the processor executes the computer program, the processing method of the data transmission process described in the first aspect of the embodiment is performed.

[0016] In a fourth aspect, an embodiment of the present application provides a non-volatile readable storage medium, wherein the non-volatile readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the processing method of the data transmission process described in the first aspect of the embodiment is performed.

[0017] Beneficial effects: In the process of sending or receiving a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, wherein the target memory descriptor chain includes multiple continuous memory descriptors, and the memory descriptor with the smallest memory address among the multiple continuous memory descriptors includes the number of the multiple continuous memory descriptors; when the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors through a first DMA request, the number of the multiple continuous memory descriptors is determined according to the memory descriptor, and a second DMA request is determined, wherein the second DMA request is used to obtain the remaining multiple continuous memory descriptors of the memory.

[0018] In the processing method of the data transmission process provided by this embodiment, in the process of sending or receiving a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, and the target memory descriptor chain includes multiple continuous memory descriptors, and the memory descriptor with the smallest memory address among the multiple continuous memory descriptors includes the number of the multiple continuous memory descriptors.

[0019] Furthermore, when the hardware of the smart network card obtains the memory descriptor through DMA, for multiple consecutive memory descriptors, after obtaining the memory descriptor with the smallest memory address through the first DMA request, the number of multiple consecutive memory descriptors can be determined according to the memory descriptor, and the second DMA request is determined to obtain the remaining multiple consecutive memory descriptors at one time, thereby significantly reducing the number of DMA requests, making full use of the Max_payload_size feature of PCIe tlp, and improving the bandwidth utilization of PCIe, thereby improving the efficiency of the smart network card in sending and receiving messages on the network as a whole, and optimizing the performance of the smart network card. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] Figure 1 is a flowchart of the steps of a data transmission method proposed in an embodiment of the present application; Figure 2 This is a schematic diagram of allocating memory descriptors during data transmission proposed in an embodiment of the present application; Figure 3 This is a schematic diagram of allocating memory descriptors during data reception provided by an embodiment of the present application; Figure 4 is a schematic diagram of obtaining a memory descriptor proposed in an embodiment of the present application; Figure 5 It is a functional module diagram of a processing device for a data transmission process proposed in an embodiment of the present application; Figure 6 is a schematic diagram of a computer device provided by an embodiment of the present application; Figure 7 It is a schematic diagram of a non-volatile readable storage medium proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] DMA: Direct Memory Access, directly access memory, read and write memory directly without CPU intervention; PCIe: peripheral component interconnect express, a high-speed serial computer expansion bus standard; Tlp: Transaction Layer Packets, which play a very important role in the PCIe communication process. Tlp is used to transfer data and control information between devices. They are the basic information transmission unit of PCIe. Virtio: An abstraction layer above devices in a paravirtualized hypervisor; Tx: sending direction; Rx: receiving direction; Descriptor Pool: Descriptor pool, the descriptor pool in Vulkan is a resource pool that manages descriptor sets (Descriptor Sets) for efficient allocation and recycling of descriptor sets; descriptor sets are data structures used in Vulkan to bind resources (such as buffers, textures, etc.) to shaders; descriptor pools ensure that the allocation and recycling process of descriptor sets is efficient and controllable.

[0024] Smart NIC is a high-performance network card dedicated to network data processing. It uses customized chips, high-speed network interfaces and powerful software support to provide faster, safer and more reliable network connection and data transmission services for data centers and enterprise networks. It can provide performance advantages and functional enhancements that many traditional network cards cannot achieve, and can also cope with increasingly complex network applications and business needs.

[0025] High performance: Smart NICs usually use dedicated chips and high-speed network interfaces to achieve ultra-high data processing rates, throughput, and flow control efficiency, thereby improving network performance and response speed.

[0026] Safe and reliable: Smart NICs can not only accelerate data transmission, but also provide better security functions and protection mechanisms, such as data encryption, access control, DDoS defense, firewalls, etc., thereby enhancing the security and stability of the network.

[0027] Support for distributed storage and computing: Smart NICs can also be combined with distributed storage, computing and other technologies to achieve more efficient data processing and analysis, while also reducing network latency and load pressure, thereby improving overall performance and efficiency.

[0028] Smart NIC (SNIC) has rich hardware offload capabilities and can provide Virtio hardware acceleration and security-related data plane offload. Unlike traditional NICs, Smart NICs offload cloud network forwarding functions, provide high-performance network forwarding capabilities, and release CPU general computing resources, including transmission and storage protocols. At the same time, they provide network programmability to customize network forwarding logic and ensure functional flexibility.

[0029] Descriptors play a vital role in memory management. They not only identify the location and size of memory areas, but also provide access control and permission management for memory areas. Through descriptors, the operating system can effectively manage the address space of processes and ensure that each process has an independent memory space, thereby preventing mutual interference between processes. Descriptors play a key role in memory management. They ensure the stability and security of the system by identifying and managing memory areas.

[0030] Virtio is a semi-virtualization technology that uses shared memory to achieve data transmission between virtual machines and host machines. The Virtio protocol is used in the smart network card hardware for software and hardware communication. During the process of sending and receiving data packets, the smart network card hardware reads the memory through DMA to obtain the memory descriptor, then parses the memory information represented in the memory descriptor, and finally reads / writes the data packet to the memory address specified by the memory descriptor through DMA.

[0031] A memory descriptor corresponds to a memory area. During the data packet sending and receiving process, large data packets usually need to be fragmented, such as splitting a large packet into multiple small packets. Therefore, multiple continuous or discontinuous memory addresses may be involved. Therefore, multiple descriptors are usually required to be chained, that is, one descriptor can point to the next descriptor. In addition, due to the decentralized aggregation characteristics of SGL, multiple descriptors are also required to be chained.

[0032] Since the length of a Virtio descriptor itself is relatively short, such as usually 16 bytes, and the length of PCIe's Max_Payload_Size is usually 256 bytes; the request length of the DMA read descriptor, such as 16 bytes, is smaller than the length of the PCIe tlp message's Max_Payload_Size, 256 bytes, which makes the PCIe tlp payload relatively low. When the descriptors are chained, only one descriptor is DMAed at a time, and the length of the DMA request is small, resulting in reduced PCIE bandwidth utilization and performance impact.

[0033] Therefore, in order to improve the bandwidth utilization of PCIe, thereby improving the performance of the Virtio-based network card in receiving or sending messages on the network, an embodiment of the present application provides a data transmission method.

[0034] Reference Figure 1 , shows a flowchart of a data transmission method in an embodiment of the present application, the method can be applied to a smart network card, and the method can specifically include the following steps: S101: In the process of sending or receiving a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, wherein the target memory descriptor chain includes multiple continuous memory descriptors, and the memory descriptor with the smallest memory address among the multiple continuous memory descriptors includes the number of the multiple continuous memory descriptors.

[0035] Memory refers to the memory of the host where the Smart NIC is deployed. In the Virtio framework, the memory descriptor is used to describe the location of data in the memory. The memory descriptor of the Virtio driver of the Smart NIC is stored in a memory area. Generally, the memory descriptors have the same size.

[0036] The memory area storing the memory descriptors of the Virtio driver is recorded as a descriptor pool, including a sending descriptor pool Tx descriptor pool and a receiving descriptor pool Rx descriptor pool. The size of the descriptor pool is related to the specifications of the smart network card.

[0037] For example, Smart NIC A can allocate X1 memory descriptors, and Smart NIC B can allocate X2 memory descriptors. The memory descriptor is used to describe the memory information of the memory area. Specifically, the memory descriptor uses multiple fields to describe the memory information of the memory area. For example, the addr field of the memory descriptor is used to indicate the starting position of the physical address space of a memory area of ​​the host; the len field usually describes the read and write properties of the memory area and the length of the read and write data; the flag field is used to indicate the characteristics of the descriptor itself, such as VRING_DESC_F_NEXT indicates whether the next field of the memory descriptor is valid, and VRING_DESC_F_INDIRECT indicates whether the descriptor points to an indirect descriptor table.

[0038] In the actual implementation process, the data packets to be read and written may be in different locations in the memory, that is, one piece of data may usually involve multiple continuous or discontinuous memory addresses.

[0039] For example, when data A needs to be obtained for transmission, part of data A is at memory address ab, and another part of data A is at memory address bc. Then, the data is not at a continuous memory address in the memory, but at a discontinuous memory address.

[0040] The driver of the smart network card needs to allocate multiple memory descriptors to correspond to the location of the data message in the memory. The hardware of the smart network card first obtains the memory descriptor corresponding to the data message in the memory through DMA request, and then obtains the data content of the data message from the corresponding memory address according to the memory descriptor; while the general DMA request only reads one memory descriptor at a time.

[0041] All memory descriptors of data A are organized into a memory descriptor chain. For example, the next field in memory descriptor 1 points to the next memory descriptor 2. Generally, after DMA memory descriptor 1, the next field in memory descriptor 1 is parsed, and then the next memory descriptor 2 pointed to by DMA memory descriptor 1 is accessed.

[0042] In the method provided in this embodiment, in the process of sending or receiving a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, and the target memory descriptor chain includes multiple memory descriptors. By way of example, in the process of the smart network card sending a message, the multiple memory descriptors comprehensively describe the location of the message to be sent in the memory, etc. After obtaining the multiple memory descriptors, the message data can be obtained from the corresponding memory and then sent.

[0043] Specifically, when the Virtio driver of the smart network card allocates a target memory descriptor chain, the target memory descriptor chain includes multiple memory descriptors with continuous memory, that is, the Virtio driver of the smart network card can allocate multiple memory descriptors with continuous memory addresses as much as possible, and the memory descriptor with the smallest memory address among the multiple continuous memory descriptors includes the number of the multiple continuous memory descriptors.

[0044] In a feasible implementation manner, during the process of sending a message, the process of the Virtio driver of the smart network card allocating a target memory descriptor chain for any message includes the following steps: A1: The Virtio driver of the smart network card determines the number of multiple memory descriptors corresponding to the message to be sent according to the size of the message to be sent.

[0045] First, when sending a message, the Virtio driver of the smart network card determines the number of multiple memory descriptors that need to be allocated according to the size of the message to be sent. For example, the maximum length of data described by each memory descriptor is limited. When the message to be sent is large, multiple memory descriptors need to be allocated to describe the location of the message to be sent in the memory.

[0046] In the actual implementation process, the number of memory descriptors can also be determined according to the size of the message to be sent and the location where the message to be sent is stored in the memory. For example, the maximum data length described by a memory descriptor is A. Although the size of a message to be sent is smaller than A, it is sufficient to be described by one memory descriptor. However, the first part of the message to be sent is located at the first memory location, and the second part of the message to be sent is located at the second memory location. At this time, two memory descriptors can be allocated, the first memory descriptor is used to describe the first memory location, and the second memory descriptor is used to describe the second memory location.

[0047] A2: According to the descriptor allocation strategy, a target memory descriptor chain is allocated in the sending descriptor pool, and the number of memory descriptors included in the target memory descriptor chain is the number of multiple memory descriptors corresponding to the message to be sent.

[0048] In this method, when a message to be sent corresponds to multiple memory descriptors, the Virtio driver of the smart network card executes a descriptor allocation strategy in a sending descriptor pool, namely, a Tx descriptor pool, wherein the descriptor allocation strategy is used to maximize the number of multiple consecutive memory descriptors when allocating multiple memory descriptors.

[0049] Reference Figure 2 , shows a schematic diagram of allocating memory descriptors during data transmission provided by an embodiment of the present application, scenario one: assuming that four memory descriptors are required for the message to be sent, all memory descriptors in the Tx descriptor pool are in an allocatable state, that is, when the descriptor is not currently used, the Virtio driver of the smart network card executes the descriptor allocation strategy, maximizes the number of multiple consecutive memory descriptors, and uses memory descriptor 1, memory descriptor 2, memory descriptor 3 and memory descriptor 4 as the target memory descriptor chain of the message to be sent, and the locations where memory descriptors 1-4 are stored in the memory are continuous, that is, these four memory descriptors are memory descriptors with continuous memory addresses.

[0050] Scenario 2: Assume that four memory descriptors are required for the message to be sent. Memory descriptor 1, memory allocator 3, and memory allocator 7-n in the Tx descriptor pool are all allocated, that is, these descriptors are currently in use. Then, the Virtio driver of the SmartNIC executes the descriptor allocation strategy to maximize the number of multiple consecutive memory descriptors. Memory descriptors 4, 5, and 6 are used as three consecutive memory descriptors, and memory descriptor 2 is used as a memory descriptor with a separate memory address. Finally, the target memory descriptor chain of the message to be sent includes memory descriptor 2 and memory descriptors 4-6.

[0051] A3: When the Virtio driver of the smart network card allocates a target memory descriptor chain for any message including multiple continuous memory descriptors, the target byte of the memory descriptor with the smallest memory address among the multiple continuous memory descriptors is written with the number of the remaining multiple continuous memory descriptors.

[0052] Specifically, the upper 12 bits of the flag field of the memory descriptor with the smallest memory address among the multiple continuous memory descriptors are used as the target byte, and the number of the remaining multiple continuous memory descriptors is written in the upper 12 bits of the flag field of the memory descriptor.

[0053] In the actual implementation process, when the hardware of the smart network card DMAs multiple memory descriptors, it usually DMAs them in sequence according to the order of the memory addresses of the memory descriptors. Therefore, for multiple consecutive memory descriptors, only the number of the remaining multiple consecutive memory descriptors can be written in the target byte of the target memory descriptor. The target memory descriptor refers to the memory descriptor that is DMAed first among the multiple consecutive memory descriptors.

[0054] For example, Figure 2 Taking scenario 1 as an example, the target memory descriptor chain corresponding to the message to be sent includes multiple memory descriptors with consecutive memory addresses, namely, memory descriptors 1-4. At this time, the number of the remaining multiple consecutive memory descriptors is written in the target byte of memory descriptor 1, that is, 3 is written in the high 12 bits of the flag field of memory descriptor 1. The value 3 in the high 12 bits of the flag field represents the number of subsequent consecutive memory descriptors, that is, after memory descriptor 1, the memory addresses of 3 memory descriptors (memory descriptors 2, 3, and 4) are consecutive.

[0055] For example, Figure 2 Taking scenario 2 as an example, the target memory descriptor chain corresponding to the message to be sent includes three memory descriptors with consecutive memory addresses, namely memory descriptors 4, 5, and 6, and a memory descriptor 2 with a separately distributed memory address; because there are no other memory descriptors with consecutive addresses with respect to memory descriptor 2, the upper 12 bits of the flag field in memory descriptor 2 do not contain the number of the remaining multiple memory descriptors with consecutive memory addresses. For example, 0 can be written to the upper 12 bits of the flag field; then, among the memory descriptors 4, 5, and 6 with consecutive memory addresses, the number of the remaining multiple memory descriptors with consecutive memory addresses is written to the upper 12 bits of the flag field of memory descriptor 4 that is first DMAed, that is, 2 is written to the upper 12 bits of the flag field of memory descriptor 4, and the value 2 in the upper 12 bits of the flag field represents the number of subsequent consecutive memory descriptors, that is, after memory descriptor 4, the memory addresses of two memory descriptors (memory descriptors 5 and 6) are consecutive.

[0056] In a feasible implementation manner, during the process of sending a message, the process of the Virtio driver of the smart network card allocating a target memory descriptor chain for any message includes the following steps: B1: Before receiving a message, the Virtio driver of the smart network card determines the maximum memory descriptor chain in the receiving descriptor pool according to the depth of the receiving queue and the descriptor allocation policy.

[0057] Before receiving a message, the size of the subsequent received message is not known, but the smart network card may receive multiple messages at a time. Therefore, according to the depth of the receive queue, the Virtio driver of the smart network card can execute the descriptor allocation strategy in the receive descriptor pool, namely the Rx descriptor pool, and allocate enough memory descriptors to form the maximum memory descriptor chain, that is, the number of memory descriptors required when the receive queue is full of received messages, and wait for the reception of the to-be-received message.

[0058] The descriptor allocation strategy is used to maximize the number of multiple consecutive memory descriptors when allocating multiple memory descriptors. Therefore, the Virtio driver of the smart network card allocates multiple memory descriptors with consecutive addresses in the Rx descriptor pool as much as possible as the target memory descriptor chain when receiving messages.

[0059] B2: According to the size of the message to be received, determine the target memory descriptor chain corresponding to the message to be received in the maximum memory descriptor chain, and the target memory descriptor chain corresponding to the message to be received includes multiple memory descriptors that are continuous.

[0060] After receiving the message to be received, according to the size of the message to be received, the target memory descriptor chain corresponding to the message to be received is determined in the target memory descriptor chain. For example, although the Virtio driver of the smart network card has allocated the maximum memory descriptor chain of the message to be received according to the receiving queue of the smart network card before the message is received, when the message is actually received, the size of the message to be received may only require the use of part of the memory descriptor chain as the target memory descriptor chain.

[0061] In the target memory descriptor chain, the number of multiple memory descriptors with continuous memory addresses is still maximized in the maximum memory descriptor chain, and the number of the remaining multiple memory descriptors with continuous memory addresses is written into the target byte of the memory descriptor that is first DMAed among the multiple memory descriptors with continuous memory addresses. For example, the high 12 bits of the flag field of the memory descriptor can be used as the target byte.

[0062] Reference Figure 3 , shows a schematic diagram of allocating memory descriptors during the data receiving process provided by an embodiment of the present application. Before data reception, the unallocated memory descriptors in the receiving descriptor pool Rx descriptor pool include memory descriptors 2-3, memory descriptors 5-9 and memory descriptors 11, and memory descriptors 2-3, memory descriptors 5-9 and memory descriptors 11 are determined as the maximum memory descriptor chain.

[0063] Assuming that after receiving the message A to be received, according to the size of the message A to be received, it is determined that the message A to be received requires 4 memory descriptors, then the number of multiple memory descriptors with maximized continuous memory addresses is selected in the maximum memory descriptor chain, that is, 4 memory descriptors with 4 continuous memory addresses are selected, that is, memory descriptors 5-8 are used as the target memory descriptor chain, and 3 is written in the high 12 bits of the flag field of memory descriptor 5, indicating that after memory descriptor 5, the memory addresses of 3 memory descriptors (memory descriptors 6, 7, and 8) are continuous.

[0064] S102: After the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors of the memory through the first DMA request, the number of the multiple continuous memory descriptors of the memory is determined according to the memory descriptor, and a second DMA request is determined, where the second DMA request is used to obtain the remaining multiple continuous memory descriptors of the memory.

[0065] Specifically, for multiple memory descriptors with continuous memory addresses, when the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors through the first DMA request, it parses the target byte in the memory descriptor to obtain the number of the remaining multiple continuous memory descriptors; and determines the second DMA request based on the number of the remaining multiple continuous memory descriptors.

[0066] When the hardware of the smart network card requests the memory descriptor in the memory through DMA, after obtaining any memory descriptor, the target byte in the memory descriptor is parsed to determine whether the memory descriptor contains the number of the remaining multiple continuous memory descriptors. If the target byte is 0, it means that the memory descriptor adjacent to the memory address of the memory descriptor is not one of the target memory descriptor chain. If the target byte is a non-zero value, it means that the memory descriptor adjacent to the memory address of the memory descriptor is one of the target memory descriptor chain. The DMA request can be combined according to the number of the remaining multiple continuous memory descriptors to obtain the remaining multiple continuous memory descriptors at one time, without performing DMA on the remaining multiple continuous memory descriptors one by one.

[0067] Reference Figure 4 , showing a schematic diagram of obtaining memory descriptors provided in an embodiment of the present application, assuming that the target memory descriptor chain corresponding to the message to be sent includes 8 memory descriptors, namely memory descriptor 1, memory descriptors 3-8 and memory descriptor 10, where memory descriptors 3-8 are 6 memory descriptors with consecutive memory addresses. Based on this method, the number of the remaining multiple memory descriptors written into the high 12 bits of the flag field of memory descriptor 3 is 5.

[0068] When the hardware of the Smart NIC is in the order of memory addresses, it includes: First, the first DMA obtains memory descriptor 1, and the high 12 bits of the flag field of memory descriptor 1 are parsed to be 0. According to the memory descriptor 3 pointed to in the next field of memory descriptor 1, it is determined that the next DMA memory descriptor is 3; Then, the second DMA memory descriptor 3 is parsed, and the high 12 bits of the flag field of memory descriptor 3 are 5, and the third DMA request is generated. The length of the generated DMA request increases, and memory descriptors 4, 5, 6, 7, and 8 can be obtained at one time, that is, one DMA request can obtain 5 memory descriptors; Next, the third DMA obtains memory descriptors 4, 5, 6, 7, and 8 at a time, and the high 12 bits of the flag field of memory descriptor 8 are parsed to be 0. According to the memory descriptor 10 pointed to in the next field of memory descriptor 8, the next DMA memory descriptor 10 is determined; Finally, for the fourth DMA memory descriptor 10, the upper 12 bits of the flag field of the memory descriptor 10 are parsed to be 0, and the next field of the memory descriptor 10 indicates that the memory descriptor 10 is the last memory descriptor of the target memory descriptor chain, so the DMA is stopped.

[0069] If the hardware of the smart network card currently obtains memory descriptors in a manner that performs DMA on memory descriptor 1, memory descriptors 3-8, and memory descriptor 10 respectively, a total of 8 DMA acquisition processes are required. However, based on the method provided in this embodiment, only 4 DMA acquisition processes are required, so that the number of memory descriptions obtained by DMA can be reduced obviously when there are many memory descriptors.

[0070] In a feasible implementation, when the message is sent or received, the Virtio driver of the smart network card releases the memory descriptor allocated for receiving the message or sending the message, and sorts the descriptors in the sending descriptor pool Txdescriptor pool and the receiving descriptor pool Rx descriptor pool according to the memory address of the descriptor for subsequent allocation.

[0071] The processing method provided in this embodiment can allocate a target memory descriptor chain for any message in the process of sending or receiving a message, where the Virtio driver of the smart network card includes multiple continuous memory descriptors, and the memory descriptor with the smallest memory address among the multiple continuous memory descriptors includes the number of the multiple continuous memory descriptors.

[0072] Furthermore, when the hardware of the smart network card obtains the memory descriptor through DMA, for multiple consecutive memory descriptors, after obtaining the memory descriptor with the smallest memory address through the first DMA request, the number of multiple consecutive memory descriptors can be determined according to the memory descriptor, and the second DMA request can be determined to obtain the remaining multiple consecutive memory descriptors at one time.

[0073] Therefore, by merging DMA requests for multiple consecutive memory descriptions, the number of DMA requests is significantly reduced, the Max_payload_size feature of PCIe tlp is fully utilized, and the bandwidth utilization of PCIe is improved, thereby improving the efficiency of sending and receiving messages on the network of the smart network card as a whole and optimizing the performance of the smart network card.

[0074] Reference Figure 5 , shows a functional module diagram of a processing device for a data transmission process provided by an embodiment of the present application, which is applied to a smart network card, and the device includes: The descriptor determination module 100 is used for allocating a target memory descriptor chain for any message by the Virtio driver of the smart network card during the process of sending or receiving a message, wherein the target memory descriptor chain includes a plurality of memory descriptors of continuous memory, and the memory descriptor with the smallest memory address among the plurality of memory descriptors of continuous memory includes the number of the plurality of memory descriptors of continuous memory; The request module 200 is used to determine the number of multiple continuous memory descriptors according to the memory descriptor after the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors of the memory through the first DMA request, and determine the second DMA request, wherein the second DMA request is used to obtain the remaining multiple continuous memory descriptors of the memory.

[0075] Optionally, the descriptor determination module includes: The target byte writing unit is used to write the target byte of the memory descriptor with the smallest memory address among the multiple continuous memory descriptors into the number of the remaining multiple continuous memory descriptors when the Virtio driver of the smart network card allocates a target memory descriptor chain including multiple continuous memory descriptors for any message.

[0076] Optionally, the request module includes: A parsing unit, configured to parse a target byte in the memory descriptor to obtain the number of the remaining multiple continuous memory descriptors of the memory after the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors of the memory through the first DMA request; The request generating unit is used to determine a second DMA request according to the number of the plurality of memory descriptors remaining continuously in the memory.

[0077] Optionally, the target byte writing unit includes: The target byte writing sub-unit is used to take the upper 12 bits of the flag field of the memory descriptor with the smallest memory address among the multiple continuous memory descriptors as the target byte, and write the number of the remaining multiple continuous memory descriptors in the upper 12 bits of the flag field of the memory descriptor.

[0078] Optionally, the descriptor determination module includes a first descriptor determination unit, configured to: In the process of sending the message, the Virtio driver of the smart network card determines the number of multiple memory descriptors corresponding to the message to be sent according to the size of the message to be sent; Allocating a target memory descriptor chain in a sending descriptor pool according to a descriptor allocation strategy, wherein the number of memory descriptors included in the target memory descriptor chain is the number of multiple memory descriptors corresponding to the message to be sent; The descriptor allocation strategy is used to maximize the number of multiple consecutive memory descriptors when allocating multiple memory descriptors.

[0079] Optionally, the descriptor determination module includes a second descriptor determination unit, configured to: Before receiving the message, the Virtio driver of the smart network card determines the maximum memory descriptor chain in the receiving descriptor pool according to the depth of the receiving queue and the descriptor allocation policy; The descriptor allocation strategy is used to maximize the number of multiple consecutive memory descriptors when allocating multiple memory descriptors.

[0080] Optionally, the second descriptor determining unit is further used to: During the process of receiving a message, according to the size of the message to be received, a target memory descriptor chain corresponding to the message to be received is determined in the maximum memory descriptor chain, and the target memory descriptor chain corresponding to the message to be received includes multiple continuous memory descriptors.

[0081] Reference Figure 6 , shows a schematic diagram of a computer device provided in an embodiment of the present application, including: at least one processor 601, and a memory 602, wherein the memory 602 stores a computer program that can be run on the processor 601, wherein the processor 601 executes the processing method of the data transmission process described in any one of the embodiments when executing the computer program.

[0082] When the processor 601 executes the computer program and executes the method for processing the data transmission process described in any one of the embodiments, the method specifically includes executing: in the process of sending or receiving a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, and the target memory descriptor chain includes multiple continuous memory descriptors, and the memory descriptor with the smallest memory address among the multiple continuous memory descriptors includes the number of multiple continuous memory descriptors; when the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors through a first DMA request, the number of multiple continuous memory descriptors is determined according to the memory descriptor, and a second DMA request is determined, and the second DMA request is used to obtain the remaining multiple continuous memory descriptors of the memory.

[0083] Reference Figure 7 , showing a schematic diagram of a non-volatile readable storage medium provided in an embodiment of the present application, wherein the non-volatile readable storage medium 700 stores a computer program 701, wherein the computer program 701, when executed by a processor, performs a processing method for a data transmission process described in any one of the embodiments.

[0084] When the computer program 701 is executed by the processor to perform the processing method of the data transmission process described in any one of the embodiments, it specifically includes the following execution: in the process of sending or receiving a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, and the target memory descriptor chain includes multiple continuous memory descriptors, and the memory descriptor with the smallest memory address among the multiple continuous memory descriptors includes the number of multiple continuous memory descriptors; when the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors through a first DMA request, the number of multiple continuous memory descriptors is determined according to the memory descriptor, and a second DMA request is determined, and the second DMA request is used to obtain the remaining multiple continuous memory descriptors of the memory.

[0085] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0086] It should be understood by those skilled in the art that the embodiments of the present application can be provided as methods, devices or computer program products. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0087] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0088] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0090] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0091] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0092] Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for processing a data transmission process, characterized in that: Applied to a smart network card, the method includes: In the process of sending or receiving a message, the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, wherein the target memory descriptor chain includes a plurality of memory descriptors that are continuous in memory, and the memory descriptor with the smallest memory address among the plurality of memory descriptors that are continuous in memory includes the number of the plurality of memory descriptors that are continuous in memory; After the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors of the memory through the first DMA request, the number of the multiple continuous memory descriptors of the memory is determined according to the memory descriptor, and a second DMA request is determined, where the second DMA request is used to obtain the remaining multiple continuous memory descriptors of the memory.

2. The method according to claim 1, characterized in that The Virtio driver of the smart network card allocates a target memory descriptor chain for any message including: When the Virtio driver of the smart network card allocates a target memory descriptor chain for any message, including multiple memory descriptors that are continuous in memory, the target byte of the memory descriptor with the smallest memory address among the multiple memory descriptors that are continuous in memory is written to the number of the remaining multiple memory descriptors that are continuous in memory.

3. The method according to claim 2, characterized in that After the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors of the memory through the first DMA request, the number of the multiple continuous memory descriptors of the memory is determined according to the memory descriptor, and the second DMA request is determined, including: After the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors of the memory through the first DMA request, the target byte in the memory descriptor is parsed to obtain the number of the remaining multiple continuous memory descriptors of the memory; A second DMA request is determined according to the number of the plurality of memory descriptors remaining continuously in the memory.

4. The method according to claim 3, characterized in that Writing the target byte of the memory descriptor with the smallest memory address among the multiple memory descriptors in the continuous memory to the number of the remaining multiple memory descriptors in the continuous memory includes: The upper 12 bits of the flag field of the memory descriptor with the smallest memory address among the multiple continuous memory descriptors are used as the target byte, and the number of the remaining multiple continuous memory descriptors is written in the upper 12 bits of the flag field of the memory descriptor.

5. The method according to claim 1, characterized in that The Virtio driver of the smart network card allocates a target memory descriptor chain for any message, including: In the process of sending the message, the Virtio driver of the smart network card determines the number of multiple memory descriptors corresponding to the message to be sent according to the size of the message to be sent; Allocating a target memory descriptor chain in a sending descriptor pool according to a descriptor allocation strategy, wherein the number of memory descriptors included in the target memory descriptor chain is the number of multiple memory descriptors corresponding to the message to be sent; The descriptor allocation strategy is used to maximize the number of multiple consecutive memory descriptors when allocating multiple memory descriptors.

6. The method according to claim 1, characterized in that The Virtio driver of the smart network card allocates a target memory descriptor chain for any message, including: Before receiving the message, the Virtio driver of the smart network card determines the maximum memory descriptor chain in the receiving descriptor pool according to the depth of the receiving queue and the descriptor allocation policy; The descriptor allocation strategy is used to maximize the number of multiple consecutive memory descriptors when allocating multiple memory descriptors.

7. The method according to claim 6, characterized in that In the process of receiving the message, the method further includes: According to the size of the message to be received, a target memory descriptor chain corresponding to the message to be received is determined in the maximum memory descriptor chain, and the target memory descriptor chain corresponding to the message to be received includes multiple memory descriptors that are continuous in memory.

8. A data transmission processing device, characterized in that: Applied to a smart network card, the device comprises: A descriptor determination module, used for allocating a target memory descriptor chain for any message by the Virtio driver of the intelligent network card during the process of sending or receiving a message, wherein the target memory descriptor chain includes a plurality of memory descriptors of continuous memory, and the memory descriptor with the smallest memory address among the plurality of memory descriptors of continuous memory includes the number of the plurality of memory descriptors of continuous memory; A request module is used to determine the number of multiple continuous memory descriptors according to the memory descriptor after the hardware of the smart network card obtains the memory descriptor with the smallest memory address among the multiple continuous memory descriptors of the memory through a first DMA request, and determine a second DMA request, wherein the second DMA request is used to obtain the remaining multiple continuous memory descriptors of the memory.

9. A computer device, characterized in that: include: At least one processor, and a memory, wherein the memory stores a computer program that can be run on the processor, wherein when the processor executes the computer program, the processing method of the data transmission process described in any one of claims 1-7 is executed.

10. A non-volatile readable storage medium, characterized in that: The non-volatile readable storage medium stores a computer program, wherein the computer program, when executed by a processor, executes the processing method of the data transmission process according to any one of claims 1 to 7.

Images