Data transmission method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a data transmission method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: storing a first data packet received by the physical machine to a first buffer area in a receiving queue through a DPDK driver deployed in the physical machine; when the DPDK driver receives a data receiving request sent by an application program in the physical machine, moving the first buffer area to a processing queue through the DPDK driver; when the data receiving request comprises data packet receiving information, moving the first buffer area to a sending queue through the DPDK driver so as to send the first data packet to a far-end memory pool for storage; and acquiring the first data packet from the far-end memory pool through a driving service deployed in the physical machine, so that the physical machine performs data processing on the first data packet when running an application program. By adopting the method, the data transmission rate can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of cloud computing, and in particular to a data transmission method, apparatus, computer equipment, readable storage medium and program product. Background Art

[0002] In recent years, a new generation of cloud computing infrastructure has been proposed in the field of cloud computing technology, namely, a computing and storage separation architecture, called converged computing. DPDK (Data Plane Development Kit) is a collection of drivers and function libraries for fast data packet processing, which is used to improve data processing performance and network throughput.

[0003] Therefore, it is possible to consider applying DPDK to aggregate computing to further improve the data transmission rate. Summary of the invention

[0004] Based on this, it is necessary to provide a data transmission method, apparatus, computer equipment, readable storage medium and program product that can improve the data transmission rate in response to the above technical problems.

[0005] In a first aspect, the present application provides a data transmission method, which is applied to a physical machine, and the method includes: storing a first data packet received by the physical machine in a first buffer in a receiving queue through a DPDK driver deployed in the physical machine; when the DPDK driver receives a data reception request sent by an application in the physical machine, moving the first buffer to a processing queue through the DPDK driver; when the data reception request includes reception data packet information, moving the first buffer to a sending queue through the DPDK driver to send the first data packet to a remote memory pool for storage; obtaining the first data packet from the remote memory pool through a driver service deployed in the physical machine, so that the physical machine can perform data processing on the first data packet when running the application.

[0006] In one of the embodiments, the method further includes: when the data reception request includes packet loss information, clearing the first data packet in the first buffer, and moving the first buffer to the sending queue and the receiving queue in sequence through the DPDK driver.

[0007] In one of the embodiments, the method further includes: after the first data packet is sent to the remote memory pool, moving the first buffer in the sending queue to the receiving queue through the DPDK driver.

[0008] In one of the embodiments, a first data packet is obtained from a remote memory pool through a driver service deployed in a physical machine, so that the physical machine can process the first data packet when running an application, including: after receiving a data acquisition instruction from the application, the driver service obtains the first data packet from the remote memory pool based on the data acquisition instruction; when a local buffer bound to a CPU core running the application in the physical machine is in a non-idle state, a second buffer is applied to the remote memory pool through the driver service, and the first data packet is stored in the second buffer, so that the CPU core running the application can obtain the first data packet from the second buffer and process the first data packet.

[0009] In one of the embodiments, the method further includes: when the physical machine runs the application, sending a second data packet to the driver service, sending the second data packet to the DPDK driver through the driver service; sending the second data packet to the network card in the physical machine through the DPDK driver, so as to send the second data packet to the target device through the network card.

[0010] In one of the embodiments, sending a second data packet to a network card in a physical machine through a DPDK driver includes: storing the second data packet in a third buffer in a receiving queue through the DPDK driver; when the DPDK driver receives a data sending request sent by an application, moving the third buffer to a processing queue through the DPDK driver; when the data sending request includes sending data packet information, moving the third buffer to a sending queue through the DPDK driver to send the second data packet to the network card in the physical machine.

[0011] In a second aspect, the present application further provides a data transmission device, applied to a physical machine, the device comprising:

[0012] A storage module, used for storing a first data packet received by the physical machine into a first buffer in a receiving queue through a DPDK driver deployed in the physical machine;

[0013] A first moving module, configured to move the first buffer to a processing queue through the DPDK driver when the DPDK driver receives a data receiving request sent by an application in the physical machine;

[0014] A second moving module is used to move the first buffer to the sending queue through the DPDK driver when the data receiving request includes receiving data packet information, so as to send the first data packet to the remote memory pool for storage;

[0015] The acquisition module is used to acquire the first data packet from the remote memory pool through the driver service deployed in the physical machine, so that the physical machine can perform data processing on the first data packet when running the application program.

[0016] In one of the embodiments, the second moving module is further used to clear the first data packet in the first buffer when the data receiving request includes packet loss information, and move the first buffer to the sending queue and the receiving queue in sequence through the DPDK driver.

[0017] In one of the embodiments, the second moving module is further used to move the first buffer in the sending queue to the receiving queue through the DPDK driver after the first data packet is sent to the remote memory pool.

[0018] In one of the embodiments, the acquisition module is specifically used to drive the service to acquire a first data packet from a remote memory pool based on the data acquisition instruction after receiving the data acquisition instruction of the application; when the local buffer bound to the CPU core running the application in the physical machine is in a non-idle state, apply for a second buffer from the remote memory pool through the driving service, and store the first data packet in the second buffer, so that the CPU core running the application can acquire the first data packet from the second buffer and perform data processing on the first data packet.

[0019] In one of the embodiments, the device also includes a sending module, which is used to send a second data packet to the driver service when the physical machine runs the application, and send the second data packet to the DPDK driver through the driver service; send the second data packet to the network card in the physical machine through the DPDK driver, so as to send the second data packet to the target device through the network card.

[0020] In one of the embodiments, the sending module is specifically used to store the second data packet in the third buffer in the receiving queue through the DPDK driver; when the DPDK driver receives a data sending request sent by the application, the third buffer is moved to the processing queue through the DPDK driver; when the data sending request includes sending data packet information, the third buffer is moved to the sending queue through the DPDK driver to send the second data packet to the network card in the physical machine.

[0021] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0022] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0023] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0024] The above-mentioned data transmission method, apparatus, computer equipment, readable storage medium and program product, first, deploy the DPDK driver in the physical machine, and based on the characteristics of DPDK itself, DPDK can avoid falling into the kernel state during data transmission, resulting in increased interruption time, so deploying the DPDK driver in the physical machine can increase the data transmission rate. Secondly, the first data packet is sent to the remote memory pool for storage. In other words, the remote memory pool is used as the memory pool when DPDK is running, which solves the problem of large memory usage caused by DPDK, which reduces the memory sales rate of the physical machine. In addition, through the DPDK driver deployed in the physical machine, the first data packet received by the physical machine is stored in the first buffer in the receiving queue. When the DPDK driver receives a data receiving request sent by the application in the physical machine, the first buffer is moved to the processing queue through the DPDK driver. When the data receiving request includes receiving data packet information, the first buffer is moved to the sending queue through the DPDK driver to send the first data packet to the remote memory pool for storage. Compared with the DPDK driver in traditional technology, the transmission of data packets between the buffer and the receiving queue and the sending queue is avoided, thereby solving the network delay overhead caused by the physical machine accessing the remote memory pool. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A schematic diagram of a data transmission architecture in an embodiment;

[0027] Figure 2 A schematic diagram of a flow chart of a data transmission method in an embodiment;

[0028] Figure 3 A schematic diagram of the principle of DPDK driving in an embodiment;

[0029] Figure 4 A schematic diagram of the principle of caching a first data packet in an embodiment;

[0030] Figure 5 is a structural block diagram of a data transmission device in one embodiment;

[0031] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] In recent years, the rapid development of general artificial intelligence, cloud computing and other Internet technologies represented by ChatGPT has led to an explosive growth in demand for diversified computing power and promoted the rapid growth of the computing infrastructure industry represented by computing centers. It has provided effective market traction for the development of different types and forms of computing centers such as general, intelligent, supercomputing and edge computing centers, and brought the digital market to a new level with larger scale, stronger computing and professional services.

[0034] The development of artificial intelligence is based on a strong computing power foundation, but due to some special reasons, the training of models through traditional stand-alone servers is greatly limited. The computing center is an infrastructure based on cloud computing technology that allocates computing resources on demand to provide large-scale cluster computing power. In the past, the cloud computing architecture system was based on an integrated architecture of computing, storage and network, which was a vertical expansion of physical machines. The computing power was still limited by the specifications of the physical machine hardware, and there was a huge computing power bottleneck, which gradually did not match the computing power demand of the industrialization of artificial intelligence.

[0035] In order to break through the computing power bottleneck of physical machines, the CXL (Compute Express Link) protocol was later used to connect physical machines in a horizontal expansion manner, aiming to reduce and solve problems such as memory capacity, memory bandwidth, and network throughput latency in high-performance computing. The CXL protocol can also provide memory sharing and memory expansion, providing faster and more flexible data exchange and processing methods. Later, based on the CXL protocol, a new generation of cloud computing infrastructure system - "aggregate computing" was proposed. Aggregate computing adopts a computing and storage separation architecture, and uses the pooling technology based on the CXL protocol to build a "computing pool" and "storage pool" for the original physical machine with integrated computing and storage, providing shared memory for multiple physical machines, and providing high-speed, low-latency memory and I / O (Input / Output) device interconnection, so that the physical machine uses a remote memory pool (remote shared memory) as runtime memory.

[0036] DPDK is a collection of drivers and function libraries for fast data packet processing, which is used to improve data processing performance and network throughput. Therefore, it is possible to consider applying DPDK to aggregate computing to further improve the data transmission rate. However, DPDK will occupy a large amount of memory on the physical machine when it is running. Therefore, this application proposes to use the remote memory pool as the memory pool when DPDK is running, making full use of the massive memory resources of the remote memory pool, and reducing the large memory occupation caused by DPDK, which leads to a reduction in the memory sales rate of the physical machine. However, accessing the remote memory pool inevitably brings about a high latency problem. Therefore, this application proposes to send the data packet directly to the receiving queue, and directly to the remote memory pool through the sending queue, avoiding the transmission of the data packet between the buffer and the receiving queue and the sending queue, simplifying the transmission process of the data packet, thereby solving the network latency overhead caused by accessing the remote memory pool.

[0037] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0038] Before introducing the data transmission method of the present application, the data transmission architecture for implementing the data transmission method is first described for ease of understanding. Figure 1 As shown, the data transmission architecture involves kernel space, interface (system call), user space, and remote memory pool. In kernel space, the DPDK driver uses the UIO (Bypass IO) framework to complete the reading of interrupt information through the corresponding interface provided by the kernel. The processing of data packets is placed in user space to avoid falling into kernel state and causing long interrupt delays. In order to access data efficiently, the memory operation object of the data packet is abstracted as an Mbuf structure, and some network information data packets are encapsulated in the Mbuf structure to specifically store the network frame buffer information and control information cache. Multiple Mbuf structures form a linked list called Re_Mbuf and store it in the memory pool. DPDK uses a circular buffer to store data information, and connects the Re_Mbuf linked list end to end to form a circular lock-free queue structure of Rte_Mempool. This application places the circular queue structure in the memory pool from the original local memory to the remote memory pool. The memory occupied by this part of the data can be called the DPDK memory pool, which reduces the memory size occupied by the DPDK runtime.

[0039] Among them, the remote memory pool is implemented based on the CXL protocol. Since the CXL protocol is based on the PCIe (Peripheral Component Interconnect express) protocol, it runs on the PCIe physical layer and has the same electrical characteristics. The CXL card can be directly inserted into the card slot of the original PCIe card, and the CPU (Central Processing Unit) in the physical machine can use the CXL card through the Flex Bus. The CXL card is transparent to its upper-layer applications, and the CPU cannot sense whether the memory being accessed is local memory or a remote memory pool. This architecture provides high scalability. In this way, applications in the user space can obtain data packets from the remote memory pool through the driver service and send data packets out through the driver service.

[0040] In an exemplary embodiment, Figure 2 As shown, a data transmission method is provided, which is described by applying the method to a physical machine, in which DPDK is installed. The data transmission method includes the following steps 201 to 204. Among them:

[0041] Step 201: A first data packet received by a physical machine is stored in a first buffer in a receiving queue through a DPDK driver deployed in the physical machine.

[0042] Step 202: When the DPDK driver receives a data receiving request sent by an application in the physical machine, the DPDK driver moves the first buffer to a processing queue.

[0043] The data receiving request carries the received data packet information or the discarded data packet information. If the application sends a data receiving request carrying the received data packet information, the DPDK driver is required to store the first data packet in the corresponding place. If the application sends a data receiving request carrying the discarded data packet information, it means that the first data packet is useless, and the DPDK driver is not required to store the first data packet in the corresponding place.

[0044] Step 203, when the data receiving request includes receiving data packet information, the first buffer is moved to the sending queue through the DPDK driver to send the first data packet to the remote memory pool for storage.

[0045] Step 204 , obtaining a first data packet from a remote memory pool through a driver service deployed in the physical machine, so that the physical machine can perform data processing on the first data packet when running an application program.

[0046] Reference Figure 3As shown, a schematic diagram of the principle of a DPDK driver is provided. After the network card in the physical machine receives the first data packet, the DPDK driver obtains the first data packet during the polling operation, and then stores the first data packet in the first buffer in the receiving queue. At this time, the receiving queue can be regarded as a received queue. When the physical machine runs the application, it interacts with the DPDK driver through the driver service, that is, sends a data reception request to the DPDK driver. Among them, the module running the application in the physical machine can be regarded as a network function (also called a network function), so sending a data reception request to the DPDK driver can be regarded as a process of the network function requesting a data packet.

[0047] When the DPDK driver receives the data receiving request, it moves the first buffer storing the first data packet to the processing queue. The module running the DPDK driver in the physical machine can also be regarded as a network function. Therefore, the DPDK driver receiving the data receiving request can be regarded as the network function receiving the data receiving request.

[0048] When the data receiving request carries the receiving data packet information, the first buffer storing the first data packet is moved to the sending queue. As mentioned above, the module running the DPDK driver can be regarded as a network function. Therefore, moving the first buffer storing the first data packet to the sending queue can be regarded as the network function sending the first data packet to the sending queue.

[0049] After the first data packet is sent from the sending queue to the remote memory pool for storage, that is, after the first data packet received by the network card is sent to the remote memory pool, the first buffer in the sending queue is empty, and the sending queue can be regarded as a sent queue. Then, the DPDK driver moves the first buffer in the sending queue to the receiving queue to ensure that the receiving queue is not empty. Then, the DPDK driver loops to perform the above-mentioned process of receiving and sending the first data packet.

[0050] When the data receiving request carries packet loss information, in one possible implementation, the first data packet in the first buffer is cleared by the DPDK driver, and the first buffer from which the first data packet is cleared is moved to the sending queue and the receiving queue in sequence by the DPDK driver, and then the first buffer is recycled to the receiving queue to ensure that the receiving queue is not empty. In another possible implementation, the first data packet in the first buffer is cleared by the DPDK driver, and the first buffer from which the first data packet is cleared is moved directly to the receiving queue by the DPDK driver to ensure that the receiving queue is not empty. As mentioned above, the module running the DPDK driver can be regarded as a network function, and therefore, clearing the first data packet in the first buffer by the DPDK driver can be regarded as a process in which the network function discards the first data packet. Then, the DPDK driver loops through the above-mentioned process of receiving and sending the first data packet.

[0051] Alternatively, the DPDK driver can use multiple transmit rings ( Figure 3 The DPDK driver can set the tail of all transmission rings at the same time and use the earliest head of all rings as the head and connect the tails. When the DPDK driver has multiple outputs, the transmission packet is mapped to the corresponding output and the packet is discarded on all other outputs. Multiple inputs can be achieved by copying multiple processing queues and receiving queues.

[0052] In traditional DPDk drivers, a fixed set of packet buffers is reused to avoid memory allocation overhead. This architecture provides network functions with better flexibility. Buffers can be reserved to reassemble packets, or buffers can be allocated from the pool to respond to non-network events. However, this architecture inevitably brings large overhead, both in terms of memory space and time. The network function introduces packet buffers, which brings additional buffer management overhead and requires the DPDk driver to implement buffer release and allocation strategies, increasing the complexity of the entire system.

[0053] based on Figure 3 It can be seen that the DPDK driver of this application completely deletes the buffer used to transmit the first data packet before the receive queue compared to the traditional DPDk driver. The DPDK driver moves the buffer directly to the receive queue, and the network function must choose to send or discard the received data packet. This simplifies the DPDK driver and avoids the transmission of data packets between the buffer and the receive queue and the send queue. Therefore, the network delay overhead caused by the physical machine accessing the remote memory pool is solved.

[0054] In addition, the DPDK driver of the present application minimizes the state by combining the receiving queue, processing queue, and sending queue into a single logical descriptor ring containing all buffers. The DPDK driver uses a simplified data structure, that is, using a receiving ring and a transmitting ring, and connecting the head of the transmitting ring to the tail of the receiving ring, thereby ensuring that the buffer that has completed the transmission is reused for reception without any intermediate steps. The hardware condition for implementing this process is based on the hardware feature of "empty transmission descriptor", that is, some descriptors in the transmission ring are allowed to fail.

[0055] In the above data transmission method, first, a DPDK driver is deployed in a physical machine, and based on the characteristics of DPDK itself, DPDK can avoid falling into kernel state during data transmission, resulting in increased interruption time, so deploying a DPDK driver in a physical machine can improve the data transmission rate. Secondly, the first data packet is sent to a remote memory pool for storage. In other words, the remote memory pool is used as a memory pool when DPDK is running, which solves the problem that the large memory occupation caused by DPDK leads to a reduction in the memory sales rate of the physical machine. In addition, through the DPDK driver deployed in the physical machine, the first data packet received by the physical machine is stored in the first buffer in the receiving queue. When the DPDK driver receives a data receiving request sent by an application in the physical machine, the first buffer is moved to the processing queue through the DPDK driver. When the data receiving request includes receiving data packet information, the first buffer is moved to the sending queue through the DPDK driver to send the first data packet to the remote memory pool for storage. Compared with the DPDK driver in the traditional technology, the transmission of data packets between the buffer and the receiving queue and the sending queue is avoided, thereby solving the network delay overhead caused by the physical machine accessing the remote memory pool.

[0056] In an exemplary embodiment, step 204 obtains the first data packet from the remote memory pool through the driver service deployed in the physical machine, so that the physical machine can process the first data packet when running the application program, including step 2041 and step 2042, wherein:

[0057] Step 2041: After receiving the data acquisition instruction from the application program, the driver service acquires a first data packet from the remote memory pool based on the data acquisition instruction.

[0058] In other words, when the physical machine runs an application and needs to obtain the first data packet, the implementation method is to read the first data packet from the remote memory pool through the driver service.

[0059] Step 2042, when the local buffer bound to the CPU core running the application in the physical machine is in a non-idle state, apply for a second buffer from the remote memory pool through the driver service, and store the first data packet in the second buffer, so that the CPU core running the application can obtain the first data packet from the second buffer and process the first data packet.

[0060] After the memory running DPDK (DPDK memory pool) is placed in the remote memory pool, the cost of multi-core access to the free buffer of the memory pool is high because the CPU needs a CAS (Compare And Set) operation for each access. In addition, since the DPDK memory pool has been placed remotely, the frequency of CPU access to the remote memory pool will also increase. Therefore, in order to reduce the cost of multi-core access to the free buffer of the memory pool and reduce the frequency of CPU access to the remote memory pool, the present application proposes to bind a local buffer (per-core-cache) to each CPU core in the physical machine, and each local buffer contains several Re_Mbuf structures (obj). In the process of taking the first data packet, each CPU core gives priority to accessing its own local buffer. If the local buffer is not idle, that is, when the local memory is insufficient, the CPU core applies to the remote memory pool for a second buffer, that is, to apply for additional space to cache the first data packet, refer to Figure 4 shown.

[0061] The above describes in detail the process of the first data packet being received by the network card and then being sent to the application program. The following describes in detail the process of the second data packet being sent from the application program and then being sent by the network card.

[0062] In an exemplary embodiment, the method further includes: sending a second data packet to a driver service when the physical machine runs an application, and sending the second data packet to a DPDK driver through the driver service; sending the second data packet to a network card in the physical machine through the DPDK driver, so as to send the second data packet to a target device through the network card.

[0063] Among them, when the physical machine runs the application, it sends the second data packet to the driver service, and the driver service sends the second data packet to the DPDK driver. This process is implemented through Figure 1 The interface in is implemented.

[0064] Sending the second data packet to the network card in the physical machine through the DPDK driver includes: storing the second data packet in a third buffer in a receiving queue through the DPDK driver; when the DPDK driver receives a data sending request sent by an application, moving the third buffer to a processing queue through the DPDK driver; when the data sending request includes sending data packet information, moving the third buffer to a sending queue through the DPDK driver to send the second data packet to the network card in the physical machine.

[0065] Continue to refer to Figure 3 After obtaining the second data packet, the DPDK driver stores the second data packet in the third buffer in the receiving queue. At this time, the receiving queue can be regarded as a received queue. When the DPDK driver receives a data sending request, it moves the third buffer storing the second data packet to the processing queue.

[0066] When the data transmission request carries the information of sending a data packet, the third buffer storing the second data packet is moved to the transmission queue. When the second data packet is sent from the transmission queue to the network card for transmission, the third buffer in the transmission queue is empty, and the transmission queue can be regarded as a transmitted queue. Then, the third buffer in the transmission queue is moved to the receiving queue through the DPDK driver to ensure that the receiving queue is not empty. Then, the DPDK driver loops and executes the above-mentioned process of receiving and sending the second data packet.

[0067] When the data transmission request carries packet loss information, in one possible implementation, the second data packet in the third buffer is cleared by the DPDK driver, and the third buffer from which the second data packet is cleared is moved to the transmission queue and the reception queue in sequence by the DPDK driver, and then the third buffer is recycled to the reception queue to ensure that the reception queue is not empty. In another possible implementation, the second data packet in the third buffer is cleared by the DPDK driver, and the third buffer from which the second data packet is cleared is moved directly to the reception queue by the DPDK driver to ensure that the reception queue is not empty.

[0068] In summary, the existing DPDK memory pools are all based on the memory of the physical machine, which inevitably leads to a large amount of memory occupation, resulting in a low memory resource sales rate and restricting the operation of other applications. Under the aggregated computing architecture, this application uses the CXL protocol to specify the process of DPDK applying for memory pool space to the remote memory pool, so that the remote memory pool is used as the memory pool when DPDK is running, which effectively reduces DPDK's memory occupation on the physical machine and increases the local memory sales space and usage space. In order to improve the efficiency of network communication, the memory distribution is optimized, and the local cache on the physical machine is set according to the CPU core, avoiding the blocking and concurrency conflict problems caused by multi-core concurrent access, and effectively improving the stability and reliability of the system.

[0069] In addition, in order to solve the network latency overhead caused by accessing the remote memory pool, this application designs a new DPDK driver based on the application scenario characteristics of aggregate computing network communication. The DPDK driver discards the network buffer part of the traditional DPDK driver when transmitting network data packets on the network card side. It effectively improves the forwarding efficiency of network data packets by reducing unnecessary copy processes of data packets and reduces the latency overhead of network transmission.

[0070] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0071] Based on the same inventive concept, the embodiment of the present application also provides a data transmission device for implementing the data transmission method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data transmission device embodiments provided below can refer to the limitations on the data transmission method above, and will not be repeated here.

[0072] In an exemplary embodiment, Figure 5 As shown, a data transmission device is provided, which is applied to a physical machine, and includes: a storage module 501, a first movement module 502, a second movement module 503 and an acquisition module 504, wherein:

[0073] The storage module 501 is used to store the first data packet received by the physical machine into the first buffer in the receiving queue through the DPDK driver deployed in the physical machine;

[0074] A first moving module 502 is configured to move the first buffer to a processing queue through the DPDK driver when the DPDK driver receives a data receiving request sent by an application in the physical machine;

[0075] The second moving module 503 is used to move the first buffer to the sending queue through the DPDK driver when the data receiving request includes receiving data packet information, so as to send the first data packet to the remote memory pool for storage;

[0076] The acquisition module 504 is used to acquire the first data packet from the remote memory pool through the driver service deployed in the physical machine, so that the physical machine can perform data processing on the first data packet when running the application program.

[0077] In one of the embodiments, the second moving module 503 is further used to clear the first data packet in the first buffer when the data receiving request includes packet loss information, and move the first buffer to the sending queue and the receiving queue in sequence through the DPDK driver.

[0078] In one embodiment, the second moving module 503 is further configured to move the first buffer in the sending queue to the receiving queue through the DPDK driver after the first data packet is sent to the remote memory pool.

[0079] In one of the embodiments, the acquisition module 504 is specifically used to drive the service to acquire a first data packet from a remote memory pool based on the data acquisition instruction after receiving the data acquisition instruction of the application; when the local buffer bound to the CPU core running the application in the physical machine is in a non-idle state, apply for a second buffer from the remote memory pool through the driving service, and store the first data packet in the second buffer, so that the CPU core running the application can acquire the first data packet from the second buffer and perform data processing on the first data packet.

[0080] In one of the embodiments, the device also includes a sending module, which is used to send a second data packet to the driver service when the physical machine runs the application, and send the second data packet to the DPDK driver through the driver service; send the second data packet to the network card in the physical machine through the DPDK driver, so as to send the second data packet to the target device through the network card.

[0081] In one of the embodiments, the sending module is specifically used to store the second data packet in the third buffer in the receiving queue through the DPDK driver; when the DPDK driver receives a data sending request sent by the application, the third buffer is moved to the processing queue through the DPDK driver; when the data sending request includes sending data packet information, the third buffer is moved to the sending queue through the DPDK driver to send the second data packet to the network card in the physical machine.

[0082] Each module in the above data transmission device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0083] In an exemplary embodiment, a computer device is provided. The computer device may be a physical machine, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as a first data packet and a second data packet. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data transmission method is implemented.

[0084] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0085] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of any one of the method embodiments described above are implemented.

[0086] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above method embodiments are implemented.

[0087] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of any one of the method embodiments described above.

[0088] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0089] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0090] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data transmission method, characterized in that: Applied to a physical machine, the method comprises: By means of a DPDK driver deployed in the physical machine, a first data packet received by the physical machine is stored in a first buffer in a receiving queue; When the DPDK driver receives a data receiving request sent by the application in the physical machine, the DPDK driver moves the first buffer to a processing queue; When the data receiving request includes receiving data packet information, the first buffer is moved to a sending queue by the DPDK driver to send the first data packet to a remote memory pool for storage; The first data packet is obtained from the remote memory pool through the driver service deployed in the physical machine, so that the first data packet is processed when the physical machine runs the application program.

2. The method according to claim 1, characterized in that The method further comprises: When the data receiving request includes packet loss information, the first data packet in the first buffer is cleared, and the first buffer is moved to the sending queue and the receiving queue in sequence through the DPDK driver.

3. The method according to claim 1, characterized in that The method further comprises: After the first data packet is sent to the remote memory pool, the first buffer in the sending queue is moved to the receiving queue through the DPDK driver.

4. The method according to claim 1, characterized in that: The acquiring the first data packet from the remote memory pool through the driver service deployed in the physical machine so as to process the first data packet when the physical machine runs the application program includes: After receiving the data acquisition instruction of the application, the driver service acquires the first data packet from the remote memory pool based on the data acquisition instruction; When the local buffer bound to the CPU core running the application in the physical machine is in a non-idle state, a second buffer is applied for from the remote memory pool through the driver service, and the first data packet is stored in the second buffer, so that the CPU core running the application can obtain the first data packet from the second buffer and perform data processing on the first data packet.

5. The method according to claim 1, characterized in that: The method further comprises: When the physical machine runs the application, it sends a second data packet to the driver service, and sends the second data packet to the DPDK driver through the driver service; The second data packet is sent to the network card in the physical machine through the DPDK driver, so that the second data packet is sent to the target device through the network card.

6. The method according to claim 5, characterized in that The sending the second data packet to the network card in the physical machine through the DPDK driver includes: The DPDK driver stores the second data packet in a third buffer in the receiving queue; When the DPDK driver receives the data sending request sent by the application, the DPDK driver moves the third buffer to the processing queue; When the data sending request includes sending data packet information, the third buffer is moved to a sending queue through the DPDK driver to send the second data packet to the network card in the physical machine.

7. A data transmission device, characterized in that: Applied to a physical machine, the device comprises: A storage module, configured to store a first data packet received by the physical machine into a first buffer in a receiving queue through a DPDK driver deployed in the physical machine; A first moving module, configured to move the first buffer to a processing queue through the DPDK driver when the DPDK driver receives a data receiving request sent by the application in the physical machine; A second moving module, configured to move the first buffer to a sending queue through the DPDK driver when the data receiving request includes receiving data packet information, so as to send the first data packet to a remote memory pool for storage; The acquisition module is used to acquire the first data packet from the remote memory pool through the driver service deployed in the physical machine, so that the physical machine can perform data processing on the first data packet when running the application program.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images