Data transmission methods and system

Abstract

Disclosed are data transmission methods and a system, which relate to the field of computers. A method comprises: a receiver transmits a data storage address and completion notification information to a transmitter; on the basis of the data storage address, the transmitter stores data to the receiver by means of a network interface card of the receiver, wherein, when transmitting a last data packet of the data, the transmitter transmits the completion notification information to the network interface card; and, once the last data packet has been stored to the receiver, the network interface card transmits the completion notification information to the receiver, so as to notify the receiver of completion of data transfer. The transmitter negotiates the completion notification information with the receiver in advance, and when transmitting the last data packet of the data, the transmitter transmits the completion notification information to the network interface card, and thus, once the last data packet has been stored to the receiver, the network interface card transmits the completion notification information to the receiver, so as to notify the receiver of completion of data transfer, eliminating the need to additionally transmit notification messages, thereby simplifying communication processes, and improving the communication efficiency.

Description

Data transmission method and system

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on January 26, 2024, with application number 202410120432.X and application name “Data Transmission Method and System”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computers, and in particular to a data transmission method and system. Background Art

[0003] Currently, communication between the source process on the sending end and the destination process on the receiving end primarily relies on the Message Passing Interface (MPI) for data exchange. To increase data transmission rates, data transmission between the sending and receiving ends can be based on Remote Direct Memory Access (RDMA) technology to reduce data transmission latency. Because both RDMA write and RDMA read operations are one-way RDMA communications, the receiving end of the data is unaware of the data. For example, if the sending end performs a write operation on the receiving end and the receiving end is unaware, the sending end needs to send an additional notification message to inform the receiving end that the data exchange is complete. Alternatively, if the receiving end performs a read operation on the sending end and the sending end is unaware, the receiving end needs to send an additional notification message to inform the sending end that the data exchange is complete. Since an additional notification message is required after data transfer between the sending and receiving ends, the communication process between the sending and receiving ends becomes complex, reducing communication efficiency. Summary of the Invention

[0004] The present application provides a data transmission method and system, thereby simplifying the communication process and effectively improving communication efficiency.

[0005] In a first aspect, a data transmission method is provided, comprising: a receiving end sending a data storage address and completion notification information to a sending end, and the sending end storing the data to the receiving end through the network card of the receiving end according to the data storage address, wherein when the sending end sends the last data packet in the data, the completion notification information is sent to the network card; after the network card stores the last data packet to the receiving end, the completion notification information is sent to the receiving end to notify the receiving end that the data move is completed.

[0006] Compared to the sending end sending an additional notification message to the receiving end to indicate the completion of data migration, which results in a complicated communication process and low communication efficiency, the solution provided by the present application is that the sending end and the receiving end negotiate the completion notification information in advance. When the sending end sends the last data packet in the data, the completion notification information is sent to the network card. After the network card stores the last data packet in the receiving end, it sends the completion notification information to the receiving end to notify the receiving end that the data migration is complete. That is, there is no need to send additional notification messages, which simplifies the communication process and improves communication efficiency.

[0007] In one possible implementation, the sending end and the receiving end communicate via the RDMA protocol, and the receiving end sends the data storage address and completion notification information to the sending end, including: the receiving end carries the completion notification information in a reserved field of the negotiation message of the data storage address, and sends the negotiation message to the sending end.

[0008] Therefore, the completion notification information is carried in the reserved field of the negotiation message without changing the format of the message, reducing the modification of the protocol, improving the compatibility of the protocol, simplifying the communication process, and improving communication efficiency.

[0009] In another possible implementation, the sending end sends the last data packet of the data, and carries the completion notification information in a reserved field of the data packet.

[0010] In another possible implementation, the method further includes: after receiving the data packet, the network card obtains completion notification information from a reserved field of the data packet.

[0011] Therefore, the completion notification information is carried in the reserved field of the business message, which reduces the modification of the protocol, improves the compatibility of the protocol, eliminates the need to send additional notification messages, simplifies the communication process, and improves communication efficiency.

[0012] In another possible implementation, the completion notification information includes a notification address and a notification value, the notification address is the address of the receiving end where the notification value is written, and the notification value is used to indicate that the data move is complete; the method also includes: the network card obtains the completion notification information from the data packet and writes the notification value into the notification address, and the receiving end confirms that the data move is complete by obtaining the notification value from the notification address.

[0013] In a second aspect, a data transmission method is provided, which includes: a sending end sends a data storage address and completion notification information to a receiving end, and the receiving end reads data through the network card of the sending end according to the data storage address, wherein when the receiving end sends the last data packet in the read data, the completion notification information is sent to the network card of the sending end; after the network card sends the data read by the last data packet to the receiving end, the network card sends the completion notification information to the sending end to notify the sending end that the data move is completed.

[0014] In one possible implementation, the sending end and the receiving end communicate via the RDMA protocol, and the sending end sends the data storage address and completion notification information to the receiving end, including: the sending end carries the completion notification information in a reserved field of the negotiation message of the data storage address, and sends the negotiation message to the receiving end.

[0015] In another possible implementation, the last data packet corresponding to the data is sent at the receiving end, and the completion notification information is carried in a reserved field of the data packet.

[0016] In another possible implementation, the method further includes: after receiving the data packet, the network card at the sending end obtains completion notification information from a reserved field of the data packet.

[0017] In another possible implementation, the completion notification information includes a notification address and a notification value, the notification address is the address of the receiving end where the notification value is written, and the notification value is used to indicate that the data move is complete; the method also includes: the network card of the sending end obtains the completion notification information from the data packet and writes the notification value into the notification address, and the sending end confirms that the data move is complete by obtaining the notification value from the notification address.

[0018] In a third aspect, a data transmission method is provided, which is executed by a sending end, and the method includes: receiving a first negotiation message, the first negotiation message including a notification value and a first notification address, and the first value of the notification value is used to indicate that the data movement is completed; sending a first data packet to a receiving end, the first data packet is used to indicate that the first data of the first service is written to the receiving end, the first data packet includes the first data of the first service, a notification value and a first notification address, the first notification address indicates the storage space of the receiving end, and the storage space of the receiving end is used to store the notification value.

[0019] In a possible implementation, the notification value and the first notification address are located in a reserved field of the first data packet.

[0020] In another possible implementation, the first data packet further includes a first storage address and access rights, wherein the first storage address indicates that the storage space of the receiving end is used to store the first data of the first service, and the access rights indicate that access to the storage space indicated by the first storage address is allowed.

[0021] When a large amount of data of the first service is transmitted between the transmitting end and the receiving end, the data of the first service may be transmitted through multiple data packets.

[0022] In another possible implementation, before sending the first data packet to the receiving end, the method further includes: sending a second data packet to the receiving end, the second data packet being used to indicate writing second data of the first service to the receiving end, the second data packet including the second data of the first service.

[0023] In another possible implementation, the second data packet further includes a notification value and a first notification address, and the second value of the notification value is used to indicate that the data migration is incomplete.

[0024] During the process of transmitting data of the first service between the sending end and the receiving end, a notification value may be carried in the intermediate data packet to indicate that the data movement is not completed, so that the receiving end can be informed of the data movement status in a timely manner.

[0025] In another possible implementation, the method also includes: sending a second negotiation message, the second negotiation message includes a notification value and a second notification address; receiving a third data packet sent by the receiving end, the third data packet is used to indicate that the first data of the second service is read from the sending end, the third data packet includes a notification value and a second notification address, and the first value of the notification value is used to indicate that the data move is completed; storing the notification value in the storage space of the sending end indicated by the second notification address; sending a fourth data packet to the receiving end, the fourth data packet includes the first data of the second service.

[0026] The data movement status is promptly learned based on the notification value carried in the data packet, that is, the data contained in the notification message is merged into the data packet, eliminating the need to receive additional notification messages, simplifying the communication process and improving communication efficiency.

[0027] In another possible implementation, the third data packet also includes a second storage address and access rights, the second storage address indicates the storage space of the sending end, and the access rights are used to indicate that access to the storage space indicated by the second storage address is allowed; the method also includes: reading the first data of the second business from the storage space indicated by the second storage address.

[0028] Therefore, according to the storage address of the storage space of the sending end contained in the data packet, the receiving end can directly perform a read operation on the storage space of the sending end, thereby improving the data transmission rate.

[0029] In another possible implementation, before receiving the third data packet sent by the receiving end, the method also includes: receiving a fifth data packet sent by the receiving end, the fifth data packet is used to indicate that the second data of the second service is read from the sending end, the fifth data packet includes a notification value and a second notification address, and the second value of the notification value is used to indicate that the data movement is not completed; sending a sixth data packet to the receiving end, the sixth data packet includes the second data of the second service.

[0030] During the process of transmitting the data of the second service between the sending end and the receiving end, a notification value may be carried in the intermediate data packet to indicate that the data movement is not completed, so that the sending end can be informed of the data movement status in a timely manner.

[0031] In a fourth aspect, a data transmission method is provided, which is executed by a receiving end, and the method includes: sending a first negotiation message, the first negotiation message including a notification value and a first notification address; receiving a first data packet sent by the sending end, the first data packet is used to indicate that the first data of the first service is written to the receiving end, the first data packet includes the first data of the first service, a notification value and a first notification address, and the first value of the notification value is used to indicate that the data transfer is completed; and storing the notification value in the storage space of the receiving end indicated by the first notification address.

[0032] In a possible implementation, the notification value and the first notification address are located in a reserved field of the first data packet.

[0033] In another possible implementation, the first data packet also includes a first storage address and access rights, the first storage address indicates the storage space of the receiving end, and the access rights are used to indicate that access to the storage space indicated by the first storage address is allowed; the method also includes: writing the first data of the first business into the storage space indicated by the first storage address.

[0034] In another possible implementation, before receiving the first data packet sent by the sending end, the method further includes: receiving a second data packet sent by the sending end, the second data packet is used to indicate writing the second data of the first service to the receiving end, and the second data packet includes the second data of the first service.

[0035] In another possible implementation, the second data packet further includes a notification value and a first notification address, and the second value of the notification value is used to indicate that the data migration is incomplete.

[0036] In another possible implementation, the method also includes: receiving a second negotiation message, the second negotiation message including a notification value and a second notification address; sending a third data packet to the sending end, the third data packet is used to indicate that the first data of the second service is read from the sending end, the third data packet includes a notification value and a second notification address, the first value of the notification value is used to indicate that the data move is completed, the second notification address indicates the storage space of the sending end, and the storage space of the sending end is used to store the notification value; receiving a fourth data packet sent by the sending end, the fourth data packet includes the first data of the second service.

[0037] In another possible implementation, the third data packet also includes a second storage address and access rights, where the second storage address indicates that the storage space of the sending end is used to store the first data of the second service, and the access rights are used to indicate that access to the storage space indicated by the second storage address is allowed.

[0038] In another possible implementation, before sending the third data packet to the sending end, the method also includes: sending a fifth data packet to the sending end, the fifth data packet is used to indicate that the second data of the second service is read from the sending end, the fifth data packet includes a notification value and a second notification address, and the second value of the notification value is used to indicate that the data movement is not completed; receiving a sixth data packet sent by the sending end, the sixth data packet includes the second data of the second service.

[0039] In a fifth aspect, a network card is provided, which is used to execute the operating steps of the method of the sending end or the receiving end in the above aspects or any possible implementation of the aspects.

[0040] In a sixth aspect, a computer device is provided, which includes a memory, a processor and a network card, wherein the memory is used to store a set of computer instructions; when the processor executes a set of computer instructions, the processor jointly with the network card executes the operating steps of the method of the sending end or the receiving end in the above aspects or any possible implementation of the aspects.

[0041] In a seventh aspect, a data transmission system is provided, which includes multiple computing nodes, and the computing nodes execute the operating steps of the method of the sending end or the receiving end in the above aspects or any possible implementation of the aspects.

[0042] In an eighth aspect, a computer-readable storage medium is provided, comprising: computer software instructions; when the computer software instructions are executed in a processor, the processor executes the operating steps of the method described at the sending end or the receiving end in the above aspects or any possible implementation of the aspects.

[0043] In the ninth aspect, a computer program product is provided. When the computer program product runs on a computer, it enables the computer to execute the operating steps of the method described at the sending end or the receiving end in the above aspects or any possible implementation of the aspects.

[0044] The technical effects brought about by any design method in the second to ninth aspects can be referred to the technical effects brought about by the first aspect or different design methods in the first aspect, and will not be repeated here.

[0045] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] FIG1 is a schematic diagram of a data transmission provided by the present application;

[0047] FIG2 is a schematic diagram of the architecture of a data transmission system provided by the present application;

[0048] FIG3 is a flow chart of a data transmission method provided by the present application;

[0049] FIG4 is a flow chart of a data transmission method provided by the present application;

[0050] FIG5 is a schematic diagram of receiving and sending a notification value provided by the present application;

[0051] FIG6 is a schematic structural diagram of a data transmission device provided by the present application;

[0052] FIG7 is a schematic structural diagram of a computer device provided in this application. DETAILED DESCRIPTION

[0053] To facilitate understanding, the main terms involved in this application are first explained.

[0054] Message Passing Interface (MPI): is a cross-language communication protocol that can run on a variety of parallel computing architectures. Message passing programs can be written in C, C++, and Fortran. MPI can be used to exchange data between a sending process and a receiving process, for example. MPI supports point-to-point communication and multi-point communication. Point-to-point communication refers to a method in which a source process communicates with a destination process. The source process is used to send data. The destination process is used to receive data. Multi-point communication includes one-to-many communication, many-to-one communication, and many-to-many communication. One-to-many communication refers to a method in which a source process communicates with multiple destination processes. Many-to-one communication refers to a method in which multiple source processes communicate with one destination process. Many-to-many communication refers to a method in which multiple source processes communicate with multiple destination processes. The goals of MPI are high performance, scalability, and portability.

[0055] Remote Direct Memory Access (RDMA) is a technology that bypasses the remote host's operating system and directly accesses the storage media of a remote host. This technology moves data from one system to another's memory without kernel intervention. This eliminates the overhead of copying storage media and context switching, conserving processor resources, improving system throughput, and reducing network communication latency.

[0056] One-side communication: A transmission method in the RDMA protocol that is suitable for transmitting large amounts of data without the other end's awareness. For example, one-side communication includes RDMA write operations and RDMA read operations.

[0057] RDMA device: A device that processes RDMA packets, used to assemble, send, receive, and process RDMA packets. For example, RDMA devices include host channel adapters and network interface cards (NICs). Network interface cards are also known as NICs.

[0058] An HPC cluster refers to a computer cluster system. An HPC cluster consists of multiple computers connected together using various interconnect technologies. Interconnect technologies can be, for example, IB, RoCE, or TCP. HPC provides ultra-high floating-point computing power, which can be used to solve the computing needs of businesses such as compute-intensive and massive data processing. The combined computing power of multiple computers connected together can be used to handle large-scale computing problems. For example, industries such as scientific research, weather forecasting, simulation experiments, biopharmaceuticals, gene sequencing, and image processing use HPC clusters to solve large-scale computing problems and computing needs. Using HPC clusters to handle large-scale computing problems can effectively shorten the computing time for processing data and improve computing accuracy. Typically, the management node in an HPC cluster can decompose computing tasks and assign the decomposed computing tasks to multiple computing nodes, which then complete the computing tasks in parallel.

[0059] Typically, HPC applications or benchmarks launch many processes. When multiple processes exchange data, they use MPI multipoint or point-to-point communication, such as MPI_Bcast, MPI_Reduce, or MPI_Send. Services for transferring large amounts of data between processes include neighbor exchange in applications, panel broadcasts in the High Performance Linpack (HPL) bench, and row exchange. The RNDV protocol can be used to transfer large amounts of data between processes. This protocol uses RDMA unilateral communication for data movement and RDMA bilateral communication for synchronous notification of data movement status. The need to send additional notification messages to indicate the completion of data movement complicates the communication process and reduces communication efficiency.

[0060] For example, as shown in Figure 1, a schematic diagram of data transmission provided by the present application. As shown in (a) in Figure 1, S110, the receiving end sends a control message to the sending end, and the control message is used to authorize the receiving end's storage medium access rights. For example, the control message includes the remote address and access rights of the receiving end. S120, the sending end performs an RDMA write operation on the receiving end. For example, the sending end sends a business message to the receiving end, and the business message includes data, remote address and access rights. S130, the sending end sends a synchronization message to the receiving end to indicate that the data move is complete. As shown in (b) in Figure 1, the business message supports RDMA write commands that carry immediate values ​​(such as: rdma write with immediate). In addition to carrying data to the receiving end, it can also carry other information, such as carrying data move completion information, and generating a completion queue entry (Completion Queue Entry, CQE) at the receiving end. After polling to cqe, the receiving end can perform subsequent operations on the corresponding immediate value content. However, the receiver generates a CQE based on the RDMA write with immediate request. After polling the CQE, the receiver takes the next action based on the content in the CQE. This requires CPU participation. However, the arrival timing of the CQE is uncertain, so the receiver can only poll indiscriminately. Non-target CQEs that arrive during the process need to be temporarily stored, and tag matching must be performed on the target CQE. This increases the complexity of hardware offload and affects performance.

[0061] In order to solve the problem of sending additional notification messages to indicate that data movement is completed, which leads to complicated communication processes and low communication efficiency, the present application provides a data transmission method, that is, carrying completion notification information in a data packet, that is, when the sending end sends the last data packet in the data, the completion notification information is sent to the network card to indicate that the data movement is completed, so that after the receiving end receives the last data packet, it can know that the data movement is completed according to the completion notification information, that is, the completion notification information contained in the notification message is merged into the business message, and there is no need to send additional notification messages, which simplifies the communication process and improves communication efficiency.

[0062] The implementation of the data transmission method provided in this application is described in detail below with reference to the accompanying drawings.

[0063] FIG2 is a schematic diagram of the architecture of a data transmission system provided by the present application. As shown in FIG2 , the data transmission system 200 includes a client 210 , a computing cluster 220 , and a storage cluster 230 .

[0064] Storage cluster 230 includes multiple storage nodes 231. A storage node 231 includes one or more controllers, a network card, and multiple hard disks. Hard disks are used to store data. Hard disks can be magnetic disks or other types of storage media, such as solid-state drives or shingled magnetic recording hard disks. The network card is used to communicate with computing nodes 221 included in computing cluster 220. The controller is used to write data to or read data from the hard disks based on read / write data requests sent by computing nodes 221. During the data reading and writing process, the controller needs to convert the addresses carried in the read / write data requests into addresses that the hard disks can recognize.

[0065] The computing cluster 220 includes multiple computing nodes 221. The computing node 221 can be a computing device, such as an accelerator card, a server, etc.

[0066] In some embodiments, computing cluster 220 may be a heterogeneous computing architecture to provide high-performance computing. For example, computing cluster 220 may include computing units with computing capabilities, such as a central processing unit (CPU), a graphics processing unit (GPU), a data processing unit (DPU), a neural processing unit (NPU), and an embedded neural-network processing unit (NPU), to provide high-performance computing.

[0067] In other embodiments, multiple computing nodes 221 are connected through network devices (such as switches, network cards, etc.) based on high-speed interconnection technology, so that the multiple computing nodes 221 can communicate with each other.

[0068] Client 210 communicates with computing cluster 220 and storage cluster 230 via network 240. For example, client 210 sends a request to computing cluster 220 via network 240, requesting that computing cluster 220 perform model training. Network 240 can be an internal enterprise network (e.g., a local area network (LAN)) or the Internet. Client 210 refers to a computer connected to network 240, also known as a workstation. Different clients can share network resources (e.g., computing resources and storage resources).

[0069] In some embodiments, computing cluster 220 also includes a control node 222. For example, the control node and the computing nodes can be independent physical devices. In another example, the control node and multiple computing nodes can be located on the same physical device. The control node can be a CPU. The multiple computing nodes include computing units such as GPUs, NPUs, and DPUs. Control node 222 is used to manage and allocate tasks, allowing multiple computing nodes to execute multiple tasks in parallel to increase data processing speed.

[0070] In this application, inter-process communication can be performed between computing nodes 221. Specifically, data can be transferred between computing nodes 221 based on RDMA unilateral operations to increase the data transfer rate between computing nodes 221. Furthermore, when the last data packet is transferred between computing nodes 221, a completion notification message is sent to the peer end so that the peer end is informed of the data transfer completion in a timely manner.

[0071] In other embodiments, client 210 is installed with client program 211. Client 210 runs client program 211 to display a user interface (UI). User 250 operates the UI to submit a request. For example, after user 250 operates the UI to submit a request, control node 222 receives the request, loads data from storage cluster 230, and distributes the data to multiple compute nodes.

[0072] Optionally, the system administrator 260 may call an application platform interface (API) 212 or a command-line interface (CLI) 213 through the client 210 to configure system information, such as the notification policy of the notification value provided in this application.

[0073] Figure 2 is merely a schematic diagram. The embodiments of this application do not limit the device connection method or number of devices in the data transmission system. For example, the data transmission system may include multiple clients. A client may connect to multiple computing nodes. Different clients may establish connections with different computing nodes.

[0074] Next, the data transmission process is described in detail with reference to the accompanying drawings.

[0075] Figure 3 is a flow chart of a data transmission method provided by this application. This description uses the inter-process communication between computing nodes shown in Figure 2 as an example. The method primarily involves communication between a sender and a receiver using the RDMA protocol. The method illustrates the sender directly performing a write operation on the receiver's storage medium based on an RDMA write operation.

[0076] Step 310: The receiving end sends a negotiation message to the sending end.

[0077] Before inter-process communication between the receiving and sending ends, the receiving processor can request a memory block from the operating system to store a notification value. The receiving end then notifies the sending end of the notification value and the notification address. The notification address is the address where the notification value is written to the receiving end's memory block. The notification value indicates the completion of the data transfer.

[0078] For example, the receiving end sends a negotiation message to the sending end, and the negotiation message includes completion notification information. The completion notification information includes a notification address and a notification value. In some embodiments, the completion notification information is located in a reserved field of the negotiation message.

[0079] The negotiation message also includes a data storage address, which is used to indicate a storage space on the receiving end for storing data moved from the sending end.

[0080] Step 320: The sending end sends at least one data packet to the receiving end.

[0081] The sending end receives the negotiation message sent by the receiving end, obtains completion notification information from the negotiation message, and stores the completion notification information in a storage medium of the sending end. For example, the sending end obtains the completion notification information from a reserved field in the negotiation message and stores the completion notification information in a network interface card (NIC) or other storage medium of the sending end. For example, the storage medium includes any one of a cache, a memory, or a flash memory.

[0082] When the sender needs to write data to the receiver, the sender directly performs a write operation on the receiver's storage medium based on the RDMA write operation, that is, the sender stores the data to the receiver through the receiver's network card according to the data storage address carried by the negotiation message.

[0083] The network card on the sending end can send a data packet to the network card on the receiving end. The data packet contains the data to be written, the data storage address carried in the negotiation message, and access permissions. The access permissions indicate that access to the storage space indicated by the data storage address is permitted. If the sending end can complete the migration of the data to be written by sending a single data packet to the receiving end, the sending end does not need to send another data packet to the receiving end. If the sending end cannot complete the migration of the data to be written by sending a single data packet to the receiving end, the sending end can send multiple data packets to the receiving end to migrate the data to be written from the sending end to the receiving end. The data contained in the multiple data packets can be different.

[0084] It should be noted that when a transmitting end sends a data packet to a receiving end, the data packet includes completion notification information. When a transmitting end sends the last data packet of multiple data packets to a receiving end, the last data packet also includes completion notification information. Completion notification information is carried in a reserved field of the data packet. Completion notification information includes a notification address and a notification value. The notification address is the address where the notification value is written to the receiving end's storage space. The notification value indicates the completion of the data transfer.

[0085] Step 330: The network card at the receiving end writes the data into the storage space indicated by the data storage address.

[0086] The receiving end receives the data packet sent by the sending end, and the network card of the receiving end writes the data in the data packet into the storage space indicated by the data storage address.

[0087] Step 340: After storing the last data packet at the receiving end, the network card at the receiving end sends a completion notification message to the receiving end to notify the receiving end that the data transfer is complete.

[0088] After the receiving end's network card writes the data to the storage space indicated by the data storage address, it can obtain completion notification information from the reserved field in the data packet and store the notification value in the receiving end's storage space indicated by the notification address. The receiving end's storage space includes the storage space of the storage medium in the receiving end or the storage space of the storage medium in the network card.

[0089] The processor at the receiving end can obtain the notification value from the storage space at the receiving end to confirm that the data move is completed.

[0090] Optionally, if the sending end needs to send multiple data packets to the receiving end, before the sending end sends the last data packet to the receiving end, the other data packets sent by the sending end to the receiving end do not carry a notification value. If the other data packets sent by the sending end to the receiving end carry a notification value, the value of the notification value can indicate that the data migration is not complete. The network card of the receiving end can store the notification value in the storage space of the receiving end indicated by the notification address. The processor in the receiving end can obtain the notification value from the storage space of the receiving end indicated by the notification address to determine that the data migration is not complete.

[0091] Optionally, data of multiple services can be transmitted between the sending end and the receiving end, and the notification addresses contained in the negotiation messages for different services sent by the receiving end to the sending end can be different, so that the network card of the receiving end stores the notification values ​​of data transfer completion of different services in the storage space indicated by different notification addresses.

[0092] Therefore, when the sending end directly performs a write operation on the storage medium of the receiving end based on the RDMA write operation, since the receiving end has no perception of whether the data movement is completed, the completion notification information is carried in the data packet. That is, when the sending end sends the last data packet in the data, the completion notification information is sent to the network card to indicate that the data movement is completed. After the receiving end receives the last data packet, it can know that the data movement is completed according to the completion notification information, that is, the completion notification information contained in the notification message is merged into the business message, and there is no need to send additional notification messages, which simplifies the communication process and improves communication efficiency.

[0093] Figure 4 is a flow chart of a data transmission method provided by this application. This description uses the inter-process communication between computing nodes shown in Figure 2 as an example. The method primarily involves communication between a sender and a receiver using the RDMA protocol. The method also illustrates the receiver performing a direct read operation on the sender's storage medium based on an RDMA read operation.

[0094] Step 410: The sending end sends a negotiation message to the receiving end.

[0095] Before the sender and receiver start inter-process communication, the sender's processor can request a memory space from the operating system for storing the notification value. The sender notifies the receiver of the notification value and notification address.

[0096] For example, the sender sends a negotiation message to the receiver, which includes completion notification information. The completion notification information includes a notification address and a notification value. The notification address is the address where the notification value is written to the sender's storage space. The notification value is used to indicate that the data transfer is complete.

[0097] In some embodiments, the completion notification information is located in a reserved field of the negotiation message.

[0098] The negotiation message also includes a data storage address, and the data storage address is used to indicate a storage space where the data is stored at the sending end.

[0099] Step 420: The receiving end sends at least one data packet to the sending end.

[0100] The receiving end receives the negotiation message sent by the sending end, obtains completion notification information from the negotiation message, and stores the completion notification information in a storage medium of the receiving end. For example, the receiving end obtains the completion notification information from a reserved field in the negotiation message and stores the completion notification information in a network interface card (NIC) or other storage medium of the receiving end. For example, the storage medium includes any one of a cache, a memory, or a flash memory.

[0101] When the receiving end needs to read data from the sending end, the receiving end directly performs a read operation on the storage medium of the sending end based on the RDMA read operation, that is, the receiving end directly reads data from the storage space of the storage medium in the sending end based on the RDMA read operation.

[0102] The receiving end's network card can send a data packet to the sending end's network card. The data packet contains the data storage address and access rights carried in the negotiation message. The access rights indicate permission to access the storage space indicated by the data storage address. If the receiving end can complete the transfer of the data to be read by sending a single data packet to the sending end, the receiving end does not need to send another data packet to the sending end. If the receiving end cannot complete the transfer of the data to be read by sending a single data packet to the sending end, the receiving end can send multiple data packets to the sending end to transfer the data to be read from the sending end to the receiving end. The data transferred can be different each time.

[0103] It should be noted that when the receiving end sends a data packet to the sending end, the data packet includes completion notification information. When the receiving end sends the last data packet of data to be read to the sending end, the last data packet also includes completion notification information. Completion notification information is carried in a reserved field of the data packet. Completion notification information includes a notification address and a notification value. The notification address is the address in the sending end's memory space where the notification value is written. The notification value indicates that the data transfer is complete.

[0104] Step 430: The network card at the sending end reads data from the storage space indicated by the data storage address.

[0105] The sending end receives the data packet sent by the receiving end, and the sending end's network card reads the data from the storage space indicated by the data storage address. The sending end sends the data to the receiving end's network card, and the receiving end's network card writes the data to the receiving end's storage medium.

[0106] Step 440: After sending the data indicated by the last data packet to the receiving end, the network card of the sending end sends a completion notification message to the sending end to notify the sending end that the data move is complete.

[0107] The sending end's network card reads data from the storage space indicated by the data storage address. After sending the read data to the receiving end, it retrieves completion notification information from the reserved field in the data packet and stores the notification value in the sending end's storage space indicated by the notification address. The sending end's storage space includes the storage space of the sending end's storage medium or the storage space of the network card's storage medium.

[0108] It should be noted that this application does not limit the order in which the network card at the receiving end writes data into the storage medium at the receiving end and the network card at the sending end stores the notification value into the storage space at the sending end indicated by the notification address.

[0109] The processor in the sending end can obtain the notification value from the storage space of the sending end to confirm that the data move is completed.

[0110] Optionally, if the receiving end needs to send multiple data packets to the sending end, before the receiving end sends the last data packet to the sending end, the other data packets sent by the receiving end to the sending end do not carry a notification value. If the other data packets sent by the receiving end to the sending end carry a notification value, the value of the notification value can indicate that the data transfer is not complete. The network card of the sending end can store the notification value in the storage space of the sending end indicated by the notification address. The processor of the sending end can obtain the notification value from the storage space of the sending end to determine that the data transfer is not complete.

[0111] Therefore, when the receiving end directly performs a read operation on the storage medium of the sending end based on the RDMA read operation, since the sending end has no perception of whether the data movement is completed, the completion notification information is carried in the data packet. That is, when the receiving end sends the last data packet in the data, the completion notification information is sent to the network card to indicate that the data movement is completed. After the sending end receives the last data packet, it can know that the data movement is completed according to the completion notification information, that is, the completion notification information contained in the notification message is merged into the business message, and there is no need to send additional notification messages, which simplifies the communication process and improves communication efficiency.

[0112] It is understood that any computing node has the above-mentioned function of receiving and sending notification values. For example, as shown in Figure 5, a schematic diagram of receiving and sending notification values ​​provided by this application is provided. It is assumed that the network card in the computing node performs the function of receiving and sending notification values.

[0113] The sending processing unit obtains the notification value and notification address from the storage medium, carries the notification value and notification address in the reserved field of the data packet, and buffers the data packet in the sending buffer unit, waiting to send the data packet.

[0114] Upon receiving a data packet, the data packet is parsed to obtain a notification value, a notification address, data, and a storage address. The receive buffer unit buffers the data. The receive processing unit retrieves the data packet from the receive buffer unit and parses the data packet to obtain a notification value, a notification address, data, and a storage address. The DMA stores the notification value in the storage space indicated by the notification address and stores the data in the storage space indicated by the storage address.

[0115] Table 1 shows the communication performance comparison between write_with_notify and traditional solutions under the same bandwidth and different packet sizes. It can be seen that the solution of this application has obvious performance advantages in medium and large data packet scenarios, and as the bandwidth increases, the range of benefited data packets will become larger.

[0116] Table 1

[0117] It is understood that in order to implement the functions in the above embodiments, the computing nodes include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily appreciate that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, this application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in hardware or in a computer software-driven hardware manner depends on the specific application scenario and design constraints of the technical solution.

[0118] The data transmission method provided by the present application is described in detail above in conjunction with Figures 1 to 5. The device provided by the present application will be described below in conjunction with Figure 6. These devices can be used to implement the functions of the computing node in the above method embodiment, and thus can also achieve the beneficial effects of the above method embodiment. In this embodiment, the device can be a computing node as shown in Figure 2, or it can be a module (such as a chip) applied to a computing node.

[0119] As shown in FIG6 , the data transmission device 600 includes a communication module 601 , a processing module 602 and a storage module 603 .

[0120] The data transmission device 600 is used to implement the function of the receiving end in the method embodiment shown in FIG. 3 .

[0121] The communication module 601 is configured to send a negotiation message, wherein the negotiation message includes a notification value and a notification address, wherein the notification value is used to indicate that the data migration is completed. For example, the communication module 601 is configured to execute step 310 .

[0122] The communication module 601 is further configured to receive a data packet, which indicates that the data contained in the data packet is to be written to the receiving end. The data packet includes the data and a notification address of a notification value, where the notification address indicates a storage space on the receiving end for storing the notification value. The transmitting end sends a notification message indicating the completion of the inclusion of the last data packet in the data.

[0123] The processing module 602 is configured to send a completion notification message to the receiving end after storing the last data packet in the receiving end, so as to notify the receiving end that the data migration is completed.

[0124] The storage module 603 is used to store notification values ​​and data, etc., so that the data transmission device can promptly know that the data transfer is completed.

[0125] It should be understood that the data transmission device 600 of the embodiment of the present application can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), wherein the PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Alternatively, when the method shown in FIG. 3 or FIG. 4 is implemented by software, the data transmission device 600 and its modules can also be software modules.

[0126] According to the embodiment of the present application, the data transmission device 600 may correspond to executing the method described in the embodiment of the present application, and the above-mentioned and other operations and / or functions of each unit in the data transmission device 600 are respectively for implementing the corresponding processes of each method in Figure 3 or Figure 4. For the sake of brevity, they will not be repeated here.

[0127] FIG7 is a schematic diagram of the structure of a computer device 700 provided in this application. As shown in FIG7 , computer device 700 includes a processor 710, a bus 720, a memory 730, a communication interface 740, a memory 750 (also referred to as a main memory unit), and a network interface card 760. Processor 710, network interface card 760, memory 730, memory 750, and communication interface 740 are connected via bus 720.

[0128] It should be understood that in this embodiment, the processor 710 may be a CPU, but may also be other general-purpose processors, digital signal processors (DSP), ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0129] The computer device 700 may also include a graphics processing unit (GPU), a neural network processing unit (NPU), a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the present application.

[0130] The communication interface 740 is used to implement communication between the computer device 700 and external devices or components.

[0131] In the present application, when the computer device 700 is used to implement the receiving end or the function of the receiving end shown in Figure 3 or Figure 4, the communication interface 740 is used to send a negotiation message or receive a data packet, or the communication interface 740 is used to receive a negotiation message or send a data packet, etc., so that the network card 760 is used to store the notification value in the storage space indicated by the notification address, and the processor 710 determines that the data move is complete.

[0132] The bus 720 may include a path for transmitting information between the above-mentioned components (such as the processor 710, the memory 750, and the storage 730). In addition to the data bus, the bus 720 may also include a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as bus 720 in the figure. The bus 720 may be a Peripheral Component Interconnect Express (PCIe) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus 720 can be divided into an address bus, a data bus, a control bus, etc.

[0133] As an example, computer device 700 may include multiple processors. The processor may be a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and / or computing units for processing data (e.g., computer program instructions).

[0134] It is worth noting that FIG7 only uses a computer device 700 including one processor 710 and one memory 730 as an example. Here, the processor 710 and the memory 730 are respectively used to indicate a type of device or equipment. In specific embodiments, the number of each type of device or equipment can be determined based on business requirements. For example, the computer device 700 may include multiple GPUs or NPUs.

[0135] Memory 750 may be a volatile memory pool or a nonvolatile memory pool, or may include both volatile and nonvolatile memory. Nonvolatile memory may be read-only memory (ROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM). Memory 750 is used to store notification values, etc.

[0136] The memory 730 may correspond to a storage medium used to store information such as notification values ​​in the above method embodiments, for example, a disk such as a mechanical hard disk or a solid-state drive.

[0137] The computer device 700 may be a general-purpose device or a dedicated device. For example, the computer device 700 may be a server or other device with computing capabilities.

[0138] It should be understood that the computer device 700 according to this embodiment may correspond to the data transmission device 600 in this embodiment, and may correspond to executing the corresponding subject in any method in Figure 3 or Figure 4, and the above-mentioned and other operations and / or functions of each module in the data transmission device 600 are respectively for realizing the corresponding processes of each method in Figure 3 or Figure 4. For the sake of brevity, they will not be repeated here.

[0139] The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium can be located in an ASIC. In addition, the ASIC can be located in a computing device. Of course, the processor and storage medium can also exist as discrete components in a computing device.

[0140] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is performed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive (SSD). The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A data transmission method, characterized in that, The method includes: The receiving end sends the first data storage address and the completion notification information to the sending end; The sending end stores the data to the receiving end through the network card of the receiving end according to the first data storage address. Among them, when the sending end sends the last data packet in the data, the completion notification information is sent to the network card; After storing the last data packet to the receiving end, the network card sends the completion notification information to the receiving end to notify the receiving end that the data migration is completed.

2. The method according to claim 1, characterized in that, The sending end and the receiving end communicate through the Remote Direct Memory Access (RDMA) protocol. The receiving end sending the first data storage address and the completion notification information to the sending end includes: The receiving end carries the completion notification information in the reserved field of the negotiation message of the first data storage address and sends the negotiation message to the sending end.

3. The method according to claim 1 or 2, characterized in that, When the sending end sends the last data packet in the data, the completion notification information is carried in the reserved field of the data packet.

4. The method according to claim 3, characterized in that, The method further includes: After receiving the data packet, the network card obtains the completion notification information from the reserved field of the data packet.

5. The method according to any one of claims 1 to 4, characterized in that The completion notification information includes a notification address and a notification value. The notification address is the address where the notification value is written to the receiving end, and the notification value is used to indicate that the data migration is completed; The method further includes: The network card obtains the completion notification information from the data packet and writes the notification value to the notification address. The receiving end confirms that the data migration is completed by obtaining the notification value from the notification address.

6. The method according to any one of claims 1 to 5, characterized in that, The data is the data written from the sending end to the receiving end, and the first data storage address indicates the storage space in the receiving end where the data is written.

7. The method according to claim 1, characterized in that The method includes: The receiving end reads data from the sending end according to the second data storage address, and the second data storage address indicates the storage space in the sending end where the data is read.

8. The method according to claim 7, characterized in that, The receiving end reading data from the sending end according to the second data storage address includes: The sending end sends the second data storage address and the completion notification information to the receiving end; The receiving end reads data through the network card of the sending end according to the second data storage address. Among them, when the receiving end sends the last data packet corresponding to the data, the completion notification information is sent to the network card; After sending the data indicated by the last data packet to the receiving end, the network card sends the completion notification information to the sending end to notify the sending end that the data migration is completed.

9. A data transmission system, characterized in that, The data transmission system includes a sending end and a receiving end. The receiving end includes a network card, and the sending end includes a network card; The receiving end is used to send the first data storage address and the completion notification information to the sending end; The sending end is used to store the data to the receiving end through the network card of the receiving end according to the first data storage address. Among them, when the sending end sends the last data packet in the data, the completion notification information is sent to the network card; The network card is used to send the completion notification information to the receiving end after storing the last data packet at the receiving end, so as to notify the receiving end that the data migration is completed.

10. The system according to claim 9, wherein The sending end communicates with the receiving end through the Remote Direct Memory Access (RDMA) protocol; The receiving end is used to carry the completion notification information in the reserved field of the negotiation message of the first data storage address and send the negotiation message to the sending end.

11. The system according to claim 9 or 10, characterized in that, When the sending end sends the last data packet in the data, the completion notification information is carried in the reserved field of the data packet.

12. The system according to claim 11, wherein After receiving the data packet, the network card is used to obtain the completion notification information from the reserved field of the data packet.

13. The system according to any one of claims 9 to 12, characterized in that, The completion notification information includes a notification address and a notification value. The notification address is the address for writing the notification value to the receiving end, and the notification value is used to indicate that the data migration is completed; The network card is used to obtain the completion notification information from the data packet and write the notification value to the notification address. The receiving end confirms that the data migration is completed by obtaining the notification value from the notification address.

14. The system according to any one of claims 9 to 13, wherein The sending end is used to send the second data storage address and the completion notification information to the receiving end; The receiving end is used to read data through the network card of the sending end according to the second data storage address. When the receiving end sends the last data packet corresponding to the data, the completion notification information is sent to the network card; The network card is used to send the completion notification information to the sending end after sending the data indicated to be read by the last data packet to the receiving end, so as to notify the sending end that the data migration is completed.

15. The system according to claim 14, characterized in that, The sending end communicates with the receiving end through the Remote Direct Memory Access (RDMA) protocol; The sending end is used to carry the completion notification information in the reserved field of the negotiation message of the second data storage address and send the negotiation message to the receiving end.

16. The system according to claim 14 or 15, characterized in that, When the receiving end sends the last data packet corresponding to the data, the completion notification information is carried in the reserved field of the data packet.

17. The system according to claim 16, wherein After receiving the data packet, the network card is used to obtain the completion notification information from the reserved field of the data packet.

18. The system according to any one of claims 14 to 17, characterized in that, The completion notification information includes a notification address and a notification value. The notification address is the address for writing the notification value to the sending end, and the notification value is used to indicate that the data migration is completed; The network card is used to obtain the completion notification information from the data packet and write the notification value to the notification address. The sending end confirms that the data migration is completed by obtaining the notification value from the notification address.

Images