Data transmission method and system

Abstract

The invention discloses a data transmission method and system, and relates to the field of computers. The method comprises the steps that a receiving end sends a data storage address and completion notification information to a sending end, the sending end stores data to the receiving end through a network card of the receiving end according to the data storage address, and when the sending end sends the last data packet in the data, the completion notification information is sent to the network card; and after the network card stores the last data packet to the receiving end, the network card sends completion notification information to the receiving end so as to notify the receiving end that data migration is completed. A sending end and a receiving end negotiate completion notification information in advance, and when the sending end sends the last data packet in data, the completion notification information is sent to a network card, so that the network card sends the completion notification information to the receiving end after storing the last data packet to the receiving end to notify the receiving end that data migration is completed. The notification message does not need to be sent additionally, the communication process is simplified, and the communication efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of computers, and particularly to a data transmission method and system. Background Art

[0002] Currently, the communication between the source process at the sending end and the destination process at the receiving end mainly performs data exchange based on the Message Passing Interface (MPI). To improve the data transmission rate, data can be transmitted between the sending end and the receiving end based on the Remote Direct Memory Access (RDMA) technology to reduce the data transmission delay. Since both the rdma write operation and the rdma read operation are one-sided communications of RDMA, the receiving end of the data is unaware. For example, when the sending end performs a write operation on the receiving end and the receiving end is unaware, the sending end needs to additionally send a notification message to inform the receiving end that the data exchange is completed; or, when the receiving end performs a read operation on the sending end and the sending end is unaware, the receiving end needs to additionally send a notification message to inform the sending end that the data exchange is completed. Since an additional notification message is required to inform that the data transfer between the sending end and the receiving end is completed after the data transfer between the sending end and the receiving end is completed, the communication process between the sending end and the receiving end is complex, reducing the communication efficiency. Summary of the Invention

[0003] This application provides a data transmission method and system, thereby simplifying the communication process and effectively improving the communication efficiency.

[0004] In a first aspect, a data transmission method is provided. The method includes: the receiving end sends a data storage address and a completion notification information to the sending end, and the sending end stores the data to the receiving end through the network card of the receiving end according to the data storage address, wherein when the sending end sends the last data packet, the completion notification information is sent to the network card; after the network card stores the last data packet to the receiving end, the completion notification information is sent to the receiving end to notify the receiving end that the data transfer is completed.

[0005] Compared with the sending end additionally sending a notification message to the receiving end to indicate that the data transfer is completed, which results in a complex communication process and low communication efficiency. In the solution provided by this application, the sending end and the receiving end negotiate the completion notification information in advance. When the sending end sends the last data packet, the completion notification information is sent to the network card, so that after the network card stores the last data packet to the receiving end, the completion notification information is sent to the receiving end to notify the receiving end that the data transfer is completed, that is, there is no need to additionally send a notification message, simplifying the communication process and improving the communication efficiency.

[0006] In a possible implementation, the sender and the receiver communicate through the RDMA protocol. The receiver sending the data storage address and the completion notification information to the sender includes: The receiver carries the completion notification information in the reserved field of the negotiation message of the data storage address and sends the negotiation message to the sender.

[0007] Thus, carrying the completion notification information in the reserved field of the negotiation message can avoid changing the message format, reduce the modification to the protocol, improve the compatibility of the protocol, simplify the communication process, and improve the communication efficiency.

[0008] In another possible implementation, when sending the last data packet in the data sent by the sender, the completion notification information is carried in the reserved field of the data packet.

[0009] In another possible implementation, the method further includes: After receiving the data packet, the network card obtains the completion notification information from the reserved field of the data packet.

[0010] Thus, carrying the completion notification information in the reserved field of the service message can reduce the modification to the protocol, improve the compatibility of the protocol, and achieve no need to send additional notification messages, simplify the communication process, and improve the communication efficiency.

[0011] In another possible implementation, the completion notification information includes a notification address and a notification value. The notification address is the address to which the notification value is written to the receiver, and the notification value is used to indicate the completion of data migration. The method further includes: The network card obtains the completion notification information from the data packet and writes the notification value to the notification address. The receiver confirms the completion of data migration by obtaining the notification value from the notification address.

[0012] In a second aspect, a data transmission method is provided. The method includes: The sender sends a data storage address and a completion notification information to the receiver. The receiver reads the data through the network card of the sender according to the data storage address. Among them, when the receiver sends the last data packet in the read data, the completion notification information is sent to the network card of the sender; After the network card sends the data indicated by the last data packet to the receiver, the completion notification information is sent to the sender to notify the sender that the data migration is completed.

[0013] In a possible implementation, the sender and the receiver communicate through the RDMA protocol. The sender sending the data storage address and the completion notification information to the receiver includes: The sender carries the completion notification information in the reserved field of the negotiation message of the data storage address and sends the negotiation message to the receiver.

[0014] In another possible implementation, when sending the last data packet corresponding to the data by the receiver, the completion notification information is carried in the reserved field of the data packet.

[0015] In another possible implementation, the method further includes: after receiving a data packet, the network card at the sending end obtains completion notification information from the reserved field of the data packet.

[0016] In another possible implementation, the completion notification information includes a notification address and a notification value. The notification address is the address where the notification value is written to the receiving end, and the notification value is used to indicate that data migration is completed. The method further includes: the network card at the sending end obtains the completion notification information from the data packet and writes the notification value to the notification address. The sending end confirms that data migration is completed by obtaining the notification value from the notification address.

[0017] In a third aspect, a data transmission method is provided. The method is executed by the sending end and includes: receiving a first negotiation message, where the first negotiation message includes a notification value and a first notification address, and a first value of the notification value is used to indicate that data migration is completed; sending a first data packet to the receiving end, where the first data packet is used to indicate writing first data of a first service to the receiving end. The first data packet includes the first data of the first service, the notification value, and the first notification address, and the first notification address indicates the storage space of the receiving end, and the storage space of the receiving end is used to store the notification value.

[0018] In a possible implementation, the notification value and the first notification address are located in the reserved field of the first data packet.

[0019] In another possible implementation, the first data packet further includes a first storage address and an access permission. The first storage address indicates that the storage space of the receiving end is used to store the first data of the first service, and the access permission is used to indicate permission to access the storage space indicated by the first storage address.

[0020] In the case where there is a large amount of data of the first service transmitted between the sending end and the receiving end, the data of the first service can be transmitted through multiple data packets.

[0021] In another possible implementation, before sending the first data packet to the receiving end, the method further includes: sending a second data packet to the receiving end, where the second data packet is used to indicate writing second data of the first service to the receiving end, and the second data packet includes the second data of the first service.

[0022] In another possible implementation, the second data packet further includes a notification value and a first notification address, and a second value of the notification value is used to indicate that data migration is not completed.

[0023] During the process of transmitting the data of the first service between the sending end and the receiving end, the notification value indicating that data migration is not completed can be carried in the intermediate data packet, so that the receiving end can timely learn about the data migration status.

[0024] In another possible implementation, the method further includes: sending a second negotiation message, where the second negotiation message includes a notification value and a second notification address; receiving a third data packet sent by the receiving end, where the third data packet is used to indicate reading the first data of the second service from the sending end, the third data packet includes a notification value and a second notification address, and a first value of the notification value is used to indicate that data migration is completed; storing the notification value into the storage space of the sending end indicated by the second notification address; and sending a fourth data packet to the receiving end, where the fourth data packet includes the first data of the second service.

[0025] According to the notification value carried in the data packet, the data migration status can be known in a timely manner, that is, the data included in the notification message is merged into the data packet, and there is no need to receive an additional notification message, which simplifies the communication process and improves the communication efficiency.

[0026] In another possible implementation, the third data packet further includes a second storage address and an access permission, where the second storage address indicates the storage space of the sending end, and the access permission is used to indicate permission to access the storage space indicated by the second storage address; the method further includes: reading the first data of the second service from the storage space indicated by the second storage address.

[0027] Thus, according to the storage address of the storage space of the sending end included in the data packet, the receiving end can directly perform a read operation on the storage space of the sending end, improving the data transmission rate.

[0028] In another possible implementation, before receiving the third data packet sent by the receiving end, the method further includes: receiving a fifth data packet sent by the receiving end, where the fifth data packet is used to indicate reading the second data of the second service from the sending end, the fifth data packet includes a notification value and a second notification address, and a second value of the notification value is used to indicate that data migration is not completed; and sending a sixth data packet to the receiving end, where the sixth data packet includes the second data of the second service.

[0029] During the process of transmitting the data of the second service between the sending end and the receiving end, a notification value can be carried in the intermediate data packet to indicate that the data migration is not completed, enabling the sending end to know the data migration status in a timely manner.

[0030] Fourthly, a data transmission method is provided, where the method is executed by the receiving end, and the method includes: sending a first negotiation message, where the first negotiation message includes a notification value and a first notification address; receiving a first data packet sent by the sending end, where the first data packet is used to indicate writing the first data of the first service to the receiving end, the first data packet includes the first data of the first service, a notification value, and a first notification address, and a first value of the notification value is used to indicate that data migration is completed; and storing the notification value into the storage space of the receiving end indicated by the first notification address.

[0031] In a possible implementation, the notification value and the first notification address are located in the reserved field of the first data packet.

[0032] In another possible implementation, the first data packet further includes a first storage address and an access permission. The first storage address indicates the storage space of the receiving end, and the access permission is used to indicate the permission to access the storage space indicated by the first storage address; the method further includes: writing the first data of the first service into the storage space indicated by the first storage address.

[0033] In another possible implementation, before receiving the first data packet sent by the sending end, the method further includes: receiving a second data packet sent by the sending end. The second data packet is used to indicate writing the second data of the first service to the receiving end, and the second data packet includes the second data of the first service.

[0034] In another possible implementation, the second data packet further includes a notification value and a first notification address. The second value of the notification value is used to indicate that the data migration is not completed.

[0035] In another possible implementation, the method further includes: receiving a second negotiation message, where the second negotiation message includes a notification value and a second notification address; sending a third data packet to the sending end. The third data packet is used to indicate reading the first data of the second service from the sending end. The third data packet includes a notification value and a second notification address. The first value of the notification value is used to indicate that the data migration is completed, and the second notification address indicates the storage space of the sending end. The storage space of the sending end is used to store the notification value; receiving a fourth data packet sent by the sending end, where the fourth data packet includes the first data of the second service.

[0036] In another possible implementation, the third data packet further includes a second storage address and an access permission. The second storage address indicates the storage space of the sending end for storing the first data of the second service, and the access permission is used to indicate the permission to access the storage space indicated by the second storage address.

[0037] In another possible implementation, before sending the third data packet to the sending end, the method further includes: sending a fifth data packet to the sending end. The fifth data packet is used to indicate reading the second data of the second service from the sending end. The fifth data packet includes a notification value and a second notification address. The second value of the notification value is used to indicate that the data migration is not completed; receiving a sixth data packet sent by the sending end, where the sixth data packet includes the second data of the second service.

[0038] In a fifth aspect, a network card is provided. The network card is used to execute the operation steps of the methods of the sending end or the receiving end in the above aspects or any possible implementation manner in the aspects.

[0039] In a sixth aspect, a computer device is provided. The computer device includes a memory, a processor, and a network card. The memory is used to store a set of computer instructions. When the processor executes the set of computer instructions, the processor, in conjunction with the network card, performs the operation steps of the method of the sending end or the receiving end in any of the above aspects or any possible implementation manner in each aspect.

[0040] In a seventh aspect, a data transmission system is provided. The computing system includes a plurality of computing nodes, and the computing nodes perform the operation steps of the method of the sending end or the receiving end in any of the above aspects or any possible implementation manner in each aspect.

[0041] In an eighth aspect, a computer-readable storage medium is provided, including: computer software instructions. When the computer software instructions run on a processor, the processor is caused to perform the operation steps of the method of the sending end or the receiving end in any of the above aspects or any possible implementation manner in each aspect.

[0042] In a ninth aspect, a computer program product is provided. When the computer program product runs on a computer, the computer is caused to perform the operation steps of the method of the sending end or the receiving end in any of the above aspects or any possible implementation manner in each aspect.

[0043] For the technical effects brought by any of the design manners in the second aspect to the ninth aspect, reference may be made to the technical effects brought by the first aspect or different design manners in the first aspect, which will not be elaborated here.

[0044] Based on the implementation manners provided in the above aspects of the present application, further combinations can be made to provide more implementation manners. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of data transmission provided by the present application;

[0046] Figure 2 It is a schematic diagram of the architecture of a data transmission system provided by the present application;

[0047] Figure 3 It is a schematic diagram of the flow of a data transmission method provided by the present application;

[0048] Figure 4 It is a schematic diagram of the flow of a data transmission method provided by the present application;

[0049] Figure 5 It is a schematic diagram of the reception and transmission of a notification value provided by the present application;

[0050] Figure 6 It is a schematic diagram of the structure of a data transmission device provided by the present application;

[0051] Figure 7 A structural schematic diagram of a computer device provided for this application. Detailed implementation manners

[0052] For ease of understanding, the main terms involved in this application are first explained.

[0053] Message Passing Interface (MPI): A cross - language communication protocol that can run on various parallel computing architectures. Message - passing programs can be written in C, C++, and Fortran. For example, data interaction can be carried out between the sending - end process and the receiving - end process using MPI. MPI supports point - to - point communication mode (point to point communications) and multi - point communication mode. The point - to - point communication mode can refer to a way of communication between a source process and a destination process. The source process is used to send data, and the destination process is used to receive data. The multi - point communication mode includes one - to - many communication mode, many - to - one communication mode, and many - to - many communication mode. The one - to - many communication mode can refer to a way of communication between a source process and multiple destination processes. The many - to - one communication mode refers to a way of communication between multiple source processes and a single destination process. The many - to - many communication mode refers to a way of communication between multiple source processes and multiple destination processes. The goal of MPI is high performance, large - scale, and portability.

[0054] Remote Direct Memory Access (RDMA): A technology that bypasses the operating system of a remote host and directly accesses the storage medium of the remote host. That is, data is moved from one system to the memory of another system without kernel intervention. Thus, the overhead of storage medium replication and context switching is eliminated, saving the computing resources of the processor, improving the system throughput, and reducing the network communication latency of the system.

[0055] One - side communication: A transmission mode in the RDMA protocol, suitable for transmitting packets with a large amount of data, and the peer is unaware. For example, one - side communication includes rdma write operation and rdma read operation.

[0056] RDMA device: A device that processes RDMA packets, used for assembling, sending, and receiving and processing RDMA packets. For example, an RDMA device includes a host channel adapter and a Network Interface Card (NIC). The Network Interface Card is abbreviated as NIC.

[0057] An HPC cluster refers to a computer cluster system. An HPC cluster contains multiple computers connected together using various interconnection technologies. The interconnection technologies can be, for example, IB, RoCE, or TCP. HPC provides extremely high floating-point computing power and can be used to address the computing requirements of computing-intensive and large-scale data processing services. The combined computing power of multiple connected computers can be used to handle large computing problems. For example, large computing problems and computing requirements involved in industries such as scientific research, weather forecasting, simulation experiments, biopharmaceuticals, gene sequencing, and image processing are solved using HPC clusters. Using an HPC cluster to handle large computing problems can effectively shorten the computing time for processing data and improve computing accuracy. Usually, the management node in an HPC cluster can decompose the computing tasks and allocate the decomposed computing tasks to multiple computing nodes, and the multiple computing nodes complete the computing tasks in parallel.

[0058] Generally, many processes are launched in HPC applications or benchmarks. When multiple processes exchange data, MPI multi-point communication or MPI point-to-point communication is used, such as MPI_Bcast, MPI_Reduce, or MPI_Send. Services that transfer large amounts of data between processes include neighbor exchange in applications, panel broadcast in the High Performance Linpack (HPL) benchmark, and row exchange. The RNDV protocol can be used to transfer large amounts of data between processes, that is, rdma one-sided communication is used to implement data transfer, and rdma two-sided communication is used to synchronize and notify the data transfer status. Since additional notification messages need to be sent to indicate the completion of data transfer, the communication process is complex and the communication efficiency is low.

[0059] Exemplarily, as Figure 1 shown, it is a schematic diagram of data transmission provided by this application. As Figure 1 shown in (a) of Figure 1As shown in (b) of , the service message supports carrying an immediate RDMA write command (e.g., rdma write with immediate). In addition to carrying data to the receiving end, it can also carry other information, such as carrying data migration completion information and generating a Completion Queue Entry (CQE) at the receiving end. After the receiving end polls the CQE, it can perform subsequent operations on the corresponding immediate content. However, the receiving end generates a CQE based on rdma write with immediate. After the receiving end polls the CQE, it will take the next action according to the content in the CQE, which requires the participation of the CPU. However, the timing of the CQE arrival is uncertain. The receiving end can only poll without discrimination. The non-target CQEs waited for midway need to be temporarily stored, and the target CQE needs to perform a tag-matching action; this increases the complexity of hardware offloading and affects performance.

[0060] To solve the problems that the additional transmission of a notification message indicating data migration completion leads to a complex communication process and low communication efficiency, this application provides a data transmission method, that is, carrying completion notification information in the data packet. That is, when the sending end sends the last data packet, it sends the completion notification information to the network card to indicate that the data migration is completed. So that after the receiving end receives the last data packet, it can know that the data migration is completed according to the completion notification information. That is, the completion notification information included in the notification message is merged into the service message, and there is no need to additionally send a notification message, which simplifies the communication process and improves the communication efficiency.

[0061] The following will describe in detail the implementation manner of the data transmission method provided by this application with reference to the accompanying drawings.

[0062] Figure 2 It is a schematic diagram of the architecture of a data transmission system provided by this application. As Figure 2 shown, the data transmission system 200 includes a client 210, a computing cluster 220, and a storage cluster 230.

[0063] The storage cluster 230 includes multiple storage nodes 231. One storage node 231 includes one or more controllers, a network card, and multiple hard disks. The hard disks are used to store data. The hard disks can be magnetic disks or other types of storage media, such as solid-state drives or shingled magnetic recording hard disks, etc. The network card is used to communicate with the computing nodes 221 included in the computing cluster 220. The controller is used to write data to the hard disk or read data from the hard disk according to the read / write data requests sent by the computing nodes 221. During the process of reading and writing data, the controller needs to convert the address carried in the read / write data request into an address that the hard disk can recognize.

[0064] The computing cluster 220 includes multiple computing nodes 221. The computing nodes 221 can be a type of computing device, such as an acceleration card, a server, etc.

[0065] In some embodiments, the computing cluster 220 can be a heterogeneous computing architecture to provide high-performance computing. For example, the computing cluster 220 can include computing units with computing capabilities such as a central processing unit (CPU), a graphics processing unit (GPU), a data processing unit (DPU), a neural processing unit (NPU), and an embedded neural-network processing unit (NPU) to provide high-performance computing.

[0066] In some other embodiments, the multiple computing nodes 221 are connected based on high-speed interconnect technology through network devices (such as switches, network cards, etc.) to enable communication between the multiple computing nodes 221.

[0067] The client 210 communicates with the computing cluster 220 and the storage cluster 230 through the network 240. For example, the client 210 sends a request to the computing cluster 220 through the network 240, requesting the computing cluster 220 to perform model training. The network 240 can refer to an enterprise internal network (such as a Local Area Network (LAN)) or the Internet. The client 210 refers to a computer connected to the network 240, and can also be called a workstation. Different clients can share resources on the network (such as computing resources, storage resources).

[0068] In some embodiments, the computing cluster 220 further includes a control node 222. For example, the control node and the computing nodes can be independent physical devices. Also, for example, the control node and the multiple computing nodes can be located in the same physical device. The control node can be a CPU. The multiple computing nodes include computing units such as GPUs, NPUs, DPUs, etc. The control node 222 is used to manage and allocate tasks, and multiple tasks are executed in parallel by the multiple computing nodes to improve the data processing rate.

[0069] In this application, inter-process communication can be performed between the computing nodes 221. Among them, data can be transmitted between the computing nodes 221 based on RDMA unilateral operations to improve the data transmission rate between the computing nodes 221. In addition, when the last data packet is transmitted between the computing nodes 221, a completion notification message is sent to the peer end to timely learn whether the data migration is completed.

[0070] In some other embodiments, the client 210 installs the client program 211. The client 210 runs the client program 211 to display a user interface (UI). The user 250 operates the user interface to submit a request. For example, the user 250 operates the user interface to submit a request. After the control node 222 obtains the request, it loads data from the storage cluster 230 and distributes the data to multiple computing nodes.

[0071] Optionally, the system administrator 260 can configure system information, etc. through the client 210 by calling the application platform interface (API) 212 or the command-line interface (CLI) interface 213, such as the notification policy of the notification value provided in this application.

[0072] Figure 2 This is only a schematic diagram. The embodiments of the present application do not limit the device connection method and the number of devices in the data transmission system. For example, the data transmission system may include multiple clients. One client can be connected to multiple computing nodes. Different clients establish connections with different computing nodes.

[0073] Next, the data transmission process will be described in detail with reference to the accompanying drawings.

[0074] Figure 3 This is a schematic flowchart of a data transmission method provided by the present application. Here, the inter-process communication between computing nodes shown in Figure 2 is used for illustration. The communication between the sending end and the receiving end through the RDMA protocol is mainly described. The direct write operation of the sending end on the storage medium of the receiving end based on the rdmawrite operation is described.

[0075] Step 310: The receiving end sends a negotiation message to the sending end.

[0076] Before the receiving end and the sending end perform inter-process communication, the processor of the receiving end can apply for a storage space for storing the notification value from the operating system. The receiving end informs the sending end of the notification value and the notification address. The notification address is the address for writing the notification value into the storage space of the receiving end. The notification value is used to indicate the completion of data migration.

[0077] For example, the receiving end sends a negotiation message to the sending end. The negotiation message includes completion notification information. The completion notification information includes the notification address and the notification value. In some embodiments, the completion notification information is located in the reserved field of the negotiation message.

[0078] Among them, the negotiation message also contains the data storage address, and the data storage address is used to indicate the storage space of the receiving end for storing the data migrated from the sending end.

[0079] Step 320: The sending end sends at least one data packet to the receiving end.

[0080] The sending end receives the negotiation message sent by the receiving end, obtains the completion notification information from the negotiation message, and stores the completion notification information in the storage medium of the sending end. For example, the sending end obtains the completion notification information from the reserved field of the negotiation message and stores the completion notification information in the network card of the sending end or other storage media. For example, the storage medium includes any one of cache, memory, or flash memory.

[0081] When the sending end needs to write data to the receiving end, the sending end directly performs a write operation on the storage medium of the receiving end based on the rdma write operation, that is, the sending end stores the data in the receiving end through the network card of the receiving end according to the data storage address carried in the negotiation message.

[0082] Among them, the network card of the sending end can send data packets to the network card of the receiving end. The data packet contains the data to be written, the data storage address carried in the negotiation message, and the access permission. The access permission is used to indicate the storage space allowed to access the data storage address. If the sending end sends one data packet to the receiving end and can complete the transfer of the data to be written, the sending end does not need to send data packets to the receiving end anymore. If the sending end sends one data packet to the receiving end and cannot complete the transfer of the data to be written, the sending end sends multiple data packets to transfer the data to be written from the sending end to the receiving end. The data contained in multiple data packets can be different.

[0083] It should be noted that when the sending end sends one data packet to the receiving end, the data packet contains the completion notification information. When the sending end sends the last data packet among multiple data packets to the receiving end, the last data packet includes the completion notification information. The completion notification information is carried in the reserved field of the data packet. The completion notification information includes the notification address and the notification value. The notification address is the address to write the notification value to the storage space of the receiving end. The notification value is used to indicate that the data transfer is completed.

[0084] Step 330: The network card of the receiving end writes the data into the storage space indicated by the data storage address.

[0085] The receiving end receives the data packet sent by the sending end, and the network card of the receiving end writes the data in the data packet into the storage space indicated by the data storage address.

[0086] Step 340: After the network card of the receiving end stores the last data packet in the receiving end, it sends the completion notification information to the receiving end to notify the receiving end that the data transfer is completed.

[0087] After the network card of the receiving end writes data into the storage space indicated by the data storage address, it can obtain the completion notification information from the reserved field in the data packet and store the notification value into the storage space of the receiving end indicated by the notification address. The storage space of the receiving end includes the storage space of the storage medium in the receiving end or the storage space of the storage medium in the network card.

[0088] The processor in the receiving end can obtain the notification value from the storage space of the receiving end and determine that the data migration is completed.

[0089] Optionally, if the sending end needs to send multiple data packets to the receiving end, before the sending end sends the last data packet to the receiving end, the other data packets sent by the sending end to the receiving end do not carry the notification value. If the other data packets sent by the sending end to the receiving end carry the notification value, the value of the notification value can indicate that the data migration is not completed. The network card of the receiving end can store the notification value into the storage space of the receiving end indicated by the notification address. The processor in the receiving end can obtain the notification value from the storage space of the receiving end indicated by the notification address and determine that the data migration is not completed.

[0090] Optionally, when multiple services can be transmitted between the sending end and the receiving end, the notification addresses included in the negotiation packets of different services sent by the receiving end to the sending end can be different, so that the network card of the receiving end can store the notification values of the completion of data migration of different services in the storage spaces indicated by different notification addresses.

[0091] Therefore, when the sending end directly performs a write operation on the storage medium of the receiving end based on the rdma write operation, since the receiving end is unaware of whether the data migration is completed, the completion notification information is carried in the data packet, that is, when the last data packet in the data sent by the sending end is sent, the completion notification information is sent to the network card to indicate that the data migration is completed. After the receiving end receives the last data packet, it can know that the data migration is completed according to the completion notification information, that is, the completion notification information included in the notification packet is merged into the service packet, and there is no need to send an additional notification packet, which simplifies the communication process and improves the communication efficiency.

[0092] Figure 4 It is a schematic flow diagram of a data transmission method provided by this application. Here, it is described by Figure 2 the inter-process communication between computing nodes shown. It mainly describes the communication between the sending end and the receiving end through the RDMA protocol. It describes that the receiving end directly performs a read operation on the storage medium of the sending end based on the rdmaread operation.

[0093] Step 410: The sending end sends a negotiation packet to the receiving end.

[0094] Before the sender and the receiver perform inter - process communication, the processor of the sender can apply to the operating system for a storage space for storing the notification value. The sender notifies the receiver of the notification value and the notification address.

[0095] For example, the sender sends a negotiation message to the receiver, and the negotiation message includes completion notification information. The completion notification information includes the notification address and the notification value. The notification address is the address where the notification value is written into the storage space of the sender. The notification value is used to indicate that the data migration is completed.

[0096] In some embodiments, the completion notification information is located in the reserved field of the negotiation message.

[0097] Among them, the negotiation message also includes a data storage address, and the data storage address is used to indicate the storage space where the data is stored at the sender.

[0098] Step 420: The receiver sends at least one data packet to the sender.

[0099] The receiver receives the negotiation message sent by the sender, obtains the completion notification information from the negotiation message, and stores the completion notification information in the storage medium of the receiver. For example, the receiver obtains the completion notification information from the reserved field of the negotiation message and stores the completion notification information in the network card of the receiver or other storage media. For example, the storage medium includes any one of a cache, a memory, or a flash memory.

[0100] When the receiver needs to read data from the sender, the receiver directly performs a read operation on the storage medium of the sender based on the rdma read operation, that is, the receiver directly reads data from the storage space of the storage medium in the sender based on the rdma read operation.

[0101] Among them, the network card of the receiver can send a data packet to the network card of the sender. The data packet contains the data storage address and the access permission carried in the negotiation message. The access permission is used to indicate that access to the storage space indicated by the data storage address is allowed. If the receiver sends a data packet to the sender and can complete the migration of the data to be read, the receiver does not need to send another data packet to the sender. If the receiver sends a data packet to the sender and cannot complete the migration of the data to be read, the receiver sends multiple data packets to the sender to migrate the data to be read from the sender to the receiver. The data migrated each time can be different.

[0102] It should be noted that when the receiving end sends a data packet to the sending end, the data packet contains a completion notification message. When the receiving end sends the last data packet among the data to be read to the sending end, the last data packet includes a completion notification message. The completion notification message is carried in the reserved field of the data packet. The completion notification message includes a notification address and a notification value. The notification address is the address for writing the notification value into the storage space of the sending end. The notification value is used to indicate that the data transfer is completed.

[0103] Step 430: The network card of the sending end reads data from the storage space indicated by the data storage address.

[0104] The sending end receives the data packet sent by the receiving end. The network card of the sending end reads data from the storage space indicated by the data storage address. The sending end sends the data to the network card of the receiving end, and the network card of the receiving end writes the data into the storage medium of the receiving end.

[0105] Step 440: After the network card of the sending end sends the data indicated by the last data packet to the receiving end, it sends the completion notification message to the sending end to notify the sending end that the data transfer is completed.

[0106] The network card of the sending end reads data from the storage space indicated by the data storage address. After sending the read data to the receiving end, it can obtain the completion notification message from the reserved field in the data packet and store the notification value into the storage space of the sending end indicated by the notification address. The storage space of the sending end includes the storage space of the storage medium in the sending end or the storage space of the storage medium in the network card.

[0107] It should be noted that this application does not limit the sequence of the network card of the receiving end writing data into the storage medium of the receiving end and the network card of the sending end storing the notification value into the storage space of the sending end indicated by the notification address.

[0108] The processor in the sending end can obtain the notification value from the storage space of the sending end and determine that the data transfer is completed.

[0109] Optionally, if the receiving end needs to send multiple data packets to the sending end, before the receiving end sends the last data packet to the sending end, the other data packets sent by the receiving end to the sending end do not carry a notification value. If the other data packets sent by the receiving end to the sending end carry a notification value, the value of the notification value can indicate that the data transfer is not completed. The network card of the sending end can store the notification value into the storage space of the sending end indicated by the notification address. The processor in the sending end can obtain the notification value from the storage space of the sending end and determine that the data transfer is not completed.

[0110] Thus, when the receiving end directly performs a read operation on the storage medium of the sending end based on the RDMA read operation, since the sending end is unaware whether the data migration is completed, a completion notification message is carried in the data packet. That is, when the receiving end sends the last data packet, the completion notification message is sent to the network card to indicate that the data migration is completed. After the sending end receives the last data packet, it can learn that the data migration is completed according to the completion notification message, that is, the completion notification message included in the notification packet is merged into the service packet, and there is no need to send an additional notification packet, which simplifies the communication process and improves the communication efficiency.

[0111] Understandably, any computing node has the function of receiving and sending notification values. Exemplarily, as Figure 5 shown, it is a schematic diagram of receiving and sending notification values provided by the present application. It is assumed that the network card in the computing node performs the function of receiving and sending notification values.

[0112] Among them, the sending processing unit obtains the notification value and the notification address from the storage medium, carries the notification value and the notification address in the reserved field of the data packet, and caches the data packet in the sending buffer unit, waiting to send the data packet.

[0113] After receiving the data packet, the data packet is parsed to obtain the notification value, the notification address, the data, and the storage address. The receiving buffer unit caches the data. The receiving processing unit obtains the data packet from the receiving buffer unit, parses the data packet to obtain the notification value, the notification address, the data, and the storage address, and the DMA stores the notification value in the storage space indicated by the notification address and stores the data in the storage space indicated by the storage address.

[0114] Table 1 shows the communication performance comparison between write_with_notify and the traditional scheme under the same bandwidth and different packet sizes. It can be seen that the scheme of the present application has obvious performance advantages in the scenario of medium and large data packets, and as the bandwidth increases, the range of beneficial data packets will be larger.

[0115] Table 1

[0116]

[0117]

[0118] It can be understood that in order to implement the functions in the above embodiments, the computing node includes the corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should easily realize that, combined with the units and method steps of each example described in the embodiments disclosed in the present application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application scenario and design constraint conditions of the technical solution.

[0119] In the above text, in combination with Figures 1 to 5 , the data transmission method provided according to the present application is described in detail. Next, in combination with Figure 6 , the devices provided according to the present application will be described. These devices can be used to implement the functions of the computing nodes in the above method embodiments, and thus can also achieve the beneficial effects possessed by the above method embodiments. In this embodiment, the device can be a computing node as shown in Figure 2 , or can also be a module (such as a chip) applied to the computing node.

[0120] Such as Figure 6 shown, the data transmission device 600 includes a communication module 601, a processing module 602, and a storage module 603.

[0121] The data transmission device 600 is used to implement the functions of the receiving end in the method embodiment shown in the above Figure 3 .

[0122] The communication module 601 is used to send a negotiation message. The negotiation message includes a notification value and a notification address. The notification value is used to indicate that the data transfer is completed. For example, the communication module 601 is used to execute step 310.

[0123] The communication module 601 is also used to receive a data packet. The data packet is used to indicate writing the data included in the data packet to the receiving end. The data packet includes data and the notification address of the notification value. The notification address indicates the storage space in the receiving end where the notification value is stored. The last data packet sent by the sending end includes a completion notification message.

[0124] The processing module 602 is used to store the last data packet in the receiving end and then send a completion notification message to the receiving end to notify the receiving end that the data transfer is completed.

[0125] The storage module 603 is used to store the notification value, data, etc., so that the data transmission device can timely learn that the data transfer is completed.

[0126] It should be understood that the data transmission device 600 according to the embodiments of the present application can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The above PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. It can also be implemented by software Figure 3 or Figure 4 When implementing the method shown, and each of its modules can also be a software module. The data transmission device 600 and each of its modules can also be software modules

[0127] The data transmission device 600 according to the embodiments of the present application can correspond to implementing the methods described in the embodiments of the present application, and the above and other operations and / or functions of each unit in the data transmission device 600 are respectively for implementing Figure 3 or Figure 4 the corresponding processes of each method in, for the sake of brevity, will not be elaborated here

[0128] Figure 7 This is a schematic structural diagram of a computer device 700 provided by the present application. As Figure 7 shown, the computer device 700 includes a processor 710, a bus 720, a memory 730, a communication interface 740, a memory 750 (which can also be referred to as the main memory unit), and a network card 760. The processor 710, the network card 760, the memory 730, the memory 750, and the communication interface 740 are connected through the bus 720

[0129] It should be understood that in this embodiment, the processor 710 can be a CPU, and this processor 710 can also be other general-purpose processors, a digital signal processor (DSP), an ASIC, an FPGA, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc

[0130] The computer device 700 may further include a graphics processing unit (GPU), a neural network processing unit (NPU), a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the solution of the present application.

[0131] The communication interface 740 is used to implement the communication between the computer device 700 and an external device or component.

[0132] In the present application, when the computer device 700 is used to implement Figure 3 or Figure 4 the receiver or the functions of the receiver shown, the communication interface 740 is used to send negotiation messages or receive data packets, or the communication interface 740 is used to receive negotiation messages or send data packets, etc., so that the network card 760 is used to store the notification value into the storage space indicated by the notification address, and the processor 710 determines that the data migration is completed.

[0133] The bus 720 may include a path for transmitting information between the above components (such as the processor 710, the memory 750, and the memory 730). In addition to the data bus, the bus 720 may also include a power bus, a control bus, a status signal bus, etc. However, for the sake of clear illustration, all kinds of buses are labeled as the bus 720 in the figure. The bus 720 may be a Peripheral Component Interconnect Express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a Compute Express Link (CXL), a Cache Coherent Interconnect for Accelerators (CCIX), etc. The bus 720 may be divided into an address bus, a data bus, a control bus, etc.

[0134] As an example, the computer device 700 may include multiple processors. The processor may be a multi-CPU processor. Here, the processor may refer to one or more devices, circuits, and / or computing units for processing data (such as computer program instructions).

[0135] It is worth noting that Figure 7Only taking the computer device 700 including one processor 710 and one memory 730 as an example herein, where the processor 710 and the memory 730 are respectively used to indicate a type of device or equipment. In a specific embodiment, the quantity of each type of device or equipment can be determined according to service requirements. For example, the computer device 700 includes multiple GPUs or NPUs.

[0136] The memory 750 can be a volatile memory pool or a non-volatile memory pool, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM). The memory 750 is used to store notification values and the like.

[0137] The memory 730 can correspond to the storage medium for storing information such as notification values in the above method embodiments. For example, a disk, such as a mechanical hard disk or a solid-state drive.

[0138] The above computer device 700 can be a general-purpose device or a special-purpose device. For example, the computer device 700 can also be a server or other devices with computing capabilities.

[0139] It should be understood that the computer device 700 according to this embodiment can correspond to the data transmission device 600 in this embodiment, and can correspond to the corresponding subject in any of the methods according to Figure 3 or Figure 4 and the above and other operations and / or functions of each module in the data transmission device 600 are respectively for implementing Figure 3 orFigure 4 The corresponding processes of the respective methods in

[0140] The method steps in this embodiment can be implemented in a hardware manner or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), register, hard disk, removable hard disk, CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. Additionally, the ASIC can be located in a computing device. Of course, the processor and the storage medium can also exist as discrete components in a computing device.

[0141] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are executed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, or other programmable devices. The computer program or instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired or wireless manner. The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive (SSD). As described above, the above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A data transmission method, characterized in that, The method comprises: The receiving end sends the first data storage address and completion notification information to the sending end; The sending end stores the data to the receiving end through the network card of the receiving end according to the first data storage address, wherein the sending end sends the completion notification information to the network card when sending the last data packet of the data; After storing the last data packet in the receiving end, the network card sends the completion notification information to the receiving end to notify the receiving end that the data migration is completed.

2. The method according to claim 1, wherein The sending end communicates with the receiving end through a remote direct memory access (RDMA) protocol, and the receiving end sends a first data storage address and completion notification information to the sending end, including: The receiving end carries the completion notification information in a reserved field of the negotiation message of the first data storage address, and sends the negotiation message to the sending end.

3. The method according to claim 1 or 2, characterized in that, The last data packet of the data is sent at the sending end, and the completion notification information is carried in a reserved field of the data packet.

4. The method according to claim 3, characterized in that, The method further comprises: After receiving the data packet, the network card obtains the completion notification information from the reserved field of the data packet.

5. The method according to any one of claims 1 to 4, characterized in that The completion notification information includes a notification address and a notification value, wherein the notification address is an address where the notification value is written to the receiving end, and the notification value is used to indicate that the data migration is completed; The method further comprises: The network card obtains the completion notification information from the data packet and writes the notification value into the notification address. The receiving end obtains the notification value from the notification address to confirm that the data migration is completed.

6. The method according to any one of claims 1 to 5, characterized in that The data is data written from the sending end to the receiving end, and the first data storage address indicates a storage space in the receiving end where the data is written.

7. The method according to claim 1, characterized in that The method comprises: The receiving end reads data from the sending end according to the second data storage address, where the second data storage address indicates a storage space in the sending end for reading the data.

8. The method according to claim 7, characterized in that, The receiving end reads data from the sending end according to the second data storage address, including: The sending end sends the second data storage address and completion notification information to the receiving end; The receiving end reads data through the network card of the sending end according to the second data storage address, wherein the completion notification information is sent to the network card when the receiving end sends the last data packet corresponding to the data; After sending the data indicated by the last data packet to be read to the receiving end, the network card sends the completion notification information to the sending end to notify the sending end that the data movement is completed.

9. A data transmission system, characterized in that, The data transmission system includes a sending end and a receiving end, the receiving end includes a network card, and the sending end includes a network card; The receiving end is used to send the first data storage address and completion notification information to the sending end; The sending end is used to store the data to the receiving end through the network card of the receiving end according to the first data storage address, wherein the completion notification information is sent to the network card when the sending end sends the last data packet of the data; The network card is used to send the completion notification information to the receiving end after storing the last data packet in the receiving end, so as to notify the receiving end that the data movement is completed.

10. The system according to claim 9, characterized in that The sending end communicates with the receiving end via a Remote Direct Memory Access (RDMA) protocol; The receiving end is configured to carry the completion notification information in a reserved field of a negotiation message of the first data storage address, and send the negotiation message to the sending end.

11. The system according to claim 9 or 10, characterized in that, The last data packet of the data is sent at the sending end, and the completion notification information is carried in a reserved field of the data packet.

12. The system according to claim 11, wherein: After receiving the data packet, the network card is used to obtain the completion notification information from the reserved field of the data packet.

13. The system according to any one of claims 9 to 12, characterized in that The completion notification information includes a notification address and a notification value, wherein the notification address is an address where the notification value is written to the receiving end, and the notification value is used to indicate that the data migration is completed; The network card is used to obtain the completion notification information from the data packet and write the notification value into the notification address. The receiving end confirms that the data migration is completed by obtaining the notification value from the notification address.

14. The system according to any one of claims 9 to 13, characterized in that The sending end is used to send the second data storage address and completion notification information to the receiving end; The receiving end is used to read data through the network card of the sending end according to the second data storage address, wherein the completion notification information is sent to the network card when the receiving end sends the last data packet corresponding to the data; The network card is used to send the completion notification information to the sending end after sending the data indicated by the last data packet to be read to the receiving end, so as to notify the sending end that the data movement is completed.

15. The system according to claim 14, wherein: The sending end communicates with the receiving end via a Remote Direct Memory Access (RDMA) protocol; The sending end is configured to carry the completion notification information in a reserved field of a negotiation message at the second data storage address, and send the negotiation message to the receiving end.

16. The system according to claim 14 or 15, characterized in that, The last data packet corresponding to the data is sent at the receiving end, and the completion notification information is carried in a reserved field of the data packet.

17. The system according to claim 16, wherein: After receiving the data packet, the network card is used to obtain the completion notification information from the reserved field of the data packet.

18. The system according to any one of claims 14 to 17, characterized in that The completion notification information includes a notification address and a notification value, wherein the notification address is an address where the notification value is written to the sending end, and the notification value is used to indicate that the data migration is completed; The network card is used to obtain the completion notification information from the data packet and write the notification value into the notification address. The sending end obtains the notification value from the notification address to confirm that the data move is completed.

Images