Data transmission method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a data transmission method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a first work request sent by a target application program, the first work request being used for a first computing power cluster to request to carry out data transmission with a second computing power cluster through employing a remote direct memory access protocol; generating a first queue pair message of a remote direct memory access protocol based on the first working request, and sending the first queue pair message to the target middleware; establishing a virtual private network tunnel between the first computing power cluster and the second computing power cluster based on the target middleware; and obtaining first transmission data corresponding to the message by the first queue, and sending the first transmission data to the second computing power cluster based on the virtual private network tunnel to execute the first work request. According to the embodiment of the invention, the RDMA service can be intelligently identified and forwarded, and the user data is safely and effectively protected, so that the operation cost of a customer is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and in particular to a data transmission method and device, electronic equipment, and storage medium. Background Art

[0002] During data transmission, current visual networking servers usually only support TCP / IP and v2v protocols, and cannot support Remote Direct Memory Access (RDMA) communication mode. Summary of the invention

[0003] In view of the above problems, a data transmission method and device, electronic device, and storage medium are proposed to overcome the above problems or at least partially solve the above problems, including:

[0004] A data transmission method is applied to a first computing power cluster, wherein the first computing power cluster is installed with a target application and a target middleware for transmitting data using a remote direct memory access protocol, the method comprising:

[0005] Obtaining a first work request sent by a target application, the first work request being used for a first computing power cluster to request a remote direct memory access protocol to perform data transmission with a second computing power cluster, the second computing power cluster being installed with a target application and a target middleware for transmitting data using the remote direct memory access protocol, the target middleware being set in kernel state for identifying and forwarding data;

[0006] Generate a first queue pair message of the remote direct memory access protocol based on the first work request, and send the first queue pair message to the target middleware;

[0007] Establishing a virtual private network tunnel between the first computing power cluster and the second computing power cluster based on the target middleware;

[0008] Obtain first transmission data corresponding to the message of the first queue, and send the first transmission data to the second computing power cluster based on the virtual private network tunnel to execute the first work request.

[0009] Optionally, obtaining the first work request sent by the target application includes:

[0010] After the first computing power cluster starts the monitoring and forwarding data service for the target middleware, a first work request sent by the target application is obtained.

[0011] Optionally, the generating a first queue pair message of a remote direct memory access protocol based on the first work request, and sending the first queue pair message to the target middleware includes:

[0012] Converting the first work request into a first queue pair message of a remote direct memory access protocol according to a preset format;

[0013] Sending the first queue pair message to a preset sending queue;

[0014] When the first queue pair message is located at the head of the sending queue and the previous message is sent, the first queue pair message is sent to the target middleware.

[0015] Optionally, it also includes:

[0016] receiving, through the virtual private network tunnel, a work completion instruction sent by the second computing power cluster, wherein the work completion instruction is an instruction for feeding back that the second computing power cluster completes the first work request based on the first transmission data;

[0017] generating a second queue pair message of the remote direct memory access protocol based on the work completion instruction;

[0018] The second queue pair message is stored in a preset completion queue.

[0019] Optionally, the establishing a virtual private network tunnel between the first computing power cluster and the second computing power cluster based on the target middleware includes:

[0020] A virtual private network tunnel is established between the first computing power cluster and the second computing power cluster based on the first queue and the message through the target middleware.

[0021] Optionally, it also includes:

[0022] When the virtual private network tunnel is a tunnel supporting a first network protocol, converting the first queue pair message into a first network protocol message according to the first network protocol;

[0023] Send the first network protocol message to the second computing power cluster through the first network protocol tunnel.

[0024] Optionally, it also includes:

[0025] receiving a second network protocol message sent by the second computing power cluster, where the second network protocol message is generated when a second work request of a target application is processed in the second computing power cluster;

[0026] Convert the second network protocol message into third queue pair message data, and send the third queue pair message data to a preset receiving queue;

[0027] When the message of the third queue pair is at the head of the receiving queue and the first computing power cluster has processed the previous message, the message of the third queue pair is removed from the receiving queue;

[0028] The second transmission data corresponding to the third queue message is obtained, and the second transmission data is sent to the second computing power cluster based on the virtual private network tunnel.

[0029] A data transmission device is applied to a first computing power cluster, wherein the first computing power cluster is installed with a target application and a target middleware for transmitting data using a remote direct memory access protocol, the device comprising:

[0030] A first work request acquisition module is used to acquire a first work request sent by a target application, the first work request is used for a first computing power cluster to request to use a remote direct memory access protocol to perform data transmission with a second computing power cluster, and the second computing power cluster is installed with a target application and a target middleware for transmitting data using the remote direct memory access protocol;

[0031] A first queue pair message sending module, used for generating a first queue pair message of the remote direct memory access protocol based on the first work request, and sending the first queue pair message to the target middleware, the target middleware is set in kernel state, and is used for identifying and forwarding data;

[0032] A virtual private network tunnel establishment module, used to establish a virtual private network tunnel between the first computing power cluster and the second computing power cluster based on the target middleware;

[0033] The first transmission data sending module is used to obtain the first transmission data corresponding to the message of the first queue, and send the first transmission data to the second computing power cluster to execute the first work request based on the virtual private network tunnel.

[0034] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the above-mentioned data transmission method when executed by the processor.

[0035] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the data transmission method described above is implemented.

[0036] The embodiments of the present invention have the following advantages:

[0037] In an embodiment of the present invention, by installing a target application and a target middleware for transmitting data using the remote direct memory access protocol in the first computing power cluster and the second computing power cluster, the middleware can intelligently identify, process and forward RDMA services, thereby establishing a virtual private network tunnel between the first computing power cluster and the second computing power cluster, and using the network tunnel to transmit transmission data corresponding to the RDMA service. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0039] Figure 1 is a flowchart of a data transmission method provided by an embodiment of the present invention;

[0040] Figure 2 It is a schematic diagram of a data transmission method in a visual network computing cluster provided by an embodiment of the present invention;

[0041] Figure 3 It is another schematic diagram of a data transmission process in a visual network computing power cluster provided by an embodiment of the present invention;

[0042] Figure 4 It is a structural flow chart of a data transmission device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Reference Figure 1 , shows a flowchart of a data transmission method provided by an embodiment of the present invention, which is applied to a first computing power cluster, and the first computing power cluster is installed with a target application and a target middleware for transmitting data using a remote direct memory access protocol, and specifically may include the following steps:

[0045] Step S101, obtaining a first work request sent by a target application;

[0046] Among them, the first work request can be used for the first computing power cluster to request the use of remote direct memory access protocol to transmit data with the second computing power cluster, and the second computing power cluster is installed with a target application and a target middleware that can be used to transmit data using the remote direct memory access protocol.

[0047] In actual applications, the first computing power cluster and the second computing power cluster are composed of multiple servers, but the communication between the two is usually based on the TCP / IP protocol and the v2v protocol. The communication protocol is usually determined according to the protocol supported by the server itself. For example, when the servers of the first computing power cluster and the second computing power cluster are Internet servers, the server communication supports the TCP / IP protocol, so that the first computing power cluster and the second computing power cluster can communicate using the TCP / IP protocol. When the first computing power cluster and / or the second computing power cluster are visual Internet servers, the server can support the TCP / IP protocol and the visual Internet protocol.

[0048] In an embodiment of the present invention, in order to enable the first computing power cluster and the second computing power cluster to support the remote direct memory access (RDMA) communication mode, a target application and a target middleware for transmitting data using the remote direct memory access protocol can be installed in the first computing power cluster and the second computing power cluster, wherein the target application can be set in user mode for user interaction to determine the work request input by the user, and the target middleware can be set in kernel mode for identifying and forwarding data. For example, when executing the writing of data of the first computing power cluster to the second computing power cluster, the target middleware can identify and forward the data of the remote direct memory access protocol in the first computing power cluster and the second computing power cluster, so that the first computing power cluster and the second computing power cluster can support the remote direct memory access protocol (RDMA) to transmit data and realize the "zero copy" mechanism (RDMA technology) of computing power data service.

[0049] In the first computing power cluster, the user can input a first work request through the target application. After obtaining the first work request sent by the target application, the first computing power cluster can perform data processing in the first computing power cluster based on the first work request to implement a "zero copy" mechanism of the computing power data server.

[0050] In one embodiment of the present invention, in the visual networking technology, the target application may be a VVoE program (visual networking virtualization overlay engine), and the middleware may be a V-Shim shim middleware. V-Shim is a network shim based on the v2v protocol to process special data packets, such as RDMA data packets, which can allow the visual networking device to identify and forward the data sent to the RDMA communication protocol (IB communication protocol). In the first computing power cluster and / or the second computing power cluster, it can be divided into user state and kernel state. The target application is in the user state and generates work requests required by the user for the user. The middleware is set in the kernel state and can be used to identify and forward data. The first computing power cluster and / or the second computing power cluster can also be provided with a network card, which is used for data transmission between computing power clusters. In the visual networking, a cascaded visual networking core server can be set between the computing power clusters, and the data transmission of the computing power clusters can be carried out through the visual networking core server.

[0051] In one embodiment of the present invention, after the first computing power cluster starts a monitoring and forwarding data service for the target middleware, the first work request sent by the target application can be obtained.

[0052] The target middleware's listening and forwarding data service can be set in the first computing power cluster and / or the second computing power cluster. When the listening and forwarding data service for the target middleware is enabled, the first computing power cluster and the second computing power cluster can intelligently identify data and perform data transmission when performing RDMA-related services. That is, the first computing power cluster and the second computing power cluster can support remote direct memory access.

[0053] Step S102, generating a first queue pair message of a remote direct memory access protocol based on the first work request, and sending the first queue pair message to a target middleware;

[0054] In the first computing power cluster, the first work request can be converted into a first queue pair message of the remote direct memory access protocol. The queue pair message is the basic unit of RDMA two-end communication. Then, the first queue pair message can be sent to the target middleware to enable the target middleware to implement remote direct memory access (RDMA) communication.

[0055] In practical applications, QP (Queue Pair) is the core concept in the RDMA (Remote Direct Memory Access) protocol, and QP can be used to manage data transmission. QP can be composed of a send queue (SQ) and a receive queue (RQ), and the application performs data transmission operations by submitting the first work request (WR) to these queues.

[0056] Specifically, the basic structure of QP can be composed of a send queue, a receive queue, and a completion queue. Among them: the send queue (SQ) can be used to store work requests (WR) for send operations, such as RDMA write, RDMA read or send. The receive queue (RQ) can be used to store work requests (WR) for receive operations, such as receiving data. The completion queue (CQ) can be used to notify the completion of the operation, including success or failure status information.

[0057] During data transmission, QP packets can be used for data transmission, protocol encapsulation, and flow control. That is, in terms of data transmission, QP packets can carry specific information about RDMA operations, such as source address, destination address, and data length. In terms of protocol encapsulation, QP packets can be encapsulated according to network types (InfiniBand, RoCE, iWARP) to ensure that the data to be transmitted can be correctly transmitted in different networks. In terms of flow control, QP packets can contain flow control information to ensure that the sending and receiving rates match to avoid packet loss or congestion.

[0058] Among them, InfiniBand: (IB) is a server and storage interconnection technology, which has the characteristics of high speed, low latency, low CPU load, high efficiency and scalability. One of the key features of InfiniBand is that it naturally supports remote direct memory access (RDMA), and its important component is coaxial cable.

[0059] RoCE: RDMA technology standard based on Ethernet, also specified by the IBTA organization. RoCE provides RDMA semantics for Ethernet. RoCE v1 is a link layer protocol that allows direct access between any two hosts in the same broadcast domain. RoCE v2 is an Internet layer protocol with Ethernet routing and forwarding capabilities. It is only supported on routers with RDMA capabilities.

[0060] iWARP: A network protocol that allows RDMA over TCP. Only supported on standard Ethernet infrastructure (with RDMA-capable switches).

[0061] The relationship between QP and RDMA operations can be reflected in the RDMA write, RDMA read, and send / receive processes. In the RDMA write process, the sender can directly write data to the receiver's memory without the receiver issuing a receive request in advance. In the RDMA read process, the sender can read data from the receiver's memory without the receiver's participation. In the send / receive process: the sender sends data, and the receiver needs to issue a receive request in advance.

[0062] The state machine of QP can include initialization, ready, and error states. When QP is created, it is in the initial state and needs to be configured and activated. After the QP configuration is completed, it enters the ready state and can receive and send data. In the error state, QP cannot process requests and needs to be reset or destroyed.

[0063] In practical applications, the QP message processing flow may include the following steps:

[0064] Step S11, first work request submission: the application submits the WR to the SQ or RQ of the QP.

[0065] Step S12, message generation: the hardware generates a QP message according to the WR, including the operation type, address and data length.

[0066] Step S13, message transmission: the QP message is transmitted to the target node through the network.

[0067] Step S14, message processing: the target node performs corresponding operations according to the QP message, such as writing into memory or reading data.

[0068] Step S15, completion notification: after the operation is completed, a completion queue item (CQE) is generated to notify the application.

[0069] In practical applications, QP messages are the key to data transmission in the RDMA protocol, so that RDMA operations can be managed through the QP send and receive queues. The QP messages in the embodiments of the present invention carry operation information, ensure efficient data transmission, and support a variety of RDMA operations to achieve optimized high-performance computing.

[0070] In one embodiment of the present invention, the process of generating a first queue pair message of the remote direct memory access protocol based on the first work request and sending the first queue pair message to the target middleware is specifically as follows:

[0071] Convert the first work request into a first queue pair message of the remote direct memory access protocol according to a preset format; send the first queue pair message to a preset sending queue; when the first queue pair message is at the head of the sending queue and the previous message is sent, send the first queue pair message to the target middleware.

[0072] The preset format may be set according to actual scenario requirements, and is not excessively limited in the embodiments of the present invention.

[0073] In actual applications, the first work request can realize the QP message management of the remote direct memory access protocol by converting it into the first queue pair message of the remote direct memory access protocol, and then the first queue pair message can be sent to the sending queue in the QP queue for storage to wait for processing. The sending queue is used to store the messages to be sent, and the computing power cluster can send data in sequence according to the order of the queues in the sending queue. Specifically, when the first queue pair message is at the head of the sending queue and the previous message is sent, the first queue pair message can be removed from the sending queue and sent to the target middleware for subsequent processing.

[0074] Step S103, establishing a virtual private network tunnel between the first computing power cluster and the second computing power cluster based on the target middleware;

[0075] In actual applications, when the target middleware receives data from the first queue, in order to realize RDMA service data transmission, a virtual private network tunnel can be established between the first computing power cluster and the second computing power cluster, and the virtual private network tunnel is bound to the target middleware. Then, the target middleware can use the virtual private network tunnel for data transmission after intelligently identifying the transmission data of the RDMA service.

[0076] In the embodiment of the present invention, the virtual private network tunnel may be a visual network VPN tunnel. In practical applications, VVoE may establish a visual network VPN transmission channel based on the visual network transmission technology.

[0077] In one embodiment of the present invention, a virtual private network tunnel is established between a first computing power cluster and a second computing power cluster based on a target middleware, including: establishing a virtual private network tunnel between the first computing power cluster and the second computing power cluster based on a first queue through the target middleware.

[0078] In the field of visual networking technology, the first computing power cluster can enable V-Shim monitoring and forwarding data services. When the target middleware receives the first queue pair message, it identifies the RDMA service, so that the first computing power cluster can serve as the calling end to establish a visual networking VPN tunnel with the second computing power cluster as the called end.

[0079] Step S104, obtaining first transmission data corresponding to the message of the first queue, and sending the first transmission data to the second computing power cluster based on the virtual private network tunnel to execute the first work request.

[0080] In actual applications, the first queue pair message is used to establish a virtual private network tunnel between the first computing power cluster and the second computing power cluster to facilitate data transmission. After the channel is established, the first computing power cluster can realize data transmission with the second computing power cluster. Therefore, in response to the user's local work request, in terms of data, the first computing power cluster can obtain the first transmission data corresponding to the first queue pair message from the local area, and then the first transmission data in the first computing power cluster can be transmitted to the second computing power cluster through the virtual private network tunnel, and then the first transmission data is received in the second computing power cluster for processing, thereby completing the first work request.

[0081] In one embodiment of the present invention, a first computing power cluster can send first transmission data to a second computing power cluster, and then receive a work completion instruction sent by the second computing power cluster through a virtual private network tunnel, where the work completion instruction is an instruction for feeding back to the second computing power cluster that the first work request is completed based on the first transmission data; a second queue pair message of a remote direct memory access protocol can be generated based on the work completion instruction; and the second queue pair message can be stored in a preset completion queue.

[0082] In actual applications, the QP queue manages the sending queue, receiving queue, and completion queue. When the second computing power cluster receives the first transmission data and completes the first work request, a work completion instruction can be generated to feedback the request completion status to the first computing power cluster. The work completion instruction is then returned to the first computing power and stored in the work completion queue of the QP queue through the middleware.

[0083] In one embodiment of the present invention, the embodiment of the present invention can be used for data transmission of a visual networking computing power cluster, so that when the virtual private network tunnel is a tunnel that supports the first network protocol, the first queue pair message is converted into a first network protocol message according to the first network protocol; and the first network protocol message is sent to the second computing power cluster through the first network protocol tunnel. The first network protocol can be a visual networking protocol.

[0084] In one embodiment of the present invention, when the second computing power cluster acts as a sender and the first computing power cluster acts as a receiver, the first computing power cluster can receive a second network protocol message sent by the second computing power cluster, where the second network protocol message is generated when a second work request of a target application is processed in the second computing power cluster; the second network protocol message is converted into third queue pair message data, and the third queue pair message data is sent to a preset receiving queue; when the third queue pair message is at the head of the receiving queue and the first computing power cluster has processed the previous message, the third queue pair message is moved out of the receiving queue; the second transmission data corresponding to the third queue message is obtained, and the second transmission data is sent to the second computing power cluster based on a virtual private network tunnel.

[0085] In actual applications, when the first computing power cluster acts as the sender and the second computing power cluster acts as the receiver, the first queue pair message is generated and stored in the sending queue, the receiving queue in the second computing power cluster stores the queue pair message corresponding to the first transmission data, and the work completion queue in the first computing power cluster receives the second queue pair message corresponding to the work completion command fed back by the second computing power cluster.

[0086] When the second computing power cluster acts as the sender and the first computing power cluster acts as the receiver, the queue pair message corresponding to the second work request is stored in the sending queue of the second computing power cluster, and the queue pair message of the second transmission data is stored in the receiving queue of the first computing power cluster. The second computing power cluster can receive the work completion instruction fed back by the first computing power cluster, and store the corresponding queue pair message in the work completion queue.

[0087] In an embodiment of the present invention, the data transmission process in the first computing power cluster and the second computing power cluster is similar, and the mechanism of intelligently identifying and forwarding RDMA services, safely and effectively protecting user data, and thus reducing the customer's operating costs is the same. When the second computing power cluster acts as the sender, the data transmission process can refer to the data transmission process when the first computing power cluster acts as the sender.

[0088] In an embodiment of the present invention, a target application and a target middleware for transmitting data using the remote direct memory access protocol can be installed in the first computing power cluster and the second computing power cluster. The middleware can intelligently identify, process and forward RDMA services, thereby establishing a virtual private network tunnel between the first computing power cluster and the second computing power cluster, and using the network tunnel to transmit transmission data corresponding to the RDMA service.

[0089] Reference Figure 2 , shows a schematic diagram of a data transmission method in a visual networking computing power cluster in an embodiment of the present invention, computing power center 1 is the first computing power cluster, that is, the RDMA service server, and computing power center 2 is the second computing power cluster, that is, the RDMA service client. Computing power center 1 and computing power center 2 are connected through the cascade core server of the V2V visual networking.

[0090] In the off-site visual network transmission 100G computing power data center, the network card interfaces of the cluster devices of computing center 1 and computing center 2 are respectively deployed with the visual network VVoE (visual network virtualization overlay engine) program. V-Shim gaskets are set in computing center 1 and computing center 2, and the 100G network card interfaces are respectively connected to the visual network core server interface. The visual network core servers in the two regions are cascaded. Register the user mac of VVoE1 to the visual network core server 1 device to access the network and assign it to the visual network number 00001. The same applies to VVoE2 and above, and assign it to the visual network number 00002. VVoE01 and 02 are respectively bound to the 100G network card of the host interface, and the communication protocol stack is bound to the V-Shim device by default.

[0091] When the user acts as an RDMA client on the AI ​​cluster of computing center 2 to request the RDMA server of the server cluster of computing center 1. The customer can choose any communication protocol stack such as InfiniBand, RoCE, iWARP, etc. to send RDMA work queue QP messages in the RDMA control layer on the cluster servers in the two regions. By default, VVoE turns on the V-Shim monitoring and forwarding data service. VVoE1, as the calling end (00001), establishes a visual network VPN tunnel with the VVoE2 called end (00002). When the QP message is received on the downlink interface of the visual router, the V-Shim shim device in the device message forwarding queue will directly encapsulate the QP message into the V2V message, and then forward this data through the visual network VPN tunnel.

[0092] Reference Figure 3 , showing a schematic diagram of the data transmission process in another visual network computing cluster in an embodiment of the present invention, in which the QP queue is composed of RDMA elements, namely, work queue element (WQE), merged with the docking element (queue pair, QP), send queue element (send queue, SQ), and receive queue (receive queue, RQ) to generate a completion queue element (completion queue element, CQE) to complete the pair (completion queue, CQ).

[0093] The WR (Work Request) sent by the application is converted into a work queue element WQE by the user driver and put into the queue QP. QP is the basic unit of RDMA two-way communication. It consists of a send queue (SQ) and a receive queue (RQ). SQ receives the post (including SEND, READ, WRITE, etc.) request sent by the application (step 1 in the figure), and RQ receives the RECV (corresponding to SEND) request sent by the application. For the RDMA write operation, V-Shim passes it to the other end RDMA transmission request (step 2 in the figure, the visual network VPN channel, the QP message of the other end), then takes out the local memory data (step 3 in the figure), performs remote write, and passes it to the other end user APP through the visual network VPN channel (step 4 in the figure). After the data transmission is completed, the network card bound to the V-Shim device generates a queue element (CQE) and puts it into the completion queue (CQ) associated with the QP (step 5 in the figure). Finally, the application receives the work completion instruction (poll) sent by the other end through the visual network VPN channel bound to the V-Shim (step 6 in the figure). At this point, it is known that the RDMA transmission is completed.

[0094] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0095] Reference Figure 4 , shows a schematic diagram of the structure of a data transmission device provided by an embodiment of the present invention, which is applied to a first computing power cluster, and the first computing power cluster is installed with a target application and a target middleware for transmitting data using a remote direct memory access protocol, and specifically may include the following modules:

[0096] A first work request acquisition module 401 is used to acquire a first work request sent by a target application, wherein the first work request is used for a first computing power cluster to request a remote direct memory access protocol to perform data transmission with a second computing power cluster, and the second computing power cluster is installed with a target application and a target middleware for transmitting data using the remote direct memory access protocol;

[0097] A first queue pair message sending module 402, configured to generate a first queue pair message of the remote direct memory access protocol based on the first work request, and send the first queue pair message to the target middleware;

[0098] A virtual private network tunnel establishing module 403, configured to establish a virtual private network tunnel between the first computing power cluster and the second computing power cluster based on the target middleware;

[0099] The first transmission data sending module 404 is used to obtain the first transmission data corresponding to the message of the first queue, and send the first transmission data to the second computing power cluster to execute the first work request based on the virtual private network tunnel.

[0100] In one embodiment of the present invention, the first work request acquisition module 401 may be used to:

[0101] After the first computing power cluster starts the monitoring and forwarding data service for the target middleware, a first work request sent by the target application is obtained.

[0102] In an embodiment of the present invention, the first queue to message sending module 402 may include:

[0103] A first queue message generating submodule, configured to convert the first work request into a first queue pair message of a remote direct memory access protocol according to a preset format;

[0104] A sending queue storage submodule, used for sending the first queue pair message to a preset sending queue;

[0105] The first queue pair message sending submodule is used to send the first queue pair message to the target middleware when the first queue pair message is located at the head of the sending queue and the previous message is sent.

[0106] In one embodiment of the present invention, the device further comprises:

[0107] a work completion instruction receiving module, configured to receive a work completion instruction sent by the second computing power cluster through the virtual private network tunnel, wherein the work completion instruction is an instruction for feeding back that the second computing power cluster completes the first work request based on the first transmission data;

[0108] A second queue pair message generating module, configured to generate a second queue pair message of the remote direct memory access protocol based on the work completion instruction;

[0109] The completion queue storage module is used to store the second queue pair message in a preset completion queue.

[0110] In one embodiment of the present invention, the virtual private network tunnel establishing module 403 may include:

[0111] A virtual private network tunnel establishment submodule is used to establish a virtual private network tunnel between the first computing power cluster and the second computing power cluster based on the first queue for the message through the target middleware.

[0112] In an embodiment of the present invention, the first transmission data sending module 404 may include:

[0113] A first network protocol message generation submodule, configured to convert the first queue pair message into a first network protocol message according to the first network protocol when the virtual private network tunnel is a tunnel supporting the first network protocol;

[0114] The first network protocol message sending submodule is used to send the first network protocol message to the second computing power cluster through the first network protocol tunnel.

[0115] In one embodiment of the present invention, the device further comprises:

[0116] A second network protocol message receiving module, used to receive a second network protocol message sent by the second computing power cluster, where the second network protocol message is generated when a second work request of a target application is processed in the second computing power cluster;

[0117] A third queue pair message data sending module, used for converting the second network protocol message into a third queue pair message data, and sending the third queue pair message data to a preset receiving queue;

[0118] A third queue message removal module, configured to remove the third queue message from the receiving queue when the third queue pair message is at the head of the receiving queue and the first computing power cluster has processed the previous message;

[0119] The second transmission data sending module is used to obtain the second transmission data corresponding to the third queue message, and send the second transmission data to the second computing power cluster based on the virtual private network tunnel.

[0120] In an embodiment of the present invention, by installing a target application and a target middleware for transmitting data using the remote direct memory access protocol in the first computing power cluster and the second computing power cluster, the middleware can intelligently identify, process and forward RDMA services, thereby establishing a virtual private network tunnel between the first computing power cluster and the second computing power cluster, and using the network tunnel to transmit transmission data corresponding to the RDMA service.

[0121] An embodiment of the present invention further provides an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and the computer program implements the above data transmission method when executed by the processor.

[0122] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above data transmission method is implemented.

[0123] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0124] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0125] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0126] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0129] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0130] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0131] The above is a detailed introduction to a data transmission method and device, an electronic device, and a storage medium. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a person skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A data transmission method, characterized in that: Applied to a first computing power cluster, the first computing power cluster is installed with a target application and a target middleware for transmitting data using a remote direct memory access protocol, the method comprising: Obtaining a first work request sent by a target application, the first work request being used by a first computing power cluster to request a remote direct memory access protocol to be used for data transmission with a second computing power cluster, the second computing power cluster being installed with a target application and a target middleware for transmitting data using the remote direct memory access protocol; Generate a first queue pair message of the remote direct memory access protocol based on the first work request, and send the first queue pair message to the target middleware, wherein the target middleware is set in kernel state for identifying and forwarding data; Establishing a virtual private network tunnel between the first computing power cluster and the second computing power cluster based on the target middleware; Obtain first transmission data corresponding to the message of the first queue, and send the first transmission data to the second computing power cluster based on the virtual private network tunnel to execute the first work request.

2. The method according to claim 1, characterized in that The obtaining of the first work request sent by the target application comprises: After the first computing power cluster starts the monitoring and forwarding data service for the target middleware, a first work request sent by the target application is obtained.

3. The method according to claim 1, characterized in that The step of generating a first queue pair message of a remote direct memory access protocol based on the first work request, and sending the first queue pair message to the target middleware includes: Converting the first work request into a first queue pair message of a remote direct memory access protocol according to a preset format; Sending the first queue pair message to a preset sending queue; When the first queue pair message is located at the head of the sending queue and the previous message is sent, the first queue pair message is sent to the target middleware.

4. The method according to claim 1, characterized in that: Also includes: receiving, through the virtual private network tunnel, a work completion instruction sent by the second computing power cluster, wherein the work completion instruction is an instruction for feeding back that the second computing power cluster completes the first work request based on the first transmission data; generating a second queue pair message of the remote direct memory access protocol based on the work completion instruction; The second queue pair message is stored in a preset completion queue.

5. The method according to claim 1, characterized in that The establishing of a virtual private network tunnel between the first computing power cluster and the second computing power cluster based on the target middleware includes: A virtual private network tunnel is established between the first computing power cluster and the second computing power cluster based on the first queue and the message through the target middleware.

6. The method according to claim 1, characterized in that The sending the first transmission data to the second computing power cluster based on the virtual private network tunnel to execute the first work request includes: When the virtual private network tunnel is a tunnel supporting a first network protocol, converting the first queue pair message into a first network protocol message according to the first network protocol; Send the first network protocol message to the second computing power cluster through the first network protocol tunnel.

7. The method according to claim 1, characterized in that Also includes: receiving a second network protocol message sent by the second computing power cluster, where the second network protocol message is generated when a second work request of a target application is processed in the second computing power cluster; Convert the second network protocol message into third queue pair message data, and send the third queue pair message data to a preset receiving queue; When the message of the third queue pair is at the head of the receiving queue and the first computing power cluster has processed the previous message, the message of the third queue pair is removed from the receiving queue; The second transmission data corresponding to the third queue message is obtained, and the second transmission data is sent to the second computing power cluster based on the virtual private network tunnel.

8. A data transmission device, characterized in that: Applied to a first computing power cluster, the first computing power cluster is installed with a target application and a target middleware for transmitting data using a remote direct memory access protocol, the device comprising: A first work request acquisition module is used to acquire a first work request sent by a target application, the first work request is used for a first computing power cluster to request to use a remote direct memory access protocol to perform data transmission with a second computing power cluster, and the second computing power cluster is installed with a target application and a target middleware for transmitting data using the remote direct memory access protocol; A first queue pair message sending module, used for generating a first queue pair message of the remote direct memory access protocol based on the first work request, and sending the first queue pair message to the target middleware, the target middleware is set in kernel state, and is used for identifying and forwarding data; A virtual private network tunnel establishment module, used to establish a virtual private network tunnel between the first computing power cluster and the second computing power cluster based on the target middleware; The first transmission data sending module is used to obtain the first transmission data corresponding to the message of the first queue, and send the first transmission data to the second computing power cluster to execute the first work request based on the virtual private network tunnel.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the data transmission method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the data transmission method according to any one of claims 1 to 7 is implemented.

Images