A data transmission method, apparatus, device, and medium

Abstract

The present application discloses a data transmission method, apparatus, device and medium, relating to the field of computer technology. This method utilizes the characteristics of low latency, low CPU overhead and high bandwidth of the RDMA protocol, that is, uses RDMA write / read operations to transmit data, uses RDMA rend / recive to transmit data operation resources and data transmission length, and manages I / O; utilizes the scalability of the Ethernet network, generates local private keys through device management and IP management, and through private key authentication management, ensures the security of access nodes and improves data transmission security; achieves the purpose of fast data transmission through multi-queue saturated data transmission. Thus, the effects of improving transmission security, reducing transmission latency and increasing transmission rate are achieved.

Description

Technical Field

[0001] The present invention relates to the field of computer technologies, and particularly to a data transmission method, apparatus, device, and medium. Background Art

[0002] Currently, with the development of technologies such as the Internet, cloud computing, the Internet of Things, and mobile Internet, data has grown explosively, bringing new challenges to traditional data transmission networks. In the face of high-performance computing, big data analysis, and high-concurrency, low-latency applications of surge-type I / O (Input / Output), the technical characteristics of the existing TCP / IP (Transmission Control Protocol / Internet Protocol) software and hardware architecture and high CPU (central processing unit) consumption of applications simply cannot meet the requirements of applications.

[0003] In summary, how to perform data transmission with low latency and high efficiency is an urgent problem to be solved currently. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a data transmission method, apparatus, device, and medium that can perform data transmission with low latency and high efficiency. The specific solutions are as follows:

[0005] In a first aspect, the present application discloses a data transmission method applied to a storage system. The storage system includes a first node and a second node. The method includes:

[0006] Obtain first information of the first node through the first node, and obtain second information of the second node through the second node;

[0007] Based on the first information and the second information, determine whether data can be transmitted between the first node and the second node through a ROCE port;

[0008] If data can be transmitted, determine a target queue for multi-queue transmission between the first node and the second node;

[0009] Send a target command representing data reading to the first node through the second node based on an RDMA read operation and second RDMA data transmission resources, and receive the target command through the first node based on an RDMA write operation and the second RDMA data transmission resources; the second RDMA data transmission resources are resources that the second node can send to the first node at any time through the target queue;

[0010] The first node sends the target data corresponding to the target command to the second node based on the RDMA write operation and the first RDMA data transmission resource, and the second node receives the target data based on the RDMA read operation and the first RDMA data transmission resource; the first RDMA data transmission resource is a resource that the first node sends to the second node through the multi-queue at any time.

[0011] Optionally, determining whether data can be transmitted between the first node and the second node through the ROCE port based on the first information and the second information includes:

[0012] The first node generates a first private key corresponding to the first node based on the first information, generates a first ROCE port private key of the ROCE network card corresponding to the first node based on the first private key, and then sends the first ROCE port private key to the second node;

[0013] The second node generates a second private key corresponding to the second node based on the second information, generates a second ROCE port private key of the ROCE network card corresponding to the second node based on the second private key, and then sends the second ROCE port private key to the first node;

[0014] The second node authenticates the first ROCE port private key, and the first node authenticates the second ROCE port private key to determine whether data can be transmitted between the first node and the second node through the ROCE port.

[0015] Optionally, before determining the target queue for multi-queue transmission between the first node and the second node, it further includes:

[0016] The first node determines the number of first queues for multi-queue transmission corresponding to the first node according to the first information, sends the number of first queues to the second node, and then receives the number of second queues;

[0017] The second node determines the number of second queues for multi-queue transmission corresponding to the second node according to the second information, sends the number of second queues to the first node, and then receives the number of first queues;

[0018] Correspondingly, determining the target queue for multi-queue transmission between the first node and the second node includes:

[0019] The first node and the second node respectively determine a target queue number for multi-queue transmission between the first node and the second node based on the first queue number and the second queue number, so as to determine a target queue for multi-queue transmission between the first node and the second node.

[0020] Optionally, the sending, by the first node, of the target data corresponding to the target command to the second node based on the RDMA write operation and the first RDMA data transmission resource includes:

[0021] The first node determines whether it is necessary to send the target data corresponding to the target command;

[0022] If it is necessary to send the target data, the first node sends the target data to the second node based on the RDMA write operation and the first RDMA data transmission resource.

[0023] Optionally, after the first node determines whether it is necessary to send the target data corresponding to the target command, the method further includes:

[0024] If it is not necessary to send the target data, the first node sends a first response message indicating the release of the target command to the second node based on the RDMA send operation and the first RDMA data transmission resource, and then the second node receives the first response message based on the RDMA Receive operation and the first RDMA data transmission resource and releases the target command based on the first response message.

[0025] Optionally, after the first node sends the target data corresponding to the target command to the second node based on the RDMA write operation and the first RDMA data transmission resource, the method further includes:

[0026] The first node determines a data address and a data length corresponding to the target data;

[0027] The first node constructs a first response message based on the data address, the data length, and an identifier of the target data sending status, and sends the first response message to the second node based on the RDMA send operation and the first RDMA data transmission resource, and then the second node obtains the first response message based on the RDMA Receive operation and the first RDMA data transmission resource.

[0028] Optionally, after the second node obtains the first response message based on the RDMA Receive operation and the first RDMA data transmission resource, the method further includes:

[0029] The second node sends a data transmission completion message to the first node based on the RDMA send operation and the second RDMA data transmission resource, and the first node receives the data transmission completion message based on the RDMA Receive operation and the second RDMA data transmission resource.

[0030] In a second aspect, the present application discloses a data transmission device applied to a storage system. The storage system includes a first node and a second node. The device includes:

[0031] An information acquisition module, configured to acquire first information of the first node through the first node and acquire second information of the second node through the second node;

[0032] A data transmission judgment module, configured to judge whether data can be transmitted between the first node and the second node through a ROCE port based on the first information and the second information;

[0033] A target queue determination module, configured to determine a target queue for multi-queue transmission between the first node and the second node if data transmission can be performed;

[0034] A command sending module, configured to send a target command representing data reading to the first node through the second node based on an RDMA read operation and a second RDMA data transmission resource; the second RDMA data transmission resource is a resource that the second node can send to the first node at any time through the target queue;

[0035] A command receiving module, configured to receive the target command through the first node based on an RDMA write operation and the second RDMA data transmission resource;

[0036] A data sending module, configured to send target data corresponding to the target command to the second node through the first node based on the RDMA write operation and a first RDMA data transmission resource; the first RDMA data transmission resource is a resource that the first node can send to the second node at any time through the multi-queue;

[0037] A data receiving module, configured to receive the target data through the second node based on the RDMA read operation and the first RDMA data transmission resource.

[0038] In a third aspect, the present application discloses an electronic device, including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, the foregoing disclosed data transmission method is implemented.

[0039] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the data transmission method disclosed above is implemented.

[0040] It can be seen that in the present application, the first node obtains the first information of the first node, and the second node obtains the second information of the second node; based on the first information and the second information, it is determined whether data can be transmitted between the first node and the second node through the ROCE port; if data can be transmitted, the target queue for multi-queue transmission between the first node and the second node is determined; the second node sends a target command representing data reading to the first node based on the RDMA read operation and the second RDMA data transmission resource, and the first node receives the target command based on the RDMA write operation and the second RDMA data transmission resource; the second RDMA data transmission resource is a resource that the second node sends to the first node at any time through the target queue; the first node sends the target data corresponding to the target command to the second node based on the RDMA write operation and the first RDMA data transmission resource, and the second node receives the target data based on the RDMA read operation and the first RDMA data transmission resource; the first RDMA data transmission resource is a resource that the first node sends to the second node at any time through the multi-queue. Thus, it can be seen that the present application uses the RDMA read operation and the RDMA write operation to achieve direct transmission between two nodes without going through the operating system, and has the characteristics of low latency, low CPU overhead, and high bandwidth; the present application performs multi-queue transmission through the target queue and transmits the first RDMA data transmission resource and the second RDMA data transmission resource at any time, realizing multi-queue saturated data transmission and accelerating the transmission speed; after the present application determines that data can be transmitted between the first node and the second node, data transmission is allowed, enhancing security and avoiding transmission errors. Description of the Drawings

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0042] Figure 1 It is a flowchart of a data transmission method provided by the present application;

[0043] Figure 2 Schematic diagram of a data transmission system structure provided by this application

[0044] Figure 3 Flowchart of a specific data transmission method provided by this application

[0045] Figure 4 Schematic diagram of a data transmission device structure provided by this application

[0046] Figure 5 Structural diagram of an electronic device provided by this application Specific implementation manners

[0047] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0048] Currently, with the development of technologies such as the Internet, cloud computing, the Internet of Things, and mobile Internet, data has grown explosively, and at the same time, it has brought new challenges to traditional data transmission networks. Facing high-performance computing, big data analysis, and high concurrency and low latency applications of surge-type IO (Input / Output, I / O input / output), the technical characteristics of the existing TCP / IP (Transmission Control Protocol / Internet Protocol) software and hardware architecture and high CPU (central processing unit) consumption of applications simply cannot meet the requirements of applications.

[0049] To overcome the above problems, this application provides a data transmission solution that can perform data transmission with low latency and high efficiency.

[0050] See Figure 1 As shown, an embodiment of this application discloses a data transmission method applied to a storage system. The storage system includes a first node and a second node. The method includes:

[0051] Step S11: Obtain first information of the first node through the first node, and obtain second information of the second node through the second node.

[0052] In the embodiment of this application, as Figure 2 shown, both the first node and the second node have Figure 2The data transmission system is extended based on an ROCE (RDMA over Ethernet) network card, where RDMA stands for Remote Direct Memory Access. The data transmission system includes OFED, RDMA_CORE, a memory management module (Memory manage), an IP management module (IP manage), a device management module (Device manage), an authentication management module (Auth manage), an IO management module (IO Build manage), a sending management module (IO send manage), and a receiving management module (IO revive manage). Additionally, there is a network device driver (NetdeviceDriver) in the system, and there is also an RDMA-related environment deployment, including OFED (driver) and RDMA_CORE (database). The first node and the second node perform data transmission based on the transmission system.

[0053] It should be noted that the RDMA protocol uses the Messaging passing through kernel method to directly transfer data from the memory of one computer to another computer without the intervention of the operating systems of both parties, achieving the characteristics of low latency, low CPU overhead, and high bandwidth. To be compatible with the existing Ethernet network, ROCE has been developed for RDMA, enabling the rapid development of RDMA technology.

[0054] In the embodiment of this application, before obtaining the first information of the first node through the first node and obtaining the second information of the second node through the second node, that is, when the storage system node is started, the deployment of the RDMA-related environments OFED and RDMA_CORE is implemented.

[0055] In the embodiment of this application, before obtaining the first information of the first node through the first node and obtaining the second information of the second node through the second node, it is also necessary to apply for memory from the corresponding OS (Operating System) through the memory management module of the first node and apply for memory from the corresponding OS through the memory management module of the second node. It should be noted that the memory management module is used to manage each module in the data transmission system (ROCE shared internal).

[0056] In the embodiment of the present application, the obtaining of the first information of the first node through the first node and the obtaining of the second information of the second node through the second node are specifically as follows: reading the first information such as the PN (Profinet, industrial Ethernet interface), CPU, and memory of the first node through the device management module of the first node, and reading the second information such as the PN, CPU, and memory of the second node through the device management module of the second node.

[0057] Step S12: Based on the first information and the second information, determine whether data can be transmitted between the first node and the second node through the ROCE port.

[0058] Step S13: If data transmission is possible, determine the target queue for multi-queue transmission between the first node and the second node.

[0059] In the embodiment of the present application, before determining the target queue for multi-queue transmission between the first node and the second node, it further includes: determining, by the first node according to the first information, the first number of queues for multi-queue transmission corresponding to the first node, and sending the first number of queues to the second node, and then receiving the second number of queues; determining, by the second node according to the second information, the second number of queues for multi-queue transmission corresponding to the second node, and sending the second number of queues to the first node, and then receiving the first number of queues. It should be noted that while determining the first number of queues, the size of the first memory pool can be determined, and while determining the second number of queues, the size of the second memory pool can be determined.

[0060] It should be noted that specifically, the memory management module of the first node determines the first number of queues for multi-queue transmission corresponding to the first node according to the first information, and sends the first number of queues to the second node, and then receives the second number of queues; the memory management module of the second node determines the second number of queues for multi-queue transmission corresponding to the second node according to the second information, and sends the second number of queues to the first node, and then receives the first number of queues. It should be noted that the process of sending the first number of queues to the second node and then receiving the second number of queues, and the process of sending the second number of queues to the first node and then receiving the first number of queues are both transmitted through a pre-existing queue between the first node and the second node.

[0061] In the embodiments of the present application, determining the target queue for multi-queue transmission between the first node and the second node includes: respectively determining, by the first node and the second node, the number of target queues for multi-queue transmission between the first node and the second node based on the first queue number and the second queue number, so as to determine the target queue for multi-queue transmission between the first node and the second node. It should be noted that the memory management module of the first node and the memory management module of the second node respectively determine the number of target queues for multi-queue transmission between the first node and the second node based on the first queue number and the second queue number, so as to determine the target queue for multi-queue transmission between the first node and the second node; It should be noted that the memory management module obtains the number of connections that the corresponding node can create, as well as the number of threads, QP numbers, and IO numbers of the connections with each other node.

[0062] It should be noted that the first node and the second node can also simultaneously obtain the queue numbers of other nodes and determine the queue numbers between them and other nodes.

[0063] Step S14: The second node sends a target command representing data reading to the first node based on the RDMA read operation and the second RDMA data transmission resource, and the first node receives the target command based on the RDMA write operation and the second RDMA data transmission resource; the second RDMA data transmission resource is the resource that the second node sends to the first node at any time through the target queue.

[0064] In the embodiments of the present application, the sending management module of the second node sends a target command representing data reading to the first node based on the RDMA read operation and the second RDMA data transmission resource, and the receiving management module of the first node receives the target command based on the RDMA write operation and the second RDMA data transmission resource; the second RDMA data transmission resource is the resource that the second node sends to the first node at any time through the target queue. It should be noted that the second node circularly sends the RDMA operation elements (address, length, key) of the second node's memory according to the memory pool information. When sending, regardless of whether the first node needs data transmission, it transmits the second RDMA data transmission resource of the second node to the first node to achieve the purpose of saturated data transmission resources. It should be noted that the memory information is sent by the IO management module to the sending management module. It should be noted that the IO management module is used to manage the assembly, disassembly, packetization, sending, and receiving of node IO.

[0065] Step S15: The first node sends the target data corresponding to the target command to the second node based on the RDMA write operation and the first RDMA data transmission resource, and the second node receives the target data based on the RDMA read operation and the first RDMA data transmission resource; the first RDMA data transmission resource is the resource that the first node sends to the second node at any time through the multi-queue.

[0066] In the embodiment of the present application, the step of the first node sending the target data corresponding to the target command to the second node based on the RDMA write operation and the first RDMA data transmission resource includes: the first node determines whether it is necessary to send the target data corresponding to the target command; if it is necessary to send the target data, the first node sends the target data to the second node based on the RDMA write operation and the first RDMA data transmission resource. It should be noted that after the receiving management module of the first node receives the target command, it needs to notify the command to the IO management module of the first node, and the IO management module of the first node determines whether it is necessary to send the target data corresponding to the target command; if it is necessary to send the target data, the IO management module needs to, according to the upper-layer task requirements, send the target data through the data transmission resource (the second RDMA data transmission resource) sent by the receiving management module to the sending management module (that is, the sending management module receives the IO transmission command sent by the IO management module according to the data transmission needs of the local node, and sends the data or command to the peer node), and then the sending management module of the first node sends the target data to the second node based on the RDMA write operation and the first RDMA data transmission resource. It should be noted that before using the IO management module, it is also necessary to obtain the number of IO resource pools and the number of threads from the memory management module through the IO management module.

[0067] It should be noted that when sending the target data through the data transmission resource sent by the receiving management module to the sending management module, if the data is too long, it is split and sent to the data management module in batches, or can be sent from the sending management module of the first node to the second node in batches.

[0068] It should be noted that if it is not necessary to send the target data, the first node sends a first response message indicating the release of the target command to the second node based on the RDMA send operation and the first RDMA data transmission resource, and then the second node receives the first response message based on the RDMA Receive operation and the first RDMA data transmission resource and releases the target command based on the first response message. It should be noted that if it is not necessary to send the target data, the IO management module of the first node needs to notify the sending management module of the first node, and the sending management module of the first node sends a first response message indicating the release of the target command to the receiving management module of the second node based on the RDMA send operation and the first RDMA data transmission resource, and then the receiving management module of the second node receives the first response message based on the RDMA Receive operation and the first RDMA data transmission resource and releases the target command based on the first response message; that is, if data does not need to be sent, only the sending management module needs to be notified to send the response of the command, so that the second node releases the command and resends the second RDMA data transmission resource.

[0069] In an embodiment of the present application, after the first node sends the target data corresponding to the target command to the second node based on the RDMA write operation and the first RDMA data transmission resource, the following steps are further included: determining, by the first node, the data address and data length corresponding to the target data; constructing, by the first node, a first response message based on the data address, data length, and the identifier of the target data sending status, and sending the first response message to the second node based on the RDMA send operation and the first RDMA data transmission resource, and then obtaining, by the second node, the first response message based on the RDMA Receive operation and the first RDMA data transmission resource. It should be noted that after the IO management module determines that data transmission is required, the data address and data length corresponding to the target data organized by the first node are notified to the sending management module. After the target data is sent to the second node, the data address and data length corresponding to the target data are determined by the sending management module of the first node; constructing, by the sending management module of the first node, a first response message based on the data address, data length, and the identifier of the target data sending status, and sending the first response message to the receiving management module of the second node based on the RDMA send operation and the first RDMA data transmission resource, and then obtaining, by the receiving management module of the second node, the first response message based on the RDMA Receive operation and the first RDMA data transmission resource. It should be noted that the IO management module of the second node can parse the first response message to obtain the data address, data length, and the target data sending status. At this time, the sending status indicates that the data transmission is successful; in addition, the sending status in other cases can also indicate data transmission failure and whether there is data.

[0070] In an embodiment of the present application, after obtaining the first response message by the second node based on the RDMA Receive operation and the first RDMA data transmission resource, the following steps are further included: the second node sends a data transmission completion message to the first node based on the RDMA send operation and the second RDMA data transmission resource, and the first node receives the data transmission completion message based on the RDMA Receive operation and the second RDMA data transmission resource. It should be noted that the sending management module of the second node sends a data transmission completion message to the first node based on the RDMA send operation and the second RDMA data transmission resource, and the receiving management module of the first node receives the data transmission completion message based on the RDMA Receive operation and the second RDMA data transmission resource. Then, the receiving management module sends the data transmission completion message to the IO management module of the first node, and the IO management module groups, parses the data of the data transmission completion message, and notifies the corresponding module according to the parsing result.

[0071] It should be noted that the RDMA Receive operation and the RDMA send operation also have the characteristics of low latency, low CPU overhead, and high bandwidth.

[0072] It can be seen that in this application, the first information of the first node is obtained through the first node, and the second information of the second node is obtained through the second node; based on the first information and the second information, it is determined whether data can be transmitted between the first node and the second node through the ROCE port; if data can be transmitted, the target queue for multi-queue transmission between the first node and the second node is determined; the target command representing data reading is sent to the first node through the second node based on the RDMA read operation and the second RDMA data transmission resource, and the target command is received by the first node based on the RDMA write operation and the second RDMA data transmission resource; the second RDMA data transmission resource is the resource that the second node can send to the first node at any time through the target queue; the target data corresponding to the target command is sent to the second node through the first node based on the RDMA write operation and the first RDMA data transmission resource, and the target data is received by the second node based on the RDMA read operation and the first RDMA data transmission resource; the first RDMA data transmission resource is the resource that the first node can send to the second node at any time through the multi-queue. Thus, in this application, direct transmission between two nodes is achieved by using the RDMA read operation and the RDMA write operation, without going through the operating system, and it has the characteristics of low latency, low CPU overhead, and high bandwidth; in this application, multi-queue transmission is performed through the target queue, and the first RDMA data transmission resource and the second RDMA data transmission resource are transmitted at any time, realizing saturated multi-queue data transmission and accelerating the transmission speed; in this application, data transmission is only allowed after it is determined that data can be transmitted between the first node and the second node, enhancing security and avoiding transmission errors.

[0073] See Figure 3 As shown, an embodiment of this application discloses a specific data transmission method, which is applied to a storage system. The storage system includes a first node and a second node. The method includes:

[0074] Step S21: Obtain the first information of the first node through the first node, and obtain the second information of the second node through the second node.

[0075] Among them, for a more specific processing procedure of step S21, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be elaborated here.

[0076] Step S22: The first node generates a first private key corresponding to the first node based on the first information, generates a first ROCE port private key of the ROCE network card corresponding to the first node based on the first private key, and then sends the first ROCE port private key to the second node.

[0077] In the embodiment of the present application, the device management module of the first node generates a first private key corresponding to the first node based on the first information, and the IP management module of the first node generates a first ROCE port private key of the ROCE network card corresponding to the first node based on the first private key, and then sends the first ROCE port private key to the second node.

[0078] It should be noted that the device management module generates an authentication private key with certain rules according to the first information for node access authentication; the IP management module is used to manage the ROCE network interface, identify and create ROCE ports, and configure, manage, and delete the IP information of the ROCE network card; the port private key is generated according to the private key of the device management module for authentication.

[0079] Step S23: The second node generates a second private key corresponding to the second node based on the second information, generates a second ROCE port private key of the ROCE network card corresponding to the second node based on the second private key, and then sends the second ROCE port private key to the first node.

[0080] In the embodiment of the present application, the device management module of the second node generates a second private key corresponding to the second node based on the second information, and the IP management module of the first node generates a second ROCE port private key of the ROCE network card corresponding to the second node based on the second private key, and then sends the second ROCE port private key to the first node.

[0081] It should be noted that the device management module generates an authentication private key with certain rules according to the second information for node access authentication; the IP management module is used to manage the ROCE network interface, identify and create ROCE ports, and configure, manage, and delete the IP information of the ROCE network card; the port private key is generated according to the private key of the device management module for authentication.

[0082] Step S24: The second node authenticates the first ROCE port private key, and the first node authenticates the second ROCE port private key to determine whether data can be transmitted between the first node and the second node through the ROCE port.

[0083] In the embodiment of the present application, the first node and the second node authenticate each other's private keys to allow data transmission between the first node and the second node through the ROCE port after successful authentication. It should be noted that the private key of the first ROCE port is authenticated through the authentication management module of the second node, and the private key of the second ROCE port is authenticated through the authentication management module of the first node.

[0084] It should be noted that the authentication management module is used to manage authentication and manage connections with other nodes. By verifying the port private key during the established connection process with the peer node, it is determined whether access to the local node is allowed.

[0085] In the embodiment of the present application, by utilizing the scalability of the Ethernet network and through authentication management, the security of the access node is ensured, and the transmission security is further improved.

[0086] Step S25: If data transmission is possible, determine the target queue for multi-queue transmission between the first node and the second node.

[0087] Among them, for a more specific processing procedure of step S25, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be elaborated herein.

[0088] Step S26: The second node sends a target command representing data reading to the first node based on the RDMA read operation and the second RDMA data transmission resource, and the first node receives the target command based on the RDMA write operation and the second RDMA data transmission resource; the second RDMA data transmission resource is the resource that the second node can send to the first node at any time through the target queue.

[0089] Among them, for a more specific processing procedure of step S26, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be elaborated herein.

[0090] Step S27: The first node sends the target data corresponding to the target command to the second node based on the RDMA write operation and the first RDMA data transmission resource, and the second node receives the target data based on the RDMA read operation and the first RDMA data transmission resource; the first RDMA data transmission resource is the resource that the first node can send to the second node at any time through the multi-queue.

[0091] Among them, for a more specific processing procedure of step S27, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be elaborated herein.

[0092] It can be seen that the first information of the first node is obtained through the first node, and the second information of the second node is obtained through the second node; the first private key corresponding to the first node is generated by the first node based on the first information, and the first ROCE port private key of the ROCE network card corresponding to the first node is generated based on the first private key, and then the first ROCE port private key is sent to the second node; the second private key corresponding to the second node is generated by the second node based on the second information, and the second ROCE port private key of the ROCE network card corresponding to the second node is generated based on the second private key, and then the second ROCE port private key is sent to the first node; the first ROCE port private key is authenticated by the second node, and the second ROCE port private key is authenticated by the first node to determine whether data can be transmitted between the first node and the second node through the ROCE port; if data can be transmitted, the target queue for multi-queue transmission between the first node and the second node is determined; the target command representing data reading is sent to the first node by the second node based on the RDMA read operation and the second RDMA data transmission resource, and the target command is received by the first node based on the RDMA write operation and the second RDMA data transmission resource; the second RDMA data transmission resource is the resource that the second node sends to the first node at any time through the target queue; the target data corresponding to the target command is sent to the second node by the first node based on the RDMA write operation and the first RDMA data transmission resource, and the target data is received by the second node based on the RDMA read operation and the first RDMA data transmission resource; the first RDMA data transmission resource is the resource that the first node sends to the second node at any time through the multi-queue. Thus, it can be seen that the present application uses the RDMA read operation and the RDMA write operation to achieve direct transmission between two nodes without going through the operating system, and has the characteristics of low latency, low CPU overhead, and high bandwidth; the present application performs multi-queue transmission through the target queue, and transmits the first RDMA data transmission resource and the second RDMA data transmission resource at any time, realizing multi-queue saturated data transmission and accelerating the transmission speed; after the present application determines that data can be transmitted between the first node and the second node by using the second ROCE port private key and the first ROCE port private key, data transmission is allowed, enhancing security and avoiding transmission errors.

[0093] See Figure 2 For the data transmission system described above, the functions of each module are as follows:

[0094] 1. When the storage system node starts up, deploy the RDMA-related environments OFED and RDMA_CORE.

[0095] 2. The memory management module applies for memory from the OS to manage each module within the ROCE sharing. Based on node information such as device CPU and memory size, determine the memory pool size and the number of queues for multi-queue transmission.

[0096] 3. The device manage (device management module, also known as the node management module) is used to read information such as the PN, CPU, and memory of this device, and generate an authentication private key with certain rules based on the device information for device access authentication.

[0097] 4. The IP manage (IP management module) is used to manage the ROCE network interface, identify and create ROCE ports, and configure, manage, and delete the IP information of the ROCE network card; generate a port private key based on the private key of the device manage module for authentication.

[0098] 5. The authentication management module is used to manage authentication and the connection with other nodes. Through the private key during the connection establishment process, verify whether the peer node can access this node. Obtain the number of connections that can be created on this node, the thread arrangement for connecting with each other device, the QP quantity, and the IO quantity through the memory management module.

[0099] 6. The IO management module is used to manage the assembly, disassembly, packetization, sending, and receiving of local IO. First, obtain the number of IO resource pools and the number of threads from the memory management module. According to the upper-layer task requirements, send the data through the data transmission resources sent by the receive management module, and send the data to be transmitted to the send management module; if the data is long, perform splitting processing; at the same time, process the data transmission completion message received by the receive module, packetize and parse the data, and notify the corresponding module according to the parsing result; after the data is sent, use the RDMA send and receive operations to notify the peer.

[0100] 7. Sending Management Module: First, according to the memory pool information issued by the IO Management Module, it circulates and sends the RDMA operation elements (address, length, key) of the local memory. When sending, regardless of whether the remote end needs data transmission, it transmits the local RDMA data transmission resources to achieve the purpose of saturating the data transmission resources. Second, according to the local data transmission needs, it receives the IO transmission commands issued by the IO Management Module and sends the data or commands to the peer node. Since the peer end cannot obtain the data length for RDMA read / write operations, the transmission length of the data is encapsulated in the response message and sent to the peer end through the RDMA send operation. The peer end can obtain the length and status (success, failure, whether there is data) of this data transmission through the RDMA Receive operation.

[0101] 8. Receiving Management Module, which is used to receive the commands sent by the remote end and the responses corresponding to the commands sent locally to the peer end. When receiving the commands from the peer end, it notifies the IO Management Module. The IO Management Module will decide whether to send data according to its own needs. If data needs to be sent, it will notify the sending management module of the organized local data address and length. After the data transmission is completed, it sends the response of this command. If data does not need to be sent, it only needs to notify the sending management module to send the response of this command, so that the peer end releases this command and re-sends the data transmission resources.

[0102] In summary, this application proposes a low-latency and high-efficiency control system for saturated communication based on ROCE network cards. This method utilizes the low-latency, low CPU overhead, and high-bandwidth characteristics of the RDMA protocol (RDMA read / write send / recive operations) (using the DMA characteristics of RDMA read / write operations to send data; using RDMA end / recive to transfer data transmission resources and data transmission length to manage IO); utilizes the scalability of the Ethernet network, generates local private keys through device management and IP management, and ensures the security of access nodes through private key authentication management to improve data transmission security; achieves the purpose of fast data transmission through multi-queue saturated data transmission. Thus, it achieves the effects of improving transmission security, reducing transmission latency, and increasing transmission rate.

[0103] It should be noted that the RDMA protocol uses the Messaging passing through kernel method to directly transfer data from the memory of one computer to another computer without the intervention of the operating systems of both parties, realizing the characteristics of low latency, low CPU overhead, and high bandwidth. To be compatible with the existing Ethernet network, RDMA has developed ROCE, which has enabled the rapid development of RDMA technology.

[0104] SeeFigure 4 As shown in Figure 4 , an embodiment of the present application discloses a data transmission device, which is applied to a storage system. The storage system includes a first node and a second node. The device includes:

[0105] An information acquisition module 11, configured to acquire first information of the first node through the first node, and acquire second information of the second node through the second node;

[0106] A data transmission judgment module 12, configured to judge whether data can be transmitted between the first node and the second node through an ROCE port based on the first information and the second information;

[0107] A target queue determination module 13, configured to determine a target queue for multi-queue transmission between the first node and the second node if data transmission can be performed;

[0108] A command sending module 14, configured to send a target command representing data reading to the first node through the second node based on an RDMA read operation and second RDMA data transmission resources; the second RDMA data transmission resources are resources that the second node can send to the first node at any time through the target queue;

[0109] A command receiving module 15, configured to receive the target command through the first node based on an RDMA write operation and the second RDMA data transmission resources;

[0110] A data sending module 16, configured to send target data corresponding to the target command to the second node through the first node based on the RDMA write operation and first RDMA data transmission resources; the first RDMA data transmission resources are resources that the first node can send to the second node at any time through the multi-queue;

[0111] A data receiving module 17, configured to receive the target data through the second node based on the RDMA read operation and the first RDMA data transmission resources.

[0112] Among them, for the more specific working processes of the above-mentioned various modules, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be elaborated herein.

[0113] It can be seen that in this application, the first information of the first node is obtained through the first node, and the second information of the second node is obtained through the second node; based on the first information and the second information, it is determined whether data can be transmitted between the first node and the second node through the ROCE port; if data can be transmitted, the target queue for multi-queue transmission between the first node and the second node is determined; the second node sends a target command representing data reading to the first node based on the RDMA read operation and the second RDMA data transmission resource, and the first node receives the target command based on the RDMA write operation and the second RDMA data transmission resource; the second RDMA data transmission resource is the resource that the second node can send to the first node at any time through the target queue; the first node sends the target data corresponding to the target command to the second node based on the RDMA write operation and the first RDMA data transmission resource, and the second node receives the target data based on the RDMA read operation and the first RDMA data transmission resource; the first RDMA data transmission resource is the resource that the first node can send to the second node at any time through the multi-queue. Thus, this application realizes direct transmission between two nodes by using the RDMA read operation and the RDMA write operation, without going through the operating system, and has the characteristics of low latency, low CPU overhead, and high bandwidth; this application performs multi-queue transmission through the target queue, and transmits the first RDMA data transmission resource and the second RDMA data transmission resource at any time, realizing multi-queue saturated data transmission and accelerating the transmission speed; after this application determines that data can be transmitted between the first node and the second node, data transmission is allowed, enhancing security and avoiding transmission errors.

[0114] Furthermore, the embodiment of this application also provides an electronic device. Figure 5 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment. The content in the figure should not be regarded as any limitation on the scope of use of this application.

[0115] Figure 5 It is a schematic structural diagram of an electronic device 20 provided by an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, an input / output interface 24, a communication interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps of the data transmission method disclosed in any of the foregoing embodiments.

[0116] In this embodiment, the power supply 23 is used to provide operating voltages for the various hardware devices on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and no specific limitation is imposed on it here; the input / output interface 24 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application needs, and no specific limitation is made here.

[0117] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, a random access memory, a magnetic disk, an optical disk, etc. The memory 22 can include a random access memory as the operating memory and a non-volatile memory for storage purposes of external memory. The storage resources thereon include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0118] Among them, the operating system 221 is used to manage the various hardware devices and the computer program 222 on the electronic device 20 through the source host. The operating system 221 can be Windows, Unix, Linux, etc. The computer program 222 can further include computer programs capable of performing other specific tasks in addition to the computer program capable of performing the data transmission method executed by the electronic device 20 disclosed in any of the foregoing embodiments.

[0119] In this embodiment, the input / output interface 24 can specifically include, but is not limited to, a USB interface, a hard disk reading interface, a serial interface, a voice input interface, a fingerprint input interface, etc.

[0120] Furthermore, an embodiment of this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the foregoing disclosed data transmission method is implemented.

[0121] For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and no further elaboration will be made here.

[0122] The computer-readable storage medium mentioned here includes a random access memory (Random Access Memory, RAM), memory, read-only memory (Read-Only Memory, ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, magnetic disks, or optical disks or any other form of storage medium known in the technical field. Among them, when the computer program is executed by a processor, the foregoing data transmission method is implemented. For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and no further elaboration will be made here.

[0123] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the data transmission methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method section.

[0124] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0125] The steps of the algorithms described in combination with the embodiments disclosed herein can be directly implemented by hardware, software modules executed by a processor, or a combination of the two. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the technical field.

[0126] Finally, it should also be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including an..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0127] The above has introduced in detail a data transmission method, device, equipment and medium provided by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A data transmission method, characterized in that Applied to a storage system, the storage system includes a first node and a second node, and the method includes: Obtain first information of the first node through the first node, and obtain second information of the second node through the second node; Based on the first information and the second information, determine whether data can be transmitted between the first node and the second node through a ROCE port; If data transmission is possible, determine a target queue for multi-queue transmission between the first node and the second node; Through the second node, send a target command representing data reading to the first node based on an RDMA read operation and a second RDMA data transmission resource, and receive the target command by the first node based on an RDMA write operation and the second RDMA data transmission resource; the second RDMA data transmission resource is a resource that the second node sends to the first node at any time through the target queue; Through the first node, send target data corresponding to the target command to the second node based on the RDMA write operation and a first RDMA data transmission resource, and receive the target data by the second node based on the RDMA read operation and the first RDMA data transmission resource; the first RDMA data transmission resource is a resource that the first node sends to the second node at any time through the multi-queue; The determining whether data can be transmitted between the first node and the second node through a ROCE port based on the first information and the second information includes: Through the first node, generate a first private key corresponding to the first node based on the first information, generate a first ROCE port private key of the ROCE network card corresponding to the first node based on the first private key, and then send the first ROCE port private key to the second node; Through the second node, generate a second private key corresponding to the second node based on the second information, generate a second ROCE port private key of the ROCE network card corresponding to the second node based on the second private key, and then send the second ROCE port private key to the first node; Authenticate the first ROCE port private key through the second node, and authenticate the second ROCE port private key through the first node to determine whether data can be transmitted between the first node and the second node through a ROCE port.

2. The data transmission method according to claim 1, wherein Before determining the target queue for multi-queue transmission between the first node and the second node, further include: Through the first node, determine a first queue number for multi-queue transmission corresponding to the first node according to the first information, send the first queue number to the second node, and then receive a second queue number; Through the second node, determine a second queue number for multi-queue transmission corresponding to the second node according to the second information, send the second queue number to the first node, and then receive the first queue number; Correspondingly, determining the target queue for multi-queue transmission between the first node and the second node includes: The first node and the second node respectively determine the number of target queues for multi-queue transmission between the first node and the second node based on the first queue number and the second queue number, so as to determine the target queue for multi-queue transmission between the first node and the second node.

3. The data transmission method according to claim 1, characterized in that, The first node sending the target data corresponding to the target command to the second node based on the RDMA write operation and the first RDMA data transmission resource includes: The first node determines whether it is necessary to send the target data corresponding to the target command; If it is necessary to send the target data, the first node sends the target data to the second node based on the RDMA write operation and the first RDMA data transmission resource.

4. The data transmission method according to claim 3, wherein After the first node determines whether it is necessary to send the target data corresponding to the target command, it further includes: If it is not necessary to send the target data, the first node sends a first response message indicating the release of the target command to the second node based on the RDMA send operation and the first RDMA data transmission resource, and then the second node receives the first response message based on the RDMA Receive operation and the first RDMA data transmission resource and releases the target command based on the first response message.

5. The data transmission method according to any one of claims 1 to 4, characterized in that After the first node sends the target data corresponding to the target command to the second node based on the RDMA write operation and the first RDMA data transmission resource, it further includes: The first node determines the data address and data length corresponding to the target data; The first node constructs a first response message based on the data address, data length, and the identifier of the target data sending status, and sends the first response message to the second node based on the RDMA send operation and the first RDMA data transmission resource, and then the second node obtains the first response message based on the RDMA Receive operation and the first RDMA data transmission resource.

6. The data transmission method according to claim 5, wherein After the second node obtains the first response message based on the RDMA Receive operation and the first RDMA data transmission resource, it further includes: The second node sends a data transmission completion message to the first node based on the RDMA send operation and the second RDMA data transmission resource, and the first node receives the data transmission completion message based on the RDMA Receive operation and the second RDMA data transmission resource.

7. A data transmission device, characterized in that, Applied to a storage system, the storage system includes a first node and a second node, and the device includes: An information acquisition module, configured to acquire first information of the first node through the first node, and acquire second information of the second node through the second node; A data transmission judgment module, configured to judge whether data can be transmitted between the first node and the second node through a ROCE port based on the first information and the second information; A target queue determination module, configured to determine a target queue for multi-queue transmission between the first node and the second node if data transmission can be performed; A command sending module, configured to send a target command representing data reading to the first node through the second node based on an RDMA read operation and a second RDMA data transmission resource; the second RDMA data transmission resource is a resource that the second node can send to the first node at any time through the target queue; A command receiving module, configured to receive the target command through the first node based on an RDMA write operation and the second RDMA data transmission resource; A data sending module, configured to send target data corresponding to the target command to the second node through the first node based on the RDMA write operation and a first RDMA data transmission resource; the first RDMA data transmission resource is a resource that the first node can send to the second node at any time through the multi-queue; A data receiving module, configured to receive the target data through the second node based on the RDMA read operation and the first RDMA data transmission resource; The data transmission judgment module is specifically configured to: generate a first private key corresponding to the first node by the first node based on the first information, generate a first ROCE port private key of the ROCE network card corresponding to the first node based on the first private key, and then send the first ROCE port private key to the second node; generate a second private key corresponding to the second node by the second node based on the second information, generate a second ROCE port private key of the ROCE network card corresponding to the second node based on the second private key, and then send the second ROCE port private key to the first node; authenticate the first ROCE port private key by the second node and authenticate the second ROCE port private key by the first node to judge whether data can be transmitted between the first node and the second node through the ROCE port.

8. An electronic device, characterized in that, It includes a processor and a memory; wherein, when the processor executes a computer program stored in the memory, the data transmission method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that, For storing a computer program; wherein, when the computer program is executed by a processor, the data transmission method according to any one of claims 1 to 6 is implemented.

Images