Data transmission system, method, device, computer equipment and storage medium

Abstract

The present application relates to a data transmission system, method, device, computer equipment, storage medium and computer program product, wherein the data transmission system includes multiple hosts, the host includes multiple nodes, the first node is used to determine the data transmission information carried when receiving a data transmission request, and determine the second node that meets the first optimized transmission condition among the multiple nodes included in the first host, and send the data transmission information; the second node is used to determine the third node that meets the second optimized transmission condition among the multiple nodes included in the second host based on the data transmission information, and send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node; the third node is used to send the target data from the third node to the destination storage address. By adopting the present application, the optimal transmission path can be used for data transmission, and the efficiency of data transmission can be improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a data transmission system, method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the rapid development of NUMA (Non Uniform Memory Access) architecture, the non-uniform memory access architecture has been widely developed in the field of data transmission. NUMA architecture is a distributed storage system including multiple hosts, each host includes multiple nodes, and the node is deployed with one or more of communication network cards, memory and hard disk. Data will be transmitted through the network between the hosts and stored in the hard disk of the host at the other end. During the data transmission process, data transmission between nodes across hosts will occur, so it is necessary to determine the data transmission path.

[0003] In the related art, when determining a data transmission path, the data transmission nodes are mainly determined randomly, which may result in a transmission path with a long transmission distance, resulting in low data transmission efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a data transmission method, apparatus, computer device, computer-readable storage medium and computer program product that can improve data transmission efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a data transmission system, wherein the data transmission system includes multiple hosts, and the host includes multiple nodes:

[0006] The first node is configured to, upon receiving a data transmission request, determine data transmission information carried by the data transmission request, determine a second node satisfying a first optimized transmission condition from among multiple nodes included in the first host, and send the data transmission information to the second node, wherein the data transmission information includes target data and a destination storage address, and the first host is a host where the first node is located;

[0007] The second node is used to determine, based on the data transmission information, a third node that meets the second optimized transmission condition among the multiple nodes included in the second host, and send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node, wherein the second host is a host where the node corresponding to the destination storage address is located;

[0008] The third node is used to send the target data from the third node to the destination storage address based on the data transmission information.

[0009] Based on the above scheme, the optimal transmission path for data transmission can be determined, and when data is transmitted across nodes and across hosts, it is ensured that the optimal transmission path can be used for data transmission, thereby improving the efficiency of data transmission and reducing the resource overhead of the data transmission system for data transmission.

[0010] In one embodiment, the first node is specifically configured to:

[0011] Obtaining first distance information between nodes in the first host corresponding to the first node;

[0012] Based on the first distance information, a second node that is closest to the first node and is deployed with a communication network card is determined.

[0013] Based on the above scheme, the nearest communication network card can be accurately determined to ensure data transmission efficiency.

[0014] In one embodiment, the second node is specifically used for:

[0015] Obtaining second distance information between nodes in the second host;

[0016] Based on the second distance information, a third node which is closest to the node corresponding to the destination storage address and is deployed with a communication network card is determined among the multiple nodes included in the second host.

[0017] Based on the above scheme, the nearest communication network card can be accurately determined to ensure data transmission efficiency.

[0018] In one embodiment, the third node is specifically used for:

[0019] Based on the second distance information, a fourth node closest to the third node and configured with a first storage space is determined among the multiple nodes included in the second host, and data transmission information including target data and a destination storage address is sent to the first storage space of the fourth node.

[0020] Based on the above scheme, the nearest communication network card can be accurately determined to ensure data transmission efficiency.

[0021] In one embodiment, the fourth node is specifically used for:

[0022] The data transmission information is received, and the target data in the data transmission information is sent to the destination storage address.

[0023] Based on the above scheme, the nearest node equipped with memory can be accurately determined to ensure data transmission efficiency.

[0024] In one of the embodiments, the data transmission system includes a target host, and the target host is any one of a plurality of hosts included in the data transmission system;

[0025] The target host is used to obtain node information of the target host, where the node information includes identification information of each node included in the target host and distance information between each of the nodes;

[0026] The target host is further used to synchronize the node information of the target host to other hosts included in the data transmission system except the target host.

[0027] Based on the above scheme, the nearest node equipped with memory can be accurately determined to ensure data transmission efficiency.

[0028] In a second aspect, the present application provides a data transmission method, applied to a first node, the method comprising:

[0029] When receiving a data transmission request, determining data transmission information carried by the data transmission request, the data transmission information including target data and a destination storage address;

[0030] Among the multiple nodes included in the first host, a second node that meets the first optimized transmission condition is determined, and the data transmission information is sent to the second node, the data transmission information is used to instruct the second node to determine a third node that meets the second optimized transmission condition among the multiple nodes included in the second host based on the data transmission information, and send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node, so that the third node sends the target data from the third node to the destination storage address based on the data transmission information, and the second host is the host where the node corresponding to the destination storage address is located.

[0031] In one of the embodiments, determining a second node satisfying a first optimized transmission condition among a plurality of nodes included in the first host comprises:

[0032] Obtaining first distance information between nodes in the first host corresponding to the first node;

[0033] Based on the first distance information, a second node that is closest to the first node and is deployed with a communication network card is determined.

[0034] In a third aspect, the present application provides a data transmission method, applied to a second node, the method comprising:

[0035] receiving data transmission information, the data transmission information including target data and a destination storage address, the data transmission information being sent by the first node to the second node after the first node determines the data transmission information carried by the data transmission request and determines a second node that meets a first optimization transmission condition among multiple nodes included in the first host, the first host being the host where the first node is located, and a first communication network card being deployed in the second node;

[0036] Among the multiple nodes included in the second host, a third node that meets the second optimized transmission condition is determined, and based on the communication connection between the first communication network card and the second communication network card, the data transmission information is sent to the third node, and the data transmission information is used to instruct the three nodes to send the target data from the third node to the destination storage address. The second host is the host where the node corresponding to the destination storage address is located, and the second communication network card is deployed in the third node.

[0037] In one of the embodiments, determining a third node satisfying the second optimized transmission condition among the multiple nodes included in the second host includes:

[0038] Obtaining second distance information between nodes in the second host;

[0039] Based on the second distance information, a third node which is closest to the node corresponding to the destination storage address and is deployed with a communication network card is determined among the multiple nodes included in the second host.

[0040] In a fourth aspect, the present application provides a data transmission method, applied to a third node, the method comprising:

[0041] Based on data transmission information, target data is sent from a third node to a destination storage address, the data transmission information including target data and destination storage address, the data transmission information is sent by the second node to the third node based on the communication connection between the first communication network card and the second communication network card after the second node determines the third node that meets the second optimized transmission condition among multiple nodes included in the second host based on the data transmission information, the second node is the node that meets the first optimized transmission condition among multiple nodes included in the first host after the first node determines the data transmission information carried by the data transmission request when receiving the data transmission request, the first host is the host where the first node is located, the second host is the host where the node corresponding to the destination storage address is located, the first communication network card is deployed in the second node, and the second communication network card is deployed in the third node.

[0042] In one embodiment, sending the target data from the third node to the destination storage address based on the data transmission information includes:

[0043] Obtain second distance information between each node in the second host, and based on the second distance information, determine, among the multiple nodes included in the second host, a fourth node that is closest to the third node and is configured with a first storage space, and send data transmission information containing target data and a destination storage address to the first storage space of the fourth node, wherein the data transmission information is used to instruct the fourth node to send the target data in the data transmission information to the destination storage address.

[0044] In a fifth aspect, the present application provides a data transmission device, applied to a first node, the device comprising:

[0045] A first receiving module, configured to determine, upon receiving a data transmission request, data transmission information carried by the data transmission request, wherein the data transmission information includes target data and a destination storage address;

[0046] The first determination module is used to determine a second node that meets the first optimized transmission condition among multiple nodes included in the first host, and send the data transmission information to the second node, the data transmission information is used to instruct the second node to determine a third node that meets the second optimized transmission condition among multiple nodes included in the second host based on the data transmission information, and send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node, so that the third node sends the target data from the third node to the destination storage address based on the data transmission information, and the second host is the host where the node corresponding to the destination storage address is located.

[0047] In a sixth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0048] When receiving a data transmission request, determining data transmission information carried by the data transmission request, the data transmission information including target data and a destination storage address;

[0049] Among the multiple nodes included in the first host, a second node that meets the first optimized transmission condition is determined, and the data transmission information is sent to the second node, the data transmission information is used to instruct the second node to determine a third node that meets the second optimized transmission condition among the multiple nodes included in the second host based on the data transmission information, and send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node, so that the third node sends the target data from the third node to the destination storage address based on the data transmission information, and the second host is the host where the node corresponding to the destination storage address is located.

[0050] In a seventh aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0051] When receiving a data transmission request, determining data transmission information carried by the data transmission request, the data transmission information including target data and a destination storage address;

[0052] Among the multiple nodes included in the first host, a second node that meets the first optimized transmission condition is determined, and the data transmission information is sent to the second node, the data transmission information is used to instruct the second node to determine a third node that meets the second optimized transmission condition among the multiple nodes included in the second host based on the data transmission information, and send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node, so that the third node sends the target data from the third node to the destination storage address based on the data transmission information, and the second host is the host where the node corresponding to the destination storage address is located.

[0053] In an eighth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0054] When receiving a data transmission request, determining data transmission information carried by the data transmission request, the data transmission information including target data and a destination storage address;

[0055] Among the multiple nodes included in the first host, a second node that meets the first optimized transmission condition is determined, and the data transmission information is sent to the second node, the data transmission information is used to instruct the second node to determine a third node that meets the second optimized transmission condition among the multiple nodes included in the second host based on the data transmission information, and send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node, so that the third node sends the target data from the third node to the destination storage address based on the data transmission information, and the second host is the host where the node corresponding to the destination storage address is located.

[0056] The above-mentioned data transmission system, method, device, computer equipment, storage medium and computer program product, wherein the data transmission system includes multiple hosts, and the host includes multiple nodes: wherein the first node is used to determine the data transmission information carried by the data transmission request when receiving the data transmission request, and determine the second node that meets the first optimized transmission condition among the multiple nodes included in the first host, and send the data transmission information to the second node; the second node is used to determine the third node that meets the second optimized transmission condition among the multiple nodes included in the second host based on the data transmission information, and send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node; the third node is used to send the target data from the third node to the destination storage address based on the data transmission information. By adopting this application, the optimal transmission path for data transmission can be determined, and in the case of data transmission across nodes and across hosts, it is ensured that the optimal transmission path can be used for data transmission, thereby improving the efficiency of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a schematic diagram of a data transmission system in one embodiment;

[0058] Figure 2 is a schematic diagram of a data transmission system in another embodiment;

[0059] Figure 3 A schematic diagram of a flow chart of a data transmission method in an embodiment;

[0060] Figure 4 A schematic diagram of a flow chart of a data transmission method in an embodiment;

[0061] Figure 5 A schematic diagram of a flow chart of a data transmission method in an embodiment;

[0062] Figure 6 A schematic diagram of a flow chart of a data transmission method in an embodiment;

[0063] Figure 7 is a structural block diagram of a data transmission device in one embodiment;

[0064] Figure 8 is a structural block diagram of a data transmission device in one embodiment;

[0065] Fig. 9 is a structural block diagram of a data transmission device in one embodiment;

[0066] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0068] With the development of technology, a NUMA architecture for distributed storage has emerged. This NUMA architecture solves the bottleneck problem of multi-core processors accessing local memory. However, in the distributed storage system (NUMA architecture), multiple devices are included, and the devices include multiple NUMA NODEs (nodes). The storage data of the local device will reach the peer device through the network and be stored in the disk of the peer device. In the above data transmission process, there will be IO delays caused by cross-node memory access, and the additional bus overhead caused by memory migration will also seriously affect the performance of the distributed storage system. NUMA is a memory structure for multiple processors.

[0069] In other words, the data path performance of the distributed storage system depends on the network card performance and disk performance on the IO path. By minimizing the distance across NUMA NODEs when accessing memory during IO under the NUMA architecture, the performance of the distributed storage system can be improved.

[0070] The embodiment of the present application provides a data transmission system 100. The data transmission system 100 can be as follows: Figure 1 As shown, it includes multiple hosts 200 (host), each host 200 includes multiple nodes (node), each node can be deployed with one or more of the first storage space, the second storage space and the communication network card (NIC), and the data transmission system 100 can also include a switch 300 (switch). Among them, each host 200 can communicate data with other hosts 200 in the data transmission system 100 through the local communication network card and the switch. The host can be a server device in the topology of the distributed storage system, and can be used as a client or a server.

[0071] In one example, if Figure 2 As shown, the data transmission system may include a first host (host 1, host1) and a second host (host 2, host2), and the first host may be connected to the second host via a switch. The first host may include multiple nodes, such as node 1 (node1), node 2 (node2), node 3 (node3) and node 4 (node4), and the second host may also include multiple nodes, such as node 1 (node1), node 2 (node2), node 3 (node3) and node 4 (node4). Among them, communication network card 1 (nic1) can be deployed in node 2 of host1, and communication network card 2 (nic2) can be deployed in node 3 of host1; hard disk 1 (disk 1) can be deployed in node 1 of host2, and hard disk 2 (disk2) can be deployed in node 4 of host2.

[0072] Based on this, there may be a data transmission request that needs to write a piece of data located in the memory of node1 to disk2 of host2 on host1. The data transmission system provided in the present application can determine the optimal data transmission path to implement the above data transmission request and perform data transmission based on the optimal data transmission path, thereby improving the efficiency of cross-node transmission of data on different hosts and improving the IO performance of the distributed storage system.

[0073] like Figure 1 As shown, an embodiment of the present application provides a data transmission system, which includes multiple hosts, each host includes multiple nodes, and the data transmission system also includes a switch.

[0074] Specifically, each node in the host can be configured with one or more of a first storage space, a second storage space and a communication network card. The first storage space can be a memory, the second storage space can be a hard disk, and the communication network card can be a network card used for communication between different hosts.

[0075] The first node is used to determine the data transmission information carried by the data transmission request when receiving the data transmission request, and determine the second node that meets the first optimized transmission condition among the multiple nodes included in the first host, and send the data transmission information to the second node.

[0076] The data transmission information includes target data and a destination storage address, and the first host is a host where the first node is located. The first node may be one of the multiple nodes included in the first host, for example, a node storing the target data. The data transmission request is used to instruct the first node to transfer the target data from the first node to the destination storage address. The destination storage address may be a hard disk address on another host.

[0077] In implementation, after receiving a data transmission request for the first node, the first node will parse the data transmission request to obtain the data transmission information carried by the data transmission request, that is, determine the target data to be transmitted in the data transmission request, and the transmission address (destination storage address) of the target transmission. The first node can determine a second node that meets the preset configuration of the optimized transmission conditions from among the multiple nodes included in the first host. In this way, the first node can send the data transmission information including the target data and the destination storage address to the second node.

[0078] Optionally, the first optimized transmission condition may be used for node screening, and the specific content of the first optimized transmission condition may be a node that is closest to the node where the target data is located and is deployed with a communication network card.

[0079] The second node is used to determine, based on the data transmission information, a third node that meets the second optimized transmission condition among the multiple nodes included in the second host, and send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node.

[0080] The second host is the host where the node corresponding to the destination storage address is located; the local communication network card of the second node may be a communication network card deployed on the second node. The second optimized transmission condition may be used for node screening, and the specific content of the second optimized transmission condition may be the node closest to the node where the destination storage address is located and where the communication network card is deployed.

[0081] In implementation, after receiving the data transmission information, the second node can determine the multiple nodes included in the second host, and screen the multiple nodes included in the second host to determine the third node that meets the second optimized transmission condition. In this way, the second node can send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node.

[0082] The third node is used to send the target data from the third node to the destination storage address based on the data transmission information.

[0083] In implementation, the third node can parse the data transmission message after the second node sends the data transmission information, obtain the target data and the destination storage address, and transfer the target data to the second storage space corresponding to the target storage address, i.e., the hard disk, based on the target storage address.

[0084] The first node in the above-mentioned data transmission system is used to determine the data transmission information carried by the data transmission request when receiving the data transmission request, and determine the second node that meets the first optimized transmission condition among the multiple nodes included in the first host, and send the data transmission information to the second node; the second node is used to determine the third node that meets the second optimized transmission condition among the multiple nodes included in the second host based on the data transmission information, and send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node; the third node is used to send the target data from the third node to the destination storage address based on the data transmission information. By adopting this application, the optimal transmission path for data transmission can be determined, and in the case of cross-node and cross-host data transmission, it is ensured that the optimal transmission path can be used for data transmission, thereby improving the efficiency of data transmission and reducing the resource overhead of the data transmission system for data transmission.

[0085] In an exemplary embodiment, the first node is specifically configured to:

[0086] Obtain first distance information between nodes in a first host corresponding to the first node; and determine a second node that is closest to the first node and is deployed with a communication network card based on the first distance information.

[0087] The first distance information may be a plurality of distance information between the first node and each node included in the first host, for example, the distance value between the first node and each other node, and so on.

[0088] During implementation, the first node can obtain distance information representing the first node and other nodes included in the first host. The first node can obtain a node sequence arranged in ascending order of distance based on the distance information between other nodes and the first node. In this way, the first node can determine whether each node is deployed with a communication network card in turn according to the arrangement order of the node sequence, and determine the node with the shortest distance and deployed with a communication network card as the second node.

[0089] In one example, the first node can also screen other nodes contained in the first host, extract multiple nodes deployed with communication network cards, and obtain distance information between the multiple nodes deployed with communication network cards and the first node, and determine the node with the shortest distance and deployed with a communication network card as the second node.

[0090] Optionally, after receiving a data transmission request for the first node, the first node may parse the data transmission request to obtain target data and a destination storage address, so that the first node may determine whether a communication network card exists locally. If a communication network card exists locally, the first node may execute the execution process of the second node in the above system embodiment, that is, the first node and the second node may be the same node.

[0091] Based on the above scheme, the nearest communication network card can be accurately determined to ensure data transmission efficiency.

[0092] In an exemplary embodiment, the second node is specifically configured to:

[0093] Obtain second distance information between nodes in the second host; based on the second distance information, determine a third node that is closest to the node corresponding to the destination storage address and is deployed with a communication network card among the multiple nodes included in the second host.

[0094] The second distance information may be a plurality of distance information between the second node and each node included in the second host.

[0095] In implementation, the second node can obtain the distance information between the multiple nodes included in the second host, and based on the distance information between the multiple nodes included in the second host and the node corresponding to the destination storage address, obtain a node sequence arranged in ascending order of distance. In this way, the second node can determine whether each node is deployed with a communication network card in turn according to the arrangement order of the node sequence, and determine the node with the smallest distance and deployed with a communication network card as the third node.

[0096] In an example, the second node can also screen the multiple nodes contained in the second host, extract multiple nodes deployed with communication network cards, and obtain the distance information between the multiple nodes deployed with communication network cards and the second node, and determine the node with the shortest distance and deployed with a communication network card as the third node.

[0097] Based on the above scheme, the nearest communication network card can be accurately determined to ensure data transmission efficiency.

[0098] In an exemplary embodiment, the third node is specifically used for:

[0099] Based on the second distance information, a fourth node closest to the third node and configured with a first storage space is determined among the multiple nodes included in the second host, and data transmission information including target data and a destination storage address is sent to the first storage space of the fourth node.

[0100] During implementation, the third node can obtain the distance information between the third node and multiple nodes included in the second host, and based on the distance information between the multiple nodes included in the second host and the third node, obtain a node sequence arranged according to the distance from small to large. In this way, the third node can determine whether each node is deployed with the first storage space in turn according to the arrangement order of the node sequence, and determine the node with the smallest distance and deployed with the first storage space as the fourth node.

[0101] In one example, the second node can also filter among the multiple nodes included in the second host, extract multiple nodes deployed with the first storage space, and obtain the distance information between the multiple nodes deployed with the first storage space and the third node, and determine the node with the shortest distance and deployed with the first storage space as the fourth node.

[0102] Optionally, after receiving the data transmission request for the third node, the third node may parse the data transmission request to obtain the target data and the destination storage address, so that the third node can determine whether the first storage space exists locally. If the first storage space exists locally, the third node may transmit the target data to the local first storage space, and send the target data stored in the local first storage space to the destination storage address, that is, the target data may be stored in the storage space of the hard disk corresponding to the destination storage address.

[0103] Based on the above scheme, the nearest node equipped with memory can be accurately determined to ensure data transmission efficiency.

[0104] In an exemplary embodiment, the fourth node is specifically configured to: receive data transmission information, and send target data in the data transmission information to a destination storage address.

[0105] In implementation, after receiving the data transmission information, the fourth node can store the target data in the local first storage space. In this way, the fourth node can send the target data stored in the local first storage space to the destination storage address, that is, store the target data in the storage space in the hard disk corresponding to the destination storage address.

[0106] Based on the above scheme, the nearest node equipped with memory can be accurately determined to ensure data transmission efficiency.

[0107] In an exemplary embodiment, the data transmission system includes a target host, and the target host is any one of a plurality of hosts included in the data transmission system.

[0108] The target host is used to obtain the node information of the target host, where the node information includes the identification information of each node included in the target host and the distance information between each node.

[0109] In implementation, the target host may obtain node information of multiple nodes deployed in the target host, that is, the target host needs to obtain identification information of multiple nodes deployed locally and distance information between the nodes.

[0110] The target host is also used to synchronize the node information of the target host to other hosts except the target host included in the data transmission system.

[0111] In implementation, after the target data obtains the local node information, the local node information can be synchronized to other hosts included in the data transmission system to which the target host belongs.

[0112] In one example, the data transmission system may include a first host, a second host, and a third host, and the target host may be any one of the above hosts, such as the first host. In this way, the first host will synchronize the node information corresponding to the first host to the second host and the third host.

[0113] Specifically, the target host can determine a first target node with a communication network card deployed locally, and a second target node with a communication network card deployed in other hosts, and establish a communication connection between the local communication network card and the communication network cards in the other hosts, that is, the target host can establish a communication connection between the communication network card on the first target node and the communication network card in the second target node in the other hosts, respectively, to facilitate data communication between different hosts in the data transmission system.

[0114] Based on the above solution, data transmission in the data transmission system can be realized and the flexibility of data transmission in the data transmission system can be increased.

[0115] Based on the same inventive concept, the embodiment of the present application also provides a data transmission method for implementing the data transmission system involved above. The implementation scheme for solving the problem provided by the data transmission method is similar to the implementation scheme recorded in the above system, so the specific limitations in one or more data transmission method embodiments provided below can refer to the limitations on the data transmission system above, and will not be repeated here.

[0116] In one embodiment, Figure 3 As shown, a data transmission method is provided, which is applied to a first node, and the data transmission method includes:

[0117] Step 302, when a data transmission request is received, determining data transmission information carried by the data transmission request, where the data transmission information includes target data and a destination storage address;

[0118] Step 304: Determine a second node that meets the first optimized transmission condition among the multiple nodes included in the first host, and send data transmission information to the second node.

[0119] Among them, the data transmission information is used to instruct the second node to determine, based on the data transmission information, a third node that meets the second optimized transmission condition among the multiple nodes included in the second host, and send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node, so that the third node sends the target data from the third node to the destination storage address based on the data transmission information. The second host is the host where the node corresponding to the destination storage address is located.

[0120] In one embodiment, if Figure 4 As shown, the specific processing process of step 304 "determining a second node that meets the optimized transmission condition among the multiple nodes included in the first host" includes:

[0121] Step 402: Obtain first distance information between nodes in a first host corresponding to the first node.

[0122] Step 404: Based on the first distance information, determine a second node that is closest to the first node and is deployed with a communication network card.

[0123] In one embodiment, Figure 5 As shown, a data transmission method is provided, which is applied to a second node, and the data transmission method includes:

[0124] Step 502: Receive data transmission information.

[0125] The data transmission information includes target data and a destination storage address. The data transmission information is sent to the second node after the first node determines the data transmission information carried by the data transmission request when receiving the data transmission request and determines the second node that meets the optimized transmission condition among multiple nodes included in the first host. The first host is the host where the first node is located, and the first communication network card is deployed in the second node.

[0126] Step 504: determine a third node that meets the second optimized transmission condition among the multiple nodes included in the second host, and send data transmission information to the third node based on the communication connection between the first communication network card and the second communication network card.

[0127] Among them, the data transmission information is used to instruct the three nodes to send the target data from the third node to the destination storage address, the second host is the host where the node corresponding to the destination storage address is located, and the second communication network card is deployed in the third node.

[0128] In one embodiment, if Figure 6As shown, the specific processing process of step 504 "determining a third node that meets the optimized transmission condition among the multiple nodes included in the second host" includes:

[0129] Step 602, obtaining second distance information between nodes in the second host;

[0130] Step 604: Based on the second distance information, determine a third node that is closest to the second node and is deployed with a communication network card among the multiple nodes included in the second host.

[0131] In one embodiment, a data transmission method is provided, which is applied to a third node. The data transmission method includes:

[0132] Based on the data transmission information, the target data is sent from the third node to the destination storage address. The data transmission information includes the target data and the destination storage address. The data transmission information is sent to the third node based on the communication connection between the first communication network card and the second communication network card after the second node determines the third node that meets the optimized transmission conditions among the multiple nodes included in the second host based on the data transmission information. The second node is the node that meets the optimized transmission conditions among the multiple nodes included in the first host after the first node determines the data transmission information carried by the data transmission request when receiving the data transmission request. The first host is the host where the first node is located, and the second host is the host where the node corresponding to the destination storage address is located. The first communication network card is deployed in the second node, and the second communication network card is deployed in the third node.

[0133] In one embodiment, sending the target data from the third node to the destination storage address based on the data transmission information includes:

[0134] Obtain second distance information between each node in the second host, and based on the second distance information, determine, among the multiple nodes included in the second host, a fourth node that is closest to the third node and is configured with a first storage space, and send data transmission information containing target data and a destination storage address to the first storage space of the fourth node, wherein the data transmission information is used to instruct the fourth node to send the target data in the data transmission information to the destination storage address.

[0135] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0136] Based on the same inventive concept, the embodiment of the present application also provides a data transmission device for implementing the data transmission method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data transmission device embodiments provided below can refer to the limitations on the data transmission method above, and will not be repeated here.

[0137] In one embodiment, Figure 7 As shown, a data transmission device 700 is provided, which is applied to a first node and includes: a first receiving module 701 and a first determining module 702, wherein:

[0138] The first receiving module 701 is used to determine the data transmission information carried by the data transmission request when receiving the data transmission request, where the data transmission information includes the target data and the destination storage address;

[0139] The first determination module 702 is used to determine a second node that meets the optimized transmission conditions among the multiple nodes included in the first host, and send data transmission information to the second node, the data transmission information is used to instruct the second node to determine a third node that meets the optimized transmission conditions among the multiple nodes included in the second host based on the data transmission information, and send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node, so that the third node sends the target data from the third node to the destination storage address based on the data transmission information. The second host is the host where the node corresponding to the destination storage address is located.

[0140] In one embodiment, the first determining module 702 is specifically configured to:

[0141] Obtain first distance information between nodes in a first host corresponding to the first node.

[0142] Based on the first distance information, a second node which is closest to the first node and is deployed with a communication network card is determined.

[0143] In one embodiment, Figure 8 As shown, a data transmission device 800 is provided, which is applied to a second node, and the data transmission device includes:

[0144] The second receiving module 801 is used to receive data transmission information, where the data transmission information includes target data and a destination storage address. The data transmission information is sent to the second node after the first node determines the data transmission information carried by the data transmission request after receiving the data transmission request and determines the second node that meets the optimized transmission condition among the multiple nodes included in the first host. The first host is the host where the first node is located, and the second node is deployed with a first communication network card.

[0145] The second determination module 802 is used to determine a third node that meets the optimized transmission conditions among the multiple nodes included in the second host, and send data transmission information to the third node based on the communication connection between the first communication network card and the second communication network card. The data transmission information is used to instruct the three nodes to send target data from the third node to the destination storage address. The second host is the host where the node corresponding to the destination storage address is located, and the second communication network card is deployed in the third node.

[0146] In one embodiment, the second determining module 802 is specifically configured to:

[0147] Obtaining second distance information between nodes in the second host;

[0148] Based on the second distance information, a third node which is closest to the second node and is deployed with a communication network card is determined among the multiple nodes included in the second host.

[0149] In one embodiment, Fig. 9 As shown, a data transmission device 900 is provided, which is applied to a third node. The data transmission device 900 includes:

[0150] The second sending module 901 is used to send the target data from the third node to the destination storage address based on the data transmission information. The data transmission information includes the target data and the destination storage address. The data transmission information is sent to the third node based on the communication connection between the first communication network card and the second communication network card after the second node determines the third node that meets the optimized transmission conditions among the multiple nodes included in the second host based on the data transmission information. The second node is the node that meets the optimized transmission conditions among the multiple nodes included in the first host after the first node determines the data transmission information carried by the data transmission request when receiving the data transmission request. The first host is the host where the first node is located, and the second host is the host where the node corresponding to the destination storage address is located. The first communication network card is deployed in the second node, and the second communication network card is deployed in the third node.

[0151] In one embodiment, the second sending module is specifically configured to:

[0152] Obtain second distance information between each node in the second host, and based on the second distance information, determine, among the multiple nodes included in the second host, a fourth node that is closest to the third node and is configured with a first storage space, and send data transmission information containing target data and a destination storage address to the first storage space of the fourth node, wherein the data transmission information is used to instruct the fourth node to send the target data in the data transmission information to the destination storage address.

[0153] Each module in the above-mentioned data transmission device 700, data transmission device 800, and data transmission device 900 can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0154] In one embodiment, a computer device is provided. The computer device may be a host, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store target data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data transmission method is implemented.

[0155] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0156] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0157] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0158] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0159] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0160] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0161] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0162] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data transmission system, characterized in that: The data transmission system includes multiple hosts, each of which includes multiple nodes: The first node is used to determine the data transmission information carried by the data transmission request when receiving the data transmission request, and determine a second node that meets the first optimized transmission condition among the multiple nodes included in the first host, and send the data transmission information to the second node, wherein the data transmission information includes target data and a destination storage address, the first host is the host where the first node is located, and the first optimized transmission condition is a node that is closest to the node where the target data is located and is deployed with a communication network card; The second node is used to determine, based on the data transmission information, a third node that meets the second optimized transmission condition among the multiple nodes included in the second host, and send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node, the second host is the host where the node corresponding to the destination storage address is located, and the second optimized transmission condition is the node that is closest to the node where the destination storage address is located and is deployed with a communication network card; The third node is used to send the target data from the third node to the destination storage address based on the data transmission information.

2. The system according to claim 1, characterized in that The first node is specifically used for: Obtaining first distance information between nodes in the first host corresponding to the first node; Based on the first distance information, a second node that is closest to the first node and is deployed with a communication network card is determined.

3. The system according to claim 1, characterized in that The second node is specifically used for: Obtaining second distance information between nodes in the second host; Based on the second distance information, a third node which is closest to the node corresponding to the destination storage address and is deployed with a communication network card is determined among the multiple nodes included in the second host.

4. The system according to claim 3, characterized in that The third node is specifically used for: Based on the second distance information, a fourth node which is closest to the third node and is configured with a first storage space is determined among the multiple nodes included in the second host, and data transmission information including target data and a destination storage address is sent to the first storage space of the fourth node.

5. The system according to claim 4, characterized in that The fourth node is specifically used for: The data transmission information is received, and the target data in the data transmission information is sent to the destination storage address.

6. The system according to any one of claims 1 to 5, characterized in that: The data transmission system includes a target host, and the target host is any one of the multiple hosts included in the data transmission system; The target host is used to obtain node information of the target host, where the node information includes identification information of each node included in the target host and distance information between each of the nodes; The target host is further used to synchronize the node information of the target host to other hosts included in the data transmission system except the target host.

7. A data transmission method, characterized in that: Applied to the first node, the method comprises: When receiving a data transmission request, determining data transmission information carried by the data transmission request, the data transmission information including target data and a destination storage address; Among the multiple nodes included in the first host, a second node that meets the first optimized transmission condition is determined, and the data transmission information is sent to the second node, the data transmission information is used to instruct the second node to determine a third node that meets the second optimized transmission condition among the multiple nodes included in the second host based on the data transmission information, and based on the communication connection between the local communication network card of the second node and the communication network card of the third node, send the data transmission information to the third node, so that the third node sends the target data from the third node to the destination storage address based on the data transmission information, the second host is the host where the node corresponding to the destination storage address is located, the first optimized transmission condition is the node that is closest to the node where the target data is located and is deployed with a communication network card, and the second optimized transmission condition is the node that is closest to the node where the destination storage address is located and is deployed with a communication network card.

8. The method according to claim 7, characterized in that The step of determining, among the plurality of nodes included in the first host, a second node that meets the optimized transmission condition comprises: Obtaining first distance information between nodes in the first host corresponding to the first node; Based on the first distance information, a second node that is closest to the first node and is deployed with a communication network card is determined.

9. A data transmission method, characterized in that: Applied to the second node, the method comprises: Receive data transmission information, the data transmission information including target data and a destination storage address, the data transmission information is sent to the second node after the first node determines the data transmission information carried by the data transmission request when receiving the data transmission request, and determines a second node that meets a first optimized transmission condition among multiple nodes included in the first host, the first host is the host where the first node is located, a first communication network card is deployed in the second node, and the first optimized transmission condition is a node that is closest to the node where the target data is located and has a communication network card deployed therein; Among the multiple nodes included in the second host, a third node that meets the second optimized transmission condition is determined, and based on the communication connection between the first communication network card and the second communication network card, the data transmission information is sent to the third node, and the data transmission information is used to instruct the three nodes to send the target data from the third node to the destination storage address. The second host is the host where the node corresponding to the destination storage address is located, and the second communication network card is deployed in the third node. The second optimized transmission condition is the node that is closest to the node where the destination storage address is located and is deployed with a communication network card.

10. The method according to claim 9, characterized in that The step of determining a third node satisfying the second optimized transmission condition from among the multiple nodes included in the second host comprises: Obtaining second distance information between nodes in the second host; Based on the second distance information, a third node which is closest to the node corresponding to the destination storage address and is deployed with a communication network card is determined among the multiple nodes included in the second host.

11. A data transmission method, characterized in that: Applied to the third node, the method includes: Based on data transmission information, target data is sent from a third node to a destination storage address, the data transmission information including the target data and the destination storage address, the data transmission information is sent by the second node to the third node based on the communication connection between the first communication network card and the second communication network card after the second node determines the third node that meets the second optimization transmission condition among multiple nodes included in the second host based on the data transmission information, the second node is the node that meets the first optimization transmission condition determined among multiple nodes included in the first host after the first node determines the data transmission information carried by the data transmission request when receiving the data transmission request, the first host is the host where the first node is located, the second host is the host where the node corresponding to the destination storage address is located, the first communication network card is deployed in the second node, the second communication network card is deployed in the third node, the first optimization transmission condition is the node that is closest to the node where the target data is located and is deployed with a communication network card, and the second optimization transmission condition is the node that is closest to the node where the destination storage address is located and is deployed with a communication network card.

12. The method according to claim 11, characterized in that The sending the target data from the third node to the destination storage address based on the data transmission information includes: Obtain second distance information between each node in the second host, and based on the second distance information, determine, among the multiple nodes included in the second host, a fourth node that is closest to the third node and is configured with a first storage space, and send data transmission information containing target data and a destination storage address to the first storage space of the fourth node, wherein the data transmission information is used to instruct the fourth node to send the target data in the data transmission information to the destination storage address.

13. A data transmission device, characterized in that: Applied to a first node, the device comprises: A first receiving module, configured to determine, upon receiving a data transmission request, data transmission information carried by the data transmission request, wherein the data transmission information includes target data and a destination storage address; The first determination module is used to determine a second node that meets the first optimized transmission condition among multiple nodes included in the first host, and send the data transmission information to the second node, the data transmission information is used to instruct the second node to determine a third node that meets the second optimized transmission condition among multiple nodes included in the second host based on the data transmission information, and send the data transmission information to the third node based on the communication connection between the local communication network card and the communication network card of the third node, so that the third node sends the target data from the third node to the destination storage address based on the data transmission information, the second host is the host where the node corresponding to the destination storage address is located, the first optimized transmission condition is the node that is closest to the node where the target data is located and is deployed with a communication network card, and the second optimized transmission condition is the node that is closest to the node where the destination storage address is located and is deployed with a communication network card.

14. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 7 to 12 are implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 7 to 12 are implemented.

Images