Data transmission method, system, device and storage medium between distributed nodes

Abstract

The present invention discloses a method, system, device, and storage medium for data transmission between distributed nodes. It relates to the field of cloud computing technology. The method includes: responding to a data transmission request and establishing a first path hierarchy diagram; obtaining the number of free connections for each pair of distributed nodes, and based on the first path hierarchy diagram and the number of free connections for each pair of distributed nodes, using an iterative method to mark the number of occupied forward connections and the number of free reverse connections; determining the connection path data between the source node and the destination node based on the number of occupied forward connections and the number of free reverse connections; and transmitting the target data from the source node to the destination node based on the connection path data. The present invention solves the technical problem in the related art of using a transit node for transit or establishing an SSL channel between any two nodes to achieve file transfer between distributed nodes of a microservice system, resulting in poor transmission effect.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing technology, and in particular to a method, system, device, and storage medium for data transmission between distributed nodes. Background Art

[0002] During file transfer in a microservice system, due to the distributed node characteristics of the microservice system, it is impossible to establish too many SSL channels between nodes. If a connection path is established between each pair of distributed nodes, n*n / 2 combinations will be generated. In the context where the source node A and the destination node B of the file transfer are sparse and random in design requirements, unnecessary overhead will be caused.

[0003] If the file transfer process is centralized and sent to a central node or transit node (e.g., a file server), then each node in need retrieves the file from the central node. This transmission method relies heavily on the central node's bandwidth, using a transfer node AC-...-B (C... is the transit node). Because each node has a limited flow policy (generally, limiting the number of connections that can be made from one other node to the current node through a maximum number of connections per IP address), the file transfer rate will be severely affected by the transmission needs of A and B. This means that the file transfer is overly dependent on the central node (file server), and connection limitations and failures will lead to processing delays at the destination node.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] The embodiments of the present invention provide a method, system, device, and storage medium for data transmission between distributed nodes, so as to at least solve the technical problem in the related art of using a transit node for transit or establishing an SSL channel between any two nodes to achieve file transmission between distributed nodes of a microservice system, resulting in poor transmission effect.

[0006] According to one aspect of an embodiment of the present invention, a method for data transmission between distributed nodes is provided, comprising: establishing a first path hierarchy diagram in response to a data transmission request, wherein the data transmission request is used to request the transmission of target data from a source node to a destination node, the first path hierarchy diagram comprising: a connection path between the source node and the destination node, each of the connection paths consisting of M distributed nodes, each of the distributed nodes in the first path hierarchy diagram being marked with a level number, where M is a positive integer; obtaining the number of free connections for each pair of the distributed nodes, and based on the first path hierarchy diagram and the number of free connections for each pair of the distributed nodes, iteratively marking the number of occupied forward connections and the number of reverse free connections number, wherein each pair of the distributed nodes is composed of two of the M distributed nodes having a connection relationship, the forward connection occupancy number is the connection occupancy number of each pair of the distributed nodes in the direction from the source node to the destination node, and the reverse free connection number is the free connection number of each pair of the distributed nodes in the direction from the destination node to the source node; based on the forward connection occupancy number and the reverse free connection number, the connection path data between the source node and the destination node is determined, wherein the connection path data includes: the number of all connection paths established from the source node to the destination node; based on the connection path data, the target data is transmitted from the source node to the destination node.

[0007] Furthermore, a first path hierarchy graph is established, including: using a breadth-first search method, starting from the source node, searching for adjacent nodes layer by layer, and when the adjacent node is found, determining whether the adjacent node is the destination node and whether the adjacent node has been marked with a level; when the adjacent node is not the destination node and the adjacent node has not been marked with a level, marking the level of the adjacent node until the level of the destination node is marked, thereby obtaining the first path hierarchy graph.

[0008] Furthermore, the number of free connections of each pair of distributed nodes is obtained, and based on the first path hierarchy graph and the number of free connections of each pair of distributed nodes, the forward connection occupancy number and the reverse free connection number are marked in an iterative manner, including: step 1, based on the number of free connections of each pair of distributed nodes in the first path hierarchy graph, a breadth-first search method is used to find the connection path between the source node and the destination node in the first path hierarchy graph, and mark the forward connection occupancy number and the reverse free connection number between the nodes in the found connection path; step 2, based on the marked forward connection occupancy number and reverse free connection number between the nodes, a second path hierarchy graph is established, and the forward connection occupancy number and the reverse free connection number of each node in the second path hierarchy graph are marked; step 3, the second path hierarchy graph is used as the first path hierarchy graph; and steps 1 to 3 are repeated until the connection path from the source node to the destination node cannot be found in the first path hierarchy graph.

[0009] Furthermore, after marking the number of occupied forward connections and the number of free reverse connections between nodes in the found connection path, the method further includes: adjusting the number of occupied forward connections according to the number of free reverse connections.

[0010] Furthermore, the number of free connections of each pair of distributed nodes is obtained, including: obtaining the maximum number of connections of the first node to the second node, wherein the first node and the second node are two distributed nodes with a connection relationship among the M distributed nodes, and the first node and the second node form a pair of distributed nodes; obtaining the number of connections between the first node and the second node; calculating the difference between the maximum number of connections and the number of connections to obtain the number of free connections of the first node and the second node.

[0011] Furthermore, based on the connection path data, transmitting the target data from the source node to the destination node includes: establishing a secure socket layer channel from the source node to the destination node based on the connection path data; and transmitting the target data from the source node to the destination node based on the secure socket layer channel.

[0012] Furthermore, each of the distributed nodes is a distributed node of a microservice system, and the target data is file data.

[0013] According to another aspect of an embodiment of the present invention, a data transmission system between distributed nodes is also provided, and the data transmission system is used for any of the data transmission methods described above, including: a distributed node; a scheduling policy processing module, wherein the scheduling policy processing module is used to respond to a data transmission request and establish a first path hierarchy diagram; based on the first path hierarchy diagram and the number of free connections of each pair of distributed nodes, an iterative method is used to mark the number of occupied forward connections and the number of reverse free connections; based on the number of occupied forward connections and the number of reverse free connections, the connection path data between the source node and the destination node is determined, wherein the data transmission request is used to request to transmit the target data from the source node to the destination node, and the first path hierarchy diagram includes: the connection path between the source node and the destination node, each of the connection paths It is composed of M distributed nodes, each distributed node in the first path hierarchical diagram is marked with a level number, each pair of distributed nodes is composed of two distributed nodes with a connection relationship among the M distributed nodes, the forward connection occupancy number is the connection occupancy number of each pair of distributed nodes in the direction from the source node to the destination node, and the reverse free connection number is the free connection number of each pair of distributed nodes in the direction from the destination node to the source node. The connection path data includes: the number of all connections established from the source node to the destination node; based on the connection path data, the target data is transmitted from the source node to the destination node; a control panel, wherein the control panel is used to call the free connection number calculation service in each distributed node to obtain the free connection number of each pair of distributed nodes.

[0014] Furthermore, the distributed node further includes: a transmission service module, wherein the transmission service module is used to provide data transmission services; and a calculation service module, wherein the calculation service module is used to provide the free connection number calculation service.

[0015] According to another aspect of an embodiment of the present invention, a data transmission device between distributed nodes is also provided, comprising: an establishment unit for responding to a data transmission request and establishing a first path hierarchy diagram, wherein the data transmission request is used to request the transmission of target data from a source node to a destination node, and the first path hierarchy diagram comprises: a connection path between the source node and the destination node, each of the connection paths being composed of M distributed nodes, each of the distributed nodes in the first path hierarchy diagram being marked with a level number, where M is a positive integer; a processing unit for obtaining the number of free connections for each pair of the distributed nodes, and based on the first path hierarchy diagram and the number of free connections for each pair of the distributed nodes, annotating the number of occupied forward connections and the number of reverse free connections in an iterative manner. a number of connections, wherein each pair of the distributed nodes is composed of two of the M distributed nodes having a connection relationship, the forward connection occupancy number is the number of connections occupied by each pair of the distributed nodes in the direction from the source node to the destination node, and the reverse free connection number is the number of free connections of each pair of the distributed nodes in the direction from the destination node to the source node; a determining unit is configured to determine connection path data between the source node and the destination node based on the forward connection occupancy number and the reverse free connection number, wherein the connection path data includes: the number of all connection paths established from the source node to the destination node; and a transmitting unit is configured to transmit the target data from the source node to the destination node based on the connection path data.

[0016] Furthermore, the establishment unit includes: a first processing sub-unit, used to adopt a breadth-first search method, starting from the source node, to search for adjacent nodes layer by layer, and when the adjacent node is found, to determine whether the adjacent node is the destination node and whether the adjacent node has been marked with a level; a marking sub-unit, used to mark the level of the adjacent node when the adjacent node is not the destination node and the adjacent node has not been marked with a level, until the level of the destination node is marked, thereby obtaining the first path level graph.

[0017] Furthermore, the processing unit includes: a second processing sub-unit, used for step 1, based on the number of free connections of each pair of distributed nodes in the first path hierarchy graph, using a breadth-first search method, to find the connection path between the source node and the destination node in the first path hierarchy graph, and mark the number of occupied forward connections and the number of reverse free connections between the nodes in the found connection path; a third processing sub-unit, used for step 2, based on the number of occupied forward connections and the number of reverse free connections between the marked nodes, to establish a second path hierarchy graph, and mark the number of occupied forward connections and the number of reverse free connections of each node in the second path hierarchy graph; a fourth processing sub-unit, used for step 3, using the second path hierarchy graph as the first path hierarchy graph; a fifth processing sub-unit, used to repeatedly execute steps 1 to 3 until the connection path from the source node to the destination node cannot be found in the first path hierarchy graph.

[0018] Furthermore, the processing unit further includes: an adjusting subunit for adjusting the forward connection occupancy number by the reverse free connection number after marking the forward connection occupancy number and the reverse free connection number between the nodes in the found connection path.

[0019] Furthermore, the processing unit also includes: a first acquisition sub-unit, used to obtain the maximum number of connections of the first node accessing the second node, wherein the first node and the second node are two of the distributed nodes with a connection relationship among the M distributed nodes, and the first node and the second node constitute a pair of distributed nodes; a second acquisition sub-unit, used to obtain the number of connections between the first node and the second node; and a calculation sub-unit, used to calculate the difference between the maximum number of connections and the number of connections to obtain the number of free connections between the first node and the second node.

[0020] Furthermore, the transmission unit includes: an establishment subunit for establishing a secure socket layer channel from the source node to the destination node based on the connection path data; and a transmission subunit for transmitting the target data from the source node to the destination node based on the secure socket layer channel.

[0021] Furthermore, each of the distributed nodes is a distributed node of a microservice system, and the target data is file data.

[0022] According to another aspect of an embodiment of the present invention, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned methods for transmitting data between distributed nodes by executing the executable instructions.

[0023] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any of the above-mentioned methods for transmitting data between distributed nodes.

[0024] In the present invention, in response to a data transmission request, a first path hierarchy graph is established, wherein the data transmission request is used to request the transmission of target data from a source node to a destination node, and the first path hierarchy graph includes: a connection path between the source node and the destination node, each connection path consisting of M distributed nodes, and each distributed node in the first path hierarchy graph is labeled with a hierarchy number, where M is a positive integer; the number of available connections for each pair of distributed nodes is obtained, and based on the first path hierarchy graph and the number of available connections for each pair of distributed nodes, a forward connection occupancy number and a reverse connection occupancy number are iteratively labeled, wherein each pair of distributed nodes consists of two distributed nodes having a connection relationship among the M distributed nodes, the forward connection occupancy number is the number of connections occupied by each pair of distributed nodes in the direction from the source node to the destination node, and the reverse connection occupancy number is the number of connections occupied by each pair of distributed nodes in the direction from the destination node to the source node; based on the forward connection occupancy number and the reverse connection occupancy number, connection path data between the source node and the destination node is determined, wherein the connection path data includes: the number of all connection paths established from the source node to the destination node; and based on the connection path data, the target data is transmitted from the source node to the destination node. This solves the technical problem of poor transmission effect in related technologies of using transit nodes for transit or establishing SSL channels between any two nodes to achieve file transfer between distributed nodes of a microservice system. In the present invention, a path hierarchy diagram between a source node and a destination node is established to find a connection path from the source node to the destination node for data transmission, thereby avoiding the high resource overhead of establishing an SSL channel between each pair of nodes in the related technologies and the situation of limited number of connections when data is transmitted uniformly through transit nodes, thereby achieving the technical effect of improving the transmission efficiency and resource utilization of data from the source node to the destination node. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0026] Figure 1 is a flowchart of an optional method for data transmission between distributed nodes according to an embodiment of the present invention;

[0027] Figure 2 is a structural diagram of an optional distributed node according to an embodiment of the present invention;

[0028] Figure 3 is a hierarchical diagram of an optional inter-node file transmission path according to an embodiment of the present invention;

[0029] Figure 4 is an optional data transmission system deployment diagram according to an embodiment of the present invention;

[0030] Figure 5 is an optional distributed inter-node file transmission path initialization diagram according to an embodiment of the present invention;

[0031] Figure 6 An optional distributed node connection strategy according to an embodiment of the present invention Figure 1 ;

[0032] Figure 7 An optional distributed node connection strategy according to an embodiment of the present invention Figure 2 ;

[0033] Figure 8 An optional distributed node connection strategy according to an embodiment of the present invention Figure 3 ;

[0034] Figure 9 is a schematic diagram of an optional distributed inter-node data transmission device according to an embodiment of the present invention;

[0035] Figure 10 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] It should be noted that the data transmission method, system and device between distributed nodes in the present disclosure can be used in the financial technology field when processing file transfer between distributed nodes under a microservice system, and can also be used in any field other than the financial technology field when processing file transfer between distributed nodes under a microservice system. The present disclosure does not limit the application field of the data transmission method, system and device between distributed nodes.

[0039] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, and file data) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0040] For ease of description, some terms or nouns involved in various embodiments of the present invention are explained below:

[0041] BFS: Breadth-first search algorithm, starting from the initial state of the problem (starting point), traverses all possible states (other nodes) according to the state transition rules (edges in the graph structure) until the terminal state (end point) is found.

[0042] SSL (Secure Sockets Layer) is a protocol that provides layered data security between applications (such as HTTP (Hypertext Transfer Protocol), Telnet (Remote Terminal Protocol), NMTP (Network Information Transfer Protocol), and FTP (File Transfer Protocol)) and TCP / IP (Transmission Control Protocol / Internet Protocol).

[0043] Sidecar Pattern: A service mesh is a dedicated infrastructure layer that handles inter-service communication in a microservices architecture. It typically implements the service mesh architecture using the sidecar pattern. The sidecar pattern deploys a service mesh proxy alongside the services to handle functions such as service discovery, traffic management, load balancing, health checks, circuit breakers and failover, and authentication.

[0044] Example 1

[0045] According to an embodiment of the present invention, a method embodiment of an optional method for transmitting data between distributed nodes is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0046] Figure 1 FIG. 1 is a flow chart of an optional method for data transmission between distributed nodes according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0047] Step S101: In response to a data transmission request, a first path hierarchy diagram is established, wherein the data transmission request is used to request the transmission of target data from a source node to a destination node. The first path hierarchy diagram includes: a connection path between the source node and the destination node, each connection path is composed of M distributed nodes, and each distributed node in the first path hierarchy diagram is marked with a level number, where M is a positive integer.

[0048] The above-mentioned data transmission request may request to transmit the target data of the source node to the destination node. The above-mentioned target data may be file data. The above-mentioned source node, destination node and M distributed nodes may be distributed nodes in the microservice system.

[0049] Figure 2 is a structural diagram of an optional distributed node according to an embodiment of the present invention, such as Figure 2 As shown, the distributed nodes in this embodiment may include: node remaining connection number calculation service, file transfer service, sidecar, network protocol stack, etc., among which the sidecar can be used for service discovery, traffic management and inspection..., the network protocol stack can provide intra-node communication and inter-node communication services, and the file transfer service can be used for data transmission.

[0050] In order to avoid the situation in the related art where file transfer relies on a central node, which may cause a delay in the destination node receiving the data sent by the source node when the central node has current limiting or failure, in this embodiment, after a data transmission request is issued, a data transmission path hierarchy diagram from the source node to the destination node (corresponding to the above-mentioned first path hierarchy diagram) can be established, and then all data transmission paths from the source node to the destination node can be determined based on the first path hierarchy diagram.

[0051] Figure 3 FIG. 1 is a schematic diagram of an optional inter-node file transmission path hierarchy according to an embodiment of the present invention. Figure 3 In the file transfer path hierarchy diagram shown, s is the source node, the node level is 0; t is the destination node, the level is 3; u and v nodes are at level 1; z and w nodes are at level 2.

[0052] Step S102, obtain the number of free connections of each pair of distributed nodes, and based on the first path hierarchy diagram and the number of free connections of each pair of distributed nodes, use an iterative method to mark the forward connection occupancy number and the reverse free connection number, wherein each pair of distributed nodes is composed of two distributed nodes with a connection relationship among M distributed nodes, the forward connection occupancy number is the number of connection occupancy of each pair of distributed nodes in the direction from the source node to the destination node, and the reverse free connection number is the number of free connections of each pair of distributed nodes in the direction from the destination node to the source node.

[0053] like Figure 3 The file transfer path hierarchy diagram shown below is combined with Figure 3 The meaning of each pair of distributed nodes is explained with examples: if distributed node s and distributed node u have a connection relationship, then su can be a pair of distributed nodes; if distributed node s and distributed node v have a connection relationship, then sv can be a pair of distributed nodes; if distributed node u and distributed node v have a connection relationship, then uv can be a pair of distributed nodes; based on the same idea, uz, uw, vw, wz, wt and zt can also form a pair of distributed nodes respectively.

[0054] To find the remaining connection data for each pair of nodes (corresponding to the aforementioned free connection data) and determine all connection path data between the source node and the destination node, the maximum number of connections per IP configured for the remaining connection calculation service of each accessed node (i.e., the maximum number of connections for each request to access the node) can be called, and the corresponding number of connections per IP corresponding to the access-applicant node in the accessed node (i.e., the number of connections for each request to access the node) can be subtracted to obtain the maximum remaining connections for the pair of nodes (corresponding to the aforementioned number of free connections for each pair of distributed nodes). Then, based on the free connection data for each pair of distributed nodes and the first path hierarchy diagram, the number of occupied forward connections and the number of free reverse connections for each pair of nodes are annotated in an iterative manner.

[0055] Figure 4 is an optional data transmission system according to an embodiment of the present invention, such as Figure 4 As shown, in this embodiment, a file transfer scheduling policy processor is established. Before each batch processing function of file transfer needs to be run, the node remaining connection number calculation service of each node is periodically called from the service grid control panel to obtain the remaining maximum connection number of all nodes and initialize it in the cache.

[0056] The above-mentioned forward connection occupancy number can be the number of connections for each pair of distributed nodes connected in the direction from the source node to the destination node, and the above-mentioned reverse free connection number can be the free connections for each pair of distributed nodes in the direction from the destination node to the source node.

[0057] Step S103: determining the connection path data between the source node and the destination node based on the number of occupied forward connections and the number of available reverse connections, wherein the connection path data includes: the number of all connection paths established from the source node to the destination node.

[0058] In this embodiment, the number of forward connections occupied and the number of reverse connections available can be determined. According to the number of forward connections available and the number of reverse connections available for each pair of nodes, all available connection paths for data transmission from the source node to the destination node can be determined to obtain the connection path data between the source node and the destination node.

[0059] Step S104: transmitting the target data from the source node to the destination node based on the connection path data.

[0060] In this embodiment, target data may be transmitted from the source node to the destination node through the SSL channel based on all connection data from the source node to the destination node determined by the path hierarchy graph.

[0061] In this embodiment, the file transfer scheduling policy processor can also push the results generated by the above steps to the corresponding distributed nodes through the service grid control panel; these nodes that receive the results establish corresponding connections with the next node through the file transfer microservice based on the two information of the number of connections and the next node. For example, the u node receives the two information of u-4-z and u-6-w, and thus initiates 4 and 6 connection requests to the z and w nodes respectively through the file transfer microservice.

[0062] It should be noted that the data transmission method between distributed nodes provided in the first embodiment of the present invention can be applied to file transmission between distributed nodes of microservices. The file transmission microservice of the distributed nodes of the microservices is based on the FTP protocol, and an SSL channel must be established based on the security principle in the transmission of confidential files. If the data is to be transmitted from node A to node B, the premise for creating an SSL channel is that there is a certificate exchange between the two nodes. This embodiment can realize a system and method for each distributed node in the microservice system to freely transmit files so that the destination node can obtain more file resources. It makes full use of the SSL channel established due to the RPC (remote procedure call) call relationship of the microservice, and transfers the file from the source node to the destination node by calling each node of the mesh graph, thereby achieving the technical effect of improving the efficiency and security of file transmission between distributed nodes in the microservice system.

[0063] Through the above steps, in this embodiment, by establishing a path hierarchy diagram between the source node and the destination node, the connection path from the source node to the destination node is found for data transmission, avoiding the high resource overhead of establishing an SSL channel between each pair of nodes in the related technology, and uniformly transmitting data through a transit node, which has a limit on the number of connections, thereby achieving the technical effect of improving the transmission efficiency and resource utilization of data transmission from the source node to the destination node. This solves the technical problem of poor transmission effect in the related technology of using a transit node for transit or establishing an SSL channel between any two nodes to achieve file transfer between distributed nodes of a microservice system.

[0064] Optionally, a first path hierarchy graph is established, including: using a breadth-first search method, starting from the source node, searching for adjacent nodes layer by layer, and when an adjacent node is found, determining whether the adjacent node is the destination node and whether the adjacent node has been marked with a level; when the adjacent node is not the destination node and the adjacent node has not been marked with a level, marking the level of the adjacent node until the level of the destination node is marked, thereby obtaining the first path hierarchy graph.

[0065] In this embodiment, in response to a data transmission request, each time the microservice call relationship ( Figure 4The registration center includes an update function) when notifying the file transfer scheduling policy processor, and can establish a file transfer path diagram between nodes (corresponding to the first path hierarchy diagram mentioned above) based on the SSL channel generated by the service call relationship between distributed nodes.

[0066] Specifically, a breadth-first-search (BFS) algorithm can be used to search for adjacent nodes layer by layer, starting from the source node. If the adjacent node is not the destination node and has already been labeled with a level, processing is skipped. If it has not yet been labeled, the level is accumulated. If the adjacent node is the destination node, the labeled level and the accumulated level are compared each time, and the maximum is taken. After BFS is completed, a hierarchical graph of the file transfer path between nodes is generated.

[0067] The following combination Figure 3 For example, BFS is used for searching, s is the source node, s level is 0; first find s's adjacent nodes u, v, and mark them as level 1; u finds u's adjacent nodes w, v, z, and since v's level has been marked and skipped, z and w's levels are marked as 2; v finds v's adjacent node w, which is also marked and skipped; w finds z and t, and marks t as level 3; z's adjacent node is t, the destination node has been marked as level 3, z's level is 2, and the cumulative result 2+1=3=t's marked level, which is not updated. So far, the level marking is complete. It should be noted that Figure 3 This is an optional path hierarchy diagram that does not display all nodes. For example, if w has other paths, resulting in the cumulative result n of w-...-t being greater than 3, the level of t is updated to the cumulative result n, achieving the technical effect of improving the efficiency of searching for connection paths from the source node to the destination node.

[0068] Optionally, the number of free connections of each pair of distributed nodes is obtained, and based on the first path hierarchy graph and the number of free connections of each pair of distributed nodes, the forward connection occupancy number and the reverse free connection number are marked in an iterative manner, including: step 1, based on the number of free connections of each pair of distributed nodes in the first path hierarchy graph, a breadth-first search method is used to find the connection path between the source node and the destination node in the first path hierarchy graph, and the forward connection occupancy number and the reverse free connection number between the nodes in the found connection path are marked; step 2, based on the forward connection occupancy number and the reverse free connection number between the marked nodes, a second path hierarchy graph is established, and the forward connection occupancy number and the reverse free connection number of each node in the second path hierarchy graph are marked; step 3, the second path hierarchy graph is used as the first path hierarchy graph; and steps 1 to 3 are repeated until no connection path from the source node to the destination node can be found in the first path hierarchy graph.

[0069] In this embodiment, based on the generated first path hierarchy diagram, before each data transmission (for example, file transfer) is required, the file transfer scheduling policy processor calls the node connection timing collection service of each node through the service grid control panel, obtains the initialized maximum number of remaining connections, and establishes a forward connection number path between each node.

[0070] Iteratively search for any path from the source node to the destination node using the BFS algorithm. Each search is along the path between nodes from the lower level to the higher level. After completing an iteration, mark the number of forward connections occupied and create the same number of reverse free connections.

[0071] Before the next iterative search, the annotations of each node level are updated again according to the method of generating the first path hierarchy diagram. After generation, the forward free connection number and the reverse free connection number are both regarded as the path from the source node to the destination node; if a search passes through the reverse path, the original forward occupancy can be directly deducted when marking the reverse occupancy. At the same time, the reverse free connection number is equal to the forward connection occupancy.

[0072] The above three actions are performed iteratively until no path from the source node to the destination node can be found; the result is output according to the number of forward connections.

[0073] For example: Figure 5 is an optional distributed inter-node file transmission path initialization diagram according to an embodiment of the present invention, combined with Figure 5 To illustrate, Figure 5 The numbers on the line segment represent the maximum number of positive remaining connections between the two corresponding nodes (simplified as "positive free connection number"). For example, the number 10 on the line segment between node s and node u means that the number of positive free connections between node s and node u is 10; the number 10 on the line segment between node s and node v means that the number of positive free connections between node s and node v is 10; the number 4 on the line segment between node u and node z means that the number of positive free connections between node u and node z is 10; the number 4 on the line segment between node u and node w means that the number of positive free connections between node u and node z is 10; 8 means that the number of free forward connections between node u and node w is 10; the number 2 on the line segment between node u and node v means that the number of free forward connections between node u and node v is 2; the number 6 on the line segment between node w and node z means that the number of free forward connections between node w and node z is 6; the number 10 on the line segment between node w and node t means that the number of free forward connections between node w and node t is 10; the number 10 on the line segment between node z and node t means that the number of free forward connections between node z and node t is 10.

[0074] Figure 6 An optional distributed node connection strategy according to an embodiment of the present invention Figure 1 ,exist Figure 5 Based on the first iteration of the BFS algorithm, the schematic diagram is as follows Figure 6 As shown, since "each search is along the path between nodes from the lower level to the higher level", the path selection of uv and wz is invalidated, and the three paths of suzt, suwt and svwt are found, and the reverse free connection number is established (see Figure 6 The number of free forward connections = the total number of free forward connections - the number of occupied forward connections.

[0075] according to Figure 6 , update the annotation of each node level again according to the method of generating the first path hierarchy graph. After generation, the number of forward free connections and the number of reverse free connections are regarded as the path from the source node to the destination node, and the connection without path and the connection between nodes at the same level are invalidated, that is, the forward connection of uv, uw and the reverse connection of us, zu, tz, tw, wv and vs are invalidated, and the result is Figure 7 The distributed inter-node connection strategy shown Figure 2 ; The BFS algorithm finds the path svwzt and marks the forward and reverse occupancy (increase and decrease are both possible, corresponding to the decrease and increase of vacancy). Figure 7 No st path can be found from the forward and reverse connection numbers; the set of the occupied numbers of forward connections between each node is the maximum scheduling strategy for file transfer from st ( Figure 8 The distributed inter-node connection strategy shown Figure 3 ). s-10-u (10 is the number of connections between the node pairs in the final strategy, equal to Figure 5 The corresponding initialized number of free forward connections minus Figure 8 The final number of forward occupied connections), u-4-z, z-9-t, s-9-v, u-6-w, v-9-w, w-5-z, z-9-t, w-10-t, the total maximum number of connections that can be established from st is the sum of the entry paths of t, which is 19.

[0076] It should be noted that in this example, no path containing a reverse connection is found in the iterative search. Figure 7 Based on this, if we remove the connection to wz and increase the flow of uz to greater than 10 (Example 2), we can find that the path found after the BFS iteration will be svwuzt. Furthermore, increasing the occupancy of the reverse connection actually reduces the occupancy of the corresponding forward connection. Example 2 will not be repeated here.

[0077] In this embodiment, by finding all connection paths between the source node and the destination node, files can be transferred through all paths, thereby achieving a technical effect of improving the efficiency of file transfer.

[0078] Optionally, after marking the number of occupied forward connections and the number of free reverse connections between nodes in the found connection path, the method further includes: adjusting the number of occupied forward connections according to the number of free reverse connections.

[0079] In this embodiment, the reverse free connection number is established to reversely hedge the forward connection occupancy number. The execution level represents the fallback operation after the forward connection number is established. In this embodiment, it can be found that the reverse free connection number is always consistent with the number of forward connection occupancy, which also reflects the significance of the design and execution levels. When re-searching, both the "forward free connection number" and the "reverse free connection number" can be considered at the same time, achieving the technical effect of ensuring the reliability and fallback of the connection path between the source node and the destination node.

[0080] Optionally, obtaining the number of free connections for each pair of distributed nodes includes: obtaining the maximum number of connections for the first node to access the second node, wherein the first node and the second node are two distributed nodes with a connection relationship among the M distributed nodes, and the first node and the second node constitute a pair of distributed nodes; obtaining the number of connections between the first node and the second node; calculating the difference between the maximum number of connections and the number of connections to obtain the number of free connections between the first node and the second node.

[0081] In this embodiment, when calculating the number of available connections for each pair of distributed nodes, the remaining connection calculation service for each connected node (corresponding to the aforementioned second node) calculates the maximum number of connections per IP address of the access requesting node (corresponding to the aforementioned first node) from the configured maximum number of connections per IP address in the connected node to obtain the maximum remaining connections for that pair of nodes (corresponding to the aforementioned number of available nodes). The service grid control panel calls the inter-node remaining connection calculation service for all microservice nodes registered with it to obtain the current value, i.e., the number of available connections for that pair of nodes.

[0082] For example: In this implementation, the "maximum number of connections per IP" numble_max_per_ip can be configured in the CMC configuration center. Each time a connection is established between nodes, each node needs to create a process to serve it. At this time, the global variable "number of connections per IP" s_ip_count_hash+1 is calculated and then compared with numble_max_per_ip. If it is less than or equal to numble_max_per_ip, the connection can be established, achieving the technical effect of improving the accuracy of calculating the number of free connections for each pair of nodes.

[0083] Optionally, transmitting the target data from the source node to the destination node based on the connection path data includes: establishing a secure socket layer channel from the source node to the destination node based on the connection path data; and transmitting the target data from the source node to the destination node based on the secure socket layer channel.

[0084] In this embodiment, after the nodes on each path from the source node to the destination node are connected to each other based on the connection path data, data is transmitted from the source node to the destination node through the SSL channel (corresponding to the above-mentioned secure socket layer channel) established between the microservices of each distributed node, thereby achieving the purpose of data transmission based on each distributed node from the source node to the destination node, and improving the technical effect of the reliability of data transmission between distributed nodes.

[0085] Optionally, each distributed node is a distributed node of a microservice system, and the target data is file data.

[0086] It should be noted that since the time point of file transfer operation in the related technology is mostly during the batch processing time period at night when the number of connections between nodes is relatively stable, in this embodiment, the specific routing method of the effective access traffic of each node (i.e., the maximum number of remaining connections) can be fully utilized to allow more files to be efficiently transferred from the source node to the destination node. It can also avoid the situation in the related technology where when file transfer is performed through a transit node, if the transit node fails, it will cause serious delays in the destination node receiving the file, thereby achieving the technical effect of improving the efficiency and reliability of file transfer between distributed nodes.

[0087] Example 2

[0088] Embodiment 2 of the present application provides an optional data transmission system between distributed nodes, which is used to execute the data transmission method between distributed nodes in embodiment 1.

[0089] In the data transmission system between distributed nodes provided in the second embodiment of the present application, the data transmission system includes: a distributed node; a scheduling strategy processing module, wherein the scheduling strategy processing module is used to respond to a data transmission request and establish a first path hierarchy diagram; based on the first path hierarchy diagram and the number of free connections of each pair of distributed nodes, an iterative method is used to mark the number of occupied forward connections and the number of free reverse connections; based on the number of occupied forward connections and the number of free reverse connections, the connection path data between the source node and the destination node is determined, wherein the data transmission request is used to request the target data to be transmitted from the source node to the destination node, and the first path hierarchy diagram includes: a connection path between the source node and the destination node, each connection path consists of M The system is composed of distributed nodes. Each distributed node in the first path hierarchy diagram is marked with a level number. Each pair of distributed nodes is composed of two distributed nodes with a connection relationship among M distributed nodes. The forward connection occupancy number is the connection occupancy number of each pair of distributed nodes from the source node to the destination node. The reverse free connection number is the free connection number of each pair of distributed nodes from the destination node to the source node. The connection path data includes: the number of all connections established from the source node to the destination node; based on the connection path data, the target data is transmitted from the source node to the destination node; the control panel is used to call the free connection number calculation service in each distributed node to obtain the free connection number of each pair of distributed nodes.

[0090] like Figure 4 The data transmission system deployment diagram shown in FIG. 1 may include multiple distributed nodes (such as Figure 4 Node a, node b, node c, node d, node e, node f... node g, node h, node i, etc. in the service grid), scheduling strategy processing module (corresponding to Figure 4 File transfer scheduling policy processor in the control panel (corresponding to Figure 4 ).

[0091] Figure 4 The file transfer scheduling policy processor in the service grid control panel can periodically obtain the maximum number of remaining connections of all nodes and initialize them in the cache by calling the node remaining connection number calculation service of each node before running the batch processing function of file transfer each time.

[0092] Each time the microservice call relationship is updated ( Figure 4 When a new SSL connection is established between the nodes (including the update function of the registration center), the file transfer scheduling policy processor is notified. Based on the SSL channel generated by the service call relationship between the nodes, a file transfer path diagram (corresponding to the first path hierarchy diagram mentioned above) is established between the nodes. Based on this first path hierarchy diagram, the connection path data between the source node and the destination node is determined, and the target data of the source node is transferred to the destination node.

[0093] In this embodiment, by establishing a path hierarchy diagram between the source node and the destination node, the connection path from the source node to the destination node is found for data transmission, thereby avoiding the high resource overhead of establishing an SSL channel between each pair of nodes in the related art, and uniformly performing data transmission through a transit node, which has a connection number limit, thereby achieving the technical effect of improving the transmission efficiency and resource utilization of data transmission from the source node to the destination node, and further solving the technical problem of poor transmission effect in the related art of using a transit node for transit or establishing an SSL channel between any two nodes to achieve file transfer between distributed nodes of a microservice system.

[0094] Optionally, the distributed node further includes: a transmission service module, wherein the transmission service module is used to provide data transmission services; and a calculation service module, wherein the calculation service module is used to provide free connection number calculation services.

[0095] like Figure 3 The distributed node structure diagram shown in FIG. 1 shows a distributed node structure diagram. In this embodiment, the distributed node may include a transmission service module (such as Figure 3 File transfer services in ) and computing service modules (such as Figure 3 In this implementation, the distributed nodes may further include: sidecars, network protocol stacks, etc., wherein the sidecars may be used for service discovery, traffic management, and inspection..., the network protocol stack may provide intra-node and inter-node communication services, and the file transfer service may be used for data transmission.

[0096] Since the time point of file transfer operation in the related art is mostly during the batch processing time period at night when the number of connections between nodes is relatively stable, the file transfer system between distributed nodes provided by the second embodiment can fully utilize the specific routing method of the effective access traffic of each node (that is, the maximum number of remaining connections), so that more files can be efficiently transferred from the source node to the destination node. It can also avoid the situation in the related art where when the file is transferred through the transit node, if the transit node fails, it will cause serious delays in the destination node receiving the file, thereby achieving the technical effect of improving the efficiency and reliability of file transfer between distributed nodes.

[0097] Example 3

[0098] Embodiment 3 of the present application provides an optional data transmission device between distributed nodes, and each implementation unit in the data transmission device corresponds to each implementation step in embodiment 1.

[0099] Figure 9 FIG is a schematic diagram of an optional distributed data transmission device between nodes according to an embodiment of the present invention. Figure 9 As shown, the data transmission device includes: an establishing unit 91, a processing unit 92, a determining unit 93, and a transmitting unit 94.

[0100] An establishing unit 91 is configured to establish a first path hierarchy graph in response to a data transmission request, wherein the data transmission request is for requesting the transmission of target data from a source node to a destination node, the first path hierarchy graph comprising: a connection path between the source node and the destination node, each connection path comprising M distributed nodes, each distributed node in the first path hierarchy graph being labeled with a level number, where M is a positive integer;

[0101] Processing unit 92 is configured to obtain the number of free connections for each pair of distributed nodes, and iteratively mark the number of occupied forward connections and the number of free reverse connections based on the first path hierarchy graph and the number of free connections for each pair of distributed nodes. Each pair of distributed nodes is composed of two distributed nodes having a connection relationship among the M distributed nodes. The number of occupied forward connections is the number of occupied connections for each pair of distributed nodes in the direction from the source node to the destination node, and the number of free reverse connections is the number of free connections for each pair of distributed nodes in the direction from the destination node to the source node.

[0102] The determining unit 93 is configured to determine the connection path data between the source node and the destination node based on the number of occupied forward connections and the number of available reverse connections, wherein the connection path data includes: the number of all connection paths established between the source node and the destination node;

[0103] The transmission unit 94 is configured to transmit target data from the source node to the destination node based on the connection path data.

[0104] In the data transmission device between distributed nodes provided in the third embodiment of the present application, a first path hierarchy diagram is established by responding to a data transmission request through an establishment unit 91, wherein the data transmission request is used to request that the target data be transmitted from the source node to the destination node, and the first path hierarchy diagram includes: a connection path between the source node and the destination node, each connection path is composed of M distributed nodes, and each distributed node in the first path hierarchy diagram is marked with a level number, where M is a positive integer; the number of free connections for each pair of distributed nodes is obtained through a processing unit 92, and based on the first path hierarchy diagram and the number of free connections for each pair of distributed nodes, the forward connection occupancy number and the number of free connections are marked in an iterative manner. The number of reverse free connections, wherein each pair of distributed nodes is composed of two distributed nodes having a connection relationship among the M distributed nodes, the forward connection occupancy number is the connection occupancy number of each pair of distributed nodes in the direction from the source node to the destination node, and the reverse free connection number is the free connection number of each pair of distributed nodes in the direction from the destination node to the source node; the determination unit 93 determines the connection path data between the source node and the destination node based on the forward connection occupancy number and the reverse free connection number, wherein the connection path data includes: the number of all connection paths established from the source node to the destination node; the transmission unit 94 transmits the target data from the source node to the destination node based on the connection path data. This solves the technical problem of poor transmission effect in related technologies of using transit nodes for transit or establishing SSL channels between any two nodes to achieve file transfer between distributed nodes of a microservice system. In this embodiment, a path hierarchy diagram is established between the source node and the destination node, and a connection path from the source node to the destination node is found for data transmission, thereby avoiding the high resource overhead of establishing an SSL channel between each pair of nodes in the related technology, and the situation of limited number of connections when data is transmitted uniformly through transit nodes, thereby achieving the technical effect of improving the transmission efficiency and resource utilization of data from the source node to the destination node.

[0105] Optionally, in the data transmission device between distributed nodes provided in Example 3 of the present application, the establishment unit includes: a first processing sub-unit, used to use a breadth-first search method to search for adjacent nodes layer by layer starting from the source node, and when the adjacent node is found, determine whether the adjacent node is the destination node and whether the adjacent node has been marked with a level; a marking sub-unit, used to mark the level of the adjacent node when the adjacent node is not the destination node and the adjacent node has not been marked with a level, until the level of the destination node is marked to obtain a first path level graph.

[0106] Optionally, in the data transmission device between distributed nodes provided in Example 3 of the present application, the processing unit includes: a second processing sub-unit, used for step 1, based on the number of free connections of each pair of distributed nodes in the first path hierarchy graph, using a breadth-first search method, to find the connection path between the source node and the destination node in the first path hierarchy graph, and mark the number of occupied forward connections and the number of reverse free connections between the nodes in the found connection path; a third processing sub-unit, used for step 2, based on the number of occupied forward connections and the number of reverse free connections between the marked nodes, to establish a second path hierarchy graph, and mark the number of occupied forward connections and the number of reverse free connections of each node in the second path hierarchy graph; a fourth processing sub-unit, used for step 3, using the second path hierarchy graph as the first path hierarchy graph; a fifth processing sub-unit, used to repeatedly execute steps 1 to 3 until no connection path from the source node to the destination node can be found in the first path hierarchy graph.

[0107] Optionally, in the data transmission device between distributed nodes provided in Example 3 of the present application, the processing unit also includes: an adjustment sub-unit, which is used to adjust the forward connection occupancy number according to the reverse free connection number after marking the forward connection occupancy number and the reverse free connection number between the nodes in the found connection path.

[0108] Optionally, in the data transmission device between distributed nodes provided in Example 3 of the present application, the processing unit also includes: a first acquisition sub-unit, used to obtain the maximum number of connections of the first node accessing the second node, wherein the first node and the second node are two distributed nodes with a connection relationship among the M distributed nodes, and the first node and the second node constitute a pair of distributed nodes; a second acquisition sub-unit, used to obtain the number of connections between the first node and the second node; and a calculation sub-unit, used to calculate the difference between the maximum number of connections and the number of connections to obtain the number of free connections between the first node and the second node.

[0109] Optionally, in the data transmission device between distributed nodes provided in Example 3 of the present application, the transmission unit includes: an establishment subunit, used to establish a secure socket layer channel from the source node to the destination node based on the connection path data; and a transmission subunit, used to transmit the target data from the source node to the destination node based on the secure socket layer channel.

[0110] Optionally, in the data transmission device between distributed nodes provided in Example 3 of the present application, each distributed node is a distributed node of a microservice system, and the target data is file data.

[0111] The above-mentioned data transmission device between distributed nodes can also include a processor and a memory. The above-mentioned establishment unit 91, processing unit 92, determination unit 93 and transmission unit 94 are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0112] The above-mentioned processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and by adjusting the kernel parameters, a path hierarchy diagram between the source node and the destination node is established to find the connection path from the source node to the destination node for data transmission, thereby avoiding the establishment of an SSL channel between each pair of nodes in the related technology, which has a large resource overhead, and uniformly transmits data through a transit node, which has a limit on the number of connections, thereby achieving the technical effect of improving the transmission efficiency and resource utilization of data transmission from the source node to the destination node.

[0113] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0114] According to another aspect of an embodiment of the present invention, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned methods for transmitting data between distributed nodes by executing the executable instructions.

[0115] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any of the above-mentioned methods for transmitting data between distributed nodes.

[0116] Figure 10 is a schematic diagram of an electronic device according to an embodiment of the present invention, such as Figure 10 As shown, an embodiment of the present invention provides an electronic device 100, which includes a processor, a memory, and a program stored in the memory and runnable on the processor. When the processor executes the program, any one of the above-mentioned methods for transmitting data between distributed nodes is implemented.

[0117] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0118] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0119] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0120] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0121] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0122] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0123] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for transmitting data between distributed nodes, characterized in that: include: In response to a data transmission request, establishing a first path hierarchy graph, wherein the data transmission request is used to request transmission of target data from a source node to a destination node, the first path hierarchy graph comprising: a connection path between the source node and the destination node, each connection path consisting of M distributed nodes, each distributed node in the first path hierarchy graph being labeled with a level number, where M is a positive integer; Obtaining the number of free connections for each pair of distributed nodes, and based on the first path hierarchy graph and the number of free connections for each pair of distributed nodes, iteratively marking the number of occupied forward connections and the number of free reverse connections, wherein each pair of distributed nodes is composed of two distributed nodes having a connection relationship among the M distributed nodes, the number of occupied forward connections is the number of occupied connections for each pair of distributed nodes in the direction from the source node to the destination node, and the number of free reverse connections is the number of free connections for each pair of distributed nodes in the direction from the destination node to the source node; Determine connection path data between the source node and the destination node based on the number of occupied forward connections and the number of available reverse connections, wherein the connection path data includes: the number of all connection paths established between the source node and the destination node; The target data is transmitted from the source node to the destination node based on the connection path data.

2. The data transmission method according to claim 1, wherein: Establish the first pathway hierarchy diagram, including: Adopting a breadth-first search method, starting from the source node, searching for adjacent nodes layer by layer, and when the adjacent node is found, determining whether the adjacent node is the destination node and whether the adjacent node has been marked with a level; In the case that the adjacent node is not the destination node and the adjacent node is not marked with a level, the levels of the adjacent nodes are marked until the level of the destination node is marked, thereby obtaining the first path level graph.

3. The data transmission method according to claim 1, wherein: Obtaining the number of free connections for each pair of the distributed nodes, and iteratively marking the number of occupied forward connections and the number of free reverse connections based on the first path hierarchy graph and the number of free connections for each pair of the distributed nodes, including: Step 1: Based on the number of free connections for each pair of distributed nodes in the first path hierarchical graph, a breadth-first search method is used to find a connection path between the source node and the destination node in the first path hierarchical graph, and the number of occupied forward connections and the number of free reverse connections between the nodes in the found connection path are marked; Step 2: establishing a second path hierarchy graph based on the marked forward connection occupancy numbers and the reverse free connection numbers between the nodes, and marking the forward connection occupancy numbers and the reverse free connection numbers of each node in the second path hierarchy graph; Step 3, using the second pathway hierarchy diagram as the first pathway hierarchy diagram; Repeat steps 1 to 3 until no connection path from the source node to the destination node can be found in the first path hierarchy graph.

4. The data transmission method according to claim 3, wherein: After marking the number of occupied forward connections and the number of free reverse connections between nodes in the found connection path, the method further includes: The number of occupied forward connections is adjusted according to the number of available reverse connections.

5. The data transmission method according to claim 1, wherein: Obtain the number of free connections between each pair of distributed nodes, including: Obtaining a maximum number of connections between a first node and a second node, wherein the first node and the second node are two distributed nodes having a connection relationship among the M distributed nodes, and the first node and the second node constitute a pair of distributed nodes; Obtaining the number of connections between the first node and the second node; The difference between the maximum number of connections and the number of connections is calculated to obtain the number of free connections of the first node and the second node.

6. The data transmission method according to claim 1, wherein: Transmitting the target data from the source node to the destination node based on the connection path data includes: establishing a secure socket layer channel from the source node to the destination node based on the connection path data; The target data is transmitted from the source node to the destination node based on the secure socket layer channel.

7. The data transmission method according to claim 1, wherein: Each of the distributed nodes is a distributed node of a microservice system, and the target data is file data.

8. A data transmission system between distributed nodes, characterized in that: The data transmission system is used to execute the data transmission method according to any one of claims 1 to 7, comprising: Distributed nodes; A scheduling strategy processing module, wherein the scheduling strategy processing module is used to respond to a data transmission request and establish a first path hierarchy diagram; based on the first path hierarchy diagram and the number of free connections for each pair of distributed nodes, an iterative method is used to mark the number of occupied forward connections and the number of free reverse connections; based on the number of occupied forward connections and the number of free reverse connections, the connection path data between the source node and the destination node is determined, wherein the data transmission request is used to request that the target data be transmitted from the source node to the destination node, and the first path hierarchy diagram includes: a connection path between the source node and the destination node, each of the connection paths consisting of M distributed nodes. The first path hierarchical graph is composed of M distributed nodes, each distributed node is marked with a level number, each pair of distributed nodes is composed of two distributed nodes with a connection relationship among the M distributed nodes, the forward connection occupancy number is the number of connections occupied by each pair of distributed nodes in the direction from the source node to the destination node, the reverse free connection number is the number of free connections of each pair of distributed nodes in the direction from the destination node to the source node, the connection path data includes: the number of all connections established from the source node to the destination node; based on the connection path data, the target data is transmitted from the source node to the destination node; A control panel, wherein the control panel is used to call a free connection number calculation service in each of the distributed nodes to obtain the free connection number of each pair of distributed nodes.

9. The data transmission system according to claim 8, characterized in that The distributed node also includes: A transmission service module, wherein the transmission service module is used to provide data transmission services; A calculation service module, wherein the calculation service module is used to provide the free connection number calculation service.

10. A data transmission device between distributed nodes, characterized in that: include: an establishing unit, configured to establish a first path hierarchical graph in response to a data transmission request, wherein the data transmission request is used to request transmission of target data from a source node to a destination node, the first path hierarchical graph comprising: a connection path between the source node and the destination node, each connection path consisting of M distributed nodes, each distributed node in the first path hierarchical graph being labeled with a level number, where M is a positive integer; a processing unit, configured to obtain the number of free connections for each pair of the distributed nodes, and iteratively mark the number of occupied forward connections and the number of unoccupied reverse connections based on the first path hierarchy graph and the number of unoccupied connections for each pair of the distributed nodes, wherein each pair of the distributed nodes is composed of two of the M distributed nodes that have a connection relationship, the number of occupied forward connections is the number of occupied connections for each pair of the distributed nodes in the direction from the source node to the destination node, and the number of unoccupied reverse connections is the number of unoccupied connections for each pair of the distributed nodes in the direction from the destination node to the source node; a determining unit, configured to determine connection path data between the source node and the destination node based on the number of occupied forward connections and the number of available reverse connections, wherein the connection path data includes: the number of all connection paths established between the source node and the destination node; A transmission unit is configured to transmit the target data from the source node to the destination node based on the connection path data.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for data transmission between distributed nodes according to any one of claims 1 to 7.

12. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the data transmission method between distributed nodes as described in any one of claims 1 to 7.

Images