A geographic distributed system communication method, device and equipment

Abstract

The present application relates to the technical field of distributed communication, and discloses a kind of geographic distributed system communication method, device and equipment.The present application can create the multiple topology nodes of one-to-one correspondence with multiple physical nodes in geographic distributed system;And, according to the communication delay between two physical nodes, the edge between two topology nodes is created, and the length of edge is the communication delay between the corresponding two physical nodes.Determine the shortest communication path between two topology nodes, and construct target topology structure based on each topology node and each shortest communication path.According to target topology structure, multiple topology nodes are grouped to obtain at least one topology node group.Based on the group leader node and member node in each topology node group, hierarchical communication between physical nodes is formed by grouping.The present application can effectively improve the efficiency of all-to-all high frequency communication in geographic distributed system by reducing long-distance delay transmission and bandwidth occupation.

Description

Technical Field

[0001] This invention relates to the field of distributed communication, and more particularly to a communication method, apparatus, and device for a geographically distributed system. Background Technology

[0002] With the globalization of information technology development, the application scope of geographic distributed systems is becoming increasingly wide, and the role of geographic distributed systems is becoming more and more important.

[0003] As data centers continue to expand geographically, frequent cross-domain communication can lead to performance bottlenecks. For example, during transaction processing, cross-domain transactions need to be coordinated across multiple data centers. The high latency of wide area networks (WANs) causes a surge in communication delays, directly impacting transaction commit times, resulting in a significant drop in system throughput and a deterioration in user experience.

[0004] Currently, there is an urgent need for a method that can effectively improve the communication efficiency of geographic distributed systems. Summary of the Invention

[0005] This invention provides a communication method, apparatus, and device for geographic distributed systems, which addresses the shortcomings of low communication efficiency in related technologies and improves the communication efficiency of geographic distributed systems.

[0006] In a first aspect, the present invention provides a communication method for a geographically distributed system, comprising: Create multiple topology nodes that correspond one-to-one with multiple physical nodes in a geographic distributed system; and create edges between two topology nodes based on the communication delay between each pair of physical nodes, wherein the length of the edge is equal to the communication delay between the two corresponding physical nodes. Based on the edges between any two of the topological nodes, determine the shortest communication path between any two of the topological nodes; A target topology is constructed based on each of the topology nodes and each of the shortest communication paths, wherein the target topology includes the edges in each of the topology nodes and each of the shortest communication paths; The plurality of topology nodes are grouped according to the target topology to obtain at least one topology node group, and each topology node group includes a group leader node and group member nodes; Based on the group leader node and the group member nodes in each of the topology node groups, the communication between each of the physical nodes is controlled.

[0007] Optionally, determining the shortest communication path between any two of the topology nodes based on the edges between each pair of the topology nodes includes: An initial topology is constructed based on each of the topology nodes and the edges between each pair of the topology nodes, wherein the initial topology includes each of the topology nodes and the edges between each pair of the topology nodes; For any two topology nodes in the initial topology, determine each communication path between the two topology nodes, calculate the length of each communication path based on the length of the edge in each communication path, and determine the shortest communication path between the two topology nodes based on the length of each communication path. The construction of the target topology based on each of the topology nodes and each of the shortest communication paths includes: In the initial topology, identify the edges that none of the shortest communication paths have passed through, and use them as target edges; In the initial topology, each target edge is pruned to obtain the target topology.

[0008] Optionally, grouping the plurality of topology nodes according to the target topology to obtain at least one topology node group includes: Each intermediate node in each of the shortest communication paths of the target topology is determined; The total number of intermediate nodes is counted. If the total number of intermediate nodes is greater than 0, at least one group leader node is determined based on the total number of intermediate nodes, and each intermediate node is traversed. For any of the intermediate nodes traversed, a first communication path passing through the intermediate node is determined in each of the shortest communication paths, the length of each of the first communication paths is determined, the maximum length is determined in the length of each of the first communication paths, and the maximum length is used as the importance of the intermediate node. For any number of group leader nodes M, the M most important values ​​are determined from the importance of each intermediate node. The intermediate nodes corresponding to the M most important values ​​are taken as initial group leader nodes. The corresponding initial topology node groups are determined according to each initial group leader node. Each initial topology node group includes the initial group leader node and the corresponding initial member nodes. The entire initial topology node group is taken as the initial grouping scheme corresponding to the number of group leader nodes. The final grouping scheme is determined based on the communication delay of each initial grouping scheme, and each initial topology node group in the final grouping scheme is determined as the topology node group, and the initial group leader node and the initial group member node in each topology node group are respectively determined as the group leader node and the group member node.

[0009] Optionally, determining the number of at least one group leader node based on the total number of intermediate nodes includes: Take the square root of the total number of intermediate nodes to obtain the corresponding square root value; If the square root value is an integer, then the square root value is directly determined to be the number M of the group leader nodes; If the square root value is not an integer, then the square root value is rounded up and rounded down to obtain the integers above and below, and both the integers above and below are determined as the number M of the group leader nodes.

[0010] Optionally, determining the corresponding initial topology node group based on each initial group leader node includes: For any first topology node other than each of the initial group leader nodes: if the shortest communication path from the first topology node to any second topology node is a direct path, then the first topology node is determined as the initial group leader node and is treated as a separate initial topology node group; wherein, the second topology node is any topology node other than the first topology node. If there is at least one non-direct path in the shortest communication path from the first topology node to any second topology node, and at least one of the non-direct paths includes the initial group leader node belonging to an intermediate node, then the first topology node is determined as a group member identity node, each non-direct path including the initial group leader node belonging to an intermediate node is determined as a second communication path, and the number of second communication paths is counted. When there is one second communication path and one initial group leader node belonging to the intermediate node in the second communication path, the first topology node is directly used as the initial group member node of the initial group leader node in the second communication path. When there are multiple second communication paths, and the number of initial group leader nodes belonging to intermediate nodes in each second communication path is 1, the second communication path with the longest length is determined in each second communication path, and the first topology node is taken as the initial group member node of the initial group leader node in the second communication path with the longest length. When the number of the second communication paths is N, and at least one of the second communication paths includes multiple initial group leader nodes belonging to intermediate nodes, in each of the initial group leader nodes belonging to intermediate nodes in each of the second communication paths, the target initial group leader node that is closest to the first topology node is determined, and the first topology node is taken as the initial group member node of the target initial group leader node; where N is an integer greater than or equal to 1; If each of the non-direct paths does not include the initial group leader node that is an intermediate node, then the first topology node is determined as the initial group leader node and is treated as a separate initial topology node group. For any initial group leader node, the initial group leader node and each corresponding initial group member node are collectively considered as an initial topology node group.

[0011] Optionally, determining the final packet scheme based on the communication delay of each of the initial packet schemes includes: Set a communication strategy for each initial topology node group in each initial grouping scheme; wherein, the communication strategy is that topology nodes in the same initial topology node group can communicate directly, while topology nodes in different initial topology node groups need to communicate indirectly through the corresponding initial group leader node; While following the communication strategy, the longest communication path in each of the initial grouping schemes is determined, and the target path length for each of the longest communication paths is determined. Determine the minimum length in each target path length, and determine the initial grouping scheme corresponding to the minimum length as the final grouping scheme.

[0012] Optionally, controlling the communication between each physical node based on the group leader node and the group member nodes in each topology node group includes: Based on the group leader node and the group member node in each of the topology node groups, the plurality of physical nodes are grouped to obtain at least one physical node group. Each physical node group includes an aggregate node corresponding to the group leader node and a regular node corresponding to the group member node. Send communication instructions to the aggregation node and the ordinary node in each physical node group; When the at least one physical node group includes a first physical node group and a second physical node group, the first physical node group includes a first aggregation node and a first ordinary node, and the second physical node group includes a second aggregation node and a second ordinary node, the communication instruction is used to trigger the first ordinary node to perform data interaction with the second ordinary node in sequence through the first aggregation node and the second aggregation node, and is also used to trigger the first aggregation node to receive sub-data sent by each ordinary node in the first physical node group, perform white data filtering on all the sub-data, obtain processed data, and send it to the target device.

[0013] Optionally, the geographic distributed system is a geographic distributed database or a geographic distributed training system.

[0014] In a second aspect, the present invention provides a geographically distributed system communication device, comprising: A node creation unit is used to create multiple topological nodes that correspond one-to-one with multiple physical nodes in a geographic distributed system; and to create edges between two of the topological nodes based on the communication delay between each pair of physical nodes, wherein the length of the edge is equal to the communication delay between the two corresponding physical nodes. The path determination unit is used to determine the shortest communication path between any two of the topological nodes based on the edges between them. A topology building unit is used to construct a target topology structure based on each of the topology nodes and each of the shortest communication paths, wherein the target topology structure includes the edges in each of the topology nodes and each of the shortest communication paths; A node grouping unit is used to group the plurality of topology nodes according to the target topology structure to obtain at least one topology node group, wherein each topology node group includes a group leader node and group member nodes. A communication control unit is used to control the communication between each physical node based on the group leader node and the group member nodes in each topology node group.

[0015] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the geographic distributed system communication method of the first aspect or any corresponding embodiment described above.

[0016] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the geographic distributed system communication method of the first aspect or any corresponding embodiment thereof.

[0017] The present invention provides a communication method, apparatus, and device for a geographic distributed system. This method can create multiple topological nodes that correspond one-to-one with multiple physical nodes in the geographic distributed system. It also creates edges between pairs of topological nodes based on the communication delay between each pair of physical nodes, with the length of each edge equal to the communication delay between the two physical nodes. Based on these edges, the shortest communication path between each pair of topological nodes is determined. A target topology structure is constructed based on each topological node and each shortest communication path, including the edges in each topological node and each shortest communication path. The multiple topological nodes are grouped according to the target topology structure to obtain at least one topological node group, with each group including a leader node and member nodes. Communication between each physical node is controlled based on the leader node and member nodes in each topological node group. This invention can effectively improve the communication efficiency between different physical nodes in a geographic distributed system. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a geographic distributed system communication method provided in an embodiment of the present invention; Figure 2 A flowchart of a target topology network construction method is provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating data interaction between different physical nodes, provided as an embodiment of the present invention. Figure 4 This is a schematic diagram of data transmission in a geographic distributed database provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of data transmission in a geographic distributed training system provided in an embodiment of the present invention; Figure 6 A flowchart illustrating another geographic distributed system communication method provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of a geographic distributed system communication device provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0021] The following is combined Figures 1-6 The present invention describes a geographic distributed system communication method.

[0022] like Figure 1 As shown, this embodiment proposes a first communication method for a geographically distributed system, which may include the following steps: S101. Create multiple topological nodes that correspond one-to-one with multiple physical nodes in the geographic distributed system; and create edges between two topological nodes based on the communication delay between two physical nodes, with the length of the edge being the communication delay between the corresponding two physical nodes.

[0023] The geographic distributed system includes multiple physical nodes, and each physical node is a distributed node within the geographic distributed system.

[0024] Specifically, this embodiment can create a unique topology node corresponding to each physical node and detect the communication latency between any two physical nodes. For any two physical nodes, an edge is created between the two topology nodes based on the communication latency between them, and the length of the edge is equal to the communication latency.

[0025] S102. Determine the shortest communication path between any two topological nodes based on the edges between them.

[0026] Specifically, in this embodiment, for any two topology nodes, all communication paths between the two topology nodes can be determined first, and the length of each communication path can be counted, thereby determining the shortest communication path with the smallest length.

[0027] S103. Construct a target topology structure based on each topology node and each shortest communication path. The target topology structure includes the edges in each topology node and each shortest communication path.

[0028] Specifically, this embodiment can construct a target topology structure consisting of all topology nodes and the shortest communication path between any two topology nodes.

[0029] Optionally, in other geographic distributed system communication methods proposed in this embodiment, step S102 may include: Construct an initial topology based on each topology node and the edges between every pair of topology nodes. The initial topology includes each topology node and the edges between every pair of topology nodes. For any two topological nodes in the initial topology, determine each communication path between the two topological nodes, calculate the length of each communication path based on the length of the edges in each communication path, and determine the shortest communication path between the two topological nodes based on the length of each communication path. At this point, step S103 may include: In the initial topology, identify all edges that the shortest communication paths have not traversed and use them as target edges; Prune each target edge in the initial topology to obtain the target topology.

[0030] like Figure 2 As shown, when the geographic distributed system includes 6 physical nodes, this embodiment can create 6 topology nodes numbered from 0 to 5. This embodiment can periodically measure the communication latency between any two physical nodes to construct a topology node structure. Figure 2 The time delay matrix shown in the second subgraph is constructed as follows: Figure 2 The initial topology is shown in the first subgraph. Subsequently, this embodiment can determine the following... Figure 2 The third subgraph shows the shortest communication paths between each pair of topological nodes. Subsequently, this embodiment can prune edges that are completely unused in all shortest communication paths within the initial topology, removing unnecessary connections and simplifying the network structure to obtain the following... Figure 2 The target topology is shown in the fourth subgraph.

[0031] S104. Group multiple topology nodes according to the target topology structure to obtain at least one topology node group. Each topology node group includes a group leader node and group member nodes.

[0032] Specifically, in this embodiment, all topology nodes can be grouped based on the target topology to obtain one or more topology node groups. Each topology node group may include a group leader node and at least one group member node.

[0033] like Figure 2 As shown, in this embodiment, the six topology nodes numbered from 0 to 5 can be divided into three topology node groups. The first topology node group includes topology nodes 1 and 0, where topology node 1 is the group leader node and topology node 0 is a group member node. The second topology node group includes topology nodes 3 and 2, where topology node 3 is the group leader node and topology node 2 is a group member node. The third topology node group includes topology nodes 5 and 4, where topology node 5 is the group leader node and topology node 4 is a group member node.

[0034] S105. Based on the leader node and member nodes in each topology node group, control the communication between each physical node.

[0035] Specifically, this embodiment can control the communication between different physical nodes in a geographic distributed system based on the leader node and member nodes in the topology node group.

[0036] Optionally, step S105 may include: Based on the leader node and member nodes in each topology node group, multiple physical nodes are grouped to obtain at least one physical node group. Each physical node group includes the aggregate node corresponding to the leader node and the ordinary node corresponding to the member node. Send communication commands to the aggregate nodes and ordinary nodes in each physical node group; When at least one physical node group includes a first physical node group and a second physical node group, the first physical node group includes a first aggregation node and a first ordinary node, and the second physical node group includes a second aggregation node and a second ordinary node, the communication command is used to trigger the first ordinary node to perform data interaction with the second ordinary node in sequence through the first aggregation node and the second aggregation node. It is also used to trigger the first aggregation node to receive sub-data sent by each ordinary node in the first physical node group, perform white data filtering on all sub-data, obtain processed data, and send it to the target device.

[0037] It is understood that this embodiment can be based on the topology node group obtained by grouping multiple topology nodes, and then group multiple physical nodes accordingly to obtain the corresponding physical node group.

[0038] Specifically, in this embodiment, for any topology node group, the physical node corresponding to the group leader node in the topology node group can be determined as an aggregate node, and the physical nodes corresponding to each member node in the topology node group can be determined as ordinary nodes. The determined aggregate node and ordinary nodes together are considered as a physical node group. At this time, this embodiment can divide multiple physical nodes into at least one physical node group, and each physical node group includes an aggregate node and at least one corresponding ordinary node.

[0039] It should be noted that this embodiment can send control commands to ordinary nodes and aggregation nodes in each physical node group to control the communication methods between different nodes. Specifically, data interaction between ordinary nodes in different physical node groups needs to be carried out through the aggregation nodes of their respective groups. The aggregation node can receive data sent by each ordinary node in the same physical node group, perform white data identification and filtering on all data, and then send the filtered data to the target device.

[0040] like Figure 2 The last child in Figure 5 As shown in the diagram, after the series of operations described above, a usable hierarchical grouping topology is clearly formed in the final small diagram. This structure can better organize and manage the nodes in the network and can deliver them to the next module for subsequent processing and operations, such as further data analysis, resource allocation, or task scheduling. This grouping method based on the delay matrix can effectively optimize the network topology and improve network performance and efficiency. The grouping method can be adjusted according to application requirements. The method shown here is one that does not undergo any significant optimization, ensuring that the forwarding process does not cause higher latency.

[0041] like Figure 3 The diagram illustrates data interaction between different physical nodes, with Data Delivery representing data interaction. The physical nodes within the dashed boxes are nodes in the same physical node group. Data transmission within the same physical node group is called intra-group transmission, while data transmission between different physical node groups is called inter-group transmission. Aggregator nodes can perform white data identification and filtering on conflict-free data blocks and data blocks containing conflicting information.

[0042] Optionally, the geographic distributed system can be a geographic distributed database or a geographic distributed training system.

[0043] like Figure 4 As shown, the geographic distributed system is a geographic distributed database. Figure 4 The example demonstrates multiple replicas (Replication 1, Replication 2, etc.), i.e., multiple physical nodes, which are assigned to different physical node groups, and aggregate nodes (such as Replication 4 and Replication N) are selected. The framework assigns these two aggregate nodes to handle white data identification and filtering operations, and aggregates data that needs to be further transmitted. Specifically, the filtering logic is to remove duplicate data (Reduce Redundant Data), for example, on Replication 1 and Replication 2... X Operations that assign a value of 1 will be merged into one entry; for the two conflicting operations Z=3 and Z=11, the AbortConflict Data operation will be performed. Figure 3 The document also labels Intra-group Communication and Inter-group Communication, demonstrating the data interaction process between different physical nodes. This embodiment uses an aggregation node to perform white data identification and filtering on data sent from different physical nodes, obtaining filtered data before retransmission, which improves the efficiency and accuracy of data processing.

[0044] like Figure 5 As shown, the geographic distributed system is a geographic distributed training system, comprising multiple data centers (DataCenters), each of which is a physical node. Data Center 1, Data Center 2, and Data Center 4 are grouped together as a physical node group, with Data Center 4 as the aggregation node and Data Center 1 and Data Center 2 as ordinary nodes. Data interaction between Data Center 1, Data Center 2, and Data Center 4 is intra-group communication. Data Center 1 and Data Center 2 send data to Data Center 4 for aggregation, performing white data identification and filtering on the database block data in the packet. Next offset refers to the offset of the next element, and WAN refers to the wide area network. Data interaction between Data Center 4 and Data Center N is inter-group communication.

[0045] Figure 5 The text also uses Data Center 1 as an example to introduce its internal workings. Specifically, Datashard is for data allocation, Worker Machine is for worker nodes, Parameter Server is for parameter servers, GlobalModel Copy is for global model replication, and LAN is for the local area network.

[0046] The geographic distributed system communication method proposed in this embodiment can create multiple topological nodes that correspond one-to-one with multiple physical nodes in the geographic distributed system; and create edges between pairs of topological nodes based on the communication latency between each pair of physical nodes, with the length of the edge being the communication latency between the corresponding two physical nodes. Based on the edges between pairs of topological nodes, the shortest communication path between each pair of topological nodes is determined. A target topology structure is constructed based on each topological node and each shortest communication path, including the edges in each topological node and each shortest communication path. Multiple topological nodes are grouped according to the target topology structure to obtain at least one topological node group, with each topological node group including a group leader node and group member nodes. Based on the group leader node and group member nodes in each topological node group, communication between each physical node is controlled. This embodiment can effectively improve the communication efficiency between different physical nodes in a geographic distributed system.

[0047] based on Figure 1 This embodiment proposes a second communication method for a geographically distributed system. In this method, step S104 may include: Determine each intermediate node in each shortest communication path of the target topology; Count the total number of intermediate nodes. If the total number of intermediate nodes is greater than 0, determine at least one group leader node based on the total number of intermediate nodes, and traverse each intermediate node. For any intermediate node encountered, determine the first communication path that passes through the intermediate node in each shortest communication path, determine the length of each first communication path, determine the maximum length in the length of each first communication path, and use the maximum length as the importance of the intermediate node. For any number of group leader nodes M, determine the M largest importance values ​​among the importance values ​​of each intermediate node, and take the intermediate nodes corresponding to the M importance values ​​as the initial group leader nodes. Determine the corresponding initial topology node groups based on each initial group leader node. Each initial topology node group includes the initial group leader node and the corresponding initial member nodes. Take each initial topology node group as a whole as the initial grouping scheme corresponding to the number of group leader nodes. The final grouping scheme is determined based on the communication delay of each initial grouping scheme, and each initial topology node group in the final grouping scheme is determined as a topology node group. The initial group leader node and the initial group member node in each topology node group are determined as the group leader node and the group member node, respectively.

[0048] In this context, an intermediate node is a node in the communication path from the source node to the destination node that is neither the source node nor the destination node. For example, in the communication path from topology node 1 to topology node 5 via topology node 3, topology node 3 is an intermediate node in that communication path.

[0049] Understandably, if the shortest communication path from the source node to the destination node consists only of the source node and the destination node, then this shortest communication path is a direct path from the source node to the destination node, and in this case, there are no intermediate nodes in the shortest communication path. For example, in the communication path from topology node 1 directly to topology node 5, there are no intermediate nodes.

[0050] Specifically, for a given intermediate node encountered during traversal, this embodiment can determine the first communication path (the shortest communication path passing through each pair of topological nodes) from the shortest communication paths between them. The lengths of all edges included in the first communication path are then summed to obtain the length of the first communication path. The maximum length of each first communication path is then determined, and this maximum length is assigned as the importance of the intermediate node. For example, if topological node 3 is traversed by two shortest communication paths, specifically the communication path from topological node 1 through topological node 3 to topological node 5 (length 14) and the communication path from topological node 1 through topological node 3 to topological node 4 (length 26), then the importance of topological node 3 is 26.

[0051] Optionally, determining the number of at least one group leader node based on the total number of intermediate nodes includes: Take the square root of the total number of intermediate nodes to obtain the corresponding square root value; If the square root value is an integer, then the square root value is directly determined to be the number of group leader nodes M; If the square root value is not an integer, then the square root value is rounded up and rounded down to obtain the integers above and below. Both the integers above and below are used to determine the number of group leader nodes M.

[0052] For example, when the total number of intermediate nodes is 9, its square root value is 3. In this embodiment, 3 can be directly determined as the number of group leader nodes. In this case, there is only one possible number of group leader nodes. As another example, when the total number of intermediate nodes is 5, its square root value is greater than 2 and less than 3. We can round up and down this square root value to obtain an integer of 3 and an integer of 2 respectively. In this case, there are two possible numbers of group leader nodes.

[0053] Specifically, in this embodiment, the number of group leader nodes for each scheme can be used to determine the corresponding group member nodes, construct an initial topology node group, thereby determining the corresponding initial grouping scheme. Then, based on the communication latency of each initial grouping scheme, the final grouping scheme can be determined.

[0054] Specifically, in this embodiment, the member nodes corresponding to each group leader node can be determined in advance.

[0055] Optionally, the above-mentioned determination of the corresponding initial topology node group based on each initial group leader node includes: For any first topology node other than each initial group leader node: if the shortest communication path from the first topology node to any second topology node is a direct path, then the first topology node is determined as the initial group leader node and is treated as a separate initial topology node group; wherein, the second topology node is a topology node other than the first topology node. If there is at least one non-direct path in the shortest communication path from the first topology node to any second topology node, and at least one non-direct path includes an initial group leader node belonging to an intermediate node, then the first topology node is determined as a group member identity node, each non-direct path including an initial group leader node belonging to an intermediate node is determined as a second communication path, and the number of second communication paths is counted. When there is one second communication path and one initial group leader node belonging to the intermediate node in the second communication path, the first topology node is directly used as the initial group member node of the initial group leader node in the second communication path. When there are multiple second communication paths, and the number of initial group leader nodes belonging to intermediate nodes in each second communication path is 1, the second communication path with the longest length is determined in each second communication path, and the first topology node is used as the initial group member node of the initial group leader node in the second communication path with the longest length. When the number of second communication paths is N, and at least one second communication path includes multiple initial group leader nodes belonging to intermediate nodes, in each initial group leader node belonging to intermediate nodes in each second communication path, the target initial group leader node that is closest to the first topology node is determined, and the first topology node is taken as the initial group member node of the target initial group leader node; where N is an integer greater than or equal to 1. If each non-direct path does not include an initial group leader node belonging to an intermediate node, then the first topology node is determined as the initial group leader node and is treated as a separate initial topology node group. For any initial group leader node, the initial group leader node and each corresponding initial group member node are treated as an initial topology node group.

[0056] The first topology node is a topology node other than the initial group leader node, and the second topology node is a topology node other than the first topology node.

[0057] Specifically, a direct path is a communication path that starts from the source node and goes directly to the destination node without passing through any intermediate nodes. A non-direct path is a communication path that starts from the source node and goes through at least one intermediate node before reaching the destination node.

[0058] Specifically, if at least one non-direct path exists in the shortest communication path from the first topology node to any second topology node, then the first topology node can be determined as the initial group member node. In this case, this embodiment needs to determine the initial group leader node corresponding to this initial group member node. This embodiment can first count all the second communication paths corresponding to the first topology node.

[0059] The second communication path starts from the first topology node, passes through the initial group leader node, and finally reaches the second topology node. It can be understood that the initial group leader node is an intermediate node in this communication path.

[0060] Specifically, when there is one second communication path corresponding to the first topology node, and there is one initial group leader node belonging to the intermediate node in the second communication path, this embodiment can directly determine the first topology node as the initial group member node of the initial group leader node in the second communication path. For example, in the communication paths starting from the first topology node 1, the first topology node 1 reaches the destination nodes 3 and 4 through the topology node 2, and the first topology node 1 directly reaches the destination nodes 5, 6, and 7. Among them, the topology node 2 is exactly the initial group leader node, so the first topology node 1 can be directly determined as the initial group member node of the topology node 2.

[0061] Specifically, when there are multiple second communication paths corresponding to the first topology node, and each second communication path has only one initial group leader node belonging to the intermediate nodes, this embodiment can determine the second communication path with the longest length and determine the first topology node as the initial group member node of the initial group leader node in the second communication path with the longest length. For example, in the communication paths starting from the first topology node 1, the first topology node 1 reaches the destination node 3 through topology node 2, the first topology node 1 reaches the destination node 5 through topology node 4, and the first topology node directly reaches the destination nodes 6 and 7. Among these, topology nodes 2 and 4 are both initial group leader nodes. In this case, if the length from the first topology node to the destination node 3 through topology node 2 is 12, and the length from the first topology node to the destination node 5 through topology node 4 is 15, then it can be considered that topology node 4 is more important than the first topology node 1, and the first topology node 1 is determined as the initial group member node of topology node 4.

[0062] Specifically, if at least one of the second communication paths corresponding to the first topology node includes multiple initial group leader nodes belonging to intermediate nodes, then among all the initial group leader nodes belonging to intermediate nodes in all the second communication paths, the target initial group leader node that is closest to the first topology node can be determined, and the first topology node can be determined as the initial member node of the target initial group leader node.

[0063] Specifically, if none of the non-direct paths corresponding to the first topology node include an initial group leader node belonging to an intermediate node, then the first topology node can be directly determined as an initial group leader node and treated as a separate initial topology node group. In this case, the initial group leader node does not have a corresponding group member node.

[0064] Specifically, in this embodiment, for any initial group leader node, the initial group leader node and all corresponding group member nodes can be treated as an initial topology node group.

[0065] Optionally, the determination of the final packet scheme based on the communication delay of each initial packet scheme includes: Set the communication strategy for each initial topology node group in each initial grouping scheme; wherein, the communication strategy is that topology nodes in the same initial topology node group can communicate directly, while topology nodes in different initial topology node groups need to communicate indirectly through the corresponding initial group leader node. Following the communication strategy, determine the longest communication path in each initial grouping scheme, and determine the target path length for each longest communication path; Determine the minimum length for each target path length, and then determine the initial grouping scheme corresponding to the minimum length as the final grouping scheme.

[0066] Specifically, this embodiment can set a first communication strategy between topology nodes in the same initial topology node group and a second communication strategy between topology nodes in different initial topology node groups.

[0067] The first communication strategy allows direct communication between topology nodes within the same initial topology node group. The second communication strategy requires communication between topology nodes in different initial topology node groups to be conducted indirectly through their respective initial group leader nodes. For example, the first initial topology node group includes the first initial group leader node, initial member node 1, and initial member node 2; the second initial topology node group includes the second initial group leader node, initial member node 3, and initial member node 4. Initial member nodes 1 and 2 can communicate directly with the first initial group leader node; initial member nodes 3 and 4 can communicate directly with the second initial group leader node. If initial member node 1 wants to communicate with initial member node 3, it must communicate with initial member node 3 sequentially through the first and second initial topology nodes. Similarly, if initial member node 4 wants to communicate with initial member node 2, it must communicate with initial member node 2 sequentially through the second and first initial topology nodes.

[0068] Specifically, this embodiment can determine the longest communication path in each initial grouping scheme, and the path length (i.e., the target path length) of each longest communication path, while following the first and second communication strategies. This embodiment can determine the minimum target path length (i.e., the minimum length) among all the target path lengths of the longest communication paths, and then determine the initial grouping scheme corresponding to the minimum length as the final grouping scheme.

[0069] The geographic distributed system communication method proposed in this embodiment can effectively group multiple topological nodes, determine the final grouping scheme, and control the communication path between multiple physical nodes based on the final grouping scheme, thereby improving the communication efficiency between physical nodes in the geographic distributed system.

[0070] In related technologies, cross-domain distributed databases such as Google Spanner, CockroachDB, Amazon Aurora, and GeoGauss hold a significant position in the market. While their multi-master architecture meets certain requirements for cross-regional data consistency and high availability, the increasing geographical distribution of data centers leads to severe performance bottlenecks due to frequent cross-domain communication. For example, during transaction processing, cross-domain transactions need to be coordinated across multiple data centers. The high latency of wide area networks (WANs) dramatically increases communication delays, directly impacting transaction commit times, resulting in a significant drop in system throughput and a deterioration in user experience. Meanwhile, the importance of distributed training in the field of artificial intelligence is self-evident. With the explosive growth of AI model parameters—such as GPT-4, LLaMA, and PaLM2 models with billions or more parameters—a single data center can no longer handle their massive computing and storage demands. Therefore, collaborative work among data centers across multiple geographical regions has become inevitable. However, the high latency and low bandwidth characteristics of WANs severely hinder the improvement of distributed training performance, becoming a key factor restricting the development of next-generation artificial intelligence.

[0071] Communication frameworks in related technologies, such as NCCL and Horovod, are primarily optimized for local area network (LAN) scenarios and face numerous challenges in wide area network (WAN) environments. NCCL experiences significantly increased synchronization overhead in high-latency environments, while Horovod is prone to performance bottlenecks under conditions of bandwidth fluctuations and high packet loss rates. Furthermore, ZeroMQ middleware used by databases like GeoGauss, lacking optimization for high-latency and bandwidth-constrained environments, is susceptible to message queuing delays and network jitter when transmitting large-scale data, leading to performance degradation.

[0072] like Figure 6 As shown, another geographic distributed system communication method proposed in this embodiment may include the following steps: Step 1: Latency and Bandwidth Collection. In this embodiment, each node first measures the round-trip latency with other nodes by periodically sending messages, and then aggregates this data to the central controller or central node to generate a network latency matrix. For network topology nodes, network information is collected to generate a bandwidth matrix.

[0073] Step 2: Grouping and Transmission Plan Generation. The grouping module uses a grouping algorithm to divide nodes into several groups. These groupings directly affect the transmission path. The system will plan the transmission path based on the current grouping situation, taking into account both the transmission load and the grouping situation, and generate the transmission plan for the current round.

[0074] In this embodiment, a latency matrix is ​​constructed by periodically measuring the round-trip latency between nodes. Simultaneously, combining node topology information and historical latency data, a clustering algorithm is used to divide the nodes into different small clusters. Based on this, a suitable aggregation node is selected for each cluster according to the node's location and connectivity within the cluster, as well as its latency characteristics. Specifically, this embodiment can employ active probing techniques, such as sending specific probe data packets, to obtain the communication latency between nodes. For the clustering algorithm, density-based clustering algorithms (such as DBSCAN) or hierarchical clustering algorithms can be selected, clustering nodes based on the combined similarity of latency and geographical information. When selecting aggregation nodes, factors such as node centrality (e.g., proximity centrality or betweenness centrality), bandwidth availability, and load conditions are comprehensively considered to ensure that the aggregation nodes can efficiently process and forward data.

[0075] Specifically, after constructing the target topology, this embodiment can monitor network status in real time, including link conditions, latency changes, and bandwidth utilization. When a violation of the latency triangle inequality is detected or a better indirect path is found, the data transmission path is dynamically adjusted, selecting to forward data through intermediate nodes to reduce transmission latency. Simultaneously, bandwidth resources are dynamically allocated based on network congestion and bandwidth requirements, prioritizing the transmission of critical data. This embodiment can utilize a latency-aware module to monitor and collect network status information. Path selection algorithms, such as an improved version of Dijkstra's algorithm, are used to find the optimal transmission path under certain constraints (such as maximum hop count limits and bandwidth requirements). For bandwidth allocation, dynamic bandwidth allocation algorithms, such as token bucket-based algorithms or proportional fairness algorithms, are employed to allocate bandwidth reasonably based on data priority and real-time bandwidth requirements.

[0076] Step 3: Data Transmission. The transmission module transmits data according to the grouping and transmission plan from the previous step. The transmission module incorporates a node identity verification mechanism; based on the node's identity, the transmission module guides the node to perform different operations.

[0077] Step 4: White Data Identification and Aggregation. If a node is determined to be an aggregation node in the transmission module, white data identification is performed. Note that the definition of white data may not be exactly the same in different applications; the filtering method here is defined according to the application. Data aggregation actually includes filtering white data and aggregating valid data. Generally, white data here can be considered as duplicate or conflicting data, and can also be further defined according to the application. Next, the data that needs to be transmitted is aggregated, generally through operations such as summation or averaging. In summary, after this step, the amount of data transmitted is reduced, but there should be no logical information loss for the application.

[0078] Specifically, this embodiment can analyze and process the data to be transmitted at the data sending end, identifying and filtering out blank data. At the aggregation node, data from nodes within the group is aggregated to reduce data transmission volume. Simultaneously, the data is organized and optimized according to application requirements, improving data effectiveness and usability. Data filtering at the aggregation node reduces the amount of data transmitted between groups, decreasing the overall data volume. Blank data is identified through data feature analysis and predefined rules. For example, in distributed database synchronization, duplicate or conflicting data is marked and filtered. Regarding aggregation operations, appropriate aggregation functions, such as summation, average calculation, or data merging operations, are used based on the data type and application scenario. Deduplication and conflict resolution methods based on application logic can also be employed to improve data transmission and processing efficiency. This embodiment can identify blank data through data feature analysis and predefined rules. For example, in distributed database synchronization, duplicate or conflicting data is marked and filtered. Regarding aggregation operations, appropriate aggregation functions, such as summation, average calculation, or data merging operations, are used based on the data type and application scenario. Deduplication and conflict resolution methods based on application logic can also be employed to improve data transmission and processing efficiency.

[0079] Step 5: Continuous Monitoring and Optimization. During each round of transmission, the system continues to monitor network dynamics to ensure its continued efficient operation in the new environment. Iterative optimization involves the system continuously adjusting based on real-time monitoring data to adapt to constantly changing network conditions.

[0080] This workflow comprehensively covers the entire process of this embodiment, from data collection, grouping and transmission plan generation, transmission cycle and white data identification and aggregation. This workflow achieves grouped and layered transmission.

[0081] It should be noted that related technologies face the challenge of reducing transmission latency and improving transmission efficiency during network transmission. For example, directly sending data in parallel can lead to long completion times (e.g., in all-to-all communication, at least 18 seconds of latency is required from node0 to node4), which affects network performance and becomes a technical challenge that needs to be addressed. This embodiment employs a dynamic path selection and adjustment module. This module optimizes network transmission through the following means: Real-time monitoring: The latency awareness module monitors the network status in real time, including link conditions, latency changes, and bandwidth utilization, providing data support for subsequent path selection and bandwidth allocation. Path selection algorithm: Path selection algorithms, such as an improved version of Dijkstra's algorithm, are used to find the optimal transmission path while meeting certain constraints (e.g., maximum hop count limit, bandwidth requirements, etc.). When a violation of the latency triangle inequality is detected or a better indirect path is found, the data transmission path is dynamically adjusted, such as forming a new transmission path from node0 to node3 via aggregation node1, thereby avoiding short latency.

[0082] This embodiment can reduce latency. As shown in the example of layered transmission, through reasonable grouping schemes and aggregation node selection, as well as dynamic path adjustment, the time to complete this round of all-to-all transmission is reduced from 18 seconds in direct parallel transmission to 17 seconds, effectively reducing transmission latency. It also improves performance by directly avoiding short latency in network transmission, achieving performance gains in completion time without affecting upper-layer applications, thus improving the overall network performance. Furthermore, it ensures the transmission of critical data: the dynamic bandwidth allocation algorithm can rationally allocate bandwidth resources according to data priority, ensuring the transmission quality and timeliness of critical data.

[0083] Specifically, the high latency and low bandwidth of wide area networks (WANs) in related technologies severely restrict the efficiency of geographically distributed collective communication, and are insufficient in optimizing communication efficiency, failing to meet the growing demand for cross-domain distributed applications. This embodiment can design new communication strategies to fully utilize the characteristics and potential optimization opportunities of WANs, significantly reducing communication latency and bandwidth overhead, and improving the efficiency of geographically distributed collective communication.

[0084] Specifically, in related technologies, the wide area network (WAN) environment undergoes dynamic changes, including changes in node status, link quality, and network topology, and communication solutions lack the ability to effectively adapt to these dynamic changes. This embodiment can construct a mechanism capable of sensing dynamic changes in the WAN in real time, dynamically adjusting communication strategies and parameters, and ensuring that the system maintains efficient operation under different network conditions.

[0085] Specifically, the hierarchical transmission strategy in related technologies still has many problems in terms of aggregation node selection, determination of the number of layers and groups, and optimization of inter-group communication, which affect its performance improvement effect. This embodiment can conduct in-depth research on the hierarchical transmission mechanism, determine the optimal aggregation node selection method, reasonable configuration of the number of layers and groups, and efficient inter-group communication strategy, so as to maximize the performance advantages of hierarchical transmission.

[0086] Specifically, in related technologies, a large amount of white data consumes bandwidth in wide area network transmission, and there is a lack of effective methods for identifying and processing white data, making it impossible to fully utilize bandwidth resources. This embodiment can design an efficient white data identification and filtering mechanism to preprocess data before transmission, reduce invalid data transmission, and optimize bandwidth allocation and utilization strategies to improve bandwidth utilization.

[0087] It should be noted that this embodiment can design a high-performance geographic distributed collective communication optimization framework. This system is specifically built for geographic distributed application scenarios and aims to innovate collective communication strategies and mechanisms, thereby enhancing the overall communication performance of cross-domain distributed systems.

[0088] This embodiment employs a hierarchical transmission strategy based on node latency, topology, and historical communication data. Through scientific clustering algorithms and a multi-factor-considered aggregation node selection method, it achieves efficient intra-group and inter-group data transmission, effectively addressing the challenges of complex WAN topologies and high latency and low bandwidth, and significantly improving communication efficiency.

[0089] This embodiment designs a dynamic path selection and adjustment module for real-time monitoring of network status. Utilizing network monitoring tools and advanced algorithms, it promptly detects and utilizes violations of the time delay triangle inequality and optimizes indirect paths to dynamically allocate bandwidth resources, ensuring critical data transmission and enhancing the system's adaptability to dynamic network changes.

[0090] This embodiment develops a precise data filtering and aggregation module. Based on data feature analysis and application requirements, it effectively identifies and filters white data, and uses appropriate aggregation functions and optimization techniques to reduce data transmission volume, improve data validity and availability, and optimize bandwidth utilization.

[0091] This embodiment constructs a dynamic adjustment and optimization mechanism during system operation. Each module continuously collects and analyzes operational data, and dynamically adjusts key aspects such as layered transmission, path selection, data processing, and protocol parameters based on the results, thereby achieving continuous improvement in system performance and maintaining a high-efficiency operating state.

[0092] like Figure 7 As shown in the figure, this embodiment proposes a communication device for a geographically distributed system, which may include: The node creation unit 701 is used to create multiple topological nodes that correspond one-to-one with multiple physical nodes in the geographic distributed system; and to create edges between two topological nodes based on the communication delay between two physical nodes, with the length of the edge being the communication delay between the two corresponding physical nodes. The path determination unit 702 is used to determine the shortest communication path between two topological nodes based on the edges between them. Topology building unit 703 is used to build a target topology structure based on each topology node and each shortest communication path, wherein the target topology structure includes the edges in each topology node and each shortest communication path; The node grouping unit 704 is used to group multiple topology nodes according to the target topology structure to obtain at least one topology node group, and each topology node group includes a group leader node and group member nodes. The communication control unit 705 is used to control the communication between each physical node based on the leader node and member nodes in each topology node group.

[0093] It should be noted that the processing procedures and beneficial effects of the node creation unit 701, path determination unit 702, topology construction unit 703, node grouping unit 704, and communication control unit 705 can be referred to respectively. Figure 1 Steps S101 to S105 are not described in detail here.

[0094] Optionally, the path determination unit 702 is further configured to: construct an initial topology structure based on each topology node and the edges between each pair of topology nodes, wherein the initial topology structure includes each topology node and the edges between each pair of topology nodes; For any two topological nodes in the initial topology, determine each communication path between the two topological nodes, calculate the length of each communication path based on the length of the edges in each communication path, and determine the shortest communication path between the two topological nodes based on the length of each communication path. Topology building block 703 is also used for: In the initial topology, identify all edges that the shortest communication paths have not traversed and use them as target edges; Prune each target edge in the initial topology to obtain the target topology.

[0095] Optionally, the node grouping unit 704 is also used for: Determine each intermediate node in each shortest communication path of the target topology; Count the total number of intermediate nodes. If the total number of intermediate nodes is greater than 0, determine at least one group leader node based on the total number of intermediate nodes, and traverse each intermediate node. For any intermediate node encountered, determine the first communication path that passes through the intermediate node in each shortest communication path, determine the length of each first communication path, determine the maximum length in the length of each first communication path, and use the maximum length as the importance of the intermediate node. For any number of group leader nodes M, determine the M largest importance values ​​among the importance values ​​of each intermediate node, and take the intermediate nodes corresponding to the M importance values ​​as the initial group leader nodes. Determine the corresponding initial topology node groups based on each initial group leader node. Each initial topology node group includes the initial group leader node and the corresponding initial member nodes. Take each initial topology node group as a whole as the initial grouping scheme corresponding to the number of group leader nodes. The final grouping scheme is determined based on the communication delay of each initial grouping scheme, and each initial topology node group in the final grouping scheme is determined as a topology node group. The initial group leader node and the initial group member node in each topology node group are determined as the group leader node and the group member node, respectively.

[0096] Optionally, the node grouping unit 704 is also used for: Take the square root of the total number of intermediate nodes to obtain the corresponding square root value; If the square root value is an integer, then the square root value is directly determined to be the number of group leader nodes M; If the square root value is not an integer, then the square root value is rounded up and rounded down to obtain the integers above and below. Both the integers above and below are used to determine the number of group leader nodes M.

[0097] Optionally, the node grouping unit 704 is also used for: For any first topology node other than each initial group leader node: if the shortest communication path from the first topology node to any second topology node is a direct path, then the first topology node is determined as the initial group leader node and is treated as a separate initial topology node group; wherein, the second topology node is a topology node other than the first topology node. If there is at least one non-direct path in the shortest communication path from the first topology node to any second topology node, and at least one non-direct path includes an initial group leader node belonging to an intermediate node, then the first topology node is determined as a group member identity node, each non-direct path including an initial group leader node belonging to an intermediate node is determined as a second communication path, and the number of second communication paths is counted. When there is one second communication path and one initial group leader node belonging to the intermediate node in the second communication path, the first topology node is directly used as the initial group member node of the initial group leader node in the second communication path. When there are multiple second communication paths, and the number of initial group leader nodes belonging to intermediate nodes in each second communication path is 1, the second communication path with the longest length is determined in each second communication path, and the first topology node is used as the initial group member node of the initial group leader node in the second communication path with the longest length. When the number of second communication paths is N, and at least one second communication path includes multiple initial group leader nodes belonging to intermediate nodes, in each initial group leader node belonging to intermediate nodes in each second communication path, the target initial group leader node that is closest to the first topology node is determined, and the first topology node is taken as the initial group member node of the target initial group leader node; where N is an integer greater than or equal to 1. If each non-direct path does not include an initial group leader node belonging to an intermediate node, then the first topology node is determined as the initial group leader node and is treated as a separate initial topology node group. For any initial group leader node, the initial group leader node and each corresponding initial group member node are treated as an initial topology node group.

[0098] Optionally, the node grouping unit 704 is also used for: Set the communication strategy for each initial topology node group in each initial grouping scheme; wherein, the communication strategy is that topology nodes in the same initial topology node group can communicate directly, while topology nodes in different initial topology node groups need to communicate indirectly through the corresponding initial group leader node. While following the communication strategy, determine the longest communication path in each initial grouping scheme, and determine the target path length for each longest communication path; Determine the minimum length for each target path length, and then determine the initial grouping scheme corresponding to the minimum length as the final grouping scheme.

[0099] Optionally, the communication control unit 705 is also used for: Based on the leader node and member nodes in each topology node group, multiple physical nodes are grouped to obtain at least one physical node group. Each physical node group includes the aggregate node corresponding to the leader node and the ordinary node corresponding to the member node. Send communication commands to the aggregate nodes and ordinary nodes in each physical node group; When at least one physical node group includes a first physical node group and a second physical node group, the first physical node group includes a first aggregation node and a first ordinary node, and the second physical node group includes a second aggregation node and a second ordinary node, the communication command is used to trigger the first ordinary node to perform data interaction with the second ordinary node in sequence through the first aggregation node and the second aggregation node. It is also used to trigger the first aggregation node to receive sub-data sent by each ordinary node in the first physical node group, perform white data filtering on all sub-data, obtain processed data, and send it to the target device.

[0100] Optionally, the geographic distributed system can be a geographic distributed database or a geographic distributed training system.

[0101] The geographic distributed system communication device proposed in this embodiment can create multiple topological nodes that correspond one-to-one with multiple physical nodes in the geographic distributed system; and create edges between pairs of topological nodes based on the communication latency between each pair of physical nodes, with the length of the edge being the communication latency between the corresponding two physical nodes. Based on the edges between pairs of topological nodes, the shortest communication path between each pair of topological nodes is determined. A target topology structure is constructed based on each topological node and each shortest communication path, including the edges in each topological node and each shortest communication path. Multiple topological nodes are grouped according to the target topology structure to obtain at least one topological node group, with each topological node group including a group leader node and group member nodes. Based on the group leader node and group member nodes in each topological node group, communication between each physical node is controlled. This embodiment can effectively improve the communication efficiency between different physical nodes in a geographic distributed system.

[0102] In this embodiment, the geographic distributed system communication device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0103] This invention also provides a computer device having the above-described features. Figure 7 The geographically distributed system communication device shown.

[0104] Please see Figure 8The present invention provides a schematic diagram of the structure of a computer device according to an optional embodiment. The computer device includes one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 Take a processor 10 as an example.

[0105] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0106] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0107] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0108] Memory 20 may include volatile memory, such as random access memory. Memory may also include non-volatile memory, such as flash memory, hard disk, or solid-state drive. Memory 20 may also include combinations of the above types of memory.

[0109] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0110] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A communication method for a geographically distributed system, characterized in that, include: Create multiple topological nodes that correspond one-to-one with multiple physical nodes in a geographic distributed system; Furthermore, based on the communication delay between each pair of physical nodes, an edge is created between each pair of topology nodes, the length of which is equal to the communication delay between the corresponding two physical nodes; Based on the edges between any two of the topological nodes, determine the shortest communication path between any two of the topological nodes; A target topology is constructed based on each of the topology nodes and each of the shortest communication paths, wherein the target topology includes the edges in each of the topology nodes and each of the shortest communication paths; The plurality of topology nodes are grouped according to the target topology to obtain at least one topology node group, and each topology node group includes a group leader node and group member nodes; Based on the group leader node and the group member nodes in each of the topology node groups, the communication between each of the physical nodes is controlled.

2. The method according to claim 1, characterized in that, The step of determining the shortest communication path between any two of the topological nodes based on the edges between them includes: An initial topology is constructed based on each of the topology nodes and the edges between each pair of the topology nodes, wherein the initial topology includes each of the topology nodes and the edges between each pair of the topology nodes; For any two topology nodes in the initial topology, determine each communication path between the two topology nodes, calculate the length of each communication path based on the length of the edge in each communication path, and determine the shortest communication path between the two topology nodes based on the length of each communication path. The construction of the target topology based on each of the topology nodes and each of the shortest communication paths includes: In the initial topology, identify the edges that none of the shortest communication paths have passed through, and use them as target edges; In the initial topology, each target edge is pruned to obtain the target topology.

3. The method according to claim 1, characterized in that, The step of grouping the plurality of topology nodes according to the target topology structure to obtain at least one topology node group includes: Each intermediate node in each of the shortest communication paths of the target topology is determined; The total number of intermediate nodes is counted. If the total number of intermediate nodes is greater than 0, at least one group leader node is determined based on the total number of intermediate nodes, and each intermediate node is traversed. For any of the intermediate nodes traversed, a first communication path passing through the intermediate node is determined in each of the shortest communication paths, the length of each of the first communication paths is determined, the maximum length is determined in the length of each of the first communication paths, and the maximum length is used as the importance of the intermediate node. For any number of group leader nodes M, the M most important values ​​are determined from the importance of each intermediate node. The intermediate nodes corresponding to the M most important values ​​are taken as initial group leader nodes. The corresponding initial topology node groups are determined according to each initial group leader node. Each initial topology node group includes the initial group leader node and the corresponding initial member nodes. The entire initial topology node group is taken as the initial grouping scheme corresponding to the number of group leader nodes. The final grouping scheme is determined based on the communication delay of each initial grouping scheme, and each initial topology node group in the final grouping scheme is determined as the topology node group, and the initial group leader node and the initial group member node in each topology node group are respectively determined as the group leader node and the group member node.

4. The method according to claim 3, characterized in that, The step of determining the number of at least one group leader node based on the total number of intermediate nodes includes: Take the square root of the total number of intermediate nodes to obtain the corresponding square root value; If the square root value is an integer, then the square root value is directly determined to be the number M of the group leader nodes; If the square root value is not an integer, then the square root value is rounded up and rounded down to obtain the integers above and below, and both the integers above and below are determined as the number M of the group leader nodes.

5. The method according to claim 3, characterized in that, The step of determining the corresponding initial topology node group based on each initial group leader node includes: For any first topology node other than each of the initial group leader nodes: if the shortest communication path from the first topology node to any second topology node is a direct path, then the first topology node is determined as the initial group leader node and is treated as a separate initial topology node group; wherein, the second topology node is any topology node other than the first topology node. If there is at least one non-direct path in the shortest communication path from the first topology node to any second topology node, and at least one of the non-direct paths includes the initial group leader node belonging to an intermediate node, then the first topology node is determined as a group member identity node, each non-direct path including the initial group leader node belonging to an intermediate node is determined as a second communication path, and the number of second communication paths is counted. When there is one second communication path and one initial group leader node belonging to the intermediate node in the second communication path, the first topology node is directly used as the initial group member node of the initial group leader node in the second communication path. When there are multiple second communication paths, and the number of initial group leader nodes belonging to intermediate nodes in each second communication path is 1, the second communication path with the longest length is determined in each second communication path, and the first topology node is taken as the initial group member node of the initial group leader node in the second communication path with the longest length. When the number of the second communication paths is N, and at least one of the second communication paths includes multiple initial group leader nodes belonging to intermediate nodes, in each of the initial group leader nodes belonging to intermediate nodes in each of the second communication paths, the target initial group leader node that is closest to the first topology node is determined, and the first topology node is taken as the initial group member node of the target initial group leader node; where N is an integer greater than or equal to 1; If each of the non-direct paths does not include the initial group leader node that is an intermediate node, then the first topology node is determined as the initial group leader node and is treated as a separate initial topology node group. For any initial group leader node, the initial group leader node and each corresponding initial group member node are collectively considered as an initial topology node group.

6. The method according to claim 3, characterized in that, The step of determining the final packet scheme based on the communication delay of each initial packet scheme includes: Set a communication strategy for each initial topology node group in each initial grouping scheme; wherein, the communication strategy is that topology nodes in the same initial topology node group can communicate directly, while topology nodes in different initial topology node groups need to communicate indirectly through the corresponding initial group leader node; While following the communication strategy, the longest communication path in each of the initial grouping schemes is determined, and the target path length for each of the longest communication paths is determined. Determine the minimum length in each target path length, and determine the initial grouping scheme corresponding to the minimum length as the final grouping scheme.

7. The method according to claim 1, characterized in that, The control of communication between each physical node based on the group leader node and the group member nodes in each topology node group includes: Based on the group leader node and the group member node in each of the topology node groups, the plurality of physical nodes are grouped to obtain at least one physical node group. Each physical node group includes an aggregate node corresponding to the group leader node and a regular node corresponding to the group member node. Send communication instructions to the aggregate node and the ordinary node in each physical node group; When the at least one physical node group includes a first physical node group and a second physical node group, the first physical node group includes a first aggregation node and a first ordinary node, and the second physical node group includes a second aggregation node and a second ordinary node, the communication instruction is used to trigger the first ordinary node to perform data interaction with the second ordinary node in sequence through the first aggregation node and the second aggregation node, and is also used to trigger the first aggregation node to receive sub-data sent by each ordinary node in the first physical node group, perform white data filtering on all the sub-data, obtain processed data, and send it to the target device.

8. The method according to claim 1, characterized in that, The geographic distributed system is either a geographic distributed database or a geographic distributed training system.

9. A communication device for a geographically distributed system, characterized in that, include: The node creation unit is used to create multiple topological nodes that correspond one-to-one with multiple physical nodes in the geographic distributed system. Furthermore, based on the communication delay between each pair of physical nodes, an edge is created between each pair of topology nodes, the length of which is equal to the communication delay between the corresponding two physical nodes; The path determination unit is used to determine the shortest communication path between any two of the topological nodes based on the edges between them. A topology building unit is used to construct a target topology structure based on each of the topology nodes and each of the shortest communication paths, wherein the target topology structure includes the edges in each of the topology nodes and each of the shortest communication paths; A node grouping unit is used to group the plurality of topology nodes according to the target topology structure to obtain at least one topology node group, wherein each topology node group includes a group leader node and group member nodes. A communication control unit is used to control the communication between each physical node based on the group leader node and the group member nodes in each topology node group.

10. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the geographic distributed system communication method according to any one of claims 1 to 8.

Images