Network data distribution method, system, device and computer readable storage medium
By selecting an initial distribution node in the network to split the subtree and using a specific batch of nodes to iteratively forward data, the problem of low server distribution efficiency when the number of network nodes is large is solved, and more efficient data transmission is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WEBANK (CHINA)
- Filing Date
- 2021-11-29
- Publication Date
- 2026-05-19
AI Technical Summary
In networks with a large number of nodes, the efficiency of servers distributing data to these nodes is low, especially when the number of clients is large. Existing technologies require a large amount of server resources to support service requests or long connections.
By selecting an initial distribution node in the network, the entire binary tree is split into subtrees, a specific batch of nodes is extracted, and these nodes are used to iteratively forward data in the network, reducing the need for direct connection between the server and all nodes, and using network nodes to jointly complete the data distribution task.
This reduces the load on the server, improves data forwarding efficiency, and makes each network node act as a data forwarder, which is more efficient than a single server distribution method.
Smart Images
Figure CN114124967B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of financial technology (Fintech), and more particularly to a network data distribution method, system, device, and computer-readable storage medium. Background Technology
[0002] With the development of computer technology, more and more technologies (big data, distributed systems, blockchain, artificial intelligence, etc.) are being applied in the financial field. The traditional financial industry is gradually transforming into financial technology (Fintech). However, due to the security and real-time requirements of the financial industry, higher demands are also being placed on computer network technology.
[0003] In a typical network cluster architecture, servers in the cluster usually act as responders, providing data to requesters. To obtain data from the server, requesters generally use two methods: First, the requester actively requests data from the server as a client. Second, the client maintains a persistent connection with the server, and the server proactively pushes data to the client in certain scenarios. In some near real-time scenarios, when the number of clients is large, if the method of clients actively requesting data from the server is adopted, a relatively large amount of server resources are needed to support the large number of service requests. If the method of clients maintaining a persistent connection with the server and the server proactively pushing data is adopted, a large amount of server resources are also needed to support the number of connections. Therefore, this reflects the problem of low efficiency in server-to-network data distribution in networks with a large number of network nodes. Summary of the Invention
[0004] The main objective of this invention is to propose a network data distribution method, system, device, and computer-readable storage medium, aiming to solve the technical problem of low efficiency in server-to-network node distribution in networks with a large number of network nodes.
[0005] To achieve the above objectives, the present invention provides a network data distribution method, the network data distribution method comprising:
[0006] Obtain the target data to be distributed, and select an initial distribution node from each network node of the pre-constructed target distributed network, wherein the target distributed network maintains routing table information of the whole network binary tree jointly mapped by each of the network nodes;
[0007] The target data and the routing table information are distributed to the initial distribution node. The routing table information is split on the initial distribution node according to its own perspective and the preset subtree splitting rules to obtain several split subtrees, and several specific batch nodes corresponding to the split subtrees are selected.
[0008] The target data and routing table information are distributed to the specific batch nodes through the initial distribution node, so that the target data and routing table information are iteratively forwarded in the target distributed network by the specific batch nodes until the network nodes in the entire network receive the target data.
[0009] Optionally, the step of splitting the routing table information on the initial distribution node according to its own perspective and preset subtree splitting rules to obtain several split subtrees includes:
[0010] From the perspective of the initial distribution node itself, starting from the root node of the entire binary tree, the subtrees that do not contain itself are split off along the direction of increasing node hierarchy, and these are the multiple split subtrees.
[0011] Optionally, the step of using the specific batch of nodes to iteratively forward the target data and the routing table information in the target distributed network until all network nodes receive the target data includes:
[0012] From the perspective of the specific batch of nodes, starting from the root node of the entire binary tree, the subtrees that do not contain themselves are split off along the direction of increasing node hierarchy, and these are used as multiple iteratively split subtrees.
[0013] Select nodes from the iteratively split subtree on the specific batch of nodes according to the routing table information;
[0014] The network nodes corresponding to the nodes in the specific batch are used as the iteration specific batch nodes, and the target data and the routing table information are distributed to the iteration specific batch nodes through the specific batch nodes;
[0015] Based on the iteration of specific batch nodes, from the perspective of the specific batch node itself, the steps of splitting out the subtrees that do not contain itself in the direction of increasing node hierarchy are performed starting from the root node of the entire network binary tree, until the network nodes of the entire network receive the target data.
[0016] Optionally, the step of distributing the target data and the routing table information to the specific batch nodes through the initial distribution node includes:
[0017] The initial distribution node distributes the target data, the routing table information, and the forwarding node list to the specific batch of nodes, wherein the forwarding node list is a list of network nodes that are currently distributing data, generated by the initial distribution node.
[0018] The step of distributing the target data and the routing table information to the iterative specific batch nodes through the specific batch nodes includes:
[0019] On the specific batch node, the iterative batch node is deduplicated using the forwarding node list, and the deduplicated iterative specific batch node is added to the forwarding node list;
[0020] The target data, the routing table information, and the list of forwarding nodes are distributed to the deduplicated iterative specific batch nodes through the specific batch nodes.
[0021] Optionally, the step of selecting a specific batch of nodes corresponding to a plurality of the split subtrees includes:
[0022] Extract the network node corresponding to the leftmost leaf node in each of the split subtrees as the node of the specific batch.
[0023] Optionally, the step of selecting an initial distribution node from the network nodes of the pre-built target distributed network includes:
[0024] Determine the network nodes that are mapped as leaf nodes in the entire binary tree, and select any one of the network nodes mapped as leaf nodes as the initial distribution node.
[0025] Optionally, before the step of obtaining the target data to be distributed, the method further includes:
[0026] Obtain the network join request sent by the network node to be joined, and obtain the node ID of the network node to be joined from the request;
[0027] The nodes to be added to the network are mapped to leaf nodes in the initial binary tree according to the preset binary tree mapping rules;
[0028] The node to be added to the network is added to a preset initial distributed network, and the node ID is added to the initial routing table information of the initial binary tree maintained by the initial distributed network, so as to construct the initial distributed network into the target distributed network.
[0029] Furthermore, to achieve the above objectives, the present invention also provides a network data distribution system, the network data distribution system comprising:
[0030] The initial node determination module is used to obtain the target data to be distributed and select an initial distribution node from each network node of the pre-constructed target distributed network, wherein the target distributed network maintains routing table information of the whole network binary tree jointly mapped by each of the network nodes;
[0031] A specific node selection module is used to distribute the target data and the routing table information to the initial distribution node, split the routing table information on the initial distribution node according to its own perspective and preset subtree splitting rules to obtain several split subtrees, and select several specific batch nodes corresponding to the split subtrees.
[0032] The data iterative distribution module is used to distribute the target data and the routing table information to the specific batch nodes through the initial distribution node, so as to use the specific batch nodes to iteratively forward the target data and the routing table information in the target distributed network until all network nodes receive the target data.
[0033] In addition, to achieve the above objectives, the present invention also provides a network data distribution device, the network data distribution device comprising: a memory, a processor, and a network data distribution program stored in the memory and executable on the processor, wherein the network data distribution program, when executed by the processor, implements the steps of the network data distribution method as described above.
[0034] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a network data distribution program, which, when executed by a processor, implements the steps of the network data distribution method described above.
[0035] In addition, to achieve the above objectives, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the network data distribution method described above.
[0036] This invention addresses the problem of low efficiency in data distribution by first identifying an initial distribution node in the network when data distribution is required. This initial node is then used to split a binary tree encompassing all nodes in the network into subtrees, extracting a specific batch of nodes and transmitting the target data to them. These nodes then iteratively forward the data according to the network's binary tree routing table. This allows the system to initially distribute data to only one node, with the distribution to other nodes handled collaboratively by all nodes in the network, eliminating the need to establish connections sequentially with each node. Therefore, when the number of nodes in the network is large, the pressure on the data server is reduced. Furthermore, by iteratively forwarding the target data to every node in the network through specific batches of nodes, each node effectively acts as a data forwarder. Compared to a server distributing data to all network nodes, this reduces the server's burden of maintaining network connections and improves data forwarding efficiency. This solves the technical problem of low efficiency in server-to-network node distribution in networks with a large number of nodes. Attached Figure Description
[0037] Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention;
[0038] Figure 2 This is a flowchart illustrating the first embodiment of the network data distribution method of the present invention;
[0039] Figure 3 This is a schematic diagram of subtree splitting in the second embodiment of the network data distribution method of the present invention;
[0040] Figure 4 This is a flowchart of a single iteration of the data distribution process in the second embodiment of the network data distribution method of the present invention.
[0041] Figure 5 A schematic diagram of filtering and deduplication at forwarding nodes in the second embodiment of the network data distribution method of the invention;
[0042] Figure 6 This is a schematic diagram of the functional modules of the network data distribution system of the present invention.
[0043] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0044] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0045] In a typical network cluster architecture, servers in the cluster usually act as responders, providing data to requesters. To obtain data from the server, requesters generally use two methods: First, the requester actively requests data from the server as a client. Second, the client maintains a persistent connection with the server, and the server proactively pushes data to the client in certain scenarios. In some near real-time scenarios, when the number of clients is large, if the method of clients actively requesting data from the server is adopted, a relatively large amount of server resources are needed to support the large number of service requests. If the method of clients maintaining a persistent connection with the server and the server proactively pushing data is adopted, a large amount of server resources are also needed to support the number of connections. Therefore, this reflects the problem of low efficiency in server-to-network data distribution in networks with a large number of network nodes.
[0046] To address the aforementioned issues, this invention provides a network data distribution method. When network data distribution is required, an initial distribution node is first determined within the network. This initial distribution node is then used to perform subtree splitting on a binary tree encompassing all nodes in the network, extracting a specific batch of nodes. The target data is then passed to these specific batch of nodes, allowing them to iteratively forward the data according to the network's binary tree routing table. This ensures that the system initially only needs to distribute data to one node in the network; the data distribution task for the remaining nodes is completed collaboratively by all nodes in the network, eliminating the need to establish connections sequentially with each node. Therefore, when the number of nodes in the network is large, the pressure on the data server is reduced. By iteratively forwarding the target data to every node in the network through specific batches of nodes, each node acts as a data forwarder. Compared to a server distributing data to all network nodes, this reduces the pressure on the server to maintain network connections and improves data forwarding efficiency, thus solving the technical problem of low efficiency in server-to-network data distribution in networks with a large number of nodes.
[0047] like Figure 1 As shown, Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention.
[0048] like Figure 1 As shown, the network data distribution system may include: a processor 1001, such as a CPU; a user interface 1003; a network interface 1004; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0049] Those skilled in the art will understand that Figure 1 The device structure shown does not constitute a limitation on the device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0050] like Figure 1As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a network data distribution program.
[0051] exist Figure 1 In the device shown, the network interface 1004 is mainly used to connect to the backend server and communicate with the backend server; the user interface 1003 is mainly used to connect to the client (programmer's end) and communicate with the client; and the processor 1001 can be used to call the network data distribution program stored in the memory 1005 and execute the operations in the following network data distribution method.
[0052] Based on the above hardware structure, an embodiment of the network data distribution method of the present invention is proposed.
[0053] Reference Figure 2 , Figure 2 This is a flowchart illustrating a first embodiment of the network data distribution method of the present invention. The network data distribution method includes:
[0054] Step S10: Obtain the target data to be distributed, and select an initial distribution node from each network node of the pre-constructed target distributed network, wherein the target distributed network maintains routing table information of the whole network binary tree jointly mapped by each network node;
[0055] In this embodiment, the present invention is applied to a computer network. The target data refers to the data that the server needs to send to the client. This can be either the client actively requesting data from the server, or the server actively pushing data to the client under certain scenarios. The target distributed network refers to a server cluster distributed network constructed by all network nodes participating in the communication. The initial distribution node refers to the network node in the target distributed network that performs data distribution first. The full-network binary tree refers to a binary tree composed of common mappings of every network node in the target distributed network. Each network node is mapped to a leaf node in the full-network binary tree, and this binary tree can actually be regarded as a routing table identifying all network nodes.
[0056] First, the network nodes participating in the communication need to be constructed into a target distributed network (hereinafter referred to as the target network). Then, when a client actively requests certain data from the target network, or when the target network currently has a data distribution task, the data to be distributed is obtained as the aforementioned target data according to the client's request or the instructions in the data distribution task. The target network can select an initial network node from the entire network of network nodes as the initial distribution node according to preset selection rules, and start the data distribution process of the entire network from this point. The preset selection rules can be to randomly select any network node as the initial distribution node, or to select the corresponding node in the network as the initial distribution node according to the request or the instructions in the task, etc.
[0057] Step S20: Distribute the target data and the routing table information to the initial distribution node. On the initial distribution node, split the routing table information according to its own perspective and the preset subtree splitting rules to obtain several split subtrees, and select several specific batch nodes corresponding to the split subtrees.
[0058] In this embodiment, the preset subtree splitting rule can be set according to the position of the initial distribution node in the entire binary tree. The split subtree is the result of splitting the entire binary tree, and there are usually multiple subtrees. Specific batch nodes refer to extracting a leaf node from each split subtree according to a certain rule, and using the network nodes corresponding to these leaf nodes as multiple specific batch nodes. The specific extraction rule could be selecting the leftmost leaf node in each split subtree, or selecting the rightmost leaf node in each split subtree, etc.
[0059] The bootstrap server in the target network distributes the target data and routing table information to the initial distribution node. Upon receiving the target data and routing table information, the initial distribution node uses the routing table information to split the entire binary tree from its own perspective, resulting in multiple subtrees. After splitting into multiple subtrees, the initial distribution node extracts leaf nodes for specific batches and then uses the network nodes corresponding to these leaf nodes as nodes for those specific batches.
[0060] Step S30: The target data and the routing table information are distributed to the specific batch nodes through the initial distribution node, so that the target data and the routing table information are iteratively forwarded in the target distributed network by the specific batch nodes until the network nodes of the entire network receive the target data.
[0061] In this embodiment, after determining the multiple specific batches of nodes to be distributed to in the next step, the initial distribution node continues to distribute the target data and routing table information to these specific batches of nodes. After receiving the target data and routing table information distributed by the initial distribution node, the specific batches of nodes further split the entire network binary tree from their own node perspective according to the routing table information. This process is iterated, transmitting the target data and routing table information sequentially until every network node in the target network has received the target data.
[0062] This embodiment provides a network data distribution method. When network data distribution is needed, the method first determines an initial distribution node in the network. Then, using this initial distribution node, a binary tree covering all nodes in the network is subtree-splittered to extract a specific batch of nodes. The target data is then passed to these specific batch of nodes, allowing them to iteratively forward the data according to the network's binary tree routing table. This means the system initially only needs to distribute data to one node in the network; the data distribution task for the remaining nodes is completed collaboratively by all nodes in the network, eliminating the need to establish connections sequentially with each node. Therefore, when the number of nodes in the network is large, the pressure on the data server is reduced. By iteratively forwarding the target data to every node in the network through specific batches of nodes, each node acts as a data forwarder. Compared to the method of a server distributing data to all network nodes, this reduces the pressure on the server to maintain network connections and improves data forwarding efficiency. This solves the technical problem of low efficiency in server-to-network node distribution in networks with a large number of nodes.
[0063] Furthermore, based on the above Figure 2 The first embodiment shown presents a second embodiment of the network data distribution method of the present invention. In this embodiment, step S20, which involves splitting the routing table information at the initial distribution node according to its own perspective and a preset subtree splitting rule to obtain several split subtrees, includes:
[0064] Step S201: From the perspective of the initial distribution node itself, starting from the root node of the entire binary tree, split the subtrees that do not contain itself along the direction of increasing node hierarchy, and form multiple split subtrees.
[0065] In this embodiment, as Figure 3As shown in the diagram. Assume the entire binary tree is mapped from 16 network nodes (labeled 0000, 0001, 0010, 0011, 0100, 0101, 0110, 0111, 1000, 1001, 1010, 1011, 1100, 1101, 1110, and 1111), with the initial distribution node mapped to node 1110 in the diagram. The splitting rule is: starting from the root node, split off the subtrees that do not contain itself, then split off the subtrees that do not contain itself from the remaining subtrees, until only the root node remains. After the splitting is complete, N subtrees will be obtained. From the perspective of node 1110 itself, we can divide it into four subtrees as shown in the dashed box in the figure. These four subtrees are: the first subtree constructed by nodes labeled 0000, 0001, 0010, 0011, 0100, 0101, 0110, and 0111; the second subtree constructed by nodes labeled 1000, 1001, 1010, and 1011; the third subtree constructed by nodes labeled 1100 and 1101; and the fourth subtree constructed by a single node labeled 1111.
[0066] This embodiment facilitates the selection of nodes for specific batches by performing subtree splitting operations.
[0067] Further, step S30, which involves using the specific batch of nodes to iteratively forward the target data and the routing table information within the target distributed network until all network nodes receive the target data, includes:
[0068] Step S301: From the perspective of the specific batch of nodes, starting from the root node of the entire binary tree, split the subtrees that do not contain themselves along the direction of increasing node hierarchy, and use them as multiple iteratively split subtrees.
[0069] Step S302: Select nodes of a specific batch from the iterative split subtree according to the routing table information on the specific batch nodes;
[0070] Step S303: The network nodes corresponding to the nodes of the specific batch are used as iterative specific batch nodes, and the target data and the routing table information are distributed to the iterative specific batch nodes through the specific batch nodes;
[0071] Step S304: Based on the iteration of specific batch nodes, perform the step of splitting out the subtrees that do not contain themselves from the perspective of the specific batch nodes themselves, starting from the root node of the entire network binary tree and proceeding in the direction of increasing node hierarchy, until the network nodes of the entire network receive the target data.
[0072] In this embodiment, it is still based on Figure 3For example, after receiving data from the initial distribution node D (1100), node 0000, one of the nodes in a specific batch, also follows the preset subtree splitting principle and splits the entire network binary tree routing table from its own perspective. It extracts the leaf nodes of the specific batch from the resulting subtrees, obtaining the network nodes corresponding to the leaf nodes labeled 1000, 0100, 0010, and 0001, and then forwards data to these iterative specific batch nodes. Node 0000 begins forwarding data sequentially to nodes 0100, 0010, and 0001, along with a forwarding node list L. The above steps are repeated for the other nodes in the specific batch (1000, 1110, and 1101), as follows... Figure 4 As shown, Figure 4 The diagram illustrates the data distribution process for one iteration. The Bootstrap Server in the diagram is the aforementioned bootstrap server. Through multiple rounds of iterative recursive processing, the process continues until all nodes in the network have received the target data.
[0073] This embodiment achieves the effect of distributing data in the node network through iterative forwarding steps. Since all nodes in the network are data forwarding nodes, compared with the previous scheme where only the server actively pushes data to all nodes in the network, the pressure on the server to maintain network connections is reduced.
[0074] Further, step S30, which involves distributing the target data and the routing table information to the specific batch nodes through the initial distribution node, includes:
[0075] Step A1: The target data, the routing table information, and the forwarding node list are distributed to the specific batch of nodes through the initial distribution node, wherein the forwarding node list is a list of network nodes that are currently distributing data, generated by the initial distribution node.
[0076] Step S303 includes:
[0077] Step S3031: On the specific batch node, the iterative batch node is deduplicated using the forwarding node list, and the deduplicated iterative specific batch node is added to the forwarding node list;
[0078] Step S3032: Distribute the target data, the routing table information, and the forwarding node list to the deduplicated iterative specific batch nodes through the specific batch nodes.
[0079] In this embodiment, following the previous embodiment, node D (1100) begins to forward data sequentially to the four nodes that need to forward data. While forwarding data, it also attaches a list L of nodes that have already forwarded data. At this moment, the nodes in L include node D (1100) which is currently sending data, as well as 0000, 1000, 1100, and 1101. After receiving the data from node D (1100), node 0000, according to the preset subtree splitting principle, splits the entire network binary tree routing table R from its own perspective. It extracts the leftmost leaf node of each subtree, obtaining 1000, 0100, 0010, and 0001. Then, it removes duplicates from the forwarding node list L attached to node D (1100), filtering out node 1000, leaving nodes 0100, 0010, and 0001. It then forwards data to these three nodes and appends nodes 0100, 0010, and 0001 to the forwarding node list L. Starting from node 0000, data is forwarded sequentially to nodes 0100, 0010, and 0001, along with a list of forwarding nodes L.
[0080] like Figure 5 As shown, taking a cluster of 16 nodes as an example, each network node splits the data into subtrees according to the binary tree routing table, and filters and adds data to the forwarding node list. The flowchart is shown below: Solid lines represent data flow paths. Dashed lines represent data that, after being filtered by the forwarding list L, does not need to be forwarded to the nodes after the split subtree.
[0081] Furthermore, step S20, which involves selecting specific batch nodes corresponding to several of the split subtrees, includes:
[0082] Step S202: Extract the network node corresponding to the leftmost leaf node in each of the split subtrees as the specific batch node.
[0083] In this embodiment, the nodes in a specific batch are specifically defined as the network nodes corresponding to the leftmost leaf node in each subtree. For example... Figure 3 As shown. The leftmost leaf nodes corresponding to node 1100 are nodes 0000, 1000, 1110, and 1101; while the leftmost leaf nodes corresponding to node 0000 are nodes 1000, 0100, 0010, and 0001; the leftmost leaf nodes corresponding to node 1000 are nodes 0000, 1100, 1010, and 1001; the leftmost leaf nodes corresponding to node 1100 are nodes 0000, 1000, 1100, and 1111; and the leftmost leaf nodes corresponding to node 1101 are nodes 0000, 1000, 1110, and 1100. Subsequent iterations follow the same pattern and will not be elaborated further.
[0084] This embodiment achieves the characteristic of finding the nearest node and distributing data in the node network by adding the steps of extracting the leftmost leaf node of each subtree as the node for forwarding data and filtering the forwarding nodes on the basis of the distributed network.
[0085] Furthermore, based on the above Figure 2 The first embodiment shown presents a third embodiment of the network data distribution method of the present invention. In this embodiment, step S10, selecting an initial distribution node from the network nodes of the pre-constructed target distributed network, includes:
[0086] Step A1: Determine the network nodes that are mapped as leaf nodes in the entire binary tree, and select any one of the network nodes mapped as leaf nodes as the initial distribution node.
[0087] In this embodiment, the initial distribution node can be determined by the guiding server selecting any node from all leaf nodes in the entire binary tree and using the network node corresponding to that node as the initial distribution node.
[0088] Furthermore, prior to step S10, the following steps are also included:
[0089] Step S01: Obtain the network join request sent by the network node to be joined, and obtain the node ID of the network node to be joined from the request;
[0090] Step S02: Map the network node to be added to the network to a leaf node in the initial binary tree according to the preset binary tree mapping rules;
[0091] Step S03: Add the network node to be added to the preset initial distributed network, and add the node ID to the initial routing table information of the initial binary tree maintained by the initial distributed network, so as to construct the initial distributed network into the target distributed network.
[0092] In this embodiment, the distributed network can specifically be a Distributed HashTable (DHT) network. As a specific implementation, a DHT network can be formed using the Kademlia algorithm. Kademlia is one of the protocol implementation schemes of DHT technology. In a DHT network based on the Kademlia algorithm (hereinafter referred to as a DHT network), the nodes participating in communication form a virtual overlay network on top of the User Datagram Protocol (UDP). The Kademlia algorithm specifies the method of node query and information exchange in the network. In the DHT network, the nodes participating in communication form a virtual overlay network on top of UDP. Each node is identified by a 160-bit (N=160) node ID, also known as the NODE ID. Typically, the node's IP address and other information are used to calculate the NODE ID using the SHA1 algorithm. Each node in the network can be mapped to a leaf node in a binary tree using the NODE ID, and each leaf node in the mapped binary tree represents a certain node in the network.
[0093] The initial network cluster will have a Bootstrap Server as the bootstrap server. When a client wants to join the DHT network, the following steps will occur:
[0094] Step 1: Network node A initializes its own node's NODE ID according to the defined NODE ID calculation rules, and then sends a request to join the network to the Bootstrap Server.
[0095] Step 2: After receiving A's request to join the network, the Bootstrap Server maps A to a leaf node in the binary tree according to the mapping rules of the Kademlia algorithm binary tree mentioned above, and then adds A's NODE ID to the maintained binary tree routing table information.
[0096] Step 3: When network node B wants to join the DHT network, repeat the above two steps in the same way.
[0097] Step 4: Once all network nodes have joined the DHT network, the Bootstrap Server will have a complete binary tree routing table information R for the entire network.
[0098] Existing DHT networks based on the Kademlia algorithm typically use XOR operations to calculate distances between nodes to find specific target nodes. This invention differs from previous methods for finding target nodes. Instead, it utilizes Kademlia's subtree partitioning rules to add a step of extracting a batch of nodes and iteratively forwarding data. This achieves data distribution within the node network, allowing the routing server to distribute data to only one node in the network when sending data. The data distribution task for the remaining nodes is completed collaboratively by all nodes in the network, eliminating the need to establish connections with each node sequentially. When the number of nodes in the network is large, this reduces the pressure on the data-distributing server. During data forwarding, because forwarding is based on the network's binary tree routing table, each node in the network acts as a data forwarder, improving data forwarding efficiency compared to a server distributing data to all network nodes.
[0099] like Figure 6 As shown, the present invention also provides a network data distribution system, the network data distribution system comprising:
[0100] The initial node determination module 10 is used to obtain the target data to be distributed and select an initial distribution node from each network node of the pre-constructed target distributed network, wherein the target distributed network maintains routing table information of the whole network binary tree jointly mapped by each of the network nodes;
[0101] The specific node selection module 20 is used to distribute the target data and the routing table information to the initial distribution node, split the routing table information on the initial distribution node according to its own perspective and preset subtree splitting rules to obtain several split subtrees, and select several specific batch nodes corresponding to the split subtrees.
[0102] The data iterative distribution module 30 is used to distribute the target data and the routing table information to the specific batch nodes through the initial distribution node, so as to use the specific batch nodes to iteratively forward the target data and the routing table information in the target distributed network until all network nodes receive the target data.
[0103] Optionally, the specific node selection module 20 includes:
[0104] The subtree splitting unit is used to split the subtrees that do not contain itself from the root node of the entire binary tree from the perspective of the initial distribution node itself, and in the direction of increasing node hierarchy, as multiple split subtrees.
[0105] Optionally, the data iteration and distribution module 30 includes:
[0106] The iterative subtree splitting unit is used to split the subtrees that do not contain themselves from the perspective of the nodes in the specific batch, starting from the root node of the entire binary tree and proceeding along the direction of increasing node hierarchy, as multiple iteratively split subtrees.
[0107] A specific node selection unit is used to select a specific batch of nodes from the iteratively split subtree according to the routing table information on the specific batch of nodes;
[0108] A data information distribution unit is used to use the network nodes corresponding to the nodes of the specific batch as iterative specific batch nodes, and distribute the target data and the routing table information to the iterative specific batch nodes through the specific batch nodes;
[0109] The step iteration execution unit is used to execute, based on the iteration specific batch of nodes, steps that split off subtrees that do not contain themselves from the perspective of the specific batch of nodes, starting from the root node of the entire network binary tree and proceeding in the direction of increasing node hierarchy, until the network nodes of the entire network receive the target data.
[0110] Optionally, the data iteration distribution module 30 further includes:
[0111] A forwarding list distribution unit is used to distribute the target data, the routing table information, and the forwarding node list to the specific batch of nodes through the initial distribution node, wherein the forwarding node list is a list of network nodes that are currently distributing data, generated by the initial distribution node.
[0112] The data information distribution unit is also used for:
[0113] On the specific batch node, the iterative batch node is deduplicated using the forwarding node list, and the deduplicated iterative specific batch node is added to the forwarding node list;
[0114] The target data, the routing table information, and the list of forwarding nodes are distributed to the deduplicated iterative specific batch nodes through the specific batch nodes.
[0115] Optionally, the specific node selection module 20 includes:
[0116] The leftmost node extraction unit is used to extract the network node corresponding to the leftmost leaf node in each of the split subtrees as the specific batch nodes.
[0117] Optionally, the initial node determination module 10 includes:
[0118] An initial node selection unit is used to determine the network nodes that are mapped as leaf nodes in the whole network binary tree, and select any one of the network nodes mapped as leaf nodes as the initial distribution node.
[0119] Optionally, the network data distribution system further includes:
[0120] The node ID acquisition module is used to acquire network join requests sent by network nodes to be joined, and to acquire the node ID of the network nodes to be joined from the requests.
[0121] The leaf node mapping module is used to map the network node to be added to the initial binary tree as a leaf node according to the preset binary tree mapping rules.
[0122] The target network construction module is used to add the network node to be added to a preset initial distributed network, and add the node ID to the initial routing table information of the initial binary tree maintained by the initial distributed network, so as to construct the initial distributed network into the target distributed network.
[0123] The present invention also provides a network data distribution device.
[0124] The network data distribution device includes a processor, a memory, and a network data distribution program stored in the memory and executable on the processor, wherein when the network data distribution program is executed by the processor, it implements the steps of the network data distribution method as described above.
[0125] The method implemented when the network data distribution program is executed can be referred to in various embodiments of the network data distribution method of the present invention, and will not be repeated here.
[0126] The present invention also provides a computer-readable storage medium.
[0127] The present invention provides a computer-readable storage medium storing a network data distribution program, which, when executed by a processor, implements the steps of the network data distribution method described above.
[0128] The method implemented when the network data distribution program is executed can be referred to in various embodiments of the network data distribution method of the present invention, and will not be repeated here.
[0129] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the network data distribution method described above.
[0130] The method implemented when the computer program is executed can be referred to in various embodiments of the network data distribution method of the present invention, and will not be repeated here.
[0131] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0132] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0133] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0134] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.
Claims
1. A network data distribution method, characterized in that, The network data distribution method includes: Obtain the target data to be distributed, and select an initial distribution node from each network node of the pre-constructed target distributed network, wherein the target distributed network maintains routing table information of the whole network binary tree jointly mapped by each of the network nodes; The target data and the routing table information are distributed to the initial distribution node. The routing table information is split on the initial distribution node according to its own perspective and the preset subtree splitting rules to obtain several split subtrees. Several specific batch nodes corresponding to several split subtrees are selected. The selection of several specific batch nodes corresponding to several split subtrees includes: extracting the network node corresponding to the leftmost leaf node in each split subtree as the specific batch node. The target data and routing table information are distributed to the specific batch nodes through the initial distribution node, so that the specific batch nodes can iteratively forward the target data and routing table information in the target distributed network until all network nodes receive the target data; The step of distributing the target data and routing table information to the specific batch nodes through the initial distribution node includes: distributing the target data, routing table information, and a forwarding node list to the specific batch nodes through the initial distribution node, so as to distribute the target data and routing table information to the iterative specific batch nodes through the specific batch nodes, wherein the forwarding node list is a list of network nodes that are currently distributing data, generated by the initial distribution node; the step of distributing the target data and routing table information to the iterative specific batch nodes through the specific batch nodes includes: deduplicating the iterative batch nodes using the forwarding node list on the specific batch nodes, and adding the deduplicated iterative specific batch nodes to the forwarding node list; and distributing the target data, routing table information, and forwarding node list to the deduplicated iterative specific batch nodes through the specific batch nodes.
2. The network data distribution method as described in claim 1, characterized in that, The step of splitting the routing table information on the initial distribution node according to its own perspective and preset subtree splitting rules to obtain several split subtrees includes: From the perspective of the initial distribution node itself, starting from the root node of the entire binary tree, the subtrees that do not contain itself are split off along the direction of increasing node hierarchy, and these are the multiple split subtrees.
3. The network data distribution method as described in claim 2, characterized in that, The step of using the specific batch of nodes to iteratively forward the target data and the routing table information in the target distributed network until all network nodes receive the target data includes: From the perspective of the specific batch of nodes, starting from the root node of the entire binary tree, the subtrees that do not contain themselves are split off along the direction of increasing node hierarchy, and these are used as multiple iteratively split subtrees. Select nodes from the iteratively split subtree on the specific batch of nodes according to the routing table information; The network nodes corresponding to the nodes in the specific batch are used as the iteration specific batch nodes, and the target data and the routing table information are distributed to the iteration specific batch nodes through the specific batch nodes; Based on the iteration of specific batch nodes, from the perspective of the specific batch node itself, the steps of splitting out the subtrees that do not contain itself in the direction of increasing node hierarchy are performed starting from the root node of the entire network binary tree, until the network nodes of the entire network receive the target data.
4. The network data distribution method as described in claim 1, characterized in that, The step of selecting an initial distribution node from the network nodes of the pre-built target distributed network includes: Determine the network nodes that are mapped as leaf nodes in the entire binary tree, and select any one of the network nodes mapped as leaf nodes as the initial distribution node.
5. The network data distribution method according to any one of claims 1-4, characterized in that, Before the step of obtaining the target data to be distributed, the method further includes: Obtain the network join request sent by the network node to be joined, and obtain the node ID of the network node to be joined from the request; The nodes to be added to the network are mapped to leaf nodes in the initial binary tree according to the preset binary tree mapping rules; The node to be added to the network is added to a preset initial distributed network, and the node ID is added to the initial routing table information of the initial binary tree maintained by the initial distributed network, so as to construct the initial distributed network into the target distributed network.
6. A network data distribution system, characterized in that, The network data distribution system includes: The initial node determination module is used to obtain the target data to be distributed and select an initial distribution node from each network node of the pre-constructed target distributed network, wherein the target distributed network maintains routing table information of the whole network binary tree jointly mapped by each of the network nodes; A specific node selection module is used to distribute the target data and the routing table information to the initial distribution node, split the routing table information on the initial distribution node according to its own perspective and a preset subtree splitting rule to obtain several split subtrees, and select specific batch nodes corresponding to several of the split subtrees; wherein, the selection of specific batch nodes corresponding to several of the split subtrees includes: extracting the network node corresponding to the leftmost leaf node in each split subtree as the specific batch node; A data iterative distribution module is used to distribute the target data and the routing table information to the specific batch nodes through the initial distribution node, so that the target data and the routing table information are iteratively forwarded in the target distributed network by the specific batch nodes until all network nodes receive the target data; wherein, the distribution of the target data and the routing table information to the specific batch nodes through the initial distribution node includes: distributing the target data, the routing table information, and a forwarding node list to the specific batch nodes through the initial distribution node, so that the target data and the routing table information are distributed to the iterative specific batch nodes through the specific batch nodes, wherein the forwarding node list is a list of network nodes that are currently distributing data, generated by the initial distribution node; the distribution of the target data and the routing table information to the iterative specific batch nodes through the specific batch nodes includes: deduplicating the iterative batch nodes on the specific batch nodes using the forwarding node list, and adding the deduplicated iterative specific batch nodes to the forwarding node list; and distributing the target data, the routing table information, and the forwarding node list to the deduplicated iterative specific batch nodes through the specific batch nodes.
7. A network data distribution device, characterized in that, The network data distribution device includes: a memory, a processor, and a network data distribution program stored in the memory and executable on the processor, wherein when the network data distribution program is executed by the processor, it implements the steps of the network data distribution method as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a network data distribution program, which, when executed by a processor, implements the steps of the network data distribution method as described in any one of claims 1 to 5.