Dynamic networking system based on hierarchical network structure and topology management method thereof

Abstract

The invention discloses a dynamic networking system based on a hierarchical network structure and a topology management method thereof. The system comprises a primary network construction module and a secondary network construction module, the primary network construction module is used for constructing a primary mesh network as a collaborative network for the selected data center node by adopting a decentralized mesh network topological structure, so that metadata of the data center node can be spread in the collaborative network, and a user can check the health state and metadata information of the node; the secondary network construction module is used for constructing a tree network by taking each data center node as a root node and realizing hierarchical management on nodes for executing collaborative tasks in the data center nodes, and a father node in the tree network has metadata of all child nodes of the father node; the data center node and the node executing the cooperative task are communicated through a bidirectional SSL protocol, and node identity credible authentication and metadata synchronization are achieved. According to the invention, dynamic credible verification of node identities can be realized.

Description

Technical Field

[0001] This invention relates to the fields of distributed networks, dynamic networking, authentication, and topology management, and in particular to a hierarchical dynamic networking system and its management method that supports collaborative processing task nodes. Background Technology

[0002] As the demand for collaborative processing tasks continues to grow, the reliable and secure transmission of data between different nodes, the dynamic adjustment of the collaborative network topology, and the real-time reliability of node identities have become key challenges in system design.

[0003] Traditional networking solutions typically rely on fixed network structures and node configurations. For centralized networking, this reliance on a leader node makes it difficult to adapt to dynamic topology changes and real-time metadata updates, requiring maintainers to manually modify configurations, resulting in low efficiency. Furthermore, traditional methods for node authentication rely on one-way authentication or static keys, posing security risks and exhibiting low efficiency. In addition, traditional networking methods lack hierarchical subnet management capabilities. A single-level structure limits the flexible organization of nodes within a subnet and metadata synchronization, making it difficult to adapt to complex hierarchical structures. Especially in large-scale collaborative computing scenarios, different nodes have varying requirements for data acquisition and propagation; a single-level network topology leads to low information update efficiency and is prone to bottlenecks. Summary of the Invention

[0004] To address the aforementioned issues, this application aims to provide a dynamic networking system based on a hierarchical network structure and its topology management method, which can achieve real-time maintenance of topology and metadata under different network environments and network topologies, and realize dynamic and reliable verification of node identities.

[0005] This invention provides a dynamic networking system based on a hierarchical network structure, characterized in that it includes a first-level network construction module and a second-level network construction module; The primary network construction module is used to construct a primary mesh network as a collaborative network for the selected data center nodes using a decentralized mesh network topology. This network is used to propagate the metadata of the data center nodes within the collaborative network and to allow users to view the health status and metadata information of the data center nodes and nodes performing collaborative tasks in the collaborative network in real time. Each data center node includes several nodes, namely sub-data center nodes. The secondary network construction module is used to construct a tree network for the nodes performing collaborative tasks in each data center node of the primary mesh network as the root node, thereby realizing hierarchical management of the nodes performing collaborative tasks in the data center node. The parent node in the tree network has the metadata of all its child nodes. The data center node and the node performing collaborative tasks communicate using a two-way SSL protocol to achieve trusted authentication of node identity and synchronization of metadata.

[0006] Preferably, each of the data center nodes loads the same dynamic encryption key as a gossip encryption key upon startup, which is used for encrypted communication with other data center nodes; the gossip encryption key is issued to the corresponding data center node after the authentication center verifies the data center node's identity.

[0007] Preferably, the nodes in the secondary tree network authenticate each other's identities and ensure trusted communication through authentication certificates and bidirectional SSL protocols.

[0008] Preferably, in the primary mesh network, data center nodes automatically elect a leader node using the Raft consensus protocol, with other data center nodes acting as follower nodes. The leader node is responsible for handling data write operations, recording write requests as log entries in its local log and replicating the recorded logs to each follower node. When a new data center node joins, the leader node designates it as a new follower node and sends the recorded logs to the new follower node, ensuring its state is synchronized with the leader node and all other follower nodes.

[0009] Preferably, nodes in the tree network periodically synchronize metadata and child node lists to their direct parent nodes and propagate them level by level to maintain the topology of the tree network.

[0010] This invention provides a topology management method for a dynamic networking system based on a hierarchical network structure, the steps of which include: 1) Select multiple data center nodes to perform the target collaborative task. For the selected data center nodes, a decentralized mesh network topology is used to construct a first-level mesh network as a collaborative network. This network is used to propagate the metadata of the data center nodes in the collaborative network and to allow users to view the health status and metadata information of the data center nodes and the nodes performing the collaborative task in the collaborative network in real time. Each data center node includes several nodes, namely sub-data center nodes. 2) Perform trusted authentication on the identity information of newly added data center nodes in the target collaborative task. After successful authentication, allow them to join the primary mesh network and dynamically perceive the resources and topology of the primary mesh network; 3) Using each data center node of the first-level mesh network as the root node, a tree network is constructed for the nodes in the data center node that perform collaborative tasks, so as to realize hierarchical management of the nodes in the data center node that perform collaborative tasks. The parent node in the tree network has the metadata of all its child nodes. The data center node and the node performing collaborative tasks communicate with each other using a two-way SSL protocol to realize trusted authentication of node identity and synchronization of metadata.

[0011] This invention is a layered, architecture-based dynamic networking system that supports real-time maintenance of the network topology when nodes dynamically join or leave. The system achieves trusted authentication of data center node identities and synchronization of metadata through a two-layer distributed network architecture and a bidirectional SSL protocol. The specific contents of this invention include...

[0012] 1. First-level mesh network structure: This implements a decentralized mesh network topology, allowing metadata from data center nodes to propagate throughout the network. Data center researchers can view the health status and metadata of data center nodes and nodes performing collaborative tasks in the collaborative network in real time; users can search for the data sources and collaborative model algorithms they need within the collaborative network according to their specific requirements.

[0013] 2. Two-level tree structure: The data center node of the first-level network can be further expanded into a root node, forming a two-level tree network with it as the root, realizing hierarchical management of nodes that perform collaborative tasks. The parent node has the metadata of all its child nodes, and the root node of the tree, that is, the data center node in the first-level network, maintains all the metadata of the subnet.

[0014] 3. Data Center Node and Collaborative Task Authentication Mechanism: In a primary mesh network, all data center nodes must load the same gossip encryption key upon startup. Only data center nodes using the same encryption key can join the mesh network and communicate with other data center nodes via encrypted gossip. Depending on the data center node's level of trust in other data center nodes and the communication channel, this key can be configured by the system administrator in each data center or requested from a unified authentication service center. The authentication center then authenticates the data center nodes and issues the key. In a secondary tree-structured network, nodes mutually authenticate each other's identities and ensure trusted communication through authentication certificates and a two-way SSL protocol.

[0015] 1. Metadata synchronization mechanism: (1) In a Tier 1 mesh network, automatic leader election is performed using the Raft consensus protocol. The leader node is responsible for handling data write operations, that is, recording write requests (write or update) in the form of log entries in the local log and replicating the log to other nodes (followers) in the cluster. When a new data center node joins, the leader node sends all entries of the current log to the new data center node to keep its state synchronized with the leader and other data center nodes in the network, thus achieving the effect of data replication.

[0016] (2) The two-level tree network maintains the topology of the tree network by periodically synchronizing metadata and child node list to its direct parent node and propagating them level by level, thus ensuring efficient propagation and updates.

[0017] Furthermore, the above method first involves all data center nodes and nodes performing collaborative tasks in the mesh network maintaining and propagating the topology and synchronization metadata of the primary mesh network. The collaborative network is defined as consisting of five data center nodes: DataCenter-A, DataCenter-B, DataCenter-C, DataCenter-D, and DataCenter-E. When a new data center node, DataCenter-F, joins, it carries metadata (MetaData). Based on the networking mechanism of the primary mesh network, the operation of joining the new data center node and the propagation of metadata are completed. Using a dynamic encryption key (encryptkey), specifically a gossip encryption key, mesh network nodes holding the gossip key perform mutual authentication and information encryption between identity nodes. Data center nodes maintain mutual health status information and membership relationships through mutual "gossip." Using a decentralized approach, each data center node communicates only with a few neighboring data center nodes, thereby propagating identity information and metadata throughout the entire network.

[0018] Furthermore, the above method maintains and propagates the topology and synchronization metadata of the secondary tree network. Assuming the primary mesh network contains the five data center nodes DataCenterA-E, each DataCenter has two child nodes SubNetNode1 and SubNetNode2, SubNetNode1 has two child nodes SubNetNode1-1 and SubNetNode1-2, and SubNetNode2 has two child nodes SubNetNode2-1 and SubNetNode2-2. Based on the secondary network's networking mechanism, the operation of adding new nodes to the collaborative network and the propagation of metadata are completed. When a child node joins the network, each node in the tree network maintains the metadata of its direct child node (i.e., the child node) and periodically synchronizes it to its direct parent data center node (i.e., the parent node). This optimizes network load, minimizes the transmission path of metadata in the network, and avoids transmission failures or the generation of dirty data caused by network fluctuations and other factors.

[0019] Furthermore, the primary mesh network uses a Gossip encryption key for trusted communication. This key is a 16-byte encryption key generated using the AES-256-GCM algorithm. During communication, each Gossip message includes a checksum to ensure that the message has not been tampered with during transmission. Data center nodes in the network need to load the same Gossip encryption key upon startup. The system supports two encryption key management mechanisms. For scenarios where nodes in the network trust the communication channel but do not trust the identity of newly joined data center nodes, the administrator will set the key in the startup parameters to verify the identity of the data center node. For scenarios where data center nodes in the network neither trust the communication channel nor the identity of new data center nodes, the system allows data center nodes to dynamically obtain encryption keys from the unified authentication service center when joining the network. The authentication center authenticates the identity of newly joined data center nodes through a two-way SSL protocol and issues a Gossip encryption key. At the same time, it maintains a list of data center nodes in the collaborative network. When a new data center node communicates with data center nodes in the collaborative network, the data center nodes in the network must first verify the validity of the new data center node's identity with the unified authentication service center to avoid key leakage and other problems caused by untrusted channels.

[0020] Furthermore, dynamic authentication is achieved between nodes in the secondary tree network using a two-way SSL protocol. The two-way SSL protocol ensures that both parties verify each other's server certificates, which can be self-signed or issued by an authorized CA. After successful authentication, a symmetric encryption key recognized by both parties is generated. All transmitted data is symmetrically encrypted using the key to ensure the confidentiality and integrity of communication.

[0021] The beneficial effects of this invention are as follows: This invention breaks through the limitations of traditional single-layer networking, can adapt to collaborative processing scenarios of different scales and levels, achieve consistency and high availability of network-wide metadata, and avoids the overhead and latency caused by network-wide broadcasting through hierarchical metadata synchronization between first-level and second-level network nodes. Attached Figure Description

[0022] Figure 1 This diagram illustrates how newly added Tier 1 network data center nodes obtain keys and authentication information from the certification authority.

[0023] Figure 2 This is a schematic diagram of the topology maintenance of a primary network data center node.

[0024] Figure 3 This is a flowchart of the two-way identity authentication process for secondary network nodes.

[0025] Figure 4 This is a diagram illustrating the maintenance of a secondary network topology.

[0026] Figure 5 This is a diagram illustrating the flow of secondary network metadata.

[0027] Figure 6 This is a diagram illustrating the primary network election and metadata synchronization. Detailed Implementation

[0028] The present invention will now be described in further detail with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0029] This embodiment provides a dynamic networking system based on a hierarchical network structure and its topology management method, as detailed below.

[0030] 1) Based on the multiple data centers currently performing the target collaborative task, organize these data center nodes into a primary network structure; 2) Based on the identity information of the newly added target collaborative task's primary network data center node, perform trusted authentication, obtain network access permission, implement the addition of the data center node to the primary network, and dynamically perceive network resources and topology. 3) Based on the scenario of metadata distribution and synchronization from the primary network data center node, organize a secondary network structure, implement the joining of secondary network nodes for the scenario of joining secondary network nodes, and dynamically perceive the network resource status; 4) Based on the above network structure, realize the propagation and synchronization of metadata in the network.

[0031] The construction process of this method is illustrated using a specific requirement and scenario as an example. The specific requirement and scenario are described as follows: There are four data centers: DataCenterA, DataCenterB, DataCenterC, and DataCenterD. DataCenterD has its own sub-data centers SubNetNode1 and SubNetNode2 within its local area network (LAN). SubNetNode1 and SubNetNode2 each have their own sub-data centers SubNetNode1-1 and SubNetNode2-1 within their respective LANs. Due to LAN limitations, SubNetNode1 can only communicate with DataCenterD and SubNetNode1-1, and SubNetNode2 can only communicate with DataCenterD and SubNetNode2-1.

[0032] Research users publish shared grassland aboveground biomass datasets on DataCenterA, DataCenterB, DataCenterC, SubNetNode1, SubNetNode2, SubNetNode1-1, and SubNetNode2-1, respectively. The metadata of the datasets are Metadata1, Metadata2, Metadata3, Metadata4-part1, Metadata4-part2, Metadata4-part3, and Metadata4-part4, respectively. Parts 1-4 of Metadata4 are non-overlapping subsets of Metadata4. A trusted network needs to be established between the above data center nodes and the nodes performing collaborative tasks. Specifically, DataCenterA, DataCenterB, and DataCenterC should all trust and communicate with the new data center node DataCenterD; DataCenterD should trust and communicate with SubNetNode1 and SubNetNode2 respectively; SubNetNode1 should trust and communicate with SubNetNode1-1; and SubNetNode2 should trust and communicate with SubNetNode1-1. It is also necessary to share the metadata of the grassland biomass dataset Metadata1, Metadata2, Metadata3, and Metadata4 in the middle of the primary network, so that all data center nodes in the primary network can access these datasets.

[0033] First, based on the existing data center nodes DataCenterA, DataCenterB, and DataCenterC, a distributed first-level network structure is constructed, prioritizing physical proximity and trust relationships between the data center nodes. This first-level network creates a decentralized mesh network. Gossip encryption keys are pre-configured in the startup parameters of DataCenterA, DataCenterB, and DataCenterC for trusted communication between data center nodes, ensuring automatic connection and network awareness upon startup. Each data center node periodically sends heartbeat packets to other data center nodes to ensure their online status. If a data center node becomes unreachable or malfunctions, the Gossip protocol can quickly detect this and propagate the status to other data center nodes, enabling dynamic maintenance of the network topology.

[0034] Secondly, when a new data center node, DataCenterD, attempts to join the primary network, it obtains a Gossip key through pre-configured settings by its administrator or by performing two-way SSL authentication with the unified authentication center. After obtaining the Gossip key, it establishes a trusted communication channel with any other primary data center node in the cluster (taking DataCenterA as an example). Upon receiving a communication request from DataCenterD, which possesses the same key as other data center nodes, DataCenterD authenticates with the unified authentication center to ensure the trustworthiness of its identity within the network. Figure 1 Upon successful joining, the network topology is updated. After DataCenterD joins the primary network, it periodically and randomly selects primary data center nodes via the Gossip protocol and transmits information to these nodes. The receiving data center nodes perform the same operation, transmitting this information to other randomly selected data center nodes, and so on. New data center nodes establish communication links with existing data center nodes and gradually synchronize the current network status and health information, such as... Figure 2 The system monitors the health status and network resource changes of each data center node in real time. If network changes are detected, the system will adjust the connection relationships between data center nodes to ensure network stability and reliable data synchronization.

[0035] Then, the data center node DataCenterD in the primary network will serve as the root node of its own secondary tree network, forming a hierarchical structure. This secondary network is managed using a parent-child relationship, with the parent node maintaining the metadata and status information of its direct child nodes. DataCenterD authenticates the identities of SubNetNode1 and SubNetNode2 via a two-way SSL protocol. Taking the authentication process between DataCenterD (the server) and SubNetNode1 (the client) as an example... Figure 3 SubNetNode1 initiates a connection request to DataCenterD, starting the TLS handshake process. DataCenterD responds by sending its server certificate, public key, and other information to SubNetNode1. SubNetNode1 verifies the certificate sent by DataCenterD to ensure its validity and trustworthiness. If the certificate verification is successful, SubNetNode1 generates its own certificate for subsequent communication. SubNetNode1 then sends its certificate to DataCenterD and encrypts a random number using DataCenterD's public key. DataCenterD verifies the received SubNetNode1 certificate to ensure its validity and authenticity and decrypts the encrypted random number sent by SubNetNode1 using its private key. Afterward, both parties begin key negotiation and encrypted communication. A shared session key is generated based on the random number using the ECDH key exchange protocol for encrypted communication. In subsequent communication, SubNetNode1 and DataCenterD use the negotiated session key to encrypt and decrypt data, ensuring secure data transmission. After DataCenterD establishes trusted connections with its lower-level nodes SubNetNode1 and SubNetNode2 according to the above process, it gradually synchronizes the current network status and health information, such as... Figure 4 The same authentication method is used for authentication and network topology organization between SubNetNode1 and SubNetNode1-1, and between SubNetNode2 and SubNetNode2-1. The system monitors the health status and network resource changes of each data center node and node performing collaborative tasks in real time to maintain the stability of the network topology and the reliability of data synchronization.

[0036] Finally, the metadata information Metadata4-part3, Metadata4-part4, and updates registered on the sub-data centers SubNetNode1-1 and SubNetNode2-1 will be periodically synchronized to their parent nodes, SubNetNode1 and SubNetNode2, respectively. SubNetNode1 and SubNetNode2 will integrate the metadata synchronized from their lower-level nodes with their own registered metadata Metadata4-part1 and Metadata4-part2, and simultaneously propagate them upwards level by level, achieving efficient data synchronization and topology maintenance. After synchronization is complete, the root node DataCenterD of the secondary network will have complete Metadata4, such as... Figure 5 Meanwhile, DataCenterD, as a member data center node of the primary network, achieves consensus and data replication through the Raft protocol and propagates metadata through the gossip protocol, ensuring data consistency across all server data center nodes. Existing data center nodes DataCenterA, DataCenterB, and DataCenterC elect a leader through the Raft consensus protocol. Taking DataCenterA's successful election as the leader as an example, when a new data center node DataCenterD joins, it defaults to a follower identity and informs the leader of its metadata (Metadata4) via a write request. The leader records the write request as a log entry in its local log and replicates this log to the other follower nodes in the cluster (DataCenterB and DataCenterC). It also sends all historical log entries to the new data center node DataCenterD, synchronizing the data state of DataCenterB, DataCenterC, and DataCenterD with the leader. Figure 6 After synchronization is complete, all data center nodes will have all the metadata of other data center nodes, namely the grassland aboveground biomass dataset metadata Metadata1, Metadata2, Metadata3, and Metadata4 registered on DataCenterA, DataCenterB, DataCenterC, and DataCenterD respectively.

[0037] Although specific embodiments of the invention have been disclosed for illustrative purposes to aid in understanding and implementing the invention, those skilled in the art will understand that various substitutions, variations, and modifications are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, the invention should not be limited to the content disclosed in the preferred embodiments, and the scope of protection claimed by the invention is defined by the claims.

Claims

1. A dynamic networking system based on a hierarchical network structure, characterized in that, It includes a primary network building module and a secondary network building module; The primary network construction module is used to construct a primary mesh network as a collaborative network for the selected data center nodes using a decentralized mesh network topology. This network is used to propagate the metadata of the data center nodes within the collaborative network and to allow users to view the health status and metadata information of the data center nodes and nodes performing collaborative tasks in the collaborative network in real time. Each data center node includes several nodes, namely sub-data center nodes. The secondary network construction module is used to construct a tree network for the nodes performing collaborative tasks in each data center node of the primary mesh network as the root node, thereby realizing hierarchical management of the nodes performing collaborative tasks in the data center node. The parent node in the tree network has the metadata of all its child nodes. The data center node and the node performing collaborative tasks communicate using a two-way SSL protocol to achieve trusted authentication of node identity and synchronization of metadata.

2. The dynamic networking system according to claim 1, characterized in that, Each data center node loads the same dynamic encryption key as a gossip encryption key upon startup, which is used for encrypted communication with other data center nodes. The gossip encryption key is issued to the corresponding data center node after the authentication center verifies the data center node's identity.

3. The dynamic networking system according to claim 2, characterized in that, The nodes in the two-level tree network authenticate each other's identities and ensure trusted communication through authentication certificates and bidirectional SSL protocols.

4. The dynamic networking system according to claim 1, 2, or 3, characterized in that, In the primary mesh network, data center nodes automatically elect a leader node using the Raft consensus protocol, with other data center nodes acting as follower nodes. The leader node is responsible for handling data write operations, recording write requests as log entries in its local log and replicating the recorded logs to each follower node. When a new data center node joins, the leader node makes it a new follower node and sends the recorded logs to the new follower node to keep its state synchronized with the leader node and all other follower nodes.

5. The dynamic networking system according to claim 1, 2, or 3, characterized in that, In the tree network, nodes periodically synchronize metadata and child node lists with their direct parent nodes, and propagate these information level by level to maintain the topology of the tree network.

6. A topology management method for a dynamic networking system based on a hierarchical network structure, comprising the following steps: 1) Select multiple data center nodes to perform the target collaborative task. For the selected data center nodes, a decentralized mesh network topology is used to construct a first-level mesh network as a collaborative network. This network is used to propagate the metadata of the data center nodes in the collaborative network and to allow users to view the health status and metadata information of the data center nodes and the nodes performing the collaborative task in the collaborative network in real time. Each data center node includes several nodes, namely sub-data center nodes. 2) Perform trusted authentication on the identity information of newly added data center nodes in the target collaborative task. After successful authentication, allow them to join the primary mesh network and dynamically perceive the resources and topology of the primary mesh network; 3) Using each data center node of the first-level mesh network as the root node, a tree network is constructed for the nodes in the data center node that perform collaborative tasks, so as to realize hierarchical management of the nodes in the data center node that perform collaborative tasks. The parent node in the tree network has the metadata of all its child nodes. The data center node and the node performing collaborative tasks communicate with each other using a two-way SSL protocol to realize trusted authentication of node identity and synchronization of metadata.

7. The method according to claim 6, characterized in that, Each data center node loads the same dynamic encryption key as a gossip encryption key upon startup, which is used for encrypted communication with other data center nodes. The gossip encryption key is issued to the corresponding data center node after the authentication center verifies the data center node's identity.

8. The method according to claim 7, characterized in that, The nodes in the two-level tree network authenticate each other's identities and ensure trusted communication through authentication certificates and bidirectional SSL protocols.

9. The method according to claim 6, characterized in that, In the primary mesh network, data center nodes automatically elect a leader node using the Raft consensus protocol, with other data center nodes acting as follower nodes. The leader node is responsible for handling data write operations, recording write requests as log entries in its local log and replicating the recorded logs to each follower node. When a new data center node joins, the leader node makes it a new follower node and sends the recorded logs to the new follower node to keep its state synchronized with the leader node and all other follower nodes.

10. The method according to claim 6, characterized in that, In the tree network, nodes periodically synchronize metadata and child node lists with their direct parent nodes, and propagate these information level by level to maintain the topology of the tree network.

Images