Dynamic generation method for network topological relation graph

Abstract

The invention provides a network topology relation graph dynamic generation method, which belongs to the technical field of network topology, and comprises the following steps: starting an improved Tarjan strong connectivity component algorithm in each subnet partition to carry out loop detection, executing distributed loop verification on cross-subnet boundary nodes, collecting ARP table entries, routing table entries, OpenFlow flow tables and virtualization platform interface data, and sending the ARP table entries, the routing table entries, the OpenFlow flow tables and the virtualization platform interface data to each subnet partition; the method comprises the following steps: performing power spectrum analysis on an optical power monitoring point on optical network equipment to identify a wavelength channel occupancy state, reconstructing a wavelength routing path of a DWDM system, and performing two-stage submission on topological data acquired by distributed detection nodes by adopting a causal consistency protocol to synchronously generate a global unified network topological relation graph; the technical problems that the incremental updating efficiency is low and the cross-layer mapping relation is difficult to reconstruct in real time when the network topology dynamically changes are solved.

Description

Technical Field

[0001] This invention belongs to the field of network topology technology, and more specifically, relates to a method for dynamically generating network topology relationship graphs. Background Technology

[0002] In existing network topology management technologies, traditional methods use SNMP polling or NetFlow analysis to collect network device status data. A centralized server periodically performs a full network scan to build the topology map. This approach is widely used in data center network monitoring and WAN management scenarios. However, existing technologies have significant drawbacks when handling large-scale network topology changes. The full scan mechanism requires re-traversing all devices and links for each topology update, with computational overhead increasing exponentially with network size. Furthermore, the centralized architecture suffers from single-point bottlenecks in distributed network environments, making it difficult to respond promptly to link jitter and sudden device status changes. In current virtualized and optical network environments, the lack of an effective correlation and reasoning mechanism between the physical and virtual layers prevents traditional topology discovery technologies from accurately establishing mappings between physical devices, virtual switches, and container networks. This leads to cross-layer path tracing failures and prolonged fault location time. In other words, existing technologies suffer from low incremental update efficiency and difficulty in real-time reconstruction of cross-layer mappings when network topology changes dynamically. Summary of the Invention

[0003] In view of this, the present invention provides a method for dynamically generating network topology relationship graphs, which can solve the technical problems of low incremental update efficiency and difficulty in real-time reconstruction of cross-layer mapping relationships when the network topology changes dynamically in the prior art.

[0004] This invention is implemented as follows: It provides a method for dynamically generating network topology graphs. Distributed probe nodes are deployed at key network aggregation points to collect information on device online events and link status changes. A Bloom filter is used to quickly identify duplicates of newly discovered device identifiers. When link status changes are detected, an incremental topology update process is triggered. A copy-on-write technique is used to create a snapshot of the current topology graph and assign a version vector identifier. A change subgraph is constructed by extracting the set of changed nodes and the set of changed edges through differential calculation. An optimal subnetting algorithm based on dynamic programming is executed on the change subgraph to allocate network nodes to different subnet partitions. A topology-aware scheduling model is used to calculate the subnetting scheme. Within each subnet partition, an improved Tarjan strongly connected component algorithm is initiated to detect loops and address cross-subnet issues. Boundary nodes extract connection summary information to perform distributed loop verification. When a loop topology is detected, the loop path is marked and the loop impact score is calculated. Based on the score, it is determined whether to perform topology reconstruction. ARP entries, routing entries, OpenFlow flow tables, and virtualization platform interface data are collected to construct a traffic matrix and calculate the latency correlation coefficient between nodes. A multi-layer association inference model is used to establish the mapping relationship between physical devices, virtual switches, and container networks. Power spectrum analysis of optical power monitoring points is performed on optical network devices to identify wavelength channel occupancy status and reconstruct the wavelength routing path of the DWDM system by combining label switching path information. A causal consistency protocol is used to perform a two-phase commit on the topology data collected by distributed probe nodes to synchronously generate a globally unified network topology graph.

[0005] Among them, the distributed probe nodes collect information on device online events and link status changes using a hybrid strategy of SNMP Trap passive listening and active ICMP probe.

[0006] Copy-on-write technology does not directly overwrite the original data when modifying it, but instead creates a copy of the data for modification, while the original data remains unchanged until all references end.

[0007] Among them, differential computation extracts the nodes and edges that have changed by comparing snapshots of the old and new topologies, and performs subsequent processing only on the changed parts.

[0008] Among them, the optimal subnet partitioning algorithm based on dynamic programming models the network partitioning problem as a multi-stage decision process. The state space is the set of allocated nodes, and the state transition corresponds to the operation of assigning a new node to a certain subnet partition. The algorithm defines a value function that accumulates the local cost of each stage. The cost includes the weighted sum of the communication traffic between nodes within the subnet partition and the cross-boundary traffic between subnet partitions. The algorithm uses memoization technology to store the calculated optimal solutions to subproblems to avoid repeated calculations. The backtracking process starts from the final state and generates a mapping scheme from nodes to subnet partitions along the optimal decision path.

[0009] The topology-aware scheduling model adopts a feature fusion computing framework based on multi-head cross-attention. The model input includes the geographic location coordinates of the nodes, link bandwidth capacity, historical traffic statistics and device type identifiers, and the output is the subnet partition affiliation probability distribution of each node.

[0010] The topology-aware scheduling model comprises a three-layer architecture: an input embedding layer, a multi-layer feature fusion layer, and an output decision layer. The input embedding layer maps the geographic location coordinates of nodes into a 128-dimensional spatial vector through a fully connected network, normalizes the link bandwidth capacity and historical traffic statistics into a 64-dimensional vector, and converts the device type identifier into a 32-dimensional vector through one-hot encoding and concatenates them to form a 224-dimensional input feature. The multi-layer feature fusion layer contains four stacked multi-head cross-attention modules. Each multi-head cross-attention module calculates the association weights between different feature dimensions through linear transformations of the query matrix, key matrix, and value matrix.

[0011] The number of attention heads in the multi-head cross-attention module is determined based on the average node degree of the current network. When the average node degree is less than 10, 4 attention heads are set; when the average node degree is between 10 and 30, 8 attention heads are set; and when the average node degree exceeds 30, 16 attention heads are set.

[0012] Among them, the improved Tarjan strongly connected component algorithm adds node access order optimization and stack space compression mechanism on the basis of the standard Tarjan algorithm. It reduces the recursion depth by prioritizing the access of nodes with larger out-degree, and immediately outputs and releases the stack space after detecting a strongly connected component.

[0013] Among them, distributed loop verification is a loop confirmation process executed collaboratively by multiple distributed probe nodes. After exchanging connection summary information, each distributed probe node independently judges whether there is a cross-subnet partition loop and confirms the existence of the loop through a majority voting mechanism.

[0014] This invention employs distributed probe nodes combined with copy-on-write technology and differential computation mechanism. By extracting changed subgraphs and executing incremental topology update processes, it avoids the computational overhead of full reconstruction, significantly improving update efficiency in large-scale network environments. This invention utilizes a dynamic programming-based optimal subnetting algorithm combined with a topology-aware scheduling model to allocate network nodes to different subnet partitions based on communication correlation and traffic characteristics. Within each subnet partition, an improved Tarjan strongly connected component algorithm is independently executed for loop detection. A distributed loop verification mechanism is used for nodes crossing subnet boundaries, overcoming the high computational complexity of traditional global loop detection algorithms in large-scale networks. Simultaneously, a multi-layer association inference model is used to fuse ARP entries, routing entries, OpenFlow flow tables, and virtualization platform interface data to establish cross-layer mapping relationships between physical devices, virtual switches, and container networks, achieving accurate end-to-end path topology reconstruction. In summary, this invention solves the technical problems mentioned in the background art, such as low incremental update efficiency and difficulty in real-time reconstruction of cross-layer mapping relationships when network topology dynamically changes. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method of the present invention.

[0016] Figure 2 A three-dimensional scatter plot showing the node distribution of the subnet.

[0017] Figure 3 The probability distribution radar map is output by the feature fusion layer of the topology-aware scheduling model.

[0018] Figure 4 A bar chart comparing the loop impact scores before and after link weight adjustment.

[0019] Figure 5 A three-dimensional surface plot of the spatiotemporal evolution trajectory of the wavelength channel occupancy state. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below.

[0021] like Figure 1 The diagram shown is a flowchart of a method for dynamically generating network topology graphs provided by this invention. This method includes the following steps:

[0022] A method for dynamically generating network topology graphs includes the following steps:

[0023] S01. Deploy distributed probe nodes at key network aggregation points. Each distributed probe node collects device online events and link status change information through a hybrid strategy of passive listening with SNMP Trap and active ICMP probe. A Bloom filter is used to quickly deduplicat newly discovered device identifiers and record basic device attribute information. The output includes a set of device identifiers, basic device attribute information, and link status change information.

[0024] S02. When the distributed probe node detects a change in link status, it triggers an incremental topology update process. It uses copy-on-write technology to create a snapshot of the current topology graph and assigns a version vector identifier. It extracts the set of changed nodes and the set of changed edges through differential calculation to form a changed subgraph. It executes an optimal subnet partitioning algorithm based on dynamic programming on the changed subgraph to allocate network nodes to different subnet partitions. It uses a topology-aware scheduling model to calculate the subnet partitioning scheme and outputs the mapping relationship between nodes and subnets.

[0025] S03. Within each subnet partition, the improved Tarjan strongly connected component algorithm is started to detect loops. Connection summary information is extracted from cross-subnet boundary nodes and distributed loop verification is performed. When a loop topology is detected, the loop path is marked and the loop impact score is calculated. Based on the loop impact score, it is determined whether to perform topology reconstruction. When the loop impact score exceeds a preset threshold, the link weight adjustment mechanism is triggered and the process returns to step S02 to re-execute subnet partitioning. When the loop impact score is lower than the preset threshold, the loop path is marked and the process proceeds to the next step.

[0026] S04. Collect ARP entries, routing entries, OpenFlow flow tables and virtualization platform interface data of network devices, construct a traffic matrix and calculate the latency correlation coefficient between nodes, establish the mapping relationship between physical devices, virtual switches and container networks through a multi-layer association inference model and output the end-to-end path topology.

[0027] S05. Perform power spectrum analysis on the optical power monitoring points of the optical network equipment to identify the occupancy status of each wavelength channel, reconstruct the wavelength routing path of the DWDM system by combining the tag switching path information, establish an optoelectronic hybrid hierarchical topology model and generate a three-dimensional spatiotemporal coordinate record of the wavelength evolution trajectory, and output the wavelength routing path and the wavelength evolution trajectory.

[0028] S06. A two-phase commit synchronization is performed on the topology data collected by the distributed probe nodes using a causal consistency protocol. Each distributed probe node updates its local topology view after confirming data consistency by comparing version vector identifiers, generates a globally unified network topology relationship map and stores it persistently. The probe interval parameter in step S01 is corrected according to the link jitter frequency in subsequent monitoring periods. When the link jitter frequency is higher than the stability threshold, the probe interval parameter is shortened; when the link jitter frequency is lower than the stability threshold, the probe interval parameter is extended.

[0029] The Bloom filter is a highly space-efficient probabilistic data structure that maps elements to different positions in an array using multiple hash functions. This allows for quick determination of whether an element exists in a set and is used in device discovery scenarios to filter known devices and avoid duplicate scanning.

[0030] The SNMP Trap is an asynchronous notification mechanism of the Simple Network Management Protocol. When a network device undergoes a state change, it proactively sends an alarm message to the management station, eliminating the need for the management station to periodically poll, thus reducing network load and improving the real-time response to events.

[0031] The ICMP probe is an Internet Control Message Protocol probe technology that determines device reachability and network latency by sending echo request messages and waiting for echo response messages from the target device, and is used to actively discover network topology.

[0032] The hybrid strategy refers to the combined use of passive listening and active probing. Passive listening receives status change events actively reported by receiving devices, while active probing periodically scans network devices to supplement unreported status information. The two complement each other to improve discovery coverage and accuracy.

[0033] The basic attribute information of the device includes the device's IP address, MAC address, device model, number of ports, firmware version, and management domain identifier, which are used to uniquely identify the network device and provide basic data for subsequent topology analysis.

[0034] The link status change information includes the link connection status, bandwidth utilization, packet loss rate, and latency jitter value, and records the link's state transition events from normal to fault or from fault to normal, as well as the transition time.

[0035] The copy-on-write technique refers to creating a copy of the data instead of directly overwriting the original data when modifying it, while keeping the original data unchanged until all references end. In topology updates, it is used to maintain historical version snapshots and support rollback operations.

[0036] The current topology snapshot is a complete mirror of the network topology at a certain moment, containing all node information, edge information, and attribute data. The snapshot technology saves the historical state of the topology for fault analysis and version comparison.

[0037] The version vector identifier is a data structure used in a distributed system to track causal relationships. Each distributed probe node maintains a vector to record the logical clock of each node. The order or concurrency of events is determined by comparing the version vector identifiers, which is used to detect topology update conflicts.

[0038] The differential computation refers to extracting changed nodes and edges by comparing snapshots of the old and new topologies, and performing subsequent processing only on the changed parts, thus avoiding the computational overhead of full reconstruction and significantly improving update efficiency in large-scale networks.

[0039] The changed node set is a collection of nodes in the topology graph that have undergone state changes, been added, or been deleted. It records the node identifier and change type, and serves as one of the processing objects for incremental updates.

[0040] The changed edge set is a set of links in the topology graph that have undergone state changes, additions, or deletions. It records the starting node, ending node, and change type of the edges, and serves as one of the processing objects for incremental updates.

[0041] The changed subgraph is a subgraph structure composed of the changed node set and the changed edge set, including newly added nodes, deleted nodes, state-changed nodes and their connection relationships, serving as the processing scope for incremental updates.

[0042] The proposed dynamic programming-based optimal subnetting algorithm models the network partitioning problem as a multi-stage decision-making process. The state space is the set of already assigned nodes, and state transitions correspond to the operation of assigning a new node to a specific subnet partition. The algorithm defines a value function that accumulates the local costs of each stage. These costs include a weighted sum of communication traffic between nodes within a subnet partition and cross-boundary traffic between subnet partitions. A memoization technique is used to store the calculated optimal solutions to subproblems, avoiding redundant computation. The backtracking process starts from the final state and generates a mapping scheme from nodes to subnet partitions along the optimal decision path, ensuring that communication overhead between subnet partitions is minimized while maintaining a relatively balanced load across subnet partitions. The algorithm's time complexity is a polynomial of the product of the number of nodes and the number of subnet partitions. The calculation process of the value function is as follows: each node in the changing subgraph is added to the subnet partition in sequence, and the weighted sum of the subnet partition cohesion and the cross-subnet partition traffic cost is calculated for each possible allocation scheme. The allocation path that minimizes the cumulative cost is selected as the optimal solution. The subnet partition cohesion is characterized by the ratio of the number of links between nodes in the subnet partition to the total number of links, and the cross-subnet partition traffic cost is characterized by the sum of the bandwidth utilization of the cross-subnet partition edges.

[0043] The subnet partition is a logical area formed by dividing network nodes according to their communication correlation and geographical location. Nodes within the same subnet partition communicate frequently and are physically close to each other, while communication between different subnet partitions is relatively sparse. The purpose of this partition is to reduce routing complexity and improve management efficiency.

[0044] The subnetting scheme includes the mapping relationship between nodes and subnets, the boundary node identifiers of each subnet partition, and the connection topology between subnet partitions, which are used to guide subsequent loop detection and path planning.

[0045] The topology-aware scheduling model adopts a feature fusion computing framework based on multi-head cross-attention. The model input includes the geographic location coordinates of the nodes, link bandwidth capacity, historical traffic statistics and device type identifiers, and the output is the subnet partition affiliation probability distribution of each node.

[0046] The topology-aware scheduling model has a three-layer architecture comprising an input embedding layer, a multi-layer feature fusion layer, and an output decision layer. The input embedding layer maps node geographic location coordinates to a 128-dimensional spatial vector using a fully connected network, encodes normalized link bandwidth capacity and historical traffic statistics into a 64-dimensional vector, and converts device type identifiers into a 32-dimensional vector using one-hot encoding. These three feature vectors are concatenated to form a 224-dimensional input feature. The multi-layer feature fusion layer contains four stacked multi-head cross-attention modules, each with eight attention heads. It calculates the association weights between different feature dimensions through linear transformations of the query matrix, key matrix, and value matrix. The query comes from features of the current node, while the keys and values ​​come from features of neighboring nodes. The attention score is calculated using a scaled dot product and then normalized using softmax to obtain the weight distribution. The weighted summed multi-head output is processed through residual connections and layer normalization, and finally undergoes a non-linear transformation via a feedforward neural network. The output decision layer uses a fully connected network to map the fused features to a logits vector equal to the number of subnet partitions, and then performs softmax activation to obtain the subnet partition affiliation probability. The number of attention heads in the multi-head cross-attention module is determined based on the average node degree of the current network. When the average node degree is less than 10, 4 attention heads are set; when the average node degree is between 10 and 30, 8 attention heads are set; and when the average node degree exceeds 30, 16 attention heads are set. This dynamic adjustment strategy ensures that the topology-aware scheduling model avoids over-parameterization in sparse networks while capturing complex association patterns in dense networks.

[0047] The steps for establishing the training dataset for the topology-aware scheduling model include: extracting network topology snapshot records from the past six months from a historical topology database; extracting node geographic location coordinates, link bandwidth capacity, historical traffic statistics, and device type identifiers for each network topology snapshot to form feature samples; having network operation and maintenance experts score the historical subnet partitioning schemes based on subnet inter-division communication overhead and load balancing, and selecting subnet partitioning schemes with scores higher than 85 as labels; performing data augmentation operations on the feature samples, including randomly deleting 10% to 20% of edges to simulate link failures, adding 5% to 15% Gaussian noise to the bandwidth values ​​to simulate measurement errors, and randomly swapping the location coordinates of 5% of nodes to simulate topology changes; and dividing the augmented feature samples into a training set and a validation set at an 8:2 ratio, with the training set containing approximately 50,000 network topology snapshot samples and the validation set containing approximately 12,000 network topology snapshot samples.

[0048] The training steps of the topology-aware scheduling model include: using the cross-entropy loss function to measure the difference between the predicted subnet partition ownership probability and the true label; adding an L2 regularization term to constrain the parameters of the topology-aware scheduling model to prevent overfitting, with the regularization coefficient set to 0.0001; using the Adam optimizer to perform gradient descent updates, with an initial learning rate set to 0.001, and the learning rate decaying to 0.9 times the original value every 10 training epochs; setting the training batch size to 64, with a total of 100 training epochs; evaluating the performance of the topology-aware scheduling model on the validation set after each training epoch, and triggering an early stopping mechanism to terminate training when the validation set loss does not decrease for 5 consecutive epochs; and selecting the model parameters with the best performance on the validation set as the final topology-aware scheduling model.

[0049] The multi-head cross-attention-based feature fusion computing framework achieves deep correlation modeling between features of different dimensions through a dynamic weight allocation mechanism, which has significant advantages over traditional feature concatenation or weighted averaging methods. This framework projects node spatial location information, link performance indicators, and device attribute information into a unified semantic space. Through an attention mechanism, it adaptively learns the contribution of each feature dimension to subnet partitioning decisions, capturing the nonlinear coupling relationship between location proximity and traffic correlation, and identifying the complex rule that geographically close but sparsely communicated nodes should not be assigned to the same subnet partition. The multi-head design enables the topology-aware scheduling model to learn multiple feature combination patterns in parallel. Different attention heads focus on different feature interaction methods, enhancing the adaptability of the topology-aware scheduling model to heterogeneous network environments and maintaining stable subnet partitioning quality even when facing abrupt changes in topology or traffic patterns. The cross-attention mechanism effectively aggregates local topology information by querying the association strength between the current node and its neighboring nodes. This enables the topology-aware scheduling model to perceive the community affiliation tendency of nodes in the network. The generated subnet partitioning scheme significantly reduces cross-subnet partition communication overhead while ensuring high cohesion within subnet partitions, thereby improving the overall network data transmission efficiency and resource utilization. This lays a good topology foundation for subsequent loop detection and path optimization.

[0050] The node-to-subnet mapping relationship is a mapping table that records the subnet partition identifier to which each network node belongs. This table is used to quickly query the subnet partition to which a node belongs and to guide routing decisions and resource allocation.

[0051] The average node degree is the arithmetic mean of the number of connected edges of all nodes in the network, reflecting the density of the network's connections. It is calculated by dividing twice the number of all edges by the total number of nodes.

[0052] The improved Tarjan strongly connected component algorithm adds node access order optimization and stack space compression mechanisms to the standard Tarjan algorithm. It reduces recursion depth by prioritizing access to nodes with larger out-degrees and immediately outputs and releases stack space after detecting strongly connected components, thereby reducing peak memory usage.

[0053] The cross-subnet boundary node is a node located at the edge of a subnet partition and directly connected to other subnet partitions. It serves as a key forwarding point for inter-subnet communication and an observation location for loop detection.

[0054] The connection summary information includes the identifier of the cross-subnet boundary node, the external subnet partition number of the connection, the link bandwidth capacity, and the latency characteristics, which are used to complete cross-subnet partition loop detection without exchanging complete topology snapshots.

[0055] The distributed loop verification is a loop confirmation process executed collaboratively by multiple distributed probe nodes. After exchanging the connection summary information, each distributed probe node independently determines whether a cross-subnet partition loop exists, and confirms the existence of the loop through a majority voting mechanism, thereby improving the reliability of detection.

[0056] The loop topology is a subgraph composed of nodes and edges that form closed loops in the network. Data packets may be circulated and forwarded in the loop topology, which may lead to broadcast storms and network paralysis, and need to be identified and handled in a timely manner.

[0057] The loop path is a complete path sequence from the starting node of the loop through several intermediate nodes and back to the starting node. The node order and link connection relationship in the loop topology are recorded to locate the loop position and analyze the cause of the loop.

[0058] The loop impact score is obtained by weighting the number of nodes in the loop path, the number of subnet partitions covered by the loop path, and the total delay of the loop path. A higher score indicates a greater potential impact of the loop topology on network performance, requiring priority handling. The calculation formula for the loop impact score is as follows: Divide the number of nodes in the loop path by the total number of nodes in the changed subgraph to obtain a node percentage normalized value; divide the number of subnet partitions covered by the loop path by the total number of subnet partitions to obtain a subnet partition percentage normalized value; divide the total delay of the loop path by the average path delay of the network to obtain a delay normalized value; assign weight coefficients of 0.4, 0.3, and 0.3 to the node percentage normalized value, the subnet partition percentage normalized value, and the delay normalized value, respectively, and then sum them to obtain the loop impact score.

[0059] The preset threshold is a pre-set boundary value for the loop impact score. When the actual loop impact score exceeds the preset threshold, it is determined that the loop topology has a serious impact on the network and topology reconstruction needs to be performed. When the actual loop impact score is lower than the preset threshold, it is determined that the loop topology has a limited impact and the current topology can be maintained. The preset threshold is determined based on historical loop statistics and service stability requirements, and is usually set between 0.6 and 0.8.

[0060] The link weight adjustment mechanism dynamically modifies the link weight value according to the congestion level of each link in the loop path, increases the weight of highly congested links and reduces their priority in routing selection, guiding traffic to detour and reducing loop pressure.

[0061] The topology reconstruction refers to re-executing the optimal subnetting algorithm based on dynamic programming according to the modified link weight values ​​based on the link weight adjustment mechanism, and generating a new subnetting scheme to eliminate or mitigate the impact of the loop topology.

[0062] The loop path marking is achieved by adding loop attribute identifiers to the nodes and edges involved in the loop path in the current topology snapshot, which is used for alarm display and subsequent route avoidance.

[0063] The ARP entry is an Address Resolution Protocol entry that records the mapping relationship between IP addresses and MAC addresses. Network devices obtain the physical address of the target device by querying the ARP entry, which is used for Layer 2 forwarding and device association analysis.

[0064] The routing table entry is a collection of routing information stored in the network device. Each routing table entry contains the destination network address, next-hop address, outgoing interface, and routing metric value, which are used to guide the three-layer forwarding of data packets.

[0065] The OpenFlow flow table is a collection of entries in a software-defined network that records packet forwarding rules. Each entry contains a matching field, action instructions, and priority. The data plane forwarding logic of the virtual network can be obtained by querying the OpenFlow flow table.

[0066] The virtualization platform interface data includes the mapping relationship between virtual machines and physical hosts, the port connection information of virtual switches, and the VLAN configuration of virtual networks, which are obtained by calling the API interface provided by the virtualization platform.

[0067] The traffic matrix is ​​a two-dimensional matrix structure that records the amount of communication traffic between any two nodes in the network. The matrix element values ​​represent the data transmission rate from the source node to the target node. By analyzing the traffic matrix, high-traffic node pairs and communication hotspots can be identified.

[0068] The latency correlation coefficient measures the degree of synchronization of latency fluctuations between two nodes. It is obtained by calculating the Pearson correlation coefficient of the latency sequence. A high correlation indicates that the two nodes share a physical link or are located on the same network path.

[0069] The multi-layer association reasoning model is a hybrid reasoning system that integrates Bayesian network probabilistic reasoning and topological constraint rules. It infers the implicit device association relationships through traffic pattern similarity, the time delay correlation coefficient, and path connectivity observation variables, and outputs the association probability and path mapping relationship between nodes.

[0070] The physical devices are the actual network hardware devices deployed, including physical switches, physical routers, and physical servers, which have real physical ports and hardware resources.

[0071] The virtual switch is a software switch running on a virtualization platform. It implements Layer 2 switching functions through software simulation and connects multiple virtual machines on the same physical host.

[0072] The container network is a virtual network in the container runtime environment. It enables communication between containers through network namespace isolation and virtual bridge connections, forming a multi-layered network architecture with the physical network and virtual machine network.

[0073] The end-to-end path topology is a complete path graph consisting of all intermediate devices and links from the source node to the target node, including cross-layer mapping relationships of the physical layer, virtual layer, and container network layer, which is used for fault location and performance analysis.

[0074] The optical network equipment is an optical transmission device deployed in an optical fiber network, including optical amplifiers, optical add-drop multiplexers, and optical cross-connect devices, used for the transmission, amplification, and switching of optical signals.

[0075] The optical power monitoring point is an optical power detection device deployed in the optical network, which can measure the optical power value of each wavelength channel in the optical fiber link in real time, and identify wavelength occupancy and signal quality through the power spectrum analysis.

[0076] The power spectrum analysis extracts frequency domain features by performing a fast Fourier transform on the optical power time series, identifies the center frequency and power distribution of different wavelength channels, and determines whether the wavelength is occupied by services and the modulation format type.

[0077] The wavelength channels are independent optical wavelength resources allocated in the dense wavelength division multiplexing system. Each wavelength channel carries an independent service signal, and different wavelength channels are transmitted in parallel in the same optical fiber without interfering with each other.

[0078] The occupancy status refers to whether the wavelength channel is currently being used by services, including idle status and occupancy status, which is determined by the power level in the power spectrum analysis.

[0079] The label switching path information is data in a multiprotocol label switching network that records the mapping relationship between labels and forwarding paths. It includes the incoming label, the outgoing label, and the next-hop address, and is used to track the complete forwarding trajectory of the label switching path.

[0080] The DWDM system is a dense wavelength division multiplexing system that transmits multiple optical signals of different wavelengths simultaneously in a single optical fiber. Each wavelength channel carries an independent service, and the transmission capacity of the optical fiber is greatly improved through wavelength multiplexing technology.

[0081] The wavelength routing path is an optical layer path formed by optical cross-connect devices and optical fiber links through which optical signals travel from the source node to the target node. Unlike the electrical layer routing path, the wavelength routing path completes the switching in the optical domain without photoelectric conversion.

[0082] The optoelectronic hybrid hierarchical topology model establishes a three-layer mapping relationship between physical layer fiber connections, optical layer wavelength routing, and electrical layer IP routing. It unifies the topology information of different layers through inter-layer association rules, supporting cross-layer path tracing and fault isolation.

[0083] The three-dimensional spatiotemporal coordinates use the spatial dimension to record the propagation path of wavelengths between network nodes and the time dimension to record the change process of the wavelength channel occupancy status, forming a spatiotemporal evolution trajectory of wavelength usage, which is used for visual analysis of wavelength resource utilization.

[0084] The wavelength evolution trajectory is a recorded sequence of the wavelength channel occupancy status changing over time, including the wavelength establishment time, holding time, and release time, and is represented by the three-dimensional spatiotemporal coordinates to form a spatiotemporal distribution feature.

[0085] The causal consistency protocol ensures that operations with causal relationships in the distributed system are executed in the same order on all distributed probe nodes. The version vector identifies and tracks operation dependencies. When a causal conflict is detected, the execution of subsequent operations is delayed until the dependent operations are completed, ensuring the logical consistency of the topology view.

[0086] The topology data includes the device basic attribute information, the link status change information, the node-to-subnet mapping relationship, the loop path marking, the end-to-end path topology, the wavelength routing path, and the wavelength evolution trajectory, which are all the basic data required to construct a complete network topology map.

[0087] The two-phase commit is a distributed transaction processing protocol. In the first phase, the coordinating node sends a preparation request to all participating distributed probe nodes and waits for a response. In the second phase, a commit or rollback instruction is sent based on the response result to ensure the atomicity of the distributed topology update operation.

[0088] The local topology view is a local network topology information maintained by a single distributed probe node, containing node, edge, and attribute data of the network area monitored by the distributed probe node. It is merged with the local topology views of other distributed probe nodes through a distributed synchronization mechanism to form a global view.

[0089] The network topology graph is a graph structure data that describes all device nodes and their connections in the network. Nodes represent network devices, edges represent link connections, and nodes and edges are accompanied by attribute information. It includes multi-layer topology relationships of physical layer, virtual layer, container layer and optical layer.

[0090] The link jitter frequency refers to the number of times the link status switches between normal and faulty per unit time. It is calculated by statistically analyzing the frequency of the link status change information. A higher link jitter frequency indicates that the link is unstable and the monitoring interval needs to be shortened.

[0091] The detection interval parameter is the time interval between two adjacent device detection operations. A shorter detection interval parameter improves the real-time performance of status updates but increases network load, while a longer detection interval parameter reduces load but may miss transient changes. The detection frequency is adaptively controlled through a feedback adjustment mechanism.

[0092] The stability threshold is a preset boundary value for the link jitter frequency. When the actual link jitter frequency exceeds the stability threshold, the network is determined to be in an unstable state and requires encrypted monitoring. When the link jitter frequency is lower than the stability threshold, the network is determined to be stable and the monitoring frequency is reduced. The stability threshold is determined based on historical statistical data and service stability requirements, and is usually set to 5 to 10 link state switches per hour.

[0093] The feedback adjustment mechanism establishes a reverse mapping relationship between the output parameters of subsequent steps and the input parameters of preceding steps. By monitoring changes in the network operating status, it dynamically adjusts the detection strategy and processing flow, forming an adaptive and optimized closed-loop control system, thereby improving the system's response speed and processing accuracy to changes in the network environment.

[0094] The present invention also provides a method for forming a dynamic network topology graph generation system by means of a computer, wherein the computer is provided with a storage medium, the storage medium stores program instructions, and the program instructions execute the above-described method when running in the computer.

[0095] The specific implementation methods of the above steps are described in detail below.

[0096] The specific implementation of step S01 is as follows: First, distributed probe nodes are deployed at the core switch, aggregation switch, and access switch of the network. These distributed probe nodes establish communication connections with the core switch, aggregation switch, and access switch through the network management interface. The distributed probe nodes activate a passive listening module, configured with SNMPTrap receiving port number 162, waiting for device online events and link status change information actively sent by network devices. Simultaneously, an active probe module is activated. This module uses the ICMP protocol to send echo request messages to devices in the network, with an initial probe interval parameter set to 30 seconds. Device reachability is determined based on the echo response messages from the devices, and the response delay is recorded. When a device online event is received or a new device response is detected, the device's IP address and MAC address are extracted as device identifiers. These identifiers are then input into a Bloom filter for deduplication. The Bloom filter uses three hash functions to map the device identifier to a 10000-bit array. The corresponding bit value in the bit array is queried. If all corresponding bits are 1, the device is considered a known device, and subsequent processing is skipped. If a bit is 0, the device is considered a new device, and the corresponding bit is set to 1. For newly discovered devices, attribute collection is performed. This involves querying the device's system object identifier (SIO) via SNMP to obtain the device model, querying the interface table to obtain the number of ports, querying the system description to obtain the firmware version, and querying the management information database to obtain the management domain identifier. This information is then combined to form the device's basic attribute information. Simultaneously, link status change information is collected, recording the link's connection status as normal or faulty. The number of inbound and outbound bytes at the link interface is collected to calculate bandwidth utilization, the link's packet loss counter is collected to calculate the packet loss rate, and the standard deviation of ICMP round-trip time is collected to calculate latency jitter. The purpose of these steps is to achieve rapid discovery and real-time status monitoring of network devices. By combining the low overhead of passive monitoring with the high coverage of active probing using a hybrid strategy, and by introducing a Bloom filter, duplicate scanning of known devices is avoided, significantly reducing network load and processing latency.

[0097] The specific implementation of step S02 is as follows: Distributed probe nodes continuously monitor link status change information. When a link status changes from normal to fault or recovers from fault to normal, an incremental topology update process is triggered. First, a copy-on-write operation is performed on the currently maintained topology graph structure to create a complete copy of the topology graph in memory as a snapshot of the current topology graph. A version vector identifier is assigned to the current topology graph snapshot, which contains the logical clock value of each distributed probe node. The previously saved historical topology graph snapshot is read, and a difference calculation is performed between the current topology graph snapshot and the historical topology graph snapshot. All nodes in the current topology graph snapshot are traversed. If a node does not exist in the historical topology graph snapshot, it is marked as a newly added node. If the attribute value of a node changes, it is marked as a state-changed node. All marked nodes are added to the set of changed nodes. Similarly, the edge set is traversed to identify newly added edges, deleted edges, and state-changed edges, and a set of changed edges is constructed. A changed subgraph is constructed based on the set of changed nodes and the set of changed edges. The changed subgraph only contains the topology elements that have changed and their one-hop neighbor nodes. An optimal subnetting algorithm based on dynamic programming is applied to the changing subgraph. The algorithm processes nodes in the changing subgraph sequentially according to topological order, traversing all possible subnetting allocation schemes for each node and calculating the subnetting cohesion and cross-subnetting traffic cost for each scheme. Subnetting cohesion is obtained by calculating the ratio of the number of edges within a subnetting to the total number of connections to nodes in that subnetting. Cross-subnetting traffic cost is obtained by accumulating the bandwidth utilization of cross-subnetting edges. The two are weighted and summed using coefficients of 0.6 and 0.4 to represent the local cost of the current state. The algorithm uses a memoized data structure to store the calculated states and their optimal costs, avoiding redundant calculations of the same subproblems. After all nodes are allocated, the algorithm backtracks from the final state to generate the mapping relationship between nodes and subnets. The node's geographical coordinates, link bandwidth capacity, historical traffic statistics, and device type identifier are input into the topology-aware scheduling model. The model calculates the correlation weights between node features using a multi-head cross-attention mechanism, outputting the probability distribution of each node's affiliation to each subnet partition. The subnet partition with the highest probability is selected as the final node allocation result, forming a complete subnet partitioning scheme. The purpose of these steps is to achieve efficient incremental updates of topology changes, reduce the processing scope through differential computation, ensure the global optimality of subnet partitioning through dynamic programming, and enhance the ability to recognize complex topology patterns through a deep learning model.

[0098] The specific implementation of step S03 is as follows: Network nodes are assigned to corresponding subnet partitions according to the subnetting scheme. Within each subnet partition, the improved Tarjan strongly connected component algorithm is started independently. The algorithm maintains a depth-first search stack and a node access timestamp array. It prioritizes nodes with larger out-degrees as the search starting point, performing a depth-first traversal from the starting point. When a node is visited, its access timestamp and minimum reachable timestamp are recorded. When the traversal returns, the timestamp of the current node is compared with the minimum reachable timestamp of its child nodes. If they are equal, the current node is determined to be the root node of the strongly connected component. All nodes from the root node to the top of the stack are popped from the search stack to form a strongly connected component. If the number of nodes in the strongly connected component is greater than one, a loop topology exists. The node sequence and edge sequence in the loop path are recorded. Connection summary information is extracted from cross-subnet boundary nodes. The connection summary information records the boundary node's identifier, the connected external subnet partition number, the bandwidth capacity of the corresponding link, and its latency characteristics. Distributed probe nodes in each subnet exchange connection summary information. By analyzing the connection relationships between boundary nodes, it determines whether a loop topology exists across subnet partitions. If the boundary of the subnet partition where node A is located connects to the subnet partition where node B is located, and the boundary of the subnet partition where node B is located connects back to the subnet partition where node A is located, a loop across subnet partitions is formed. After detecting a loop topology, a loop impact score is calculated. The number of nodes in the loop path and the number of subnet partitions crossed by the loop path are counted. The total loop path delay is obtained by summing the delay values ​​of all edges on the loop path. The node percentage normalized value is obtained by dividing the number of nodes by the total number of nodes in the changing subgraph. The subnet partition percentage normalized value is obtained by dividing the number of subnet partitions by the total number of subnet partitions. The delay normalized value is obtained by dividing the total loop path delay by the average path delay of the network. The loop impact score is obtained by weighting and summing the three normalized values ​​with weight coefficients of 0.4, 0.3, and 0.3. The algorithm determines whether the loop impact score exceeds a preset threshold of 0.7. If it does, a link weight adjustment mechanism is triggered, increasing the weight of links in the loop path with bandwidth utilization exceeding 80% to 1.5 times their original value. After adjustment, the algorithm returns to step S02 to re-execute subnet partitioning. If the loop impact score is below the preset threshold of 0.7, loop attribute identifiers are added to nodes and edges in the loop path to form loop path markers. These loop path markers are then output, and the process proceeds to the next step. The purpose of these steps is to efficiently detect loop topology structures in the network and assess their impact. This is achieved by reducing memory consumption through an improved strongly connected component algorithm, realizing global detection of cross-subnet partition loops through distributed collaboration, and enabling intelligent decision-making for loop handling strategies through impact scoring.

[0099] The specific implementation of step S04 is as follows: ARP entries, routing entries, OpenFlow flow tables, and virtualization platform interface data are collected from network devices as input data for multi-layer association reasoning. ARP entries are parsed to extract the mapping relationship between IP addresses and MAC addresses; routing entries are parsed to extract routing information between the destination network address and the next-hop address; OpenFlow flow tables are parsed to extract flow matching rules and forwarding actions; and the virtualization platform API interface is called to obtain the binding relationship between virtual machines and physical hosts, the port mapping of virtual switches, and VLAN configuration information. A traffic matrix is ​​constructed based on the collected data. The row and column indices of the traffic matrix are network node identifiers, and the matrix element values ​​are the data transmission rates between corresponding node pairs, calculated using byte counters and timestamps in the flow tables. Delay measurement is performed on any two nodes in the network. The delay sequence between nodes is obtained using ICMP messages with timestamp options or the round-trip time of a TCP three-way handshake. The Pearson correlation coefficient of the delay sequence is calculated as the delay correlation coefficient. A correlation coefficient close to 1 indicates that the delay fluctuations of the two nodes are highly synchronized. The traffic matrix, latency correlation coefficient, ARP mapping, routing information, flow table rules, and virtualization mapping are input into a multi-layer association inference model. The model uses a Bayesian network to establish a probabilistic dependency structure between observed variables and implicit associations. It infers communication associations between virtual machines through traffic pattern similarity, infers physical path sharing between nodes through latency correlation coefficients, and verifies the rationality of the inference results through topology constraint rules. The model outputs association probability values ​​between nodes, selects node pairs with association probabilities greater than 0.8 to establish association edges, and constructs a cross-layer mapping topology for physical devices, virtual switches, and container networks. The complete forwarding path from the source physical device through the virtual switch to the target container is traced, recording the device and interface information for each hop along the path to form an end-to-end path topology. The purpose of these steps is to accurately identify the associations between devices in virtualized and containerized environments, improve inference accuracy by fusing multi-source data, handle uncertainty through probabilistic models, and achieve complete path reconstruction from the physical layer to the application layer.

[0100] The specific implementation of step S05 is as follows: Optical power monitoring is performed on the optical network equipment deployed in the optical network. The optical power monitoring point collects the optical power value of each wavelength channel every 100 milliseconds. The collected optical power time series is input into a Fast Fourier Transform algorithm for power spectrum analysis. The algorithm converts the time-domain optical power signal into a frequency-domain power spectrum, identifies the center frequency corresponding to the power peak in the spectrum, and determines the wavelength channel number corresponding to the peak based on the standard wavelength spacing of 100 GHz in dense wavelength division multiplexing (DWDM) systems. A threshold judgment is performed on the power level of each wavelength channel. If the power level is higher than the service signal threshold minus 20 dBm, the wavelength channel is determined to be in an occupied state; if the power level is lower than the noise threshold minus 40 dBm, it is determined to be in an idle state. Label switching path information is queried from the network management system. Label switching path information records the ingress, egress, and forwarding interfaces of each label switching router in the multi-protocol label switching network. The end-to-end label switching path is reconstructed by tracing the label mapping relationship. The wavelength channel occupancy status obtained from optical power monitoring is correlated and matched with the label switching path information to determine the label switching path corresponding to the service carried by each wavelength channel, thereby restoring the wavelength routing path in the DWDM system. A hybrid optoelectronic hierarchical topology model is established. The physical layer records the start and end nodes and lengths of optical fibers, the optical layer records the center frequency and routing path of wavelength channels, and the electrical layer records IP addresses and routing table entries. Inter-layer mapping relationships are established through optoelectronic conversion nodes. Three-dimensional spatiotemporal coordinates are assigned to each wavelength channel. The X and Y axes represent the geographical location of the wavelength channel in the network topology, the Z axis represents the wavelength frequency, and the time axis records the moments when the wavelength channel changes from idle to occupied and back to idle, forming a wavelength evolution trajectory. The purpose of these steps is to achieve refined topology management at the optical network layer, accurately identify wavelength occupancy through power spectral analysis, reconstruct end-to-end optical paths through tag switching path tracing, and intuitively display the dynamic usage of wavelength resources through a three-dimensional spatiotemporal representation.

[0101] The specific implementation of step S06 is as follows: Each distributed probe node encapsulates the topology data it has collected and processed locally. This topology data includes basic device attribute information, link status change information, node-to-subnet mapping relationships, loop path markers, end-to-end path topology, wavelength routing paths, and wavelength evolution trajectories. One distributed probe node is selected as the coordinating node. The coordinating node sends a preparation submission request to all participating distributed probe nodes, carrying the topology data to be updated and its version vector identifier. Upon receiving the preparation request, each distributed probe node checks the version vector identifier and compares it with its locally maintained version vector identifier to determine if there is a causal conflict. If the clock value of a node in the received version vector identifier is less than the corresponding local value, a causal conflict exists, and a rejection response is returned to the coordinating node; otherwise, an acceptance response is returned. The coordinating node collects all responses. If all responses are accepted, a submission command is sent to each node; if there is a rejection, a rollback command is sent, and the update is delayed until the causal dependency is resolved. The distributed probe node receiving the submission command writes the topology data into its local topology view, updates its own logical clock in its local version vector identifier by 1, and completes the local topology view update operation. The coordinating node aggregates the local topology views of all distributed probe nodes, merging the physical layer topology, virtual layer topology, container layer topology, and optical layer topology. This eliminates duplicate nodes and edges, resolves attribute conflicts, and generates a globally unified network topology graph. This graph is persistently stored in a graph database, with node and edge indexes to support fast queries. Simultaneously, a background monitoring task is initiated to count the number of link status changes within subsequent monitoring periods and calculate the link jitter frequency per unit time. If the link jitter frequency exceeds a stability threshold of 8 times per hour, the probe interval parameter is shortened by 20%. If the frequency is below the stability threshold of 5 times per hour, the probe interval parameter is extended by 20%. The modified probe interval parameter is sent to the active probe module in step S01 via control commands to dynamically adjust the probe frequency. The purpose of these steps is to ensure global consistency of topology data in a distributed environment. Data conflicts are avoided through causal consistency protocols and two-phase commit mechanisms, and adaptive optimization of the probe strategy is achieved through a feedback adjustment mechanism, improving the system's responsiveness to network environment changes and resource utilization efficiency.

[0102] It should be noted that the key technical ideas of this invention include an event-driven incremental topology update mechanism, a subnetting optimization strategy integrating dynamic programming and deep learning, and a probabilistic reasoning method for multi-level network relationships. The event-driven incremental topology update mechanism triggers the update process by monitoring network device state change events. Combining copy-on-write technology and differential computation, it only processes changed topology elements. Compared to traditional periodic full scans, this significantly reduces computational overhead and network load. In large-scale dynamic network environments, it can achieve sub-second topology state synchronization, avoiding service interruptions and resource waste caused by full reconstruction. Simultaneously, version vector identification tracks causal relationships, ensuring data consistency in a distributed environment, enabling the system to accurately capture transient topology changes and quickly respond to network faults. The subnetting optimization strategy integrating dynamic programming and deep learning models the network partitioning problem as a multi-stage decision-making process. Dynamic programming ensures the finding of the globally optimal subnetting allocation scheme under constraints. Simultaneously, a topology-aware scheduling model based on multi-head cross-attention learns complex correlation patterns between node features. Compared to traditional heuristic partitioning algorithms, this approach more accurately identifies implicit community structures and communication affinity between nodes. The generated subnetting scheme maintains high cohesion within subnets while minimizing cross-subnet communication overhead, providing a clear topological foundation for subsequent loop detection and path optimization, significantly improving network manageability and operational efficiency. The probabilistic inference method for multi-layer network relationships integrates traffic matrix analysis, latency correlation calculation, and topology constraint rules. It employs Bayesian networks to establish a probabilistic mapping between observed data and implicit relationships. Compared to traditional methods relying solely on ARP and routing tables, this method accurately infers cross-layer device mapping relationships in virtualized and containerized environments. It solves the visibility loss problem caused by the separation of physical and logical topologies in software-defined networking and network function virtualization scenarios, achieving complete end-to-end path reconstruction from the physical layer to the application layer. This provides precise topology data for fault location and performance optimization. The three key technical approaches work synergistically to form a complete dynamic network topology generation solution. The incremental update mechanism provides efficient topology data acquisition and synchronization capabilities, the subnetting optimization strategy provides a structured topology organization method, and the multi-layer association inference method provides accurate cross-layer mapping relationships. These three elements work together to enable the system to accurately construct and maintain multi-dimensional network topology maps encompassing the physical, virtual, container, and optical layers in real-time in large-scale heterogeneous network environments. Compared to traditional single-dimensional topology management methods, this significantly improves the completeness and accuracy of topology information, laying a solid foundation for automated and intelligent network operation and maintenance.

[0103] It should be noted that this invention also solves the following technical problem: the high computational complexity of loop detection algorithms in large-scale network environments leads to insufficient real-time performance. Traditional loop detection methods perform strongly connected component searches on the global topology graph. The algorithm's time complexity is proportional to the product of the number of network nodes and edges. When the network size reaches tens of thousands of nodes, a single detection can take tens of seconds or even minutes, which cannot meet the real-time loop identification requirements in scenarios with rapidly changing link states. This invention uses a dynamic programming-based optimal subnetting algorithm to allocate network nodes to different subnet partitions, significantly reducing the number of nodes and edges within each subnet partition. Within each subnet partition, an improved Tarjan strongly connected component algorithm is executed independently for loop detection, decomposing the global loop search problem into multiple local subproblems to be solved in parallel. The time complexity of loop detection in a single subnet partition is only related to the size of the subnet partition. Parallel computing mechanisms reduce the overall detection time to the second level. Distributed loop verification is completed by exchanging connection summary information for nodes crossing subnet boundaries, avoiding frequent transmission and redundant calculations of global topology snapshots, and significantly improving the real-time performance and accuracy of loop detection in large-scale network environments.

[0104] It should be noted that this invention also solves the following technical problem: the difficulty in tracing the correlation between wavelength routing paths and IP routing paths in optical and electrical network convergence scenarios. Traditional network topology management technologies treat the optical layer and electrical layer as independent systems. The optical network management system only focuses on wavelength channel allocation and optical power monitoring, while the electrical network management system only focuses on IP routing and traffic scheduling. There is a lack of a unified correlation mechanism between the two. When service traffic passes through both optical layer wavelength switching and electrical layer IP routing simultaneously, maintenance personnel cannot obtain complete end-to-end path information, leading to difficulties in locating cross-layer faults. This invention identifies the occupancy status of each wavelength channel by performing power spectrum analysis on optical power monitoring points of optical network devices, reconstructs the wavelength routing path of the DWDM system by combining tag switching path information, establishes an optoelectronic hybrid hierarchical topology model to uniformly represent the three-layer mapping relationship of physical layer fiber connection, optical layer wavelength routing and electrical layer IP routing, uses three-dimensional spatiotemporal coordinates to record wavelength evolution trajectory to form spatiotemporal distribution characteristics of wavelength usage, and associates optical layer wavelength routing path with electrical layer end-to-end path topology through inter-layer association rules, realizing cross-layer path tracing and unified topology view construction in the scenario of optical network and electrical network convergence.

[0105] Specifically, the principle of this invention is as follows: The core technical problem solved by this invention lies in decomposing global topology updates into two mutually coordinating processes: local incremental processing and hierarchical mapping reasoning. Copy-on-write technology maintains the current topology snapshot unchanged while creating a copy for modification. Differential computation extracts only the changed node set and changed edge set to form the changed subgraph, shrinking the processing scope of incremental updates from all network nodes to the changed region, and reducing computational complexity from global scale to the scale of the change. The optimal subnet partitioning algorithm based on dynamic programming allocates network nodes to different subnet partitions through a multi-stage decision-making process, ensuring frequent communication and close physical distance between nodes within a subnet partition, while communication between subnet partitions is relatively sparse. The improved Tarjan algorithm independently performs loop detection within each subnet partition, decomposing the global loop search problem into multiple local subproblems solved in parallel. Nodes across subnet boundaries complete distributed loop verification through connection summary information exchange, avoiding frequent transmission of global topology snapshots. The multi-layer association inference model establishes probabilistic associations between nodes through traffic matrices and latency correlation coefficients, and infers the mapping relationship between physical devices, virtual switches, and container networks by combining topology constraint rules. This breaks through the limitation of traditional topology discovery technology being limited to a single network layer and achieves accurate tracking of cross-layer paths.

[0106] The following provides a specific embodiment 1 of the present invention. In this embodiment 1, the specific implementation of steps S01, S05 and S06 is the same as that described above, and will not be repeated here. The specific implementation of other steps is described in detail below.

[0107] The specific implementation of step S02 is as follows: When link status change information is detected, the system immediately triggers the incremental topology update process, uses copy-on-write technology to take a snapshot of the current topology graph and assigns a version vector identifier, the version vector identifier being... dimensional vector ,in This refers to the number of distributed probe nodes, typically ranging from 5 to 20, determined based on the network size. Indicates the first The logical clock value of each node, in units of times. After extracting changed nodes and edges by comparing old and new snapshots using differential calculation, and constructing a changed subgraph, an optimal subnetting algorithm based on dynamic programming is executed. The value function calculation formula is expressed as follows:

[0108] ;

[0109] In the formula, For those already allocated There are nodes and the current set of nodes to be assigned is . The minimum cumulative cost at that time, dimensionless; The current node number being processed. The value range is from 1 to the total number of nodes in the changing subgraph; This is the current set of nodes to be assigned. For the first One node; This represents the total number of subnet partitions. Number the subnet partition; To make nodes Assigned to subnet partition The communication cost within the subnet partition is dimensionless. To make nodes Assigned to subnet partition The resulting cross-subnet partition communication cost is dimensionless.

[0110] in, The calculation formula is expressed as follows:

[0111] ;

[0112] In the formula, This is the subnet partition cohesion weighting coefficient, with an empirical value of 0.6, and is dimensionless. Subnet partitioning Add node Number of internal links, in units of links; Subnet partitioning Add node Total number of links, in units of links.

[0113] The calculation formula is expressed as follows:

[0114] ;

[0115] In the formula, This is the weighting coefficient for cross-subnet partition traffic cost, with an empirical value of 0.4, and is dimensionless. For the set of cross-subnet partition edges; Used as an identifier for edges; For the edge The bandwidth utilization rate, ranging from 0 to 1, is dimensionless; As a reference bandwidth utilization rate, it is usually taken as 0.8, which is dimensionless.

[0116] After receiving the geographic location coordinates of nodes, link bandwidth capacity, historical traffic statistics, and device type identifiers, the topology-aware scheduling model inputs the embedding layer to map the geographic location coordinates into a 128-dimensional vector, encode the bandwidth capacity and historical traffic statistics into a 64-dimensional vector, and convert the device type identifier into a 32-dimensional vector, concatenating them to form a 224-dimensional input feature. The formula for calculating the attention score in the multi-head cross-attention module is as follows:

[0117] ;

[0118] In the formula, The query matrix has a dimension equal to the number of nodes multiplied by the feature dimension. The key matrix has a dimension equal to the number of nodes multiplied by the feature dimension. It is a value matrix, with dimensions equal to the number of nodes multiplied by the feature dimension; Represents the transpose of the key matrix; The dimension of the key vector is usually set to 64, and the unit is dimension. This is a scaling factor used to prevent the gradient of the softmax function from vanishing due to an excessively large dot product. The output decision layer obtains the subnet partition assignment probability through softmax activation, completing the mapping relationship between nodes and subnets.

[0119] The specific implementation of step S03 is as follows: Within each subnet partition, an improved Tarjan strongly connected component algorithm is initiated. This algorithm traverses nodes using a depth-first search, maintaining access timestamps and minimum backtracking points, prioritizing nodes with higher out-degrees to reduce recursion depth. After extracting connection digest information from cross-subnet boundary nodes, each distributed probe node independently determines cross-subnet partition loops and confirms them through a majority voting mechanism. The formula for calculating the loop impact score is as follows:

[0120] ;

[0121] In the formula, The loop influence score is dimensionless. This represents the number of nodes in the loop path, expressed in units. The total number of nodes in the variable subgraph, expressed in units of nodes; The number of subnet partitions covered by the loop path, expressed in units. This represents the total number of subnet partitions, expressed in units. This represents the total loop path delay, in milliseconds (ms). This represents the average path delay of the network, measured in milliseconds (ms).

[0122] when When the congestion level exceeds a preset threshold (typically set between 0.6 and 0.8), the link weight value is dynamically adjusted based on the congestion level of each link. The link weight adjustment formula is as follows:

[0123] ;

[0124] In the formula, The adjusted link weights are dimensionless. The original link weights are dimensionless. For link identification; This is the weighting adjustment factor, with an empirical value of 1.5, and it is dimensionless. For link The bandwidth utilization rate, ranging from 0 to 1, is dimensionless; This is the bandwidth utilization threshold, typically set to 0.7, and is dimensionless. After adjustment, return to step S02 to re-execute subnetting; otherwise, output the loop path marker and proceed to the next step.

[0125] The specific implementation of step S04 is as follows: After the system collects ARP entries, routing entries, OpenFlow flow tables, and virtualization platform interface data from network devices, it constructs a traffic matrix. ,in Represents a node To the node The data transmission rate is expressed in Mbit / s. The latency correlation coefficient is calculated using the Pearson correlation coefficient formula, as follows:

[0126] ;

[0127] In the formula, For nodes and nodes The time delay correlation coefficient is dimensionless. This represents the number of time-delay sampling points, typically ranging from 100 to 500, determined based on the sampling period, and is expressed in units of points. The sampling sequence number; For nodes No. The delay measurement is in milliseconds (ms). For nodes No. The delay measurement is in milliseconds (ms). For nodes The average latency, in milliseconds (ms); For nodes The mean latency is expressed in milliseconds. The multi-layered correlation inference model integrates traffic pattern similarity, latency correlation coefficients, and path connectivity observation variables. It establishes a mapping relationship between physical devices, virtual switches, and container networks through Bayesian network probabilistic inference, outputting the end-to-end path topology for subsequent analysis.

[0128] It should be noted that the variables involved in this embodiment are explained in detail in Table 1.

[0129] Table 1. Variable Explanation Table

[0130]

[0131] To better understand and implement this invention, a specific application scenario of the invention is provided below as an example 2: To verify the effectiveness of the invention, technicians built an experimental environment. By constructing a large enterprise campus network topology scenario containing 850 network nodes and 1240 links, typical events such as dynamic online / offline status changes of network devices, sudden changes in link status, and fluctuations in traffic load were simulated to verify the real-time performance and accuracy of the network topology graph dynamic generation method in complex network environments. In the experimental environment, 12 distributed probe nodes were deployed in the server room where the network backbone switches are located. Each probe node is responsible for monitoring the status information of approximately 70 network devices. The network nodes include 36 core switches, 124 aggregation switches, 456 access switches, 158 servers, 42 firewalls, and 34 load balancers. The basic attribute information of the devices includes fields such as IP address, MAC address, device model, and number of ports. The experimental scenario sets the link bandwidth range to 100Mbps to 40Gbps, the link latency range to 0.5ms to 25ms, and the network topology is organized according to a three-layer architecture of core layer, aggregation layer, and access layer.

[0132] The technicians first configured the SNMP Trap listening service and ICMP probe task for the distributed probe nodes. The SNMP Trap listening port was set to 162, and the initial ICMP probe interval was set to 30 seconds. The Bloom filter uses three hash functions to map to a bit array of length. The storage space was optimized, and the false positive rate was kept below 0.01%. The first 600 seconds of the experiment were the network initialization phase, during which each distributed probe node collected basic attribute information from all 850 nodes, discovered device identifier sets, and recorded information such as device model, number of ports, and firmware version. At 720 seconds into the experiment, technicians triggered a link status change event, simulating a connection interruption due to device failure on two core links and five aggregation layer links, while three new access switches were added online. The distributed probe nodes immediately received eight link status change alarms via SNMP Trap and confirmed the connectivity of the newly added devices via ICMP probes.

[0133] Upon detecting a link state change, the system triggers an incremental topology update process, using copy-on-write technology to create a snapshot of the current topology graph and assigning a version vector identifier of v128. Differential calculations extract a set of changed nodes containing 8 nodes (5 state-changed nodes and 3 newly added nodes) and a set of changed edges containing 7 deleted edges and 9 newly added edges, forming a changed subgraph with 16 elements. Technicians then execute a dynamic programming-based optimal subnetting algorithm on this subgraph, with a 16×8 state space representing the possible allocation schemes of the 16 nodes across 8 subnet partitions. The algorithm records the state transition matrix of the value function during computation and uses memoization technology to avoid redundant calculations of subproblems. The final subnetting scheme allocates 850 nodes to the 8 subnet partitions, as shown below. Figure 2 As shown.

[0134] As shown in Table 2, the number of nodes and the number of boundary nodes in each subnet partition are relatively evenly distributed. The number of nodes in subnet partition 1 to subnet partition 8 are 124, 109, 118, 95, 107, 102, 98 and 97 respectively, and the number of boundary nodes are 8, 7, 9, 6, 7, 8, 6 and 7 respectively.

[0135] Table 2 Subnet Partition Node Distribution Table

[0136]

[0137] The topology-aware scheduling model plays a crucial role in subnetting. Its inputs include the geographic coordinates of 850 nodes, the bandwidth capacity of 1240 links, historical traffic statistics for the past 24 hours, and device type identifiers. The input embedding layer maps geographic coordinates to 128-dimensional vectors, encodes bandwidth and traffic data into 64-dimensional vectors, and converts device types into 32-dimensional vectors through one-hot encoding. These three feature vectors are concatenated to form a 224-dimensional input feature. Since the average node degree in the current network is 2.92, a multi-head cross-attention module is configured with four attention heads. After processing through four feature fusion layers, the output decision layer generates an 8-dimensional probability distribution vector, such as... Figure 3 As shown, this represents the probability of each node belonging to one of the eight subnet partitions. Technicians trained the model using a training set containing 50,000 topology snapshots. The validation set loss reached its minimum of 0.0342 in the 67th round and did not decrease for five consecutive rounds, triggering an early stopping mechanism to terminate training.

[0138] An improved Tarjan strongly connected component algorithm was used within each subnet partition to detect loops. The algorithm prioritizes visiting nodes with higher out-degrees to reduce recursion depth. A loop topology containing 5 nodes was detected in subnet partition 3. The loop path was from node N215 to N348 to N502 to N671 to N783 and back to N215, with a total latency of 78ms. Further analysis of connection digests at cross-subnet boundary nodes revealed a cross-subnet partition loop between subnet partition 3 and subnet partition 6, involving boundary nodes N348 and N671. A distributed loop verification mechanism confirmed the loop's existence through a majority vote of 12 probe nodes; the voting results showed that 11 nodes confirmed the loop's existence.

[0139] The loop impact score is calculated based on three factors: the number of nodes in the loop path, the number of covered subnets, and the total path delay. The normalized value for node proportion is 5 divided by 850, yielding 0.0059; the normalized value for subnet proportion is 2 divided by 8, yielding 0.25; and the normalized value for delay is 78 divided by the network average path delay of 12.6, yielding 6.19. Weighting coefficients of 0.4, 0.3, and 0.3 are assigned to the three normalized values ​​respectively, and then summed to obtain a loop impact score of 1.93. Since this score exceeds the preset threshold of 0.7, the system triggers a link weight adjustment mechanism, increasing the weight of the three links in the loop path with bandwidth utilization exceeding 85% to 2.5 times their original value, guiding traffic around to alleviate loop pressure. After adjusting the link weights, the system returns to the subnetting step and re-executes the optimal subnetting algorithm based on dynamic programming to generate a new subnetting scheme to eliminate the high-impact loop, such as... Figure 4 As shown.

[0140] Technicians collected 6842 ARP entries, 3156 routing entries, 2487 OpenFlow flow table entries, and 1234 virtualization platform interface data entries from network devices, constructing an 850×850 dimensional traffic matrix. By calculating the Pearson correlation coefficient of the inter-node latency sequences, a latency correlation coefficient matrix was obtained, identifying 318 highly correlated node pairs, indicating that these node pairs share physical links. A multi-layer association inference model, integrating Bayesian network probabilistic inference and topology constraint rules, established a mapping relationship between physical devices, virtual switches, and container networks, successfully identifying the network connection relationships of 346 virtual machines and 512 container instances running on 158 physical servers. The end-to-end path topology includes a three-layer mapping of the physical layer, virtual layer, and container layer, completely recording the cross-layer path sequence from the source container through the virtual switch, physical network interface card, and physical switch to the target container.

[0141] The experimental scenario included a DWDM system composed of 24 optical network devices, each supporting 80 wavelength channels with a center wavelength range of 1530nm to 1565nm and a channel spacing of 50GHz. Technicians performed power spectrum analysis on 48 optical power monitoring points, extracting frequency domain features using Fast Fourier Transform, identifying a total of 1536 currently occupied wavelength channels, representing an occupancy rate of 80%. The wavelength routing paths of the DWDM system were reconstructed by combining label switching path information, establishing a hybrid optoelectronic topology model containing 24 optical nodes and 1536 wavelength channels. Three-dimensional spatiotemporal coordinates were used to record the wavelength evolution trajectory, with a time dimension sampling interval of 10s and a spatial dimension recording the wavelength propagation path between optical nodes, forming a 3600-second wavelength usage evolution trajectory. Figure 5 As shown.

[0142] Each distributed probe node employs a causal consistency protocol to perform a two-phase commit synchronization of topology data. In the first phase, the coordinating node sends a preparation request to the 11 participating nodes, and all nodes respond with confirmation within 200ms. In the second phase, the coordinating node sends a commit command to complete the topology data synchronization. After confirming data consistency through version vector identification comparison, the 12 probe nodes update their local topology views and generate a globally unified network topology graph, which is persistently stored in the graph database. Technical personnel analyzed the link jitter frequency during subsequent monitoring periods and found that the link state switching frequency reached 12 times per hour during the 1800s to 2400s of the experiment, exceeding the stability threshold of 10 times. The system automatically shortened the probe interval parameter from 30s to 15s. During the 3000s to 3600s of the experiment, the link state switching frequency dropped to 4 times per hour, below the stability threshold. The system then extended the probe interval parameter to 45s, achieving adaptive control of the probe frequency.

[0143] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for dynamically generating network topology graphs, characterized in that, Distributed probe nodes are deployed at key network aggregation points to collect device online events and link status change information. A Bloom filter is used to quickly deduplicat newly discovered device identifiers. When link status change information is detected, an incremental topology update process is triggered. A copy-on-write technique is used to create a snapshot of the current topology graph and assign a version vector identifier. A change subgraph is constructed by extracting the set of changed nodes and the set of changed edges through differential computation. An optimal subnetting algorithm based on dynamic programming is executed on the change subgraph to allocate network nodes to different subnet partitions. A topology-aware scheduling model is used to calculate the subnetting scheme. Within each subnet partition, an improved Tarjan strongly connected component algorithm is initiated to detect loops and extract connection summary information from nodes crossing subnet boundaries. The distributed loop verification process involves marking the loop path and calculating the loop impact score when a loop topology is detected. Based on the score, it determines whether to perform topology reconstruction. ARP entries, routing entries, OpenFlow flow tables, and virtualization platform interface data are collected to construct a traffic matrix and calculate the latency correlation coefficient between nodes. A multi-layer association inference model is used to establish the mapping relationship between physical devices, virtual switches, and container networks. Power spectrum analysis of optical power monitoring points is performed on optical network devices to identify wavelength channel occupancy status and reconstruct the wavelength routing path of the DWDM system by combining label switching path information. A causal consistency protocol is used to perform a two-phase commit on the topology data collected by distributed probe nodes to synchronously generate a globally unified network topology graph.

2. The method according to claim 1, characterized in that, The distributed probe nodes collect information on device online events and link status changes using a hybrid strategy of passive SNMP Trap listening and active ICMP probe.

3. The method according to claim 2, characterized in that, Copy-on-write technology does not directly overwrite the original data when modifying it, but instead creates a copy of the data for modification, while the original data remains unchanged until all references end.

4. The method according to claim 3, characterized in that, Differential computation extracts changed nodes and edges by comparing snapshots of the old and new topologies, and performs subsequent processing only on the changed parts.

5. The method according to claim 4, characterized in that, The optimal subnet partitioning algorithm based on dynamic programming models the network partitioning problem as a multi-stage decision-making process. The state space is the set of allocated nodes, and the state transition corresponds to the operation of assigning a new node to a certain subnet partition. The algorithm defines a value function that accumulates the local costs of each stage. The cost includes the weighted sum of the communication traffic between nodes within the subnet partition and the cross-boundary traffic between subnet partitions. The algorithm uses memoization technology to store the calculated optimal solutions to subproblems to avoid redundant calculations. The backtracking process starts from the final state and generates a mapping scheme from nodes to subnet partitions along the optimal decision path.

6. The method according to claim 5, characterized in that, The topology-aware scheduling model adopts a feature fusion computing framework based on multi-head cross-attention. The model input includes the geographic location coordinates of the nodes, link bandwidth capacity, historical traffic statistics and device type identifiers, and the output is the subnet partition affiliation probability distribution of each node.

7. The method according to claim 6, characterized in that, The topology-aware scheduling model comprises a three-layer architecture: an input embedding layer, a multi-layer feature fusion layer, and an output decision layer. The input embedding layer maps the geographic location coordinates of nodes into a 128-dimensional spatial vector through a fully connected network, normalizes the link bandwidth capacity and historical traffic statistics and encodes them into a 64-dimensional vector, and converts the device type identifier into a 32-dimensional vector through one-hot encoding and concatenates them to form a 224-dimensional input feature. The multi-layer feature fusion layer contains four stacked multi-head cross-attention modules. Each multi-head cross-attention module calculates the association weights between different feature dimensions through linear transformations of the query matrix, key matrix, and value matrix.

8. The method according to claim 7, characterized in that, The number of attention heads in the multi-head cross-attention module is determined based on the average node degree of the current network. When the average node degree is less than 10, 4 attention heads are set; when the average node degree is between 10 and 30, 8 attention heads are set; and when the average node degree exceeds 30, 16 attention heads are set.

9. The method according to claim 8, characterized in that, The improved Tarjan strongly connected component algorithm adds node access order optimization and stack space compression mechanisms to the standard Tarjan algorithm. It reduces the recursion depth by prioritizing the access of nodes with larger out-degrees, and immediately outputs and releases the stack space after detecting a strongly connected component.

10. The method according to claim 9, characterized in that, Distributed loop verification is a loop confirmation process executed collaboratively by multiple distributed probe nodes. After exchanging connection summary information, each distributed probe node independently determines whether a cross-subnet partition loop exists, and confirms the existence of the loop through a majority voting mechanism.

Images