A network topology self-discovery management method and system in a distributed cloud environment

Abstract

The application relates to the technical field of distributed cloud, in particular to a network topology self-discovery management method and system under a distributed cloud environment, which comprises the following steps: each cloud center deploys a distributed detector to perform network equipment topology information self-discovery; the distributed detector converts network topology information into standard format data and pushes the standard format data to a cloud management center; the cloud management center platform stores network topology data into object storage and provides network topology real-time visual display and external API interfaces; the network topology self-discovery management method and system under the distributed cloud environment have the beneficial effects that each cloud data center pushes the obtained detailed topology data to the cloud management center through SDWAN, the cloud management center can centrally gather network equipment topology information from each cloud platform, and a comprehensive and accurate global view is formed.

Description

Technical Field

[0001] This invention relates to the field of distributed cloud technology, specifically to a method and system for self-discovery and management of network topology in a distributed cloud environment. Background Technology

[0002] As the scale of distributed cloud services continues to expand, the complexity of network architecture is also growing exponentially, which undoubtedly places more stringent demands on the efficient management and fine-grained control of the entire network. To improve overall network management efficiency and ensure higher levels of network stability, it is crucial to seamlessly synchronize real-time network topology information from each cloud data center to the cloud management center. Relying on manual methods to collect topology data from network devices within each cloud center and manually input it into the cloud management platform is extremely inefficient and difficult to adapt to the rapidly changing network environment of today. While SNMP (Simple Network Management Protocol) can serve as a solution, enabling automatic device discovery and status monitoring, this method requires all devices to have SNMP services enabled and correctly configured community strings or provide a consistent MIB (Management Information Base). In practical applications, this reliance may lead to some devices being unable to be effectively identified or have their configuration details fully obtained due to configuration issues.

[0003] LLDP (Link Layer Discovery Protocol) focuses on probing information about directly connected network devices. While it excels at topology discovery at the local connection layer, it is unable to reveal topological relationships across multiple hops or deep, complex network structures. Furthermore, if network device failures in cloud data centers fail to trigger timely alarm notifications containing detailed topology information, the speed and effectiveness of problem diagnosis and resolution will be severely impacted, threatening the overall stability of the network.

[0004] Therefore, building a management system that can accurately reflect the overall network situation and has intelligent alarm functions is particularly crucial for ensuring the healthy operation of the network in a distributed cloud environment. Summary of the Invention

[0005] The purpose of this invention is to provide a network topology self-discovery management method and system in a distributed cloud environment to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a network topology self-discovery management method in a distributed cloud environment, the method comprising the following steps:

[0007] Step 1: Deploy distributed detectors in each cloud center to perform self-discovery of network device topology information;

[0008] Step 2: Distributed detectors convert network topology information into standard format data and push it to the cloud management center;

[0009] Step 3: The cloud management center platform stores network topology data in object storage, providing real-time visualization of the network topology and external API interfaces;

[0010] Step 4: Each distributed detector uploads network status changes to the cloud management center in real time to ensure network topology consistency between the cloud management center and each cloud center;

[0011] Step 5: Each distributed detector periodically performs link probing on network devices to promptly detect link anomalies in the network.

[0012] Preferably, step one includes deploying distributed detectors in each cloud data center, performing automatic network topology discovery tasks according to a preset time interval based on a configuration file, and collecting network device information and link status data.

[0013] Preferably, step two includes storing network topology data in a standard format in a local database and uploading it to the object storage system of the cloud management center; the object storage system sets corresponding access permission policies according to different cloud centers to ensure data security and compliance.

[0014] Preferably, step three includes the management center implementing a network topology visualization display at the cloud center level, displaying the network structure and status of the cloud center in real time through a large screen; the cloud management center provides standard API interfaces to the outside world, enabling other services or systems to easily obtain and operate network topology information.

[0015] Preferably, step four includes each cloud center continuously monitoring the status changes of network devices and reporting them to the cloud management center in real time; when an event that meets the predefined alarm rules is detected, the cloud management center will automatically generate a work order and notify the corresponding front-line maintenance personnel to handle it, while updating the network topology visualization interface to ensure that the problem can be responded to in a timely manner; each cloud center detects link anomalies, converts the fault information into work orders and dispatches them to front-line personnel in order to quickly locate and resolve network connectivity problems.

[0016] A network topology self-discovery management method for a distributed cloud environment is provided. The network topology self-discovery management system for a distributed cloud environment comprises an information discovery module, a data push module, a display module, a data upload module, and a link detection module.

[0017] The information discovery module deploys distributed detectors in each cloud center to perform self-discovery of network device topology information;

[0018] The data push module, a distributed detector, converts network topology information into standard format data and pushes it to the cloud management center;

[0019] The display module stores network topology data in object storage on the cloud management center platform, providing real-time visualization of network topology and external API interfaces;

[0020] The data upload module allows each distributed detector to upload network status changes to the cloud management center in real time, ensuring the consistency of network topology between the cloud management center and each cloud center.

[0021] The link detection module uses distributed detectors to periodically detect link problems in network devices and promptly identify link anomalies in the network.

[0022] Preferably, the information discovery module deploys distributed detectors in each cloud data center, performs automatic network topology discovery tasks according to a preset time interval based on a configuration file, and collects network device information and link status data.

[0023] Preferably, in the data push module, network topology data is stored in a local database in a standard format and uploaded to the object storage system of the cloud management center; the object storage system sets corresponding access permission policies according to different cloud centers to ensure data security and compliance.

[0024] Preferably, the display module allows the management center to visualize the network topology at the cloud center level, displaying the network structure and status of the cloud center in real time on a large screen; the cloud management center provides standard API interfaces to the outside world, enabling other services or systems to easily obtain and operate network topology information.

[0025] Preferably, in the data upload module, each cloud center continuously monitors the status changes of network devices and reports them to the cloud management center in real time. When an event that meets the predefined alarm rules is detected, the cloud management center automatically generates a work order and notifies the corresponding front-line maintenance personnel to handle it. At the same time, it updates the network topology visualization interface to ensure that the problem can be responded to in a timely manner. Each cloud center detects link anomalies and converts the fault information into work orders to be dispatched to front-line personnel in order to quickly locate and resolve network connection problems.

[0026] Compared with the prior art, the beneficial effects of the present invention are:

[0027] This invention proposes a network topology self-discovery management method and system in a distributed cloud environment. Each cloud data center pushes detailed topology data to the cloud management center via SDWAN. The cloud management center then centrally aggregates network device topology information from various cloud platforms, forming a comprehensive and accurate global view. Detectors deployed in each cloud center continuously receive SNMP Trap messages from network device status changes and transmit the Trap data to the cloud management center via the SDWAN network, ensuring that the visualized network structure diagram displayed by the cloud management center is always synchronized with the actual network device status of each cloud center. The network topology alarm system in each cloud center can perform timed detection of link connectivity. When any anomaly is detected, the system quickly triggers an alarm mechanism and automatically assigns a work order to the frontline operations and maintenance team to ensure that network problems are handled promptly, thereby effectively preventing significant business impacts caused by network failures. In the immediate event of a network device failure in a cloud center, based on a pre-established network topology model and related information, the system can quickly locate the scope of the failure and its potential chain reactions, providing accurate and timely decision support for operations and maintenance personnel, facilitating rapid judgment and appropriate emergency measures. Attached Figure Description

[0028] Figure 1 This is a flowchart of the method of the present invention;

[0029] Figure 2 This is a schematic diagram of the detector principle of the present invention;

[0030] Figure 3 This is a flowchart of the network status active reporting process of the present invention;

[0031] Figure 4 This is a flowchart of the network topology alarm process of the present invention;

[0032] Figure 5 This is a flowchart of the ICMP protocol-based detection process of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the present invention clear and complete, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only some, not all, embodiments of the present invention, and are merely illustrative of the embodiments of the present invention. They are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Example 1

[0035] This invention provides a technical solution: a network topology self-discovery management method in a distributed cloud environment, the method comprising the following steps:

[0036] Step 1: Deploy distributed detectors in each cloud center to perform self-discovery of network device topology information; deploy distributed detectors in each cloud data center to perform automatic network topology discovery tasks according to the configuration file at preset time intervals, and collect network device information and link status data.

[0037] Step 2: The distributed detector converts the network topology information into standard format data and pushes it to the cloud management center; the network topology data is stored in a local database in standard format and uploaded to the object storage system of the cloud management center; the object storage system sets corresponding access permission policies according to different cloud centers to ensure data security and compliance.

[0038] Step 3: The cloud management center platform stores network topology data in object storage, providing real-time visualization of network topology and external API interfaces; the management center realizes network topology visualization at the cloud center level, displaying the network structure and status of the cloud center in real time through a large screen; the cloud management center provides standard API interfaces to the outside world, enabling other services or systems to easily obtain and manipulate network topology information.

[0039] Step 4: Each distributed detector uploads network status changes to the cloud management center in real time, ensuring network topology consistency between the cloud management center and each cloud center; each cloud center continuously monitors network device status changes and reports them to the cloud management center in real time; when an event matching predefined alarm rules is detected, the cloud management center automatically generates a work order and notifies the relevant frontline operations and maintenance personnel for handling, while updating the network topology visualization interface to ensure timely response to problems; each cloud center detects link anomalies, converts fault information into work orders, and dispatches them to frontline personnel for rapid location and resolution of network connectivity issues.

[0040] Step 5: Each distributed detector periodically performs link probing on network devices to promptly detect link anomalies in the network.

[0041] Example 2

[0042] Based on Example 1, the overall process is described below:

[0043] 1. Highly efficient distributed detection systems have been successfully deployed in various cloud data centers.

[0044] 2. The LLDP and SNMP protocols have been successfully enabled on the managed network devices.

[0045] 3. As the scheduled task starts, the detector uses an automatic discovery mechanism combining LLDP and SNMP protocols to deeply explore the network device topology and accurately converts the data format according to a preset template, thus successfully completing the collection of network device topology information. For detailed technical implementation information, please refer to section 4.2.

[0046] 4. Based on the SDWAN network architecture, the cloud center detector will promptly push the network topology information it acquires in real time to the message middleware Kafka for storage and transfer.

[0047] 5. As the core processing node, the cloud management center actively consumes message streams from Kafka, meticulously parses the received network device details data, and after a series of careful processing and integration, securely stores it in a highly available object storage system.

[0048] 6. The cloud management center further utilizes visualization technology to display detailed and intuitive network topology diagrams divided by each cloud data center, providing operations and maintenance personnel with a clear and three-dimensional view of the network structure.

[0049] 7. To facilitate the integration and use of third-party systems, the cloud management center provides a series of standardized API interfaces, enabling external services to easily obtain the latest network topology information.

[0050] See the overall flowchart Figure 1 Overall process description

[0051] Network device topology detection core implementation description:

[0052] See the core process for network device topology detection. Figure 2 The following is a detailed implementation description:

[0053] 1. After the detectors deployed in each cloud center start their scheduled tasks, they first load and parse a preset configuration file. The configuration file contains the parameters required to connect to the device, such as device type, IP address, username, and password.

[0054] 2. The detector uses the ConnectHandler function of the netmiko library to connect to the device and uses the send_command function to execute commands to obtain device information.

[0055] 3. Use regular expressions to parse the device's LLDP information and add neighboring device and port information to the LLDP list.

[0056] 4. The detector uses the networkx library to build a topology graph data structure. It iterates through the LLDP list from step three, treating neighboring devices and ports as nodes and adding edges to represent the relationships between them.

[0057] 4. The detector uses the SNMP protocol to send GET or GETNEXT query requests to all devices in the network that support the protocol, thereby obtaining relevant data such as the type of these devices, interface status, IP and MAC addresses, and subnetting, and thus gaining a comprehensive understanding of the working status and logical connection relationships of the devices.

[0058] 5. Deeply integrate and analyze the data collected based on SNMP and LLDP protocols, extract the neighbor relationships established between devices through LLDP, and construct a detailed and accurate network topology map based on this.

[0059] 6. The cloud center will display the constructed network topology visualization information in real time, intuitively reflecting the entire network architecture.

[0060] 7. Frontline operations and maintenance personnel in the cloud center need to carefully check the generated network topology information to determine whether the topology obtained by automatic discovery is complete and intact.

[0061] a) If the automatic discovery results meet the requirements, the automatically generated network topology information will be stored in a graph database specifically used to store complex relational data;

[0062] b) If there are deficiencies or non-compliance with specific standards, the operations and maintenance personnel will continue to perform subsequent steps to improve the network topology information.

[0063] 8. Operation and maintenance personnel manually customize and optimize network topology information according to actual conditions to ensure its accuracy, and finally update the manually improved network topology information to the graph database for further management and analysis.

[0064] Instructions for Real-Time Network Device Status Reporting Process:

[0065] See the real-time reporting process for network device status. Figure 3 The following is a detailed implementation description:

[0066] 1. Distributed detection systems are deployed in each cloud data center.

[0067] 2. When any change occurs in the status of the managed network device, the device will proactively send a Trap notification message to the detector via the SNMP protocol.

[0068] 3. When the distributed detector receives a Trap message, it will parse the message content according to pre-set rules and save it to a local database for further processing.

[0069] 4. After the parsing is completed, the distributed detector will push alarm or status change information to the cloud management center to ensure data synchronization.

[0070] 5. After receiving the real-time status reports from network devices, the cloud management center parses and uploads them to the object storage system for subsequent analysis.

[0071] 6. Based on the status of network devices, the cloud management center updates the visualization of their network topology information in real time, intuitively reflecting the status changes of the entire network architecture.

[0072] 7. Based on the preset business strategies and alarm triggering rules, the cloud management center system will make a judgment:

[0073] a) If the current change meets the preset alarm triggering conditions, the system determines that the network status has become abnormal, and then automatically generates the corresponding alarm event and quickly assigns the work order to the front-line operation and maintenance personnel for handling.

[0074] b) If the current state change does not meet any preset rules, the system will consider it to be a change within the normal fluctuation range and therefore ignore the event to avoid false alarms and waste of resources.

[0075] Network topology alarm process description:

[0076] See the real-time reporting process for network device status. Figure 4 The following is a detailed implementation description:

[0077] 1. Deploy distributed detector modules in each cloud data center.

[0078] 2. After the scheduled task starts, the detector module will actively connect to and read the information stored in the network topology database.

[0079] 3. The detector acquires link status data throughout the network.

[0080] 4. For each link information, the system repeatedly calls the ICMP-based probe function to perform connectivity checks one by one. For details on the implementation principle, please refer to Part 4.5 regarding ICMP-based connectivity detection technology.

[0081] a) If the link is abnormal, the detector will assemble an alarm message according to the predefined rule set and record it in the local database, while triggering the subsequent processing flow.

[0082] b) If the link is normal, the detector continues to perform the same detection process on the next link until the detection of all links in the network is completed.

[0083] 5. After the detection is completed, the alarm information is pushed to the cloud management center in real time through the SDWAN network.

[0084] 6. After receiving the alarm information, the cloud management center parses it and stores the alarm details in the object storage system to ensure the integrity and traceability of historical records.

[0085] 7. Based on the received alarm content, the cloud management center updates the network topology view in real time, intuitively displaying the latest status of the affected areas and the overall network.

[0086] 8. Based on the specific content and priority of the alarm information, the cloud management center will automatically dispatch work orders to the front-line operation and maintenance team of the cloud service center to guide them to respond quickly and resolve related issues.

[0087] ICMP protocol-based detection process description:

[0088] See the real-time reporting process for network device status. Figure 5 The following is a detailed implementation description:

[0089] 1. Read the configuration file. The main contents of the configuration file include the timeout period and the maximum number of probes.

[0090] 2. Set the number of probes to 0. Execute the ping command.

[0091] 3. Determine if the command execution time has expired.

[0092] 1) If the timeout occurs, increment the number of probes by 1 and continue to step 5.

[0093] 2) If the timeout has not occurred, proceed to the next step.

[0094] 4. The system determines whether the ping is successful based on the command execution result.

[0095] 1) If the ping fails, increment the probe count by 1 and proceed to the next step.

[0096] 2) If the ping is successful, the test result is considered normal.

[0097] 5. The system determines whether the number of detection attempts exceeds the set limit.

[0098] 1) If the detection timeout occurs, the detection round ends and the detection result is judged to be abnormal.

[0099] 2) If the number of probe attempts has not expired, continue probing the link.

[0100] Example 3

[0101] Based on Example 2, a network topology self-discovery management system in a distributed cloud environment is proposed. The system consists of an information discovery module, a data push module, a display module, a data upload module, and a link detection module.

[0102] The information discovery module deploys distributed detectors in each cloud center to perform self-discovery of network device topology information; it also deploys distributed detectors in each cloud data center to perform automatic network topology discovery tasks according to the configuration file at preset time intervals, collecting network device information and link status data.

[0103] The data push module, a distributed detector, converts network topology information into standard format data and pushes it to the cloud management center. The network topology data is stored in a local database in a standard format and then uploaded to the object storage system of the cloud management center. The object storage system sets corresponding access permission policies according to different cloud centers to ensure data security and compliance.

[0104] The display module stores network topology data in object storage on the cloud management center platform, providing real-time visualization of network topology and external API interfaces. The management center implements network topology visualization at the cloud center level, displaying the network structure and status of the cloud center in real time on a large screen. The cloud management center provides standard API interfaces to the outside world, enabling other services or systems to easily obtain and manipulate network topology information.

[0105] The data upload module allows each distributed detector to upload network status changes to the cloud management center in real time, ensuring network topology consistency between the cloud management center and each cloud center. Each cloud center continuously monitors the status changes of network devices and reports them to the cloud management center in real time. When an event matching predefined alarm rules is detected, the cloud management center automatically generates a work order and notifies the relevant frontline operations and maintenance personnel for handling, while updating the network topology visualization interface to ensure timely response to problems. Each cloud center detects link anomalies, converts fault information into work orders, and dispatches them to frontline personnel for rapid location and resolution of network connectivity issues.

[0106] The link detection module uses distributed detectors to periodically detect link problems in network devices and promptly identify link anomalies in the network.

[0107] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for self-discovery and management of network topology in a distributed cloud environment, characterized in that: The method includes the following steps: Step 1: Deploy distributed detectors in each cloud center to perform self-discovery of network device topology information; Step 2: Distributed detectors convert network topology information into standard format data and push it to the cloud management center; Step 3: The cloud management center platform stores network topology data in object storage, providing real-time visualization of the network topology and external API interfaces; Step 4: Each distributed detector uploads network status changes to the cloud management center in real time to ensure network topology consistency between the cloud management center and each cloud center; Step 5: Each distributed detector periodically performs link probing on network devices to promptly detect link anomalies in the network; Step two includes storing network topology data in a standard format in a local database and uploading it to the object storage system of the cloud management center; the object storage system sets corresponding access permission policies according to different cloud centers to ensure data security and compliance; Step four includes each cloud center continuously monitoring the status changes of network devices and reporting them to the cloud management center in real time; when an event that meets the predefined alarm rules is detected, the cloud management center will automatically generate a work order and notify the relevant front-line maintenance personnel to handle it, while updating the network topology visualization interface to ensure that the problem can be responded to in a timely manner; each cloud center detects link anomalies, converts the fault information into work orders and dispatches them to front-line personnel in order to quickly locate and resolve network connectivity problems.

2. The network topology self-discovery management method in a distributed cloud environment according to claim 1, characterized in that: Step one involves deploying distributed detectors in various cloud data centers to perform automatic network topology discovery tasks according to the configuration file at preset time intervals, and collecting network device information and link status data.

3. The network topology self-discovery management method in a distributed cloud environment according to claim 1, characterized in that: Step 3 includes enabling the management center to visualize the network topology at the cloud center level, displaying the network structure and status of the cloud center in real time on a large screen; the cloud management center provides standard API interfaces to the outside world, enabling other services or systems to easily obtain and operate network topology information.

4. A network topology self-discovery management system in a distributed cloud environment according to any one of claims 1-3, characterized in that: The system comprises an information discovery module, a data push module, a display module, a data upload module, and a link detection module; The information discovery module deploys distributed detectors in each cloud center to perform self-discovery of network device topology information; The data push module, a distributed detector, converts network topology information into standard format data and pushes it to the cloud management center; The display module stores network topology data in object storage on the cloud management center platform, providing real-time visualization of network topology and external API interfaces; The data upload module allows each distributed detector to upload network status changes to the cloud management center in real time, ensuring the consistency of network topology between the cloud management center and each cloud center. The link detection module uses distributed detectors to periodically detect link problems in network devices and promptly identify link anomalies in the network.

5. A network topology self-discovery management system in a distributed cloud environment according to claim 4, characterized in that: The information discovery module deploys distributed detectors in each cloud data center and performs automatic network topology discovery tasks according to the configuration file at preset time intervals, collecting network device information and link status data.

6. A network topology self-discovery management system in a distributed cloud environment according to claim 5, characterized in that: The data push module stores network topology data in a standard format in a local database and uploads it to the object storage system of the cloud management center. The object storage system sets corresponding access permission policies according to different cloud centers to ensure data security and compliance.

7. A network topology self-discovery management system in a distributed cloud environment according to claim 6, characterized in that: The display module enables the management center to visualize the network topology at the cloud center level, displaying the network structure and status of the cloud center in real time on a large screen. The cloud management center provides standard API interfaces to the outside world, allowing other services or systems to easily obtain and manipulate network topology information.

8. A network topology self-discovery management system in a distributed cloud environment according to claim 7, characterized in that: The data upload module continuously monitors the status changes of network devices in each cloud center and reports them to the cloud management center in real time. When an event that meets the predefined alarm rules is detected, the cloud management center automatically generates a work order and notifies the relevant front-line maintenance personnel to handle it. At the same time, it updates the network topology visualization interface to ensure that the problem can be responded to in a timely manner. Each cloud center detects link anomalies and converts the fault information into work orders to be dispatched to front-line personnel in order to quickly locate and resolve network connectivity problems.

Images