A method and system for monitoring complex network devices

Abstract

This invention relates to a method and system for monitoring complex network devices, belonging to the field of network device monitoring technology. The invention employs a distributed device monitoring architecture, distributing device monitoring tasks across multiple nodes. A load balancing strategy is used to allocate monitoring tasks, ensuring high availability and scalability of the system. Simultaneously, since the data responded by the protocol adaptation layer uses a unified data model, data processing can be performed on the monitoring nodes to remove erroneous data, convert it into a data model that meets the requirements of the data table, and analyze the collected data before storing it in the database, ensuring data real-time performance and reliability. The protocol adaptation layer proposed in this invention is compatible with multiple protocol types, reducing coupling between protocols and improving compatibility and flexibility. Through a dual-cluster deployment, the cluster size is dynamically adjusted according to the number of devices, enhancing data acquisition capabilities and reliability.

Description

Technical Field

[0001] This invention belongs to the field of network device monitoring technology, specifically relating to a method and system for monitoring complex network devices. Background Technology

[0002] With the rapid development of information technology and the Internet, the types and numbers of network devices are increasing daily, and the complexity of network systems is also constantly rising. To ensure the normal operation and security of networks, network device monitoring technology has become an important research direction. Network device monitoring technology mainly includes real-time monitoring and management of devices such as routers, switches, servers, and firewalls. By collecting the operating status and performance data of these devices, analyzing and processing this data, potential problems can be identified and resolved in a timely manner, thereby ensuring the efficient, stable, and secure operation of the network.

[0003] Currently, network device monitoring technologies mainly include the following:

[0004] (1) Simple Network Management Protocol (SNMP)

[0005] SNMP is one of the most widely used network management protocols. It collects various operational information from network devices through SNMP agents and transmits this information to the SNMP management system for processing and analysis. SNMP's main advantages lie in its simplicity and ease of implementation. Network administrators can use SNMP to obtain device status information, performance data, configuration parameters, etc., thereby achieving device monitoring and management. However, SNMP also has some drawbacks, such as its relatively poor security, making it vulnerable to attacks. Furthermore, SNMP may encounter performance bottlenecks when processing large amounts of device data.

[0006] (2) Monitoring technology based on traffic analysis

[0007] Traffic analysis-based monitoring technologies monitor device status and performance by analyzing network traffic. Representative tools in this category include NetFlow and sFlow. NetFlow, proposed by Cisco, collects and analyzes network traffic packets to provide detailed insights into network usage, detect abnormal traffic, and identify potential security threats. sFlow is a sampling technique that provides real-time network performance monitoring through sampling and analysis of network traffic. The advantage of traffic analysis technologies lies in their ability to provide detailed network usage and traffic patterns, helping to identify abnormal behavior and potential problems within the network. However, these technologies are complex to implement and require significant computational and storage resources.

[0008] (3) Log analysis technology

[0009] Log analysis technology collects and analyzes log files generated by network devices to obtain detailed information about device operation, including error messages and access records. Commonly used log analysis tools include Syslog and Splunk. Syslog is a standard log transmission protocol widely used in various network devices. Splunk is a powerful log analysis platform capable of real-time analysis and processing of large amounts of log data. Log analysis technology can help network administrators promptly identify and locate faults, improving network maintainability and reliability. Its drawbacks include the large volume of log data and the complexity of analysis and processing, especially when dealing with massive amounts of log data, requiring efficient analysis algorithms and powerful computing capabilities.

[0010] Monitoring various types of network devices, such as routers, switches, servers, and firewalls, in a complex network environment is a significant challenge. These devices typically support different protocols, such as ICMP, HTTP, HTTPS, SNMP, and Syslog, and their data formats and communication mechanisms vary considerably. Furthermore, the sheer volume of status and performance data from these network devices presents a challenge in terms of real-time, efficient, and reliable data collection and processing. The following are the main technical difficulties currently faced in building network device monitoring systems:

[0011] (1) Multi-protocol compatibility issues: With the rapid development of information technology and the Internet, the types of network devices are constantly increasing, and the protocols of different types of devices are different. This poses a huge challenge to device monitoring. Whenever a new protocol device is connected to the network, the system has to be re-adapted, which greatly increases the cost of system maintenance. How to be compatible with and process data of multiple protocols on a unified platform will be a very tricky problem.

[0012] (2) Real-time performance and reliability of data acquisition: SNMP may encounter performance bottlenecks when processing a large amount of device data. Monitoring and log analysis technologies based on traffic analysis have high computation and storage requirements. As the number of devices increases, the real-time performance of data acquisition cannot be guaranteed, which may lead to data loss or delay. Summary of the Invention

[0013] (a) Technical problems to be solved

[0014] The technical problem to be solved by this invention is how to provide a method and system for monitoring complex network devices to solve problems such as multi-protocol compatibility, data real-time performance and reliability.

[0015] (II) Technical Solution

[0016] To address the aforementioned technical problems, this invention proposes a complex network device monitoring system, which includes: a system main service, a monitoring task scheduler, monitoring nodes, a protocol adaptation scheduler, and a protocol adapter.

[0017] The main system service is the primary service of the monitoring system. Located in the situation display layer, it provides data interfaces to the situation display page and issues monitoring tasks to the monitoring task scheduler. As the power source of the system, the main system service drives the monitoring system to operate normally. Once it crashes, other services will also stop running.

[0018] The monitoring task scheduler, located in the monitoring service layer, is a component used to allocate and schedule monitoring nodes. The monitoring task scheduler records the status of monitoring nodes and monitoring tasks, and allocates and schedules monitoring tasks for monitoring nodes based on the real-time status, so that monitoring tasks can be processed efficiently.

[0019] The monitoring node, located in the monitoring service layer, is a service component for processing device status data. The data collected by the protocol adapter is processed by the monitoring node, and the processed data is saved to the database.

[0020] The protocol adapter scheduler, located in the protocol adapter layer, is used to allocate and schedule protocol adapters. The protocol adapter scheduler records the status of the protocol adapter and the status of the protocol adapter request. Based on the real-time status, it allocates and schedules the protocol adapters required by the monitoring nodes, so that the protocol adapters are used efficiently.

[0021] Protocol adapters, located in the protocol adaptation layer, are used to connect to network devices in the network device layer and obtain status data. Protocol adapters connect to specified devices through a specified protocol, obtain status data, and respond to the monitoring node; each protocol adapter can only connect to one protocol.

[0022] This invention also provides a method for monitoring complex network devices, the method comprising the following steps:

[0023] S1. The system main service initiates a device monitoring task. When initiating a device monitoring task, the task parameters must include at least the unique identifier of the network device and the monitoring protocol; otherwise, the device monitoring task cannot be initiated. The system main service distributes the device monitoring task to the monitoring task scheduler. After receiving the monitoring task, the monitoring task scheduler saves the current monitoring task to the monitoring task list.

[0024] S2. The monitoring task scheduler checks if there are any other unassigned monitoring tasks before the current monitoring task. If so, it assigns the previous monitoring tasks first until all tasks before the current task have been assigned. Then, it starts assigning the current monitoring task. After the monitoring task is assigned, the monitoring task scheduler checks if there are any idle nodes in the monitoring node cluster. If not, the current task is in a waiting state until an idle monitoring node appears. The monitoring task scheduler then assigns the current monitoring task to the idle monitoring node, and the assignment of the current monitoring task is completed.

[0025] S3. After receiving a monitoring task, the monitoring node requests a protocol adapter for the specified protocol from the protocol adapter scheduler based on the monitoring protocol parameters in the task data. After receiving the request, the protocol adapter scheduler adds the request to the request list.

[0026] S4. The protocol adaptation scheduler checks whether there are any unprocessed requests with the same protocol before this request. If so, it processes the previous requests with the same protocol first, until all requests with the same protocol before this request have been processed, and then starts processing this request.

[0027] S5. After the protocol adapter scheduler starts processing this request, it checks whether there is an idle adapter with the same protocol in the protocol adapter cluster. If not, the request will wait until an idle protocol adapter appears. The protocol adapter scheduler will then assign the idle protocol adapter to the monitoring node. The monitoring node will obtain network device status data by calling the data interface provided by the protocol adapter and perform data processing. The monitoring task will then be completed.

[0028] (III) Beneficial Effects

[0029] This invention proposes a method and system for monitoring complex network devices, which achieves the following objectives:

[0030] (1) Multi-protocol compatibility: The system can be compatible with and process data from multiple protocols, reducing complexity and maintenance costs.

[0031] (2) Real-time performance and reliability: Real-time and efficient data acquisition and processing are achieved, ensuring data integrity and real-time performance.

[0032] (3) Intuitive visualization: Rich charts and customizable views improve the ease of use of the system and help users quickly understand and analyze data. Attached Figure Description

[0033] Figure 1 This is a diagram of the architecture of the present invention;

[0034] Figure 2 This is a schematic diagram of the system architecture of the present invention;

[0035] Figure 3 This is a flowchart of the device monitoring process of the present invention;

[0036] Figure 4 This is a schematic diagram of the protocol adaptation layer.

[0037] Figure 5 Data processing flowchart for monitoring services;

[0038] Figure 6 This is a system functional block diagram of the present invention. Detailed Implementation

[0039] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0040] This invention proposes a method and system for monitoring complex network devices, aiming to solve problems such as multi-protocol compatibility, data real-time performance, and reliability.

[0041] (1) Multi-protocol compatibility

[0042] The system designed in this invention can be compatible with and process data from multiple protocols. Even if a device with a new protocol is connected to the network, it only needs to convert the data of that protocol into a unified data format that meets the requirements of this system to complete the adaptation, thereby reducing the complexity and maintenance cost of the system.

[0043] (2) Real-time performance and reliability

[0044] This invention significantly improves data processing capabilities by distributing data processing nodes and balancing processing tasks, achieving real-time and efficient data acquisition and processing, and ensuring data integrity and real-time performance.

[0045] (3) Intuitive visual display

[0046] The complex network device monitoring system designed in this invention provides an intuitive visualization page, rich charts and customizable views, which improves the ease of use of the system and helps users quickly understand and analyze data.

[0047] (I) Overall Framework

[0048] This invention designs a complex network device monitoring system that supports distributed network device monitoring using multiple protocols. The solution divides the complex network device monitoring framework into four layers, such as... Figure 1 As shown:

[0049] The lowest layer is the network device layer, which is the infrastructure that constitutes the network environment, including routers, switches, servers, firewalls, etc.

[0050] The next layer is the protocol adaptation layer, which consists of various different protocol adapters. Each protocol adapter is independent of the others, operates independently, and does not affect the others.

[0051] The next layer above is the monitoring service layer, which consists of multiple monitoring nodes and is responsible for cleaning, transforming, analyzing, and storing the data.

[0052] The top layer is the situation display layer, which can display the changing trends of performance indicators, network device topology relationships, abnormal alarm information, etc.

[0053] (II) System Architecture

[0054] Architecture such as Figure 2 As shown, the service components included in the complex network device monitoring system are: system main service, monitoring task scheduler, monitoring node, protocol adapter scheduler, protocol adapter, etc.

[0055] The main system service is the primary service of the monitoring system, located in the situation display layer. It provides data interfaces to the situation display page and issues monitoring tasks to the monitoring task scheduler. As the power source of the system, the main system service drives the monitoring system to operate normally. Once it crashes, other services will also stop running.

[0056] The monitoring task scheduler, located in the monitoring service layer, is a component used to allocate and schedule monitoring nodes. The monitoring task scheduler records the status of monitoring nodes and monitoring tasks, and allocates and schedules monitoring tasks for monitoring nodes based on the real-time status, so that monitoring tasks can be processed efficiently.

[0057] The monitoring node, located in the monitoring service layer, is a service component for processing device status data. The data collected by the protocol adapter is processed by the monitoring node, and the processed data is saved to the database.

[0058] The protocol adapter scheduler, located in the protocol adapter layer, is used to allocate and schedule protocol adapters. The protocol adapter scheduler records the status of protocol adapters and the status of protocol adapter requests. Based on the real-time status, it allocates and schedules the protocol adapters required by the monitoring nodes, so that the protocol adapters can be used efficiently.

[0059] Protocol adapters, located in the protocol adaptation layer, are used to connect to network devices in the network device layer and obtain status data. Protocol adapters connect to specified devices through a specified protocol, obtain status data, and respond to the monitoring node; each protocol adapter can only connect to one protocol.

[0060] The system employs a dual-cluster deployment architecture consisting of a monitoring node cluster and a protocol adapter cluster. The monitoring process is driven by the main system service, which initiates monitoring tasks based on the monitoring task scheduler. Upon receiving a monitoring task, the monitoring task scheduler allocates it to monitoring nodes according to the status of the monitoring node cluster. If the size of the monitoring node cluster cannot meet the monitoring task requirements in a timely manner, the cluster can be dynamically expanded to add more monitoring nodes. After receiving a monitoring task, a monitoring node requests a protocol adapter from the protocol adapter scheduler. The scheduler allocates a protocol adapter for the specified protocol to the monitoring node based on the status of the protocol adapter cluster, enabling data collection from network devices. If no protocol adapter is available, an adapter for the specified protocol can be added to meet the network device monitoring task. If no protocol adapter is available for the specified protocol, it can be adapted individually, and the adapted adapter can be registered with the protocol adapter scheduler.

[0061] The distributed device monitoring architecture of this invention significantly improves the system's data acquisition capabilities and reliability, ensures the real-time performance and integrity of the data, and avoids data loss and delays.

[0062] (III) Monitoring Process

[0063] This invention also provides a method for monitoring complex network devices, such as... Figure 3 As shown, the method includes the following steps:

[0064] S1. The system main service initiates a device monitoring task. When initiating a device monitoring task, the task parameters must include at least the unique identifier of the network device and the monitoring protocol; otherwise, the device monitoring task cannot be initiated. The system main service distributes the device monitoring task to the monitoring task scheduler. After receiving the monitoring task, the monitoring task scheduler saves the current monitoring task to the monitoring task list.

[0065] S2. The monitoring task scheduler checks if there are any other unassigned monitoring tasks before the current monitoring task. If so, it assigns the previous monitoring tasks first until all tasks before the current task have been assigned. Then, it starts assigning the current monitoring task. After the monitoring task is assigned, the monitoring task scheduler checks if there are any idle nodes in the monitoring node cluster. If not, the current task is in a waiting state until an idle monitoring node appears. The monitoring task scheduler then assigns the current monitoring task to the idle monitoring node, and the assignment of the current monitoring task is completed.

[0066] S3. After receiving a monitoring task, the monitoring node requests a protocol adapter for the specified protocol from the protocol adapter scheduler based on the monitoring protocol parameters in the task data. After receiving the request, the protocol adapter scheduler adds the request to the request list.

[0067] S4. The protocol adaptation scheduler checks whether there are any unprocessed requests with the same protocol before this request. If so, it processes the previous requests with the same protocol first, until all requests with the same protocol before this request have been processed, and then starts processing this request.

[0068] S5. After the protocol adapter scheduler starts processing this request, it checks whether there is an idle adapter with the same protocol in the protocol adapter cluster. If not, the request will wait until an idle protocol adapter appears. The protocol adapter scheduler will then assign the idle protocol adapter to the monitoring node. The monitoring node will obtain network device status data by calling the data interface provided by the protocol adapter and perform data processing. The monitoring task will then be completed.

[0069] (iv) Protocol Adaptation

[0070] In the equipment monitoring process, acquiring equipment status data is the most crucial step. In today's complex network environment, monitoring various network devices is particularly important. With the increasing variety and number of devices, different devices communicate using different protocols, posing challenges to network monitoring. To address the issue of multi-protocol compatibility, this invention designs an innovative protocol adaptation layer to convert data from different protocols into a unified data format, facilitating centralized management and monitoring.

[0071] The protocol adaptation layer can be viewed as an intermediate layer, located between the network device layer and the monitoring service layer. It receives data packets from various devices, parses and converts them into a unified format, and then passes the data to the monitoring service layer.

[0072] The working principle of the protocol adaptation layer is as follows: Figure 4 As shown. Before starting, the protocol adapter needs to be configured with relevant parameters, such as supported protocols and the IP address of the protocol adapter scheduler. After starting, the protocol adapter will automatically connect to the protocol adapter scheduler based on the configured IP address and send a registration request. After receiving the registration request from the protocol adapter, the protocol adapter scheduler will save the connected protocol adapter information to the protocol adapter registry. When saving the protocol adapter information, the protocol adapter scheduler will group the protocol adapters according to the protocol type to reduce the retrieval time during allocation.

[0073] After receiving the assigned task, the monitoring node sends a request to the protocol adapter scheduler to obtain a protocol adapter. Upon receiving the request, the scheduler does not immediately assign a protocol adapter to the monitoring node; instead, it saves the request to the protocol adapter request table. In the protocol adapter request table, the scheduler retrieves the corresponding protocol type from the request, groups the requests according to the protocol type, and queues each group of protocol adapter requests according to the request time, with each new request placed at the end of the queue. After saving, the scheduler checks if there is an idle protocol adapter with the same protocol in the protocol adapter registry. If so, it assigns the protocol adapter to the first request in the queue of the same protocol request in the protocol adapter request table and marks the protocol adapter's status as "working." This process continues until a new protocol adapter request or an idle protocol adapter appears, at which point the scheduler searches and assigns the adapter.

[0074] After being assigned to a designated monitoring node, the protocol adapter sends a device monitoring request to the protocol adapter, which includes the network device's connection information in the request parameters. Upon receiving the connection information, the protocol adapter establishes a protocol connection and data exchange with the designated device, obtains status information, converts the protocol data into a unified data type, and responds to the monitoring node. After obtaining the device status data, the protocol adapter sends a status recovery request to the protocol adapter scheduler again. Upon receiving the request, the protocol adapter scheduler restores the device's status to "idle," and the protocol adapter completes one device monitoring task.

[0075] The protocol adaptation layer greatly simplifies network device monitoring in multi-protocol environments by providing a unified data interface. This solution not only improves system compatibility and flexibility but also lays a solid foundation for future expansion and maintenance. Through this innovative design, network administrators can more efficiently monitor and manage a variety of devices in complex network environments.

[0076] (V) Data Processing

[0077] This invention adopts a distributed device monitoring architecture, distributing device monitoring tasks across multiple nodes and allocating monitoring tasks through a load balancing strategy to ensure high availability and scalability of the system. At the same time, since the data responded by the protocol adaptation layer is data of a unified data model, the data can be processed on the monitoring nodes to clean up erroneous data, convert it into a data model that meets the requirements of the data table, analyze the collected data, and save it to the database, ensuring the real-time performance and reliability of the data.

[0078] The data processing flow of the monitoring node is as follows: Figure 5As shown, after receiving device status data based on a unified data model from the protocol adapter, the monitoring node first cleans the data and verifies its legality, such as checking for null device unique identifiers or duplicate IP addresses. If the data is deemed invalid, it is discarded, and the process ends. If the data meets the verification rules, the monitoring node converts it into data that conforms to different database table structures, such as device basic data, device CPU data, and device memory data. Based on the converted data, the monitoring node analyzes whether various device parameters meet alarm conditions, such as whether the device is offline, whether the CPU utilization exceeds the threshold, and whether the memory utilization exceeds the threshold. If the device parameters meet the alarm conditions, alarm data is generated, such as device offline alarm, device CPU alarm, and device memory alarm. Finally, the converted device status data and device alarm data are saved to the database for system display and data auditing.

[0079] (VI) System Design

[0080] Based on the above complex network device monitoring solutions, this invention designs a complex network device monitoring system to facilitate users' monitoring and maintenance of network devices. The system functional architecture is as follows: Figure 6 As shown, the system is divided into five major functional modules: homepage status display, device asset management, network device topology, device alarm reminders, and system configuration management.

[0081] The homepage status display module is used to summarize and display the status of network devices. It provides visual statistics through rich charts and dashboards, and can display information such as the top 5 alarm devices, unprocessed alarms, asset overview, and abnormal device statistics.

[0082] The equipment asset management module is used for maintaining the equipment asset list. In addition to maintaining the basic information of the equipment list, it can also view the operating status information of key performance indicators of the equipment, such as CPU utilization and memory utilization curves. It also has remote operation and maintenance functions for the equipment, supporting remote connection protocols such as SSH and TELNET, improving the efficiency of users' operation and maintenance of remote network equipment.

[0083] The network device topology module is used to dynamically generate the topology of network devices and monitor data traffic in the network.

[0084] The device alarm notification module is used for alarm functions for abnormal information of network devices. The system classifies alarm information into categories such as abnormal performance indicators, device offline alarms, and unauthorized device access alarms, and performs statistical analysis by type and level. It also supports the function of processing alarm information and records the user's processing operations for device alarm information.

[0085] The system configuration management module is used to manage the configuration of system-related parameters. Monitoring rule settings configure the rules for issuing monitoring tasks, enabling functions such as pausing, starting, and controlling the issuance of monitoring tasks. Alarm threshold settings allow for setting thresholds for different types of performance parameters and different levels of abnormal alarms; for example, setting a general-level CPU utilization alarm threshold and a critical-level memory utilization alarm threshold. The data acquisition log management module provides functions for querying and managing device status data acquisition logs.

[0086] The complex network device monitoring system of the present invention uses rich visualization methods to enable users to intuitively see the operating status and performance data of network devices, quickly identify problems, and improve the system's flexibility and ease of use.

[0087] Key points of this invention:

[0088] This invention designs a method and system for monitoring complex network devices, which is suitable for monitoring scenarios with a large number of complex network devices and diverse protocols. Through the protocol adaptation layer proposed in this invention, it is compatible with multiple protocol types, reduces the coupling between protocols, and improves the compatibility and flexibility of protocols. Through the dual-cluster deployment method, the cluster size is dynamically adjusted according to the number of devices, thereby improving data acquisition capabilities and reliability.

[0089] Effects of the invention:

[0090] The complex network device monitoring method and system of the present invention achieve the following objectives:

[0091] (1) Multi-protocol compatibility: The system can be compatible with and process data from multiple protocols, reducing complexity and maintenance costs.

[0092] (2) Real-time performance and reliability: Real-time and efficient data acquisition and processing are achieved, ensuring data integrity and real-time performance.

[0093] (3) Intuitive visualization: Rich charts and customizable views improve the ease of use of the system and help users quickly understand and analyze data.

[0094] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A complex network device monitoring system, characterized in that, The system includes: main system service, monitoring task scheduler, monitoring nodes, protocol adaptation scheduler, and protocol adapter; The main system service is the primary service of the monitoring system. Located in the situation display layer, it provides data interfaces to the situation display page and issues monitoring tasks to the monitoring task scheduler. As the power source of the system, the main system service drives the monitoring system to operate normally. Once it crashes, other services will also stop running. The monitoring task scheduler, located in the monitoring service layer, is a component used to allocate and schedule monitoring nodes. The monitoring task scheduler records the status of monitoring nodes and monitoring tasks, and allocates and schedules monitoring tasks for monitoring nodes based on the real-time status, so that monitoring tasks can be processed efficiently. The monitoring node, located in the monitoring service layer, is a service component for processing device status data. The data collected by the protocol adapter is processed by the monitoring node, and the processed data is saved to the database. The protocol adapter scheduler, located in the protocol adapter layer, is used to allocate and schedule protocol adapters. The protocol adapter scheduler records the status of the protocol adapter and the status of the protocol adapter request. Based on the real-time status, it allocates and schedules the protocol adapters required by the monitoring nodes, so that the protocol adapters are used efficiently. Protocol adapters, located in the protocol adaptation layer, are used to connect to network devices in the network device layer and obtain status data. Protocol adapters connect to specified devices through a specified protocol, obtain status data, and respond to the monitoring node; each protocol adapter can only connect to one protocol. in, After receiving the assigned task, the monitoring node sends a request to the protocol adapter scheduler to obtain a protocol adapter. Upon receiving the request, the scheduler does not immediately assign a protocol adapter to the monitoring node; instead, it saves the request to the protocol adapter request table. In the protocol adapter request table, the scheduler retrieves the corresponding protocol type from the request, groups the requests according to the protocol type, and queues each group of protocol adapter requests according to the request time, with each new request placed at the end of the queue. After saving, the scheduler checks if there is an idle protocol adapter with the same protocol in the protocol adapter registry. If so, it assigns the protocol adapter to the first request in the queue of the same protocol request in the protocol adapter request table and marks the protocol adapter's status as "working." This process continues until a new protocol adapter request or an idle protocol adapter appears, at which point the scheduler searches and assigns the adapter.

2. The complex network device monitoring system as described in claim 1, characterized in that, The system adopts a dual-cluster deployment architecture consisting of a monitoring node cluster and a protocol adapter cluster; The monitoring process is driven by the main system service. The main system service initiates monitoring tasks according to the monitoring task schedule. After receiving the monitoring task, the monitoring task scheduler allocates the monitoring task to the monitoring node according to the status of the monitoring node cluster, and the monitoring node performs the subsequent monitoring tasks. If the size of the monitoring node cluster cannot meet the monitoring task requirements in a timely manner, the monitoring node cluster can be dynamically expanded to add a certain number of monitoring nodes to meet the monitoring task requirements. After receiving a monitoring task, the monitoring node will request a protocol adapter from the protocol adapter scheduler. The protocol adapter scheduler assigns protocol adapters for a specified protocol to monitoring nodes based on the status of the protocol adapter cluster, and performs data collection on network devices. If there are no available protocol adapters, you can add protocol adapters for the specified protocols to meet the network device monitoring tasks. If no protocol adapter is specified, the protocol can be adapted separately, and the adapted protocol adapter can be registered to the protocol adapter scheduler.

3. The complex network device monitoring system as described in claim 1, characterized in that, The protocol adaptation layer is an intermediate layer located between the network device layer and the monitoring service layer. It receives data packets from various devices, parses and converts them into a unified format, and then passes the data to the monitoring service layer.

4. The complex network device monitoring system as described in claim 3, characterized in that, Before starting, the protocol adapter needs to be configured with relevant parameters. After startup, the protocol adapter will automatically connect to the protocol adapter scheduler according to the configured protocol adapter scheduler IP address and send a registration request; After receiving a registration request from a protocol adapter, the protocol adapter scheduler saves the connected protocol adapter information to the protocol adapter registry. When saving the protocol adapter information, the protocol adapter scheduler groups the protocol adapters according to the protocol type to reduce the retrieval time during allocation.

5. The complex network device monitoring system as described in claim 4, characterized in that, After the protocol adapter is assigned to a designated monitoring node, the monitoring node will send a device monitoring request to the protocol adapter, which will include the network device's connection information in the request parameters. After receiving the connection information, the protocol adapter establishes a protocol connection and data interaction with the designated device, obtains status information, converts the protocol data into data of a unified data type, and responds to the monitoring node. After the protocol adapter completes the acquisition of device status data, it will send a status recovery request to the protocol adapter scheduler again. After receiving the request, the protocol adapter scheduler will restore the status to "idle", and the protocol adapter will complete one device monitoring task.

6. The complex network device monitoring system as described in claim 5, characterized in that, After receiving device status data based on a unified data model from the protocol adapter, the monitoring node first cleans the data and verifies its legality. If the data is deemed invalid, it is discarded, and the process ends. If the data meets the verification rules, the monitoring node converts the data to conform to different database table structures. Based on the converted data, it analyzes whether the device's various parameters meet the alarm conditions. If the device's parameters meet the alarm conditions, alarm data is generated. Finally, the converted device status data and device alarm data are saved to the database for system display and data auditing.

7. The complex network device monitoring system as described in claim 6, characterized in that, Converting data to meet different database table structures includes: converting to device basic data, device CPU data, and device memory data; alarm conditions include: whether the device is offline, whether the CPU utilization rate exceeds the threshold, and whether the memory utilization rate exceeds the threshold; alarm data includes: device offline alarms, device CPU alarms, and device memory alarms.

8. The complex network device monitoring system as described in claim 1, characterized in that, The system's functional modules include: homepage status display, device asset management, network device topology, device alarm alerts, and system configuration management; The homepage status display module is used to summarize and display the status of network devices. It uses rich charts and dashboards for visual statistics, displaying the top 5 alarm devices, unprocessed alarms, asset overview, and abnormal device statistics. The equipment asset management module is used for maintaining the equipment asset list. In addition to maintaining the basic information of the equipment list, it can also view the operating status information of key performance indicators of the equipment. It also has the function of remote operation and maintenance of equipment and supports remote connection protocols. The network device topology module is used to dynamically generate the topology of network devices and monitor data traffic in the network. The device alarm notification module is used for alarm functions for abnormal information of network devices. The system classifies alarm information into performance indicator abnormal alarms, device offline alarms, and unauthorized device access alarms, and performs statistical analysis by type and level; it also supports alarm information processing functions to record user processing operations on device alarm information. The system configuration management module is used to manage the configuration of system-related parameters; the monitoring rule setting is used to configure the rules for issuing monitoring tasks, and to enable pausing, starting, and controlling the issuance of monitoring tasks; the alarm threshold setting is used to set thresholds for different types of performance parameters and different levels of abnormal alarms; and the data collection log management is used to query and manage the data collection logs of device status data.

9. A method for monitoring complex network devices based on the system according to any one of claims 1-8, characterized in that, The method includes the following steps: S1. The system main service initiates a device monitoring task. When initiating a device monitoring task, the task parameters must include at least the unique identifier of the network device and the monitoring protocol; otherwise, the device monitoring task cannot be initiated. The system main service distributes the device monitoring task to the monitoring task scheduler. After receiving the monitoring task, the monitoring task scheduler saves the current monitoring task to the monitoring task list. S2. The monitoring task scheduler checks if there are any other unassigned monitoring tasks before the current monitoring task. If so, it assigns the previous monitoring tasks first until all tasks before the current task have been assigned. Then, it starts assigning the current monitoring task. After the monitoring task is assigned, the monitoring task scheduler checks if there are any idle nodes in the monitoring node cluster. If not, the current task is in a waiting state until an idle monitoring node appears. The monitoring task scheduler then assigns the current monitoring task to the idle monitoring node, and the assignment of the current monitoring task is completed. S3. After receiving the monitoring task, the monitoring node requests the protocol adapter of the specified protocol from the protocol adapter scheduler according to the monitoring protocol parameters in the task data. After receiving a request, the protocol adaptation scheduler adds the request to the request list; S4. The protocol adaptation scheduler checks whether there are any unprocessed requests with the same protocol before this request. If so, it processes the previous requests with the same protocol first, until all requests with the same protocol before this request have been processed, and then starts processing this request. S5. After the protocol adapter scheduler starts processing this request, it checks whether there is an idle adapter with the same protocol in the protocol adapter cluster. If not, the request will wait until an idle protocol adapter appears. The protocol adapter scheduler will then assign the idle protocol adapter to the monitoring node. The monitoring node will obtain network device status data by calling the data interface provided by the protocol adapter and perform data processing. The monitoring task will then be completed.

Images