Network equipment index acquisition method and device, equipment, storage medium and product

Abstract

The invention discloses a network equipment index collection method and device, equipment, a storage medium and a product, and relates to the technical field of network management, the method comprises the steps that a first micro-service node in a collection end is called to acquire a network equipment index collection task, and the first micro-service node comprises a main task distributor and a standby task distributor; if the main task distributor does not respond to the request of the standby task distributor in the preset service connection period, the standby task distributor is switched to be in a main use state, and the standby task distributor is called to send a network equipment index collection task to a second micro-service node in the collection end, the second micro-service node is a node preset based on a multi-active mechanism; and controlling the second micro-service node through the first micro-service node, and executing a network equipment index acquisition task. According to the method, the first micro-service node with the main / standby switching mechanism and the second micro-service node with the multi-active mechanism are deployed at the acquisition end, so that the full-link availability of the network equipment index acquisition method is improved.

Description

Technical Field

[0001] The present application relates to the field of network management technology, and in particular to a method, device, equipment, storage medium and product for collecting network equipment indicators. Background Art

[0002] With the development of science and technology and the progress of society, network communications have developed rapidly, and at the same time, the information on the network has also developed explosively. In order to monitor information such as network traffic, it is necessary to collect network equipment indicators. Common commercial or open source network equipment indicator collection software usually includes collection modules, processing modules, storage modules, alarm modules and display modules. These products have achieved certain results in network equipment monitoring and data collection. However, the existing technology lacks the master-slave mechanism and multi-active mechanism in the architectural design of the task distributor of the collection equipment, which reduces the full-link availability of the equipment. Once a link fails, it is easy to cause interruption and loss of data collection.

[0003] Therefore, how to improve the full-link availability of network equipment indicator collection methods is a problem that needs to be solved urgently.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a network equipment indicator collection method, device, equipment, storage medium and computer program product, aiming to solve the technical problem of how to improve the full-link availability of the network equipment indicator collection method.

[0006] To achieve the above object, the present application proposes a network device indicator collection method, which is applied to a network device indicator collection system. The network device indicator collection system includes a collection end, and the method includes:

[0007] Calling a first microservice node in the collection end to obtain a network device indicator collection task, wherein the first microservice node includes a main task distributor and a backup task distributor;

[0008] If the main task distributor does not respond to the service connection request of the backup task distributor within the preset service connection period, the backup task distributor is switched to the active state, and the backup task distributor is called to send the network device indicator collection task to the second microservice node in the collection end, wherein the second microservice node is a node pre-set based on the multi-active mechanism;

[0009] The second microservice node is controlled by the first microservice node to execute the network device indicator collection task.

[0010] Optionally, the network device indicator collection system includes a master control end, and the step of calling the first microservice node in the collection end to obtain the network device indicator collection task includes:

[0011] Calling the first microservice node in the acquisition end to receive the task configuration data sent by the master control end, parsing the task configuration data based on the task configuration data, and obtaining a task name and a task switch;

[0012] The scheduled creation task and distribution thread of the first microservice node are updated based on the task name and the task switch to obtain the network device indicator collection task.

[0013] Optionally, the second microservice node includes a plurality of data collectors, and the step of controlling the second microservice node through the first microservice node to perform the network device indicator collection task includes:

[0014] Traversing the uniform resource locators URLs corresponding to the communication interfaces of the plurality of data collectors respectively through the first microservice node to obtain a first URL circular queue;

[0015] Obtaining first URL response status data through the first microservice node, traversing and reading each URL in the first URL circular queue, so as to update the first URL circular queue based on the first URL response status data, and obtain a second URL circular queue;

[0016] The target data collector cluster corresponding to the second URL circular queue is controlled by the first microservice node to execute the network device indicator collection task.

[0017] Optionally, the step of traversing and reading each URL in the first URL circular queue to update the first URL circular queue based on the first URL response status data to obtain a second URL circular queue includes:

[0018] For any target URL read through the traversal, judging whether the data collector corresponding to the target URL can respond to the network device indicator collection task based on the first URL response status data;

[0019] If not, remove the target URL from the first URL circular queue;

[0020] This time, each URL in the first URL circular queue is traversed until the second URL circular queue is obtained.

[0021] Optionally, the second microservice node further includes a plurality of data processors, and the step of controlling the target data collector cluster corresponding to the second URL circular queue through the first microservice node to execute the network device indicator collection task includes:

[0022] Obtaining a system configuration file through the first microservice node, determining a target network device corresponding to the network device indicator collection task based on the system configuration file, and collecting data on the target network device to obtain target indicator data;

[0023] Acquire a third URL circular queue corresponding to the plurality of data processors and second URL response status data corresponding to the plurality of data processors through the target data collector cluster, update the third URL circular queue based on the second URL response status data to obtain a fourth URL circular queue, and determine a target data processor based on the fourth URL circular queue;

[0024] The target indicator data is processed by the target data processor to obtain an indicator processing result.

[0025] Optionally, the step of processing the target indicator data by the target data processor to obtain an indicator processing result includes:

[0026] Sending the indicator processing result to the third microservice node of the acquisition end, wherein the third microservice node includes a primary data reporter and a backup data reporter;

[0027] The step of sending the indicator processing result to the third microservice node of the acquisition end includes:

[0028] When the target data recorder cannot send the indicator processing result to the master data recorder, obtain the master-slave switching time of the historical data recorder and the current system time;

[0029] If the absolute difference between the active-standby switching time of the historical data reporter and the current system time is less than a preset time threshold, the data reporter communication interface in the memory of the third microservice node is configured as the communication interface of the standby data reporter to control the target data processor to send the indicator data result to the standby data reporter based on the communication interface of the standby data reporter.

[0030] Optionally, the method further comprises:

[0031] Obtaining a system configuration file through the first microservice node to determine a load balancing frequency of the network device indicator collection task based on the system configuration file;

[0032] Calculate the task delivery interval based on the load balancing frequency;

[0033] The network device indicator collection task is sent to the second microservice node through the first microservice node based on the task delivery interval.

[0034] Optionally, the first microservice node reads based on the system configuration file to obtain a threshold event generation rule, and synchronizes the threshold event generation rule to the target data processor;

[0035] The target data processor matches the collection object of the network device indicator collection task based on the threshold event generation rule to obtain a collection object matching result;

[0036] When the collection object matching result is a successful match, if the target indicator data meets the threshold alarm condition in the threshold event generation rule, the threshold alarm data is generated by the target data processor, and the threshold alarm data is converted into visual data to push the threshold alarm data to the target user.

[0037] Optionally, obtain the heartbeat Kafka data of each microservice node in the collection end, and determine whether there is data missing based on the heartbeat Kafka data;

[0038] If there is data missing, a supplementary recording instruction is generated based on the site system configuration, and the second microservice node is controlled to perform an automatic supplementary recording task based on the supplementary recording instruction.

[0039] Optionally, the step of controlling the second microservice node to perform the automatic supplementary recording task based on the supplementary recording instruction includes:

[0040] If the supplementary recording instruction includes a task identification ID, constructing a target local backup file index path corresponding to the task ID based on the task ID;

[0041] The backup file is retrieved based on the target local backup index path to obtain the target backup file corresponding to the task ID, so as to control the second microservice node to automatically supplement the recording based on the target backup file.

[0042] Optionally, the step of controlling the second microservice node to perform the automatic supplementary recording task based on the supplementary recording instruction includes:

[0043] If the supplementary recording instruction includes a collection time interval, controlling the second microservice node to construct a full backup file index path set, so as to call a file reading method to retrieve a full backup file set based on the full backup file index path set;

[0044] The full backup file set is screened based on the collection time interval to obtain a target backup file that meets the collection time interval, so as to control the second microservice node to automatically supplement the recording based on the target backup file.

[0045] In addition, to achieve the above purpose, the present application also proposes a network equipment indicator collection device, the network equipment indicator collection device comprising:

[0046] A task acquisition module, used to call the first microservice node in the collection end to obtain a network device indicator collection task, wherein the first microservice node includes a main task distributor and a backup task distributor;

[0047] A task issuing module, used for switching the backup task distributor to the active state if the primary task distributor does not respond to the service connection request of the backup task distributor within a preset service connection period, and calling the backup task distributor to send the network device indicator collection task to the second microservice node in the collection end, wherein the second microservice node pre-deploys a multi-active server;

[0048] A task execution module is used to control the second microservice node through the first microservice node to execute the network device indicator collection task.

[0049] In addition, to achieve the above-mentioned purpose, the present application also proposes a network equipment indicator collection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the network equipment indicator collection method described above.

[0050] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the network device indicator collection method described above are implemented.

[0051] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the network device indicator collection method described above are implemented.

[0052] One or more technical solutions proposed in this application have at least the following technical effects:

[0053] The present application obtains the network device indicator collection task by calling the first microservice node in the collection end, wherein the first microservice node includes a main task distributor and a backup task distributor; if the main task distributor does not respond to the service connection request of the backup task distributor within the preset service connection period, the backup task distributor is switched to the main state, and the backup task distributor is called to send the network device indicator collection task to the second microservice node in the collection end, wherein the second microservice node is a node pre-set based on a multi-active mechanism; the second microservice node is controlled by the first microservice node to execute the network device indicator collection task. On the basis of obtaining the network device indicator collection task, the embodiment of the present application deploys a first microservice node with a main-backup switching mechanism and a second microservice node with a multi-active mechanism at the collection end to execute the network device indicator collection task, thereby improving the full-link availability of the network device indicator collection method, thereby avoiding interruption and loss of data collection. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0056] Figure 1 This is a schematic diagram of the system architecture of the network equipment indicator collection system for this application;

[0057] Figure 2 This is a flow chart of the first embodiment of the network device index collection method of the present application;

[0058] Figure 3 A schematic diagram of a process of performing active / standby switching of a task distributor provided in Embodiment 1 of the present application;

[0059] Figure 4 A schematic diagram of the process of issuing system configuration provided in Example 1 of the present application;

[0060] Figure 5 A schematic diagram of a process of sending tasks from a master control end to a collection end provided in the second embodiment of the present application;

[0061] Figure 6 A schematic diagram of the working principle of a data collector with a multi-active mechanism provided in Example 3 of the present application;

[0062] Figure 7 A schematic diagram of the process of performing active / standby switching of a data reporter provided in Embodiment 4 of the present application;

[0063] Figure 8 A schematic diagram of a process for implementing load balancing provided in Embodiment 5 of the present application;

[0064] Fig. 9 A schematic diagram of the process of generating and issuing threshold event rules provided in Example 6 of the present application;

[0065] Fig.10 A schematic diagram of the process of automatic supplementary recording at the acquisition end provided in Embodiment 7 of the present application;

[0066] Fig.11 A schematic diagram of the process flow of the acquisition terminal self-monitoring provided in Embodiment 7 of the present application;

[0067] Fig.12 This is a schematic diagram of the module structure of a network device indicator collection device for network indicator collection in an embodiment of the present application;

[0068] Fig.13 This is a schematic diagram of the device structure of the hardware operating environment involved in the network device indicator collection method in the embodiment of the present application.

[0069] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0070] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0071] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0072] The main solution of the embodiment of the present application is: calling the first microservice node in the collection end to obtain the network device indicator collection task, wherein the first microservice node includes a main task allocator and a backup task allocator; if the main task allocator does not respond to the service connection request of the backup task allocator within a preset service connection period, switching the backup task allocator to the main state, calling the backup task allocator to send the network device indicator collection task to the second microservice node in the collection end, wherein the second microservice node is a node pre-set based on a multi-active mechanism; controlling the second microservice node through the first microservice node to execute the network device indicator collection task.

[0073] Since the existing technology lacks a master-slave mechanism and a multi-active mechanism in the architectural design of the task distributor of the collection equipment, the full-link availability of the equipment is reduced. Once a link fails, it is easy to cause interruption and loss of data collection.

[0074] Based on the solution of the present application, on the basis of obtaining the network device indicator collection task, a first microservice node with a master-slave switching mechanism and a second microservice node with a multi-active mechanism are deployed at the collection end to execute the network device indicator collection task, thereby improving the full-link availability of the network device indicator collection method, thereby avoiding interruption and loss of data collection.

[0075] It should be noted that the execution subject of this embodiment can be a computing service device with network device indicator collection, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a network device indicator collection system, etc. The following takes the network device indicator collection system as an example to illustrate this embodiment and the following embodiments.

[0076] At present, the network equipment indicator collection system in the relevant network equipment indicator collection method usually includes a collection module, a processing module, a storage module, an alarm module and a display module, which can realize data collection, processing, storage and alarm functions. Although these technologies have achieved certain results in network equipment monitoring and data collection, they all have more or less the following defects:

[0077] The collection tasks are highly coupled and not flexible enough. The collection tasks in the existing system usually share configuration files, and it is impossible to achieve complete independence between tasks. Modifying the configuration requires restarting the collector. This makes the operation complicated when dynamically adding or starting and stopping collection tasks, affecting the overall flexibility of the network device indicator collection system.

[0078] The high availability of the entire link is poor, and there is a risk of data loss in data collection. The existing technology often lacks a master-slave and multi-active mechanism in the architectural design of the task distributor, collector, data processor, and data reporter. Once a link fails, it is easy to cause interruption and loss of data collection.

[0079] Poor scalability, difficult to adapt to different scales of collection needs. The deployment of collectors and data processors in existing network equipment indicator collection systems is often not flexible enough, and it is difficult to dynamically expand or reduce according to the actual collection volume and resource conditions, resulting in resource waste or performance bottlenecks.

[0080] Load balancing is not comprehensive and the system stability is poor. The existing technology is often not comprehensive enough in terms of load balancing, and cannot achieve sequential collection load balancing of collection tasks in the collection interval time, resulting in unstable system performance under high concurrency.

[0081] The generation delay of alarm events is large. The alarm events of the existing network equipment indicator collection system are usually generated uniformly by the master control end after data collection and processing are completed, resulting in delays in the generation and push of alarm events and failure to respond to network anomalies in a timely manner.

[0082] Based on the above considerations, the present application proposes a network device indicator collection method to make the following improvements to the above defects: through independent configuration between tasks, the collection, processing, and push processes are isolated from each other and do not interfere with each other. They can be dynamically added, started, and stopped, so that the collection tasks are independent and the management and control are flexible; through the master-slave architecture and automatic switching of the task distributor, the multi-active architecture and local data backup and re-recording of the collector, the multi-active architecture and local data backup and re-recording of the data processor, the master-slave and automatic switching, self-monitoring and heartbeat monitoring of the data reporter; the data collector and the data processor can expand and reduce the number of deployments according to the collection volume of different collection sites, and can accurately configure the number of threads of each type of collection task for each service according to the server performance. It can flexibly select whether to use a cache circular queue and the size of the circular queue according to the resource and collection volume configuration to improve scalability and support flexible deployment of collection sites with different equipment volumes; support load balancing of multi-active services and sequential collection load balancing in the collection interval time, so that the comprehensive load balancing service is more stable; for the indicator collection configuration, event push is generated from the collection end immediately after the collection and processing is completed, thereby reducing the delay in the generation of the preceding event.

[0083] Reference Figure 1 , Figure 1 This is a schematic diagram of the system architecture of the network equipment indicator collection system; Figure 1As shown, the method proposed in the embodiment of the present application is mainly used in the network equipment indicator collection system. The system proposed in the present application adopts a microservice architecture design. The collection end receives the tasks and data issued by the master control end through the master control end-collection end task and data dedicated distribution channel, and transmits data back to the master control end through Kafka. Kafka is a stream processing platform for building real-time data pipelines and stream applications. It stores and manages data through Topics. Topics can be regarded as message classifiers in Kafka. Producers publish messages to specific topics, and consumers subscribe to messages from topics. This topic-based messaging mechanism enables Kafka to process large amounts of data in a high-performance and low-latency manner. The topics in Kafka can be configured with multiple partitions, and each partition can be placed on different servers, so that horizontal expansion can be achieved and throughput can be improved. Partitions can be replicated on different servers in the cluster to improve fault tolerance and ensure that data is not lost. Kafka has a mechanism called heartbeat for monitoring and managing consumer status. After subscribing to a topic, consumers will continuously send heartbeat requests to the Kafka cluster to indicate their active status and maintain subscription to the topic. The heartbeat mechanism helps track consumer status, automatically reassign partitions, manage consumer groups, and update metadata, which improves Kafka's fault tolerance, resource management, and consistency of data processing. The above design concepts and architecture make Kafka an ideal choice for processing large-scale real-time data streams.

[0084] like Figure 1 As shown, the network equipment index collection system of the present application includes a collection end and a master control end, wherein the master control end includes two parts: a master control end front end and a master control end back end. The master control end is mainly responsible for user interaction, providing an interface for administrators to create / update collection tasks, collection task management, event generation rule management, and collection end system configuration management, and provides monitoring and display functions. The master control back end receives the task configuration from the front end to control the collection end to perform the reception and transmission of the collection task, and sends the collection task to the collection end, and generates the rule by sending the threshold event to the collection end. At the same time, the master control end back end receives the index data and collection data results published by the collection end to the corresponding topic through the heartbeat Kafka, analyzes the heartbeat data to detect the operating status of each collection node and stores it. In addition to the normal processing flow, the master control end also realizes automatic data re-recording in the case of abnormal data through the local backup and automatic re-recording process of the collection end data and the self-monitoring process of the collection end, ensuring the overall stability and reliability of the system.

[0085] It should be noted that the collection end is responsible for specific data collection, processing and uploading to ensure the efficient operation of the system and the reliability of data. The collection end consists of four main components: TaskMaker (task distributor), Collector (data collector), Processor (data processor) and DataHub (data reporter). The collection end adopts a master-slave and multi-active architecture design, and achieves high availability and load balancing through the master-slave switching of the task distributor, the multi-active processing of the data collector and data processor, and the master-slave switching of the data reporter. The following will describe the functions of the above four main components:

[0086] The task distributor adopts a master-slave architecture to ensure high availability of task distribution. The main functions of the task distributor are task reception, task distribution, and master-slave switching. When the system is operating normally, the master task distributor receives the collection task configuration issued by the master control end, parses the task category and parameters, and creates a scheduled task based on the parsing results to distribute the collection task to the corresponding data collector. The backup task distributor regularly synchronizes the task configuration with the master task distributor, monitors the status of the master task distributor, and automatically takes over the task distribution when the master fails. When the task distributor distributes collection tasks to the data collector, if three consecutive attempts to distribute collection tasks to the data collector fail, the data collector is replaced to complete the collection based on the multi-active mechanism of the data collector. When adding and changing the threshold event generation rules at the master control end, the threshold event rules are issued to the data processor.

[0087] The data collector adopts a multi-active architecture to ensure load balancing and high efficiency of data collection. When the system is running, multiple data collector instances run simultaneously. According to the collection tasks distributed by the task distributor, data collection requests are sent to network devices based on the network communication protocol. After collecting data, the data collector stores the data in a local backup file to ensure data traceability. After pre-processing, the collection results are sent by the data collector to the data processor for data processing. When the data collector sends the collection result data to the data processor, if three consecutive attempts to send to the data processor fail, the data processor is replaced to complete the data processing. At the same time, the data collector implements network connectivity diagnosis based on the ICMP protocol to obtain information such as the online status of network devices, network delay, and packet loss rate.

[0088] The data processor adopts a multi-active architecture to ensure load balancing and high efficiency of data processing. After receiving the collected data sent by the data collector, the data processor generates an alarm event according to the preset threshold event generation rules and records it in the local backup file to ensure the integrity of the data and events. Then, it attempts to send the processed data and alarm events to the data reporter through the URL circular queue three times in a row. If the sending fails, it is determined that the current data reporter has failed, and the data reporter master-slave switching process is executed to achieve resending of processed data and alarm events after the master-slave data reporter switches.

[0089] The data reporter adopts a master-slave architecture to ensure high availability of data upload. The master data reporter receives the processed data and alarm events sent by the data processor, uploads the received data to the data storage component of the master control end through Kafka, and sends the alarm event to the EventAlerter (event alarm) service through a dedicated channel. In addition, the master data reporter receives the heartbeat packets sent regularly by the data collector and data processor to determine whether each service is operating normally, and generates a heartbeat alarm when the heartbeat is not sent normally. In addition, the heartbeat Kafka is also used by the master control end to issue a re-recording instruction when data loss is detected. The backup data reporter regularly synchronizes the data upload status with the master data reporter. When the master data reporter fails, the backup data reporter automatically takes over the data upload task to ensure the continuity of data upload.

[0090] In addition, it should be noted that in the embodiment of the present application, the network device indicator collection system can collect network device indicators based on SNMP (Simple Network Management Protocol) and ICMP (Internet Control Message Protocol). Among them, SNMP and ICMP are widely used in the field of network management and equipment monitoring. The main components of SNMP include Manager (management station), Agent (agent), and MIB (management information base). Manager is a server that runs management applications and is used to monitor and control SNMP agents; Agent is software running on managed devices, responsible for receiving and processing requests from SNMP management stations; MIB contains definitions and information of all managed objects, structured into a hierarchical structure, and MIB contains the OID (identifier) ​​of each object. OID is organized in a tree structure in MIB, and each node has a unique number, usually represented as a string of numbers, such as 1.3.6.1.2.1.1.1, representing a specific management object. Based on OID, SNMP can obtain device management status information, performance data, configuration parameters, etc. SNMP mainly uses GET (get), GET-NEXT (get the next), SET (set), and TRAP (alarm or notification) instructions to manage devices. SNMP obtains the value of a specified OID through a GET request, obtains the value of the next OID through GET-NEXT to implement MIB traversal, sets the value of a specified OID through a SET request, and sets an agent through TRAP to send an alarm or notification to the management station.

[0091] Specifically, when SNMP performs device monitoring, the Manager sends a GET request to the agent to obtain the value of a certain OID, such as obtaining the CPU usage of the device. The Agent searches for the corresponding OID and returns its value to the management station. When SNMP performs device control, the Manager sends a SET request to the agent to set the value of a certain OID, such as modifying the configuration parameters of the device. The Agent performs the configuration modification and returns a confirmation response. When SNMP performs exception handling, the Agent will actively send a TRAP message to notify the Manager of the abnormal situation. After receiving the TRAP message, the Manager takes corresponding measures, such as alarming, recording logs, or automatically performing repair operations. ICMP is used to transmit control messages, such as sending error messages, diagnostic information, and control commands. ICMP messages are generally used for network troubleshooting and diagnosis. Based on SNMP and ICMP protocols, the network device indicator collection system of this embodiment realizes the acquisition and analysis of indicator data of network devices, and ensures efficient operation of the network and rapid fault response.

[0092] Reference Figure 2 , Figure 2 This is a flow chart of the first embodiment of the network device indicator collection method of the present application.

[0093] In this embodiment, the network device indicator collection method is applied to a network device indicator collection system, the network device indicator collection system includes a collection end, and the network device indicator collection method includes steps S10 to S30:

[0094] Step S10, calling the first microservice node in the collection end to obtain a network device indicator collection task, wherein the first microservice node includes a main task distributor and a backup task distributor;

[0095] It should be noted that the microservice architecture of the network device indicator collection system in this embodiment improves the scalability, maintainability and reliability of the system by splitting large applications into multiple small, independent services. The setting of the main task allocator and the backup task allocator is to ensure the continuity and stability of task allocation and prevent task interruption caused by single point failure. The technical purpose is to ensure that network device indicator collection tasks can be allocated and executed in a timely and effective manner under any circumstances.

[0096] Step S20, if the main task distributor does not respond to the service connection request of the backup task distributor within the preset service connection period, the backup task distributor is switched to the active state, and the backup task distributor is called to send the network device indicator collection task to the second microservice node in the collection end, wherein the second microservice node is a node pre-set based on the multi-active mechanism;

[0097] Reference Figure 3 , Figure 3 A schematic diagram of a process flow of performing active / standby switching of a task distributor provided in the first embodiment of the network device index collection method of the present application; Figure 3As shown in the figure, the task distributor is divided into the main task distributor and the backup task distributor. Among them, the main task distributor is marked as "main TaskMaker" and the backup task distributor is marked as "backup TaskMaker". After the main TaskMaker is started, it listens for configuration changes through the configuration file interface listening thread. The thread listens for changes in the configuration file through a file change listener. If the configuration file changes, the configuration file interface listening thread first updates the configuration, then obtains the IP of the backup TaskMaker from the microservice IP dictionary that contains the mapping relationship between the entire site microservice and the IP port, and then calls the configuration synchronization method to synchronize the configuration of all services that are not the main Taskmaker service. The backup TaskMaker regularly sends the configuration to the main one through the heartbeat synchronization interface, and determines whether the configuration has changed according to the response of the main TaskMaker. If there is a change, the original configuration is overwritten and updated synchronously. When the main TaskMaker fails, the backup TaskMaker fails to get a response from the main TaskMaker after two consecutive attempts to synchronize the heartbeat, and the difference between the current system time and the last state switching time exceeds 10 minutes (if it exceeds 10 minutes, it means that the switch between the main and backup TaskMakers has exceeded a certain time, otherwise frequent switching is likely to cause system problems). When the above conditions are met, the backup TaskMaker will initiate a master-slave switch and automatically switch to the main TaskMaker. When switching, the backup first pushes the alarm data through the alarm event service, then reads the current system configuration from the system configuration file, and then changes the IP corresponding to the keyword that uniquely identifies the main TaskMaker in the microservice IP dictionary in the current configuration to the IP of the backup TaskMaker, and rewrites it into the configuration file. At this time, the backup TaskMaker reads the new system configuration to confirm that the identity has been switched to the main one, and immediately tries to synchronize the configuration to all services three times in a row before starting to take over the task distribution. Once the original main node recovers, it will re-declare the main state and synchronize the task configuration, and the original backup detects that the configuration file has changed, and then switches back to the backup state. This mechanism ensures seamless switching when the main node fails, improving the stability and reliability of the system.

[0098] Step S30: Control the second microservice node through the first microservice node to execute the network device indicator collection task.

[0099] Reference Figure 4 , Figure 4 A schematic diagram of the process of issuing system configuration provided in the first embodiment of the network device index collection method of this application; Figure 4 As shown, in this embodiment, the system configuration is sent to the task distributor of the collection end through the master control end, and then the system configuration is synchronized to other microservice nodes in the collection end through the task distributor of the collection end.

[0100] Specifically, after the administrator creates / updates the system configuration through the front end of the microservice WebServer (master control end), the WebServer sends the task configuration in JSON (JavaScript Object Notation, lightweight data exchange) format to the main TaskMaker of the collection end. After the main TaskMaker receives the system configuration JSON file, it calls the file change processing method through the configuration monitoring thread to process the JSON file. The method first compares the old configuration file with the new configuration file. If the file content changes, it is overwritten and written to the local configuration folder. After the system configuration file update detection thread detects the change of the system configuration file, it reads the new configuration file into the memory and restarts all threads related to the collection task to ensure that the new configuration takes effect. Subsequently, the main TaskMaker reads the collector status field in the system configuration to confirm that it is the main one and triggers configuration synchronization, and obtains the IP addresses of all TaskMaker standby instances and other collection ends from the microservice IP dictionary, and synchronizes the configuration file to the TaskMaker standby instance and other collection end services through the POST (HTTP protocol-based request) method. After the TaskMaker standby instance receives the system configuration JSON file synchronized by the main instance, it overwrites it and writes it to the local configuration folder, and starts the configuration file update detection thread, reads the new configuration file into the memory, and restarts all threads related to the collection task to ensure that the new configuration takes effect. The TaskMaker confirms itself as the backup TaskMaker through the collector status field, and does not trigger configuration synchronization. Other services on the collection side (such as data collectors, data processors, and data reporters) also receive the system configuration JSON file, overwrite it to the local configuration folder, start the configuration file update detection thread, read the new configuration file into memory, restart all collection task-related threads, ensure that the new configuration takes effect, and confirm that the service is running normally. The entire process ensures the real-time and effectiveness of the configuration through configuration file update detection and task thread restart, and realizes efficient, stable and flexible collection task management.

[0101] The present application adopts the above scheme, specifically obtains the network device index collection task by calling the first microservice node in the collection end, wherein the first microservice node includes a main task distributor and a backup task distributor; if the main task distributor does not respond to the service connection request of the backup task distributor within the preset service connection period, the backup task distributor is switched to the main state, and the backup task distributor is called to send the network device index collection task to the second microservice node in the collection end, wherein the second microservice node is a node pre-set based on the multi-active mechanism; the second microservice node is controlled by the first microservice node to execute the network device index collection task. The present application designs a highly available and scalable architecture for various microservices at the collection end. The main and standby and automatic switching design of the microservice task distributor. The task distributor is divided into main and standby. After the main task distributor is started, it listens to and updates the system configuration and synchronizes the task configuration of all services. The backup task distributor periodically sends the configuration to the main one to synchronize the update. If the main task distributor fails, the backup task distributor will automatically switch to the main one and take over the task distribution after failing to receive the main response for two consecutive times. Once the original master task distributor is restored, it will re-declare the master state and synchronize the task configuration, while the original backup task distributor returns to the backup state. This embodiment ensures seamless switching when the master node fails, and improves the stability and reliability of the system.

[0102] Based on the network device indicator collection method proposed in this application, the administrator using the network device indicator collection system can create or update the system configuration through the WebServer front end, and then send this information to the collection end TaskMaker in JSON format through the WebServer. After receiving, the main TaskMaker overwrites the configuration folder. These configurations are detected by the file monitoring thread and read and write the configuration into the memory after the update, and restart all the threads related to the collection task. Subsequently, the main TaskMaker reads the collector status field in the system configuration to confirm that it is the main and trigger configuration synchronization. After receiving the configuration file synchronized by the main, the backup TaskMaker overwrites the local configuration folder. These configurations are detected by the file monitoring thread and read and write the configuration into the memory after the update, and restart all the threads related to the collection task. Subsequently, the backup TaskMaker reads the collector status field in the system configuration to confirm that it is the backup and does not trigger configuration synchronization. After receiving, the other collection end overwrites the configuration folder. These configurations are detected by the file monitoring thread and read and write the configuration into the memory after the update, and restart all the threads related to the collection task. The entire process ensures the real-time and effectiveness of the configuration through the update detection of the configuration file and the restart of the task thread, and realizes efficient, stable and flexible collection task management.

[0103] Based on the first embodiment of the present application, the second embodiment of the present application is proposed. In the second embodiment of the present application, the same or similar content as the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated later. In this embodiment, the network device indicator collection system includes a master control end, and the first microservice node in the collection end is called to obtain a network device indicator collection task. Step S10 may include:

[0104] Step S101, calling the first microservice node in the acquisition end to receive the task configuration data sent by the master control end, parsing the task configuration data based on the task configuration data, and obtaining the task name and task switch;

[0105] Step S102: based on the task name and the task switch, the scheduled creation task and distribution thread of the first microservice node are updated to obtain the network device indicator collection task.

[0106] Specifically, refer to Figure 5 , Figure 5 A schematic diagram of a process of sending tasks from a master control end to a collection end provided in the second embodiment of the network device index collection method of the present application; Figure 5 As shown in the figure, after the administrator creates / updates the task through the front end of the microservice master control end (WebServer), the WebServer sends the task configuration in JSON format to the TaskMaker on the collection end. After receiving it, the main TaskMaker updates the local task configuration file. After the configuration file monitoring thread detects the configuration update, it first reads the new task configuration to obtain the task name and task switch, and then closes the scheduled creation task and distribution thread of the existing task with the same name. If the task switch is on, a new scheduled creation task and distribution thread are created according to the new configuration to achieve the task update. If the task switch is off, the scheduled task and distribution thread are closed. In order to achieve the synchronization of the collection task configuration on the main and standby TaskMaker, the main TaskMaker calls the main TaskMaker to standby interface to synchronize the latest configuration file to the standby. The standby TaskMaker sends a heartbeat packet to the main TaskMaker through the standby TaskMaker to master interface to synchronize the local task configuration file list, and updates the local configuration file according to the response of the main TaskMaker. In addition, the main TaskMaker regularly cleans up the timed collection task configuration by comparing the current system time and the task shutdown time to ensure the efficiency of task management and the stable operation of the system. In this way, the real-time update and accurate execution of network equipment indicator collection tasks can be guaranteed, improving system performance.

[0107] In addition, it should be noted that the embodiment of the present application also provides a specific configuration method for the network device indicator collection task:

[0108] The data structure of the collection task configuration is in the form of key-value pairs, including the task name (uniquely identifies the task), task switch, whether the collection alarm is triggered, whether event push is enabled, and other basic information. In addition, the collection task configuration also includes Kafka information (server address, topic, user name and password) and detailed collection configuration. The first collection configuration is shown in Table 1 below:

[0109]

[0110]

[0111] Table 1

[0112] Specifically, the first type of collection task is centered on the collection item, and collects data of all collection items from the target site. Its collection configuration includes the collection configuration name, collection timing expression, and multiple collection items (each collection item has its own collection item name, OID, collection timing sub-expression, expression value type) and multiple target sites (each target collection site has its own IP address, SNMP group name, SNMP version) and other information. The collection timing expression is an expression used to set the execution time of the scheduled task, which consists of six space-separated fields, representing minutes, hours, dates, months, and days of the week (and an optional year field), to accurately control the execution time of the task.

[0113] The collection configuration corresponding to the second collection task type is shown in Table 2 below:

[0114]

[0115]

[0116] Table 2

[0117] Specifically, as shown in Table 2, the second type of collection task is centered on the collection target site and collects data of specific collection items from the target site. Its collection configuration includes the collection configuration name, collection timing expression, and information of multiple target sites (each target collection site has its own IP address, SNMP community name, SNMP version, and specific collection item configuration), where the collection items have their own collection item name, oid, collection timing sub-expression, and expression value type.

[0118] In addition, it should be noted that the configuration of different expression value types can refer to Table 3:

[0119]

[0120]

[0121] Table 3

[0122] As shown in Table 3, when valueType is 1, the collection result is a difference value, which is applicable to numerical cumulative indicators. The OID of the collection object is the actual OID. The collection result of the current collection needs to be calculated by difference with the collection result of the previous collection, that is, the final reported collection result is the result of subtracting the previous result from the current result. When valueType is 3, the OID of the collection object is the actual OID, and no calculation is required. The collection result corresponding to the OID is reported. When valueType is 2, the collection result is the difference time average, which is applicable to numerical cumulative indicators. The value of OID is empty. cronExpression is a variable expression that only references 3 types of collection results. When calculating, the difference between the current collection value and the previous collection value of the 3 types of collection results pointed to by the referenced variable is used as the numerator, and the time interval between the current collection time and the previous collection time is used as the denominator. The final difference time average is calculated and reported. When valueType is 5, the value of OID is empty, and cronExpression is a calculation formula that references multiple collection results of types 1, 2, and 3. The final result of the current indicator is calculated and reported based on the final collection results of other referenced indicators and the expression.

[0123] This application adopts the above scheme. After creating / updating the task through the front end of the master control end (WebServer), the WebServer sends the task configuration to the collection end TaskMaker in JSON format. After receiving it, TaskMaker updates the local task configuration file. After the file monitoring thread detects the configuration update, it reads the updated task configuration, obtains the task name and task switch, closes the scheduled creation task and distribution thread of the existing task with the same name, and then if the task switch is on, creates a new scheduled creation task and distribution thread according to the new configuration, thereby updating or closing the task. In addition, the main task distributor regularly cleans up the timed task configuration to ensure the efficiency of task management and the stable operation of the system. In this way, the real-time update and accurate execution of the network equipment indicator collection task can be guaranteed, thereby improving the system performance.

[0124] Based on the first embodiment of the present application, the third embodiment of the present application is proposed. In the third embodiment of the present application, the same or similar content as the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated later. In this embodiment, the second microservice node includes multiple data collectors, and the step of controlling the second microservice node through the first microservice node to perform the network device indicator collection task includes:

[0125] Step A1, traversing the uniform resource locators URLs corresponding to the communication interfaces of the plurality of data collectors through the first microservice node to obtain a first URL circular queue;

[0126] Step A2: obtaining first URL response status data through the first microservice node, traversing and reading each URL in the first URL circular queue, so as to update the first URL circular queue based on the first URL response status data, and obtain a second URL circular queue;

[0127] Step A3: Control the target data collector cluster corresponding to the second URL circular queue through the first microservice node to execute the network device indicator collection task.

[0128] It should be noted that both the microservice data collector (referred to as Collector in this embodiment) and the data processor (referred to as Processor in this embodiment) adopt a multi-active design.

[0129] Reference Figure 6 , Figure 6 A schematic diagram of the working principle of a data collector with a multi-active mechanism provided in the third embodiment of the network device index collection method of the present application; Figure 6 As shown, TaskMaker reads all data collector communication interfaces from the system management configuration file and stores them in a URL circular queue. A circular queue is a ring-shaped data structure that can be seen as connecting queues end to end in order, and circularly using a fixed-length list to store URLs without increasing storage space. Thus, a first URL circular queue can be obtained.

[0130] Further, the traversal reads each URL in the first URL circular queue to update the first URL circular queue based on the first URL response status data to obtain a second URL circular queue, and the step A2 may include:

[0131] Step A21, for any target URL read through the traversal, judging whether the data collector corresponding to the target URL can respond to the network device indicator collection task based on the first URL response status data;

[0132] Step A22, if not, removing the target URL from the first URL circular queue;

[0133] Step A23, this time traversing each URL in the first URL circular queue until obtaining the second URL circular queue.

[0134] Specifically, when TaskMaker distributes collection tasks to Collector, it takes the URL from the Collector's communication interface circular queue and distributes the collection tasks through the POST method, thereby achieving load balancing of multiple active Collectors. If three consecutive attempts to send the collection task to the currently read URL fail, it is determined that the data collector to which this URL interface belongs is no longer available. TaskMaker first pushes an alarm through the alarm event HTTP API, then temporarily removes this URL from the circular queue, puts the collection task back into the corresponding collection task queue, and continues to read the next URL for sending.

[0135] The embodiment of the present application is based on the above scheme, and TaskMaker reads the Collector communication interface from the system configuration file and stores them in the URL circular queue. When TaskMaker distributes collection tasks to Collector, it takes the URL from the Collector's communication interface circular queue for sending, so as to achieve load balancing of multiple active Collectors. If the sending to the extracted URL fails three times in a row, the URL will be temporarily removed from the circular queue and the next URL will be sent. In addition, TaskMaker can control the frequency of sending collection tasks according to the system configuration to ensure that the collection tasks are sent and completed within the collection interval, and at the same time achieve sequential collection load balancing in the collection interval time dimension, avoiding high concurrent centralized collection causing severe resource overhead of collection services and network equipment.

[0136] In a feasible implementation manner, the second microservice node further includes a plurality of data processors, and the target data collector cluster corresponding to the second URL circular queue is controlled by the first microservice node to execute the network device indicator collection task, and step A3 may include:

[0137] Step A31, obtaining a system configuration file through the first microservice node, determining a target network device corresponding to the network device indicator collection task based on the system configuration file, and collecting data from the target network device to obtain target indicator data;

[0138] Step A32, obtaining, through the target data collector cluster, a third URL circular queue corresponding to the multiple data processors and second URL response status data corresponding to the multiple data processors, updating the third URL circular queue based on the second URL response status data to obtain a fourth URL circular queue, and determining a target data processor based on the fourth URL circular queue;

[0139] Step A33, processing the target indicator data by the target data processor to obtain an indicator processing result.

[0140] Specifically, the system configuration file is used to determine the target network devices of the network device index collection task, and collect data from these target network devices to finally obtain the target index data.

[0141] Specifically, the Collector (i.e., the data collector in the target data collector cluster) reads the Processor (i.e., the target data processor) communication interface from the system configuration file and stores them in the third URL circular queue. More specifically, when the Collector sends the collection result to the Processor, it takes out the URL from the third URL circular queue of the communication interface of the Processor and sends it through the POST method, thereby realizing the load balancing of multiple active Processors. Thus, the target data collector cluster obtains the second URL response status data corresponding to the multiple data processors. Based on the second URL response status data, if three consecutive attempts to send the collection result to the extracted URL fail, it is determined that the data processor corresponding to the URL is no longer available. The Collector first pushes an alarm through the alarm event HTTP API, and then temporarily removes this URL from the third URL circular queue, thereby obtaining the fourth URL circular queue, and then determines the communicable target data processor based on the fourth URL circular queue, and puts the collection task back into the collection queue and takes the next URL for sending. In addition, TaskMaker regularly reads the new system configuration file to update the circular queue and maintain the continuity and reliability of task allocation, so as to improve the system's adaptive ability and overall stability when the collector fails.

[0142] In the embodiment of the present application, through the above scheme, the Collector reads the communication interface of the Processor from the system configuration file and stores them in the URL circular queue. When the Collector sends the collection results to the Processor, it takes the URL from the circular queue of the communication interface of the Processor and sends it, so as to achieve load balancing of multiple active Processors. If the sending to the extracted URL fails three times in a row, the URL will be temporarily removed from the circular queue and the next URL will be taken for sending. In addition, TaskMaker periodically reloads the system configuration file to update the circular queue and maintain the continuity and reliability of task allocation. In this way, the system's adaptive ability and overall stability can be improved when the data collector fails.

[0143] Based on the third embodiment of the present application, a fourth embodiment of the present application is proposed. In the fourth embodiment of the present application, the same or similar contents as those in the above-mentioned third embodiment can be referred to the above introduction, and will not be repeated later. In this embodiment, the target indicator data is processed by the target data processor to obtain the indicator processing result, and step A33 may include:

[0144] Step A40, sending the indicator processing result to the third microservice node of the collection end, wherein the third microservice node includes a primary data reporter and a backup data reporter;

[0145] The step of sending the indicator processing result to the third microservice node of the acquisition end includes:

[0146] Step A401, when the target data device cannot send the indicator processing result to the main data reporter, obtain the main and standby switching time of the historical data reporter and the current system time;

[0147] Step A402: If the absolute difference between the active-standby switching time of the historical data reporter and the current system time is less than a preset time threshold, the data reporter communication interface in the memory of the third microservice node is configured as the communication interface of the standby data reporter to control the target data processor to send the indicator data result to the standby data reporter based on the communication interface of the standby data reporter.

[0148] Reference Figure 7 , Figure 7 A schematic diagram of a process of performing active / standby switching of a data reporter provided in the fourth embodiment of the network device index collection method of the present application; Figure 7As shown, the third microservice node includes a primary data reporter and a backup data reporter; the data reporter (DataHub) also adopts a primary-backup and automatic switching design. The Processor reads the primary-backup interface information of the DataHub from the system configuration, and attempts to send the processed data to the communication interface of the primary DataHub through the POST method. If three consecutive attempts to send fail, it is determined that the primary is faulty, and the Processor puts the data back into the processing queue and initiates a DataHub primary-backup switching request. At this time, the Processor compares the current system time with the last DataHub switching time to check whether it has been more than ten minutes since the last primary-backup switching (if it exceeds 10 minutes, it means that the primary-backup DataHub switching has exceeded a certain time, otherwise frequent switching is likely to cause system problems). If this time threshold is exceeded, the Processor will actively initiate the primary-backup switching, update the DataHub primary-backup communication interface information in the memory, and notify the primary TaskMaker to synchronize this change to the system configuration through the DataHub change notification interface. After receiving the change notification, the primary TaskMaker first calls the DataHub change interface to update the system configuration, then writes the updated properties into the system configuration file, and notifies all related services to synchronize the new configuration information. This ensures that when the DataHub main interface fails, the system can quickly switch to the backup interface to maintain the continuity of data transmission and the stable operation of the system.

[0149] Through the above scheme, this embodiment realizes the master-slave setting and automatic switching design of the microservice data reporter. The Processor reads the master-slave interface information of the DataHub from the system configuration and attempts to send the processed data to the master interface. If three consecutive attempts to send fail, the Processor will put the data back into the processing queue and check the time since the last master-slave switch. If it exceeds ten minutes, the Processor will initiate a master-slave switch, update the DataHub interface information in the memory, and notify TaskMaker to synchronize this change to the system configuration. TaskMaker then updates the system configuration and notifies all related services to synchronize the new configuration information. In this way, it can be ensured that the system can quickly switch to the backup interface when the primary interface fails, maintaining the continuity of data transmission and the stable operation of the system.

[0150] Based on the first embodiment of the present application, the fifth embodiment of the present application is proposed. In the fifth embodiment of the present application, the same or similar contents as those in the above-mentioned first embodiment can be referred to the above introduction, and will not be repeated later. In this embodiment, the method further includes:

[0151] Step B1, obtaining a system configuration file through the first microservice node to determine the load balancing frequency of the network device indicator collection task based on the system configuration file;

[0152] Step B2, calculating the task delivery interval based on the load balancing frequency;

[0153] Step B3: Sending the network device indicator collection task to the second microservice node through the first microservice node based on the task delivery interval.

[0154] Specifically, TaskMaker can control the frequency of sending collection tasks according to the system configuration file. Figure 8 , Figure 8 A flowchart of implementing load balancing provided in the fifth embodiment of the network device index collection method of the present application; Figure 8 As shown in the figure, when sending a collection task to the Collector of the multi-active mechanism, the main TaskMaker first obtains all collection task configurations from the system configuration file and adds all collection tasks to the collection job queue in sequence. When sending a task, the main TaskMaker first takes out the first collection task from the collection job queue, then reads the task parameters, and then sends the collection task to the communication interface of the Collector to complete the sending. Next, the main TaskMaker determines whether the load balancing frequency ratio set in the configuration file is greater than 0. If it is less than or equal to 0, there is no need to perform load balancing, and the parameters of the next collection task are directly read from the queue. If it is greater than 0, the interval time for sending the next collection task is calculated and set, and the main TaskMaker suspends sending collection tasks until the interval time ends. Next, the main TaskMaker checks whether the task queue is empty. If it is not empty, it takes the configuration of the next task from the queue and repeats the above process. If it is empty, it means that all collection tasks have been sent.

[0155] Based on the above solution, this embodiment obtains the system configuration file through the first microservice node to determine the load balancing frequency of the network device indicator collection task based on the system configuration file; calculates the task delivery interval based on the load balancing frequency; and sends the network device indicator collection task to the second microservice node based on the task delivery interval through the first microservice node. Based on the above process, TaskMaker introduces appropriate delays between the processes of issuing collection tasks to achieve sequential collection load balancing in the collection interval time dimension, avoiding high concurrent centralized collection causing severe collection service and network equipment resource overhead.

[0156] Based on the third embodiment of the present application, the sixth embodiment of the present application is proposed. In the sixth embodiment of the present application, the same or similar contents as those in the above-mentioned third embodiment can be referred to the above introduction, and will not be repeated later. In this embodiment, the method further includes:

[0157] Step C1, reading by the first microservice node based on the system configuration file to obtain a threshold event generation rule, and synchronizing the threshold event generation rule to the target data processor;

[0158] Step C2, the target data processor matches the collection object of the network device indicator collection task based on the threshold event generation rule to obtain a collection object matching result;

[0159] Step C3, when the collection object matching result is a successful match, if the target indicator data meets the threshold alarm condition in the threshold event generation rule, the threshold alarm data is generated by the target data processor, and the threshold alarm data is converted into visual data to push the threshold alarm data to the target user.

[0160] Reference Fig. 9 , Fig. 9 A flowchart of generating and sending threshold events provided in the sixth embodiment of the network device indicator collection method of the present application; Fig. 9 As shown in the figure, the administrator creates or updates the event generation rules through the WebServer front end, and then sends the event generation rules to the collection end TaskMaker in JSON format through a dedicated delivery channel. After receiving the event generation rules, the main TaskMaker updates the local configuration file. After these configuration changes are detected by the file monitoring thread, the main TaskMaker first obtains the IP addresses of all Processors and standby TaskMakers in the microservice IP dictionary, and then calls the event rule configuration synchronization method to send the event generation rules to the event generation rule synchronization interface to synchronize the event generation rule configuration to all Processors and standby TaskMakers. After receiving the new event generation rule configuration, the Processor first determines the rule type and updates the corresponding rule local configuration file. After the file monitoring thread detects the configuration update, it calls the initialize event rule parameter method to read the updated rule configuration, and creates a threshold event generation rule based on the configuration and caches it in the application memory. After receiving the event rule configuration JSON, the standby TaskMaker writes it to the local corresponding configuration file. The file monitoring thread does not trigger the rule update when it detects the configuration update and confirms that it is in the standby state.

[0161] Specifically, the embodiment of the present application also provides specific field settings of the threshold event rule, refer to Table 4:

[0162]

[0163]

[0164]

[0165] Table 4

[0166] As shown in Table 4, the threshold event rule configuration adopts the form of key-value pairs, including basic task information (such as the abbreviation of the collection site name, task ID), IP interface information dictionary (bandwidth mapping, interface list and type), and collection configuration (IP list, indicator name, rule item list, collection namespace and extended information). Among them, the data structure of the rule item list is a JSON list, and each rule item contains the calculation type, event level, event name, threshold and number of matches. When the calculation type is 1, if the value corresponding to the event name exceeds the threshold, an alarm is pushed; when the calculation type is 2, if the value corresponding to the event name does not exceed the threshold, an alarm is pushed; when the calculation type is other values, if the value corresponding to the event name can be matched with the calculation type, an alarm is pushed. The master control end sends the configuration file to the main TaskMaker of the collection end through a dedicated sending channel. TaskMaker receives and parses the configuration file, extracts the basic task information and collection configuration, creates and distributes the collection task, and synchronizes the configuration information to the standby TaskMaker to ensure the reliability of configuration sending and the efficiency of task distribution.

[0167] When the Processor processes the data results collected by the Collector, it first determines whether the collected data meets the threshold conditions. If it meets the alarm threshold, it queries the historical event cache dictionary of the DataHub main service through the interface, and constructs a key query dictionary based on the device IP, interface name (empty string if not), and event name to determine whether there are historical alarm events. If there are no historical alarm events, a new event data object is created and related properties are set, such as event level, event name, indicator name calculation type, threshold, etc. Then the Processor adds the event data to the historical event through the interface and sends it to the DataHub for caching. If the corresponding historical alarm event is queried, the creation time, indicator value, number of matches, and indicator mapping of the historical event data object are updated, and the number of matches is incremented once. If the number of event matches exceeds the cumulative number of alarm rule configurations, the alarm event is sent to the event alarm end (EventAlerter) service for processing and the DataHub interface is called to update the corresponding historical alarm event in its historical event cache dictionary. If the number of event matches does not exceed the cumulative number of alarm rule configurations, the DataHub interface is called to update the corresponding historical alarm event in its historical event cache dictionary. If the alarm threshold condition is not met, check whether there is historical event data. If there is a corresponding historical alarm event, it is determined that the event has been restored and the alarm event is sent to the event alarm end (EventAlerter) service for processing and the DataHub interface is called to delete the corresponding historical alarm event in its historical event cache dictionary. Among them, the historical event cache dictionary of the DataHub main service uses the device IP, interface name (empty string if not available), and event name as keys, and the event data object as value. Its query, add, update, and delete interfaces are configured in the site system configuration file, and are recorded in the memory variable when the Processor reads the configuration file. This design ensures that the system can generate threshold events in a timely and effective manner, improving the efficiency of event discovery to processing.

[0168] The embodiment of the present application uses the above scheme to create or update event generation rules specifically through the WebServer front end, and then sends this information to the collection end TaskMaker in JSON format through the WebServer. After receiving, TaskMaker updates the local configuration file. After these configurations are detected to be updated by the file monitoring thread, the configuration files are synchronized to the backup service of TaskMaker and all Processors. After receiving, the Processor updates the local configuration file. After the file monitoring thread detects the configuration update, it reads the updated rule configuration and creates a threshold event generation rule cached in the application memory. When the Processor processes the collected data, it matches the collected data with the collection object and compares the threshold event generation conditions according to these threshold event generation rules. Once the collection object is successfully matched and the event generation conditions are met, the corresponding threshold event is generated and sent to the event alert end (EventAlerter) service for processing. In this way, it can ensure that the system can generate threshold events in a timely and effective manner, thereby improving the efficiency of event discovery to processing.

[0169] Based on the first embodiment of the present application, a seventh embodiment of the present application is proposed. In the seventh embodiment of the present application, the same or similar contents as those in the above-mentioned first embodiment can be referred to the above introduction, and will not be repeated later. In this embodiment, the method further includes:

[0170] Step C1, obtaining the heartbeat Kafka data of each microservice node in the acquisition end, and judging whether there is data missing based on the heartbeat Kafka data;

[0171] Step C2: if there is data missing, a supplementary recording instruction is generated based on the site system configuration, and the second microservice node is controlled to perform an automatic supplementary recording task based on the supplementary recording instruction.

[0172] Specifically, refer to Fig.10 , Fig.10 A flow chart of automatic supplementary recording at the collection end provided in the seventh embodiment of the network device index collection method of the present application; Fig.10As shown in the figure, the data collector (Collector) and the data processor (Processor) poll all the collection task lists to obtain the collected data according to the collection site system settings. If the data has been collected, a new backup file is created in a specific data type format to back up the collected data locally. In addition, the Collector and Processor regularly clean up the backup files according to the upper limit of the backup file storage time set locally to ensure sufficient local storage space and the efficiency of querying backup files. In the normal data processing process, the Processor obtains the output channel corresponding to the data type from the system configuration and sends data to the data reporting interface of the data reporter (DataHub). The DataHub will synchronize the heartbeat package of the details of the lost tasks within the last heartbeat interval through the heartbeat Kafka. The self-monitoring module of the master control end parses the heartbeat message to determine whether there is data loss. Once data loss is detected, it can be determined whether to automatically generate a collection loss event and send an event alarm (EventAlerter) service according to the site system configuration. At the same time, the self-monitoring module of the master control end can also determine whether to issue a re-recording instruction based on the site system configuration. The data collector (Collector) and the data processor (Processor) receive the re-recording instruction and call the re-recording interface by task ID or the re-recording interface by collection time according to the instruction type to retrieve and resend the lost data from the local backup.

[0173] In a feasible implementation manner, the step of controlling the second microservice node to perform the automatic supplementary recording task based on the supplementary recording instruction includes:

[0174] Step D1, if the supplementary recording instruction includes a task identification ID, constructing a target local backup file index path corresponding to the task ID based on the task ID;

[0175] Step D2, retrieving the backup file based on the target local backup index path, obtaining the target backup file corresponding to the task ID, so as to control the second microservice node to automatically supplement the recording based on the target backup file.

[0176] Specifically, if the supplementary recording instruction received by the Collector is to supplement the recording according to the task ID, the Collector first builds a local backup file index path based on the task ID, then calls the file reading method to retrieve the backup file from the corresponding index path, and then finds the backup data corresponding to the task ID from the search results, and finally sends the data back to the Processor through the corresponding Processor communication interface in the system configuration file for processing.

[0177] In another feasible implementation manner, the step of controlling the second microservice node to perform the automatic supplementary recording task based on the supplementary recording instruction includes:

[0178] Step E1: if the supplementary recording instruction includes a collection time interval, controlling the second microservice node to construct a full backup file index path set, so as to call a file reading method to retrieve a full backup file set based on the full backup file index path set;

[0179] Step E2: screening the full backup file set based on the collection time interval to obtain a target backup file that meets the collection time interval, so as to control the second microservice node to automatically supplement the recording based on the target backup file.

[0180] Specifically, if the supplementary recording instruction received by the Collector is to supplement the recording according to the task collection time, the Collector first constructs and obtains the index path of all local backup files, and then calls the file reading method to retrieve the backup file from the corresponding index path, and then extracts the collection time of the data from the search results, and then filters the backup data within the time range according to the task start time and end time, and finally sends the data to the Processor through the corresponding Processor communication interface in the system configuration file for processing. The process of supplementary recording after the Processor receives the supplementary recording instruction is similar to the Collector's supplementary recording process. After the Processor obtains the backup data, it uses the POST method again to send the Collector or its own supplementary recording data to the DataHub to complete the automatic supplementary recording. The above process realizes rapid data recovery and ensures data integrity.

[0181] This embodiment uses the above scheme, and the data collector (Collector) and the data processor (Processor) perform local backup when the corresponding collected data is obtained according to the collection site system settings. In the normal data processing process, the collected data is sent to the designated output channel through the data reporter, and the data reporter synchronizes the heartbeat package of the details of the lost task within the last heartbeat interval through the heartbeat Kafka. The self-monitoring module of the master control end parses the heartbeat message to determine whether there is data loss. Once the data loss is detected, it can determine whether to automatically generate the collection loss event to send the event alarm (EventAlerter) service according to the site system configuration. At the same time, the self-monitoring module of the master control end can also determine whether to issue a supplementary recording instruction according to the site system configuration. If the data collector (Collector) and the data processor (Processor) receive the supplementary recording instruction, they can retrieve and resend the lost data from the local backup, thereby realizing rapid data recovery and ensuring data integrity.

[0182] Furthermore, the embodiment of the present application also proposes a self-monitoring process of the network equipment indicator collection system.

[0183] Reference Fig.11 , Fig.11 The flowchart of the collection end self-monitoring provided in the seventh embodiment of the network device index collection method of the present application is as follows. Fig.11 As shown, the task distributor (TaskMaker), data collector (Collector), data processor (Processor) and data reporter (DataHub) all send heartbeat packets to DataHub through POST requests, which record the IP address of the site to which they belong, their own roles, and details of the tasks lost during the last heartbeat interval. DataHub transmits this data to the master control end through the heartbeat Kafka. If the self-monitoring module of the master control end does not receive data from a microservice at the production collection site for two consecutive heartbeat cycles, it will first identify this as a timeout, then generate a collection service abnormal event, and push an event alarm (EventAlerter); then, add or update the alarm record. If the self-monitoring module of the master control end detects that the heartbeat data of a microservice at the production collection site has returned to normal, it will first push a recovery alarm and then delete the corresponding alarm record. In addition to passively receiving heartbeat data, the master control end self-monitoring module will also actively detect the key performance indicators of each microservice at each site; during active detection, the master control end self-monitoring module first obtains the IP addresses of all services from the microservice IP dictionary, and then sends a GET query request to the health indicator value query interface of all services based on the IP address, requesting each service to upload CPU usage, disk usage, memory usage, etc. When the detection is successful, the master control end self-monitoring module first stores the return results of each service in the database. If the return result is higher than the threshold, the indicator over-threshold event is generated based on the result and the event alarm (EventAlerter) service is sent, and finally the alarm record is added or updated. If the return result returns to below the threshold, a recovery alarm event is generated and the event alarm (EventAlerter) service is sent, and finally the alarm record is deleted. When the detection fails, the collection service abnormal event is also generated and the event alarm (EventAlerter) service is sent.

[0184] In this embodiment, through the above scheme, the task distributor (TaskMaker), data collector (Collector), data processor (Processor) and data reporter (DataHub) all send heartbeat packets to DataHub to record the site to which they belong, their own roles, and the details of the tasks lost in the last heartbeat interval. These data are transmitted to the master control end through the heartbeat Kafka. If the self-monitoring module of the master control end does not receive data for two consecutive heartbeat cycles for a microservice at the production collection site, it will be identified as a timeout and automatically generate an event alert (EventAlerter) service for abnormal events of the collection service. In addition to passively receiving heartbeats, the self-monitoring module of the master control end will also actively detect the key performance indicators of each microservice at each site, such as CPU, memory and disk, and generate an event alert (EventAlerter) service for indicators exceeding the threshold event based on the results; if the detection fails, an event alert (EventAlerter) service for abnormal events of the collection service will also be generated.

[0185] This application also provides a network equipment index collection device, please refer to Fig.12 , the network equipment indicator collection device comprises:

[0186] The task acquisition module 10 is used to call the first microservice node in the collection end to obtain the network device indicator collection task, wherein the first microservice node includes a main task distributor and a backup task distributor;

[0187] The task issuing module 20 is used to switch the backup task distributor to the active state if the primary task distributor does not respond to the service connection request of the backup task distributor within a preset service connection period, and call the backup task distributor to send the network device indicator collection task to the second microservice node in the collection end, wherein the second microservice node pre-deploys a multi-active server;

[0188] The task execution module 30 is used to control the second microservice node through the first microservice node to execute the network device indicator collection task.

[0189] The network equipment index collection system includes a master control terminal, and the task issuing module 30 is also used for:

[0190] Calling the first microservice node in the acquisition end to receive the task configuration data sent by the master control end, parsing the task configuration data based on the task configuration data, and obtaining a task name and a task switch;

[0191] The scheduled creation task and distribution thread of the first microservice node are updated based on the task name and the task switch to obtain the network device indicator collection task.

[0192] The second microservice node includes a plurality of data collectors, and the task acquisition module 10 is further used for:

[0193] Traversing the uniform resource locators URLs corresponding to the communication interfaces of the plurality of data collectors respectively through the first microservice node to obtain a first URL circular queue;

[0194] Obtaining first URL response status data through the first microservice node, traversing and reading each URL in the first URL circular queue, so as to update the first URL circular queue based on the first URL response status data, and obtain a second URL circular queue;

[0195] The target data collector cluster corresponding to the second URL circular queue is controlled by the first microservice node to execute the network device indicator collection task.

[0196] The task acquisition module 10 is also used for:

[0197] For any target URL read through the traversal, judging whether the data collector corresponding to the target URL can respond to the network device indicator collection task based on the first URL response status data;

[0198] If not, remove the target URL from the first URL circular queue;

[0199] This time, each URL in the first URL circular queue is traversed until the second URL circular queue is obtained.

[0200] The second microservice node further includes a plurality of data processors, and the task execution module 30 is further used for:

[0201] Obtaining a system configuration file through the first microservice node, determining a target network device corresponding to the network device indicator collection task based on the system configuration file, and collecting data on the target network device to obtain target indicator data;

[0202] Acquire a third URL circular queue corresponding to the plurality of data processors and second URL response status data corresponding to the plurality of data processors through the target data collector cluster, update the third URL circular queue based on the second URL response status data to obtain a fourth URL circular queue, and determine a target data processor based on the fourth URL circular queue;

[0203] The target indicator data is processed by the target data processor to obtain an indicator processing result.

[0204] The task execution module 30 is also used for:

[0205] Sending the indicator processing result to the third microservice node of the acquisition end, wherein the third microservice node includes a primary data reporter and a backup data reporter;

[0206] The step of sending the indicator processing result to the third microservice node of the acquisition end includes:

[0207] When the target data recorder cannot send the indicator processing result to the master data recorder, obtain the master-slave switching time of the historical data recorder and the current system time;

[0208] If the absolute difference between the active-standby switching time of the historical data reporter and the current system time is less than a preset time threshold, the data reporter communication interface in the memory of the third microservice node is configured as the communication interface of the standby data reporter to control the target data processor to send the indicator data result to the standby data reporter based on the communication interface of the standby data reporter.

[0209] The task issuing module 20 is also used for:

[0210] Obtaining a system configuration file through the first microservice node to determine a load balancing frequency of the network device indicator collection task based on the system configuration file;

[0211] Calculate the task delivery interval based on the load balancing frequency;

[0212] The network device indicator collection task is sent to the second microservice node through the first microservice node based on the task delivery interval.

[0213] The task acquisition module 10 is also used for:

[0214] The first microservice node reads the system configuration file to obtain a threshold event generation rule, and synchronizes the threshold event generation rule to the target data processor;

[0215] The target data processor matches the collection object of the network device indicator collection task based on the threshold event generation rule to obtain a collection object matching result;

[0216] When the collection object matching result is a successful match, if the target indicator data meets the threshold alarm condition in the threshold event generation rule, the threshold alarm data is generated by the target data processor, and the threshold alarm data is converted into visual data to push the threshold alarm data to the target user.

[0217] The task execution module 30 is also used for:

[0218] Obtain the heartbeat Kafka data of each microservice node in the collection end, and determine whether there is data missing based on the heartbeat Kafka data;

[0219] If there is data missing, a supplementary recording instruction is generated based on the site system configuration, and the second microservice node is controlled to perform an automatic supplementary recording task based on the supplementary recording instruction.

[0220] The task execution module 30 is also used for:

[0221] If the supplementary recording instruction includes a task identification ID, constructing a target local backup file index path corresponding to the task ID based on the task ID;

[0222] The backup file is retrieved based on the target local backup index path to obtain the target backup file corresponding to the task ID, so as to control the second microservice node to automatically supplement the recording based on the target backup file.

[0223] The task execution module 30 is also used for:

[0224] If the supplementary recording instruction includes a collection time interval, controlling the second microservice node to construct a full backup file index path set, so as to call a file reading method to retrieve a full backup file set based on the full backup file index path set;

[0225] The full backup file set is screened based on the collection time interval to obtain a target backup file that meets the collection time interval, so as to control the second microservice node to automatically supplement the recording based on the target backup file.

[0226] The network equipment indicator collection device provided by the present application adopts the network equipment indicator collection method in the above embodiment, which can solve the technical problem of how to improve the full-link availability of the network equipment indicator collection method. Compared with the prior art, the beneficial effects of the network equipment indicator collection device provided by the present application are the same as the beneficial effects of the network equipment indicator collection method provided by the above embodiment, and the other technical features in the network equipment indicator collection device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0227] The present application provides a network device indicator collection device, which includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the network device indicator collection method in the above-mentioned embodiment one.

[0228] Reference below Fig.13, which shows a schematic diagram of the structure of a network device indicator collection device suitable for implementing the embodiment of the present application. The network device indicator collection device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.13 The network device indicator collection device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0229] like Fig.13 As shown, the network device index collection device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the network device index collection device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the network device indicator collection device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a network device indicator collection device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0230] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0231] The network device indicator collection device provided by the present application adopts the network device indicator collection method in the above embodiment, which can solve the technical problem of how to improve the full-link availability of the network device indicator collection method. Compared with the prior art, the beneficial effects of the network device indicator collection device provided by the present application are the same as the beneficial effects of the network device indicator collection method provided by the above embodiment, and the other technical features in the network device indicator collection device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0232] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0233] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0234] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the network device indicator collection method in the above-mentioned embodiment.

[0235] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0236] The above-mentioned computer-readable storage medium may be included in the network device indicator collection device; or may exist independently without being assembled into the network device indicator collection device.

[0237] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the network device indicator collection device, the network device indicator collection device: calls the first microservice node in the collection end to obtain the network device indicator collection task, wherein the first microservice node includes a main task distributor and a backup task distributor; if the main task distributor does not respond to the service connection request of the backup task distributor within a preset service connection period, the backup task distributor is switched to the main state, and the backup task distributor is called to send the network device indicator collection task to the second microservice node in the collection end, wherein the second microservice node is a node pre-set based on a multi-active mechanism; the second microservice node is controlled by the first microservice node to execute the network device indicator collection task.

[0238] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0239] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0240] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0241] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned network device indicator collection method, and can solve the technical problem of how to improve the full-link availability of the network device indicator collection method. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the network device indicator collection method provided in the above-mentioned embodiment, and will not be repeated here.

[0242] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned network device indicator collection method when executed by a processor.

[0243] The computer program product provided by this application can solve the technical problem of how to improve the full-link availability of the network device indicator collection method. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the network device indicator collection method provided by the above embodiment, which will not be repeated here.

[0244] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for collecting network equipment indicators, characterized in that: Applied to a network equipment index collection system, the network equipment index collection system includes a collection end, and the method includes: Calling a first microservice node in the collection end to obtain a network device indicator collection task, wherein the first microservice node includes a main task distributor and a backup task distributor; If the main task distributor does not respond to the service connection request of the backup task distributor within the preset service connection period, the backup task distributor is switched to the active state, and the backup task distributor is called to send the network device indicator collection task to the second microservice node in the collection end, wherein the second microservice node is a node pre-set based on the multi-active mechanism; The second microservice node is controlled by the first microservice node to execute the network device indicator collection task.

2. The method according to claim 1, characterized in that The network device indicator collection system includes a master control terminal, and the step of calling the first microservice node in the collection terminal to obtain a network device indicator collection task includes: Calling the first microservice node in the acquisition end to receive the task configuration data sent by the master control end, parsing the task configuration data based on the task configuration data, and obtaining a task name and a task switch; The scheduled creation task and distribution thread of the first microservice node are updated based on the task name and the task switch to obtain the network device indicator collection task.

3. The method according to claim 1, characterized in that The second microservice node includes a plurality of data collectors, and the step of controlling the second microservice node through the first microservice node to perform the network device indicator collection task includes: Traversing the uniform resource locators URLs corresponding to the communication interfaces of the plurality of data collectors respectively through the first microservice node to obtain a first URL circular queue; Obtaining first URL response status data through the first microservice node, traversing and reading each URL in the first URL circular queue, so as to update the first URL circular queue based on the first URL response status data, and obtain a second URL circular queue; The target data collector cluster corresponding to the second URL circular queue is controlled by the first microservice node to execute the network device indicator collection task.

4. The method according to claim 3, characterized in that The step of traversing and reading each URL in the first URL circular queue to update the first URL circular queue based on the first URL response status data to obtain a second URL circular queue includes: For any target URL read through the traversal, judging whether the data collector corresponding to the target URL can respond to the network device indicator collection task based on the first URL response status data; If not, remove the target URL from the first URL circular queue; This time, each URL in the first URL circular queue is traversed until the second URL circular queue is obtained.

5. The method according to claim 3, characterized in that The second microservice node also includes a plurality of data processors, and the step of controlling the target data collector cluster corresponding to the second URL circular queue through the first microservice node to execute the network device indicator collection task includes: Obtaining a system configuration file through the first microservice node, determining a target network device corresponding to the network device indicator collection task based on the system configuration file, and collecting data on the target network device to obtain target indicator data; Acquire a third URL circular queue corresponding to the plurality of data processors and second URL response status data corresponding to the plurality of data processors through the target data collector cluster, update the third URL circular queue based on the second URL response status data to obtain a fourth URL circular queue, and determine a target data processor based on the fourth URL circular queue; The target indicator data is processed by the target data processor to obtain an indicator processing result.

6. The method according to claim 5, characterized in that The step of processing the target indicator data by the target data processor to obtain the indicator processing result includes: Sending the indicator processing result to the third microservice node of the acquisition end, wherein the third microservice node includes a primary data reporter and a backup data reporter; The step of sending the indicator processing result to the third microservice node of the acquisition end includes: When the target data recorder cannot send the indicator processing result to the master data recorder, obtain the master-slave switching time of the historical data recorder and the current system time; If the absolute difference between the active-standby switching time of the historical data reporter and the current system time is less than a preset time threshold, the data reporter communication interface in the memory of the third microservice node is configured as the communication interface of the standby data reporter to control the target data processor to send the indicator data result to the standby data reporter based on the communication interface of the standby data reporter.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining a system configuration file through the first microservice node to determine a load balancing frequency of the network device indicator collection task based on the system configuration file; Calculate the task delivery interval based on the load balancing frequency; The network device indicator collection task is sent to the second microservice node through the first microservice node based on the task delivery interval.

8. The method according to claim 5, characterized in that The method further comprises: The first microservice node reads the system configuration file to obtain a threshold event generation rule, and synchronizes the threshold event generation rule to the target data processor; The target data processor matches the collection object of the network device indicator collection task based on the threshold event generation rule to obtain a collection object matching result; When the collection object matching result is a successful match, if the target indicator data meets the threshold alarm condition in the threshold event generation rule, the threshold alarm data is generated by the target data processor, and the threshold alarm data is converted into visual data to push the threshold alarm data to the target user.

9. The method according to claim 1, characterized in that The method further comprises: Obtain the heartbeat Kafka data of each microservice node in the collection end, and determine whether there is data missing based on the heartbeat Kafka data; If there is data missing, a supplementary recording instruction is generated based on the site system configuration, and the second microservice node is controlled to perform an automatic supplementary recording task based on the supplementary recording instruction.

10. The method according to claim 9, characterized in that The step of controlling the second microservice node to perform the automatic supplementary recording task based on the supplementary recording instruction includes: If the supplementary recording instruction includes a task identification ID, constructing a target local backup file index path corresponding to the task ID based on the task ID; The backup file is retrieved based on the target local backup index path to obtain the target backup file corresponding to the task ID, so as to control the second microservice node to automatically supplement the recording based on the target backup file.

11. The method according to claim 9, characterized in that The step of controlling the second microservice node to perform the automatic supplementary recording task based on the supplementary recording instruction includes: If the supplementary recording instruction includes a collection time interval, controlling the second microservice node to construct a full backup file index path set, so as to call a file reading method to retrieve a full backup file set based on the full backup file index path set; The full backup file set is screened based on the collection time interval to obtain a target backup file that meets the collection time interval, so as to control the second microservice node to automatically supplement the recording based on the target backup file.

12. A network equipment index collection device, characterized in that: The device comprises: A task acquisition module, used to call the first microservice node in the collection end to obtain a network device indicator collection task, wherein the first microservice node includes a main task distributor and a backup task distributor; A task issuing module, used for switching the backup task distributor to the active state if the primary task distributor does not respond to the service connection request of the backup task distributor within a preset service connection period, and calling the backup task distributor to send the network device indicator collection task to the second microservice node in the collection end, wherein the second microservice node pre-deploys a multi-active server; A task execution module is used to control the second microservice node through the first microservice node to execute the network device indicator collection task.

13. A network equipment indicator collection device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the network device indicator collection method according to any one of claims 1 to 11.

14. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the network device indicator collection method according to any one of claims 1 to 11 are implemented.

15. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the network device indicator collection method according to any one of claims 1 to 11 are implemented.

Images