A switch data collection method and system based on high-availability Prometheus

Abstract

The present invention discloses a switch data collection method and system based on highly available prometheus, wherein the collection method is implemented based on an SNMP collector cluster in a prometheus architecture, and includes: determining a corresponding SNMP collector node in an SNMP collector cluster according to a collection task; detecting whether the SNMP collector node cache has monitoring data corresponding to the collection task; if so, transmitting the monitoring data back to the prometheus node corresponding to the collection task; if not, acquiring the monitoring data and transmitting it back to the prometheus node. The present invention collects switch monitoring data and backs up multiple copies of data, and uses a distributed task scheduling algorithm to perform task control on the collection cluster, thereby effectively solving the problems of response delay or collection data loss caused by performing multiple collection tasks at the same time in the prior art, and significantly improving the collection efficiency and quality of switch monitoring data.

Description

Technical Field

[0001] The present invention belongs to the technical field of data monitoring, and in particular relates to a switch data collection method and system, equipment and medium based on high-availability Prometheus. Background Art

[0002] Prometheus is an open source complete data monitoring solution. It is based on a new model of centralized rule calculation, unified analysis and alarm. It is widely used in the monitoring field with its powerful data storage characteristics. Since the prometheus architecture itself only supports single-machine deployment, in the traditional switch data collection process, a single prometheus node sends the collection task to the snmp-exporter (SNMP collector, that is, the component used by the prometheus node for monitoring and collection). The snmp-exporter obtains the data of the specified switch from the SNMP protocol and provides it to the prometheus node. There is also a way to achieve high availability by deploying multiple prometheus nodes and collecting the same service, but in the high-availability prometheus architecture, multiple prometheus nodes work at the same time, and the snmp-exporter performs multiple collection tasks on the same switch at the same time. The traffic load on the switch side increases sharply, which is easy to cause collection task response delays and collection data loss, thereby greatly affecting the efficiency of switch data collection. Summary of the invention

[0003] The technical problem to be solved by the present invention is to overcome the above-mentioned defects in the prior art and provide a switch data collection method and system, device and medium based on high-availability Prometheus.

[0004] The present invention solves the above technical problems through the following technical solutions:

[0005] The present invention provides a switch data collection method based on high-availability Prometheus, which is implemented based on an SNMP collector cluster in a Prometheus architecture and is used to collect monitoring data of a distributed switch; the SNMP collector cluster includes at least two SNMP collector nodes, and the data collection method includes the following steps:

[0006] According to the collection task information sent by the prometheus node, determining the SNMP collector node corresponding to the collection task in the SNMP collector cluster;

[0007] Detect whether the SNMP collector node corresponding to the collection task has the monitoring data corresponding to the collection task;

[0008] If yes, the monitoring data is transmitted back to the prometheus node corresponding to the collection task; if no, the monitoring data is obtained through the SNMP collector node and transmitted back to the prometheus node corresponding to the collection task.

[0009] Preferably, the step of determining the SNMP collector node corresponding to the collection task in the SNMP collector cluster according to the collection task information sent by the prometheus node includes:

[0010] Get the collection task information sent by at least two prometheus nodes respectively;

[0011] All the collection task information is randomly distributed to each SNMP collector node in the SNMP collector cluster;

[0012] Determine the target SNMP collector node corresponding to the collection task according to the collection task obtained by each SNMP collector node and the currently available SNMP collector nodes recorded by the SNMP collector node;

[0013] Send the collection task to the target SNMP collector node.

[0014] Preferably, the step of determining the target SNMP collector node corresponding to the collection task according to the collection task acquired by each SNMP collector node and the currently available SNMP collector nodes recorded by the SNMP collector node comprises:

[0015] Reading the first list in each SNMP collector node respectively, wherein the first list records the IP addresses of the SNMP collector nodes that are currently in a state of being able to execute collection tasks;

[0016] Performing a consistent hash operation based on the first list to generate a Hash ring, each node of the Hash ring corresponds to the IP address of the SNMP collector node whose current state is to execute the collection task;

[0017] The IP address of the distributed switch corresponding to the collection task is mapped to the Hash ring through a consistent hash operation to determine the target SNMP collector node corresponding to the collection task.

[0018] Preferably, the monitoring data is obtained through the SNMP collector node and transmitted back to Place The steps of the prometheus node corresponding to the collection task include:

[0019] If the SNMP collector node is in an idle state, the monitoring data is obtained based on the SNMP protocol, and is transmitted back to the prometheus node corresponding to the collection task through the cache;

[0020] If the SNMP collector node is not in an idle state, blocking and waiting is performed until the SNMP collector node is in an idle state, the monitoring data is obtained based on the SNMP protocol, and the data is transmitted back to the prometheus node corresponding to the collection task through the cache.

[0021] The present invention also provides a switch data collection system based on high-availability prometheus, which is implemented based on an SNMP collector cluster in a prometheus architecture and is used to collect monitoring data of the distributed switch; the SNMP collector cluster includes at least two SNMP collector nodes, and the data collection system includes:

[0022] A load balancing module, used to determine the SNMP collector node corresponding to the collection task in the SNMP collector cluster according to the collection task information sent by the prometheus node;

[0023] The collection execution module is used to detect whether the SNMP collector node corresponding to the collection task has the monitoring data corresponding to the collection task; if so, the monitoring data is returned to the prometheus node corresponding to the collection task; if not, the monitoring data is obtained through the SNMP collector node and returned to the prometheus node corresponding to the collection task.

[0024] Preferably, the load balancing module includes:

[0025] A task acquisition unit is used to respectively acquire the collection task information sent by at least two prometheus nodes;

[0026] A task allocation unit, used for randomly allocating all the collection task information to each SNMP collector node in the SNMP collector cluster;

[0027] A node determination unit, configured to determine a target SNMP collector node corresponding to the collection task according to the collection task acquired by each SNMP collector node and the currently available SNMP collector nodes recorded by the SNMP collector node;

[0028] The task sending unit is used to send the collection task to the target SNMP collector node.

[0029] Preferably, the node determination unit includes:

[0030] The list reading subunit is used to read the first list in each SNMP collector node respectively, wherein the first list records the IP addresses of the SNMP collector nodes whose current status is that they can execute the collection task;

[0031] A hash table generation subunit, configured to perform a consistent hash operation based on the first list to generate a hash ring, wherein each node of the hash ring corresponds to an IP address of the SNMP collector node whose current state is to be able to execute the collection task;

[0032] The target node determination subunit is used to map the IP address of the distributed switch corresponding to the collection task to the hash ring through a consistent hash operation to determine the target SNMP collector node corresponding to the collection task.

[0033] Preferably, the collection execution module is specifically used for: if the SNMP collector node is in an idle state, obtaining the monitoring data based on the SNMP protocol, and transmitting it back to the prometheus node corresponding to the collection task through the cache; if the SNMP collector node is not in an idle state, blocking and waiting until the SNMP collector node is in an idle state, obtaining the monitoring data based on the SNMP protocol, and transmitting it back to the prometheus node corresponding to the collection task through the cache.

[0034] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned switch data collection method based on high-availability Prometheus when executing the computer program.

[0035] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned switch data collection method based on high-availability Prometheus is implemented.

[0036] The positive and progressive effect of the present invention is that the present invention provides a switch data collection method and system, device and medium based on high-availability prometheus, which can determine the SNMP collector node corresponding to the collection task in the SNMP collector cluster according to the collection task information sent by the prometheus node, and then perform the collection and return of the monitoring data of the distributed switch according to the detection result of whether the SNMP collector node has the monitoring data corresponding to the collection task, thereby improving the existing problems of the high-availability prometheus architecture. The present invention effectively solves the problem of collection task response delay or collection data loss caused by the switch performing multiple collection tasks at the same time in the existing high-availability prometheus architecture by performing switch monitoring data collection and multi-copy data backup, and using a distributed task scheduling algorithm to control the collection cluster, and significantly improves the collection efficiency and quality of the switch monitoring data. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flow chart of a switch data collection method based on high-availability Prometheus according to Example 1 of the present invention.

[0038] Figure 2 This is a schematic diagram of the health detection of the SNMP collector in Example 1 of the present invention.

[0039] Figure 3 The figure is a schematic diagram of the selection process of the SNMP collector in Example 1 of the present invention.

[0040] Figure 4 This is a schematic diagram of the process of the SNMP collector executing a collection task in Example 1 of the present invention.

[0041] Figure 5 This is a module diagram of a switch data collection system based on high-availability Prometheus according to Example 2 of the present invention.

[0042] Figure 6 This is a structural block diagram of an electronic product according to Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0043] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0044] The switch data collection method based on high-availability Prometheus provided in this embodiment can be executed in an intelligent terminal, a computer terminal, a network device, a chip, a chip module or a similar computing device. Mentioning "embodiment" in this application means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0045] It should be understood that the "system", "device", "unit" and / or "module" used herein is a method for distinguishing different components, elements, parts, parts or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions. As shown in this specification, unless the context clearly indicates an exception, the words "one", "a", "a" and / or "the" do not specifically refer to the singular, but may also include the plural. Generally speaking, the terms "comprises" and "includes" only indicate the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements.

[0046] Example 1

[0047] This embodiment specifically provides a switch data collection method based on high-availability prometheus. The data collection method is implemented based on an SNMP collector cluster in a prometheus architecture and is used to collect monitoring data of distributed switches. The monitoring data includes but is not limited to switch status, name, number of I / O messages, etc.; wherein the SNMP collector cluster includes at least two SNMP collector nodes.

[0048] Specifically, the collection process can be implemented by setting the following three modules:

[0049] The open source monitoring component prometheus is responsible for issuing collection tasks to the load balancing component, and collecting and storing monitoring data; the load balancing component is responsible for randomly and evenly distributing the collection tasks issued by multiple prometheus nodes to the nodes in the SNMP collector cluster; the SNMP collector, which is an improved version of the switch data collection component snmp-exporter, has data caching and synchronization task functions, is responsible for using the SNMP protocol to collect switch data, and provides data to prometheus. Multiple SNMP collectors can form an SNMP collector cluster, support high availability, and avoid single point failure problems. Of course, those skilled in the art will know that the collection method of this embodiment is not limited by the modules and their names defined above.

[0050] The data collection method includes the following steps:

[0051] S1. According to the collection task information sent by the prometheus node, determine the SNMP collector node corresponding to the collection task in the SNMP collector cluster;

[0052] S2. Detect whether the SNMP collector node cache corresponding to the collection task has the monitoring data corresponding to the collection task; if so, execute S3: transmit the monitoring data back to the prometheus node corresponding to the collection task; if not, execute S4: obtain the monitoring data through the SNMP collector node and transmit it back to the prometheus node corresponding to the collection task. Those skilled in the art will know that to determine whether the SNMP collector node has the monitoring data corresponding to the collection task, the SNMP collector node cache can be detected.

[0053] As a preferred implementation, according to the collection task information sent by the prometheus node, the step of determining the SNMP collector node corresponding to the collection task in the SNMP collector cluster includes:

[0054] Get the collection task information sent by at least two prometheus nodes respectively;

[0055] All collection task information is randomly distributed to each SNMP collector node in the SNMP collector cluster;

[0056] Determine the target SNMP collector node corresponding to the collection task according to the collection task obtained by each SNMP collector node and the currently available SNMP collector nodes recorded by the SNMP collector node;

[0057] Send the collection task to the target SNMP collector node.

[0058] Among them, the process of sending Prometheus collection tasks is that in the high-availability prometheus architecture, N prometheus nodes collect monitoring data of M switches at the same time, and each prometheus node sends M collection tasks to the load balancing component. After receiving the collection tasks, the load balancing component randomly and evenly distributes the collection tasks to each node of the SNMP collector cluster, and uses the distributed task scheduling algorithm to non-center and adaptively select the SNMP collector nodes with good running status in the SNMP collector cluster to perform the collection tasks. For the judgment of the available status of the SNMP collector node, it can be based on the regular detection of the running status of each node in the SNMP collector cluster, and each update their own available node IP address table in this period to achieve synchronization. The selected SNMP collector node collects data from the specified switch according to the IP address of the switch included in the collection task, and provides the data to the prometheus node that sent the collection task.

[0059] refer to Figure 2 , multiple SNMP collectors constitute an SNMP collector cluster. Each node in the SNMP collector cluster periodically accesses its own health detection http interface to obtain the status information of other nodes in the cluster. The health detection interface is as follows:

[0060] http: / / SNMP collector's ip:port / health. If the returned text is "ok", it means that the node is running normally. Otherwise, it means that the node is not running normally.

[0061] After obtaining the status information of other nodes in the cluster, the node writes the addresses of nodes that are operating normally during the current period into its own available node IP address table and updates it regularly.

[0062] The written content is: (assuming that node 3 is not running normally)

[0063] SNMP collector 1IP;

[0064] SNMP collector 2IP;

[0065] SNMP collector 4IP;

[0066] …

[0067] The prometheus collection task delivery process is carried out by N prometheus nodes in the high-availability prometheus architecture to collect monitoring data from M switches. Each prometheus node delivers M switch collection tasks to the load balancing component. One collection task represents one switch collection. The collection task is an http request, which contains the address of the switch to be collected this time. The request is as follows:

[0068] http: / / load balancing component ip:port?target=switch IP;

[0069] As a preferred implementation, the steps of determining the target SNMP collector node corresponding to the collection task according to the collection task obtained by each SNMP collector node and the currently available SNMP collector nodes recorded by the SNMP collector node include:

[0070] Read the first list in each SNMP collector node respectively, the first list records the IP address of the SNMP collector node whose current status is to execute the collection task;

[0071] Perform consistent hashing based on the first list to generate a Hash ring, each node of the Hash ring corresponds to an IP address of an SNMP collector node that is currently in a state of being able to execute a collection task;

[0072] The IP address of the distributed switch corresponding to the collection task is mapped to the hash ring through consistent hashing to determine the target SNMP collector node corresponding to the collection task.

[0073] As a preferred implementation, the steps of acquiring monitoring data through the SNMP collector node and transmitting it back to the prometheus node corresponding to the collection task include:

[0074] If the SNMP collector node is in idle state, it obtains monitoring data based on the SNMP protocol and transmits it back to the prometheus node corresponding to the collection task through the cache;

[0075] If the SNMP collector node is not in an idle state, it will block and wait until the SNMP collector node is in an idle state, obtain monitoring data based on the SNMP protocol, and transmit it back to the prometheus node corresponding to the collection task through the cache.

[0076] For details, please refer to the SNMP collector selection process. Figure 3 When the load balancing component receives the switch collection tasks from N prometheus nodes, it uses the distributed task scheduling algorithm to select the appropriate SNMP collector to perform this collection task for each task. The specific process of the algorithm is as follows:

[0077] 1. Load balancing. The load balancing component randomly distributes the collection tasks to all collector nodes in the SNMP collector cluster to avoid a single node performing multiple collection tasks and improve collection efficiency.

[0078] 2. Select the node to execute the task. SNMP collector A receives the collection task after load balancing and selects the node to execute the task:

[0079] 1) Read all available IP addresses in the current available node IP table of SNMP collector A, perform consistent hashing and generate a hash ring. Each node on the ring represents a node in the current cluster that is running normally.

[0080] 2) Read the IP address of the switch whose information needs to be collected in this collection task, and use consistent hashing to map it on the hash ring in 1);

[0081] 3) Search clockwise from the hash value of the switch IP address, and the IP address corresponding to the first hash value encountered is the IP address of SNMP collector B that performs this collection task;

[0082] 3. Task forwarding. SNMP collector A sends the collection task to SNMP collector B selected in step 2, and collector B executes the collection task.

[0083] The SNMP collector execution process can be found in the attached Figure 4 After the above SNMP collector selection process, SNMP collector B is selected to perform this collection task. The process of SNMP collector node collecting switch data is as follows:

[0084] 1. Query its own cache. After receiving the collection task, the SNMP collector node first queries its own cache. If the switch data that needs to be collected currently already exists in its own cache, it reads the cache data and provides it to Prometheus through the collect interface provided by Prometheus.

[0085] 2. Query the node execution status. If the switch data that needs to be collected does not exist in its own cache, further determine the node execution task status. When the node is in an idle state, it obtains the switch data from the SNMP protocol. After the collection is completed, the data is stored in the cache and provided to Prometheus. When the node is executing a collection task, the collection task is blocked and waited until the node completes the ongoing collection task. The task is released from the blocking wait and executes step 1.

[0086] Through this method, multiple prometheus nodes can simultaneously obtain data from multiple switches. This method uses SNMP collectors and distributed task scheduling algorithms to control the switch collection frequency, ensuring that the switch side is not affected by the increase in traffic load in enterprise-level monitoring, and improving the efficiency and quality of switch collection.

[0087] As a specific example, there are the following Prometheus servers P1, P2, P3, P4; SNMP collectors S1, S2, S3, S4, S5 and load balancing server D1.

[0088] 1. SNMP collectors S1, S2, S3, S4, and S5 form an SNMP collector cluster. S1 requests the health interface from other SNMP collectors in the cluster every minute, and writes the address of the SNMP collector that returns "ok" into its own available node IP address table. Other collectors access nodes in the cluster except themselves in the same way and update the available node IP address table.

[0089] 2. Prometheus servers P1, P2, P3, and P4 send 1,000 collection tasks to the load balancing server every 5 minutes. Each collection task carries a different switch IP address, with the goal of collecting data from 1,000 different switches.

[0090] 3. The load balancing server D1 receives 4000 tasks sent by 4 Prometheus servers and randomly and evenly distributes the tasks to each SNMP collector node in the SNMP collector cluster.

[0091] 4. After load balancing, the collection task T1 is assigned to the SNMP collector node S1. S1 reads its own currently available node IP address table and the switch IP address carried by the collection task T1, and builds a hash ring through the consistent hash algorithm.

[0092] 5. Search clockwise from the hash value after calculation of the switch IP address carried by collection task T1. The IP address corresponding to the first hash value encountered is the IP address of the SNMP collector S2 that executes collection task T1.

[0093] 6. SNMP collector S1 forwards collection task T1 to SNMP collector S2, and S2 executes collection task T1.

[0094] 7. After receiving the collection task, SNMP collector S2 queries its own cache. If the cache already contains the switch data that needs to be collected, it reads the cache data and provides it to Prometheus for use; if the cache does not contain the switch data, it further determines the collector status.

[0095] If the SNMP collector S2 is in an idle state, it collects switch data through the SNMP protocol, stores the data in the cache after the collection is completed, and provides it to Prometheus; if the collector S2 is executing the collection task T2, the collection task T1 is blocked and waits until T2 is completed, and then the process described in 7 is repeated.

[0096] The switch data collection method based on high-availability prometheus provided in this embodiment is based on the improved application of the SNMP collector. Compared with the original snmp-exporter version, it adds data caching and synchronization task functions, controls the switch collection task frequency and provides switch data to multiple prometheus nodes at the same time; by designing a distributed task scheduling algorithm, it is realized without the need for third-party task allocation and scheduling components, and the SNMP collector node that executes the collection task is selected by its own cluster in a decentralized and adaptive manner, which is more practical.

[0097] Example 2

[0098] Corresponding to the switch data collection method based on high-availability Prometheus introduced above, this embodiment also provides a switch data collection system based on high-availability Prometheus.

[0099] Specifically, Figure 5 As shown, the switch data collection system based on high-availability prometheus of this embodiment is implemented based on the SNMP collector cluster in the prometheus architecture, and is used to collect monitoring data of distributed switches; the SNMP collector cluster includes at least two SNMP collector nodes, and the data collection system includes:

[0100] The load balancing module 1 is used to determine the SNMP collector node corresponding to the collection task in the SNMP collector cluster according to the collection task information sent by the prometheus node;

[0101] The collection execution module 2 is used to detect whether the SNMP collector node corresponding to the collection task has monitoring data corresponding to the collection task; if so, the monitoring data is transmitted back to the prometheus node corresponding to the collection task; if not, the monitoring data is obtained through the SNMP collector node and transmitted back to the prometheus node corresponding to the collection task. As a preferred embodiment, the load balancing module 1 includes a task acquisition unit, which is used to respectively obtain the collection task information sent by at least two prometheus nodes; a task allocation unit, which is used to randomly allocate all the collection task information to each SNMP collector node in the SNMP collector cluster; a node determination unit, which is used to determine the target SNMP collector node corresponding to the collection task according to the collection tasks obtained by each SNMP collector node and the currently available SNMP collector nodes recorded by the SNMP collector node; a task sending unit, which is used to send the collection task to the target SNMP collector node.

[0102] As a preferred implementation, the node determination unit includes:

[0103] The list reading subunit is used to read the first list in each SNMP collector node respectively, where the first list records the IP addresses of the SNMP collector nodes that are currently in a state of being able to execute the collection task;

[0104] A hash table generation subunit, configured to perform a consistent hash operation based on the first list to generate a hash ring, wherein each node of the hash ring corresponds to an IP address of an SNMP collector node that is currently in a state of being able to execute a collection task;

[0105] The target node determination subunit is used to map the IP address of the distributed switch corresponding to the collection task to the hash ring through a consistent hash operation to determine the target SNMP collector node corresponding to the collection task.

[0106] As a preferred implementation, the collection execution module 2 is specifically used for: if the SNMP collector node is in an idle state, obtaining monitoring data based on the SNMP protocol, and transmitting it back to the prometheus node corresponding to the collection task through the cache; if the SNMP collector node is not in an idle state, blocking and waiting until the SNMP collector node is in an idle state, obtaining monitoring data based on the SNMP protocol, and transmitting it back to the prometheus node corresponding to the collection task through the cache.

[0107] The switch data collection system based on high-availability prometheus provided in this embodiment is based on the improved application of the SNMP collector. Compared with the original snmp-exporter version, it adds data caching and synchronization task functions, controls the switch collection task frequency and provides switch data to multiple prometheus nodes at the same time; by designing a distributed task scheduling algorithm, it is realized without the need for third-party task allocation and scheduling components, and the SNMP collector node that executes the collection task is selected by its own cluster in a decentralized and adaptive manner, which is more practical.

[0108] Example 3

[0109] Figure 6 The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the switch data collection method based on high-availability Prometheus in the above embodiment is implemented. Figure 6 The electronic device 30 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0110] like Figure 6As shown, the electronic device 30 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).

[0111] The bus 33 includes a data bus, an address bus, and a control bus.

[0112] The memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (ROM) 323 .

[0113] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0114] The processor 31 executes various functional applications and data processing by running the computer program stored in the memory 32, such as the switch data collection method based on high-availability Prometheus as described above in the present invention.

[0115] The electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 35. Furthermore, the model generating device 30 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 36. Figure 6 As shown, the network adapter 36 communicates with other modules of the model-generated device 30 via the bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0116] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0117] Example 4

[0118] This embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps in the switch data collection method based on high-availability prometheus in the above embodiment are implemented. The readable storage medium can be more specifically, but not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0119] In a possible implementation, the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps in the switch data collection method based on high-availability prometheus as described above. Among them, the program code for executing the present invention can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device, partially on the remote device, or completely on the remote device.

[0120] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A switch data collection method based on high-availability prometheus, characterized in that: Based on the SNMP collector cluster in the prometheus architecture, it is used to collect monitoring data of distributed switches; the SNMP collector cluster includes at least two SNMP collector nodes, and the data collection method includes the following steps: According to the collection task information sent by the prometheus node, determining the SNMP collector node corresponding to the collection task in the SNMP collector cluster; Detect whether the SNMP collector node corresponding to the collection task has the monitoring data corresponding to the collection task; If yes, the monitoring data is transmitted back to the prometheus node corresponding to the collection task; if no, the monitoring data is obtained through the SNMP collector node and transmitted back to the prometheus node corresponding to the collection task; The step of determining the SNMP collector node corresponding to the collection task in the SNMP collector cluster according to the collection task information sent by the prometheus node includes: Get the collection task information sent by at least two prometheus nodes respectively; The load balancing component randomly distributes all the collection task information to each SNMP collector node in the SNMP collector cluster; The SNMP collector node that receives the collection task determines the target SNMP collector node corresponding to the collection task according to the acquired collection task and the currently available SNMP collector nodes recorded by the SNMP collector node; Sending the collection task to the target SNMP collector node; The step of the SNMP collector node receiving the collection task determining the target SNMP collector node corresponding to the collection task according to the acquired collection task and the currently available SNMP collector nodes recorded by the SNMP collector node comprises: Reading the first list in each SNMP collector node respectively, wherein the first list records the IP addresses of the SNMP collector nodes that are currently in a state of being able to execute collection tasks; Performing a consistent hash operation based on the first list to generate a Hash ring, each node of the Hash ring corresponds to the IP address of the SNMP collector node whose current state is to execute the collection task; The IP address of the distributed switch corresponding to the collection task is mapped to the Hash ring through a consistent hash operation to determine the target SNMP collector node corresponding to the collection task.

2. The switch data collection method based on high-availability Prometheus according to claim 1, characterized in that: The step of acquiring the monitoring data through the SNMP collector node and transmitting the monitoring data back to the prometheus node corresponding to the collection task includes: If the SNMP collector node is in an idle state, the monitoring data is obtained based on the SNMP protocol, and is transmitted back to the prometheus node corresponding to the collection task through the cache; If the SNMP collector node is not in an idle state, blocking and waiting is performed until the SNMP collector node is in an idle state, the monitoring data is obtained based on the SNMP protocol, and the data is transmitted back to the prometheus node corresponding to the collection task through the cache.

3. A switch data collection system based on high-availability Prometheus, characterized in that: Based on the SNMP collector cluster in the prometheus architecture, it is used to collect monitoring data of distributed switches; the SNMP collector cluster includes at least two SNMP collector nodes, and the data collection system includes: A load balancing module, used to determine the SNMP collector node corresponding to the collection task in the SNMP collector cluster according to the collection task information sent by the prometheus node; A collection execution module is used to detect whether the SNMP collector node corresponding to the collection task has the monitoring data corresponding to the collection task; if so, the monitoring data is transmitted back to the prometheus node corresponding to the collection task; if not, the monitoring data is obtained through the SNMP collector node and transmitted back to the prometheus node corresponding to the collection task; The load balancing module includes: A task acquisition unit is used to respectively acquire the collection task information sent by at least two prometheus nodes; A task allocation unit, used for a load balancing component to randomly allocate all the collection task information to each SNMP collector node in the SNMP collector cluster; A node determination unit, configured to determine, by the SNMP collector node that receives the collection task, a target SNMP collector node corresponding to the collection task according to the acquired collection task and the currently available SNMP collector nodes recorded by the SNMP collector node; A task sending unit, configured to send the SNMP collector node that receives the collection task to the target SNMP collector node; The node determination unit comprises: The list reading subunit is used to read the first list in each SNMP collector node respectively, wherein the first list records the IP addresses of the SNMP collector nodes whose current status is that they can execute the collection task; A hash table generation subunit, configured to perform a consistent hash operation based on the first list to generate a hash ring, wherein each node of the hash ring corresponds to an IP address of the SNMP collector node whose current state is to be able to execute the collection task; The target node determination subunit is used to map the IP address of the distributed switch corresponding to the collection task to the hash ring through a consistent hash operation to determine the target SNMP collector node corresponding to the collection task.

4. The switch data collection system based on high-availability Prometheus according to claim 3, characterized in that: The collection execution module is specifically used for: if the SNMP collector node is in an idle state, obtaining the monitoring data based on the SNMP protocol, and transmitting it back to the prometheus node corresponding to the collection task through the cache; if the SNMP collector node is not in an idle state, blocking and waiting until the SNMP collector node is in an idle state, obtaining the monitoring data based on the SNMP protocol, and transmitting it back to the prometheus node corresponding to the collection task through the cache.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the switch data collection method based on high-availability Prometheus according to any one of claims 1 to 2 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the switch data collection method based on high-availability Prometheus is implemented as described in any one of claims 1-2.

Images