Monitoring data acquisition method and device and monitoring system

Abstract

The invention discloses a monitoring data acquisition method and device and a monitoring system.The acquisition method is executed by a monitoring server and comprises the steps that a detection task is pulled concurrently according to idle detection threads in a detection thread pool, the detection task indicates a to-be-detected target cluster, and the target cluster is a cluster of a monitored target component; executing the detection task according to the detection thread pulled to the detection task, and obtaining host information of the target cluster; and connecting to a corresponding host according to the host information, executing a detection script corresponding to the target component on the corresponding host, and obtaining a detection result containing a plurality of monitoring index items as monitoring data of the target component. By implementing the method, different detection scripts can be selected according to the types of different clusters, the dynamic information of the key indexes of the clusters can be accurately collected in real time, and operation and maintenance personnel can conveniently make an optimal decision according to the dynamic information.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a monitoring data collection method, device and monitoring system. Background Art

[0002] Monitoring is a tool and process for measuring and managing technical systems. With the expansion of the business scale of technical systems and the increasing requirements for system stability and high availability, cluster deployment has become the mainstream. The corresponding monitoring system also needs to keep pace with the times. It is not limited to a single machine, but also needs to understand the system environment in the cluster dimension, diagnose cluster faults, formulate cluster capacity plans, etc., and provide the cluster with performance information, cost information, and status information of the technical system.

[0003] At present, the relevant technology is to monitor a single host through the zabbix monitoring system. Specifically, the zabbix monitoring system is an enterprise-level open source solution based on the WEB interface that provides distributed system monitoring and network monitoring functions. When the service has been deployed through a cluster, a single machine failure may not affect the service, resulting in the existing monitoring system being unable to detect whether the cluster has failed. Summary of the invention

[0004] In view of this, the embodiments of the present invention provide a monitoring data collection method, device and monitoring system to solve the problem that the existing monitoring system cannot detect whether a cluster fails.

[0005] According to the first aspect, an embodiment of the present invention provides a method for collecting monitoring data, which is executed by a monitoring server, including: according to an idle detection thread in a detection thread pool, concurrently pulling detection tasks, the detection task indicates a target cluster to be detected, and the target cluster is a cluster of a monitored target component; executing the detection task according to the detection thread pulled to the detection task, and obtaining host information of the target cluster; connecting to a corresponding host according to the host information, executing a detection script corresponding to the target component on the corresponding host, and obtaining a detection result containing several monitoring index items as monitoring data of the target component.

[0006] Optionally, the total number of detection threads in the detection thread pool is determined according to the total number of clusters of monitored components and physical resources of the monitoring server.

[0007] Optionally, the detection task is generated in the distributed scheduling system through the following process: triggering the monitoring task of each monitored component according to the idle scheduling thread in the scheduling thread pool; obtaining the cluster information of the target component according to the target scheduling thread that triggers the monitoring task of the target component; generating a corresponding number of detection tasks according to the number of clusters indicated by the cluster information.

[0008] Optionally, the collection method also includes: saving the detection results to a preset time series database; in response to a monitoring data display request, selecting matching detection results from the preset time series database for display, and / or, issuing an alarm when the detection results trigger an alarm rule.

[0009] Optionally, the monitoring indicator items include general indicator items and difference indicator items corresponding to the target components.

[0010] Optionally, the general monitoring items include at least one of the following: node usage status information, shard usage status information, cluster usage status information, memory usage, CPU usage, and disk usage.

[0011] Optionally, the target component is a remote dictionary service Redis component, and the difference index item includes a cache hit rate.

[0012] Optionally, the target component is a message queue, and the difference indicator item includes the number of delayed messages and / or throughput.

[0013] According to a second aspect, an embodiment of the present invention provides a monitoring data collection device, which is applied to a monitoring server and is used to implement the monitoring data collection method described in any implementation of the first aspect.

[0014] According to the third aspect, an embodiment of the present invention provides a monitoring data collection device, including: a detection task pulling module, used to concurrently pull detection tasks according to idle detection threads in a detection thread pool, wherein the detection task indicates a target cluster to be detected, and the target cluster is a cluster of a monitored target component; a host information acquisition module, used to execute the detection task according to the detection thread pulled to the detection task, and obtain the host information of the target cluster; a monitoring data acquisition module, used to connect to a corresponding host according to the host information, execute a detection script corresponding to the target component on the corresponding host, and obtain a detection result containing several monitoring index items as the monitoring data of the target component.

[0015] According to the fourth aspect, an embodiment of the present invention provides a monitoring system, including a monitoring server and a distributed scheduling system; the monitoring server is deployed with the monitoring data collection device described in the second aspect or the third aspect; the distributed scheduling system is used to trigger the monitoring tasks of each monitored component according to the idle scheduling threads in the scheduling thread pool; obtain the cluster information of the target component according to the target scheduling thread that triggers the monitoring task of the target component; and generate a corresponding number of detection tasks according to the number of clusters indicated by the cluster information.

[0016] According to the fifth aspect, an embodiment of the present invention provides a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the one processor, and the instructions are executed by the at least one processor so that the at least one processor performs the steps of the monitoring data collection method described in the first aspect or any one embodiment of the first aspect.

[0017] According to the sixth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the monitoring data collection method described in the first aspect or any one of the embodiments of the first aspect are implemented.

[0018] The technical solution of the present invention has the following advantages:

[0019] The present invention provides a monitoring data collection method, device and monitoring system, wherein the collection method comprises: according to the idle detection threads in the detection thread pool, concurrently pulling detection tasks, the detection tasks indicate the target cluster to be detected, and the target cluster is the cluster of the monitored target component; according to the detection thread that pulls the detection task, the detection task is executed to obtain the host information of the target cluster; according to the host information, the corresponding host is connected to, and the detection script corresponding to the target component is executed on the corresponding host to obtain the detection result containing several monitoring index items as the monitoring data of the target component.

[0020] By implementing the present invention, the problem in the related art that the existing monitoring system cannot detect whether a cluster failure occurs is solved. Combined with the created detection thread pool, multiple detection tasks are pulled concurrently, and then the detection tasks are executed to generate detection results of monitoring indicator items, that is, the monitoring data of the cluster is collected. Different detection scripts can be selected according to the types of different clusters, and the dynamic information of the key indicators of the cluster can be collected in real time and accurately, which is convenient for operation and maintenance personnel to make the best decision based on the dynamic information. Before a failure or problem occurs, the system can be repaired and managed in advance according to the changes in the monitoring data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A flowchart of a specific example of a method for collecting monitoring data in an embodiment of the present invention;

[0023] Figure 2 is a principle block diagram of a specific example of a monitoring data collection device in an embodiment of the present invention;

[0024] Figure 3 FIG. 4 is a specific example diagram of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0026] Monitoring is one of the most important links in the use of technical systems. The existing monitoring process is mostly to collect system indicators of a single machine through zabbix. Zabbix is ​​an enterprise-level open source solution based on the WEB interface that provides distributed system monitoring and network monitoring functions. At present, the monitoring of host CPU, memory, network, file system and other indicators is relatively stable if the zabbix system is used to implement it.

[0027] However, zabbix only provides monitoring of single-machine system indicators. For services deployed in cluster mode, a single machine failure does not necessarily affect the service. If the cluster fails without being noticed, then even if monitoring is performed, it is necessary to reconsider whether the content being monitored is reasonable. In other words, there may be a situation where a failure has occurred but has not been detected.

[0028] Based on the above background, in order to solve the problem in the related technology that the existing monitoring system cannot detect whether a cluster fails, the embodiments of the present invention provide a monitoring data collection method, device and monitoring system, with the purpose of providing a cluster monitoring data collection method based on a distributed scheduling system, realizing real-time and timely collection of monitoring data, and realizing effective, timely and accurate monitoring of file systems and clusters.

[0029] The embodiment of the present invention provides a method for collecting monitoring data. In this embodiment, the collection method can be executed by a monitoring server, such as Figure 1 As shown, the method includes:

[0030] Step S11: according to the idle detection threads in the detection thread pool, concurrently pull detection tasks, wherein the detection tasks indicate the target cluster to be detected, and the target cluster is the cluster of the monitored target component;

[0031] In this embodiment, the volume information of the target cluster may be information used to characterize the number of nodes and the number of shards in the target cluster, and the server capacity information may be the number of CPUs in the server where the acquisition method described in the embodiment is executed, that is, the speed at which the CPU executes the computing task. The thread pool may include multiple threads, and a thread may be a carrier or container for executing a task. Specifically, based on the volume information of the target cluster that needs to be detected and the computing power of the CPU of the server that executes the detection task, a detection thread pool is created, that is, the number of tasks that the server can pull concurrently is determined, that is, the number of tasks executed simultaneously, that is, the number of threads in the detection thread pool.

[0032] In this embodiment, the detection task can be a task that has been triggered and needs to be executed. The thread pool can include multiple threads. One thread can pull one task to execute, and multiple threads can pull multiple tasks to execute concurrently. Specifically, the pulled detection task is used to indicate a target cluster with detection, which refers to a cluster of the monitored target components.

[0033] Step S12: Execute the detection task according to the detection thread of the pulled detection task to obtain the host information of the target cluster; in this embodiment, the target cluster may include multiple nodes, one node may be represented by a host, and the host information may be information storing multiple features such as target cluster topology information, node information, shard information, and type information. Specifically, the detection task may be executed by the detection thread of the detection task to obtain the host information corresponding to the target cluster.

[0034] Step S13: Connect to the corresponding host according to the host information, execute the detection script corresponding to the target component on the corresponding host, and obtain the detection result containing several monitoring index items as the monitoring data of the target component; in this embodiment, the detection script can be the script type information of the specific execution of the detection logic, and specifically, the detection script type can be determined by the detection logic of the components composed of various clusters; the monitoring index item can be the indicator type information of the specific monitoring target cluster, for example, it can include CPU usage, etc. Specifically, when executing the detection task, the type information of the detection script that specifically executes the detection logic and the monitoring index item information that needs to be collected can be determined according to the acquired host information.

[0035] In this embodiment, the detection script is executed, and the detection result generated by the detection script is obtained as the monitoring data of the target component; in this embodiment, the detection result can be the real-time usage of each monitoring indicator item in the target cluster, for example, at the current moment, the memory usage rate in the target cluster is A%, the disk usage rate is B%, the CPU usage rate is C%, and so on.

[0036] A monitoring data collection method provided by the present invention includes: according to an idle detection thread in a detection thread pool, concurrently pulling detection tasks, the detection task indicates a target cluster to be detected, and the target cluster is a cluster of a monitored target component; executing the detection task according to the detection thread pulled to the detection task, and obtaining host information of the target cluster; connecting to a corresponding host according to the host information, executing a detection script corresponding to the target component on the corresponding host, and obtaining a detection result containing several monitoring index items as monitoring data of the target component.

[0037] By implementing the present invention, the problem in the related art that the existing monitoring system cannot detect whether a cluster failure occurs is solved. Combined with the created detection thread pool, multiple detection tasks are pulled concurrently, and then the detection tasks are executed to generate detection results of monitoring indicator items, that is, the monitoring data of the cluster is collected. Different detection scripts can be selected according to the types of different clusters, and the dynamic information of the key indicators of the cluster can be collected in real time and accurately, which is convenient for operation and maintenance personnel to make the best decision based on the dynamic information. Before a failure or problem occurs, the system can be repaired and managed in advance according to the changes in the monitoring data.

[0038] As an optional implementation manner of the present invention, the total number of detection threads in the detection thread pool mentioned in the above embodiment may be determined according to the total number of clusters of each monitored component and the physical resources of the monitoring server.

[0039] As an optional implementation manner of the present invention, the detection task described in any of the above embodiments is generated in a distributed scheduling system, and the main process of generating the detection task includes:

[0040] Step 1: Trigger the monitoring task of each monitored component according to the idle scheduling thread in the scheduling thread pool;

[0041] Step 2: According to the target scheduling thread that triggers the monitoring task of the target component, the cluster information of the target component is obtained;

[0042] Step 3: Generate a corresponding number of detection tasks according to the number of clusters indicated by the cluster information.

[0043] In this embodiment, the number of detection tasks generated corresponds to the number of clusters corresponding to the target components, thereby optimizing the allocation of available resources and monitoring the corresponding components.

[0044] As an optional implementation of the present invention, in the process of generating the detection task, the detection task can also be configured and the triggering frequency of the detection task can be determined according to the target demand; in this embodiment, the target demand can be a user demand determined according to the actual application scenario, and the detection task and the triggering rule of the detection task are configured in the target distributed scheduling system according to the target demand. Specifically, the detection task can only be pulled by the thread pool after it is triggered in the target distributed scheduling system. Therefore, the triggering frequency of the detection task needs to be pre-configured, for example, the triggering frequency of the detection task can be set to 60 seconds / time.

[0045] Then, according to the trigger frequency, the trigger time of the detection task is determined; in this embodiment, according to the pre-configured trigger frequency, the interval time at which the detection task is triggered can be determined. For example, when the trigger frequency is 60 seconds / time, it can be determined that the detection task will be triggered once every minute, that is, the detection task can be determined to be triggered every minute.

[0046] When the trigger time is reached, the detection task is triggered and the detection task can be generated. In this embodiment, when the time interval between the last detection task being triggered and the current time is one minute, the trigger time has been reached, and the detection task is triggered, and a task record is generated, that is, the detection task is generated.

[0047] A monitoring data collection method provided by an embodiment of the present invention combines the high-frequency trigger frequency and arrival trigger time of the equipment in the target distributed scheduling system, and the detection task is automatically triggered, which can realize the high-frequency execution of the detection task, that is, the real-time status information of the cluster monitoring indicators can be obtained in time.

[0048] As an optional implementation of the present invention, in the above step S13, the detection script and monitoring index items corresponding to the target component can be determined by the following process:

[0049] First, the type of the target cluster is determined according to the host information. In this embodiment, the type of the target cluster can be determined according to the topology information of the target cluster in the host information based on the acquired host information. Specifically, the type of the target cluster can include Redis cluster, Zookeeper cluster, Schedule cluster and other clusters.

[0050] Secondly, according to the type of the target cluster, determine the detection script and monitoring index items. In this embodiment, according to the type of the target cluster, determine the detection logic of the components of the cluster, and determine the type of the detection script according to the detection logic; and according to the type of the target cluster, determine the monitoring index items required by the target cluster, including general index items and difference index items, wherein the difference index items are specifically determined according to the types of different clusters.

[0051] Specifically, the detection logic of the component may be, for example, that the Redis cluster performs detection by command line instructions cluster nodes (configuration information of the cluster to which the currently connected node belongs), and the Zookeeper cluster performs detection by connecting to all nodes of the cluster.

[0052] A monitoring data collection method provided by an embodiment of the present invention combines different types of clusters and detection logic of different clusters to determine difference indicator items and detection scripts adapted to the cluster, so as to obtain indicator information related to the cluster operation status, and then make an accurate judgment on the cluster operation status, which is conducive to operation and maintenance personnel to make management instructions and decisions that are most beneficial to the cluster.

[0053] As an optional implementation of the present invention, the collection method also includes: saving the detection results to a preset time series database; in response to a monitoring data display request, selecting matching detection results from the preset time series database for display, and / or issuing an alarm when the detection results trigger an alarm rule. In this embodiment, the detection results returned from executing the detection task are saved to a preset time series database, which can be used for the display of subsequent monitoring indicator data and for docking with the alarm module. Specifically, when a monitoring data display request initiated by a user is obtained, in response to the monitoring data display request, the detection results indicated by the monitoring data display request are selected from the preset time series database for display;

[0054] Alternatively, when it is determined through the data records in the preset time series library that the data of a certain monitoring indicator of a certain cluster has exceeded the reasonable threshold range, that is, when the detection result triggers the alarm rule, an alarm is issued, and the process of issuing an alarm can be to generate an alarm message and connect it to the alarm module to promptly remind the operation and maintenance personnel to maintain the cluster. Specifically, a certain indicator in a certain cluster can be the memory usage rate, and the corresponding reasonable threshold range of the memory usage rate can be 30%-60%. When at the current moment, the memory usage rate of the current cluster returned by the detection task is 59%, it can be determined that the memory usage rate of the cluster is about to exceed the reasonable threshold range. At this time, it is necessary to generate a primary alarm message to notify the operation and maintenance personnel that they need to monitor this indicator and this cluster to prevent the occurrence of failures.

[0055] At the current moment, when the memory usage of the current cluster returned by the detection task is 69%, it can be determined that the memory usage of the cluster has exceeded the reasonable threshold range. At this time, an emergency alarm message needs to be generated to notify the operation and maintenance personnel to immediately monitor this indicator and this cluster, to repair the cluster, and to ensure the normal operation of the cluster.

[0056] It should be noted that the above two implementation modes, namely, the display in response to the monitoring data display request, and the monitoring detection results triggering the alarm rules to issue an alarm, do not affect each other, and there is no order restriction. The corresponding operations can be implemented according to the actual situation, and can also be implemented in parallel based on the actual situation. The present invention is not limited to this.

[0057] As an optional implementation of the present invention, the monitoring indicator items include general indicator items and difference indicator items corresponding to the target components. In this embodiment, the general monitoring indicator items may include one or more of the following indicators: availability, traffic, latency, success rate, saturation and other indicators. Among them, availability can be used to characterize the available status of each node in the cluster, for example, it can be normal or faulty; it can also be used to characterize the available status of each shard in the cluster, for example, it can be normal or faulty; it can also be used to characterize the available status of the cluster as a whole, for example, it can be normal or faulty. Specifically, the general monitoring indicator items may also include at least one of node usage status information, shard usage status information, cluster usage status information, memory usage, CPU usage and disk usage.

[0058] The difference indicators vary depending on the target component. Specifically, for the remote dictionary service Redis component, the difference indicators may be cache hit rate, expiration time ratio, etc.; for the RocketMQ message queue component, the difference indicators may be the number of delayed messages and / or throughput, etc.

[0059] The embodiment of the present invention provides a method for collecting monitoring data, which can implement the detection logic of different cluster types, and can determine the correct monitoring indicators for different clusters. It can also optimize the monitoring indicator items and clarify the detection means of the cluster status through problems encountered in daily operation and maintenance, so as to achieve the purpose of accurately detecting the monitoring targets.

[0060] The embodiment of the present invention provides a monitoring data collection device, such as Figure 2 As shown, including:

[0061] The detection task pulling module 21 is used to concurrently pull detection tasks based on the idle detection threads in the detection thread pool. The detection task indicates the target cluster to be detected, and the target cluster is the cluster of the monitored target components. For detailed implementation content, please refer to the relevant description of step S11 in the above method embodiment.

[0062] The host information acquisition module 22 is used to execute the detection task according to the detection thread that pulls the detection task, and obtain the host information of the target cluster; the detailed implementation content can refer to the relevant description of step S12 in the above method embodiment.

[0063] The monitoring data acquisition module 23 is used to connect to the corresponding host according to the host information, execute the detection script corresponding to the target component on the corresponding host, and obtain the detection result containing several monitoring index items as the monitoring data of the target component. For detailed implementation content, please refer to the relevant description of step S13 in the above method embodiment.

[0064] A monitoring data collection device provided by the present invention includes: a detection task pulling module 21, which is used to concurrently pull detection tasks according to idle detection threads in a detection thread pool, wherein the detection task indicates a target cluster to be detected, and the target cluster is a cluster of a monitored target component; a host information acquisition module 22, which is used to execute the detection task according to the detection thread that pulls the detection task, and obtain the host information of the target cluster; a monitoring data acquisition module 23, which is used to connect to a corresponding host according to the host information, execute a detection script corresponding to the target component on the corresponding host, and obtain a detection result containing several monitoring index items as the monitoring data of the target component.

[0065] By implementing the present invention, the problem in the related art that the existing monitoring system cannot detect whether a cluster failure occurs is solved. Combined with the created detection thread pool, multiple detection tasks are pulled concurrently, and then the detection tasks are executed to generate detection results of monitoring indicator items, that is, the monitoring data of the cluster is collected. Different detection scripts can be selected according to the types of different clusters, and the dynamic information of the key indicators of the cluster can be collected in real time and accurately, which is convenient for operation and maintenance personnel to make the best decision based on the dynamic information. Before a failure or problem occurs, the system can be repaired and managed in advance according to the changes in the monitoring data.

[0066] As an optional implementation of the present invention, the monitoring data acquisition module 23 may determine the detection script and monitoring index items corresponding to the target component by executing the following process:

[0067] First, the type of the target cluster is determined according to the host information. In this embodiment, the type of the target cluster can be determined according to the topology information of the target cluster in the host information based on the acquired host information. Specifically, the type of the target cluster can include Redis cluster, Zookeeper cluster, Schedule cluster and other clusters.

[0068] Secondly, according to the type of the target cluster, determine the detection script and monitoring index items. In this embodiment, according to the type of the target cluster, determine the detection logic of the components of the cluster, and determine the type of the detection script according to the detection logic; and according to the type of the target cluster, determine the monitoring index items required by the target cluster, including general index items and difference index items, wherein the difference index items are specifically determined according to the types of different clusters.

[0069] Specifically, the detection logic of the component may be, for example, that the Redis cluster performs detection by command line instructions cluster nodes (configuration information of the cluster to which the currently connected node belongs), and the Zookeeper cluster performs detection by connecting to all nodes of the cluster.

[0070] As an optional implementation of the present invention, the monitoring data collection device can be applied to a monitoring server and is specifically used to execute the monitoring data collection method described in any of the above embodiments.

[0071] The embodiment of the present invention further provides a monitoring system, which includes: a monitoring server and a distributed scheduling system; the monitoring server is deployed with a monitoring data collection device as described in any of the above embodiments;

[0072] Among them, the distributed scheduling system is used to trigger the monitoring tasks of each monitored component according to the idle scheduling threads in the scheduling thread pool; obtain the cluster information of the target component according to the target scheduling thread that triggers the monitoring task of the target component; and generate a corresponding number of detection tasks according to the number of clusters indicated by the cluster information.

[0073] The embodiment of the present invention also provides a computer device, such as Figure 3 As shown, the computer device may include a processor 31 and a memory 32, wherein the processor 31 and the memory 32 may be connected via a bus 30 or in other ways. Figure 3 The connection via bus 30 is taken as an example.

[0074] The processor 31 may be a central processing unit (CPU). The processor 31 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0075] The memory 32 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the monitoring data collection method in the embodiment of the present invention. The processor 31 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 32, that is, the monitoring data collection method in the above method embodiment is implemented.

[0076] The memory 32 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created by the processor 31, etc. In addition, the memory 32 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 32 may optionally include a memory remotely arranged relative to the processor 31, and these remote memories may be connected to the processor 31 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0077] The one or more modules are stored in the memory 32 and when executed by the processor 31, perform the following steps: Figure 1 The monitoring data collection method in the illustrated embodiment.

[0078] For details of the above computer equipment, please refer to Figure 1 The corresponding related descriptions and effects in the illustrated embodiments can be understood and will not be repeated here.

[0079] An embodiment of the present invention further provides a non-transitory computer-readable medium, which stores computer instructions. The computer instructions are used to enable a computer to execute a monitoring data collection method as described in any one of the above embodiments, wherein the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (SSD), etc.; the storage medium may also include a combination of the above-mentioned types of memories.

[0080] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A method for collecting monitoring data, characterized in that: The method is executed by a monitoring server and includes: According to the idle detection threads in the detection thread pool, concurrently pull detection tasks, wherein the detection tasks indicate the target cluster to be detected, and the target cluster is the cluster of the monitored target component; Execute the detection task according to the detection thread that pulls the detection task, and obtain the host information of the target cluster; A corresponding host is connected to according to the host information, a detection script corresponding to the target component is executed on the corresponding host, and a detection result including several monitoring index items is obtained as monitoring data of the target component.

2. The method according to claim 1, characterized in that The total number of detection threads in the detection thread pool is determined according to the total number of clusters of each component being monitored and the physical resources of the monitoring server.

3. The method according to claim 1, characterized in that The detection task is generated in the distributed scheduling system through the following process: Trigger the monitoring tasks of each monitored component according to the idle scheduling threads in the scheduling thread pool; Acquire cluster information of the target component according to the target scheduling thread that triggers the monitoring task of the target component; A corresponding number of detection tasks are generated according to the number of clusters indicated by the cluster information.

4. The method according to claim 1, characterized in that: Also includes: Saving the detection results to a preset time series database; In response to a monitoring data display request, a matching detection result is selected from the preset time series database for display, and / or an alarm is issued when the detection result triggers an alarm rule.

5. The method according to claim 1, characterized in that: The monitoring index items include general index items and difference index items corresponding to the target components.

6. The method according to claim 5, characterized in that The general monitoring items include at least one of the following: node usage status information, shard usage status information, cluster usage status information, memory usage rate, CPU usage rate and disk usage rate.

7. The method according to claim 5, characterized in that The target component is a remote dictionary service Redis component, and the difference index item includes a cache hit rate.

8. The method according to claim 5, characterized in that The target component is a message queue, and the difference indicator item includes the number of delayed messages and / or throughput.

9. A monitoring data collection device, characterized in that: The method is applied to a monitoring server and is used to implement the monitoring data collection method as claimed in any one of claims 1 to 8.

10. A monitoring system, characterized in that: It includes a monitoring server and a distributed scheduling system; the monitoring server is deployed with the monitoring data collection device according to claim 9; The distributed scheduling system is used to trigger the monitoring tasks of each monitored component according to the idle scheduling threads in the scheduling thread pool; obtain the cluster information of the target component according to the target scheduling thread that triggers the monitoring task of the target component; and generate a corresponding number of detection tasks according to the number of clusters indicated by the cluster information.

Images