Monitoring parameter acquisition method and device

Abstract

The invention discloses a monitoring parameter acquisition method and device, relates to the technical field of data processing, and mainly aims at achieving the real-time performance and comprehensiveness of monitoring parameter acquisition. According to the main technical scheme, the method comprises the steps that according to a first preset period, a general storage parameter server is used for obtaining full monitoring parameters of at least one monitoring object through a pull mode and / or a push mode; wherein the full monitoring parameters refer to all monitoring parameters related to a monitoring object; according to a second preset period, using the real-time storage parameter server to obtain core monitoring parameters of the at least one monitoring object through a pull mode and / or a push mode; wherein the core monitoring parameters are monitoring parameters conforming to a preset rule; the second preset period is smaller than the first preset period and is a second-level period.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a monitoring parameter acquisition method and device. Background Art

[0002] With the development of information technology, in addition to digitizing business operation data, operation and maintenance and infrastructure teams are also paying more and more attention to the monitoring of application systems, servers, and middleware. Common monitoring indicator collection tools collect indicator information scattered across different servers, business systems, and middleware to the backend for storage, query through query statements, and then perform visual display, analysis, and alarms.

[0003] At present, the commonly used monitoring indicator collection tools are mainly divided into two modes: push and pull. The pull mode means that the monitoring indicator collection tool actively sends a data acquisition request to the target object, and the target object returns the corresponding indicator after receiving the request. The push mode means that the target object actively pushes the indicator data to the monitoring indicator collection tool, and the monitoring indicator collection tool then stores and processes it.

[0004] However, the push mode supports fewer target objects, and although the collection speed is fast, the collected target objects are not complete. Although the pull mode pulls more target objects, the indicators of some target objects are too large, resulting in a longer collection time and slower collection speed. Summary of the invention

[0005] In view of the above problems, the present invention provides a monitoring parameter collection method and device, the main purpose of which is to achieve both real-time and comprehensive collection of monitoring parameters.

[0006] In order to solve the above technical problems, the present invention proposes the following solutions:

[0007] In a first aspect, the present invention provides a monitoring parameter acquisition method, the method comprising:

[0008] According to a first preset period, using a general storage parameter server to obtain full monitoring parameters of at least one monitoring object through a pull mode and / or a push mode; wherein the full monitoring parameters refer to all monitoring parameters related to the monitoring object;

[0009] According to a second preset period, the core monitoring parameters of at least one monitored object are obtained by using a real-time storage parameter server through a pull mode and / or a push mode; wherein the core monitoring parameters are monitoring parameters that comply with preset rules; and the second preset period is less than the first preset period and is a period of seconds.

[0010] In a second aspect, the present invention provides a monitoring parameter acquisition device, comprising:

[0011] A first acquisition unit, according to a first preset period, uses a general storage parameter server to obtain the full monitoring parameters of at least one monitoring object through a pull mode and / or a push mode; wherein the full monitoring parameters refer to all monitoring parameters related to the monitoring object;

[0012] The second acquisition unit is used to obtain the core monitoring parameters of at least one monitored object through a pull mode and / or a push mode using a real-time storage parameter server according to a second preset period; wherein the core monitoring parameters are monitoring parameters that comply with preset rules; and the second preset period is less than the first preset period and is a period of seconds.

[0013] In order to achieve the above-mentioned object, according to a third aspect of the present invention, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the method described in the first aspect.

[0014] In order to achieve the above-mentioned purpose, according to the fourth aspect of the present invention, there is provided an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, all or part of the steps of the method described in the first aspect are implemented.

[0015] In order to achieve the above-mentioned object, according to a fifth aspect of the present invention, there is provided a computer program product, comprising a computer program / instructions, which implement the steps of the method described in the first aspect when executed by a processor.

[0016] By means of the above technical scheme, the monitoring parameter collection method and device provided by the present invention are due to the fact that there are currently fewer target objects that support the push mode. Although the collection speed is fast, the collected target objects are not complete. Although the pull mode pulls more complete target objects, the monitoring parameters of some target objects are too large, resulting in a longer collection time and a slower collection speed. To this end, the present invention uses a general storage parameter server to obtain the full monitoring parameters of at least one monitored object through a pull mode and / or a push mode according to a first preset period; wherein the full monitoring parameters refer to all monitoring parameters related to the monitored object; the full monitoring parameters can be collected; according to a second preset period, the real-time storage parameter server is used to obtain the core monitoring parameters of at least one monitored object through a pull mode and / or a push mode, which can realize the collection of core monitoring parameters; wherein the core monitoring parameters are monitoring parameters that comply with preset rules, and the number of core monitoring parameters can be controlled to increase the push speed; the second preset period is less than the first preset period, and is a second-level period. The present invention can realize the use of different periods, and adopt a combination of pull mode and push mode to realize the collection of full monitoring parameters and core monitoring parameters. In this way, the real-time and comprehensiveness of the collected monitoring parameters can be taken into account.

[0017] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0019] Figure 1 A flow chart of a monitoring parameter collection method provided by an embodiment of the present invention is shown;

[0020] Figure 2 A flow chart of a second monitoring parameter collection method provided by an embodiment of the present invention is shown;

[0021] Figure 3 A technical architecture diagram provided by an embodiment of the present invention is shown;

[0022] exist Figure 3 middle,

[0023] Minor Node Exporter is a custom micro-server parameter collector used to collect the core monitoring parameters of the server;

[0024] Node Exporter is a general server parameter collector, used to collect all monitoring parameters of the server;

[0025] Push Task is a core monitoring parameter push task, which is used to push core monitoring parameters to the corresponding Pushgateway;

[0026] Pull Exporter is a general microservice parameter collector, which is used to collect all monitoring parameters of microservices.

[0027] Pushgateway is a monitoring parameter push gateway used to store monitoring parameters;

[0028] RealTime Prometheus Server is a real-time storage parameter server used to collect and store the core monitoring parameters of all monitored objects;

[0029] Common Prometheus Server is a general storage parameter server used to collect and store all monitoring parameters of all monitored objects;

[0030] Push metrics are monitoring parameters pushed to Pushgateway.

[0031] Short pull is a task for real-time storage parameter server to pull core monitoring parameters.

[0032] Long pull is a task for pulling all monitoring parameters from the general storage parameter server.

[0033] Figure 4 A flow chart of a third monitoring parameter collection method provided by an embodiment of the present invention is shown;

[0034] Figure 5 A block diagram showing a monitoring parameter acquisition device provided by an embodiment of the present invention is shown;

[0035] Figure 6 A block diagram showing the composition of another monitoring parameter acquisition device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0036] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0037] Terminology explanation:

[0038] Pull mode: The data collection system will actively initiate a request to obtain the indicator data of the target object (such as a server, application, or other monitoring target). The data collection system implements the indicator collection of the target object by defining a job, that is, the data collection system will regularly poll the exposed interface of the target object, pull the latest indicator data, and store it for subsequent query and analysis.

[0039] Push mode: The data source (i.e., the target object, such as a server, application, or other monitoring target) actively pushes the indicator data to the data collection system. In the push mode, the target object sends the indicator data to an intermediate component, usually called Pushgateway, periodically or when a specific event occurs. Pushgateway acts as an intermediary service that receives data from multiple target objects and allows the data collection system to pull data from Pushgateway periodically.

[0040] Currently, there are relatively few target objects that support the push mode. Although the acquisition speed is fast, the target objects collected are incomplete. Although the pull mode pulls more target objects, the monitoring parameters of some target objects are too large, resulting in a long acquisition time and a slow acquisition speed. To address this problem, the inventors thought of first obtaining the full monitoring parameters and core monitoring parameters, and then pushing the full monitoring parameters and core monitoring parameters to the corresponding parameter collector based on different acquisition cycles.

[0041] To this end, an embodiment of the present invention provides a monitoring parameter collection method, through which the real-time and comprehensive collection of monitoring parameters are achieved. The specific execution steps are as follows: Figure 1 As shown, the execution subject is an electronic device, and the electronic device executes the following steps, which at least include steps 100-200.

[0042] 100. According to a first preset period, use a general storage parameter server to obtain full monitoring parameters of at least one monitored object through a pull mode and / or a push mode.

[0043] Among them, the full monitoring parameters refer to all monitoring parameters related to the monitoring object, for example: the monitoring object can be a microservice, server, and middleware, etc., and the monitoring parameters can be process-level monitoring parameters (this type of monitoring parameters focuses on the status and performance of the process running on the monitoring object, such as CPU utilization, memory usage, number of threads, etc.), business monitoring parameters (this type of monitoring parameters focuses on the business-level performance of the services provided by the monitoring object, such as transaction success rate, transaction response time, error rate, etc.), API interface level monitoring parameters (this type of monitoring parameters focuses on the performance of the API interface provided by the monitoring object, such as the number of API requests, response time, failure rate, etc.), etc., which are not specifically limited in this embodiment.

[0044] This step uses different methods for obtaining full monitoring parameters according to different monitoring objects, which are not specifically limited in this embodiment. For example, the server uses Node Exporter to collect full monitoring parameters related to the server, and sets Pull Exporter in the microservice to collect full monitoring parameters.

[0045] Since the monitoring parameters collected in this step are full monitoring parameters, the first preset period must be set to ensure that the general storage parameter server can pull all full monitoring parameters. For example, the first preset period can be 15s or 30s, which is not specifically limited in this embodiment.

[0046] After obtaining all monitoring parameters of at least one monitoring object, based on the first preset period, the universal parameter collector pulls all monitoring parameters in a pull mode and / or a push mode. This step can achieve comprehensiveness of collected monitoring parameters.

[0047] It should be noted that the specific selection of pull mode or push mode, or both, depends on the type of monitoring object. Different monitoring object types use different monitoring parameter data collection methods. If Pushgateway is involved, push mode is used. If the monitoring parameter data can be directly pulled, pull mode is selected. If both situations exist, then both pull mode and push mode are selected.

[0048] 200. According to a second preset period, using a real-time storage parameter server to obtain core monitoring parameters of at least one monitored object through a pull mode and / or a push mode.

[0049] The core monitoring parameters are monitoring parameters that comply with preset rules, which may be pre-set monitoring parameter types, such as monitoring parameters related to modules such as processes, memory, and hard disks, which are not specifically limited in this embodiment.

[0050] This step adopts different ways of obtaining core monitoring parameters according to different monitoring objects, which are not specifically limited in this embodiment. For example: the server uses Minor Node Exporter to obtain monitoring parameters that meet preset rules from the full amount of monitoring parameters as core monitoring parameters; and sets Push Task in the microservice to collect core monitoring parameters.

[0051] The second preset period is smaller than the first preset period and is a period in seconds.

[0052] Since the monitoring parameters collected in this step are core monitoring parameters, and the core monitoring parameters need to be collected quickly, the speed needs to be in seconds. Therefore, when setting the second preset period, it is necessary to ensure that all core monitoring parameters can be quickly transmitted to the real-time storage parameter server. For example, the second preset period can be 1s, which is not specifically limited in this embodiment.

[0053] After the step of obtaining the core monitoring parameters of at least one monitoring object, based on the second preset period, the real-time storage parameter server obtains the core monitoring parameters of at least one monitoring object through a pull mode and / or a push mode. This step can achieve the real-time nature of collecting the core monitoring parameters.

[0054] It should be noted that the specific selection of pull mode or push mode, or both, depends on the type of monitoring object. Different monitoring object types use different monitoring parameter data collection methods. If Pushgateway is involved, push mode is used. If the monitoring parameter data can be directly pulled, pull mode is selected. If both situations exist, then both pull mode and push mode are selected.

[0055] A specific monitoring parameter is selected as an example to illustrate how to collect it according to the proposed monitoring parameter collection method.

[0056] Example Parameters

[0057] The CPU usage is selected as an example. CPU usage is a common monitoring parameter that reflects the usage of server CPU resources and is very important for evaluating system performance and stability.

[0058] Example of monitoring parameter collection method

[0059] 1. Full monitoring parameter collection

[0060] According to the first preset cycle, use the general storage parameter server to obtain the full amount of monitoring parameters. Monitoring object: server

[0061] Acquisition method: Use Node Exporter to collect all monitoring parameters related to the server.

[0062] The first preset period: for example, 15 seconds or 30 seconds.

[0063] Collection content: All monitoring parameters including CPU usage.

[0064] Mode: Pull all monitoring parameters from the server in pull mode.

[0065] 2. Core monitoring parameter collection

[0066] According to the second preset period, the core monitoring parameters are obtained using the real-time storage parameter server

[0067] Monitoring object: Server

[0068] Acquisition method: Use Minor Node Exporter to filter out core monitoring parameters, such as CPU usage, from all monitoring parameters.

[0069] The second preset period: for example, 1 second.

[0070] Collection content: core monitoring parameters, such as collecting only CPU usage.

[0071] Mode: Pull core monitoring parameters from the server in pull mode.

[0072] Example

[0073] Suppose you are monitoring the performance of a server and want to obtain its comprehensive performance data (full monitoring parameters) and key performance parameters (core monitoring parameters), especially CPU usage.

[0074] Full monitoring parameters: All monitoring parameters of the server are collected through Node Exporter every 15 seconds or 30 seconds, including but not limited to CPU usage, memory usage, disk I / O, etc.

[0075] Core monitoring parameters: Every second, the Minor Node Exporter filters out key parameters from all monitoring parameters, such as CPU usage, and pulls them from the real-time storage parameter server.

[0076] In this way, you can obtain comprehensive performance data (for example, updated every 15 seconds or 30 seconds), and you can also track changes in key parameters in real time (for example, CPU usage is updated every 1 second).

[0077] Regarding steps 100 and 200, it should be noted that monitoring parameter templates are stored in the monitoring system, and these templates include the type setting / change of the monitored object, the list of monitoring parameters required for different types of monitored objects, the type of acquisition server, and the list of server addresses. This means that through the monitoring parameter template, the type of a certain monitored object, the parameters required to be monitored for the monitored object, the type of acquisition server corresponding to the monitored object and the monitoring parameters, and the list of server addresses of the corresponding type can be preset or changed at any time, that is, the monitoring requirements of various types of monitored objects can be preset using the monitoring parameter template.

[0078] In some embodiments, the parameter control parameter template also includes a parameter collection mode of the collection server, so as to configure the parameter collection mode of the collection server, such as a pull mode and a push mode. In one example, the parameter collection mode of each type of collection server can be a preset recommendation, for example, it can be recommended according to a recommendation rule.

[0079] The recommendation rule may be based on the historical usage ratio of parameter collection modes of various types of collection servers. For example, for a general storage parameter server, if the number of times the push mode is used is greater than a first threshold, the push mode is recommended. For another example, for a real-time storage parameter server, if the number of times the pull mode is used is greater than a second threshold, the pull mode is recommended.

[0080] The recommendation rule can be based on the current collection requirements. For example, since the push mode is more suitable for fewer monitoring parameters, when the collected monitoring parameters are core monitoring parameters or a small number of partial monitoring parameters, the push mode is recommended. For another example, since the pull mode is more suitable for more monitoring parameters, when the collected monitoring parameters are all monitoring parameters or most monitoring parameters, the pull mode is recommended.

[0081] The recommendation rule may also be based on the resource occupancy of the acquisition server. The embodiment of the present application may obtain the resource occupancy of the acquisition server. For example, since the push mode is more suitable for monitoring fewer parameters, when the resource occupancy rate of the acquisition server is higher than the third threshold, the push mode is recommended. For another example, since the pull mode is more suitable for monitoring more parameters, when the resource occupancy rate of the acquisition server is lower than the third threshold, the pull mode is recommended.

[0082] The recommendation rule can also be based on the resource occupancy of the monitored object. The embodiment of the present application can obtain the resource occupancy of the monitored object. For example, since the push mode is more suitable for monitoring fewer parameters, when the resource occupancy rate of the monitored object is higher than the fourth threshold, the push mode is recommended. For another example, since the pull mode is more suitable for monitoring more parameters, when the resource occupancy rate of the monitored object is lower than the fourth threshold, the pull mode is recommended.

[0083] The recommendation rule can also be recommended based on the resource occupancy of the acquisition server and the resource occupancy of the monitored object. The embodiment of the present application can obtain the resource occupancy of the acquisition server and the resource occupancy of the monitored object. For example, since the push mode is more suitable for monitoring fewer parameters, when the resource occupancy rate of the acquisition server is higher than the third threshold and the resource occupancy rate of the monitored object is higher than the fourth threshold, the push mode is recommended. For another example, since the pull mode is more suitable for monitoring more parameters, when the resource occupancy rate of the acquisition server is lower than the third threshold and the resource occupancy rate of the monitored object is lower than the fourth threshold, the pull mode is recommended. For another example, when the resource occupancy rate of the acquisition server is higher than the third threshold and the resource occupancy rate of the monitored object is lower than the fourth threshold, or when the resource occupancy rate of the acquisition server is lower than the third threshold and the resource occupancy rate of the monitored object is higher than the fourth threshold, it can be recommended in combination with the current acquisition needs. When the collected monitoring parameters are core monitoring parameters or a small number of partial monitoring parameters, the push mode is recommended. When the resource occupancy rate of the acquisition server is lower than the third threshold, the pull mode is recommended. For another example, when the resource occupancy rate of the collection server is higher than the third threshold and the resource occupancy rate of the monitored object is lower than the fourth threshold, or the resource occupancy rate of the collection server is lower than the third threshold and the resource occupancy rate of the monitored object is higher than the fourth threshold, the recommendation can be made based on the historical usage ratio of the parameter collection modes of each type of collection server, that is, if the proportion of the number of times the general storage parameter server adopts the push mode is greater than the first threshold, then the push mode is recommended, or, if the proportion of the number of times the real-time storage parameter server adopts the pull mode is greater than the second threshold, then the pull mode is recommended.

[0084] In one embodiment, the types of monitoring objects include servers, microservices, and middleware, the parameters to be monitored (i.e., monitoring parameters) include full monitoring parameters and core monitoring parameters, and the collection server types include general storage parameter servers and real-time storage parameter servers. In order to clearly show the correspondence between the monitoring object type and the server type, the following lists three types of monitoring objects and their corresponding server types, as well as how to collect full monitoring parameters and core monitoring parameters. When the type of monitoring object is selected as a server, for the full monitoring parameters: use Node Exporter to collect full monitoring parameters related to the server, such as CPU usage, memory usage, disk I / O, etc. These parameters are obtained by the general storage parameter server through pull mode. For core monitoring parameters: use Minor Node Exporter to filter out core monitoring parameters, such as CPU usage, from the full monitoring parameters. These parameters are obtained by the real-time storage parameter server through pull mode.

[0085] When the monitoring object type is selected as microservice, for full monitoring parameters: use a general microservice parameter collector (such as Pull Exporter) to obtain full monitoring parameters related to microservices, including but not limited to process-level parameters, business parameters, and API interface-level parameters. These parameters are obtained by the general storage parameter server in pull mode. For core monitoring parameters: collect core monitoring parameters by setting Push Task and push them to the parameter push gateway (such as Pushgateway), and then obtain them by the real-time storage parameter server in pull mode.

[0086] When the type of monitoring object is selected as middleware, for full monitoring parameters: if the middleware has a parameter pull interface, the full monitoring parameters of the middleware are provided through the interface, and are obtained by the general storage parameter server in pull mode. If the middleware does not have a parameter pull interface, the full monitoring parameters of the middleware are pulled through the proxy tool, pushed to the parameter push gateway, and then obtained by the general storage parameter server in pull mode. For core monitoring parameters: if the middleware has a parameter pull interface, the core monitoring parameters are filtered from it, and obtained by the general storage parameter server in pull mode. If the middleware does not have a parameter pull interface, the core monitoring parameters are pulled through the proxy tool, pushed to the parameter push gateway, and then obtained by the real-time storage parameter server in pull mode.

[0087] That is to say, when a monitoring object needs to collect both full monitoring parameters and core monitoring parameters, the corresponding servers can be selected to process these two different types of monitoring parameters. Specifically, the full monitoring parameters are collected by the general storage parameter server, which pulls or pushes data in a longer period (such as a few minutes); while the core monitoring parameters are collected by the real-time storage parameter server, which pulls or pushes data in a shorter period (such as a few seconds). This method can ensure that the real-time and comprehensiveness of the monitoring system are improved without adding too much complexity.

[0088] Assume that you need to monitor the performance of a server and want to obtain its comprehensive performance data (full monitoring parameters) and key performance parameters (core monitoring parameters), especially CPU usage. Then collect all monitoring parameters of the server through Node Exporter every 15 or 30 seconds, including but not limited to CPU usage, memory usage, disk I / O, etc. Every 1 second, filter out key parameters from the full monitoring parameters through Minor Node Exporter, such as CPU usage, and pull them through the real-time storage parameter server. In this way, comprehensive data coverage and real-time updates of key indicators can be ensured to meet the needs of different monitoring objects.

[0089] When a monitoring object needs to be monitored, the monitoring system first identifies the type of the monitoring object. Then, based on the type, the required monitoring parameter set and the corresponding server address list are extracted from the monitoring parameter template, and one or more servers are selected as the main server and the backup server according to the preset rules. Among them, the main server is used to send the acquired parameters to the general storage parameter server or the real-time storage parameter server, and the backup server sends the monitoring parameters to the general storage parameter server or the real-time storage parameter server according to the configured rules when the main server is inactive or fails. Finally, a monitoring parameter configuration file for the corresponding monitoring object is generated, and the configuration file is sent to the selected server, so that the server can automatically obtain the relevant monitoring parameters according to the information in the configuration file. The configuration file contains information such as the monitoring parameter list, the collection method, and the collection cycle. Specifically, the configuration file specifies which monitoring parameters should be collected by the general storage parameter server and which monitoring parameters should be collected by the real-time storage parameter server, for example: the full monitoring parameters are collected by the general storage parameter server, and the core monitoring parameters are collected by the real-time storage parameter server. The configuration file also presets the collection cycle of different monitoring parameters, such as the collection cycle of the full monitoring parameters is longer (such as a few minutes), and the collection cycle of the core monitoring parameters is shorter (such as seconds). In addition, the configuration file presets the collection mode of different monitoring parameters, namely pull mode or push mode.

[0090] The rules for selecting servers include, but are not limited to, load balancing, random selection, or priority-based selection. Specifically:

[0091] Load balancing: The monitoring system can monitor the load of each server in real time and select the server with the lowest current load as the primary server.

[0092] Random selection: A server is selected as the primary server from the server address list through a random algorithm.

[0093] Priority selection: The server address list contains the priority information of each server, and the monitoring system selects the main server based on the priority.

[0094] In addition, the status of the primary server and the backup server is synchronized through the heartbeat packet mechanism. When the primary server fails to send the heartbeat packet on time, the backup server takes over the role of the primary server. That is, when the backup server does not receive the heartbeat packet from the primary server for a period of time, it immediately takes over the role of the primary server and starts sending monitoring parameters to the general storage parameter server or the real-time storage parameter server.

[0095] Based on the above Figure 1 It can be seen from the implementation of the embodiment that the present invention provides a monitoring parameter collection method, which aims to solve the inherent contradiction between real-time and comprehensiveness in traditional monitoring technology by adopting different memories to pull different types of monitoring parameters. Traditional monitoring technology can either only monitor a few key indicators in real time, or can comprehensively collect all indicators but cannot achieve high-frequency data updates. The present invention realizes effective management and optimization of monitoring parameters by setting up two servers - a general storage parameter server and a real-time storage parameter server - to process the full amount of monitoring parameters and the core monitoring parameters respectively.

[0096] Specifically, the general storage parameter server in the present invention obtains all monitoring parameters, i.e., full monitoring parameters, from the monitored object by pulling or pushing according to a pre-set first cycle. This process ensures that all relevant data is recorded by the system. At the same time, the real-time storage parameter server follows a shorter second cycle set at the second level, focusing on frequently pulling or receiving core monitoring parameters that are critical to the health of the system. These core monitoring parameters are a part of the important parameters selected according to preset rules, and the quantity is subject to certain control to ensure the speed of data push.

[0097] Therefore, the design of the present invention overcomes the conflict between real-time and comprehensiveness in traditional monitoring solutions, so that the monitoring system can not only quickly capture important changes in the system, but also comprehensively record all details of its operation. By flexibly adjusting the data collection cycle, the present invention can achieve the collection of core monitoring parameters in seconds or even shorter cycles while ensuring the overall performance of the system, ensuring that no full monitoring information is missed. This mechanism greatly enhances the value of monitoring data, enhances the flexibility and responsiveness of the system, and ultimately effectively improves the overall efficiency of the monitoring system.

[0098] When the monitoring objects are servers, microservices, and middleware, in order to complete the comprehensive collection of all monitoring parameters, in some embodiments of the present application, such as Figure 2 As shown, the aforementioned 100 includes 110-150.

[0099] 110. When the monitoring object is a microservice, the general microservice parameter collector is used to obtain the full monitoring parameters related to the microservice.

[0100] When the monitoring object is a microservice, the general microservice parameter collector should be used to obtain the full amount of monitoring parameters related to the microservice. Figure 3 As shown in the figure, the Pull Exporter provided for microservices supports a variety of full monitoring parameters such as process-level parameters, business parameters, and API interface-level parameters to ensure the coverage of monitoring parameter collection.

[0101] Specifically, when the monitoring object is a microservice, a general microservice parameter collector such as PullExporter is used to obtain the full monitoring parameters related to the microservice. Due to the characteristics of the microservice architecture, a single service may run in different processes or even on different machines, so a mechanism is needed to uniformly collect the monitoring information of these scattered services. Pull Exporter is such a tool, which can support the collection of process-level parameters, business parameters, and API interface-level parameters, etc., to ensure comprehensive coverage of monitoring data.

[0102] In the present invention, the use of Pull Exporter to solve the background technical problems is mainly reflected in the following aspects:

[0103] Comprehensiveness: Pull Exporter can collect various parameters of microservices, including process status, business performance, and API performance, making the monitoring data more complete and helping to fully understand the health status of microservices.

[0104] Real-time and flexibility: By configuring a reasonable pull cycle, you can obtain the status changes of microservices in a timely manner, which is crucial for discovering potential problems. In addition, Pull Exporter can flexibly adjust the collection frequency as needed to adapt to different monitoring needs.

[0105] Compatibility and ease of use: Pull Exporter is designed to be easily integrated into the existing microservice architecture without requiring extensive modifications to the service itself. At the same time, it supports multiple parameter types, improving compatibility with different types of microservices.

[0106] Through the above measures, the present invention can effectively improve the real-time and comprehensiveness of the monitoring system without increasing excessive complexity, and solves the shortcomings of traditional monitoring methods in terms of real-time and comprehensiveness.

[0107] 120. When the monitored object is a server, a general server parameter collector is used to obtain all monitoring parameters related to the server.

[0108] When the monitoring object is a server, a general server parameter collector should be used to obtain all monitoring parameters related to the server. Figure 3 As shown in the figure, server monitoring uses Node Exporter to collect all monitoring parameters. Considering that there are many collected parameters, involving network, process, CPU, memory, hard disk and other aspects, the collection period is usually set to 10 seconds, 15 seconds or 30 seconds to ensure the effective collection of monitoring parameters.

[0109] Specifically, when the monitoring object is a server, a general server parameter collector such as Node Exporter is used to obtain all monitoring parameters related to the server. This is based on the following reasons:

[0110] Comprehensiveness: Node Exporter can collect data from multiple modules on the server, including but not limited to network, process, CPU, memory, hard disk, etc. This means that you can get comprehensive information about the running status of the server, which is crucial for diagnosing problems and preventing failures.

[0111] Standardization and ease of use: Node Exporter is part of the Prometheus monitoring system, which provides a standardized way to collect various monitoring indicators on operating systems such as Linux and Windows. This allows consistent monitoring strategies to be maintained even for different server environments, reducing the difficulty of implementing monitoring.

[0112] Flexibility of collection cycle: The collection cycle can be flexibly set according to the server load and business needs. For example, for key performance indicators, a shorter collection cycle (such as 10 seconds) can be set, while for some non-critical indicators, a longer time interval (such as 30 seconds) can be selected to balance the real-time nature of monitoring data and collection overhead.

[0113] Through the above measures, Node Exporter can provide real-time monitoring data updates without affecting server performance. This is very important for timely detection and response of problems. Traditional monitoring methods may ignore certain specific monitoring indicators or fail to cover all important performance parameters. Using Node Exporter can ensure that monitoring parameters are collected more comprehensively and reduce blind spots in monitoring. By continuously monitoring various server indicators, server resources can be managed more effectively to avoid resource waste or service interruptions caused by insufficient resources.

[0114] In summary, using Node Exporter as the collector for server monitoring can not only meet the comprehensiveness requirements of monitoring, but also optimize resource usage through flexible collection cycle settings and reduce fault response time through real-time monitoring, thereby effectively solving the real-time and comprehensiveness problems in traditional monitoring methods.

[0115] 130. When the monitored object is a middleware and the middleware has a parameter pulling interface, the full monitoring parameters of the middleware are provided through the parameter pulling interface.

[0116] When the monitoring object is a middleware, and the middleware has a parameter pull interface, you can directly use the interface to obtain the full monitoring parameters of the middleware. This method is suitable for middleware that have set up and exposed parameter pull interfaces, because they can directly provide the required monitoring information through the interface, simplifying the data collection process.

[0117] like Figure 3 As shown in the figure, some middleware directly sets the parameter pull interface and exposes the interface to the outside; at the same time, some middleware does not directly set the parameter pull interface. The processing methods for these two types of middleware are different. In particular, for those middleware that directly set the parameter pull interface and expose the interface to the outside, you can: pull the full monitoring parameters from the parameter pull interface directly set and exposed by the middleware through CommonPrometheus Server. The full monitoring parameters usually include table-level and user-level monitoring data, and the collection period can be set to 15 seconds or 30 seconds.

[0118] The reasons for directly setting the parameter pull interface are:

[0119] Directness and convenience: For those middlewares that have set up parameter pull interfaces and exposed them to the outside, you can directly obtain all monitoring parameters through this interface. This method reduces the intermediate links and makes the acquisition of monitoring parameters more direct and efficient.

[0120] Comprehensiveness: Through the parameter pull interface, you can obtain a variety of monitoring parameters including table level and user level from the middleware, ensuring the comprehensiveness of the monitoring data, which is crucial for a comprehensive assessment of the working status of the middleware.

[0121] Standardization and ease of use: The exposed parameter pulling interfaces are usually designed according to standard protocols, which means that these interfaces can be easily integrated into existing monitoring systems, reducing the difficulty of implementing monitoring.

[0122] In other words, compared to traditional monitoring methods that may only obtain some monitoring parameters but not all key performance indicators, using the exposed parameter pull interface can ensure full coverage of monitoring data and reduce blind spots in monitoring. By using Common Prometheus Server to regularly pull full monitoring parameters from the parameter pull interface of the middleware, you can flexibly set the collection cycle (such as 15 seconds or 30 seconds) according to business needs to ensure the real-time nature of monitoring data, which is very important for timely discovery and resolution of problems. Continuous monitoring of various indicators of the middleware helps to better manage middleware resources and prevent resource waste or service interruptions due to insufficient resources.

[0123] Through the above measures, the present invention can effectively improve the real-time and comprehensiveness of the monitoring system without increasing too much complexity, and solve the shortcomings of traditional monitoring methods in terms of real-time and comprehensiveness. Especially for those middlewares that have exposed parameter pull interfaces, this method can realize the comprehensive collection of their full monitoring parameters.

[0124] 140. When the monitoring object is a middleware and the middleware does not have a parameter pulling interface, the full amount of monitoring parameters of the middleware are pulled through the proxy tool and pushed to the corresponding parameter push gateway.

[0125] The parameter push gateway is used to store all monitoring parameters of the middleware.

[0126] like Figure 3As shown, some middleware directly sets the parameter pull interface and exposes the interface to the outside; some middleware does not directly set the parameter pull interface. For those middleware that do not directly set the parameter pull interface, special processing is required. Specifically, this type of middleware needs to use proxy tools (for example: Jmx Exporter, which can convert parameters in JMX format to parameters in Prometheus format) to pull the full amount of monitoring parameters and push these parameters to the Pushgateway used in conjunction with Common Prometheus Server. Common PrometheusServer can then pull the stored full amount of monitoring parameters from Pushgateway.

[0127] Reasons to use a proxy tool:

[0128] Compatibility and conversion: Middleware that does not directly set the parameter pull interface usually does not have the ability to directly expose monitoring parameters. In this case, proxy tools such as Jmx Exporter can act as a bridge to convert the monitoring parameters inside the middleware into Prometheus format so that these parameters can be recognized and used by the monitoring system.

[0129] Centralized management: By pushing monitoring parameters to Pushgateway, centralized management of monitoring data can be achieved. This not only facilitates the maintenance of monitoring data, but also simplifies the overall architecture of the monitoring system.

[0130] Improved flexibility: The existence of proxy tools allows the monitoring system to adapt to more types of middleware, increasing the flexibility of the system. Even if new middleware needs to be monitored in the future, as long as a suitable proxy tool can be found, it can be easily integrated into the existing system.

[0131] It can be seen that by pulling all monitoring parameters through the proxy tool and pushing them to Pushgateway, the comprehensiveness of the monitoring data can be ensured. Even if the middleware itself does not provide a direct parameter pulling interface, it is possible to achieve comprehensive collection of all monitoring parameters related to the middleware in this way. Although the collection cycle of full monitoring parameters is long, by pushing these parameters to Pushgateway, it can ensure that Common Prometheus Server can obtain the latest monitoring data in a timely manner when needed, thereby improving the real-time performance of the monitoring system. The combination of proxy tools and Pushgateway eliminates the need for the monitoring system to directly deal with each different middleware, thereby reducing the complexity of the monitoring system.

[0132] Through the above measures, the present invention can effectively improve the real-time and comprehensiveness of the monitoring system without increasing too much complexity, and solve the shortcomings of traditional monitoring methods in terms of real-time and comprehensiveness. Especially for those middlewares that do not directly expose parameter pull interfaces, this method can realize the comprehensive collection of their full monitoring parameters.

[0133] 150. According to a first preset period, use a general storage parameter server to pull all monitoring parameters.

[0134] From steps 110-140, the full monitoring parameters of the microservices, servers, and middleware can be obtained, and then according to the first preset period, the full monitoring parameters are pulled using a general storage parameter server.

[0135] like Figure 3 As shown in the figure, Common Prometheus Server pulls the full monitoring parameters collected by Pull Exporter, the full monitoring parameters collected by Node Exporter, the full monitoring parameters provided by the parameter pulling interface of the middleware, and the full monitoring parameters stored by the Pushgateway corresponding to the middleware through long pull (for example, the collection period is 15s or 30s), thereby realizing the comprehensive collection of the full monitoring parameters of multiple monitoring objects and ensuring the integrity of the collection of the full monitoring parameters.

[0136] It should be noted that: the transmission scheme composed of steps 110-130 and step 150 is a pull mode; the transmission scheme composed of step 140 and step 150 is a push mode.

[0137] The technical solution composed of the above steps 110-150 uses special collection tools (such as Pull Exporter, Node Exporter, Jmx Exporter, etc.) and interfaces to achieve comprehensive coverage of monitoring data for the different characteristics of microservices, servers and middleware. This solution can not only quickly capture important changes in the system, but also comprehensively record all details of its operation. By reasonably setting the collection cycle, combining the pull and push modes, and using standardized monitoring formats (such as Prometheus format), the comprehensiveness and ease of use of monitoring data are ensured. At the same time, through centralized management and flexible adjustment of the collection frequency, the complexity of the monitoring system is reduced, effectively solving the shortcomings of traditional monitoring methods in terms of comprehensiveness. Although the above solution focuses on the comprehensiveness of monitoring data, reasonable collection cycle settings can also help improve the real-time nature of monitoring data.

[0138] Based on the above Figure 2It can be seen from the implementation of the embodiment that the present invention provides a monitoring parameter collection method. This embodiment adopts different full monitoring parameter collection devices set in different monitoring objects, and then combines the pull mode and the push mode to complete the comprehensive collection of full monitoring parameters. When the monitoring objects are servers and microservices, in order to collect core monitoring parameters in real time, in some embodiments of the present application, such as Figure 4 As shown, the aforementioned 200 includes 210-230.

[0139] It should be noted that most middleware cannot filter out some monitoring parameters according to certain rules before or during the collection of monitoring parameters. This is because the original design of middleware is to provide specific application services rather than specialized monitoring. As a result, they usually lack built-in complex monitoring parameter filtering mechanisms. On the one hand, the main purpose of middleware is to provide services (such as message queues, database connection pools, etc.) rather than monitoring, so built-in complex monitoring filtering logic does not meet its main responsibilities. On the other hand, the introduction of complex monitoring parameter filtering mechanisms may increase additional computing overhead and affect the performance of the middleware itself. In addition, developing and maintaining a unified monitoring parameter filtering mechanism for different middleware requires additional development resources and maintenance costs, and the interfaces and protocols of different middleware vary greatly, making it difficult to standardize. Finally, middleware tends to expose as much monitoring data as possible to meet the monitoring needs of different users, which in turn increases the comprehensiveness and flexibility of monitoring data. Therefore, most middleware does not provide monitoring parameter filtering functions by default, but relies on external monitoring systems to filter and process data.

[0140] Since most middleware cannot filter its monitoring parameters, this means that all generated monitoring parameters will be collected. If the middleware cannot filter the monitoring parameters, then all generated monitoring parameter data will be collected, which may result in a very large amount of data, thereby affecting the efficiency and real-time performance of data processing. Especially when second-level monitoring is required, a large amount of monitoring parameter data may cause delays in data transmission and processing, thereby affecting the performance of the system. Therefore, in order to ensure the real-time performance and efficiency of the monitoring system, the following embodiments of the present invention mainly analyze servers and microservices as monitoring objects, that is, the function of second-level monitoring is performed.

[0141] However, in order to enable the middleware to have the function of second-level monitoring and solve the data processing efficiency and real-time problems caused by a large number of monitoring parameters, the present invention provides the following comprehensive solutions:

[0142] Intelligent filtering and aggregation: Use the intelligent filtering function of the monitoring system (such as Prometheus's PromQL) to perform preliminary filtering and aggregation of data at the collection end, retaining only key monitoring parameters and reducing unnecessary data transmission.

[0143] Hierarchical monitoring strategy: Design a hierarchical monitoring strategy, first collect key basic monitoring parameters (such as CPU usage, memory usage, etc.), and then gradually add necessary advanced monitoring parameters according to actual needs. This can ensure real-time performance and control data volume.

[0144] Dynamically adjust the collection frequency: Dynamically adjust the collection frequency according to the actual load of the middleware. Increase the collection frequency during business peaks and reduce the collection frequency during business troughs to balance the pressure of real-time and data processing.

[0145] Use proxy tools for preliminary processing: Use proxy tools (such as Jmx Exporter) to perform preliminary data filtering or aggregation to reduce the amount of data transmitted to Pushgateway and reduce the burden on the monitoring system.

[0146] Introduce a stream processing framework: Introduce a stream processing framework (such as Apache Kafka, Apache Flink, etc.) to realize real-time processing and filtering of monitoring data, reducing the real-time processing pressure of the monitoring system.

[0147] Set threshold alarms: Set threshold alarms for key indicators. Alarms are triggered and detailed data is recorded only when certain preset conditions are reached, rather than continuously collecting all data.

[0148] Incremental monitoring: For some indicators, incremental monitoring is used to record only the changes relative to the last collection, rather than recording complete data every time.

[0149] Optimize the middleware itself: For middleware within the controllable range, you can consider enhancing its own monitoring capabilities, such as developing plug-ins or modules to implement the filtering function of monitoring parameters. This requires considering the monitoring requirements during the design stage of the middleware.

[0150] Use a dedicated monitoring agent: For middleware that cannot directly enhance monitoring capabilities, a dedicated monitoring agent software can be developed. The agent software is responsible for preprocessing and filtering data locally, and then sending the processed data to the monitoring system.

[0151] Through these measures, the real-time and efficiency of the monitoring system can be improved without significantly increasing the complexity of the monitoring system, so that the middleware has the ability to monitor in seconds. These solutions comprehensively consider data volume control, real-time requirements and the overall performance of the system, aiming to achieve comprehensive and efficient monitoring.

[0152] 210. When the monitoring object is a microservice, the core monitoring parameter push task is used to obtain a preset number of core monitoring parameters and push them to the corresponding parameter push gateway.

[0153] Wherein, the parameter push gateway is used to store the core monitoring parameters of the microservice;

[0154] like Figure 3 As shown in the figure, Push Task basic dependency packages are provided for microservices. Push Task supports important parameters such as core API interface latency, throughput, and load, i.e., core monitoring parameters. Its pull cycle can be 1s. Second-level push is achieved through second-level Push Task scheduling to ensure the real-time performance of core monitoring parameters pushed to Pushgateway. This Pushgateway is used to pull core monitoring parameters from RealTime Prometheus Server.

[0155] This step pushes core monitoring parameters to the corresponding Pushgateway through a custom Push Task scheduled task, which is used to collect microservice monitoring parameters in seconds.

[0156] 220. When the monitored object is a server, a custom micro-server parameter collector is used to obtain a preset number of core monitoring parameters from the full monitoring parameters.

[0157] like Figure 3 As shown in the figure, server monitoring uses Minor Node Exporter to filter out core monitoring parameters from all monitoring parameters provided by Node Exporter, including but not limited to core monitoring parameters of modules such as network, process, CPU, memory, and hard disk. Minor Node Exporter ensures that the number of core monitoring parameters is within the preset number, for example, 100, which is used for RealTime Prometheus Server to pull core monitoring parameters in real time, so as to achieve the core monitoring parameters of the server collected in seconds.

[0158] 230. According to a second preset period, use the real-time storage parameter server to pull the core monitoring parameters.

[0159] From steps 210-220, the core monitoring parameters used by the microservice and the server can be obtained. Then, according to the second preset period, the core monitoring parameters are pulled by using the real-time storage parameter server.

[0160] like Figure 3 As shown in the figure, RealTime Prometheus Server pulls the core monitoring parameters collected by Minor Node Exporter and the core monitoring parameters stored in Pushgateway corresponding to the microservice through short pull (for example: collection period 1s), realizes real-time collection of core monitoring parameters of multiple monitoring objects, and ensures the speed of core monitoring parameter collection.

[0161] It should be noted that: the transmission scheme composed of step 210 and step 230 is a pull mode; the transmission scheme composed of step 220 and step 230 is a push mode.

[0162] The technical solution composed of the above steps 210-230 solves the real-time and resource optimization problems in microservice architecture and server monitoring by designing a reasonable monitoring data collection, processing and storage mechanism. For microservice monitoring, the solution uses a custom Push Task scheduled task to collect core monitoring parameters from each service instance and push them to Pushgateway in real time, and then the Prometheus server pulls these data to achieve second-level monitoring; for server monitoring, the key parameters are selected from the full monitoring parameters by deploying Minor Node Exporter, and the number of these parameters is controlled, which also achieves second-level collection. Finally, these core monitoring parameters are pulled by the real-time storage server according to the preset period, which ensures the real-time and effectiveness of the monitoring data, while reducing the pressure of data transmission and storage, and improving the overall performance of the monitoring system. In addition, this technical solution can also flexibly adjust the data collection cycle to achieve the core monitoring parameter collection of seconds or even shorter periods while ensuring the overall performance of the system, ensuring that no full monitoring information is missed.

[0163] Based on the above Figure 4 It can be seen from the implementation of the embodiment that the present invention provides a monitoring parameter collection method. This embodiment adopts different collection devices of core monitoring parameters set in different monitoring objects, and then combines the pull mode and the push mode to complete the real-time collection of core monitoring parameters.

[0164] In order to better explain the solution provided in the embodiments of the present application, the following examples are given.

[0165] 1. Settings

[0166] (1) Set up two server hosts, configured as "RealTime Prometheus Server" and "CommonPrometheus Server" respectively.

[0167] Specifically: download the Prometheus installation package, install two Prometheus servers, and configure them as "RealTimePrometheus Server" and "Common Prometheus Server" respectively.

[0168] (2) Build a corresponding Pushgateway for the RealTime Prometheus Server to collect the core monitoring parameters of microservices in real time at the second level.

[0169] Specifically: download the Pushgateway installation package, install Pushgateway, specify the Pushgateway port number through the web listening address, specify the persistence file through the persistence file name, and load it when Pushgateway restarts; set the pull mode common parameters of RealTime Prometheus Server, and edit the corresponding configuration file to enable RealTime Prometheus Server to pull core monitoring parameters.

[0170] (3) Build a corresponding Pushgateway for Common Prometheus Server to collect full monitoring parameters of the middleware in non-real-time seconds.

[0171] Specifically: download the Pushgateway installation package, install Pushgateway, specify the Pushgateway port number through the web listening address, specify the persistence file through the persistence file name, and load it when Pushgateway restarts; set the pull mode common parameters of RealTime Prometheus Server, and edit the corresponding configuration file to implement Common Prometheus Server to pull core monitoring parameters.

[0172] (4) Deploy and install Node Exporter on the monitored object "server", customize Minor NodeExporter and deploy and install it.

[0173] Specifically: first download the Node Exporter installation package and unzip it, then start Node Exporter to install it. Customize the Minor Node Exporter plug-in: Use the Node Exporter source code library for secondary development, or collect monitoring parameters in a customized way, and set the monitoring parameters in the metrics (monitoring parameter) format of Prometheus. Set different port numbers for NodeExporter and Minor Node Exporter to distinguish them and avoid conflicts.

[0174] (5) Deploy Pull Exporter and Push Task on the monitored object “microservice”.

[0175] Specifically: add custom Exporter monitoring parameters through code according to actual business needs. PushTask supports important parameters such as core API interface latency, throughput, and load, i.e. core monitoring parameters, whose pull cycle can be 1s. Push Task scheduling can achieve second-level push, ensuring the real-time push of core monitoring parameters to Pushgateway.

[0176] (6) In the middleware that does not have a parameter pulling interface, install an agent tool to collect all monitoring parameters.

[0177] 2. Collection part

[0178] like Figure 3 As shown, the details are as follows:

[0179] (1) In a microservice, the Pull Exporter collects all monitoring parameters of the microservice, and the Push Task collects the core monitoring parameters of the microservice and pushes them to the corresponding Pushgateway.

[0180] (2) In the server, Node Exporter collects all monitoring parameters of the server, and Minor Node Exporter collects the core monitoring parameters of the server.

[0181] (3) In the middleware, the pull interface directly set up provides full monitoring parameters, and the proxy tool is also used to collect the full monitoring parameters of the middleware without a pull interface.

[0182] (4) Common Prometheus Server pulls all monitoring parameters from (1) to (3) through long pull.

[0183] (5) RealTime Prometheus Server pulls the core monitoring parameters in (1)-(2) through short pull.

[0184] Furthermore, as a response to the above Figure 1 In order to realize the method shown in the figure, the embodiment of the present invention also provides a monitoring parameter acquisition device for the above Figure 1 The device embodiment corresponds to the aforementioned method embodiment. For ease of reading, this device embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspond to all the contents of the aforementioned method embodiment. Figure 3 As shown, the device comprises:

[0185] The first acquisition unit 31 obtains the full monitoring parameters of at least one monitored object by using a general storage parameter server in a pull mode and / or a push mode according to a first preset period; wherein the full monitoring parameters refer to all monitoring parameters related to the monitored object;

[0186] The second acquisition unit 32 is used to obtain the core monitoring parameters of at least one monitored object through a pull mode and / or a push mode using a real-time storage parameter server according to a second preset period; wherein the core monitoring parameters are monitoring parameters that comply with preset rules; and the second preset period is less than the first preset period and is a period of seconds.

[0187] Furthermore, the embodiment of the present invention also provides another monitoring parameter collection device for implementing the above method. This device embodiment corresponds to the above method embodiment. For ease of reading, this device embodiment will not repeat the details of the above method embodiment one by one, but it should be clear that the device in this embodiment can correspond to all the contents of the above method embodiment. Figure 4 As shown, the device comprises:

[0188] The first acquisition unit 31 obtains the full monitoring parameters of at least one monitored object by using a general storage parameter server in a pull mode and / or a push mode according to a first preset period; wherein the full monitoring parameters refer to all monitoring parameters related to the monitored object;

[0189] The second acquisition unit 32 is used to obtain the core monitoring parameters of at least one monitored object through a pull mode and / or a push mode using a real-time storage parameter server according to a second preset period; wherein the core monitoring parameters are monitoring parameters that comply with preset rules; and the second preset period is less than the first preset period and is a period of seconds.

[0190] Furthermore, the first acquisition unit 31 includes:

[0191] The first acquisition module 311 is used to obtain all monitoring parameters related to the microservice using a general microservice parameter collector when the monitoring object is a microservice;

[0192] The pulling module 315 is used to use the general storage parameter server to pull the full monitoring parameters obtained from the first obtaining module 311 according to a first preset period.

[0193] Furthermore, the first acquisition unit 31 includes:

[0194] The second acquisition module 312 is used to acquire all monitoring parameters related to the server by using a general server parameter collector when the monitored object is a server;

[0195] The pulling module 315 is used to use the general storage parameter server to pull the full monitoring parameters obtained from the second obtaining module 312 according to a first preset period.

[0196] Furthermore, the first acquisition unit 31 includes:

[0197] The third acquisition module 313 is used to provide the full monitoring parameters of the middleware through the parameter pulling interface when the monitoring object is the middleware and the middleware has a parameter pulling interface;

[0198] The pulling module 315 is used for the third pulling, according to the first preset period, using the general storage parameter server to pull the full amount of monitoring parameters obtained from the third acquisition module 313.

[0199] Furthermore, the first acquisition unit 31 includes:

[0200] The fourth acquisition module 314 is used for, when the monitoring object is a middleware and the middleware does not have a parameter pulling interface, pulling the full amount of monitoring parameters of the middleware through the proxy tool and pushing them to the corresponding parameter pushing gateway, wherein the parameter pushing gateway is used to store the full amount of monitoring parameters of the middleware;

[0201] The pulling module 315 is used to use the general storage parameter server to pull the full monitoring parameters obtained from the fourth acquisition module 314 according to the first preset period.

[0202] Furthermore, the second acquisition unit 32 includes:

[0203] The first acquisition module 321 is used for, when the monitoring object is a microservice, using the core monitoring parameter push task to acquire a preset number of the core monitoring parameters and push them to the corresponding parameter push gateway, wherein the parameter push gateway is used to store the core monitoring parameters of the microservice;

[0204] The pulling module 323 is used to use the real-time storage parameter server to pull the core monitoring parameters obtained from the first obtaining module 321 according to a second preset period.

[0205] Furthermore, the second acquisition unit 32 includes:

[0206] The second acquisition module 322 is used for acquiring a preset number of the core monitoring parameters from the full monitoring parameters by using a custom micro server parameter collector when the monitoring object is a server;

[0207] The pulling module 323 is used to use the real-time storage parameter server to pull the core monitoring parameters obtained from the second obtaining module 322 according to a second preset period.

[0208] Furthermore, an embodiment of the present invention further provides a processor, the processor is used to run a program, wherein the program executes the above Figure 1 The monitoring parameter collection method described in .

[0209] Furthermore, an embodiment of the present invention further provides a storage medium, wherein the storage medium is used to store a computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the above Figure 1 The monitoring parameter collection method described in .

[0210] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0211] It is understandable that the related features in the above methods and devices can be referenced to each other. In addition, the "first", "second" and the like in the above embodiments are used to distinguish the embodiments, but do not represent the advantages and disadvantages of the embodiments.

[0212] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0213] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific languages ​​is for disclosing the best mode of the present invention.

[0214] In addition, the memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0215] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0216] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0217] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0218] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0219] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0220] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0221] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0222] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0223] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0224] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A monitoring parameter collection method, characterized in that: The method comprises: According to a first preset period, using a general storage parameter server to obtain full monitoring parameters of at least one monitoring object through a pull mode and / or a push mode; wherein the full monitoring parameters refer to all monitoring parameters related to the monitoring object; According to a second preset period, the core monitoring parameters of at least one monitored object are obtained by using a real-time storage parameter server through a pull mode and / or a push mode; wherein the core monitoring parameters are monitoring parameters that comply with preset rules; and the second preset period is less than the first preset period and is a period of seconds.

2. The method according to claim 1, characterized in that The method of obtaining full monitoring parameters of at least one monitoring object by using a general storage parameter server in a pull mode and / or a push mode according to a first preset period includes: When the monitoring object is a microservice, a general microservice parameter collector is used to obtain all monitoring parameters related to the microservice; According to a first preset period, the general storage parameter server is used to pull the full amount of monitoring parameters.

3. The method according to claim 2, characterized in that The method of obtaining full monitoring parameters of at least one monitoring object by using a general storage parameter server in a pull mode and / or a push mode according to a first preset period includes: When the monitored object is a server, a general server parameter collector is used to obtain all monitoring parameters related to the server; According to a first preset period, the general storage parameter server is used to pull the full amount of monitoring parameters.

4. The method according to claim 3, characterized in that The method of obtaining full monitoring parameters of at least one monitoring object by using a general storage parameter server in a pull mode and / or a push mode according to a first preset period includes: When the monitoring object is a middleware, and the middleware has a parameter pulling interface, the full monitoring parameters of the middleware are provided through the parameter pulling interface; According to a first preset period, the general storage parameter server is used to pull the full amount of monitoring parameters.

5. The method according to claim 4, characterized in that The method of obtaining full monitoring parameters of at least one monitoring object by using a general storage parameter server in a pull mode and / or a push mode according to a first preset period includes: When the monitoring object is a middleware, and the middleware does not have a parameter pulling interface, the full amount of monitoring parameters of the middleware are pulled through an agent tool and pushed to the corresponding parameter push gateway, wherein the parameter push gateway is used to store the full amount of monitoring parameters of the middleware; According to a first preset period, the general storage parameter server is used to pull the full amount of monitoring parameters.

6. The method according to any one of claims 1 to 5, characterized in that: The method of obtaining the core monitoring parameters of at least one monitoring object by using the real-time storage parameter server in a pull mode and / or a push mode according to the second preset period includes: When the monitoring object is a microservice, a preset number of core monitoring parameters are acquired by using the core monitoring parameter push task and pushed to a corresponding parameter push gateway, wherein the parameter push gateway is used to store the core monitoring parameters of the microservice; According to a second preset period, the core monitoring parameters are pulled using the real-time storage parameter server.

7. The method according to claim 6, characterized in that The method of obtaining the core monitoring parameters of at least one monitoring object by using the real-time storage parameter server in a pull mode and / or a push mode according to the second preset period includes: When the monitored object is a server, a preset number of the core monitoring parameters are obtained from the full monitoring parameters using a custom micro-server parameter collector; According to a second preset period, the core monitoring parameters are pulled using the real-time storage parameter server.

8. A monitoring parameter acquisition device, characterized in that: include: A first acquisition unit, according to a first preset period, uses a general storage parameter server to obtain the full monitoring parameters of at least one monitoring object through a pull mode and / or a push mode; wherein the full monitoring parameters refer to all monitoring parameters related to the monitoring object; The second acquisition unit is used to obtain the core monitoring parameters of at least one monitored object through a pull mode and / or a push mode using a real-time storage parameter server according to a second preset period; wherein the core monitoring parameters are monitoring parameters that comply with preset rules; and the second preset period is less than the first preset period and is a period of seconds.

9. A storage medium comprising a stored program, characterized in that: When the program is running, the device where the storage medium is located is controlled to execute the monitoring parameter collection method described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, all or part of the steps of the monitoring parameter collection method according to any one of claims 1 to 7 are implemented.

Images