Scalable and pluggable cloud native monitoring system platform and monitoring method thereof

Abstract

The present invention provides an expandable and pluggable cloud native monitoring system platform and a monitoring method thereof, including an operation portal subsystem, a basic component subsystem and an implementation target subsystem; the implementation target subsystem runs in the monitoring target in the form of a plug-in, executes a script or a probe program according to a remote instruction to collect the operating status of the monitoring target, and generates real-time monitoring index data; the basic component subsystem is deployed in each firewall area, collects and summarizes the real-time monitoring index data of all monitoring targets in the firewall area, and performs persistent storage, and provides a query interface to the outside; the operation portal subsystem edits and mobilizes the monitoring scripts and probe programs of each monitoring target of the implementation target subsystem, and uniformly manages the configuration of the basic component subsystem. Through this scheme, the present invention realizes the batch monitoring deployment and personalized monitoring configuration of large-scale computing nodes under the cloud native system.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud native monitoring, and specifically, to an extensible and pluggable cloud native monitoring system platform and a monitoring method thereof, and in particular, preferably to an implementation scheme of an extensible and pluggable cloud native monitoring system platform. Background Art

[0002] Under the cloud-native application architecture, the original monolithic system has become a collaboration of many microservices. The interruption of one microservice will cause the entire system to be interrupted or put the system into a fault state. Therefore, monitoring these microservices has become a necessary task.

[0003] Indicators are an indispensable part of the production system and an important basis for the stable operation of the system. Especially in a cloud-native environment, a good indicator monitoring system plays an important role in the efficient and smooth operation of cloud-native applications.

[0004] The Chinese invention patent document with publication number CN113268398A discloses a business monitoring method and monitoring system for a cloud native data warehouse, which method includes: synchronizing data generated by various business fields to the cloud native data warehouse, querying corresponding data from the cloud native data warehouse and performing indicator calculation based on pre-set indicator calculation configuration data to generate indicator data; calculating the indicator data according to preset alarm rules to obtain alarm information, and notifying the alarm recipient when the alarm information reaches a preset alarm threshold.

[0005] In view of the above-mentioned prior art, the inventor believes that in the current field of cloud-native monitoring technology, the monitoring system's collection of monitoring indicators depends on the monitoring indicator collection component it provides, also known as a probe (exporter). There are still many deficiencies in the scalability and ease of use of probes. Although cloud-native monitoring components represented by Prometheus provide users with a variety of types of probes (exporters) through an active community, it is still difficult to meet the diverse needs of large-scale enterprise-level scenarios. The specific reasons are as follows:

[0006] After users download and obtain the probe (exporter), they need to manually deploy and install it on the server where the monitoring target is located, which is not convenient for large-scale management. At present, the industry generally adopts manual operation and maintenance. The management difficulty increases exponentially with the increase in the number of machines and probes, and the ease of use is insufficient. The probe (exporter) itself is also an executable program. If you want to generate personalized or customized monitoring indicators, you need to modify the existing probe program or rewrite the probe program, which is difficult for operation and maintenance personnel and lacks scalability. Summary of the invention

[0007] In view of the defects in the prior art, the object of the present invention is to provide an expandable and pluggable cloud-native monitoring system platform and a monitoring method thereof.

[0008] According to the present invention, an expandable and pluggable cloud-native monitoring system platform is provided, including an operation portal subsystem, a basic component subsystem and an implementation target subsystem;

[0009] The implementation target subsystem runs in the monitoring target in the form of a plug-in, executes scripts or probe programs according to remote instructions to collect the operating status of the monitoring target, and generates real-time monitoring indicator data;

[0010] The basic component subsystem is deployed in each firewall area, collects and aggregates the real-time monitoring indicator data of all monitoring targets in the firewall area, stores them persistently, and provides a query interface to the outside world;

[0011] The operation portal subsystem edits and mobilizes the monitoring scripts and probe programs of each monitoring target of the implementation target subsystem, and uniformly manages the configuration of the basic component subsystem.

[0012] Preferably, the implementation target subsystem includes a monitoring plug-in module;

[0013] The monitoring plug-in module receives remote instructions, downloads and updates monitoring probes and public and private letter scripts, and schedules the operation. According to the results of the monitoring script operation, it generates real-time monitoring indicator data and exposes the interface for collection.

[0014] The monitoring plug-in module provides an instruction receiving interface, instruction authentication, public-private letter script result conversion, public-private letter script operation scheduling, probe program operation scheduling, script download and update, and probe program download and update.

[0015] Preferably, the monitoring plug-in module is a WEB application program written in JAVA programming language, which exposes HTTP interface for remote program to transmit instructions;

[0016] The instruction authentication of the monitoring plug-in module is based on the signature algorithm of the domestic hash algorithm to obtain the signature value;

[0017] The signature value is placed in the request header of the HTTP request by the instruction caller;

[0018] The monitoring plug-in module recalculates the signature based on the information in the instruction HTTP request and verifies whether the signature in the request header is legal.

[0019] Preferably, the basic component subsystem includes a Prometheus module, a middle platform gateway module and a file gateway module;

[0020] The Prometheus module accesses the monitoring plug-in and the monitoring interface exposed by the probe, aggregates and collects the monitoring data and stores it persistently in the local time series database, and then provides a query interface for the time series database externally;

[0021] The middle platform gateway module is responsible for forwarding the instructions of the monitoring plug-in and forwarding the requests for Prometheus monitoring queries;

[0022] The file gateway module provides transit storage for probe files, monitoring plug-in files, and public and private letter script files, and distributes them to each target server.

[0023] Preferably, the instructions received by the monitoring plug-in module include script and probe program download and update, and the monitoring plug-in module accesses the middle station gateway in the same network area as the running server to download the target file medium;

[0024] The public and private letter scripts are text files, which are downloaded and stored in the specified directory. The monitoring plug-in module accesses the script file at a predetermined frequency and executes the commands in the script file, accesses and collects the operation information of the monitoring target, and then returns the operation information as the result to the monitoring plug-in module. The monitoring plug-in module collects the results and converts them into Prometheus format, provides a Prometheus capture interface, and provides the Prometheus module for collection at any time.

[0025] The probe program is a computer program. After downloading, run the installation script to complete the installation. When the installation is complete, the monitoring plug-in module calls the running script to start the probe program. The probe program collects data from the monitoring target and converts it into Prometheus format, provides a Prometheus crawling interface, and provides Prometheus module collection.

[0026] Preferably, if the configuration of the public or private letter script needs to be modified during the operation of the monitoring plug-in module, the remote end sends an instruction to update the configuration to the monitoring plug-in module, the monitoring plug-in module saves the new configuration locally in the form of a file, and restarts the corresponding public or private letter script or probe program according to the new configuration.

[0027] Preferably, the cloud-native monitoring system is linked with the data center availability system, which deploys the monitoring plug-in modules to all servers in batches and adds them to the scheduled tasks of the operating system to regularly determine the survival of the program so that the monitoring plug-in modules run as expected in each server.

[0028] Preferably, the operation portal subsystem includes a monitoring plug-in management module, a probe management module, a public and private letter management module and a configuration center module;

[0029] The monitoring plug-in management module provides management functions for the monitoring plug-ins that have been deployed on the server, including self-update and configuration management;

[0030] The probe management module provides an upload operation for the probe program, installs or updates the uploaded probe program to the server where the monitoring plug-in has been deployed, and performs operation scheduling and configuration update on the probe program;

[0031] The public and private letter management module provides the addition and editing operations of public monitoring function scripts and private monitoring function scripts, sends or updates the public monitoring function scripts to the server where the monitoring plug-in has been deployed, and performs operation scheduling and configuration updates on the public and private letter scripts;

[0032] The configuration center module is responsible for centralized management of the configuration of the Prometheus module, the middle platform gateway module, and the file gateway module.

[0033] Preferably, the basic component subsystem is deployed in each firewall zone;

[0034] In each firewall area, the Prometheus module is implemented based on the open source monitoring system Prometheus, reads the configuration file generated by the configuration center module, accesses the monitoring plug-in in the firewall area and the HTTP interface exposed by the probe program, and collects and stores monitoring indicator data;

[0035] The middle station gateway module is responsible for forwarding network requests for monitoring plug-in instructions and monitoring data queries across firewall areas. When a network request needs to be transmitted across firewall areas, the network request is sent to the middle station gateway in the area where the request sender is located, and then the middle station gateway sends the request to the middle station gateway in the network area where the target is located. The middle station gateway in the network area where the target is located finally sends the request to the target;

[0036] The file gateway module is responsible for distributing public and private letter script files and probe program files within each firewall area. When the monitoring plug-in module receives an update or installs and downloads a public or private letter script and a probe program, it accesses the file gateway within the current firewall area based on the domain name, obtains the file media and downloads it locally.

[0037] According to a monitoring method of an extensible and pluggable cloud native monitoring system platform provided by the present invention, the extensible and pluggable cloud native monitoring system platform is applied, and the following steps are included:

[0038] Step S1: Synchronize the target server information from the configuration management database to the operation portal subsystem;

[0039] Step S2: The operation portal subsystem converts the target server information synchronized in step S1 into an operation instruction for the target server, and sends the request to the middle platform gateway;

[0040] Step S3: The middle platform gateway forwards the operation instruction from the operation portal subsystem to the target server;

[0041] Step S4: If the target server receives an operation instruction of the file download type, it requests the file from the file gateway;

[0042] Step S5: When the file gateway receives the file download request and cannot find the file locally, it requests the file from the operation portal subsystem and then returns;

[0043] Step S6: The operation portal subsystem converts the target server information synchronized in step S1 into Prometheus configuration and sends it to each Prometheus module;

[0044] Step S7: The Prometheus module collects monitoring data for the target server according to the latest configuration.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. Through this solution, the present invention realizes batch monitoring deployment and personalized monitoring configuration of large-scale computing nodes in a cloud native system;

[0047] 2. Through this solution, the present invention realizes the dynamic addition of monitoring indicators by online writing of public and private letter scripts to meet customized monitoring needs;

[0048] 3. Through this solution, the present invention realizes the adaptation to the multi-firewall network architecture of the bank, and realizes centralized monitoring and display through a lower amount of cross-firewall network data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0050] Figure 1 This is a diagram of an implementation scheme of an expandable and pluggable cloud-native monitoring system platform of the present invention;

[0051] Figure 2 It is the detailed block diagram of the monitoring plug-in module;

[0052] Figure 3 For full compatibility with extensible plug-in monitoring method diagram. DETAILED DESCRIPTION

[0053] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0054] The embodiment of the present invention discloses an implementation scheme of an expandable and pluggable cloud native monitoring system platform, such as Figure 1 As shown, it can be divided into three subsystems: operation portal, basic components, and implementation goals. The details of each subsystem are as follows:

[0055] 1. Implement the target subsystem:

[0056] This subsystem consists of multiple modules, specifically: (1) Monitoring plug-in module, which receives remote instructions, downloads and updates monitoring probes and public and private scripts, and schedules their operation. It also generates real-time monitoring indicator data based on the results of the monitoring scripts, and exposes interfaces for external collection. The details of this module are as follows: Figure 2 As shown, it can be seen that the module can also be divided according to its functions: providing instruction receiving interface, instruction authentication, public and private letter script and probe program operation scheduling, public and private letter script result conversion, script and probe program download and update. Specifically, the monitoring plug-in module is a WEB (World Wide Web) application written in the JAVA programming language, which exposes an HTTP (Hyper Text Transfer Protocol) interface for remote programs to convey instructions. In order to prevent the instruction interface from being accessed by malicious calls, this program also designs the function of instruction authentication, and the program designs a signature algorithm based on SM3 (domestic hash algorithm). The details are as follows:

[0057] The first step is to extract the HTTP request method.

[0058] The second step is to extract the request URL information, including the domain name and the path part of the URI. (The full name of URL in English is uniform resource locator, and the Chinese translation is uniform resource locator. The full name of URI in English is Uniform Resource Identifier, and the Chinese translation is uniform resource identifier.)

[0059] The third step is to extract the HTTP request message body.

[0060] The fourth step is to concatenate the above request method, URL information, HTTP request message body and the key negotiated between systems.

[0061] The fifth step is to calculate the SM3 (a domestic cryptographic algorithm) value of the string concatenated in the fourth step, which is the signature value.

[0062] The signature value will be placed in the request header of the HTTP request by the instruction caller; the monitoring plug-in module recalculates the signature based on the information in the instruction HTTP request and verifies whether the signature in the request header is legal.

[0063] The instructions received by the monitoring plug-in module are mainly divided into public and private letter scripts and probe program operation scheduling and script and probe program download and update as well as configuration update. When the instruction is script and probe program download and update, the monitoring plug-in module will first access the middle platform gateway in the same network area as the server on which it runs to download the target file medium. On the one hand, the public and private letter scripts are text files, so they can be stored in the specified directory after downloading. The public and private letter scripts support Python (a programming language) and Shell (a programming language). The monitoring plug-in accesses and executes the script file at a certain frequency. The commands in the script file will access and collect the operation information of the monitoring target, such as servers and middleware applications, and then return the operation information as the result to the monitoring plug-in. The monitoring plug-in collects these results and converts them into real-time monitoring indicators in Prometheus format, provides a standard Prometheus crawling interface, and provides Prometheus modules for collection at any time. On the other hand, the probe program is a computer program, so after the monitoring plug-in is downloaded, its installation script will be run to complete the installation. When the installation is complete, the monitoring plug-in will call its running script to start the probe program. The probe program will automatically collect data from the monitored target and convert it to Prometheus format, provide a standard Prometheus crawling interface, and provide Prometheus module collection. During the operation, if you need to modify the configuration of the public and private letter scripts, such as the label name when the script result is converted to Prometheus format, or change the port where the probe program runs, the remote end will send an instruction to update the configuration to the monitoring plug-in module. The monitoring plug-in module will save the new configuration to the local in the form of a file, and restart the corresponding public and private letter script or probe program according to the new configuration. In addition, in order to facilitate remote configuration management, the monitoring plug-in module also provides an HTTP interface for viewing the local public and private letter or probe program configuration.

[0064] In this system, the monitoring plug-in module itself is a computer program, which is responsible for the operation scheduling of public and private letter scripts and probe programs, as well as the communication with remote programs. It is the key to this monitoring system. In order to enable the monitoring plug-in module to be automatically deployed in batches to the target server and ensure its availability. This system is linked with the external system-the data center availability system. The data center availability system will deploy the monitoring plug-in module to all servers in batches and add it to the scheduled tasks of the operating system. It will regularly determine the survival of this program. If it is not alive, it will try to pull it up. To ensure that this monitoring plug-in module will run as expected in each server.

[0065] 2. Basic component subsystem:

[0066] This subsystem consists of multiple modules, specifically:

[0067] (1) Prometheus module: This module collects and aggregates monitoring data by accessing the monitoring plug-ins and monitoring interfaces exposed by the probes, stores the data persistently in the local time series database, and then provides a query interface for the time series database.

[0068] (2) The middle platform gateway module is responsible for forwarding the instructions of the monitoring plug-in and forwarding the requests for Prometheus monitoring queries.

[0069] (3) File gateway module, which provides transit storage for probe files, monitoring plug-in files, and public and private letter script files, and distributes them to each target server.

[0070] The relationship between the above modules, such as Figure 1As shown in the figure, the Prometheus module, the middle platform gateway module, and the file gateway module are the basic components of the system. The combination of these three modules as the smallest unit will be deployed in each firewall area to adapt to the network architecture of multiple firewalls and reduce the amount of network data across firewalls. Specifically, in each firewall area, the Prometheus module is implemented based on the open source monitoring system Prometheus. It reads the configuration file generated by the configuration center module, accesses the monitoring plug-in in this firewall area and the HTTP interface exposed by the probe program, and collects and stores monitoring indicator data. The middle platform gateway module mainly undertakes the network request forwarding function of monitoring plug-in instructions and monitoring data queries across firewall areas. When a network request needs to be transmitted across firewall areas, the request will be sent to the middle platform gateway in the area where the request sender is located, and then this middle platform gateway will send the request to the middle platform gateway in the network area where the target is located. The middle platform gateway in the network area where the target is located will finally send the request to the specific target. The file gateway is responsible for distributing public and private letter script files and probe program files in each firewall area. When the monitoring plug-in module receives an update or installs a public and private letter script and probe program, it will access the file gateway in the current firewall area according to the domain name, obtain the file medium and download it locally. As a file transfer station, when it receives a download request from the monitoring plug-in module, if it finds that the file is not locally, it will send it to the remote ( Figure 1 The operation portal subsystem of the monitoring plug-in module will request the media, download it locally, and then return the media to the download request of the monitoring plug-in module. In addition, when new file media is uploaded or updated, the operation portal subsystem will actively push the file to the file gateway to achieve a "preheating" effect.

[0071] 3. Operation portal subsystem:

[0072] This subsystem consists of multiple modules, specifically:

[0073] (1) Monitoring plug-in management module, which provides management functions for monitoring plug-ins that have been deployed on the server, including self-update and configuration management. This module allows users to centrally manage deployed monitoring plug-ins on the page, including starting and stopping monitoring plug-in programs, configuration changes of monitoring plug-in programs, and sub-updates of monitoring plug-in programs. After the above management operations occur, the module will generate an instruction, which will be sent to the middle-end gateway in the firewall area where the operation portal is located via an HTTP request, and then forwarded to the target monitoring plug-in.

[0074] (2) Probe management module, which provides upload operations for probe programs, installs or updates the uploaded probe programs to the server where the monitoring plug-in has been deployed, and schedules their operation and updates their configuration. This module allows users to operate one or more probe programs in the target server on the page, including installing a new probe program on a server node, updating the probe program, uninstalling the probe program, setting the operation cycle of the probe program, and updating the configuration of the probe program. After the above operations occur, the module will generate an instruction, which will be sent to the middle-office gateway in the firewall area where the operation portal is located via an HTTP request, and then forwarded to the target monitoring plug-in. The monitoring plug-in will then complete the operation scheduling or download update of the probe program according to the instruction.

[0075] (3) Public and private letter management module, which provides the addition and editing operations for public monitoring function scripts and private monitoring function scripts, and distributes or updates the public monitoring function scripts to the server where the monitoring plug-in has been deployed, and performs operation scheduling and configuration updates. Public monitoring function scripts are highly versatile and can be distributed to all servers. Private function scripts can only be applied to a certain system, so their distribution range is the servers belonging to a certain system. When public and private letters are distributed, you can choose the independent configuration of each distribution server and the operation scheduling configuration, such as monitoring the size of the folder. When each public letter is distributed, you can enter a different path as a parameter, and for example, you can configure it to be executed only between 1 and 2 a.m. every night. After the user operates in the public and private letter management module, the module will generate an instruction, which is sent to the middle-end gateway in the firewall area where the operation portal is located through an HTTP request, and then forwarded to the target monitoring plug-in. The monitoring plug-in then schedules the operation or downloads and updates the public and private letter script files according to the instruction.

[0076] (4) Alarm management module: This module provides configuration for alarm rules and then performs cyclic calculations on the monitoring data collected in the Prometheus module according to the rules. When the rule threshold is reached, an alarm is issued. The alarm is in the form of an HTTP request and is sent to the data center availability system. The data center availability system then sends it to the user via email, text message, etc.

[0077] (5) Visual monitoring dashboard module: This module issues query requests in Prometheus format and then displays the returned monitoring data in a graphical manner.

[0078] (6) Configuration center module, which is responsible for centralized management of the configuration of the Prometheus module, the middle platform gateway module, and the file gateway module.

[0079] Each module in the above-mentioned operation portal subsystem is a WEB program as a whole, which constitutes an operation portal subsystem and runs in a firewall area. All servers and their operating system information will be collected by the data center CMDB (a database for managing various configuration information of devices in the enterprise IT architecture). The CMDB will be synchronized to the operation portal subsystem at regular intervals, and then the configuration center module will convert the information of each server node in the CMDB into Prometheus configuration and send it to the Prometheus module in each firewall area. Since the Prometheus modules are distributed in various firewall areas, the monitoring data query request of the visual monitoring disk module will first be sent to the middle platform gateway in the firewall area where the operation portal subsystem is located, and then forwarded to the middle platform gateway in the firewall area where the target Prometheus module is located, and finally reach the Prometheus module. The Prometheus module request will be calculated based on the locally stored monitoring data and the request will be returned by the original path. The full name of CMDB in English is Configuration Management Database, and the Chinese translation is Configuration Management Database.

[0080] The present invention implements the target subsystem in the form of a plug-in and runs it in the monitoring target. According to remote instructions, it executes scripts or probe programs to collect the operating status of the monitoring target and generates real-time monitoring index data, which is collected and summarized by a set of basic component subsystems in each firewall area for persistent storage. Users can manage each monitoring target and each monitoring component through the operation portal subsystem. Through this solution, batch monitoring deployment and personalized monitoring configuration of large-scale computing nodes in the cloud native system are realized; it is realized by writing public and private letter scripts online and dynamically adding monitoring indicators to meet customized monitoring needs; it is realized to adapt to the bank's multi-firewall network architecture, and realize centralized monitoring display through a low amount of cross-firewall network data.

[0081] The embodiment of the present invention also discloses a fully compatible and extensible plug-in monitoring system and method, such as Figure 3 As shown, the following steps are included:

[0082] Step S1: Synchronize the target server information from CMDB to the operation portal subsystem.

[0083] Step S2: The monitoring plug-in management, monitoring probe management, and public and private letter management modules in the operation portal subsystem convert the target server information synchronized in step S1 into operation instructions for the target server, such as file download, operation scheduling, etc., and send the request to the middle platform gateway.

[0084] Step S3: The middle platform gateway forwards the operation instructions from the operation portal subsystem to the target server.

[0085] Step S4: If the target server receives an operation instruction of the file download type, it will request the file from the file gateway.

[0086] Step S5: When the file gateway receives a file download request and cannot find the file locally, it will request the file from the operation portal subsystem and then return.

[0087] Step S6: The configuration center module in the operation portal subsystem converts the target server information synchronized in step S1 into Prometheus configuration and sends it to each Prometheus module.

[0088] Step S7: The Prometheus module collects monitoring data for the target server according to the latest configuration.

[0089] The implementation scheme of the scalable and pluggable cloud-native monitoring system platform of the present invention designs three subsystems: operation portal, basic components, and implementation targets. The implementation target subsystem runs in the monitoring target in the form of a plug-in, and executes scripts or probe programs to collect monitoring indicators of the monitoring target according to remote instructions. The basic component subsystem accesses and collects monitoring indicator data for monitoring targets located in various firewall areas, and provides a unified query capability to the outside world. In the operation portal subsystem, the monitoring scripts and probe programs of each monitoring target of the implementation target subsystem can be edited and mobilized, and the configuration of the basic component subsystem can be uniformly managed, and the final monitoring effect page can be displayed to the user.

[0090] That is, the implementation target subsystem runs in the monitoring target in the form of a plug-in. According to remote instructions, it executes scripts or probe programs to collect the operating status of the monitoring target and generate real-time monitoring indicator data; the basic component subsystem will be deployed in each firewall area. Each subsystem will collect and summarize the real-time monitoring indicator data of all monitoring targets in the firewall area, and store them persistently through the Prometheus time series database, and provide a query interface for the monitoring indicator data under the computer room dimension; the operation portal subsystem edits and mobilizes the monitoring scripts and probe programs of each monitoring target of the implementation target subsystem, and uniformly manages the configuration of the basic component subsystem.

[0091] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0092] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. An extensible and pluggable cloud-native monitoring system platform, characterized in that: It includes the operation portal subsystem, basic component subsystem and implementation target subsystem; The implementation target subsystem runs in the monitoring target in the form of a plug-in, executes scripts or probe programs according to remote instructions to collect the operating status of the monitoring target, and generates real-time monitoring indicator data; The basic component subsystem is deployed in each firewall area, collects and aggregates the real-time monitoring indicator data of all monitoring targets in the firewall area, stores them persistently, and provides a query interface to the outside world; The operation portal subsystem edits and mobilizes the monitoring scripts and probe programs of each monitoring target of the implementation target subsystem, and uniformly manages the configuration of the basic component subsystem; The implementation target subsystem includes a monitoring plug-in module; The monitoring plug-in module receives remote instructions, downloads and updates monitoring probes and public and private letter scripts, and schedules the operation. According to the results of the monitoring script operation, it generates real-time monitoring indicator data and exposes the interface for collection. The monitoring plug-in module provides an instruction receiving interface, instruction authentication, public-private letter script result conversion, public-private letter script operation scheduling, probe program operation scheduling, script download and update, and probe program download and update; The monitoring plug-in module is a WEB application written in JAVA programming language, which exposes HTTP interface for remote program to transmit instructions; The instruction authentication of the monitoring plug-in module is based on the signature algorithm of the domestic hash algorithm to obtain the signature value; The signature value is placed in the request header of the HTTP request by the instruction caller; The monitoring plug-in module recalculates the signature based on the information in the instruction HTTP request and verifies whether the signature in the request header is legal; The basic component subsystem includes a Prometheus module, a middle platform gateway module, and a file gateway module; The Prometheus module accesses the monitoring plug-in and the monitoring interface exposed by the probe, aggregates and collects the monitoring data and stores it persistently in the local time series database, and then provides a query interface for the time series database externally; The middle platform gateway module is responsible for forwarding the instructions of the monitoring plug-in and forwarding the requests for Prometheus monitoring queries; The file gateway module provides transit storage for probe files, monitoring plug-in files, and public and private letter script files, and distributes them to each target server; The cloud-native monitoring system is linked to the data center availability system. The data center availability system deploys monitoring plug-in modules to all servers in batches and adds them to the scheduled tasks of the operating system to regularly determine the survival of the program, so that the monitoring plug-in modules can run as expected in each server. The operation portal subsystem includes a monitoring plug-in management module, a probe management module, a public and private letter management module, and a configuration center module; The monitoring plug-in management module provides management functions for the monitoring plug-ins that have been deployed on the server, including self-update and configuration management; The probe management module provides an upload operation for the probe program, installs or updates the uploaded probe program to the server where the monitoring plug-in has been deployed, and performs operation scheduling and configuration update on the probe program; The public and private letter management module provides the addition and editing operations of public monitoring function scripts and private monitoring function scripts, sends or updates the public monitoring function scripts to the server where the monitoring plug-in has been deployed, and performs operation scheduling and configuration updates on the public and private letter scripts; The configuration center module is responsible for centralized management of the configuration of the Prometheus module, the middle platform gateway module, and the file gateway module.

2. The scalable and pluggable cloud-native monitoring system platform according to claim 1, characterized in that: The instructions received by the monitoring plug-in module include script and probe program download and update, and the monitoring plug-in module accesses the middle station gateway in the same network area as the running server to download the target file medium; The public and private letter scripts are text files, which are downloaded and stored in the specified directory. The monitoring plug-in module accesses the script file at a predetermined frequency and executes the commands in the script file, accesses and collects the operation information of the monitoring target, and then returns the operation information as the result to the monitoring plug-in module. The monitoring plug-in module collects the results and converts them into Prometheus format, provides a Prometheus capture interface, and provides the Prometheus module for collection at any time. The probe program is a computer program. After downloading, run the installation script to complete the installation. When the installation is complete, the monitoring plug-in module calls the running script to start the probe program. The probe program collects data from the monitoring target and converts it into Prometheus format, provides a Prometheus crawling interface, and provides Prometheus module collection.

3. The scalable and pluggable cloud-native monitoring system platform according to claim 2, characterized in that: During the operation of the monitoring plug-in module, if the configuration of the public or private letter script needs to be modified, the remote end sends an instruction to update the configuration to the monitoring plug-in module. The monitoring plug-in module saves the new configuration locally in the form of a file and restarts the corresponding public or private letter script or probe program according to the new configuration.

4. The scalable and pluggable cloud-native monitoring system platform according to claim 1, characterized in that: The basic component subsystem is deployed in each firewall area; In each firewall area, the Prometheus module is implemented based on the open source monitoring system Prometheus, reads the configuration file generated by the configuration center module, accesses the monitoring plug-in in the firewall area and the HTTP interface exposed by the probe program, and collects and stores monitoring indicator data; The middle station gateway module is responsible for forwarding network requests for monitoring plug-in instructions and monitoring data queries across firewall areas. When a network request needs to be transmitted across firewall areas, the network request is sent to the middle station gateway in the area where the request sender is located, and then the middle station gateway sends the request to the middle station gateway in the network area where the target is located. The middle station gateway in the network area where the target is located finally sends the request to the target; The file gateway module is responsible for distributing public and private letter script files and probe program files within each firewall area. When the monitoring plug-in module receives an update or installs and downloads a public or private letter script and a probe program, it accesses the file gateway within the current firewall area based on the domain name, obtains the file media and downloads it locally.

5. A monitoring method for an expandable and pluggable cloud-native monitoring system platform, applying the expandable and pluggable cloud-native monitoring system platform described in any one of claims 1 to 4, comprising the following steps: Step S1: Synchronize the target server information from the configuration management database to the operation portal subsystem; Step S2: The operation portal subsystem converts the target server information synchronized in step S1 into an operation instruction for the target server, and sends the request to the middle platform gateway; Step S3: The middle platform gateway forwards the operation instruction from the operation portal subsystem to the target server; Step S4: If the target server receives an operation instruction of the file download type, it requests the file from the file gateway; Step S5: When the file gateway receives the file download request and cannot find the file locally, it requests the file from the operation portal subsystem and then returns; Step S6: The operation portal subsystem converts the target server information synchronized in step S1 into Prometheus configuration and sends it to each Prometheus module; Step S7: The Prometheus module collects monitoring data for the target server according to the latest configuration.

Images