Dynamic classification and aggregation calculation method and device for cloud monitoring resource indexes

Abstract

The invention relates to the technical field of cloud resource monitoring, and discloses a dynamic classification aggregation calculation method and device for cloud monitoring resource indexes, and the method comprises the steps: obtaining cloud monitoring resource index information and cloud monitoring resource grouping information, and generating aggregation task elements based on the cloud monitoring resource index information and the cloud monitoring resource grouping information; according to the aggregation task elements, dynamically classifying and aggregating parameters in the cloud monitoring resource grouping information to obtain an aggregation calculation task; and executing the aggregation calculation task to obtain a cloud monitoring resource index aggregation result. Classification and aggregation strategies can be flexibly adjusted according to actual requirements, and analyzability and practicability of a monitoring system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud resource monitoring, and in particular to a method and device for dynamically classifying and aggregating cloud monitoring resource indicators. Background Art

[0002] In IT (Information Technology) infrastructure monitoring, monitoring data is very complex and large, involving multiple levels such as servers, network equipment, and storage devices. Monitoring data is collected at a high frequency, resulting in a huge amount of data containing a lot of noise and fluctuations. Directly analyzing and visualizing this raw data is often not practical because the data volume is too large and it is difficult to effectively extract useful information.

[0003] Although indicator aggregation is a key step in processing large-scale monitoring data, related methods have limitations of fixedness and staticness, and cannot be flexibly adjusted and dynamically adapted to changing monitoring needs. They often lack customized processing for different monitoring products and are difficult to meet diverse needs. Summary of the Invention

[0004] In view of this, the present invention provides a dynamic classification and aggregation calculation method and device for cloud monitoring resource indicators to solve the problem that the indicator aggregation method of monitoring data cannot be flexibly adjusted and dynamically adapted to changing monitoring needs, and is difficult to customize for different monitoring products.

[0005] In a first aspect, the present invention provides a method for dynamic classification and aggregation calculation of cloud monitoring resource indicators, the method comprising:

[0006] Obtain cloud monitoring resource indicator information and cloud monitoring resource grouping information, and generate aggregation task elements based on the cloud monitoring resource indicator information and cloud monitoring resource grouping information;

[0007] According to the aggregation task elements, the parameters in the cloud monitoring resource grouping information are dynamically classified and aggregated to obtain the aggregate computing task;

[0008] Execute the aggregation calculation task to obtain the aggregation results of cloud monitoring resource indicators.

[0009] The dynamic classification and aggregation calculation method for cloud monitoring resource indicators provided in this embodiment can effectively organize different types of monitoring data through cloud monitoring resource grouping information, so that it can be more targeted when performing indicator aggregation, thereby improving the efficiency and accuracy of data analysis, and then flexibly adjusting the classification and aggregation strategies according to actual needs, dynamically classifying and aggregating the parameters in the cloud monitoring resource grouping information to obtain aggregation calculation tasks, ensuring real-time monitoring and aggregation calculation of cloud monitoring resource indicators, being able to flexibly respond to changing monitoring needs, reducing unnecessary computing burdens, and improving the analyzability and practicality of the monitoring system. It not only improves the efficiency of the monitoring system, but also enhances the real-time insight into resource usage, providing important support for enterprise decision-making.

[0010] In an optional embodiment, according to the aggregation task elements, the parameters in the cloud monitoring resource grouping information are dynamically classified and aggregated to obtain the aggregate computing task, including:

[0011] Extract the parameters corresponding to the aggregation task elements; the parameters corresponding to the aggregation task elements include monitoring class name, resource indicator name, unique label, PromQL template and aggregation period;

[0012] Parse the parameters of the PromQL template and replace the parameters in the PromQL template with the parameters in the cloud monitoring resource group information;

[0013] Extract the aggregation method in the PromQL template, and use the aggregation method to dynamically splice the parameters in the cloud monitoring resource group information to obtain the PromQL statement;

[0014] Create aggregation calculation tasks based on PromQL statements and aggregation cycles.

[0015] The dynamic classification and aggregation calculation method for cloud monitoring resource indicators provided in this embodiment automatically generates PromQL statements according to different resource grouping information and multiple aggregation methods, which can meet diverse aggregation needs and enable users to adjust aggregation methods and configuration information at any time according to needs, thereby achieving low-latency query and efficient aggregation calculation, and improving the response speed and data processing capabilities of the monitoring system.

[0016] In an optional implementation, creating an aggregation computing task based on a PromQL statement and an aggregation period includes:

[0017] Add the PromQL statement to the loaded aggregate information list and determine whether the loaded aggregate information list is empty;

[0018] If the loaded aggregation information list is empty, the PromQL statement is analyzed to generate an aggregation calculation task;

[0019] Or, if the loaded aggregation information list is not empty, it is judged whether the PromQL statements of adjacent aggregation periods are the same, and if the PromQL statements of adjacent aggregation periods are different, the aggregation calculation task is updated.

[0020] The cloud monitoring resource index dynamic classification aggregation calculation method provided in the embodiment realizes timely updating of the aggregation calculation task by analyzing whether the loaded aggregation information list is empty, and improves the efficiency of the monitoring system.

[0021] In an optional implementation, the method further comprises:

[0022] The cloud monitoring resource index aggregation result is labeled and expanded, and the cloud monitoring resource index aggregation result after label expansion is stored in the time series database.

[0023] The cloud monitoring resource index dynamic classification aggregation calculation method provided in the embodiment facilitates subsequent user query and analysis of monitoring data by performing label expansion on the cloud monitoring resource index aggregation result.

[0024] In an optional implementation, the method further comprises:

[0025] The timing task is acquired, and the aggregation task element and the aggregation calculation task are periodically updated based on the timing task.

[0026] The cloud monitoring resource index dynamic classification aggregation calculation method provided in the embodiment periodically updates the cloud monitoring resource index information and the cloud monitoring resource group information by timing management of the aggregation calculation task, ensures that the aggregation calculation task always remains in the latest state, and improves the real-time performance and reliability of the monitoring system.

[0027] In an optional implementation, periodically updating the aggregation task element and the aggregation calculation task based on the timing task comprises:

[0028] The dynamic aggregation calculation task template and the cloud monitoring resource grouping task template are created;

[0029] The timing parameter is acquired, and the timing task is constructed based on the timing parameter and the dynamic aggregation calculation task template or the cloud monitoring resource grouping task template;

[0030] The cloud monitoring resource index information and the cloud monitoring resource grouping information are periodically acquired based on the timing task to update the aggregation task element and the aggregation calculation task.

[0031] The cloud monitoring resource index dynamic classification aggregation calculation method provided in the embodiment can flexibly adjust the classification and aggregation strategy according to actual needs by timing management of the aggregation task element and the aggregation calculation task, reduces unnecessary calculation burden, and improves the analyzability and practicability of the monitoring system.

[0032] In a second aspect, the present invention provides a device for dynamically classifying and aggregating cloud monitoring resource indicators, the device comprising:

[0033] A configuration management module is used to obtain cloud monitoring resource indicator information and cloud monitoring resource grouping information, and generate aggregation task elements based on the cloud monitoring resource indicator information and cloud monitoring resource grouping information;

[0034] The dynamic aggregation module is used to dynamically classify and aggregate the parameters in the cloud monitoring resource grouping information according to the aggregation task elements to obtain the aggregated computing task;

[0035] The execution module is used to execute the aggregation computing task and obtain the aggregation results of cloud monitoring resource indicators.

[0036] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, computer instructions being stored in the memory, and the processor executing the computer instructions to thereby execute the dynamic classification, aggregation and calculation method of cloud monitoring resource indicators of the above-mentioned first aspect or any corresponding embodiment thereof.

[0037] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the dynamic classification and aggregation calculation method for cloud monitoring resource indicators of the above-mentioned first aspect or any corresponding embodiment thereof.

[0038] In a fifth aspect, the present invention provides a computer program product comprising computer instructions, the computer instructions being used to enable a computer to execute the dynamic classification and aggregation calculation method for cloud monitoring resource indicators of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 This is a flow chart of a method for dynamic classification and aggregation calculation of cloud monitoring resource indicators according to an embodiment of the present invention;

[0041] Figure 2 This is a flowchart of another method for dynamic classification and aggregation calculation of cloud monitoring resource indicators according to an embodiment of the present invention;

[0042] Figure 3is a flow diagram of another cloud monitoring resource index dynamic classification and aggregation calculation method according to an embodiment of the present application;

[0043] Figure 4 is a flow diagram of a cloud monitoring resource index dynamic classification and aggregation calculation method according to an embodiment of the present application;

[0044] Figure 5 is a dynamic aggregation flow diagram according to an embodiment of the present application;

[0045] Figure 6 is a structural block diagram of a cloud monitoring resource index dynamic classification and aggregation calculation device according to an embodiment of the present application;

[0046] Figure 7 is a hardware structure diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0048] The embodiments of the present application provide a cloud monitoring resource index dynamic classification and aggregation calculation method. It should be noted that the cloud monitoring resource index dynamic classification and aggregation calculation method provided by the embodiments of the present application can have a cloud monitoring resource index dynamic classification and aggregation calculation device as an execution subject. The cloud monitoring resource index dynamic classification and aggregation calculation device can be implemented as part or all of an electronic device in a software, hardware or combination of software and hardware manner. The electronic device can be a server or a terminal. The server in the embodiments of the present application can be a single server or a server cluster composed of multiple servers. The terminal in the embodiments of the present application can be a smart phone, a personal computer, a tablet computer, a wearable device, a smart robot or other smart hardware devices. In the following method embodiments, the execution subject is taken as an example of the electronic device.

[0049] An embodiment of the present invention provides a dynamic classification, aggregation and calculation method for cloud monitoring resource indicators, which divides the monitoring data according to different resource groups, for example, according to specific monitoring products such as cloud hosts, physical machines and cloud hard disks, or according to monitoring categories such as network equipment, databases, load balancers, etc. Each resource group contains a group of related monitoring data, which is convenient for aggregation and analysis; through resource grouping, different types of monitoring data are effectively organized, so that indicator aggregation can be more targeted, thereby improving the efficiency and accuracy of data analysis, and using dynamic configuration, timing management and classification aggregation, it is possible to flexibly adjust the classification and aggregation strategies according to actual needs, reduce unnecessary computing burden, and improve the analyzability and practicality of the monitoring system.

[0050] According to an embodiment of the present invention, an embodiment of a method for dynamic classification, aggregation and calculation of cloud monitoring resource indicators is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0051] In this embodiment, a dynamic classification and aggregation calculation method for cloud monitoring resource indicators is provided, which can be used for the above-mentioned electronic devices. Figure 1 Flowchart of a dynamic classification and aggregation calculation method for cloud monitoring resource indicators according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0052] Step S101: Obtain cloud monitoring resource indicator information and cloud monitoring resource grouping information, and generate aggregation task elements based on the cloud monitoring resource indicator information and cloud monitoring resource grouping information.

[0053] Specifically, cloud monitoring refers to a series of activities and technologies for performance monitoring and health checks of computing resources, applications, and services deployed in the cloud. Cloud monitoring can help enterprises and organizations continuously track various indicators in their cloud environments to ensure the normal operation of these resources and services and to be able to respond quickly when failures occur.

[0054] Furthermore, resource grouping refers to the logical classification and organization of monitoring data according to different resource categories. Specifically, resource grouping is a method of categorizing data from multiple monitoring objects for better management and analysis of this data. Resource grouping can be performed based on different criteria, such as by specific monitoring products (such as cloud hosts, physical machines, and cloud hard drives) or functional categories of monitored objects (such as network devices, databases, and load balancers). Resource indicators are specific numerical values ​​used to measure and evaluate the performance and health of various resources in a cloud environment. The concept of "resource grouping" is used to logically classify and organize them for better management and analysis of monitoring data. Cloud monitoring resource indicator information includes monitoring categories (such as computing, storage, and network resources), indicator names (such as CPU usage and memory usage), unique tags (such as cloud host ID and instance ID), PromQL templates (such as aggregation of CPU usage), and aggregation periods (such as 60 seconds, 1 minute, and 1 hour). Task elements are the basic information required to perform aggregate calculations, including resource indicators, aggregation periods, and PromQL templates. These elements form the basis for generating specific computing tasks, enabling the system to perform calculations based on specific needs.

[0055] Furthermore, cloud monitoring resource indicator information (such as monitoring category, indicator name, unique label, PromQL template and aggregation period) is obtained from the configuration management platform, and cloud monitoring resource grouping information is obtained from the cache, and aggregation task elements are generated based on the cloud monitoring resource indicator information and cloud monitoring resource grouping information.

[0056] Furthermore, the configuration management platform supports the classification and division of different monitoring object resources and the dynamic registration of resource indicators. The registered information includes not only the monitoring category and indicator name, but also unique tags, PromQL templates, and aggregation cycles.

[0057] Furthermore, the resource indicators registered by the configuration management platform are periodically obtained. Specifically, the full information is read from the database of the configuration management platform, and then parsed and stored in the memory. That is, all registered cloud monitoring resource indicator information is obtained, and the cloud monitoring resource indicator information is parsed and stored in the locally loaded aggregation information list to ensure that the system can obtain the latest monitoring data in real time.

[0058] Furthermore, the cloud monitoring resource indicator information and resource grouping information are periodically obtained from the cache, and are parsed and stored in a local resource grouping information list for use in subsequent aggregation calculations.

[0059] Step S102 : dynamically classify and aggregate the parameters in the cloud monitoring resource grouping information according to the aggregation task elements to obtain an aggregate computing task.

[0060] Step S103: Execute the aggregation calculation task to obtain the cloud monitoring resource indicator aggregation result.

[0061] The dynamic classification and aggregation calculation method for cloud monitoring resource indicators provided in this embodiment can effectively organize different types of monitoring data through cloud monitoring resource grouping information, so that it can be more targeted when performing indicator aggregation, thereby improving the efficiency and accuracy of data analysis, and then flexibly adjusting the classification and aggregation strategies according to actual needs, dynamically classifying and aggregating the parameters in the cloud monitoring resource grouping information to obtain aggregation calculation tasks, ensuring real-time monitoring and aggregation calculation of cloud monitoring resource indicators, being able to flexibly respond to changing monitoring needs, reducing unnecessary computing burdens, and improving the analyzability and practicality of the monitoring system. It not only improves the efficiency of the monitoring system, but also enhances the real-time insight into resource usage, providing important support for enterprise decision-making.

[0062] In this embodiment, a dynamic classification and aggregation calculation method for cloud monitoring resource indicators is provided, which can be used for the above-mentioned electronic devices. Figure 2 Flowchart of a dynamic classification and aggregation calculation method for cloud monitoring resource indicators according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0063] Step S201: Obtain cloud monitoring resource indicator information and cloud monitoring resource group information, and generate aggregation task elements based on the cloud monitoring resource indicator information and cloud monitoring resource group information. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0064] Step S202 : dynamically classify and aggregate the parameters in the cloud monitoring resource grouping information according to the aggregation task elements to obtain an aggregate computing task.

[0065] Specifically, according to the aggregation task elements, classification aggregation is performed according to the parameters in the resource grouping information, and PromQL statements are dynamically spliced ​​to create an aggregation calculation task.

[0066] The above step S202 includes:

[0067] Step S2021, extracting parameters corresponding to the aggregation task elements; the parameters corresponding to the aggregation task elements include monitoring class name, resource indicator name, unique label, PromQL template and aggregation period.

[0068] Specifically, the task elements that need to be aggregated are updated in a timely manner according to the current monitoring needs and resource status to complete the parameter parsing; for example, when the monitoring indicators of a resource group change, the relevant aggregated task elements are automatically obtained, and then the relevant parameters in the aggregated task elements are extracted.

[0069] Step S2022: parse the parameters of the PromQL template and replace the parameters in the PromQL template with the parameters in the cloud monitoring resource grouping information.

[0070] Specifically, the parameters in the PromQL template are parsed, and the parameters in the PromQL template are matched and replaced according to the cloud monitoring resource grouping information.

[0071] Step S2023: extract the aggregation method in the PromQL template, and use the aggregation method to dynamically splice the parameters in the cloud monitoring resource grouping information to obtain a PromQL statement.

[0072] Specifically, the aggregation method in the PromQL template is extracted and parsed, and multiple PromQLs that need to be aggregated are generated according to different methods. The aggregation methods can include sum, average, maximum, etc., and aggregation by dimensions such as instance ID and region is also supported.

[0073] Among them, PromQL (Prometheus Query Language) is a query language used to process time series data. PromQL can express complex query logic through simple syntax, supports multiple arithmetic operators and time series operators, and most operations are performed on time series data sets (vectors). PromQL statements allow users to select and aggregate time series from time series data to implement monitoring indicator queries and alarm triggering.

[0074] Step S2024: Create an aggregation computing task based on the PromQL statement and the aggregation period.

[0075] In some optional implementations, the above step S2024 includes:

[0076] Step a1: Add the PromQL statement to the loaded aggregate information list, and determine whether the loaded aggregate information list is empty.

[0077] Specifically, during the initial aggregation, the aggregation calculation task is registered according to the configuration information, and all aggregation PromQL statements are loaded.

[0078] In step a2, if the loaded aggregation information list is empty, the PromQL statement is analyzed to generate an aggregation calculation task.

[0079] Specifically, if the loaded aggregation information list is empty, it means that no aggregation PromQL statements have been loaded. In this case, you need to analyze the PromQL statements to generate aggregation tasks that need to be processed.

[0080] Step a3: Alternatively, if the loaded aggregation information list is not empty, determine whether the PromQL statements of adjacent aggregation periods are the same; if the PromQL statements of adjacent aggregation periods are different, update the aggregation calculation task.

[0081] Specifically, if the loaded aggregation information list is not empty, it means that the aggregation PromQL statement has been loaded. Then you need to compare whether the PromQL statement has changed to update the aggregation calculation task regularly.

[0082] Step S203: Execute the aggregation calculation task to obtain the cloud monitoring resource indicator aggregation result.

[0083] Specifically, PromQL statements are executed to batch query the time series database to obtain the resource indicator aggregation results, and the results are pushed to the time series database after label expansion.

[0084] Furthermore, the tag extension is performed on the aggregation results of cloud monitoring resource indicators, and the tag-extended aggregation results of cloud monitoring resource indicators are stored in a time series database; wherein, tag extension refers to adding additional metadata information (tags) to the aggregation results to facilitate subsequent query and analysis. By extending tags, users can better identify and classify monitoring data.

[0085] The time series database uses Victoria-metrics, which provides data storage capabilities. The raw indicator data of different products (such as CPU usage, memory usage, etc.) is reported to Victoria-metrics for subsequent aggregation and analysis, and stores the aggregation results of cloud monitoring resource indicators. Victoria-metrics is an open source, high-performance time series database that is very suitable for applications that require fast response, high throughput, and scalability. Its low-latency query, high availability, and ease of use make it an ideal choice for monitoring, logging, and analyzing time series data.

[0086] The dynamic classification and aggregation calculation method for cloud monitoring resource indicators provided in this embodiment automatically generates PromQL statements according to different resource grouping information and multiple aggregation methods, which can meet diverse aggregation needs and enable users to adjust aggregation methods and configuration information at any time according to needs, thereby achieving low-latency query and efficient aggregation calculation, and improving the response speed and data processing capabilities of the monitoring system.

[0087] In this embodiment, a dynamic classification and aggregation calculation method for cloud monitoring resource indicators is provided, which can be used for the above-mentioned electronic devices. Figure 3 Flowchart of a dynamic classification and aggregation calculation method for cloud monitoring resource indicators according to an embodiment of the present invention. Figure 3As shown, the flow includes the following steps:

[0088] At step S301, cloud monitoring resource index information and cloud monitoring resource grouping information are acquired, and an aggregation task element is generated based on the cloud monitoring resource index information and the cloud monitoring resource grouping information. For details, please refer to Figure 2 At step S201 of the embodiment shown, no further elaboration is given here.

[0089] At step S302, according to the aggregation task element, parameters in the cloud monitoring resource grouping information are dynamically classified and aggregated to obtain an aggregation calculation task. For details, please refer to Figure 2 At step S202 of the embodiment shown, no further elaboration is given here.

[0090] At step S303, the aggregation calculation task is executed to obtain a cloud monitoring resource index aggregation result. For details, please refer to Figure 2 At step S203 of the embodiment shown, no further elaboration is given here.

[0091] At step S304, a timing task is acquired, and the aggregation task element and the aggregation calculation task are periodically updated based on the timing task.

[0092] Specifically, the above step S304 includes:

[0093] At step S3041, a dynamic aggregation calculation task template and a cloud monitoring resource grouping task template are created.

[0094] Specifically, a task template (Task Template) is created, including the name, description, and execution logic of the template, etc.

[0095] At step S3042, a timing parameter is acquired, and a timing task is constructed based on the timing parameter and the dynamic aggregation calculation task template or the cloud monitoring resource grouping task template.

[0096] Specifically, the timing parameter includes parameters such as task ID and timing period.

[0097] Further, the created timing task is stored, and each CronTask (timing task) instance includes the unique identifier of the task (i.e., task ID), timing rule, task template, and parameters, etc.

[0098] Further, the timing task is added to a task pool, which is used to manage the timing task, including creating a new CronTask instance, starting and stopping the timer.

[0099] At step S3043, the cloud monitoring resource index information and the cloud monitoring resource grouping information are periodically acquired based on the timing task to update the aggregation task element and the aggregation calculation task.

[0100] Furthermore, the scheduled tasks in the task pool are started to periodically obtain cloud monitoring resource indicator information and cloud monitoring resource grouping information, update the aggregation task elements, and at the same time, update the aggregation computing tasks according to the aggregation period.

[0101] The dynamic classification and aggregation calculation method of cloud monitoring resource indicators provided in this embodiment can periodically update cloud monitoring resource indicator information and cloud monitoring resource group information by regularly managing aggregation calculation tasks, ensuring that the aggregation calculation tasks are always kept up to date, thereby improving the real-time and reliability of the monitoring system.

[0102] The following describes a specific embodiment to illustrate the specific steps of a dynamic classification and aggregation calculation method for cloud monitoring resource indicators.

[0103] Example 1:

[0104] like Figure 4 As shown, a dynamic classification and aggregation calculation method for cloud monitoring resource indicators includes the following modules and steps:

[0105] Configuration management module: obtains resource indicator information from the configuration management platform, obtains resource grouping information from the cache, and generates task elements that need to be aggregated based on the indicator information and aggregation information.

[0106] Dynamic Aggregation: This module aggregates tasks based on the parameters in the resource grouping information and dynamically constructs PromQL statements to create aggregate calculation tasks. Aggregation methods include sum, average, and maximum, and supports aggregation by dimensions such as instance ID and region.

[0107] Timing management module: Periodically obtains resource indicator information and resource grouping information, updates aggregation task elements, and updates aggregation computing tasks according to the aggregation period.

[0108] Data storage module: The victoria-metrics time series database provides data storage capabilities. The raw indicator data of different products is reported to victoria-metrics, and the aggregation results are also stored in it.

[0109] In the above technical solution, the specific implementation process of the configuration management module is as follows:

[0110] Step 1: The timing management module periodically obtains resource indicators registered by the configuration management platform. Specifically, it reads the full information from the database of the configuration management platform, and then parses and stores it in memory.

[0111] An example of a PromQL template for resource indicators is as follows:

[0112] {

[0113] #method#by(idc,job)(cpu_util{idc="#idc#",job="virtual_machine",uuid="~"

[0114] #uuid#"}); max, min, avg, sum

[0115] }

[0116] The structure used to store aggregated PromQL information is as follows:

[0117]

[0118] Among them, MonitorClassRule is the monitoring class, MonitorClassRule.Name is the monitoring class name, and MetricRule is the resource indicator information collection, including the aggregation PromQL, indicator name, and aggregation period.

[0119] Step 2: The timing management module periodically obtains the resource grouping information in the cache, then parses and stores it in the memory. This step ensures that the system can update the resource grouping information in a timely manner so that the latest grouping standards can be used in subsequent calculations.

[0120] In the above technical solution, the timing management module includes:

[0121] Scheduled task pool: used to manage scheduled tasks, including creating new Cron instances, starting and stopping Cron timers; the task pool is defined as follows:

[0122]

[0123]

[0124] Task list: used to store the created scheduled tasks. Each CronTask instance includes the task's unique identifier, timing rules, task template, and parameters.

[0125] Task template: used to create TaskTemplate task templates, including the template name, description, and execution logic. Task templates can be defined as:

[0126]

[0127] In the above technical solution, the specific implementation process of the timing management module is as follows:

[0128] Step 1: Create an aggregate computing task template; Example of creating a dynamic aggregate computing task template:

[0129]

[0130] Step 2: Create a scheduled task with parameters such as task ID, calculation period, calculation task template, monitoring item rules, etc.

[0131] Step 3: Add the scheduled task to the task pool, which can be expressed as:

[0132]

[0133]

[0134] In the above technical solution, if Figure 5 As shown in the figure, the dynamic aggregation module has the following specific implementation process:

[0135] Step 1: Update the task elements that need to be aggregated based on the current monitoring requirements and resource status, and complete parameter parsing. For example, when the monitoring indicators of a resource group change, the relevant task elements are automatically obtained. Among them, obtaining task elements is the basic step, and resource indicator information and resource grouping parameters are the key parameters that constitute PromQL.

[0136] Step 2: Based on the task information obtained in step 1, it can be dynamically spliced ​​into the required PromQL according to different categories and methods.

[0137] Step 3: Create an aggregate computing task based on the task elements obtained in steps 1 and 2, and then add it to the task pool.

[0138]

[0139] Step 4: Execute PromQL batch query on the time series database to obtain the resource indicator aggregation results, expand the labels on the results, and push them to the time series database.

[0140] Resource indicator query TSDB:

[0141]

[0142]

[0143] Expand the labels of the query results and construct the final data structure to be stored:

[0144]

[0145] for lname,lval:=range sample.Metric{

[0146] if string(lname)!= "idc"{

[0147] continue

[0148] }

[0149] labels=append(labels,prompb.Label{Name:string(lname),Value:string(lval)})

[0150] }

[0151] samples:=[]prompb.Sample{{Value:float64(sample.Value),Timestamp:now.Unix()*1e3}}

[0152] ts:=prompb.TimeSeries{Labels:labels,Samples:samples}

[0153] tsList=append(tsList,ts)

[0154] }

[0155] In the above technical solution, the specific implementation process of step 2 of the dynamic aggregation module is as follows:

[0156] Step 2.1: Extract task element information, including monitoring class name, resource indicator name, unique label, PromQL template, and aggregation period.

[0157] Step 2.2: Parse the parameters in the PromQL template and perform matching and replacement based on the resource grouping information. Then extract and parse the aggregation method in the PromQL template and generate multiple PromQLs that need to be aggregated according to different methods.

[0158] In the above technical solution, the specific implementation process of step 3 of the dynamic aggregation module, creating an aggregate computing task, is as follows:

[0159] Step 3.1: When the aggregation module is first loaded, it is necessary to register the aggregation task according to the configuration information and load all the aggregation PromQL.

[0160] Step 3.2: If the loaded aggregation information list is empty, it means that no aggregation PromQL has been loaded. In this case, you need to analyze the PromQL to generate the aggregation tasks that need to be processed.

[0161] Step 3.3: If the loaded aggregation information list is not empty, it means that the aggregation PromQL has been loaded. You need to compare whether the PromQL has changed and update the aggregation task regularly.

[0162] The advantages and beneficial effects of the above embodiments are:

[0163] (1) Grouping and classification aggregation of monitoring data: Classify monitoring data according to different resource groups and encapsulate them into flexible dynamic resource indicators, so that indicator aggregation can be more targeted and reduce unnecessary computing burden.

[0164] (2) Dynamic configuration of parameters and timing management: By dynamically configuring aggregation parameters, the aggregation strategy can be flexibly adjusted according to actual needs, avoiding the rigidity problem caused by fixed parameters, so that the system can better adapt to rapidly changing monitoring needs; at the same time, by timing the management of aggregation tasks, resource indicators and resource group information can be periodically updated to ensure that the aggregation tasks are always kept up to date, thereby improving the real-time performance and reliability of the monitoring system.

[0165] (3) Dynamic classification and aggregation calculation can be realized: PromQL statements are automatically generated according to different resource grouping information and multiple aggregation methods to meet diverse aggregation needs. This mechanism enables users to adjust the aggregation method and configuration information at any time according to needs, achieving low-latency query and efficient aggregation calculation, and improving the response speed and data processing capabilities of the monitoring system.

[0166] In this embodiment, a dynamic classification and aggregation computing device for cloud monitoring resource indicators is also provided, which is used to implement the above-mentioned embodiments and preferred implementation methods. The details that have been explained will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and conceivable.

[0167] This embodiment provides a dynamic classification and aggregation computing device for cloud monitoring resource indicators, such as Figure 6 Shown, including:

[0168] Configuration management module 601, used to obtain cloud monitoring resource indicator information and cloud monitoring resource grouping information, and generate aggregation task elements based on the cloud monitoring resource indicator information and cloud monitoring resource grouping information;

[0169] Dynamic aggregation module 602, configured to dynamically classify and aggregate the parameters in the cloud monitoring resource grouping information according to the aggregation task elements to obtain an aggregated computing task;

[0170] The execution module 603 is used to execute the aggregation computing task and obtain the cloud monitoring resource indicator aggregation result.

[0171] In some optional implementations, the dynamic aggregation module 602 includes:

[0172] The extraction unit is used to extract the parameters corresponding to the aggregation task elements; the parameters corresponding to the aggregation task elements include the monitoring class name, resource indicator name, unique label, PromQL template and aggregation period;

[0173] The parsing unit is used to parse the parameters of the PromQL template and replace the parameters in the PromQL template with the parameters in the cloud monitoring resource grouping information;

[0174] A splicing unit is used to extract the aggregation method in the PromQL template and dynamically splice the parameters in the cloud monitoring resource grouping information using the aggregation method to obtain a PromQL statement;

[0175] The first creation unit is used to create an aggregation calculation task based on a PromQL statement and an aggregation period.

[0176] In some optional implementations, the creating unit includes:

[0177] Add a subunit to add the PromQL statement to the loaded aggregate information list and determine whether the loaded aggregate information list is empty;

[0178] The generation subunit is used to analyze the PromQL statement and generate the aggregation calculation task if the loaded aggregation information list is empty;

[0179] The judgment subunit is used to determine whether the PromQL statements of adjacent aggregation cycles are the same if the loaded aggregation information list is not empty. If the PromQL statements of adjacent aggregation cycles are different, the aggregation calculation task is updated.

[0180] In some optional embodiments, the method further includes:

[0181] The tag extension module is used to perform tag extension on the aggregation results of cloud monitoring resource indicators and store the tag-extended aggregation results of cloud monitoring resource indicators in a time series database.

[0182] In some optional embodiments, the method further includes:

[0183] The scheduled update module is used to obtain scheduled tasks and periodically update the aggregation task elements and aggregation calculation tasks based on the scheduled tasks.

[0184] In some optional implementations, the timing update module includes:

[0185] The second creation unit is used to create a dynamic aggregation computing task template and a cloud monitoring resource grouping task template;

[0186] A construction unit is used to obtain timing parameters and construct a scheduled task based on the timing parameters and a dynamic aggregation computing task template or a cloud monitoring resource grouping task template;

[0187] The acquisition unit is used to periodically acquire cloud monitoring resource indicator information and cloud monitoring resource grouping information based on the scheduled task to update the aggregation task elements and aggregation computing tasks.

[0188] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0189] In this embodiment, a dynamic classification and aggregation computing device for cloud monitoring resource indicators is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0190] The embodiment of the present invention also provides a computer device having the above Figure 6 A dynamic classification and aggregation computing device for cloud monitoring resource indicators is shown.

[0191] See also Figure 7 , Figure 7 is a structural diagram of a computer device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of a GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0192] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0193] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0194] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0195] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0196] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 7 The bus connection is taken as an example.

[0197] The input device 30 can receive input digital or character information and generate key signal input related to user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display, and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0198] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0199] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0200] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A dynamic classification and aggregation calculation method for cloud monitoring resource indicators, characterized in that: The method comprises: Obtaining cloud monitoring resource indicator information and cloud monitoring resource grouping information, and generating aggregation task elements based on the cloud monitoring resource indicator information and the cloud monitoring resource grouping information; According to the aggregation task elements, the parameters in the cloud monitoring resource grouping information are dynamically classified and aggregated to obtain an aggregate computing task; Execute the aggregation calculation task to obtain the cloud monitoring resource indicator aggregation result.

2. The method according to claim 1, characterized in that The step of dynamically classifying and aggregating the parameters in the cloud monitoring resource grouping information according to the aggregation task elements to obtain an aggregate computing task includes: Extract the parameters corresponding to the aggregation task elements; the parameters corresponding to the aggregation task elements include monitoring class name, resource indicator name, unique label, PromQL template and aggregation period; Perform parameter parsing on the PromQL template, and replace the parameters in the PromQL template with the parameters in the cloud monitoring resource grouping information; Extracting the aggregation method in the PromQL template, and dynamically splicing the parameters in the cloud monitoring resource grouping information using the aggregation method to obtain a PromQL statement; The aggregation computing task is created based on the PromQL statement and the aggregation period.

3. The method according to claim 2, characterized in that The creating the aggregate computing task based on the PromQL statement and the aggregation period includes: Add the PromQL statement to the loaded aggregate information list, and determine whether the loaded aggregate information list is empty; If the loaded aggregation information list is empty, the PromQL statement is analyzed to generate the aggregation calculation task; Alternatively, if the loaded aggregation information list is not empty, it is determined whether the PromQL statements of adjacent aggregation cycles are the same; if the PromQL statements of adjacent aggregation cycles are different, the aggregation calculation task is updated.

4. The method according to claim 1, wherein Also includes: Perform tag expansion on the cloud monitoring resource indicator aggregation result, and store the cloud monitoring resource indicator aggregation result after tag expansion in a time series database.

5. The method according to claim 1, wherein Also includes: Obtain a scheduled task, and periodically update the aggregation task element and the aggregation computing task based on the scheduled task.

6. The method according to claim 5, characterized in that The periodically updating the aggregate task elements and the aggregate computing tasks based on the scheduled tasks includes: Create dynamic aggregate computing task templates and cloud monitoring resource grouping task templates; Acquire timing parameters, and construct the scheduled task based on the timing parameters and the dynamic aggregate computing task template or the cloud monitoring resource grouping task template; The cloud monitoring resource indicator information and the cloud monitoring resource grouping information are periodically obtained based on the scheduled task to update the aggregation task elements and the aggregation computing task.

7. A dynamic classification and aggregation computing device for cloud monitoring resource indicators, characterized in that: The device comprises: A configuration management module is used to obtain cloud monitoring resource indicator information and cloud monitoring resource grouping information, and generate aggregation task elements based on the cloud monitoring resource indicator information and the cloud monitoring resource grouping information; A dynamic aggregation module, configured to dynamically classify and aggregate the parameters in the cloud monitoring resource grouping information according to the aggregation task elements to obtain an aggregated computing task; The execution module is used to execute the aggregation calculation task and obtain the cloud monitoring resource indicator aggregation result.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the dynamic classification and aggregation calculation method for cloud monitoring resource indicators according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the dynamic classification and aggregation calculation method for cloud monitoring resource indicators according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the dynamic classification and aggregation calculation method for cloud monitoring resource indicators according to any one of claims 1 to 6.

Images