A monitoring system and method

Abstract

The present application provides a monitoring system and method, which collect a resource monitoring data set, determine an initial data selection parameter according to the resource monitoring data set, and select a first resource monitoring data subset through the initial data selection parameter; determine the squared correlation distance corresponding to each resource monitoring data point in the resource monitoring data set according to the first resource monitoring data subset; determine a set of correlation distance deviation values according to the squared correlation distance, obtain the squared correlation distance corresponding to the minimum correlation distance deviation value, and further determine the resource monitoring data selection parameter; select a second resource monitoring data subset according to the resource monitoring data selection parameter, and determine the resource monitoring data correlation distance corresponding to each resource monitoring data point in the resource monitoring data set through the second resource monitoring data subset; issue an alarm when the resource monitoring data correlation distance is higher than a preset critical value, so as to solve the technical problem of low alarm accuracy rate for abnormal resource usage.

Description

Technical Field

[0001] This application relates to the technical field of resource usage monitoring. More specifically, this application relates to a monitoring system and method. Background Art

[0002] Prometheus is an open-source system and service monitoring tool that is widely used to collect, store, and query metric data of various applications and systems. Using the Prometheus monitoring system, resource usage, performance metrics, application status, etc. can be monitored, and real-time alerts for abnormal situations can be issued.

[0003] The method for monitoring resource usage based on Prometheus includes steps such as installing and configuring the Prometheus monitoring system, importing Exporters, setting monitoring metrics, configuring alert rules, setting dashboards, and data storage. Through this monitoring method, the resource usage of the system and applications can be grasped in real time, abnormalities can be detected in a timely manner, and troubleshooting and performance optimization can be carried out. In the Prometheus monitoring system, administrators can set monitoring metrics, alert rules, and data storage policies according to actual needs to achieve high customization. By setting alert rules, the system can automatically monitor whether the metric values exceed the set thresholds and trigger alert notifications in a timely manner when abnormal situations occur, ensuring that key issues can be resolved in a timely manner and guaranteeing the stability and reliability of the Prometheus monitoring system. The Prometheus monitoring system also provides a powerful data visualization function. By displaying the real-time change trends of monitoring metrics through dashboards, administrators can quickly identify performance bottlenecks and abnormal behaviors. However, in the prior art, when analyzing the abnormal usage of resource monitoring data through the Prometheus monitoring system and issuing abnormal alerts based on the results of resource abnormal analysis, there is a technical problem of low accuracy in resource abnormal usage alerts. Summary of the Invention

[0004] This application provides a monitoring system and method to solve the technical problem of low accuracy in resource abnormal usage alerts.

[0005] To solve the above technical problem, this application adopts the following technical solutions:

[0006] In a first aspect, this application provides a monitoring method, including the following steps:

[0007] Collect resource monitoring data through the Prometheus monitoring system to obtain a resource monitoring data set, determine initial data selection parameters according to the resource monitoring data set, and select a first subset of resource monitoring data through the initial data selection parameters;

[0008] Determine the squared correlation distance corresponding to each resource monitoring data point in the resource monitoring data set according to the first subset of resource monitoring data;

[0009] Determine a set of relevant distance deviation values based on the squared relevant distances corresponding to each resource monitoring data point, and then determine the squared relevant distance corresponding to the minimum relevant distance deviation value. Determine the resource monitoring data selection parameter according to the squared relevant distance corresponding to the minimum relevant distance deviation value;

[0010] Select a second subset of resource monitoring data according to the resource monitoring data selection parameter, and determine the relevant distance of the resource monitoring data corresponding to each resource monitoring data point in the resource monitoring data set through the second subset of resource monitoring data;

[0011] Take the resource monitoring data points with relevant distances of resource monitoring data higher than the preset critical value as resource usage anomaly information for warning.

[0012] In some embodiments, collect resource monitoring data through the Prometheus monitoring system, and the obtained resource monitoring data set specifically includes:

[0013] Determine the median value of the resource monitoring data collected through the Prometheus monitoring system;

[0014] Fill in the missing values of the collected resource monitoring data according to the median value of the resource monitoring data to obtain a resource monitoring data set.

[0015] In some embodiments, determining the initial data selection parameter according to the resource monitoring data set specifically includes:

[0016] Determine the total amount of data of the resource monitoring data points in the resource monitoring data set;

[0017] Determine the data dimension of the resource monitoring data points in the resource monitoring data set;

[0018] Determine the initial data selection parameter according to the total amount of data of the resource monitoring data points and the data dimension of the resource monitoring data points in the resource monitoring data set, wherein the initial data selection parameter is determined according to the following formula:

[0019]

[0020] Wherein, represents the initial data selection parameter, represents the resource monitoring data set is the total amount of data of the resource monitoring data points in represents the resource monitoring data set is the data dimension of the resource monitoring data points in represents the resource monitoring data set.

[0021] In some embodiments, determining the squared correlation distance corresponding to each resource monitoring data point in the resource monitoring dataset according to the first subset of resource monitoring data specifically includes:

[0022] Determining the data mean vector and the data covariance matrix of the first subset of resource monitoring data;

[0023] Taking the data mean vector as the data position parameter of the first subset of resource monitoring data;

[0024] Taking the data covariance matrix as the data scale parameter of the first subset of resource monitoring data;

[0025] Determining the squared correlation distance corresponding to each resource monitoring data point in the resource monitoring dataset according to the data position parameter and the data scale parameter of the first subset of resource monitoring data, wherein the squared correlation distance is determined according to the following formula;

[0026]

[0027] Wherein, represents the squared correlation distance corresponding to the resource monitoring data point corresponding thereto, represents the th resource monitoring data point in the resource monitoring dataset, represents the first subset of resource monitoring data data position parameter, represents the first subset of resource monitoring data data scale parameter, represents the transpose operation.

[0028] In some embodiments, determining the squared correlation distance corresponding to the minimum correlation distance deviation value and determining the resource monitoring data selection parameter according to the squared correlation distance corresponding to the minimum correlation distance deviation value specifically include:

[0029] Sorting the squared correlation distances of all resource monitoring data to obtain a sequence of squared correlation distances;

[0030] Determining the resource monitoring data point corresponding to the minimum correlation distance deviation value, and further determining the sequence position of the squared correlation distance corresponding to the resource monitoring data point in the sequence of squared correlation distances;

[0031] Determining the resource monitoring data selection parameter according to the sequence position and the initial data selection parameter.

[0032] In some embodiments, selecting the second subset of resource monitoring data according to the resource monitoring data selection parameter specifically includes:

[0033] Select a number of resource monitoring data points in the resource monitoring data set that is equal to the selected parameter value of the resource monitoring data;

[0034] Combine the resource monitoring data points into a second resource monitoring data subset in chronological order.

[0035] In some embodiments, the preset critical value is determined according to the selected parameter of the resource monitoring data.

[0036] In a second aspect, the present application provides a monitoring system, including a resource anomaly warning unit, and the resource anomaly warning unit includes:

[0037] A first resource monitoring data subset selection module, configured to collect resource monitoring data through a Prometheus monitoring system to obtain a resource monitoring data set, determine an initial data selection parameter according to the resource monitoring data set, and select a first resource monitoring data subset through the initial data selection parameter;

[0038] A correlation distance square determination module, configured to determine the correlation distance square corresponding to each resource monitoring data point in the resource monitoring data set according to the first resource monitoring data subset;

[0039] A resource monitoring data selection parameter determination module, configured to determine a correlation distance deviation value set according to the correlation distance square corresponding to each resource monitoring data point, and further determine the correlation distance square corresponding to the smallest correlation distance deviation value, and determine the resource monitoring data selection parameter according to the corresponding correlation distance square;

[0040] A resource monitoring data correlation distance determination module, configured to select a second resource monitoring data subset according to the resource monitoring data selection parameter, and determine the resource monitoring data correlation distance corresponding to each resource monitoring data point in the resource monitoring data set through the second resource monitoring data subset;

[0041] A resource anomaly warning module, configured to use the resource monitoring data points with the resource monitoring data correlation distance higher than the preset critical value as resource usage anomaly information for warning.

[0042] In a third aspect, the present application provides a computer device, which includes a memory and a processor. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the above-mentioned monitoring method.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions or codes are stored. When the instructions or codes are run on a computer, the computer is enabled to execute the above-mentioned monitoring method.

[0044] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0045] In a monitoring system and method provided by this application, Prometheus monitoring system is used to collect resource monitoring data to obtain a resource monitoring data set. Initial data selection parameters are determined according to the resource monitoring data set, and a first subset of resource monitoring data is selected through the initial data selection parameters; the squared correlation distance corresponding to each resource monitoring data point in the resource monitoring data set is determined according to the first subset of resource monitoring data; a set of correlation distance deviation values is determined according to the squared correlation distance corresponding to each resource monitoring data point, and then the squared correlation distance corresponding to the minimum correlation distance deviation value is determined. Resource monitoring data selection parameters are determined according to the corresponding squared correlation distance; a second subset of resource monitoring data is selected according to the resource monitoring data selection parameters, and the resource monitoring data correlation distance corresponding to each resource monitoring data point in the resource monitoring data set is determined through the second subset of resource monitoring data; resource monitoring data points with resource monitoring data correlation distances higher than a preset critical value are used as resource usage abnormal information for warning.

[0046] In this application, first, by filling in missing values, the influence of missing values in the resource monitoring data set on resource monitoring data analysis is reduced. Second, determining the squared correlation distance corresponding to each resource monitoring data point in the resource monitoring data set helps to identify abnormal resource usage situations. Then, by determining the correlation distance deviation value, the influence of abnormal data on overall data analysis can be reduced, and the accuracy of resource monitoring data analysis can be improved. Finally, by determining and comparing the resource monitoring data correlation distances, abnormal resource monitoring data points can be detected and warned, so as to solve the technical problem of low accuracy rate of resource abnormal usage warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0048] Figure 1 is an exemplary flowchart of a monitoring method shown in some embodiments of this application;

[0049] Figure 2 is a schematic diagram of exemplary hardware and / or software of a resource anomaly warning unit shown in some embodiments of this application;

[0050] Figure 3It is a schematic structural diagram of a computer device for implementing a monitoring method shown in some embodiments of the present application. Detailed implementation manners

[0051] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0052] The embodiments of the present application provide a monitoring system and method. The core is to collect resource monitoring data through the Prometheus monitoring system to obtain a resource monitoring data set, determine initial data selection parameters according to the resource monitoring data set, and select a first resource monitoring data subset through the initial data selection parameters; determine the squared correlation distance corresponding to each resource monitoring data point in the resource monitoring data set according to the first resource monitoring data subset; determine a set of correlation distance deviation values according to the squared correlation distance corresponding to each resource monitoring data point, and then determine the squared correlation distance corresponding to the minimum correlation distance deviation value. Determine resource monitoring data selection parameters according to the corresponding squared correlation distance; select a second resource monitoring data subset according to the resource monitoring data selection parameters, and determine the resource monitoring data correlation distance corresponding to each resource monitoring data point in the resource monitoring data set through the second resource monitoring data subset; use the resource monitoring data points with the resource monitoring data correlation distance higher than the preset critical value as resource usage abnormal information for warning, so as to solve the technical problem of low accuracy of resource abnormal usage warning.

[0053] To better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners. Refer to Figure 1 , this figure is an exemplary flowchart of a monitoring method shown in some embodiments of the present application. The monitoring method 100 mainly includes the following steps:

[0054] In step 101, collect resource monitoring data through the Prometheus monitoring system to obtain a resource monitoring data set, determine initial data selection parameters according to the resource monitoring data set, and select a first resource monitoring data subset through the initial data selection parameters.

[0055] It should be noted that the Prometheus monitoring system itself is an open-source monitoring system that can collect and store resource monitoring data, and then obtain a resource monitoring data set. When collecting resource monitoring data, it is first necessary to import an Exporter or an Agent. An Exporter is a process or program for collecting resource monitoring data, and an Agent is a proxy program responsible for collecting and sending resource monitoring data to the Prometheus monitoring system. In this application, the resource monitoring data set is a multi-dimensional data set containing a series of resource monitoring data points sorted by time. Among them, each resource monitoring data point contains monitoring metric features such as CPU usage, memory occupancy, or disk read / write.

[0056] In some embodiments, collecting resource monitoring data through the Prometheus monitoring system to obtain a resource monitoring data set can be specifically implemented in the following manner, that is:

[0057] Determine the median value of the resource monitoring data collected through the Prometheus monitoring system;

[0058] Fill in the missing values of the collected resource monitoring data according to the median value of the resource monitoring data to obtain a resource monitoring data set.

[0059] Specifically, when implementing, determine the median value of all resource monitoring data. In this application, the median value of the data is the middle value after sorting the resource monitoring data by value. In the resource monitoring data set, there may be missing values, that is, there is no resource monitoring data at some time points. Use the determined median value of the data to fill in the missing resource monitoring data, that is, set the resource monitoring data value at the missing point to the median value of the data. After completing the calculation of the median value of the data and filling in the missing values, a complete resource monitoring data set can be obtained. Through the above method, the impact of missing data on the subsequent analysis of resource usage can be reduced.

[0060] In some embodiments, determining the initial data selection parameters according to the resource monitoring data set can be specifically implemented in the following manner, that is:

[0061] Determine the total amount of data of the resource monitoring data points in the resource monitoring data set;

[0062] Determine the data dimension of the resource monitoring data points in the resource monitoring data set;

[0063] Determine the initial data selection parameters according to the total amount of data of the resource monitoring data points and the data dimension of the resource monitoring data points in the resource monitoring data set. Specifically, when implementing, the initial data selection parameters can be determined according to the following formula:

[0064]

[0065] Wherein, Represents the initial data selection parameter, Represents the resource monitoring data set The total amount of data of the resource monitoring data points in Represents the resource monitoring data set The data dimension of the resource monitoring data points in Represents the resource monitoring data set. It should be noted that the size of the initial data selection parameter value determines the number of resource monitoring data points selected from the resource monitoring data set, that is, the value of the initial data selection parameter is the number of selected resource monitoring data points, and the data dimension of the resource monitoring data points represents the number of monitoring index features included in each resource monitoring data point.

[0066] In some embodiments, a first resource monitoring data subset is selected through the initial data selection parameter. After determining the initial data selection parameter according to the above method, the same number of resource monitoring data points as the value of the initial data selection parameter are selected from the resource monitoring data set, and these resource monitoring data points are combined into a first resource monitoring data subset in chronological order.

[0067] It should be noted that by filling in the missing values, the impact of the missing values in the resource monitoring data set on the resource monitoring data analysis is reduced, ensuring the continuity of the resource monitoring data. After selecting the first resource monitoring data subset, more efficient resource monitoring data analysis can be performed.

[0068] In step 102, the squared correlation distance corresponding to each resource monitoring data point in the resource monitoring data set is determined according to the first resource monitoring data subset.

[0069] In some embodiments, the squared correlation distance corresponding to each resource monitoring data point in the resource monitoring data set can be specifically determined by the following method according to the first resource monitoring data subset, that is:

[0070] Determine the data mean vector and data covariance matrix of the first resource monitoring data subset;

[0071] Take the data mean vector as the data position parameter of the first resource monitoring data subset;

[0072] Take the data covariance matrix as the data scale parameter of the first resource monitoring data subset;

[0073] Determine the squared correlation distance corresponding to each resource monitoring data point in the resource monitoring data set according to the data position parameter and data scale parameter of the first resource monitoring data subset. Specifically, when implemented, the squared correlation distance can be determined according to the following formula;

[0074]

[0075] Wherein, Represents a resource monitoring data point The square of the relevant distance Represents the th resource monitoring data point in the resource monitoring dataset Represents the data position parameter of the first resource monitoring data subset Represents the first resource monitoring data subset The data scale parameter Represents a transpose operation. It should be noted that the square of the relevant distance represents the square of the relevant distance, and the relevant distance is an index for measuring the difference between the resource monitoring data points in the resource monitoring dataset and the first resource monitoring data subset. The smaller the relevant distance, the smaller the difference degree between the resource monitoring data points in the resource monitoring dataset and the first resource monitoring data subset, and the larger the relevant distance, the larger the difference degree between the resource monitoring data points in the resource monitoring dataset and the first resource monitoring data subset.

[0076] In the above embodiment, for the first resource monitoring data subset, calculate the data mean value of each monitoring index feature and form a data mean value vector. The data mean value vector contains the average value of each monitoring index feature in the first resource monitoring data subset. Calculate the data covariance matrix of the first resource monitoring data subset according to the data mean value vector. Regard the calculated data mean value vector as the data position parameter of the first resource monitoring data subset. The data position parameter describes the overall position of the first resource monitoring data subset in terms of dimensions, that is, the average level of the monitoring index features of the first resource monitoring data subset. Regard the calculated data covariance matrix as the data scale parameter of the first resource monitoring data subset. The data scale parameter reflects the degree of diffusion of the first resource monitoring data subset in different dimensions, that is, the correlation between different monitoring index features in the first resource monitoring data subset.

[0077] It should be noted that by using the data position parameter and data scale parameter of the first resource monitoring data subset, the square of the relevant distance of each resource monitoring data point in the resource monitoring dataset can be determined, which helps to identify abnormal resource usage situations and take timely measures to avoid system failures.

[0078] In step 103, determine a set of relevant distance deviation values according to the square of the relevant distance corresponding to each resource monitoring data point, and then determine the square of the relevant distance corresponding to the smallest relevant distance deviation value. Determine the resource monitoring data selection parameter according to the corresponding square of the relevant distance.

[0079] It should be noted that after obtaining the square of the relevant distance of each resource monitoring data point, it is necessary to sort the squares of the relevant distances of all resource monitoring data points in ascending order to obtain a sequence of squares of relevant distances.​

[0080] In some embodiments, the relevant distance deviation value set may be determined according to the relevant squared distances corresponding to each resource monitoring data point in the following specific manner, that is:

[0081] For each resource monitoring data point, determine the mean of the relevant squared distances according to the relevant squared distances and the initial data selection parameter.

[0082] Obtain the initial data selection parameter.

[0083] Determine the variation degree of the relevant squared distances according to the mean of the relevant squared distances and the initial data selection parameter. Specifically, in implementation, the variation degree of the relevant squared distances may be determined according to the following formula:

[0084]

[0085] Where, represents the variation degree of the relevant squared distance corresponding to the th relevant squared distance in the relevant squared distance sequence, represents the relevant squared distance of the resource monitoring data point ; represents the th resource monitoring data point in the resource monitoring data set, represents the mean of the relevant squared distances corresponding to the th relevant squared distance in the relevant squared distance sequence, represents the initial data selection parameter;

[0086] Compare the variation degree of the relevant squared distances with a preset variation degree of the relevant squared distances to obtain the relevant distance deviation value of the resource monitoring data point, and further determine the relevant distance deviation value set.

[0087] It should be noted that after determining the variation degree of the relevant squared distances of a resource monitoring data point, subtract the preset variation degree of the relevant squared distances from the variation degree of the relevant squared distances to obtain a difference value, take the absolute value of the difference value, and use it as the relevant distance deviation value of the resource monitoring data point. Specifically, in implementation, the preset variation degree of the relevant squared distances is generally determined according to the dimension of the resource monitoring data. For example, assuming that the dimension of the resource monitoring data is , then the value of the preset variation degree of the relevant squared distances may be . After obtaining the relevant distance deviation values corresponding to each resource monitoring data in the resource monitoring data set through the above method, combine all the relevant distance deviation values together in chronological order to form the relevant distance deviation value set.

[0088] In addition, it should be noted that by calculating the square of the relevant distance and determining the relevant distance deviation value, it can help optimize the selection of parameters for resource monitoring data. The relevant distance deviation value can reflect the relative deviation degree between each resource monitoring data point in the resource monitoring data set and the first resource monitoring data subset.

[0089] In some embodiments, further determine the square of the relevant distance corresponding to the minimum relevant distance deviation value. The specific method for determining the parameter selected for resource monitoring data according to the corresponding square of the relevant distance can be as follows:

[0090] Sort the squares of the relevant distances of all resource monitoring data to obtain a sequence of squares of relevant distances;

[0091] Determine the resource monitoring data point corresponding to the minimum relevant distance deviation value, and then determine the sequence position of the square of the relevant distance of the resource monitoring data point in the sequence of squares of relevant distances;

[0092] Determine the parameter selected for resource monitoring data according to the sequence position and the initial data selection parameter. When specifically implemented, the parameter selected for resource monitoring data can be determined according to the following formula:

[0093]

[0094] Where, represents the parameter selected for resource monitoring data, represents the initial data selection parameter, represents the sequence position of the square of the relevant distance in the sequence of squares of relevant distances.

[0095] It should be noted that by determining the relevant distance deviation value, the influence of abnormal data on the overall data analysis can be reduced. By considering the relevant distance deviation value, abnormal data can be better filtered, and the stability and reliability of resource monitoring data analysis can be improved. By determining the parameter selected for resource monitoring data through the relevant distance deviation value, unnecessary calculation and storage overhead can be reduced, and the accuracy of resource monitoring data analysis can be improved.

[0096] In step 104, select a second resource monitoring data subset according to the parameter selected for resource monitoring data, and determine the relevant distance of the resource monitoring data corresponding to each resource monitoring data point in the resource monitoring data set through the second resource monitoring data subset.

[0097] In some embodiments, the specific method for selecting a second resource monitoring data subset according to the parameter selected for resource monitoring data can be as follows:

[0098] Select a number of resource monitoring data points in the resource monitoring data set that is equal to the value of the parameter selected for resource monitoring data;

[0099] Combine the resource monitoring data points into a second resource monitoring data subset in chronological order.

[0100] It should be noted that by the above steps, determining the second resource monitoring data subset can reduce the scale of the data set, thereby reducing the complexity of resource monitoring data processing and calculation.

[0101] In some embodiments, to determine the resource monitoring data correlation distance corresponding to each resource monitoring data point in the resource monitoring data set through the second resource monitoring data subset, the following method can be specifically adopted, that is:

[0102] Determine the data mean vector and data covariance matrix of the second resource monitoring data subset;

[0103] Determine the resource monitoring data correlation distance corresponding to each resource monitoring data point in the resource monitoring data set according to the data mean vector and data covariance matrix of the second resource monitoring data subset. Specifically, when implemented, the resource monitoring data correlation distance can be determined according to the following formula;

[0104]

[0105] where, represents the resource monitoring data correlation distance corresponding to the resource monitoring data point represents the th resource monitoring data point in the resource monitoring data set, represents the second resource monitoring data subset data mean vector, represents the second resource monitoring data subset data covariance matrix, represents the transpose operation. It should be noted that the resource monitoring data correlation distance is an index used to measure the difference between the resource monitoring data points in the resource monitoring data set and the second resource monitoring data subset. The smaller the resource monitoring data correlation distance, the smaller the difference degree between the resource monitoring data points in the resource monitoring data set and the second resource monitoring data subset. The larger the resource monitoring data correlation distance, the larger the difference degree between the resource monitoring data points in the resource monitoring data set and the second resource monitoring data subset. And the larger the resource monitoring data correlation distance, the greater the possibility that the resource monitoring data point is abnormal.

[0106] In addition, it should be noted that by determining the resource monitoring data correlation distance, the similarity and difference between the resource monitoring data points in the resource monitoring data set and the second resource monitoring data subset can be found, which helps to detect abnormal resource monitoring data points.

[0107] ​In step 105, the resource monitoring data points with the resource monitoring data - related distance higher than the preset critical value are used as resource usage abnormal information for alarming.

[0108] In some embodiments, the resource monitoring data points with the resource monitoring data - related distance higher than the preset critical value are used as resource usage abnormal information for alarming. When specifically implemented, according to the preset critical value, it is determined which resource monitoring data points have a resource monitoring data - related distance higher than the preset critical value. For the resource monitoring data points with a resource monitoring data - related distance higher than the preset critical value, they are regarded as resource usage anomalies and an alarm notification is triggered. The alarm notification should include specific information about the anomaly, such as the timestamp of the abnormal resource monitoring data point, the corresponding monitoring metric notification, and the specific value, so that the operation and maintenance personnel can quickly locate the problem.

[0109] In some embodiments, the preset critical value is determined according to the resource monitoring data selection parameter. It should be noted that the resource monitoring data - related distance follows a chi - square distribution with the degree of freedom being the value of the resource monitoring data selection parameter That is , usually assuming the confidence level to be 97.5%, the preset critical value can be obtained as When the resource monitoring data - related distance of the resource monitoring data point is higher than the preset critical value, that is , then it is determined that the resource monitoring data point is an abnormal resource monitoring data point, and this resource monitoring data point is used as resource usage abnormal information for alarming, so as to improve the accuracy of resource abnormal usage alarming.

[0110] In this application, first, by filling in the missing values, the influence of the missing values in the resource monitoring data set on the resource monitoring data analysis is reduced. Secondly, determining the square of the correlation distance corresponding to each resource monitoring data point in the resource monitoring data set helps to identify abnormal resource usage situations. Then, by determining the correlation distance deviation value, the influence of abnormal data on the overall data analysis can be reduced and the accuracy of the resource monitoring data analysis can be improved. Finally, by determining and comparing the resource monitoring data - related distances, abnormal resource monitoring data points can be detected and alarmed to solve the technical problem of low accuracy of resource abnormal usage alarming.

[0111] In addition, on the other hand of this application, in some embodiments, this application provides a monitoring system, which also includes a resource abnormal alarm unit. Refer to Figure 2, This figure is a schematic diagram of the exemplary hardware and / or software of the resource anomaly warning unit shown in some embodiments of the present application. The resource anomaly warning unit 200 includes: a first resource monitoring data subset selection module 201, a correlation distance square determination module 202, a resource monitoring data selection parameter determination module 203, a resource monitoring data correlation distance determination module 204, and a resource anomaly warning module 205, which are described as follows:

[0112] The first resource monitoring data subset selection module 201. In the present application, the first resource monitoring data subset selection module 201 is mainly used to collect resource monitoring data through the Prometheus monitoring system to obtain a resource monitoring data set, determine an initial data selection parameter according to the resource monitoring data set, and select a first resource monitoring data subset through the initial data selection parameter;

[0113] The correlation distance square determination module 202. In the present application, the correlation distance square determination module 202 is mainly used to determine the correlation distance square corresponding to each resource monitoring data point in the resource monitoring data set according to the first resource monitoring data subset;

[0114] The resource monitoring data selection parameter determination module 203. In the present application, the resource monitoring data selection parameter determination module 203 is mainly used to determine a correlation distance deviation value according to the correlation distance square corresponding to each resource monitoring data point, and then determine the correlation distance square corresponding to the minimum correlation distance deviation value, and determine a resource monitoring data selection parameter according to the corresponding correlation distance square;

[0115] The resource monitoring data correlation distance determination module 204. In the present application, the resource monitoring data correlation distance determination module 204 is mainly used to select a second resource monitoring data subset according to the resource monitoring data selection parameter, and determine the resource monitoring data correlation distance corresponding to each resource monitoring data point in the resource monitoring data set through the second resource monitoring data subset;

[0116] The resource anomaly warning module 205. In the present application, the resource anomaly warning module 205 is mainly used to use the resource monitoring data points with the resource monitoring data correlation distance higher than the preset critical value as resource usage anomaly information for warning.

[0117] The examples of the monitoring system and method provided by the embodiments of the present application are introduced in detail above. It can be understood that, in order to implement the above functions, the corresponding device includes the corresponding hardware structure and / or software module for executing each function. Those skilled in the art should easily realize that, combining the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0118] In some embodiments, the present application further provides a computer device, which includes a memory and a processor. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the above-mentioned monitoring method.

[0119] In some embodiments, referring to Figure 3 , the dashed line in this figure indicates that the unit or the module is optional. This figure is a schematic structural diagram of a computer device for a monitoring method provided by an embodiment of the present application. The above-mentioned monitoring method in the above embodiments can be implemented by Figure 3 the computer device shown. The computer device 300 includes at least one processor 301, a memory 302, and at least one communication unit 305. The computer device 300 can be a terminal device, a server, or a chip.

[0120] The processor 301 can be a general-purpose processor or a dedicated processor. For example, the processor 301 can be a central processing unit (CPU). The CPU can be used to control the computer device 300, execute software programs, and process the data of software programs. The computer device 300 can also include a communication unit 305 for implementing signal input (reception) and output (transmission).

[0121] For example, the computer device 300 can be a chip, and the communication unit 305 can be the input and / or output circuit of the chip. Or, the communication unit 305 can be the communication interface of the chip. The chip can be a component of a terminal device, a network device, or other devices.

[0122] Again, for example, the computer device 300 can be a terminal device or a server, and the communication unit 305 can be the transceiver of the terminal device or the server. Or, the communication unit 305 can be the transceiver circuit of the terminal device or the server.

[0123] The computer device 300 may include one or more memories 302, on which there is a program 304 that can be run by the processor 301 to generate instructions 303, enabling the processor 301 to execute the methods described in the above method embodiments according to the instructions 303. Optionally, data (such as a target audit model) may also be stored in the memory 302. Optionally, the processor 301 may also read the data stored in the memory 302. This data may be stored at the same storage address as the program 304, or it may be stored at a different storage address from the program 304.

[0124] The processor 301 and the memory 302 may be provided separately or integrated together. For example, they may be integrated on a system on chip (SOC) of a terminal device.

[0125] It should be understood that each step of the above method embodiments may be completed by a logic circuit in hardware form or instructions in software form in the processor 301. The processor 301 may be a central processing unit, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices. For example, discrete gate, transistor logic devices, or discrete hardware components.

[0126] Those skilled in the art should understand that the embodiments of the present application may be provided as a method, a system, or a computer program product. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0127] For example, in some embodiments, the present application also provides a computer-readable storage medium, in which instructions or code are stored. When the instructions or code run on a computer, the computer is enabled to execute the above monitoring method.

[0128] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they know the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0129] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these modifications and variations.

Claims

1. A monitoring method, characterized in that, It includes the following steps: Collect resource monitoring data through the Prometheus monitoring system to obtain a resource monitoring data set, determine initial data selection parameters based on the resource monitoring data set, and select a first subset of resource monitoring data through the initial data selection parameters; Determine the squared correlation distance corresponding to each resource monitoring data point in the resource monitoring data set according to the first subset of resource monitoring data; Determine a set of correlation distance deviation values according to the squared correlation distance corresponding to each resource monitoring data point, and further determine the squared correlation distance corresponding to the minimum correlation distance deviation value. Determine the resource monitoring data selection parameters according to the squared correlation distance corresponding to the minimum correlation distance deviation value; Select a second subset of resource monitoring data according to the resource monitoring data selection parameters, and determine the resource monitoring data correlation distance corresponding to each resource monitoring data point in the resource monitoring data set through the second subset of resource monitoring data; Use the resource monitoring data points with a resource monitoring data correlation distance higher than the preset critical value as resource usage anomaly information for alarm.

2. The method according to claim 1, wherein Collecting resource monitoring data through the Prometheus monitoring system to obtain a resource monitoring data set specifically includes: Determine the median value of the resource monitoring data collected through the Prometheus monitoring system; Fill in the missing values of the collected resource monitoring data according to the median value of the resource monitoring data to obtain a resource monitoring data set.

3. The method according to claim 1, characterized in that, Determining the initial data selection parameters according to the resource monitoring data set specifically includes: Determine the total amount of data of the resource monitoring data points in the resource monitoring data set; Determine the data dimension of the resource monitoring data points in the resource monitoring data set; Determine the initial data selection parameters according to the total amount of data of the resource monitoring data points and the data dimension of the resource monitoring data points in the resource monitoring data set. Among them, the initial data selection parameters are determined according to the following formula: Among them, represents the initial data selection parameter, represents the resource monitoring data set the total amount of resource monitoring data points in the resource monitoring data set, represents the resource monitoring data set the data dimension of the resource monitoring data points in the resource monitoring data set, represents the resource monitoring data set.

4. The method according to claim 1, wherein Determining the squared correlation distance corresponding to each resource monitoring data point in the resource monitoring data set according to the first subset of resource monitoring data specifically includes: Determine the data mean vector and data covariance matrix of the first subset of resource monitoring data; Use the data mean vector as the data position parameter of the first subset of resource monitoring data; Use the data covariance matrix as the data scale parameter of the first subset of resource monitoring data; Determine the squared correlation distance corresponding to each resource monitoring data point in the resource monitoring data set according to the data position parameter and data scale parameter of the first subset of resource monitoring data. Among them, the squared correlation distance is determined according to the following formula; Among them, represents the resource monitoring data point corresponding squared correlation distance, represents the th resource monitoring data point in the resource monitoring dataset, represents the data position parameter of the first resource monitoring data subset and represents the data scale parameter of the first resource monitoring data subset and represents the transpose operation.

5. The method according to claim 1, characterized in that, Determine the squared correlation distance corresponding to the minimum correlation distance deviation value. Determining the resource monitoring data selection parameters according to the squared correlation distance corresponding to the minimum correlation distance deviation value specifically includes: Sort the squared correlation distances of all resource monitoring data to obtain a squared correlation distance sequence; Determine the resource monitoring data point corresponding to the minimum correlation distance deviation value, and further determine the sequence position of the squared correlation distance corresponding to the resource monitoring data point in the squared correlation distance sequence; Determine the resource monitoring data selection parameters according to the sequence position and the initial data selection parameters.

6. The method according to claim 1, characterized in that, Selecting a second subset of resource monitoring data according to the selected parameters of the resource monitoring data specifically includes: Selecting a number of resource monitoring data points in the resource monitoring data set that is equal to the value of the selected parameter of the resource monitoring data; Combining the resource monitoring data points into a second subset of resource monitoring data in chronological order.

7. The method according to claim 1, wherein The preset critical value is determined according to the selected parameters of the resource monitoring data.

8. A monitoring system, characterized in that, Including a resource anomaly warning unit, the resource anomaly warning unit includes: A first subset of resource monitoring data selection module, configured to collect resource monitoring data through the Prometheus monitoring system to obtain a resource monitoring data set, determine an initial data selection parameter according to the resource monitoring data set, and select a first subset of resource monitoring data through the initial data selection parameter; A related distance square determination module, configured to determine the related distance square corresponding to each resource monitoring data point in the resource monitoring data set according to the first subset of resource monitoring data; A resource monitoring data selection parameter determination module, configured to determine a related distance deviation value set according to the related distance square corresponding to each resource monitoring data point, and then determine the related distance square corresponding to the minimum related distance deviation value, and determine the resource monitoring data selection parameter according to the related distance square corresponding to the minimum related distance deviation value; A resource monitoring data related distance determination module, configured to select a second subset of resource monitoring data according to the selected parameters of the resource monitoring data, and determine the resource monitoring data related distance corresponding to each resource monitoring data point in the resource monitoring data set through the second subset of resource monitoring data; A resource anomaly warning module, configured to use the resource monitoring data points with the resource monitoring data related distance higher than the preset critical value as resource usage anomaly information for warning.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes a monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium, in which instructions or codes are stored, and when the instructions or codes are run on a computer, the computer is caused to execute a monitoring method according to any one of claims 1 to 7.

Images