Time sequence monitoring index abnormity rapid detection method and device

Abstract

The invention relates to the field of artificial intelligence, and particularly provides a time sequence monitoring index abnormity rapid detection method and device.The method comprises the steps that firstly, historical time sequence monitoring index data are classified, and an index data sample library is constructed; then, respectively constructing Gaussian process regression models according to the index types, predicting index data of the current time, and performing correlation analysis on the predicted data and real-time data of the system to obtain correlation coefficients of the two groups of data; the correlation coefficient is compared with a fault threshold value, if the correlation coefficient is lower than the threshold value, an abnormal warning is given out, and finally warning root judgment and restoration are conducted through manual auditing. Compared with the prior art, human participation in the operation and maintenance inspection process can be reduced to a great extent, and the labor cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and specifically provides a method and device for quickly detecting abnormalities in time series monitoring indicators. Background Art

[0002] During the system operation and maintenance process, the system monitoring indicator data mostly exists in the form of time series waveforms. When the real-time data matches the historical data, it is considered a normal state. When the real-time data does not match the historical data, it is considered an abnormal state and an abnormal alarm is issued. With the deepening of the information process, the system is becoming more and more complex, making the types of monitoring indicators for system operation and maintenance more and more diverse, and the indicator data is constantly generated as the system runs. The manual operation and maintenance means cannot quickly process the massive time series monitoring indicator data and cannot meet the real-time requirements of system operation and maintenance.

[0003] With the rapid development of artificial intelligence technology, it is becoming more and more common in all walks of life to use the learning, efficiency, and reasoning characteristics of artificial intelligence technology to replace manual labor in tedious and repetitive work. How to solve the problems of low efficiency and inability to achieve real-time alarms in the face of massive time series indicator data during system operation and maintenance inspections, as well as the problems that the effect of manual inspections is greatly affected by emotions and mental states, the error rate is too high, and it is difficult to trace are problems that technical personnel in this field urgently need to solve. Summary of the invention

[0004] The present invention aims at the above-mentioned deficiencies of the prior art and provides a highly practical method for quickly detecting abnormalities of time series monitoring indicators.

[0005] A further technical task of the present invention is to provide a device for rapid detection of abnormalities in timing monitoring indicators that is reasonably designed, safe and applicable.

[0006] The technical solution adopted by the present invention to solve its technical problem is: A method for rapid detection of abnormalities in time series monitoring indicators, firstly, classifying historical time series monitoring indicator data and building an indicator data sample library; Then, a Gaussian process regression model is constructed according to the indicator type to predict the indicator data of the current time, and a correlation analysis is performed on the predicted data and the real-time data of the system to obtain the correlation coefficient of the two sets of data; the correlation coefficient is compared with the fault threshold. If the correlation coefficient is lower than the threshold, an abnormal warning is issued. Finally, the root cause of the alarm is determined and repaired through manual review.

[0007] Further, the following steps are included: S1. Classify and label the historical time series monitoring indicators to form an indicator data sample library that can be used for machine learning algorithm training models; S2. Use the Gaussian process regression algorithm to learn and train different types of historical time series monitoring indicator data in the indicator data sample library, and establish a Gaussian process regression model, that is, a detection model. The input data of the model is the time series time, and the output is the monitoring indicator value; S3. Use the constructed Gaussian process regression model, i.e., the detection model, to predict the current time series monitoring indicator data and perform correlation analysis with the system real-time indicator data; S4. Conduct manual review to determine the authenticity of the fault; S5. If it is determined to be a fault, the root cause of the alarm can be determined and the fault can be repaired based on the alarm sequence time and indicator type.

[0008] Furthermore, in step S2, the specific steps of constructing the detection model are as follows: S2.1. Consider a training data set consisting of n input-output pair samples consisting of historical time series monitoring indicator data in the indicator data sample library. , the input vector is a time series column vector, is the column vector of monitoring indicators corresponding to the time, where and The dimensions are the same, and the two are written as a functional relationship ; S2.2 Definitions The set of random variables in the function space obeys a joint Gaussian distribution, that is, for a test input vector , whose predicted output distribution is an n+1-dimensional joint Gaussian distribution consisting of n known training outputs and one unknown test output, which can be expressed as: ; Then the mean function of the joint Gaussian distribution is expressed as: ; The variance of the joint Gaussian distribution is expressed as: ; The mean of this Gaussian distribution is the prediction result, and the variance is a measure of the confidence of the prediction result; in It represents the vector and The covariance between represents the test input vector and the covariance vector between the training input vector, specifically expressed as ,and It represents the covariance matrix formed by the covariance function between the training input vectors, and its matrix elements can be expressed as ; S2.3, determine the choice of covariance function; S2.4, log-likelihood estimation of hyperparameters; S2.5. Use the gradient optimization method to optimize the values ​​of hyperparameters.

[0009] Furthermore, in step S2.3, for the basic monitoring indicators of the CPU utilization indicator, memory usage indicator, video memory usage indicator and GPU temperature indicator system, a periodic covariance function is selected for the construction of such indicator detection model, and the periodic covariance function is expressed as: ; For order indicators and application call indicators, the rational quadratic covariance function is selected for the construction of such indicator detection models. The rational quadratic covariance function is expressed as: ; In this covariance function the hyperparameter The number of input time series vectors and The dimensions are the same, that is, each detection time point corresponds to a hyperparameter; For traffic indicators, a composite covariance function constructed using a periodic covariance function and a rational quadratic covariance function is used to construct a detection model for such indicators, which is specifically expressed as: ; The hyperparameters , , , and The value of is obtained by maximizing its log-likelihood estimate.

[0010] Furthermore, in step S2.4, for Gaussian process regression, the log-likelihood estimate of the hyperparameters is: ; in represents the vector composed of all hyperparameters, Representation Matrix The value of the determinant of the elements in .

[0011] Furthermore, in step S2.5, the gradient optimization method is used to optimize the value of the hyperparameter to obtain the hyperparameter that maximizes the log-likelihood estimate. In this process, the partial derivative of the log-likelihood estimate with respect to each hyperparameter needs to be calculated, and the calculation formula can be expressed as: ; in, It means to find the trace of the matrix, that is, the sum of the diagonal elements of the matrix.

[0012] Furthermore, in step S3, in the art, the monitoring index is time series data, and its value will change continuously according to the time change, and has time continuity. Therefore, when performing correlation analysis, the value change curves of the predicted data and the real-time monitoring data in the same time series should be selected to perform correlation analysis; Assume that the time series data to be predicted is selected within the same time series and real-time monitoring of time series data , its correlation coefficient The calculation formula is: ; in and Forecasting time series data and real-time monitoring of time series data The mean of It is compared with the fault threshold set by the system. If the correlation coefficient is higher than the threshold, it is considered normal. If the correlation coefficient is lower than the threshold, it is considered abnormal and an abnormal alarm is issued.

[0013] Furthermore, in step S5, if it is a false alarm, it is fed back to the indicator data sample library after manual annotation, the indicator data sample library is updated, and the detection model is updated through re-learning and calibration.

[0014] A device for quickly detecting abnormality of a time series monitoring indicator, comprising: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to execute a method for quickly detecting abnormalities in timing monitoring indicators.

[0015] Compared with the prior art, the method and device for quickly detecting abnormality of time series monitoring indicators of the present invention have the following outstanding beneficial effects: The present invention selects different covariance functions according to the characteristics of different types of monitoring index data, uses the Gaussian process regression algorithm to learn the historical time series monitoring index data, constructs a Gaussian process regression model to predict the monitoring index data of the current time, and performs correlation analysis with the real-time monitoring index data. It judges whether it is abnormal according to the correlation result. This process has the characteristics of parallelization and fast processing speed. The response time can reach the second level, and the root cause of the alarm can be judged according to the alarm timing and indicator type, which greatly improves the system operation and maintenance efficiency.

[0016] Moreover, this process greatly reduces human involvement in the operation and maintenance inspection process, thereby reducing labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Attached Figure 1 The present invention is a flowchart of a method for rapid detection of abnormalities of time series monitoring indicators. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] A best embodiment is given below: like Figure 1 As shown, a method for quickly detecting abnormalities of time series monitoring indicators in this embodiment first classifies historical time series monitoring indicator data and constructs an indicator data sample library.

[0021] Then, a Gaussian process regression model is constructed according to the indicator type to predict the indicator data of the current time, and a correlation analysis is performed on the predicted data and the real-time data of the system to obtain the correlation coefficient of the two sets of data; the correlation coefficient is compared with the fault threshold. If the correlation coefficient is lower than the threshold, an abnormal warning is issued, and then the root cause of the alarm is determined and repaired through manual review.

[0022] The specific steps are as follows: S1. Classify and label the historical time series monitoring indicator data according to types such as CPU utilization indicator, memory usage indicator, video memory usage indicator, GPU temperature indicator, order indicator, application call count indicator, and traffic indicator to form an indicator data sample library that can be used for machine learning algorithm training models.

[0023] The characteristic of this type of monitoring indicator is that its value changes continuously over time and has time continuity. Therefore, its continuous change trend within a certain period of time needs to be considered in the anomaly detection process.

[0024] S2. Use the Gaussian process regression algorithm to learn and train different types of historical time series monitoring indicator data in the indicator data sample library, and establish a Gaussian process regression model, that is, a detection model. The input data of the model is the time series time, and the output is the monitoring indicator value.

[0025] This model can establish a joint Gaussian distribution consisting of predicted data and historical data through maximum likelihood estimation based on historical data, which is used to regress the input time series to predict the output value and realize the prediction of the indicator data at the current time.

[0026] The specific steps to build the detection model are: S2.1. Consider a training data set consisting of n input-output pair samples consisting of historical time series monitoring indicator data in the indicator data sample library. , the input vector is a time series column vector, is the column vector of monitoring indicators corresponding to the time, where and The dimensions are the same, and the two are written as a functional relationship .

[0027] S2.2 Definitions The set of random variables in the function space obeys a joint Gaussian distribution, that is, for a test input vector , whose predicted output distribution is an n+1-dimensional joint Gaussian distribution consisting of n known training outputs and one unknown test output, which can be expressed as: ; Then the mean function of the joint Gaussian distribution can be expressed as: ; The variance of the joint Gaussian distribution can be expressed as: ; The mean of this Gaussian distribution is the prediction result, and the variance is a measure of the confidence of the prediction result. It represents the vector and The covariance between Represents the test input vector and the covariance vector between the training input vector, specifically expressed as ,and It represents the covariance matrix formed by the covariance function between the training input vectors, and its matrix elements can be expressed as .

[0028] It can be seen that the prediction results of the model and the confidence of the prediction results are directly related to the covariance function. Therefore, when the covariance function is determined, the model can be established.

[0029] S2.3. Determine the selection of covariance function. In the present invention, different covariance functions are selected to construct the detection model according to the characteristics of different types of monitoring indicator data to obtain the best effect.

[0030] For basic system monitoring indicators such as CPU utilization index, memory usage index, video memory usage index, GPU temperature index, etc., these indicator data have obvious periodic changes. The present invention selects a periodic covariance function for the construction of such indicator detection model. The periodic covariance function is expressed as: ; For order indicators and application call indicators, these indicator data are jumpy. The present invention selects a rational quadratic covariance function for the construction of such indicator detection model. The rational quadratic covariance function is expressed as: ; In this covariance function the hyperparameter The number of input time series vectors and The dimension is the same, that is, each detection time point corresponds to a hyperparameter, which can better fit the jumpiness of order indicators and application call indicators.

[0031] Regarding traffic indicators, such indicators have both periodicity and jumpiness. The present invention uses a composite covariance function constructed by a periodic covariance function and a rational quadratic covariance function to construct a detection model for such indicators, which is specifically expressed as: ; Among them, the hyperparameters , , , and The value of can be obtained by maximizing its log-likelihood estimate.

[0032] S2.5, Log-likelihood estimation, for Gaussian process regression, the log-likelihood estimate of the hyperparameters is: ; in represents the vector composed of all hyperparameters, Representation Matrix The value of the determinant of the elements in .

[0033] Gradient optimization method uses gradient optimization method to optimize the value of hyperparameters to obtain the hyperparameters that maximize the log-likelihood estimate. In this process, the partial derivative of the log-likelihood estimate for each hyperparameter needs to be calculated. The calculation formula can be expressed as: ; in It means to find the trace of the matrix, that is, the sum of the diagonal elements of the matrix.

[0034] S3. Use the constructed Gaussian process regression model, i.e., the detection model, to predict the current time series monitoring indicator data and perform correlation analysis with the system real-time indicator data. In this field, the monitoring indicator is time series data, and its value will change continuously according to the time change, and has time continuity. Therefore, when performing correlation analysis, it is necessary to select the value change curve of the predicted data and the real-time monitoring data in the same time series to perform correlation analysis. Assume that the predicted time series data in the same time series is selected and real-time monitoring of time series data , its correlation coefficient The calculation formula is: ; in and Forecasting time series data and real-time monitoring of time series data The mean of It is compared with the fault threshold set by the system. If the correlation coefficient is higher than the threshold, it is considered normal. If the correlation coefficient is lower than the threshold, it is considered abnormal and an abnormal alarm is issued.

[0035] S4. Perform manual review to determine the authenticity of the fault.

[0036] S5. If it is determined to be a fault, the root cause of the alarm can be determined and the fault can be repaired according to the alarm timing and indicator type; if it is a false alarm, it will be fed back to the indicator data sample library after manual labeling, the indicator data sample library will be updated, and the detection model will be updated through re-learning and calibration.

[0037] Based on the above method, a time series monitoring indicator abnormality rapid detection device in this embodiment includes: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to execute a method for quickly detecting abnormalities in timing monitoring indicators.

[0038] The above-mentioned specific implementations are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above-mentioned specific implementations. Any technical solutions that conform to the above-mentioned specific implementations of the present invention and any appropriate changes or substitutions made by ordinary technicians in the relevant technical field shall fall within the patent protection scope of the present invention.

[0039] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for rapid detection of abnormality of time series monitoring indicators, characterized in that: First, classify the historical time series monitoring indicator data and build an indicator data sample library; Then, a Gaussian process regression model is constructed according to the indicator type to predict the indicator data of the current time, and a correlation analysis is performed on the predicted data and the real-time data of the system to obtain the correlation coefficient of the two sets of data; the correlation coefficient is compared with the fault threshold. If the correlation coefficient is lower than the threshold, an abnormal warning is issued. Finally, the root cause of the alarm is determined and repaired through manual review.

2. The method for rapid detection of abnormality of time series monitoring indicators according to claim 1, characterized in that: The steps are as follows: S1. Classify and label the historical time series monitoring indicators to form an indicator data sample library that can be used for machine learning algorithm training models; S2. Use the Gaussian process regression algorithm to learn and train different types of historical time series monitoring indicator data in the indicator data sample library, and establish a Gaussian process regression model, that is, a detection model. The input data of the model is the time series time, and the output is the monitoring indicator value; S3. Use the constructed Gaussian process regression model, i.e., the detection model, to predict the current time series monitoring indicator data and perform correlation analysis with the system real-time indicator data; S4. Conduct manual review to determine the authenticity of the fault; S5. If it is determined to be a fault, the root cause of the alarm can be determined and the fault can be repaired based on the alarm sequence time and indicator type.

3. The method for rapid detection of abnormality of time series monitoring indicators according to claim 2 is characterized in that: In step S2, the specific steps of building the detection model are as follows: S2.

1. Consider a training data set consisting of n input-output pair samples consisting of historical time series monitoring indicator data in the indicator data sample library. , the input vector is a time series column vector, is the column vector of monitoring indicators corresponding to the time, where and The dimensions are the same, and the two are written as a functional relationship ; S2.2 Definitions The set of random variables in the function space obeys a joint Gaussian distribution, that is, for a test input vector , whose predicted output distribution is an n+1-dimensional joint Gaussian distribution consisting of n known training outputs and one unknown test output, which can be expressed as: ； Then the mean function of the joint Gaussian distribution is expressed as: ； The variance of the joint Gaussian distribution is expressed as: ； The mean of this Gaussian distribution is the prediction result, and the variance measures the confidence of the prediction result; in It represents the vector and The covariance between Represents the test input vector and the covariance vector between the training input vector, specifically expressed as ,and It represents the covariance matrix formed by the covariance function between the training input vectors, and its matrix elements can be expressed as ; S2.3, determine the choice of covariance function; S2.4, log-likelihood estimation of hyperparameters; S2.

5. Use the gradient optimization method to optimize the values ​​of hyperparameters.

4. The method for rapid detection of abnormality of time series monitoring indicators according to claim 3 is characterized in that: In step S2.3, for the basic monitoring indicators of the CPU utilization index, memory usage index, video memory usage index and GPU temperature index system, a periodic covariance function is selected to construct a detection model for such indicators. The periodic covariance function is expressed as: ； For order indicators and application call indicators, the rational quadratic covariance function is selected for the construction of such indicator detection models. The rational quadratic covariance function is expressed as: ； In this covariance function the hyperparameter The number of input time series vectors and The dimensions are the same, that is, each detection time point corresponds to a hyperparameter; For traffic indicators, a composite covariance function constructed using a periodic covariance function and a rational quadratic covariance function is used to construct a detection model for such indicators, which is specifically expressed as: ； The hyperparameters , , , and The value of is obtained by maximizing its log-likelihood estimate.

5. A method for rapid detection of abnormality of time series monitoring indicators according to claim 4, characterized in that: In step S2.4, for Gaussian process regression, the log-likelihood estimate of the hyperparameters is: ； in represents the vector composed of all hyperparameters, Representation Matrix The value of the determinant of the elements in .

6. A method for rapid detection of abnormality of time series monitoring indicators according to claim 5, characterized in that: In step S2.5, the gradient optimization method is used to optimize the value of the hyperparameter to obtain the hyperparameter that maximizes the log-likelihood estimate. In this process, the partial derivative of the log-likelihood estimate for each hyperparameter needs to be calculated, and the calculation formula can be expressed as: ； in, It means to find the trace of the matrix, that is, the sum of the diagonal elements of the matrix.

7. A method for rapid detection of abnormality of time series monitoring indicators according to claim 6, characterized in that: In step S3, in this field, the monitoring indicator is time series data, and its value will change continuously according to the time change, and has time continuity. Therefore, when performing correlation analysis, it is necessary to select the value change curve of the predicted data and the real-time monitoring data in the same time series to perform correlation analysis; Assume that the time series data to be predicted is selected within the same time series and real-time monitoring of time series data , its correlation coefficient The calculation formula is: ； in and Forecasting time series data and real-time monitoring of time series data The mean of It is compared with the fault threshold set by the system. If the correlation coefficient is higher than the threshold, it is considered normal. If the correlation coefficient is lower than the threshold, it is considered abnormal and an abnormal alarm is issued.

8. The method for rapid detection of abnormality of time series monitoring indicators according to claim 6, characterized in that: In step S5, if it is a false alarm, it is fed back to the indicator data sample library after manual annotation, the indicator data sample library is updated, and the detection model is updated through re-learning and calibration.

9. A device for rapid detection of abnormality of time series monitoring indicators, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 8.

Images