A method and device for processing monitoring data

Abstract

One or more embodiments of this specification provide a method and device for processing monitoring data. The method includes: obtaining the time series of key monitoring indicators and the time series of multiple candidate monitoring indicators; using a heuristic algorithm, taking the time series of the key monitoring indicators and the time series of the multiple candidate monitoring indicators as input data, and inputting them into a preset convex function serving as an objective function for optimization to obtain integral coefficients indicating whether there is a linear relationship between the time series of the key monitoring indicators and the time series of each candidate monitoring indicator among the multiple candidate monitoring indicators; determining, according to the integral coefficients, the candidate monitoring indicators that have a linear relationship with the key monitoring indicators from the multiple candidate monitoring indicators. Through the above embodiments, in a scenario where the monitoring data includes a large number of monitoring indicators, the labor cost and time cost can be saved, and the mining efficiency can be improved.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of application technology, and in particular, to a method and device for processing monitoring data. Background Art

[0002] The monitoring system connected to the business system can assist the operation and maintenance personnel to perceive the operating status of the business system in a timely manner. The operation and maintenance personnel can usually analyze the various monitoring indicators contained in the monitoring data to determine whether the business system is running normally, which business links in the business corresponding to the business system have problems, etc.

[0003] In the process of analyzing monitoring indicators, it is very important to explore the correlation between different monitoring indicators. In practical applications, business personnel can usually sort out key business indicators first, and then for applications associated with the key business indicators, business personnel and developers jointly analyze the source code logic to mine the key monitoring indicators corresponding to the key business indicators and other monitoring indicators related to the key monitoring indicators from monitoring data such as log files.

[0004] It can be seen that in scenarios where monitoring data includes massive monitoring indicators, manually mining the correlation between monitoring indicators not only has the problems of high labor cost, high time cost, and low mining efficiency, but also makes it difficult to mine the correlation between monitoring indicators across code logic. Summary of the invention

[0005] The present application provides a method for processing monitoring data, the method comprising:

[0006] Obtain the time series of key monitoring indicators and the time series of multiple candidate monitoring indicators;

[0007] A heuristic algorithm is used to take the time series of the key monitoring indicator and the time series of the multiple candidate monitoring indicators as input data, and input them into a preset convex function as an objective function for optimization solution, so as to obtain an integer coefficient representing whether there is a linear relationship between the time series of the key monitoring indicator and the time series of each candidate monitoring indicator in the multiple candidate monitoring indicators;

[0008] According to the integral point coefficient, a candidate monitoring indicator having a linear relationship with the key monitoring indicator is determined from the multiple candidate monitoring indicators.

[0009] Optionally, the optimization objectives of the objective function include:

[0010] Solve for an optimal coefficient matrix multiplied with the first matrix when the straight-line distance between a second matrix formed by multiplying the first matrix and the coefficient matrix and a third matrix formed by the time series of the key monitoring indicators is minimized; wherein the first matrix is ​​formed by the time series of the multiple candidate monitoring indicators, and the coefficient matrix is ​​formed by the integer coefficients between the time series of the key monitoring indicators and the time series of each of the multiple candidate monitoring indicators;

[0011] The constraint condition of the objective function includes: the integer point coefficient obtained by solving is an integer.

[0012] Optionally, the value of the integral point coefficient includes 0 and 1; wherein, when the integral point coefficient is 0, it indicates that there is no linear relationship between the time series of the candidate monitoring indicator corresponding to the integral point coefficient and the time series of the key monitoring indicator; when the integral point coefficient is 1, it indicates that there is a linear relationship between the time series of the candidate monitoring indicator corresponding to the integral point coefficient and the time series of the key monitoring indicator;

[0013] Determining, according to the integral point coefficient, a candidate monitoring indicator having a linear relationship with the key monitoring indicator from the multiple candidate monitoring indicators, including:

[0014] Determine the integer point coefficients whose values ​​are 1 in the coefficient matrix;

[0015] The candidate monitoring indicator corresponding to the integer coefficient whose value is 1 in the coefficient matrix is ​​determined as the candidate monitoring indicator having a linear relationship with the key monitoring indicator.

[0016] Optionally, the heuristic algorithm includes a simulated annealing algorithm;

[0017] The objective function is optimized by using a simulated annealing algorithm, including:

[0018] Step a, generating the first matrix according to the input time series of the plurality of candidate monitoring indicators, and generating the third matrix according to the input time series of the key monitoring indicators;

[0019] Step b, using a preset initial temperature as the current temperature, and using a preset initial coefficient matrix as the current optimal coefficient matrix of the objective function, and multiplying the first matrix by the current optimal coefficient matrix to obtain a second matrix, and calculating a first straight-line distance between the second matrix and the third matrix as the objective function value corresponding to the current optimal coefficient matrix;

[0020] Step c, randomly generating a new coefficient matrix of the objective function, and multiplying the first matrix by the new coefficient matrix to obtain a new second matrix, and calculating a second straight-line distance between the new second matrix and the third matrix as the objective function value corresponding to the new coefficient matrix;

[0021] Step d: if the second straight-line distance is less than the first straight-line distance, the new coefficient matrix is ​​updated as the current optimal coefficient matrix of the objective function, and the second straight-line distance is updated as the objective function value corresponding to the current optimal coefficient matrix; if the second straight-line distance is not less than the first straight-line distance, the new coefficient matrix is ​​updated as the current optimal coefficient matrix of the objective function according to a preset probability, and the second straight-line distance is updated as the objective function value corresponding to the current optimal coefficient matrix;

[0022] Step e, iteratively executing steps c to d at the current temperature until the number of iterations reaches a preset maximum number of inner-layer cycles, and then multiplying the current temperature by a preset attenuation coefficient to obtain a new current temperature;

[0023] Step g: If the new current temperature is not less than the preset termination temperature, iteratively execute steps c to e at the new current temperature until the new current temperature is less than the termination temperature, and then determine the current optimal coefficient matrix of the objective function as the optimal coefficient matrix obtained by solving the objective function.

[0024] Optionally, the constraint condition of the objective function further includes: among the multiple candidate monitoring indicators, the number of candidate monitoring indicators that have a linear relationship with the key monitoring indicator does not exceed a preset maximum number of linear associations;

[0025] A new coefficient matrix of the objective function is randomly generated, including:

[0026] Calculating partial derivatives of the objective function in directions corresponding to the plurality of candidate monitoring indicators, respectively, and sorting the calculated partial derivatives, so as to determine, based on the sorting results, directions of fastest gradient descent corresponding to a first number of candidate monitoring indicators from the partial derivatives that are not less than 0; wherein the first number is not greater than the maximum number of linear associations;

[0027] A new coefficient matrix of the objective function is randomly generated, and the integer coefficients corresponding to the first number of candidate monitoring indicators in the new coefficient matrix are set to 1.

[0028] Optionally, the method further includes:

[0029] Randomly masking the non-zero elements in the new coefficient matrix;

[0030] Performing random sampling in a closed interval from 0 to a preset value to obtain a second number; wherein the difference between the maximum linear correlation number and the number of integer coefficients with a value of 1 in the new coefficient matrix is ​​greater than or equal to the preset value;

[0031] extracting a second number of candidate monitoring indicators from the plurality of candidate monitoring indicators without replacement;

[0032] The integer point coefficients corresponding to the second number of candidate monitoring indicators in the new coefficient matrix are set to 1.

[0033] Optionally, the method further includes:

[0034] Get the time series of multiple monitoring indicators extracted from the monitoring log file;

[0035] Calculating the correlation coefficients between the time series of the multiple monitoring indicators and the time series of the key monitoring indicator respectively; wherein the correlation coefficient represents the degree of correlation between the time series of each monitoring indicator in the multiple monitoring indicators and the time series of the key monitoring indicator;

[0036] According to the correlation coefficient, at least some of the monitoring indicators whose time series have the highest correlation coefficient with the time series of the key monitoring indicator are screened out from the multiple monitoring indicators as the multiple candidate monitoring indicators.

[0037] The present application also provides a monitoring data processing device, the device comprising:

[0038] An acquisition unit, used to acquire a time series of a key monitoring indicator and a time series of multiple candidate monitoring indicators;

[0039] A solution unit, used to adopt a heuristic algorithm, take the time series of the key monitoring indicator and the time series of the multiple candidate monitoring indicators as input data, and input them into a preset convex function as an objective function for optimization solution, so as to obtain an integer coefficient representing whether there is a linear relationship between the time series of the key monitoring indicator and the time series of each candidate monitoring indicator in the multiple candidate monitoring indicators;

[0040] A determination unit is used to determine, according to the integral point coefficient, a candidate monitoring indicator that has a linear relationship with the key monitoring indicator from the multiple candidate monitoring indicators.

[0041] The present application also provides an electronic device, comprising a communication interface, a processor, a memory and a bus, wherein the communication interface, the processor and the memory are interconnected via the bus;

[0042] The memory stores machine-readable instructions, and the processor executes the above method by calling the machine-readable instructions.

[0043] The present application also provides a machine-readable storage medium, which stores machine-readable instructions. When the machine-readable instructions are called and executed by a processor, the above method is implemented.

[0044] Through the above embodiments, by adopting a heuristic algorithm, the time series of the key monitoring indicator and the time series of multiple candidate monitoring indicators are used as input data, and input into a convex function as an objective function for optimization solution. According to the integer coefficients obtained by the solution, the candidate monitoring indicators that have a linear relationship with the key monitoring indicator can be determined from the multiple candidate monitoring indicators, thereby realizing automatic mining of the linear relationship between different monitoring indicators; and in the scenario where the monitoring data includes massive monitoring indicators, it can not only reduce the requirements for user experience and technical level, save labor costs and time costs, and improve mining efficiency, but also perform linear relationship mining on monitoring indicators across code logic. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0046] Figure 1 is a flowchart of a method for processing monitoring data shown in an exemplary embodiment;

[0047] Figure 2 is a flowchart of a simulated annealing algorithm shown in an exemplary embodiment;

[0048] Figure 3 It is a structural schematic diagram of an electronic device in which a monitoring data processing device is located, shown in an exemplary embodiment;

[0049] Figure 4 It is a block diagram of a monitoring data processing device shown in an exemplary embodiment. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0051] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0052] As businesses become increasingly complex and the number of users continues to grow, it becomes increasingly difficult to determine whether business systems are operating in an expected manner.

[0053] The monitoring system connected to the business system can assist the operation and maintenance personnel to perceive the operating status of the business system in a timely manner. The monitoring system can record a large amount of runtime information of the business system in monitoring data such as log files and traces files. The operation and maintenance personnel can usually count the key fields in the monitoring data (such as error codes, result codes, call types, etc.) to obtain the number of times these key fields appear in a specified time interval to form a time series of these statistical indicators, which are also monitoring indicators. By analyzing each monitoring indicator, the operation and maintenance personnel can determine whether the business system is running normally, which business links in the business corresponding to the business system have problems, etc.

[0054] In the process of analyzing monitoring indicators, it is very important to explore the correlation between different monitoring indicators. On the one hand, the correlation between monitoring indicators can help operation and maintenance personnel improve the efficiency and accuracy of locating problems. For example, if the time series of a certain error code of a business suddenly increases, the operation and maintenance personnel can perceive that the business is abnormal; further, if it is known that there is a correlation between the error code of the business and the result code of a call to a certain interface, and the time series of the result code also suddenly increases, the operation and maintenance personnel can judge that the business abnormality is likely related to the interface.

[0055] In related technologies, business personnel can usually first sort out key business indicators, and then for applications related to the key business indicators, business personnel and developers jointly analyze the source code logic to mine key monitoring indicators corresponding to the key business indicators and other monitoring indicators related to the key monitoring indicators from monitoring data such as log files.

[0056] It can be seen that in the scenario where the monitoring data includes a large number of monitoring indicators, the manual method of mining the correlation between monitoring indicators depends on the experience and technical level of business personnel and developers, and it also takes a lot of time to parse the source code logic and understand the content of monitoring data such as log files, resulting in high labor costs, high time costs, and low mining efficiency. In addition, it is also difficult to mine the correlation between monitoring indicators across code logic using manual methods.

[0057] In view of this, the present specification aims to propose a technical solution that can automatically mine the linear relationship between monitoring indicators.

[0058] The core concept of this manual is:

[0059] In the scenario where the monitoring data includes a large number of monitoring indicators, it is difficult to quickly calculate the time series of several candidate monitoring indicators that have a linear relationship with the time series of key monitoring indicators from the time series of a large number of candidate monitoring indicators. Therefore, by converting the problem of calculating the linear relationship between the time series of monitoring indicators into a general problem of optimizing convex functions (that is, convex optimization problem), various relatively mature algorithms can be used to solve the convex optimization problem (OPT, convex optimization problem), and then according to the approximate optimal solution of the convex optimization problem obtained, the linear relationship between the time series of monitoring indicators can be mined.

[0060] Moreover, since the solution space of the objective function is restricted to integer numbers by adding constraints to the objective function, the convex optimization problem described above can be specifically a general integer-point convex optimization problem; for example, the integer-point convex optimization problem can be specifically a general quadratic integer-point programming problem (IQP, Integer Quadratic Programming).

[0061] Among them, since the integer point convex optimization problem is an NPH (NP-hard) problem, that is, a problem to which all NP (Non-deterministic Polynomial) problems can be reduced within the polynomial time complexity, it is very suitable to be solved by a heuristic algorithm.

[0062] For example, the heuristic algorithm may specifically include, but is not limited to, a simulated annealing algorithm, an ant colony algorithm, an artificial neural network, etc., which are not specifically limited in this specification.

[0063] During implementation, the time series of the key monitoring indicator and the time series of multiple candidate monitoring indicators can be obtained; further, a heuristic algorithm can be used to take the time series of the key monitoring indicator and the time series of the multiple candidate monitoring indicators as input data, and input them into a preset convex function as an objective function for optimization solution, so as to obtain the integer coefficients that characterize whether there is a linear relationship between the time series of the key monitoring indicator and the time series of each candidate monitoring indicator in the multiple candidate monitoring indicators; further, based on the integer coefficients obtained by optimizing the convex function, the candidate monitoring indicators that have a linear relationship with the key monitoring indicator can be determined from the multiple candidate monitoring indicators.

[0064] It can be seen that in the embodiments shown above, by adopting a heuristic algorithm to optimize the convex function as the objective function, according to the integer coefficients obtained by the solution, the candidate monitoring indicators that have a linear relationship with the key monitoring indicators can be determined from multiple candidate monitoring indicators, thereby realizing automatic mining of the linear relationship between different monitoring indicators; and in the scenario where the monitoring data includes massive monitoring indicators, it can not only reduce the requirements for user experience and technical level, save manpower and time costs, and improve mining efficiency, but also perform linear relationship mining on monitoring indicators across code logic.

[0065] It should be noted that, in the related art, the heuristic algorithm can usually only be used to solve feasible solutions; while in the technical solution of the present specification, by converting the objective function into a convex function and adding constraints to the convex function as the objective function, the solution space is limited to integer numbers, and the optimization problem of mining the linear relationship between multiple monitoring indicators can be converted into a general integer-point convex optimization problem, so that the heuristic algorithm can be used to solve the objective function and the global optimal solution can be solved.

[0066] In practical applications, linear relationship mining between monitoring indicators has important business value and is widely used in business fields such as fault analysis, semantic mining, and data verification.

[0067] For example, if the time series of a key monitoring indicator can be expressed as the sum of the time series of multiple candidate monitoring indicators, then this linear relationship can to a certain extent reflect the causal relationship between the multiple candidate monitoring indicators and the key monitoring indicator; that is, when the time series of the key monitoring indicator is abnormal, the reason for the abnormality in the time series of the key monitoring indicator can be determined based on whether the time series of the multiple candidate monitoring indicators are abnormal.

[0068] For example, a monitoring indicator recorded in the monitoring data (such as the error code of a certain business) may exist independently, and the operation and maintenance personnel need to consult relevant documents or contact developers and business personnel to understand the semantics of the monitoring indicator; if it is known that there is a linear relationship between the error code of the business and the result code of calling certain interfaces, the operation and maintenance personnel can combine the semantics of the result codes of these interfaces to understand the semantics of the error code of the business.

[0069] For another example, if the time series of a key monitoring indicator can be expressed as the sum of the time series of multiple candidate monitoring indicators, then once this linear relationship is destroyed, it can be said that at least one of the monitoring indicators has an abnormality; especially in business systems with idempotence requirements, data verification based on the linear relationship between monitoring indicators can more effectively detect anomalies.

[0070] The technical solutions in this specification are described below through specific embodiments and in combination with specific application scenarios.

[0071] See also Figure 1 , Figure 1 FIG. 1 is a flowchart of a method for processing monitoring data according to an exemplary embodiment. The method for processing monitoring data may perform the following steps:

[0072] Step 102: Obtain the time series of the key monitoring indicator and the time series of multiple candidate monitoring indicators;

[0073] Step 104: using a heuristic algorithm, taking the time series of the key monitoring indicator and the time series of the multiple candidate monitoring indicators as input data, and inputting them into a preset convex function as an objective function for optimization solution, so as to obtain an integer coefficient representing whether there is a linear relationship between the time series of the key monitoring indicator and the time series of each candidate monitoring indicator in the multiple candidate monitoring indicators;

[0074] Step 106: According to the integral point coefficient, determine a candidate monitoring indicator having a linear relationship with the key monitoring indicator from the multiple candidate monitoring indicators.

[0075] In step 102, a time series of a key monitoring indicator and a time series of a plurality of candidate monitoring indicators may be obtained.

[0076] For example, the time series of key monitoring indicators can be obtained as Z s，0 , and the time series of K candidate monitoring indicators can be obtained respectively as Z s，1 , Z s，2 ,……,Z s，K , where K is a positive integer greater than 1, s = 1, ..., T. For example: Z s，0 It can be used to characterize the time series of the key monitoring indicators in the 1st time interval to the Tth time interval.

[0077] In an illustrated embodiment, before step 102, the method may further include: acquiring monitoring data to be processed, wherein the monitoring data may include a plurality of monitoring indicators.

[0078] For example, a monitoring log file can be obtained from the monitoring system, and key fields such as error code, result code, call type, etc. can be extracted from the text content of the monitoring log file as monitoring indicators. Furthermore, an aggregation operation can be performed on the number of each monitoring indicator to count the number of times each monitoring indicator appears in a specified time interval to obtain a time series of the monitoring indicator.

[0079] Wherein, in step 102, the key monitoring indicator may include a monitoring indicator that needs to be mined for linear relationship among several monitoring indicators included in the monitoring data. The candidate monitoring indicators may include other monitoring indicators among the several monitoring indicators except the key monitoring indicator.

[0080] In one embodiment shown, before step 102, the method may further include: obtaining time series of multiple monitoring indicators extracted from the monitoring log file; respectively calculating the correlation coefficients between the time series of the multiple monitoring indicators and the time series of the key monitoring indicator; wherein the correlation coefficient represents the degree of correlation between the time series of each monitoring indicator in the multiple monitoring indicators and the time series of the key monitoring indicator; and according to the correlation coefficient, selecting at least some of the monitoring indicators whose time series have the highest correlation coefficient with the time series of the key monitoring indicator from the multiple monitoring indicators as the multiple candidate monitoring indicators.

[0081] For example, the time series of key monitoring indicators extracted from the monitoring log file can be obtained as Z s，0 , and the time series of N monitoring indicators are Z s，1 , Z s，2 ,……,Z s，N, where N is a positive integer greater than K; further, the time series Z of each monitoring indicator can be calculated separately s，1 , Z s，2 ,……,Z s，N Time series Z of key monitoring indicators s，0 The correlation coefficient between them can be used to determine the degree of correlation between the time series of each monitoring indicator in the N monitoring indicators and the time series of the key monitoring indicator; further, according to the calculated correlation coefficient, the K monitoring indicators with the highest correlation coefficient between their time series and the time series of the key monitoring indicator can be screened out from the N monitoring indicators as the candidate monitoring indicators.

[0082] Among them, this specification does not specifically limit the specific implementation method of calculating the correlation coefficient between the time series of the multiple monitoring indicators and the time series of the key monitoring indicators. In one possible embodiment, the correlation coefficient may include a Pearson correlation coefficient; wherein, when the Pearson correlation coefficient is 0, it indicates that the two time series are uncorrelated; when the absolute value of the Pearson correlation coefficient is greater than 0, it indicates that the two time series are correlated, and the larger the absolute value, the stronger the correlation.

[0083] It should be noted that, in the embodiments shown above, by respectively calculating the correlation coefficients between the time series of the multiple monitoring indicators and the time series of the key monitoring indicators, K candidate monitoring indicators (for example, N can be a positive integer in the order of ten thousand, and K can be a positive integer in the order of thousand) whose time series have the highest correlation with the time series of the key monitoring indicators can be screened out from the N monitoring indicators as input data of the objective function, thereby excluding the monitoring indicators among the N monitoring indicators that are not related to the key monitoring indicators, reducing the matrix dimension of the input data of the objective function, and reducing the computational complexity of optimizing the solution to the objective function.

[0084] In step 104, a heuristic algorithm can be used to take the time series of the key monitoring indicator and the time series of the multiple candidate monitoring indicators as input data, and input them into a preset convex function as an objective function for optimization solution to obtain the integer coefficient that characterizes whether there is a linear relationship between the time series of the key monitoring indicator and the time series of each candidate monitoring indicator in the multiple candidate monitoring indicators.

[0085] Among them, in step 104, the optimization goal of the objective function can be: to solve the integer coefficients corresponding to each of the K candidate monitoring indicators when the sum of the time series of some of the K candidate monitoring indicators is closest to the time series of the key monitoring indicator.

[0086] For example, when the time series of key monitoring indicators is Z s，0 , and the time series of K candidate monitoring indicators are Z s，1 , Z s，2 ,……,Z s，K After that, Z s，0 , Z s，1 , Z s，2 ,……,Z s，K As input data, it is input into the objective function f(x) for optimization solution; the algorithm adopted for optimizing the objective function f(x) can be a heuristic algorithm; further, the time series Z representing the key monitoring indicators can be obtained s，0 and the time series Z of K candidate monitoring indicators s，1 , Z s，2 ,……,Z s，K Is there a linear relationship between the integral coefficients [x 1 , x 2 , ..., x k ].

[0087] In a possible embodiment, the value of the integral point coefficient may include 0 and 1; wherein, when the integral point coefficient takes the value of 0, it can represent that there is no linear relationship between the time series of the candidate monitoring indicator corresponding to the integral point coefficient and the time series of the key monitoring indicator; when the integral point coefficient takes the value of 1, it can represent that there is a linear relationship between the time series of the candidate monitoring indicator corresponding to the integral point coefficient and the time series of the key monitoring indicator.

[0088] For example, x 1 =1, the time series Z of the candidate monitoring indicator can be represented s，1 Time series Z of key monitoring indicators s，0 There is a linear relationship between them. For example, x 2 = 0, the time series Z of the candidate monitoring indicator can be represented s，2 Time series Z of key monitoring indicators s，0 There is no linear relationship between them.

[0089] In one embodiment shown, in step 104, the optimization objective of the objective function may specifically include: solving the optimal coefficient matrix multiplied with the first matrix when the straight-line distance between the second matrix formed by multiplying the first matrix and the coefficient matrix and the third matrix formed by the time series of the key monitoring indicators is minimized; wherein the first matrix is ​​composed of the time series of the multiple candidate monitoring indicators, and the coefficient matrix is ​​composed of the integer coefficients between the time series of the key monitoring indicators and the time series of each candidate monitoring indicator in the multiple candidate monitoring indicators. The constraint conditions of the objective function may specifically include: the integer coefficients obtained by solving are integers.

[0090] For example, the first matrix A can be represented by the following formula:

[0091] A: =(Z s，1 , Z s，2 , ..., Z s，K )

[0092] The coefficient matrix can be represented as x, the second matrix can be represented as Ax, and the third matrix can be represented as Z s，0 , the objective function can be represented by the following formula:

[0093]

[0094] x i ∈{0,1}, where i=1,...,K

[0095] Wherein f(x) is a convex function as the objective function; the optimization goal of the objective function f(x) can be to solve the problem between the second matrix Ax and the third matrix Z s，0 When the straight-line distance between is the smallest, the approximate optimal solution of the coefficient matrix x; the constraint condition of the objective function can be x i ∈{0, 1}, that is, each element included in the coefficient matrix x obtained by solving can only take the value of 0 or 1.

[0096] It should be noted that in the above-mentioned embodiment, the original objective function Z can be transformed into s，0 = Ax is transformed into a convex function min x ||Ax-Z s，0 || 2 , and then use a heuristic algorithm to optimize the convex function as the objective function, and then the approximate optimal solution of the original objective function can be obtained based on the obtained approximate optimal solution of the convex function.

[0097] In addition, it should be noted that in the embodiment shown above, the conversion of the original objective function into the corresponding convex function by the second norm is merely an exemplary description and does not make any special limitation to this specification; in practical applications, those skilled in the art may also convert the original objective function into the corresponding convex function by other means, which are not illustrated one by one here.

[0098] In one embodiment shown, the heuristic algorithm may include a simulated annealing algorithm. In this case, in step 104, the objective function is optimized using the simulated annealing algorithm, which may specifically include:

[0099] Step a, generating the first matrix according to the input time series of the plurality of candidate monitoring indicators, and generating the third matrix according to the input time series of the key monitoring indicators;

[0100] Step b, using a preset initial temperature as the current temperature, and using a preset initial coefficient matrix as the current optimal coefficient matrix of the objective function, and multiplying the first matrix by the current optimal coefficient matrix to obtain a second matrix, and calculating a first straight-line distance between the second matrix and the third matrix as the objective function value corresponding to the current optimal coefficient matrix;

[0101] Step c, randomly generating a new coefficient matrix of the objective function, and multiplying the first matrix by the new coefficient matrix to obtain a new second matrix, and calculating a second straight-line distance between the new second matrix and the third matrix as the objective function value corresponding to the new coefficient matrix;

[0102] Step d: if the second straight-line distance is less than the first straight-line distance, the new coefficient matrix is ​​updated as the current optimal coefficient matrix of the objective function, and the second straight-line distance is updated as the objective function value corresponding to the current optimal coefficient matrix; if the second straight-line distance is not less than the first straight-line distance, the new coefficient matrix is ​​updated as the current optimal coefficient matrix of the objective function according to a preset probability, and the second straight-line distance is updated as the objective function value corresponding to the current optimal coefficient matrix;

[0103] Step e, iteratively executing steps c to d at the current temperature until the number of iterations reaches a preset maximum number of inner-layer cycles, and then multiplying the current temperature by a preset attenuation coefficient to obtain a new current temperature;

[0104] Step g: If the new current temperature is not less than the preset termination temperature, iteratively execute steps c to e at the new current temperature until the new current temperature is less than the termination temperature, and then determine the current optimal coefficient matrix of the objective function as the optimal coefficient matrix obtained by solving the objective function.

[0105] For example, see Figure 2 , Figure 2 FIG. 1 is a flow chart of a simulated annealing algorithm shown in an exemplary embodiment. The initial temperature can be preset as T S , the termination temperature is T e , the attenuation coefficient is α, and the maximum number of inner layer cycles is N 0 , and for convenience, we can also write f(x): = || Ax-Z s，0 || 2 In step a, a first matrix A can be generated according to the time series of the input K candidate monitoring indicators: s，1， Z s，2 , ..., Z s，K ), and the third matrix Z can be generated according to the time series of the key monitoring indicators input s，0 In step b, the current temperature t = T S , and, the initial coefficient matrix x can be randomly initialized, and the current optimal coefficient matrix And, the objective function value f(x) corresponding to the current optimal coefficient matrix can be calculated, and the objective function value corresponding to the current optimal coefficient matrix can be set In step c, a new coefficient matrix x′ of the objective function can be randomly generated according to a preset update function g(x), and the objective function value f(x′) corresponding to the new coefficient matrix x′ can be calculated; in step d, if f(x′)<f(x), the current optimal coefficient matrix And can also If f(x′)>f(x), then the current optimal coefficient matrix can be set according to the preset probability And can also In step f, at the current temperature t=T S The next iteration executes steps c to d until the number of iterations reaches the maximum number of inner loops N. 0 , then according to t = t * α, the new current temperature can be obtained; in step g, if t ≥ T e , then the steps c to e can be iteratively performed at the new current temperature t until t < T e , then the current optimal coefficient matrix of the objective function The optimal coefficient matrix is ​​determined to be the optimal coefficient matrix obtained by solving the objective function.

[0106] Among them, in the step d, the current optimal coefficient matrix is ​​set according to the preset probability Specifically, it may include: let δ = f(x′) - f(x), x ~ P(w), where,

[0107]

[0108] That is, we can use the probability Accept the new coefficient matrix x' as the current optimal coefficient matrix for the objective function.

[0109] It should be noted that in the above-described embodiments, only an exemplary description is given of a process of optimizing the objective function using a simulated annealing algorithm. For the specific implementation process of the simulated annealing algorithm, please refer to the relevant technology for the parts that are not described in detail, and no further description is given here. For example, regarding the preset values ​​of the initial temperature, the termination temperature, the attenuation coefficient, and the maximum number of inner layer cycles, those skilled in the art can flexibly set them according to engineering experience values, and this specification does not limit them; regarding the initial coefficient matrix, it can be randomly generated, or it can be flexibly set by those skilled in the art according to engineering experience values.

[0110] In one embodiment shown, in step 104, the constraint of the objective function may also include: among the multiple candidate monitoring indicators, the number of candidate monitoring indicators that have a linear relationship with the key monitoring indicator does not exceed the preset maximum number of linear associations; in other words, in the optimal coefficient matrix obtained by optimizing the objective function, the number of elements with a value of 1 does not exceed the preset maximum number of linear associations. For example, according to engineering experience, the maximum number of linear associations can be set to L, where L is a positive integer and L<<K<<N, for example, L can be a positive integer of the order of ten; the constraint of the objective function can be characterized as: x i ∈{0,1}and|{i:x i =1}|≤L.

[0111] In this case, in the process of randomly generating a new coefficient matrix of the objective function, the first number of elements can be set to 1. In implementation, in the step c, randomly generating a new coefficient matrix of the objective function can specifically include: calculating the partial derivatives of the objective function in the directions corresponding to the multiple candidate monitoring indicators, and sorting the calculated partial derivatives to determine the direction of the fastest gradient descent corresponding to the first number of candidate monitoring indicators from the partial derivatives not less than 0 based on the sorting result; wherein the first number is not greater than the maximum number of linear associations; randomly generating a new coefficient matrix of the objective function, and setting the integer coefficients corresponding to the first number of candidate monitoring indicators in the new coefficient matrix to 1.

[0112] For example, using the formula shown below:

[0113]

[0114] The partial derivatives of the objective function f(x) in the directions corresponding to the K candidate monitoring indicators can be calculated as And the calculated partial derivatives can be sorted, for example, from large to small, so that in, It is used to characterize the largest partial derivative among the calculated K partial derivatives, and so on. Further, based on the sorting result, we can let x′=x, and let x j ′=1, where That is, the partial derivative with the fastest gradient descent and the first quantity can be determined from the partial derivatives that are not less than 0. The direction corresponding to the candidate monitoring indicator; further, a new coefficient matrix of the objective function f(x) can be randomly generated, and the coefficients in the new coefficient matrix corresponding to the first quantity can be The coefficients corresponding to the candidate monitoring indicators are set to 1, so that the new coefficient matrix after the setting process can be used as the new coefficient matrix randomly generated in the step c.

[0115] In this case, in the process of randomly generating a new coefficient matrix of the objective function, in addition to setting the first number of elements to 1, other elements of random numbers and random positions may also be set to 0. In implementation, in the step c, randomly generating a new coefficient matrix of the objective function may further include: setting the integer coefficients corresponding to the candidate monitoring indicators whose partial derivatives in the corresponding directions are less than 0 in the randomly generated new coefficient matrix to 0.

[0116] For example, a new coefficient matrix of the objective function f(x) is randomly generated, and the coefficients in the new coefficient matrix corresponding to the first quantity are After the coefficient corresponding to the candidate monitoring indicator is set to 1, we can calculate Let x j ′=0,j∈S_; that is, if the calculated partial derivative of the objective function in the direction corresponding to a candidate monitoring indicator is less than 0, the coefficient corresponding to the candidate monitoring indicator in the randomly generated new coefficient matrix can be set to 0, thereby the new coefficient matrix after the setting to 0 can be used as the new coefficient matrix randomly generated in the step c.

[0117] It should be noted that in the related art, when solving the quadratic integer programming problem by using the simulated annealing algorithm, the uniform gradient descent method can usually be used to add random disturbances to the current temperature level to generate a new solution to the objective function, resulting in the generated new solution being a non-integer and having a slow convergence speed. In the technical solution of this specification, when solving the quadratic integer programming problem based on the framework of the simulated annealing algorithm, a new strategy for randomly generating new solutions is also designed. The difference lies in that: on the one hand, since the solution space of the objective function is limited to integer numbers, the new solution of the objective function can be randomly generated by setting the random number and random position elements in the new coefficient matrix to 1 and / or 0; on the other hand, by selecting the partial direction with the fastest gradient descent for calculation, the convergence speed can be accelerated and the efficiency of solving the objective function can be improved.

[0118] In addition, it should be noted that in the above embodiments, the first quantity is taken as This is only an exemplary description and does not impose any special limitation on this specification. For example, the first number may also be L, or the first number may be any positive integer less than L.

[0119] In one embodiment shown, in step c, the method may also include: randomly masking non-zero elements in the new coefficient matrix; performing random sampling in a closed interval from 0 to a preset value to obtain a second number; wherein the difference between the maximum linear correlation number and the number of integer coefficients with a value of 1 in the new coefficient matrix is ​​greater than or equal to the preset value; extracting a second number of candidate monitoring indicators from the multiple candidate monitoring indicators without replacement; and setting the integer coefficients corresponding to the second number of candidate monitoring indicators in the new coefficient matrix to 1.

[0120] For example, after randomly generating a new coefficient matrix x′, the non-zero elements in the new coefficient matrix x′ can be randomly masked first; further, L′~μ[0, L-|{i:x i ′=1}|], where |{i:x i′=1}| is used to characterize the number of integer coefficients with a value of 1 in the new coefficient matrix x′, that is, it can be found in the closed interval [0, L-|{i:x i ′=1}|] to obtain a second number L′; further, L′ elements can be selected without replacement, and x j ′=1,where j=j 1 , ..., j L′ That is, the L′ elements selected from the new coefficient matrix can be set to 1 and used as the new coefficient matrix randomly generated in step c.

[0121] In one possible embodiment, the preset maximum number of linear associations L can be 8. If the number of elements with a value of 1 in the randomly generated new coefficient matrix x′ is 5, the preset value can be a positive integer less than 3. Furthermore, random sampling can be performed in the closed interval [0, 3] to obtain the second number L′, that is, the second number L′ can be any positive integer greater than or equal to 0 and less than or equal to 3.

[0122] It should be noted that, in the above-described embodiment, although the convergence speed can be accelerated by selecting the partial direction with the fastest gradient descent for calculation, it is possible that the new coefficient matrix randomly generated in each inner loop is the same, resulting in the approximate optimal solution of the objective function no longer changing, and being mistakenly believed to have found the global optimal solution; therefore, the non-zero elements can be randomly masked, and the second number of elements can be selected and set to 1, which can speed up the convergence speed while avoiding falling into the local optimal solution. Moreover, for the new coefficient matrix updated by the above processing method, the number of coefficients with a value of 1 included therein is still random and less than the maximum linear association number L.

[0123] In another illustrated embodiment, in the step c, the method further includes: if the randomly generated new coefficient matrix is ​​the same as the current optimal coefficient matrix, then re-randomly initializing and generating a new coefficient matrix for the objective function.

[0124] In step 106, after optimizing the objective function and obtaining the integer coefficients that characterize whether there is a linear relationship between the time series of the key monitoring indicator and the time series of each candidate monitoring indicator among the multiple candidate monitoring indicators, the candidate monitoring indicators that have a linear relationship with the key monitoring indicator can be determined from the multiple candidate monitoring indicators based on the integer coefficients.

[0125] For example, when optimizing the objective function f(x), we can obtain the time series Z representing the key monitoring indicators. s，0and the time series Z of K candidate monitoring indicators s，1 , Z s，2 ,……,Z s，K Is there a linear relationship between the integral coefficients [x 1 , x 2 , ..., x k ]After that, the time series Z of the key monitoring indicators is characterized according to a certain integral coefficient s，0 There is a linear relationship between the candidate monitoring indicator and the corresponding time series of the candidate monitoring indicator, and it can be determined that the candidate monitoring indicator has a linear relationship with the key monitoring indicator, and the time series Z of the key monitoring indicator is characterized according to a certain integral coefficient s，0 If there is no linear relationship between the time series of the corresponding candidate monitoring indicator, it can be determined that there is no linear relationship between the candidate monitoring indicator and the key monitoring indicator; that is, it can be approximately considered that the sum of the time series of the candidate monitoring indicators that have a linear relationship with the key monitoring indicator is equal to the time series of the key monitoring indicator.

[0126] In one embodiment shown, the value of the integral point coefficient may include 0 and 1; wherein, when the integral point coefficient takes the value of 0, it represents that there is no linear relationship between the time series of the candidate monitoring indicator corresponding to the integral point coefficient and the time series of the key monitoring indicator; when the integral point coefficient takes the value of 1, it represents that there is a linear relationship between the time series of the candidate monitoring indicator corresponding to the integral point coefficient and the time series of the key monitoring indicator; in this case, in the step 106, according to the integral point coefficient, the candidate monitoring indicator having a linear relationship with the key monitoring indicator is determined from the multiple candidate monitoring indicators, which may specifically include: determining the integral point coefficient whose value is 1 in the coefficient matrix; determining the candidate monitoring indicator corresponding to the integral point coefficient whose value is 1 in the coefficient matrix as the candidate monitoring indicator having a linear relationship with the key monitoring indicator.

[0127] For example, if the integer coefficient [x 1 , x 2 , x 3 ]=[1,0,1], then the integral coefficient x with the value of 1 can be determined 1 、x 3 There is a linear relationship between the corresponding candidate monitoring indicator and the key monitoring indicator, and the integral coefficient x is 0. 2 There is no linear relationship between the corresponding candidate monitoring indicators and the key monitoring indicators; that is, it can be approximately considered that Z s，0 =Z s，1 +Z s，3 .

[0128] It can be seen from the above technical scheme that by taking the time series of the key monitoring indicator and the time series of multiple candidate monitoring indicators as input data, and inputting them into the convex function as the objective function for optimization solution, the candidate monitoring indicators that have a linear relationship with the key monitoring indicator can be determined from the multiple candidate monitoring indicators according to the integer coefficients obtained by the solution, thereby realizing automatic mining of the linear relationship between different monitoring indicators; and in the scenario where the monitoring data includes massive monitoring indicators, it can not only reduce the requirements for user experience and technical level, save labor costs and time costs, and improve mining efficiency, but also perform linear relationship mining on monitoring indicators across code logic.

[0129] Corresponding to the above-mentioned embodiment of the method for processing monitoring data, this specification also provides an embodiment of a device for processing monitoring data.

[0130] See also Figure 3 , Figure 3 It is a schematic diagram of the structure of an electronic device in which a processing device for monitoring data is located, which is shown as an exemplary embodiment. At the hardware level, the device includes a processor 302, an internal bus 304, a network interface 306, a memory 308, and a non-volatile memory 310, and of course may also include hardware required for other services. One or more embodiments of this specification can be implemented based on software, such as the processor 302 reading the corresponding computer program from the non-volatile memory 310 into the memory 308 and then running it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0131] See also Figure 4 , Figure 4 FIG. 1 is a block diagram of a monitoring data processing device according to an exemplary embodiment. The monitoring data processing device can be applied to Figure 3 In the electronic device shown in the figure, the technical solution of this specification is implemented. Wherein, the monitoring data processing device may include:

[0132] An acquisition unit 402 is used to acquire a time series of a key monitoring indicator and a time series of multiple candidate monitoring indicators;

[0133] A solving unit 404 is used to adopt a heuristic algorithm, take the time series of the key monitoring indicator and the time series of the multiple candidate monitoring indicators as input data, and input them into a preset convex function as an objective function for optimization solution, so as to obtain an integer coefficient representing whether there is a linear relationship between the time series of the key monitoring indicator and the time series of each candidate monitoring indicator in the multiple candidate monitoring indicators;

[0134] The determination unit 406 is configured to determine, according to the integral point coefficient, a candidate monitoring indicator having a linear relationship with the key monitoring indicator from among the multiple candidate monitoring indicators.

[0135] In this embodiment, the optimization objectives of the objective function include:

[0136] Solve for an optimal coefficient matrix multiplied with the first matrix when the straight-line distance between a second matrix formed by multiplying the first matrix and the coefficient matrix and a third matrix formed by the time series of the key monitoring indicators is minimized; wherein the first matrix is ​​formed by the time series of the multiple candidate monitoring indicators, and the coefficient matrix is ​​formed by the integer coefficients between the time series of the key monitoring indicators and the time series of each of the multiple candidate monitoring indicators;

[0137] The constraint condition of the objective function includes: the integer point coefficient obtained by solving is an integer.

[0138] In this embodiment, the value of the integral coefficient includes 0 and 1; wherein, when the integral coefficient is 0, it indicates that there is no linear relationship between the time series of the candidate monitoring indicator corresponding to the integral coefficient and the time series of the key monitoring indicator; when the integral coefficient is 1, it indicates that there is a linear relationship between the time series of the candidate monitoring indicator corresponding to the integral coefficient and the time series of the key monitoring indicator;

[0139] The determining unit 406 is specifically configured to:

[0140] Determine the integer point coefficients whose values ​​are 1 in the coefficient matrix;

[0141] The candidate monitoring indicator corresponding to the integer coefficient whose value is 1 in the coefficient matrix is ​​determined as the candidate monitoring indicator having a linear relationship with the key monitoring indicator.

[0142] In this embodiment, the heuristic algorithm includes a simulated annealing algorithm;

[0143] The solving unit 404 is specifically used for:

[0144] Step a, generating the first matrix according to the input time series of the plurality of candidate monitoring indicators, and generating the third matrix according to the input time series of the key monitoring indicators;

[0145] Step b, using a preset initial temperature as the current temperature, and using a preset initial coefficient matrix as the current optimal coefficient matrix of the objective function, and multiplying the first matrix by the current optimal coefficient matrix to obtain a second matrix, and calculating a first straight-line distance between the second matrix and the third matrix as the objective function value corresponding to the current optimal coefficient matrix;

[0146] Step c, randomly generating a new coefficient matrix of the objective function, and multiplying the first matrix by the new coefficient matrix to obtain a new second matrix, and calculating a second straight-line distance between the new second matrix and the third matrix as the objective function value corresponding to the new coefficient matrix;

[0147] Step d: if the second straight-line distance is less than the first straight-line distance, the new coefficient matrix is ​​updated as the current optimal coefficient matrix of the objective function, and the second straight-line distance is updated as the objective function value corresponding to the current optimal coefficient matrix; if the second straight-line distance is not less than the first straight-line distance, the new coefficient matrix is ​​updated as the current optimal coefficient matrix of the objective function according to a preset probability, and the second straight-line distance is updated as the objective function value corresponding to the current optimal coefficient matrix;

[0148] Step e, iteratively executing steps c to d at the current temperature until the number of iterations reaches a preset maximum number of inner-layer cycles, and then multiplying the current temperature by a preset attenuation coefficient to obtain a new current temperature;

[0149] Step g: If the new current temperature is not less than the preset termination temperature, iteratively execute steps c to e at the new current temperature until the new current temperature is less than the termination temperature, and then determine the current optimal coefficient matrix of the objective function as the optimal coefficient matrix obtained by solving the objective function.

[0150] In this embodiment, the constraint condition of the objective function also includes: among the multiple candidate monitoring indicators, the number of candidate monitoring indicators that have a linear relationship with the key monitoring indicator does not exceed a preset maximum number of linear associations;

[0151] The solving unit 404 is specifically used for:

[0152] Calculating partial derivatives of the objective function in directions corresponding to the plurality of candidate monitoring indicators, respectively, and sorting the calculated partial derivatives, so as to determine, based on the sorting results, directions of fastest gradient descent corresponding to a first number of candidate monitoring indicators from the partial derivatives that are not less than 0; wherein the first number is not greater than the maximum number of linear associations;

[0153] A new coefficient matrix of the objective function is randomly generated, and the integer coefficients corresponding to the first number of candidate monitoring indicators in the new coefficient matrix are set to 1.

[0154] In this embodiment, the solving unit 404 is further configured to:

[0155] Randomly masking the non-zero elements in the new coefficient matrix;

[0156] Performing random sampling in a closed interval from 0 to a preset value to obtain a second number; wherein the difference between the maximum linear correlation number and the number of integer coefficients with a value of 1 in the new coefficient matrix is ​​greater than or equal to the preset value;

[0157] extracting a second number of candidate monitoring indicators from the plurality of candidate monitoring indicators without replacement;

[0158] The integer point coefficients corresponding to the second number of candidate monitoring indicators in the new coefficient matrix are set to 1.

[0159] In this embodiment, the device further comprises a screening unit, which is used to:

[0160] Get the time series of multiple monitoring indicators extracted from the monitoring log file;

[0161] Calculating the correlation coefficients between the time series of the multiple monitoring indicators and the time series of the key monitoring indicator respectively; wherein the correlation coefficient represents the degree of correlation between the time series of each monitoring indicator in the multiple monitoring indicators and the time series of the key monitoring indicator;

[0162] According to the correlation coefficient, at least some of the monitoring indicators whose time series have the highest correlation coefficient with the time series of the key monitoring indicator are screened out from the multiple monitoring indicators as the multiple candidate monitoring indicators.

[0163] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0164] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. Ordinary technicians in this field can understand and implement it without paying creative work.

[0165] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0166] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0167] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0168] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0169] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0170] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0171] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "one", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0172] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0173] The above description is merely a preferred embodiment of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the scope of protection of one or more embodiments of the present specification.

Claims

1. A method for processing monitored data, the method comprises: obtaining time series of key monitored metrics and time series of multiple candidate monitored metrics; using a heuristic algorithm, taking the time series of the key monitored metric and the time series of the multiple candidate monitored metrics as input data, and inputting them into a preset convex function serving as an objective function for optimization solution, so as to obtain integral point coefficients indicating whether there is a linear relationship between the time series of the key monitored metric and the time series of each candidate monitored metric among the multiple candidate monitored metrics; determining, according to the integral point coefficients, candidate monitored metrics having a linear relationship with the key monitored metric from the multiple candidate monitored metrics; The optimization objective of the objective function includes: solving the integral point coefficients corresponding to each candidate monitored metric among the multiple candidate monitored metrics when the sum of the time series of some candidate monitored metrics among the multiple candidate monitored metrics is closest to the time series of the key monitored metric.

2. The method according to claim 1, the optimization objective of the objective function includes: solving an optimal coefficient matrix multiplied by the first matrix when the straight-line distance between a second matrix formed by multiplying a first matrix and a coefficient matrix and a third matrix formed by the time series of the key monitored metric is the smallest; wherein, the first matrix is formed by the time series of the multiple candidate monitored metrics, and the coefficient matrix is formed by the integral point coefficients between the time series of the key monitored metric and the time series of each candidate monitored metric among the multiple candidate monitored metrics; The constraint condition of the objective function includes: the obtained integral point coefficients are integers.

3. The method according to claim 2, the values of the integral point coefficients include 0 and 1; wherein, when the value of the integral point coefficient is 0, it indicates that there is no linear relationship between the time series of the candidate monitored metric corresponding to this integral point coefficient and the time series of the key monitored metric; when the value of the integral point coefficient is 1, it indicates that there is a linear relationship between the time series of the candidate monitored metric corresponding to this integral point coefficient and the time series of the key monitored metric; Determining, according to the integral point coefficients, candidate monitored metrics having a linear relationship with the key monitored metric from the multiple candidate monitored metrics includes: determining the integral point coefficients with a value of 1 in the coefficient matrix; determining the candidate monitored metrics corresponding to the integral point coefficients with a value of 1 in the coefficient matrix as the candidate monitored metrics having a linear relationship with the key monitored metric.

4. The method according to claim 3, the heuristic algorithm includes a simulated annealing algorithm; Using the simulated annealing algorithm to perform optimization solution on the objective function includes: Step a, generating the first matrix according to the input time series of the multiple candidate monitored metrics, and generating the third matrix according to the input time series of the key monitored metric; Step b: Take the preset initial temperature as the current temperature, and take the preset initial coefficient matrix as the current optimal coefficient matrix of the objective function. Multiply the first matrix by the current optimal coefficient matrix to obtain a second matrix, and calculate the first straight-line distance between the second matrix and the third matrix as the objective function value corresponding to the current optimal coefficient matrix; Step c: Randomly generate a new coefficient matrix of the objective function. Multiply the first matrix by the new coefficient matrix to obtain a new second matrix, and calculate the second straight-line distance between the new second matrix and the third matrix as the objective function value corresponding to the new coefficient matrix; Step d: If the second straight-line distance is less than the first straight-line distance, update the new coefficient matrix as the current optimal coefficient matrix of the objective function, and update the second straight-line distance as the objective function value corresponding to the current optimal coefficient matrix; if the second straight-line distance is not less than the first straight-line distance, update the new coefficient matrix as the current optimal coefficient matrix of the objective function with a preset probability, and update the second straight-line distance as the objective function value corresponding to the current optimal coefficient matrix; Step e: Iteratively execute Step c to Step d at the current temperature until the number of iterations reaches the preset maximum number of inner loop times, then multiply the current temperature by the preset attenuation coefficient to obtain a new current temperature; Step g: If the new current temperature is not less than the preset termination temperature, iteratively execute Step c to Step e at the new current temperature until the new current temperature is less than the termination temperature, then determine the current optimal coefficient matrix of the objective function as the optimal coefficient matrix obtained by solving the objective function.

5. The method according to claim 4, the constraint condition of the objective function further includes: Among the multiple candidate monitoring indicators, the number of candidate monitoring indicators having a linear relationship with the key monitoring indicator does not exceed the preset maximum linear correlation number; Randomly generating a new coefficient matrix of the objective function includes: Calculate the partial derivatives of the objective function in the directions corresponding to the multiple candidate monitoring indicators respectively, and sort the calculated partial derivatives. Based on the sorting result, determine the direction with the fastest gradient descent corresponding to the first number of candidate monitoring indicators from the partial derivatives not less than 0; wherein, the first number is not greater than the maximum linear correlation number; Randomly generate a new coefficient matrix of the objective function, and set the integral coefficients corresponding to the first number of candidate monitoring indicators in the new coefficient matrix to 1.

6. The method according to claim 5, the method further includes: Randomly occlude the non-zero elements in the new coefficient matrix; Randomly sample within the closed interval from 0 to the preset value to obtain a second number; wherein, the difference between the maximum linear correlation number and the number of integral coefficients with a value of 1 in the new coefficient matrix is greater than or equal to the preset value; Extract a second quantity of candidate monitoring metrics without replacement from the multiple candidate monitoring metrics; Set the integral coefficients corresponding to the second quantity of candidate monitoring metrics in the new coefficient matrix to 1.

7. The method according to claim 1, the method further comprises: Obtain the time series of multiple monitoring metrics extracted from the monitoring log file; Calculate the correlation coefficients between the time series of the multiple monitoring metrics and the time series of the key monitoring metric respectively; wherein, the correlation coefficient characterizes the correlation degree between the time series of each monitoring metric in the multiple monitoring metrics and the time series of the key monitoring metric; According to the correlation coefficients, screen out at least some of the monitoring metrics from the multiple monitoring metrics whose correlation coefficients between the time series and the time series of the key monitoring metric are the highest, as the multiple candidate monitoring metrics.

8. A monitoring data processing device, the device comprises: An acquisition unit, configured to acquire the time series of a key monitoring metric and the time series of multiple candidate monitoring metrics; A solution unit, configured to use a heuristic algorithm to input the time series of the key monitoring metric and the time series of the multiple candidate monitoring metrics as input data into a convex function preset as an objective function for optimization solution, so as to obtain integral coefficients characterizing whether there is a linear relationship between the time series of the key monitoring metric and the time series of each candidate monitoring metric in the multiple candidate monitoring metrics; A determination unit, configured to determine, according to the integral coefficients, the candidate monitoring metrics that have a linear relationship with the key monitoring metric from the multiple candidate monitoring metrics; The optimization objective of the objective function includes: solving the integral coefficients corresponding to each candidate monitoring metric in the multiple candidate monitoring metrics when the sum of the time series of some candidate monitoring metrics in the multiple candidate monitoring metrics is closest to the time series of the key monitoring metric.

9. An electronic device, comprising a communication interface, a processor, a memory and a bus, and the communication interface, the processor and the memory are interconnected through the bus; Machine-readable instructions are stored in the memory, and the processor executes the method according to any one of claims 1 to 7 by calling the machine-readable instructions.

10. A machine-readable storage medium, storing machine-readable instructions, and when the machine-readable instructions are called and executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images