A system performance bottleneck detection method based on reinforcement learning

Abstract

The present application belongs to the field of abnormal root cause analysis, and relates to a system performance bottleneck detection method based on reinforcement learning. The system performance bottleneck detection method is as follows: first, extract system performance index data; second, find the first abnormal time period and abnormal dimension using threshold method; third, perform causal inference on abnormal data and further root cause analysis. The present application can effectively solve the system performance bottleneck detection problem in a high-load environment, help system administrators identify and solve problems faster, reduce the risk of system crashes, and improve the stability and reliability of the system, so that the system can respond to user requests faster and improve user satisfaction. The present application can be applied to a wider range of root cause analysis problems, effectively helping operation and maintenance personnel solve system performance bottleneck detection problems using artificial intelligence methods, and has good applicability and robustness.

Description

Technical Field

[0001] This invention belongs to the field of anomaly root cause analysis and relates to a system performance bottleneck detection method based on reinforcement learning. Background Technology

[0002] With the continuous development of computer technology and the expanding application scope of computer systems, the requirements for computer system performance are also increasing. System performance bottlenecks refer to the root causes of declining computer system performance; therefore, accurately and quickly detecting and diagnosing system performance bottlenecks has become one of the important research directions in computer system performance analysis.

[0003] System performance bottleneck detection aims to identify and locate performance bottlenecks in a system by monitoring and analyzing performance data from various aspects, thereby achieving performance optimization. Traditional computer system performance analysis methods are typically based on statistical methods or rule engines, including performance monitoring, bottleneck testing, log analysis, system profiling, and data analysis. While these methods can provide a certain level of diagnostic accuracy, their efficiency and precision are insufficient to meet the requirements of modern computer system performance analysis. To address these issues, researchers have begun exploring the use of artificial intelligence technologies to improve the efficiency and accuracy of computer system performance analysis.

[0004] In practical applications, root cause analysis is a commonly used technique that helps engineers quickly and accurately identify the root causes of performance bottlenecks. Within root cause analysis, machine learning techniques can help engineers automate data analysis and pinpoint the true causes of system performance bottlenecks. Therefore, in recent years, an increasing number of researchers have begun exploring methods for applying machine learning techniques to root cause analysis.

[0005] Causal inference is a widely used technique in root cause analysis. The basic idea of ​​causal inference is to infer the root cause of a system by observing the causal relationships between variables. Specifically, causal inference first identifies the relationships between variables, and then infers the causal relationships between them based on these relationships. In root cause analysis, causal inference can help engineers quickly and accurately identify performance bottlenecks in a system. Because performance bottlenecks are usually caused by the combined effects of multiple factors, only by comprehensively considering the relationships between these factors can the root cause of the performance bottleneck be truly found. Causal inference can effectively handle such complex relationships, thereby achieving accurate root cause analysis.

[0006] In recent years, reinforcement learning has been found to be applicable to causal inference and has achieved excellent performance. Inspired by this, this invention applies reinforcement learning to root cause analysis algorithms and experimentally demonstrates its effectiveness.

[0007] In the field of root cause analysis, the technical advantages of this reinforcement learning-based system performance bottleneck detection method are as follows: This method can identify abnormal system performance conditions, extract useful features and patterns from complex data, mine causal relationships between data, and efficiently and accurately identify the bottlenecks causing system performance anomalies. Furthermore, this method has good scalability and reusability, and can be applied to root cause analysis tasks in different fields. Therefore, this reinforcement learning-based system performance bottleneck detection method is a root cause analysis method with broad application prospects and practical value. Summary of the Invention

[0008] This invention addresses the problem of system performance bottleneck detection by providing a root cause analysis method based on reinforcement learning. This method enables the automatic detection of performance bottlenecks in systems under high load conditions, thereby effectively improving system stability and performance.

[0009] To achieve the specific objectives, the technical solution of the present invention is as follows:

[0010] A system performance bottleneck detection method based on reinforcement learning, the overall flowchart is as follows: Figure 1 As shown, the steps are as follows:

[0011] The first step is to extract system performance metrics data:

[0012] In high-stress testing environments, various performance metrics are collected, including but not limited to CPU utilization, memory usage, operating system kernel call counts, disk I / O, virtual machine resource usage, and network transmission speed, to identify system performance bottlenecks. This data is then extracted and normalized for more accurate analysis.

[0013] The second step is to use the threshold method to identify the initial abnormal time period and abnormal dimension.

[0014] 2.1 Calculate the high and low thresholds for each dimension of the extracted data. The threshold calculation method uses the n-sigma method.

[0015] 2.2 Identify anomalies based on threshold values ​​for each dimension, and find the earliest anomaly time point and the corresponding anomaly dimension.

[0016] 2.3 Divide the time period before and after the earliest anomaly point. Data within this time period will be used as input data for the subsequent root cause analysis algorithm.

[0017] The third step is to perform causal inference on the abnormal data and then conduct root cause analysis.

[0018] 3.1 The Actor-Critic algorithm is used to discover causal relationships between different dimensions within anomaly time periods. The anomalous data obtained in step two is input into the Actor-Critic algorithm model. After training until convergence, the output is a graph adjacency matrix with side length equal to the number of dimensions of the input data, i.e., the causal graph. The structure of the Actor-Critic-based causal inference algorithm is as follows: Figure 2 As shown; the Actor structure in the Actor-Critic network is as follows: Figure 3 As shown; the Attention module structure in the GAT network is as follows Figure 4 As shown in the figure; the reward value, the evaluation metric for algorithm training, changes iteratively with the number of training iterations. Figure 5 As shown.

[0019] The fractional function for the graph structure is as follows: First, the BIC score of the cause-effect graph is defined, with the following formula:

[0020]

[0021] The first term on the right is the likelihood function. Let represent the predicted value of the k-th item in the i-th dimension of the observed sample x, n represent the number of samples (i.e., the time length), d represent the number of dimensions, and ε is a decimal value to avoid the logarithm being zero; here, it is taken as 10. -8 The second term in the right-hand equation is a penalty term, and m represents the number of edges in the graph.

[0022] The graph scores were then normalized to obtain the final scores for the causal graph. The specific formula is as follows:

[0023] S(G)=(S BIC (G)-S l ) / (S u -S l )

[0024] Where S l and S u These are the high and low thresholds for the causal graph score, S. l S is the fraction of a directed graph where all elements except the diagonal are 1. u The score for a graph where all values ​​are 0.

[0025] The GAT network structure in Actor is as follows: The GAT network in Actor is a neural network model based on an attention mechanism. In this model, GAT is composed of multiple stacked attention modules, and the specific composition of the attention modules is as follows: First, a one-dimensional convolutional layer is used to extract features from the input sequence to obtain a feature vector; then, the feature vector is passed through two convolutional layers to calculate attention coefficients. The calculation of attention coefficients requires adding the outputs of the two convolutional layers and activating them, followed by a softmax transformation to ensure that the sum of the attention coefficients is 1; the attention coefficients and the feature sequence are passed through a dropout layer, and some elements are randomly set to zero; next, the feature sequence is weighted and summed according to the attention coefficients to obtain the encoding of the relationship features of all embedded nodes; finally, the feature vector of the input sequence is added to the output vector using a residual connection, and a nonlinear transformation is performed on it using an activation function to obtain the final output vector.

[0026] The features extracted by the GAT network in the Actor are used to generate an adjacency matrix in a graph structure through bilinear product and Bernoulli sampling, as follows: In the Actor structure, the feature encoding extracted by the GAT network is bilinearly producted with the learnable weights to obtain the final adjacency probability distribution. The formula for calculating the bilinear product is as follows:

[0027]

[0028] Where W is the learning weight matrix, x i x j p is the vector of the i-th and j-th dimensions obtained through GAT encoding. ij (W) represents the adjacency probability from node i to j. Finally, Bernoulli sampling is performed on the probability distribution between nodes to convert the probability between each pair of nodes into binary samples to obtain the adjacency matrix of the generated graph, i.e., the causal graph. This process masks the current node by subtracting a large negative value (i.e., 100000000) from the probability matrix and multiplying it by a mask, ensuring that it does not connect to itself.

[0029] 3.2 Calculate the Pearson correlation coefficient of the input data and take its absolute value to obtain the correlation matrix.

[0030] 3.3 The transition probability matrix of the edges is obtained by performing forward, backward, and self-transitions based on the causal graph and the correlation matrix.

[0031] 3.4 Perform a random walk based on the transition probability matrix of the edges to obtain the final list of abnormal root cause scores.

[0032] The beneficial effects of this invention are:

[0033] This method effectively addresses the problem of system performance bottleneck detection under high load environments, helping system administrators identify and resolve issues more quickly, reducing the risk of system crashes, and improving system stability and reliability. This allows the system to respond to user requests faster, increasing user satisfaction. This invention can be applied to a wider range of root cause analysis problems, effectively assisting operations and maintenance personnel in solving system performance bottleneck detection issues using artificial intelligence methods, demonstrating excellent applicability and robustness. Attached Figure Description

[0034] Figure 1 A flowchart illustrating a system performance bottleneck detection method based on reinforcement learning;

[0035] Figure 2 A schematic diagram of the causal inference algorithm based on Actor-Critic;

[0036] Figure 3 A schematic diagram of the Actor structure in the Actor-Critic network;

[0037] Figure 4 Schematic diagram of the Attention module structure in the GAT network;

[0038] Figure 5 A diagram illustrating the changes in reward values ​​generated by the Actor-Critic algorithm's causal graph. Detailed Implementation

[0039] To make the technical solution of the present invention clearer, the present invention will be further described below with reference to the accompanying drawings. The present invention is implemented in specific steps:

[0040] A system performance bottleneck detection method based on reinforcement learning, the overall flowchart is as follows: Figure 1 As shown, the steps are as follows:

[0041] The first step is to extract system performance metrics data.

[0042] System performance testing is conducted under high-pressure environments to collect various performance metrics. To obtain accurate and reliable performance data, the testing environment needs to resemble a real production environment; for example, load testing tools can be used to simulate real request traffic and concurrent user counts.

[0043] The second step is to use the threshold method to identify the earliest starting time period and the anomaly dimension.

[0044] (1) Calculate the high and low thresholds for each dimension of the extracted data.

[0045] The high and low thresholds for each dimension are calculated using the n-sigma thresholding method. The specific calculation formula is as follows:

[0046] y low / high =y mean ±n*y std

[0047] Where y high and y low These represent the high and low thresholds for a certain dimension of data, y mean and y std These represent the mean and standard deviation of the data in this dimension, respectively, with n being a variable parameter that takes different values ​​depending on the dataset.

[0048] (2) Determine anomalies based on the thresholds of each dimension, and find the earliest abnormal time point and the corresponding abnormal dimension. For each dimension of the data, find the abnormal time points that are greater than the high threshold and less than the low threshold, and compare their sizes. The time point with the smallest value is the earliest abnormal time point, and record the time point and its corresponding abnormal dimension.

[0049] (3) Divide the abnormal time period before and after the earliest abnormal point. The data within this time period will be used as the input data for the subsequent root cause analysis algorithm.

[0050] The third step is to perform root cause analysis on the abnormal data.

[0051] (1) The Actor-Critic algorithm was used to discover the causal relationships between different dimensions during abnormal time periods.

[0052] The abnormal data obtained in the second step is input into the Actor-Critic algorithm model, and the model structure diagram is as follows: Figure 2 As shown in the figure. This part of the algorithm converges after approximately 30 rounds of training (the convergence speed varies depending on the dataset), and outputs a graph adjacency matrix with a side length equal to the dimension of the input data, i.e., a causal graph. The reward value, the evaluation metric for this algorithm, changes iteratively with the number of training iterations, as shown in the figure. Figure 5 As shown.

[0053] First, we define a fractional function to evaluate the quality of a cause-effect graph. The BIC score formula for a cause-effect graph is defined as follows:

[0054]

[0055] The first term on the right is the likelihood function. Let represent the predicted value of the k-th item in the i-th dimension of the observed sample x, n represent the number of samples (i.e., the time length), d represent the number of dimensions, and ε is a decimal value to avoid the logarithm being zero; here, it is taken as 10. -8 The second term in the right-hand equation is a penalty term, and m represents the number of edges in the graph.

[0056] The graph scores were then normalized to obtain the final scores for the causal graph. The specific formula is as follows:

[0057] S(G)=(S BIC (G)-S l ) / (S u -S l )

[0058] Where S l and S u These are the high and low thresholds for the causal graph score, S. l S is the fraction of a directed graph where all elements except the diagonal are 1. u The score for a graph where all values ​​are 0.

[0059] Next, a network model based on the Actor-Critic algorithm is used to search for the causal graph with the best score. The Actor network is responsible for generating the graph, and the Critic network is responsible for judging the quality of the generated graph. In the network framework, the Actor uses a graph attention mechanism (GAT) to extract feature relationships and generates the adjacency matrix of the graph structure through bilinear product and Bernoulli sampling. The Critic uses two fully connected layers to predict the score of the generated graph. The Actor network structure is as follows: Figure 3 As shown.

[0060] The GAT network is a neural network model based on the attention mechanism. In this model, GAT is composed of multiple stacked attention modules, and the network structure of the attention modules is as follows: Figure 4 As shown, the specific components are as follows: First, a one-dimensional convolutional layer is used to extract features from the input sequence to obtain a feature vector; then, the feature vector is passed through two convolutional layers to calculate attention coefficients. The calculation of attention coefficients requires adding the outputs of the two convolutional layers and activating them, followed by a softmax transformation to ensure that the sum of the attention coefficients is 1; the attention coefficients and the feature sequence are passed through a dropout layer, where a portion of the elements are randomly set to zero; next, the feature sequence is weighted and summed according to the attention coefficients to obtain the encoding of all embedded node relationship features; finally, a residual connection is used to add the feature vector of the input sequence to the output vector, and an activation function is used to perform a non-linear transformation to obtain the final output vector.

[0061] In the Actor structure, the feature codes extracted by the GAT network are bilinearly productd with the learnable weights to obtain the final adjacency probability distribution. The formula for calculating the bilinear product is as follows:

[0062]

[0063] Where W is the learning weight matrix, x i x j p is the vector of the i-th and j-th dimensions obtained through GAT encoding. ij(W) represents the adjacency probability from node i to j. Finally, Bernoulli sampling is performed on the probability distribution between nodes to convert the probability between each pair of nodes into binary samples to obtain the adjacency matrix of the generated graph, i.e., the causal graph. This process masks the current node by subtracting a large negative value (i.e., 100000000) from the probability matrix and multiplying it by a mask, ensuring that it does not connect to itself.

[0064] The training objective of the Actor-Critic algorithm is to maximize the reward. In the definition of the reward, in addition to the score of the generated graph, an acyclic constraint is added. The specific formula is as follows:

[0065] rewards: = -S(G) - αS(A)

[0066] Where α≥0 is the penalty parameter, A is the generating graph matrix, and S(A) is the acyclic constraint, defined as:

[0067] S(A): = trace(e A )-d

[0068] (2) Calculate the Pearson correlation coefficient of the input data and take its absolute value to obtain the correlation matrix.

[0069] (3) Obtain the edge transition probability matrix by performing forward, backward, and self-transitions based on the causal graph and correlation matrix. Taking the abnormal dimension identified in step two as the starting node, firstly, multiply all edges in the causal graph by the correlation coefficient between the end node and the starting node of that edge to obtain the node transition probability matrix P. Normalize matrix P by the first dimension. Then, traverse all outgoing edges, calculate the probability of forward transition based on P, update the edge transition probability matrix M, and normalize M by the third dimension. Traverse all incoming edges, calculate the probability of backward transition to update M and normalize it, then multiply by the backward transition coefficient. Generate self-loops, obtain the in-situ transition probability and normalize it, finally obtaining the edge transition probability matrix. During the calculation process, the influence strength coefficients of the previous and next nodes need to be set to 0.1 and 0.2 respectively.

[0070] (4) Perform a random walk based on the transition probability matrix of the edges to obtain the list of abnormal root cause scores.

[0071] Starting with the anomaly dimension identified in step two, the algorithm randomly moves to the next node each time, according to the given edge transition probability matrix. During this process, if a node cannot be moved to another, the algorithm terminates the current round early to avoid infinite loops. After multiple rounds, the number of times each node is visited is counted, serving as its relevance score. Finally, a list is created by mapping the relevance scores to their corresponding service names, sorted in descending order of relevance score; this is the final list of root cause anomaly scores.

[0072] PR@k represents the probability of the true value among the first k indicators; a larger value indicates a higher accuracy in root cause analysis. Table 1 shows the accuracy of this method on different datasets. As can be seen, this method can effectively detect the root cause of anomalies, thereby helping operations personnel analyze performance bottlenecks.

[0073] Table 1. Accuracy of root cause analysis on different datasets

[0074]

Claims

1. A system performance bottleneck detection method based on reinforcement learning, characterized in that, The steps are as follows: The first step is to extract system performance metrics data: In a high-stress testing environment, various performance metrics are collected, including CPU utilization, memory usage, operating system kernel call count, disk I / O, virtual machine resource usage, and network transmission speed, in order to identify system performance bottlenecks. These data are extracted and normalized for more accurate analysis. The second step is to use the threshold method to identify the earliest starting time period and the anomaly dimension. 2.1 Calculate the high and low thresholds for each dimension of the extracted data; the threshold calculation method uses the n-sigma method; 2.2 Identify anomalies based on threshold values ​​for each dimension, and determine the earliest anomaly time point and its corresponding anomaly dimension; 2.3 Divide the time period before and after the earliest outlier into anomaly time periods. Data within this time period will be used as input data for subsequent root cause analysis algorithms. The third step is to perform causal inference and root cause analysis on the abnormal data. 3.1 The causal relationship between different dimensions in the abnormal time period is discovered by using the causal inference algorithm based on Actor-Critic; the abnormal data obtained in the second step is input into the causal inference algorithm model based on Actor-Critic, and after training until the result converges, the output is a graph adjacency matrix with a side length equal to the number of dimensions of the input data, i.e., the causal graph. A causal inference algorithm model based on Actor-Critic is constructed, defining a graph structure score function and a reinforcement learning reward value based on the score function. This model can search for the causal graph with the best score. The causal inference algorithm model based on Actor-Critic is in the Actor-Critic network framework. The Actor uses the GAT network to extract feature relationships and generates the adjacency matrix of the graph structure, i.e., the causal graph, by passing the features through bilinear product and Bernoulli sampling. The Critic uses two fully connected layers to predict the generated graph score. 3.2 Calculate the Pearson correlation coefficient of the input data and take its absolute value to obtain the correlation matrix; 3.3 Based on the causal graph and the correlation matrix, perform forward, backward, and self-transitions to obtain the edge transition probability matrix; 3.4 Perform a random walk based on the transition probability matrix of the edges to obtain the list of abnormal root cause scores.

2. The system performance bottleneck detection method based on reinforcement learning as described in claim 1, characterized in that, In step 3.1, the fractional function of the graph structure is defined as follows: First, the BIC score of the causal graph is defined, with the following formula: The first term on the right is the likelihood function. Let represent the predicted value of the k-th item in the i-th dimension of the observed sample x, where n represents the number of samples (i.e., the time length) and d represents the number of dimensions. To avoid small values ​​where the logarithm argument is zero, we use 10 here. -8 The second term in the right-hand equation is a penalty term, and m represents the number of edges in the graph. Then, the scores of the graph are normalized to obtain the final score of the causal graph. The specific formula is as follows: in and These are the high and low thresholds for the causal graph score, respectively. The fraction of a directed graph where all elements except the diagonal are 1. The score for a graph where all values ​​are 0.

3. The system performance bottleneck detection method based on reinforcement learning as described in claim 1 or 2, characterized in that, In step 3.1, the GAT network structure in the Actor is as follows: The GAT network in the Actor is a neural network model based on the attention mechanism. In this model, GAT is composed of multiple attention modules stacked together, and the composition of the attention modules is as follows: First, a one-dimensional convolutional layer is used to extract features from the input sequence to obtain a feature vector; then, the feature vector is passed through two convolutional layers to calculate the attention coefficients. The calculation of the attention coefficients requires adding the outputs of the two convolutional layers and activating them, followed by a softmax transformation to ensure that the sum of the attention coefficients is 1; the attention coefficients and the feature sequence are passed through a dropout layer, and some of the elements are randomly set to zero; next, the feature sequence is weighted and summed according to the attention coefficients to obtain the encoding of the embedded node relationship features; finally, the feature vector of the input sequence is added to the output vector using a residual connection, and a nonlinear transformation is performed on it using an activation function to obtain the final output vector.

4. A system performance bottleneck detection method based on reinforcement learning as described in claim 1 or 2, characterized in that, In step 3.1, the features extracted by the GAT network in the Actor are used to generate an adjacency matrix of a graph structure through bilinear product and Bernoulli sampling, specifically as follows: In the Actor structure, the feature encoding extracted by the GAT network is bilinearly producted with the learnable weights to obtain the final adjacency probability distribution; the bilinear product calculation formula is as follows: Where W is the learning weight matrix. , The vector of dimensions i and j obtained through GAT encoding. Let i be the adjacency probability from node i to j; finally, Bernoulli sampling is performed on the probability distribution between nodes to convert the probability between each pair of nodes into binary samples to obtain the adjacency matrix of the generated graph, i.e., the causal graph. This process involves subtracting a large negative value (100000000) from the probability matrix and then multiplying it by a mask to block the current node, ensuring that it does not connect to itself.

5. The system performance bottleneck detection method based on reinforcement learning as described in claim 2, characterized in that, In step 3.1, the reinforcement learning reward value based on the fractional function is as follows: In the definition of the reinforcement learning reward, in addition to the score of the generated graph, an acyclic constraint is also added, and the specific formula is: in Let A be the generation graph matrix and S(A) be the penalty parameter, and S(A) be the acyclic constraint, defined as: 。

Images