A transformer-based microservice performance anomaly detection method and system

Abstract

The application discloses a kind of microservice performance anomaly detection methods based on Transformer, comprising: obtaining the span log of microservice system, the span with same tracking ID in span log is combined together, to obtain multiple different call chains, using depth-first search algorithm searches all paths of the call chain from root node to leaf node, to obtain multiple service call sequences in the call chain and its corresponding multiple response time sequences, each service call sequence is input into the detection model trained in advance, to obtain the probability distribution of each feature value in the path vector set corresponding to the service call sequence, according to the probability distribution of service feature value output by detection model, the probability of service actual feature value in path vector is obtained as the abnormal score of service, according to the abnormal score of service, whether service is the service that appears performance anomaly in call chain is judged.The application can solve the technical problems of poor convergence speed and detection effect of existing method model.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of deep learning and microservice system operation and maintenance, and more particularly relates to a microservice performance anomaly detection method and system based on Transformer for distributed tracking. BACKGROUND

[0002] With the continuous development of cloud computing technology, many software applications begin to be deployed on cloud computing platforms, and monolithic architecture is gradually replaced by service-oriented architecture. More and more companies choose to migrate from monolithic architecture to microservice architecture, and build their core business systems based on microservice architecture. However, the failure of software services based on microservice architecture has also affected the normal use of users and caused not small losses to enterprises. It is very important to discover and diagnose failures in time to ensure the high reliability and high availability of online services. With the increase in system size and complexity, traditional manual operation becomes difficult. Due to the complex interaction between services and the huge operation and maintenance data, the problem of anomaly detection for microservice systems is challenging.

[0003] There are some machine learning methods that try to solve the problem of microservice anomaly detection. The first method encodes the call chain as a service tracking vector, which is used as input to train a deep Bayesian neural network with posterior flow to learn the normal patterns of call chains. By comparing the patterns of call traces with the learned normal call trace patterns, it is determined whether there is an abnormal situation; the second method models the call chain under the normal state of the application based on a variational autoencoder, and uses the response time of the services in the call chain to construct the call chain into one-dimensional time series data for training. In detection, reconstruction error is used to determine whether the call chain is abnormal; the third method proposes a multi-modal long short term memory (LSTM) network model, which learns the normal patterns of service sequences and response time sequences in call chain data by combining two single-modal networks. One network is used to learn the probability distribution of services that may be called after a service, and the other network is used to learn the probability distribution of service response time. In online detection, the model judges whether the system has appeared abnormal by predicting the events that may occur after an event and the possible response time of the event.

[0004] However, the above methods all have some defects that cannot be ignored: the effective dimension of the feature vector constructed by the first method is too sparse, which will affect the convergence speed and detection effect of the model to some extent; the second method does not consider the correlation between the response time of the service and the calling path, so it cannot well model the normal distribution of the response time of the service, resulting in low efficiency of microservice performance anomaly detection; the third method uses LSTM, which is difficult to capture the long-distance dependency relationship between services in the calling chain, so that the model is difficult to learn the calling relationship between services, resulting in inaccurate detection results; in addition, the above methods cannot detect abnormal services in the calling chain in a fine-grained manner, lack of explainability for the detected abnormal calling chain, and cannot quickly locate the root cause of the anomaly, so they cannot meet the business scenarios of effective discovery and accurate root cause positioning of microservice performance anomalies. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the present application provides a microservice performance anomaly detection method and system based on Transformer, which aims to solve the technical problems that the effective dimension of the feature vector constructed by the first method in the background art is too sparse, which will affect the convergence speed and detection effect of the model to some extent, and the second method does not consider the correlation between the response time of the service and the calling path, so it cannot well model the normal distribution of the response time of the service, resulting in low efficiency of microservice performance anomaly detection, and the third method uses LSTM, which is difficult to capture the long-distance dependency relationship between services in the calling chain, so that the model is difficult to learn the calling relationship between services, resulting in inaccurate detection results, and the above methods cannot detect abnormal services in the calling chain in a fine-grained manner, lack of explainability for the detected abnormal calling chain, and cannot quickly locate the root cause of the anomaly, so they cannot meet the business scenarios of effective discovery and accurate root cause positioning of microservice performance anomalies.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present application, a microservice performance anomaly detection method based on Transformer is provided, comprising the following steps:

[0007] (1) Obtain the span logs of the microservice system, use Python to combine the spans with the same trace ID in the span logs together to obtain multiple different call chains, for each call chain, use a depth-first search algorithm to search all paths of the call chain from the root node to the leaf node to obtain multiple service call sequences and their corresponding multiple response time sequences in the call chain, input each response time in each response time sequence corresponding to each service call sequence obtained to a pre-established CART regression tree to obtain a feature value corresponding to each response time, and all feature values corresponding to each response time sequence corresponding to each service call sequence in the call chain constitute a feature value sequence corresponding to the service call sequence, that is, a path vector set corresponding to the service call sequence.

[0008] (2) Input each service call sequence obtained in step (1) into a pre-trained detection model to obtain a probability distribution of each feature value in the path vector set corresponding to the service call sequence.

[0009] (3) According to the probability distribution of the service feature value output by the detection model in step (2), obtain the probability of the actual feature value of the service in the path vector as the anomaly score of the service.

[0010] (4) Determine whether the service is a service with performance anomaly in the call chain according to the anomaly score of the service obtained in step (3).

[0011] Preferably, the CART regression tree is obtained by the following steps:

[0012] (1-1) Obtain multiple response times of a service, sort the response times of the service in ascending order to obtain an ordered variable set D.

[0013] D={(x1,y1),(x2,y2),(x3,y3),…,(x n ,y n )}

[0014] Where x i represents the position of the i th response time of the service in the ordered variable set, y i represents the numerical value of the i th response time of the service, and i∈[1,n], where n represents the total number of response times of the service.

[0015] (1-2) Traverse the split point s i between the position x i of the i th response time of the service in the ordered variable set and the position x i+1 of the i+1 th response time of the service in the ordered variable set obtained in step (1-1), that is, calculate each split point si The error square sum m(s) of the two divided parts i Select s that minimizes the error square sum m(s) as the optimal split point s j The ordered variable set D is divided into left subset (y1, y2,..., y j ) and right subset (y j , y j+1 ,..., y j+2 ) with s n as the split point.

[0016] (1-3) For the left subset and right subset obtained by splitting the set with the optimal split point s j , set the scale factor, and repeatedly perform the above step (1-2), i.e. repeatedly split the two subsets in a recursive manner until the split stop condition is met, and then obtain the average variance of the left subset (y1, y2,..., y j ) and the right subset (y j+1 , y j+2 ,..., y n ) as the average variance corresponding to the scale factor.

[0017] (1-4) Increase the scale factor in a recursive manner, and repeat the above step (1-3) to obtain the average variances corresponding to multiple scale factors.

[0018] (1-5) Select the scale factor corresponding to the minimum average variance and the left subset and the right subset from the multiple average variances obtained in steps (1-3) and (1-4), and obtain a decision tree with M sub-trees, represented by Y1, Y2,..., Y M , where Y i = {y i |y s ≤y i <y s+1}, and calculate the mean value c i of each sub-tree as the output of each sub-tree.

[0019] Preferably, the error square sum m(s i ) corresponding to the split point s i in step (1-2) is calculated using the following formula:

[0020]

[0021] Where R1 = {x1, x2,..., x i}, R2 = {x i+1 , x i+2 ,... x n}, c1 and c2 are the average values of the left subset (y1, y2,..., y j ) and the right subset (y j+1 , y j+2 ,..., y n ), respectively.

[0022] Step (1-5) is performed using the following formula:

[0023]

[0024] where Nm represents the number of elements of the sub-tree Y m .

[0025] Preferably, the detection model is a Transfomer model, which includes a word embedding layer, a position encoding layer, an encoder-decoder layer and an output layer connected in sequence, and the specific structure is as follows:

[0026] The first layer is the word embedding layer, and the input is a C-dimensional integer vector (where C represents the length of the input sequence), and each integer represents the index of a word in the input sequence. Each index is mapped to a d model dimensional word vector using an embedding matrix of size Vxd model (where V is a natural number, the size of which is related to the number of services and service feature values in the data set, the more services, the larger the value of V, d model is a constant, usually 64), and the output is a Cxd model word vector composed of C d model dimensional word vectors.

[0027] The second layer is the position encoding layer, and the input is the Cxd model word vector obtained by the first layer. A d model dimensional vector is calculated as the position encoding vector of each d model dimensional word vector using the sine and cosine position encoding method, and the word vector and the position encoding vector are added to output a Cxd model dimensional vector.

[0028] The third layer is the encoder-decoder layer, and the input is the Cxd model dimensional vector obtained by the second layer, which is subjected to matrix transformation and calculation to output a CxV dimensional vector.

[0029] The fourth layer is the output layer, and the input is the CxV dimensional vector obtained by the third layer. A fully connected layer and a Softmax layer are used to perform linear transformation and normalization processing on the input to output a CxV dimensional vector.

[0030] Preferably, the detection model is trained by the following steps:

[0031] (2-1) Obtain the span logs of the microservice system, preprocess the span logs to obtain the training set composed of service call sequences [op1, op2,..., op a ] and path vector sets [D1, D2,..., D a ].

[0032] (2-2) For the service call sequence [op1, op2,..., op a ] (where op i is composed of service name and service operation name) and path vector set [D1, D2,..., D a ] in the training set obtained in step (2-1), use an embedding matrix of size Vxd model (where V is a natural number related to the number of different services in the data set, the more services, the larger the value of V, d model is a constant, usually 64) to map each element in the service call sequence and the path vector set into a word vector, respectively, to represent the semantic information and feature information of the service, use sine and cosine position encoding to perform d model dimensional position encoding on each element's word vector, and add the position encoding corresponding to each element to the word vector to obtain the summation result of the element, and the summation results corresponding to all elements in the service call sequence and the path vector set form the input vector sets [S1, S2,..., S a ] and [T1, T2,..., T a ] of the encoder-decoder layer in the Transformer model, respectively.

[0033] (2-3) Input the input vector set [S1, S2,..., S a ] obtained in step (2-2) into the encoder of the Transformer model to obtain the feature matrix Y of the input vector set using the self-attention mechanism.

[0034] (2-4) Input the vector [T1, T2,..., T n ] obtained in step (2-2) into the decoder of the Transformer model to obtain the feature matrix Z using the masked self-attention mechanism.

[0035] (2-5) Take the feature matrix Y calculated in step (2-3) as the K and V matrices, take the feature matrix Z calculated in step (2-4) as the Q matrix, and input the Q, K, and V matrices into the encoder-decoder attention module in the decoder of the detection model to obtain a new feature matrix F. Input the new feature matrix F into a fully connected layer network with a softmax activation function to obtain the probability distribution of the service feature values predicted by the model.

[0036] (2-6) Based on the service feature value probability distribution predicted by the model obtained in step (2-5), the detection model is iteratively trained using the cross-entropy loss function until the detection model converges, thus obtaining the trained detection model.

[0037] Preferably, in step (2-2), for a given position pos in the service call sequence and path vector and dimension i in the word vector (where i∈[1, a]), the position encoding is calculated as follows:

[0038]

[0039]

[0040] Among them, PE (pos，i) d represents the position code of the i-th dimension at position pos. model Indicates the dimension of the embedding vector.

[0041] Preferably, step (2-3) specifically involves:

[0042] First, through the weight matrix W q W k and W v For each input vector S in the set of input vectors i A linear transformation is performed to obtain the query vector Q, key vector K, and value vector, respectively. During training, the weight matrix W... q W k and W v It will be continuously updated;

[0043] Then, the input vector S is calculated using the query vector and the key vector. i Attention score ∝ i,j .

[0044]

[0045] Where q1, q2, ..., q n This indicates a query for elements k1, k2, ..., k in vector Q. n Represents the elements in the key vector K;

[0046] By ∝ i,j The resulting attention score matrix represents the degree of dependency between different services in the service call sequence.

[0047] Next, use the Softmax function to apply the ∝ i,j Normalization yields ∝′ i，j .

[0048]

[0049] where t∈[1, n];

[0050] Subsequently, the value vector is weighted and calculated with the normalized correlation score to obtain a vector Y representing the service context i .

[0051]

[0052] where v1, v2,..., v n represent the elements in the value vector V;

[0053] Finally, a two-layer fully connected network FFN is used to perform a nonlinear transformation on all vectors Y1, Y2,..., Y n to obtain the feature matrix Y of the service call sequence after the encoder processing.

[0054] Preferably, the cross-entropy loss function L in step (2-6) is:

[0055]

[0056] where T y represents the length of the target sequence, y t,i represents the target sequence T y At time step t, the one-hot representation of the i-th feature value, represents the probability value of the i-th feature value predicted by the detection model at time step t.

[0057] Preferably, step (4) first obtains the abnormal score of the service corresponding to the service call sequence obtained in step (3) and records it. When the performance of the initial detection system is abnormal, a fixed threshold TH is set for the abnormal scores of all services to determine whether the service is abnormal, i.e. when the abnormal score of the service is less than 0.6, it is determined that the service is the service with performance anomaly in the call chain. In the subsequent detection process, first, the cumulative distribution function CDF of the historical abnormal score of the service is obtained according to the recorded historical abnormal score of the service, and then the cumulative distribution value is calculated according to the current abnormal score of the service and the cumulative distribution function. If the cumulative distribution value is less than the preset threshold, it is determined that the service is the service with performance anomaly in the call chain.

[0058] According to another aspect of the present application, a microservice performance anomaly detection system based on Transformer is provided, comprising the following steps:

[0059] The first module is used for obtaining span logs of a micro-service system, using Python to combine spans with the same tracking ID in the span logs together to obtain multiple different call chains, using a depth-first search algorithm to search all paths of each call chain from a root node to a leaf node to obtain multiple service call sequences and multiple response time sequences corresponding to the call chain, inputting each response time in each response time sequence corresponding to each service call sequence obtained into a pre-established CART regression tree to obtain a feature value corresponding to each response time, and all feature values corresponding to each response time sequence corresponding to each service call sequence in the call chain constitute a feature value sequence corresponding to the service call sequence, that is, a path vector set corresponding to the service call sequence.

[0060] The second module is used for inputting each service call sequence obtained by the first module into a pre-trained detection model to obtain a probability distribution of each feature value in the path vector set corresponding to the service call sequence.

[0061] The third module is used for obtaining a probability of an actual feature value of a service in the path vector as an abnormal score of the service according to the probability distribution of the service feature value output by the detection model in the second module.

[0062] The fourth module is used for judging whether the service is a service with performance anomaly in the call chain according to the abnormal score of the service obtained by the third module.

[0063] Overall, the above technical solutions conceived by the present application can achieve the following beneficial effects compared with the prior art:

[0064] (1) The present application uses the structure of the call chain and the service performance index (response time) to construct the call chain into service call sequences and path vectors, avoiding the problem of excessively sparse vectors of the constructed call chain. Therefore, the technical problem of difficulty in convergence of the model and poor detection effect caused by excessively sparse effective dimensions of the input vector is solved.

[0065] (2) The present application uses the Transformer self-attention mechanism to capture the calling relationship between services in steps (2-2) to (2-3), thus solving the technical problem of inaccurate detection effect caused by the inability to capture the long-distance dependency relationship between services.

[0066] (3) The application can effectively learn the normal response time distribution of services in different calling chains because the application uses step (2-5) to use the Transformer encoder-decoder attention mechanism to establish the association between the response time of the service and the calling path of the service, and integrates these relationships into model training, so as to solve the technical problem that the existing microservice performance anomaly detection method does not consider the association between the service response time and the calling path, and the anomaly detection effect is poor;

[0067] (4) Because the application uses steps (3) to (4), the probability of the actual feature value of the service in the calling chain is obtained by using the probability distribution output by the model to judge the anomaly, so that the abnormal service in the calling chain can be identified in detail during anomaly detection, and therefore the technical problem that the abnormal service in the abnormal calling chain cannot be detected, and the root cause of the anomaly cannot be quickly and accurately located. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 is the overall framework diagram of the microservice performance anomaly detection method based on the Transformer of the application;

[0069] Figure 2 is the comparison of the anomaly detection effect of the application on the AIops challenge data set and the open source microservice Train-Ticket data set and the F1-Score index of the existing method;

[0070] Figure 3 is the flowchart of the microservice performance anomaly detection method based on the Transformer of the application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and do not limit the application. In addition, the technical features involved in each embodiment of the application described below can be combined with each other as long as they do not conflict with each other.

[0072] The basic idea of the application is to use the Transformer model attention mechanism to capture the calling relationship between services, and use the encoder-decoder attention module in the Transformer model to establish the association between the response time of the service and the calling path, so as to more effectively learn the normal distribution of the service response time, and improve the detection effect of the microservice performance anomaly.

[0073] As Figure 1 and Figure 3As shown, the present application provides a Transformer-based microservice performance anomaly detection method, comprising the following steps:

[0074] (1) Obtain the span logs of the microservice system, use Python to combine the spans with the same trace ID in the span logs together to obtain multiple different call chains, for each call chain, use a depth-first search algorithm to search all paths of the call chain from the root node to the leaf node to obtain multiple service call sequences and their corresponding multiple response time sequences in the call chain, input each response time in each response time sequence corresponding to each service call sequence to a pre-established CART regression tree to obtain a feature value corresponding to each response time, and all feature values corresponding to each response time sequence corresponding to each service call sequence in the call chain constitute a feature value sequence corresponding to the service call sequence, i.e. a path vector set corresponding to the service call sequence.

[0075] The advantage of this step is that the structure information of the call chain and the service response time are used to construct the call chain into a service call sequence and a path vector, avoiding the problem of excessively sparse feature vectors of the constructed call chain, and enabling more efficient service normal mode modeling based on the service call sequence and the path vector.

[0076] Specifically, the CART regression tree is obtained by the following steps:

[0077] (1-1) Obtain multiple response times of a service, sort the response times of the service in ascending order to obtain an ordered variable set D.

[0078] D={(x1,y1),(x2,y2),(x3,y3),...,(x n ,y n )}

[0079] Where x i represents the position of the i-th response time of the service in the ordered variable set, y i represents the numerical value of the i-th response time of the service, and i∈[1,n], where n represents the total number of response times of the service.

[0080] (1-2) Traverse the position x i of the i-th response time of the service in the ordered variable set and the position x i+1 of the i+1-th response time of the service in the ordered variable set in the ordered variable set D obtained in step (1-1) between the split points s i , i.e. Calculate the error sum of squares m(s i of the two parts divided by each split point s i). Select s that minimizes the error sum of squares m(s) as the optimal split point s j , with s j as the split point, the ordered variable set D is divided into a left subset (y1, y2,..., y j ) and a right subset (y j+1 , y j+2 ,..., y n ).

[0081] Specifically, in this step, the split point s i corresponding to the error sum of squares m(s i ) is obtained.

[0082]

[0083] Where R1 = {x1, x2,..., x i}, R2 = {x i+1 , x i+2 ,... x n}, c1 and c2 are the average values of the left subset (y1, y2,..., y j ) and the right subset (y j+1 , y j+2 ,..., y n ) respectively.

[0084] (1-3) For the left subset and the right subset obtained by splitting the set with the optimal split point s j in step (1-2), by setting a scaling factor and repeatedly performing the above step (1-2), that is, continuously dividing the two subsets in a recursive manner until the division stopping condition is met (that is, when the number of elements in the left subset or the right subset obtained by division is less than the number of elements in the ordered variable set D multiplied by the scaling factor), the average variance of the left subset (y1, y2,..., y j ) and the right subset (y j+1 , y j+2 ,..., y n ) obtained is taken as the average variance corresponding to the scaling factor.

[0085] In this step, the scaling factor is set to 0.1;

[0086] (1-4) Continuously increase the scaling factor in a recursive manner and repeat the above step (1-3) to obtain the average variances corresponding to multiple scaling factors;

[0087] Specifically, in this step, the value of the scaling factor is continuously increased by 0.05 as an interval until it equals 0.5.

[0088] (1-5) From the multiple average variances obtained in steps (1-3) and (1-4), select the scaling factor corresponding to the minimum average variance, as well as the left and right subsets, to obtain a decision tree with M subtrees, labeled Y1, Y2, ..., Y... M It means that Y i ={y i |y s ≤y i <y s+1} Calculate the mean c of each subtree. i As the output of each subtree:

[0089]

[0090] Where Nm represents subtree Y m The number of elements.

[0091] The advantage of the above sub-steps (1-1) to (1-5) is that by using the CART regression tree to bin the response time of each service, the number of feature values ​​of each service is effectively reduced, the computational complexity of the model is reduced, and the interference of response time noise and fluctuations on the model is reduced, thereby improving the detection effect of the model.

[0092] (2) Input each service call sequence obtained in step (1) into the pre-trained detection model to obtain the probability distribution of each feature value in the path vector set corresponding to the service call sequence.

[0093] The detection model of this invention is a Transformer model, which includes a word embedding layer, a position encoding layer, an encoder-decoder layer, and an output layer connected in sequence, with the specific structure as follows:

[0094] The first layer is the word embedding layer. The input is a C-dimensional integer vector (where C represents the length of the input sequence), and each integer represents the index of a word in the input sequence. This is achieved using a vector of size V×d. model (Where V is a natural number, its size is related to the number of services and their feature values ​​in the dataset; the more services there are, the larger the value of V will be, d) model The embedding matrix (where is a constant, typically 64) maps each index to a d... model The word vectors are 3D, and the output consists of C d-dimensional words. model C×d consisting of word vectors model Word vectors.

[0095] The second layer is the position encoding layer, whose input is the C×d obtained from the first layer. model The word vectors are encoded using sine and cosine positional encoding for each d. model Calculate a d-dimensional word vector modelThe position encoding vector is a V-dimensional vector, and the word vector and the position encoding vector are added to output a Cxd model dimensional vector.

[0096] The third layer is an encoder-decoder layer, and the input of the third layer is the Cxd model dimensional vector obtained by the second layer, which is subjected to matrix transformation and calculation to output a CxV-dimensional vector.

[0097] The fourth layer is an output layer, and the input of the fourth layer is the CxV-dimensional vector obtained by the third layer, which is subjected to linear transformation and normalization processing by using a fully connected layer and a Softmax layer, and the output is a CxV-dimensional vector.

[0098] Specifically, the detection model of the application is obtained by the following steps:

[0099] (2-1) Obtain the span log of the micro-service system, and preprocess the span log to obtain a training set composed of a service call sequence [op1, op2,..., op a ] and a path vector set [D1, D2,..., D a ].

[0100] In this step, the span log is collected from the micro-service system by the distributed monitoring tool Jaeger, and includes timestamp, service name, service operation name, span ID, trace ID and other information.

[0101] The preprocessing process of the span log in this step is exactly the same as the processing process in step (1) above, and will not be repeated here.

[0102] (2-2) For the service call sequence [op1, op2,..., op a ](wherein op i is composed of service name and service operation name) and the path vector set [D1, D2,..., D a ] in the training set obtained in step (2-1), an embedding matrix with a size of Vxd model (wherein V is a natural number, the size of which is related to the number of different services in the data set, the more the number of services, the larger the value of V, and d model is a constant, usually 64) is used to map each element in the service call sequence and the path vector set into a word vector, respectively, to represent the semantic information and feature information of the service, and the sine and cosine position encoding is used to perform d modelThe position encoding of each element in the service call sequence and the path vector is calculated, and the position encoding corresponding to each element is added to the word vector to obtain the summation result of the element. The summation results of all elements in the service call sequence and the path vector set respectively constitute the input vector set [S1, S2,..., S a ] and [T1, T2,..., T a ] of the encoder-decoder layer in the Transformer model.

[0103] For a given position pos in the service call sequence and the path vector and a dimension i (where i∈[1, a]) in the word vector, the position encoding is calculated as follows:

[0104]

[0105]

[0106] where PE (pos，i) represents the position encoding of the i-th dimension of the position pos. d model represents the dimension of the embedding vector.

[0107] Specifically, the parameter of the position encoding is a value calculated based on the position and the dimension of the word vector. When i is even, the value is When i is odd, the value is For each dimension i of a given position pos, the corresponding position encoding value is composed of different periodic sine and cosine curves. For example, when i=0, the corresponding sine function has a period of 2n; when i=1, the corresponding cosine function has a period of 2π. If two different positions pos a and pos b have the same encoding value in a certain dimension i, the difference between them is equal to the period of the curve in that dimension, i.e., |pos a -pos b | = T i . In other dimensions k∈{0, 1, 2,..., d-1} (k≠i), since T k ≠ T i , the encoding of pos a and pos b in dimension k will not be equal. Therefore, this position encoding method can ensure that different positions are encoded into different values in all d model dimensions, so that the elements at different positions in the sequence can obtain unique position encodings.

[0108] (2-3) The input vector set [S1, S2,..., S aThe encoder of the input Transformer model is inputted to obtain a feature matrix Y of the input vector set by using a self-attention mechanism.

[0109] Specifically, first, each input vector S q in the input vector set is linearly transformed by weight matrices W k , W v and W i to obtain a query vector Q, a key vector K and a value vector V, respectively. During the training process, the weight matrices W q , W k and W v are constantly updated.

[0110] Then, the attention score a i of the input vector S i，j is calculated by using the query vector and the key vector.

[0111]

[0112] where q1, q2,..., q n represent elements in the query vector Q, and k1, k2,..., k n represent elements in the key vector K.

[0113] The attention score matrix composed of a i，j represents the degree of dependence between different services in the service invocation sequence.

[0114] Next, the Softmax function is used to normalize a i,j to obtain a′ i，j .

[0115] a′ i，j = Softmax(a i，1 , a i，2 ,..., a i，n ) = exp(a i,j ) / ∑ t exp(a i,t )

[0116] where t∈[1, n];

[0117] Subsequently, the vector Y i representing the service context is calculated by weighting and summing the value vector and the normalized relevance score.

[0118]

[0119] wherein v1, v2,..., v n denotes an element in the value vector V;

[0120] Finally, a two-layer fully connected network FFN is used to perform a nonlinear transformation on all vectors Y1, Y2,..., Y n to obtain the feature matrix Y of the service call sequence after the encoder processing.

[0121] The above sub-steps (2-2) to (2-3) have the advantage that the input vector of the service call sequence is processed by using the self-attention mechanism, which can more fully capture the context information of the service and more effectively model the normal response time distribution of the service.

[0122] (2-4) input the vector [T1, T2,..., T n ] obtained in step (2-2) into the Transformer model decoder to obtain the feature matrix Z by using the masked self-attention mechanism.

[0123] Specifically, the calculation process of this step is the same as that of step (2-3) and will not be repeated here.

[0124] (2-5) input the feature matrix Y calculated in step (2-3) as the K and V matrices, and input the feature matrix Z calculated in step (2-4) as the Q matrix, and input the Q, K, and V matrices into the encoder-decoder attention module in the decoder of the detection model to obtain a new feature matrix F. Input the new feature matrix F into a fully connected layer network with a softmax activation function to obtain the probability distribution of the service feature value predicted by the model.

[0125] Specifically, the process after inputting the encoder-decoder attention module in this step is exactly the same as that of step (2-3) and will not be repeated here.

[0126] The advantage of this step is that, by using the encoder-decoder attention mechanism, the correlation between the service response time and the service call path can be established, and the model can learn the normal distribution of the service response time more accurately.

[0127] (2-6) according to the probability distribution of the service feature value predicted by the model obtained in step (2-5), and using the cross-entropy loss function to iteratively train the detection model until the detection model converges, thereby obtaining the trained detection model.

[0128] Specifically, this step is to constantly update the parameters of the detection model by minimizing the value of the cross-entropy loss function, so that the detection model can learn the normal distribution of the service response time.

[0129] Preferably, the cross-entropy loss function L is:

[0130]

[0131] where T y denotes the length of the target sequence, y t，i denotes the target sequence T y at time step t, the one-hot representation of the i-th feature value, denotes the probability value of the i-th feature value predicted by the detection model at time step t.

[0132] (3) According to the probability distribution of the service feature value output by the detection model in step (2), the probability of the actual service feature value in the path vector is obtained as the anomaly score of the service.

[0133] (4) According to the anomaly score of the service obtained in step (3), it is judged whether the service is the service that appears performance anomaly in the call chain.

[0134] Specifically, this step first obtains the anomaly score of the service corresponding to the service call sequence in step (3) and records it down. In the initial (i.e. the first day) detection of the performance anomaly of the system, a fixed threshold TH (TH = 0.6) is set for the anomaly score of all services to judge whether the service is abnormal, i.e. when the anomaly score of the service is less than 0.6, it is determined that the service is the service that appears performance anomaly in the call chain. In the subsequent detection process (not the first day), first, the cumulative distribution function (Cumulative Distribution Function, abbreviated as CDF) of the historical (the previous day or a week, preferably the previous day) anomaly score of the service is obtained according to the recorded historical anomaly score of the service, and then the cumulative distribution value is calculated according to the current anomaly score of the service and the cumulative distribution function. If the cumulative distribution value is less than the preset threshold (which is 0.1 in the present application), it is determined that the service is the service that appears performance anomaly in the call chain.

[0135] The advantages of the above steps (3) to (4) are that the probability distribution of each service feature value output by the model can be used to detect the service that appears performance anomaly in the call chain in a fine-grained manner, which facilitates the root cause positioning of the performance anomaly problem of the microservice system.

[0136] The application provides a distributed tracking-oriented, Transformer-based microservice performance anomaly detection method, which aims to accurately detect system anomalies and locate the root cause of the anomalies in a microservice system in a timely manner, thereby guaranteeing the high availability and high reliability of the service. The self-attention mechanism of the Transformer is used to capture the calling relationship between services, and the response time and calling path of the services are associated through an encoder-decoder architecture, thereby effectively modeling the normal distribution of the service response time, using the trained model to detect microservice performance anomalies online, and using the calling relationship between services to quickly deduce the root cause on the basis of anomaly detection, so that the discovery and diagnosis of microservice performance anomalies are more accurate. It is convenient for operation and maintenance personnel to take effective measures in a timely manner, and the reliability of the system is improved.

[0137] Experimental results

[0138] The experimental environment of the application: the CPU is 8 AMD Ryzen 7 3700X@3.60GHz, the GPU is 1 NVIDIA GeForce RTX 3060, the memory is 16GB DDR4, the hard disk capacity is 2TB, and the algorithm in the paper is programmed and implemented under the Windows 10 operating system using Python and the machine learning library pytorch.

[0139] In order to illustrate the effectiveness of the detection model in the application and the superiority of the application for service performance anomaly detection, the application has made relevant tests on the AIops dataset and the Train-Ticket dataset. The model is compared with the existing methods AEVB, MultimodalLSTM and TraceAnomaly in terms of effect, and table 1 gives the comparison results. Among them, P, R and F represent the three evaluation indexes of precision, recall and comprehensive performance of the model respectively.

[0140] Table 1: Anomaly detection effect of different methods

[0141]

[0142] As can be seen from Table 1, compared with the three existing anomaly detection methods, the present application improves the precision by 48.6%, 30.2% and 3.5% on average and the recall by 34.7%, 11.1% and 4.1% on average under two data sets, and it can be seen that the microservice performance anomaly detection method proposed in the present application improves the detection effect on the two data sets. In addition, the microservice performance anomaly root cause positioning method based on the Transformer proposed in the present application, on the one hand, adopts a self-attention mechanism to capture the calling relationship between services and more fully extracts the context information of the services, and on the other hand, uses an encoder-decoder attention mechanism to establish the association between the service feature values and the service calling sequence, thereby improving the modeling effect of the normal distribution of the service response time and improving the detection effect of the service performance anomaly.

[0143] In order to illustrate that the root cause positioning algorithm based on anomaly detection can accurately locate the root cause service of the anomaly, the present application carries out related tests on the AIops data set. The model is compared with the existing methods SBFL, MonitorRank and TraceAnomaly, and Table 2 gives the comparison results. Among them, A@k (k = 1, 2, 3) represents the probability that the root cause is contained in the Top-k result, that is, the proportion of the number of root cause services contained in the top k results to the total number of abnormal traces. As can be seen from Table 2, compared with the three existing anomaly detection methods, the accuracy of the root cause positioning of the present application is improved by 35.4% and 6.1% respectively. These results show that the algorithm based on anomaly detection can accurately locate the root cause of the anomaly, and effectively solves the problem of low efficiency of microservice performance anomaly detection.

[0144] Table 2 Comparison of root cause positioning effects of different methods

[0145]

[0146] Those skilled in the art will readily understand that the above description is only the preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting microservice performance anomalies based on Transformer, characterized in that, Includes the following steps: (1) Obtain the span log of the microservice system. Use Python to combine spans with the same tracking ID in the span log to obtain multiple different call chains. For each call chain, use the depth-first search algorithm to search all paths from the root node to the leaf node of the call chain to obtain multiple service call sequences and their corresponding multiple response time sequences in the call chain. Input each response time in each response time sequence corresponding to each service call sequence into the pre-built CART regression tree to obtain the feature value corresponding to each response time. All the feature values ​​corresponding to each response time sequence of each service call sequence in the call chain constitute the feature value sequence corresponding to the service call sequence, that is, the path vector set corresponding to the service call sequence. (2) Input each service call sequence obtained in step (1) into the pre-trained detection model to obtain the probability distribution of each feature value in the path vector set corresponding to the service call sequence; the detection model is a Transformer model, which includes a word embedding layer, a position encoding layer, an encoder-decoder layer and an output layer connected in sequence, and the specific structure is as follows: The first layer is the word embedding layer, and the input is a word embedding layer. An integer vector of dimension , where The input sequence represents the length of the input sequence, and each integer represents the index of a word in the input sequence. A variable of size is used. The embedding matrix maps each index to a Word vectors of dimension, where It is a natural number, and its size is related to the number of services and the number of feature values ​​of each service in the dataset. The more services there are, the larger the value of the feature value of each service. The larger the value of , the better. The constant is set to 64, and the output is determined by... indivual Composed of 3D word vectors Word vectors; The second layer is the position coding layer, whose input is the result obtained from the first layer. The word vectors are encoded using sine and cosine positional encoding for each word. Calculate a word vector of dimension 1 The word vector is used as its positional encoding vector, and the word vector and the positional encoding vector are added together to output a... dimensional vector; The third layer is the encoder-decoder layer, whose input is the result of the second layer. A 3D vector is subjected to matrix transformations and calculations to output the result. dimensional vector; The fourth layer is the output layer, and its input is the result obtained from the third layer. A 3D vector is linearly transformed and normalized using a fully connected layer and a Softmax layer, and the output is a... vector (3) Based on the probability distribution of the service feature values ​​output by the detection model in step (2), obtain the probability of the actual service feature values ​​in the path vector as the service anomaly score; (4) Determine whether the service is a service with performance abnormality in the call chain based on the service abnormality score obtained in step (3).

2. The microservice performance anomaly detection method based on Transformer according to claim 1, characterized in that, CART regression trees are obtained through the following steps: (1-1) Obtain multiple response times for a service, sort the service response times in ascending order to obtain an ordered set of variables. ; ， in This represents the position of the i-th response time of the service in the ordered set of variables. Let i represent the value of the i-th response time of the service, where i ∈ [1, n], and n represents the total number of response times of the service. (1-2) Traverse the ordered set of variables obtained in step (1-1) The position of the i-th response time of the service in the ordered set of variables The position of the (i+1)th response time of the service in the ordered set of variables The dividing point between ,Right now Calculate each split point Sum of squared errors of the two parts Choose the option that makes the sum of squared errors... The smallest s is taken as the optimal split point. ,by The ordered set of variables is used as the split point. Divided into left subset and right subset ; (1-3) Regarding the optimal split point in step (1-2) Regarding the left and right subsets obtained from splitting the set, by setting a scaling factor and repeatedly performing the above steps (1-2), that is, recursively partitioning these two subsets until the partitioning stopping condition is met, the left subset is obtained. and right subset The average variance is taken as the average variance corresponding to the scaling factor; (1-4) Continuously increase the scaling factor in a recursive manner and repeat the above steps (1-3) to obtain the average variance corresponding to multiple scaling factors; (1-5) From the multiple average variances obtained in steps (1-3) and (1-4), select the scaling factor corresponding to the minimum average variance, as well as the left and right subsets, to obtain a decision tree with M subtrees. Then, use... It means that among them Calculate the mean of each subtree. As the output of each subtree.

3. The microservice performance anomaly detection method based on Transformer according to claim 2, characterized in that, Obtain the split point in step (1-2) Corresponding sum of squared errors The following formula is used: ， in , , and Left subset and right subset The average value; Steps (1-5) use the following formula: ， Where Nm represents the subtree The number of elements.

4. The microservice performance anomaly detection method based on Transformer according to claim 3, characterized in that, The detection model is trained through the following steps: (2-1) Obtain the span logs of the microservice system and preprocess the span logs to obtain the service call sequence. and path vector set The training set consists of; (2-2) Service call sequences in the training set obtained in step (2-1) and path vector set ,in It consists of a service name and a service operation name, using a size of [size missing]. The embedding matrix maps each element in the service call sequence and path vector set to a word vector, representing the semantic and feature information of the service, respectively. Sine and cosine positional encoding is used to process the word vector of each element. The positional encoding of each element is calculated, and the positional encoding of each element is summed with the word vector to obtain the summation result for that element. The summation results of all elements in the service call sequence and path vector set constitute the input vector set of the encoder-decoder layer in the Transformer model. and ; (2-3) The set of input vectors obtained in step (2-2) The feature matrix of the input vector set is obtained in the encoder of the Transformer model using a self-attention mechanism. ; (2-4) The vector obtained in step (2-2) The input is fed into the Transformer model decoder to obtain the feature matrix using a masked self-attention mechanism. ; (2-5) The feature matrix obtained in step (2-3) The characteristic matrix is ​​obtained by calculating steps (2-4) using the K and V matrices. As the Q matrix, the Q, K, and V matrices are input into the encoder-decoder attention module in the decoder of the detection model to obtain a new feature matrix. ; to the new feature matrix Input a fully connected layer network with softmax activation function to obtain the probability distribution of the service feature values ​​predicted by the model; (2-6) Based on the probability distribution of service feature values ​​predicted by the model obtained in step (2-5), the detection model is iteratively trained using the cross-entropy loss function until the detection model converges, thereby obtaining the trained detection model.

5. The microservice performance anomaly detection method based on Transformer according to claim 4, characterized in that, In step (2-2), for a given position in the service call sequence and path vector and the dimension in word vectors , where i The location code is calculated as follows: ， in, Indicates position The Dimensional position encoding; Indicates the dimension of the embedding vector.

6. The microservice performance anomaly detection method based on Transformer according to claim 5, characterized in that, Steps (2-3) are as follows: First, through the weight matrix , and For each input vector in the set of input vectors A linear transformation is performed to obtain the query vector Q, key vector K, and value vector, respectively. During training, the weight matrix... , and It will be continuously updated; Then, the input vector is calculated using the query vector and the key vector. Attention score ; ， Where q1, q2, ..., q n This indicates a query for elements k1, k2, ..., k in vector Q. n Represents the elements in the key vector K; Depend on The attention score matrix represents the degree of dependency between different services in the service call sequence; Next, use the Softmax function to... Normalization yields ; ， Where t∈[1,n]; Subsequently, a vector representing the service context is obtained by weighting and summing the value vector with the normalized relevance score. ; ， Where v1, v2, ..., v n This represents an element in the value vector V; Finally, a two-layer fully connected network FFN is used for all vectors. After performing a nonlinear transformation, the final feature matrix of the service call sequence after encoder processing is obtained. .

7. The microservice performance anomaly detection method based on Transformer according to claim 6, characterized in that, The cross-entropy loss function L in step (2-6) is: ， in, Indicates the length of the target sequence. Represents the target sequence At time step t, the one-hot representation of the i-th eigenvalue is... This represents the probability value of the i-th feature value predicted by the detection model at time step t.

8. The microservice performance anomaly detection method based on Transformer according to claim 1, characterized in that, Step (4) First, obtain and record the abnormal score of the service corresponding to the service call sequence obtained in step (3). When initially detecting the performance abnormality of the system, set a fixed threshold TH for the abnormal score of all services to determine whether the service is abnormal. That is, when the abnormal score of the service is less than 0.6, it is determined that the service is a service with performance abnormality in the call chain. In the subsequent detection process, first obtain the cumulative distribution function CDF of the historical abnormal score of the service based on the recorded historical abnormal score of the service, and then calculate the cumulative distribution value based on the current abnormal score of the service and the cumulative distribution function. If the cumulative distribution value is less than the preset threshold, it is determined that the service is a service with performance abnormality in the call chain.

9. A microservice performance anomaly detection system based on Transformer, characterized in that, Includes the following steps: The first module is used to obtain the span logs of the microservice system. Python is used to group spans with the same trace ID in the span logs together to obtain multiple different call chains. For each call chain, a depth-first search algorithm is used to search all paths from the root node to the leaf node of the call chain to obtain multiple service call sequences and their corresponding multiple response time sequences in the call chain. Each response time in each response time sequence corresponding to each service call sequence is input into a pre-built CART regression tree to obtain the feature value corresponding to each response time. All the feature values ​​corresponding to each response time sequence of each service call sequence in the call chain constitute the feature value sequence corresponding to the service call sequence, that is, the path vector set corresponding to the service call sequence. The second module is used to input each service call sequence obtained from the first module into the pre-trained detection model to obtain the probability distribution of each feature value in the path vector set corresponding to the service call sequence. The detection model is a Transformer model, which includes a word embedding layer, a position encoding layer, an encoder-decoder layer, and an output layer connected in sequence, with the following specific structure: The first layer is the word embedding layer, and the input is a word embedding layer. An integer vector of dimension , where The input sequence represents the length of the input sequence, and each integer represents the index of a word in the input sequence. A variable of size is used. The embedding matrix maps each index to a Word vectors of dimension, where It is a natural number, and its size is related to the number of services and the number of feature values ​​of each service in the dataset. The more services there are, the larger the value of the feature value of each service. The larger the value of , the better. The constant is set to 64, and the output is determined by... indivual Composed of 3D word vectors Word vectors; The second layer is the position coding layer, whose input is the result obtained from the first layer. The word vectors are encoded using sine and cosine positional encoding for each word. Calculate a word vector of dimension 1 The word vector is used as its positional encoding vector, and the word vector and the positional encoding vector are added together to output a... dimensional vector; The third layer is the encoder-decoder layer, whose input is the result of the second layer. A 3D vector is subjected to matrix transformations and calculations to output the result. dimensional vector; The fourth layer is the output layer, and its input is the result obtained from the third layer. A 3D vector is linearly transformed and normalized using a fully connected layer and a Softmax layer, and the output is a... ; The third module is used to obtain the probability of the actual service feature value in the path vector as the service anomaly score based on the probability distribution of the service feature value output by the detection model in the second module. The fourth module is used to determine whether a service is experiencing performance issues in the call chain based on the service exception score obtained from the third module.

Images