A cloud database anomaly detection method and system based on a transformer

Abstract

The application discloses a cloud database anomaly detection method based on a Transformer, and comprises the following steps: acquiring key performance indicator (KPI) data of a cloud database, performing dimension reduction processing on the collected KPI data of the cloud database to obtain the KPI data of the cloud database after the dimension reduction processing; performing preprocessing on the KPI data of the cloud database after the dimension reduction processing to obtain cloud database KPI sample data; inputting the obtained KPI data of the cloud database into a cloud database anomaly detection model TransAD that is pre-trained to obtain an anomaly detection result. The application can solve the technical problem that the existing cloud database anomaly detection method neglects the characteristics of a cloud database cluster, only considers the correlation between different KPIs of the database itself, and does not consider the correlation between databases, so that the anomaly state of the database cannot be measured from the overall level of the cluster, and thus the detection accuracy is low.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of abnormality detection in cloud database intelligent operation and maintenance, and more particularly relates to a cloud database abnormality detection method and system based on a Transformer. BACKGROUND

[0002] With the rapid development of Internet technology, the generation of massive data makes data storage and management face greater challenges. Database technology is an effective solution to massive data storage and is receiving more and more attention and support. As cloud databases are being used more and more widely in enterprises, their stability is attracting attention, and abnormality detection is used to monitor key performance indicators (KPIs) of cloud database systems continuously and issue alarms in a timely manner to abnormal situations to ensure the stability of cloud database systems.

[0003] Since cloud database KPI data is time series data, the mainstream method is to regard cloud database abnormality detection as a time series abnormality detection problem and use time series abnormality detection technology to detect abnormalities in cloud database KPI time series data. In a cloud database cluster, there are two dimensions of correlation between cloud database KPIs. The first dimension is the internal relationship between different KPIs of the same cloud database, i.e., the correlation within the cloud database. The second dimension is that some of the same KPIs of different cloud databases have similar trends, i.e., the correlation between cloud databases. Since the features extracted by traditional machine learning-based time series abnormality detection methods are too shallow, the current mainstream time series abnormality detection methods mainly tend to be deep learning methods. Time series abnormality detection methods based on deep learning can be divided into two categories. The first category of methods only focuses on the state of the cloud data itself and considers the correlation between different KPIs of the same cloud database. The second category of methods focuses on the similarity of the change trends of the same KPIs of different cloud databases and detects abnormal cloud databases by detecting the deviation of the change trends between cloud databases.

[0004] However, the cloud database anomaly detection method has some defects that cannot be ignored: first, the first method ignores the characteristics of the cloud database cluster (multiple cloud databases in the cluster work cooperatively and the load is relatively balanced), only considers the correlation between different KPIs of the database itself, and does not consider the correlation between databases, so it cannot measure the abnormal state of the database from the overall level of the cluster, thus resulting in low detection accuracy; second, the second method finds that some same KPIs of different cloud databases in the cluster have similar change trends, and detects the anomaly by detecting the deviation of the change trend of the KPIs between the cloud databases, thus improving the performance of cloud database anomaly detection, but non-anomalous factors such as workload changes will cause the load of the database to deviate and fluctuate, thus also causing the change trend of the same KPIs of different cloud databases to deviate and cause false positives, thus still having the problem of low accuracy. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the present application provides a cloud database anomaly detection method and system based on Transformer, which aims to solve the technical problems that the existing cloud database anomaly detection method ignores the characteristics of the cloud database cluster, only considers the correlation between different KPIs of the database itself, and does not consider the correlation between databases, thus cannot measure the abnormal state of the database from the overall level of the cluster, thus resulting in low detection accuracy, and non-anomalous factors such as workload changes will cause the load of the database to deviate and fluctuate, thus also causing the change trend of the same KPIs of different cloud databases to deviate and cause false positives, thus resulting in low detection accuracy.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present application, a cloud database anomaly detection method based on Transformer is provided, comprising the following steps:

[0007] A. Obtain the key performance indicator (KPI) data of the cloud database, and perform dimensionality reduction processing on the collected KPI data of the cloud database to obtain the KPI data of the cloud database after dimensionality reduction processing.

[0008] B. Preprocess the KPI data of the cloud database after dimensionality reduction processing in step A to obtain cloud database KPI sample data.

[0009] C. Input the KPI data of the cloud database obtained in step B into a pre-trained cloud database anomaly detection model TransAD to obtain an anomaly detection result.

[0010] Preferably, step A specifically comprises collecting KPI data of the cloud database at fixed time intervals through a programmatic interface provided by the cloud database, and then selecting KPI data of the cloud database most relevant to cloud database anomalies from the collected KPI data of the cloud database based on expert experience as the KPI data of the cloud database after dimension reduction, the KPI data of the cloud database after dimension reduction being two-dimensional data with time dimension and index dimension, and containing n KPI vectors, wherein n ranges from 10 to 80.

[0011] Preferably, step B specifically comprises filling missing values in the KPI data of the cloud database caused by network jitter and the like using linear interpolation to obtain filled multi-dimensional KPI data of the cloud database; and dividing the filled multi-dimensional KPI data of the cloud database into multi-dimensional KPI subsequence samples with fixed time span size and no overlap using a sliding window with a size of K, each multi-dimensional KPI subsequence sample being two-dimensional data with time dimension and index dimension, the size of the time dimension being K, and the size of the index dimension being the number n of KPIs in the KPI data of the cloud database after dimension reduction.

[0012] Preferably, the cloud database anomaly detection model comprises three parts connected in sequence, namely a Transformer encoder, a cloud database embedding layer and an anomaly decision layer.

[0013] The first layer is the Transformer encoder, which has multiple encoding blocks, each encoding block containing a multi-head self-attention sublayer and a feedforward neural network sublayer, and the specific structure is as follows:

[0014] The multi-head self-attention sublayer encodes the KPI vectors in the KPI data of multiple cloud databases under the same cluster to obtain KPI encoding vectors, wherein the number of KPI sample data of the cloud database is m, each cloud database KPI sample data contains n KPI vectors, and each KPI vector contains k data points, and the number of KPI vectors is m*n.

[0015] The feedforward neural network sublayer takes the KPI encoding vectors output by the multi-head self-attention sublayer as input, and linearly transforms the KPI encoding vectors to obtain linearly transformed KPI encoding vectors.

[0016] The second layer is the cloud database embedding layer, which is a fully connected neural network with 1024 neurons, and takes the KPI encoding vectors output by the first layer as input, and performs feature fusion processing on the n KPI encoding vectors belonging to the same database to output a cloud database embedding vector that can represent the running state of the cloud database.

[0017] The third layer is an anomaly decision layer, which is a fully connected neural network with 1024 neurons, and the input is the cloud database embedding vector output by the second layer, and the embedding vectors of multiple cloud databases in the cluster are processed for feature fusion to output an anomaly probability vector with m+1 dimensions.

[0018] Preferably, the cloud database anomaly detection model is obtained by the following steps:

[0019] (1) Obtain cloud database KPI data, and perform dimension reduction processing on the collected cloud database KPI data to obtain cloud database KPI data after dimension reduction processing.

[0020] (2) Preprocess the cloud database KPI data after dimension reduction processing in step (1) to obtain cloud database KPI sample data.

[0021] (3) Perform anomaly KPI value injection on the cloud database KPI sample data obtained in step (2) to obtain labeled cloud database KPI sample data.

[0022] (4) Divide the labeled cloud database KPI sample data obtained in step (3) into a training set and a test set according to a ratio of 7:3.

[0023] (5) For the cloud database KPI sample data in the training set obtained in step (4), input the KPI samples of multiple cloud databases in the cluster under the same time window into the Transformer encoder to obtain KPI encoding vectors that fuse the correlation features of the cloud databases.

[0024] (6) Input the KPI encoding vectors obtained in step (5) into the cloud database embedding layer, and the cloud database embedding layer fuses the n KPI encoding vectors belonging to the same cloud database to obtain cloud database embedding vectors that can represent the running state of the cloud database.

[0025] (7) Input the cloud database embedding vectors obtained in step (6) into the anomaly decision layer, and the anomaly decision layer fuses the embedding vectors of m cloud databases in the cluster (obtained by processing the KPI sample data of multiple cloud databases belonging to the same cluster in the same time window) to obtain an anomaly probability vector with m+1 elements.

[0026] (8) Further process the anomaly probability vector obtained in step (7) using the Softmax and Argmax functions to obtain the anomaly cloud database number, i.e., the anomaly detection result.

[0027] Specifically, the anomaly cloud database number 0 indicates no anomaly, and other numbers indicate that the corresponding numbered database is abnormal.

[0028] (9) Calculate the total loss function based on the anomaly probability vector obtained in step (7) and the true labels of the cloud database KPI sample data, and iteratively train the TransAD model using the back propagation method until the TransAD model converges, thereby obtaining a preliminarily trained TransAD model.

[0029] (10) Verify the TransAD model preliminarily trained in step (9) using the test set obtained in step (4) until the detection accuracy obtained reaches the optimal value, thereby obtaining a trained TransAD model.

[0030] Preferably, step (5) comprises the following sub-steps:

[0031] (5-1) Input the KPI vector KPI i,j (where i represents the i-th cloud database number in the cluster, j represents the j-th KPI vector in the cloud database sample data, and i∈[1,m], j∈[1,n]) into the Q network of the Transformer encoder to obtain the query vector.

[0032] (5-2) Input different KPI vectors (KPI 1,1 , KPI i,2 , KPI i,j-1 , KPI i,j+1 ,…, KPI i,n ) belonging to the same cloud database and the same KPI vectors (KPI 2,j , KPI 3,j ,…, KPI i-1,j , KPI i+1,j … KPI m，j ) of different cloud databases into the K network of the Transformer encoder to obtain the key vector.

[0033] (5-3) Calculate the attention weight vector by dot product based on the query vector obtained in step (1) and the key vector obtained in step (2). Each weight in the attention weight vector represents the degree of attention of the KPI i,j vector to the KPI vector corresponding to the weight.

[0034] (5-4) According to the attention weight vector calculated in step (5-3), weight and fuse the KPI vectors that need to be paid attention to (i.e. different KPI vectors in the KPI sample data of the same cloud database, and the same KPI vectors in the KPI sample data of different cloud databases) to obtain the KPI encoding vector with fused correlation features.

[0035] Preferably, the total loss function comprises an L2 regularization loss function, an anomaly classification loss function calculated based on a cross-entropy loss function, and a contrast loss function:

[0036]

[0037] wherein the first term of the formula is an L2 regularization loss, the second term is an anomaly classification loss calculated based on a cross-entropy loss function, and the third term is a contrast loss. In the first term, λ is a coefficient of an L2 regularization term, and ω is a TransAD network weight vector; in the second term, β is a coefficient of an anomaly classification loss calculated based on a cross-entropy loss function, and L CE represents a cross-entropy loss function, Y i and Y i represent a real label vector and a predicted probability vector of the KPI sample of the i-th cloud database, respectively; in the third term, M represents the number of cloud databases in the cluster, and L d is a distance loss function for calculating a sample pair distance loss, I i and I j represent cloud database embedding vectors of the i-th cloud database KPI sample data and the j-th cloud database KPI sample data after being processed by a Transformer encoder and a cloud database embedding layer, respectively, and Y i,j represents whether the i-th cloud database and the j-th cloud database are similar (both being abnormal or both being normal are similar, and the value is 1; otherwise, the value is 0.

[0038] Preferably, the contrast loss function calculation process is as follows:

[0039] (9-1) Obtain embedding vectors of m cloud databases in the cluster, calculate the distance between each other, and thus obtain the cosine distance of each pair of cloud database embedding vectors.

[0040] (9-2) Based on the cosine distance of the cloud database sample pairs calculated in step (9-1), further calculation and processing are performed to obtain the cosine loss between the cloud database KPI sample data pairs;

[0041] (9-3) The cosine loss between each cloud database sample pair calculated in step (9-2) is accumulated to obtain the distance loss of all cloud database sample pairs.

[0042] According to another aspect of the present application, a cloud database anomaly detection system based on Transformer is provided, comprising:

[0043] The first module is configured to obtain key performance indicator (KPI) data of a cloud database, and perform dimension reduction processing on the collected KPI data of the cloud database to obtain the KPI data of the cloud database after dimension reduction processing.

[0044] The second module is configured to preprocess the KPI data of the cloud database after the dimension reduction processing of the first module to obtain cloud database KPI sample data.

[0045] The third module is configured to input the KPI data of the cloud database obtained by the second module into the pre-trained cloud database anomaly detection model TransAD to obtain an anomaly detection result.

[0046] Overall, compared with the prior art, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0047] (1) The present application adopts step (3), which extracts the correlation features of cloud database KPI within and between cloud databases through the Transformer encoder, and detects cloud database anomalies using the correlation features in the two dimensions, thereby effectively improving the precision rate while maintaining a high recall rate, and improving the overall performance of cloud database anomaly detection.

[0048] (2) The present application adopts step (3), which can realize that the Transformer encoder only extracts the correlation features of cloud databases within or between cloud databases through the mask matrix, so as to determine the influence of the correlation features of cloud database KPI within and between cloud databases on cloud database anomaly detection.

[0049] (3) The present application introduces a contrast loss function into the TransAD model, which enables the model to learn the similarity (distance) between cloud databases. This similarity is universal and can adapt to different cloud database KPI data sets, so the model has good adaptability to mixed cloud database KPI data sets (cloud database KPI data sets with multiple clusters). BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a framework diagram of the cloud database anomaly detection method based on the Transformer of the present application;

[0051] Figure 2 is a network structure diagram of the cloud database anomaly detection model TransAD of the present application;

[0052] Figure 3 is a schematic diagram of the data augmentation process of the cloud database KPI sample;

[0053] Figure 4 is a schematic diagram of the encoding process of the KPI vector by the Transformer encoder;

[0054] Figure 5is a schematic diagram of a mask matrix used in the attention mechanism of the present application.

[0055] Figure 6 is a flow chart of the cloud database anomaly detection method based on the Transformer of the present application. DETAILED DESCRIPTION

[0056] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0057] As shown in Figure 1 and Figure 6 The present application provides a cloud database anomaly detection method based on the Transformer, comprising the following steps:

[0058] A, acquiring the Key Performance Indicators (KPI) data of the cloud database, and performing dimension reduction processing on the collected KPI data of the cloud database to obtain the KPI data of the cloud database after dimension reduction processing.

[0059] Specifically, first, the KPI data of the cloud database is collected at a fixed time interval through the programming interface provided by the cloud database, and then the KPI data of the cloud database most relevant to the cloud database anomaly is selected from the collected KPI data of the cloud database based on expert experience as the KPI data of the cloud database after dimension reduction processing. The KPI data of the cloud database after dimension reduction processing is two-dimensional data with time dimension and index dimension, which contains n KPI vectors, wherein the value range of n is 10 to 80, and the preferred value is 14.

[0060] The granularity of KPI data collection of the cloud database in this step is 60 times / minute to 1 time / minute, and the preferred value is 12 times per minute.

[0061] The KPI data of the cloud database after dimension reduction processing in this step includes 14 KPI data such as CpuUseRate, MemoryUseRate, InnodbDataWrites, etc.

[0062] The advantage of this step is that by selecting the appropriate cloud database KPI collection granularity and selecting the KPI most relevant to the anomaly, the computing complexity of the anomaly detection model is reduced while the anomaly characteristics of the cloud database KPI time series data are retained.

[0063] B, pre-process the KPI data of the cloud database after dimension reduction processing in step A to obtain cloud database KPI sample data.

[0064] Specifically, first, the missing values in the KPI data of the cloud database caused by network jitter and the like are filled using linear interpolation to obtain the filled multi-dimensional KPI data of the cloud database; then the filled multi-dimensional KPI data of the cloud database is divided into multi-dimensional KPI subsequence samples with a fixed time span size and no overlap using a sliding window with a size of K (wherein the value range of K is 10 to 100, and preferably 20), each multi-dimensional KPI subsequence sample is two-dimensional data with a time dimension and an index dimension, the size of the time dimension is K, and the size of the index dimension is the number n of KPIs in the KPI data of the cloud database after dimension reduction processing.

[0065] C, input the KPI data of the cloud database obtained in step B into the pre-trained cloud database anomaly detection model TransAD to obtain an anomaly detection result.

[0066] Specifically, this step first inputs the KPI data of the cloud database obtained in step (2) into the cloud database anomaly detection model; then the cloud database anomaly detection model encodes the KPI vector in the KPI data of the cloud database, and in the encoding process, the information of the same KPI vector belonging to different cloud databases and the information of different KPI vectors belonging to the same cloud database are fused to obtain a KPI encoding vector that fuses the correlation features of the two dimensions of the cloud database and the cloud database; finally, the cloud database anomaly detection model judges the cloud database anomaly by using the KPI encoding vector.

[0067] The advantage of this step is that by using the Transformer encoder to capture the correlation features between the same KPIs of different cloud databases and the correlation features between different KPIs of the same cloud database to detect cloud database anomalies, cloud database anomalies can be detected from both the state of the cloud database itself and the overall level of the cloud database cluster, thereby improving the accuracy of anomaly detection.

[0068] Figure 2 is the structure of the cloud database anomaly detection model of the application, which includes three parts of a Transformer encoder, a cloud database embedding layer (a full connection layer) and an anomaly decision layer (a full connection layer) connected in sequence.

[0069] The first layer is the Transformer encoder, the Transformer encoder has a plurality of encoding blocks, each encoding block contains a multi-head self-attention sublayer and a feedforward neural network sublayer, and the specific structure is as follows:

[0070] The multi-head self-attention sublayer encodes KPI vectors (the number of KPI vectors is m*n) in KPI data (the number of KPI sample data of the cloud database is m, each KPI sample data of the cloud database contains n KPI vectors, and each KPI vector contains k data points) of multiple cloud databases in the same cluster to obtain KPI encoding vectors.

[0071] The advantage of the step is that the correlation features of the KPI vectors in the cloud database KPI sample data in different aspects can be captured by using the multi-head self-attention layer (containing multiple self-attention heads, each self-attention head maps the KPI vectors into different feature spaces and extracts the correlation features between the KPI vectors).

[0072] The feedforward neural network sublayer linearly transforms the KPI encoding vectors output by the multi-head self-attention sublayer to obtain linearly transformed KPI encoding vectors.

[0073] The second layer is a cloud database embedding layer (which is a fully connected neural network with 1024 neurons), and the input of the cloud database embedding layer is the KPI encoding vectors output by the first layer. The n KPI encoding vectors belonging to the same database are subjected to feature fusion processing to output a cloud database embedding vector that can represent the running state (health or abnormality) of the cloud database.

[0074] Specifically, the input data is an n*k-dimensional feature tensor, and the output data is a 1024-dimensional feature vector.

[0075] The third layer is an anomaly decision layer (which is a fully connected neural network with 1024 neurons), and the input of the anomaly decision layer is the cloud database embedding vector output by the second layer. The embedding vectors of multiple cloud databases in the cluster are subjected to feature fusion processing to output an m+1-dimensional anomaly probability vector.

[0076] Specifically, the probability vector output by the module needs to be further processed to obtain the number of abnormal cloud databases. First, the Softmax is subjected to normalization processing, and then the Argmax function is used to obtain the subscript of the maximum abnormal value in the anomaly probability vector. The subscript represents the number of the cloud database that is abnormal (when the value is 0, it means that there is no abnormality).

[0077] The advantage of the module is that the abnormal cloud database can be decided by considering the information of the cluster as a whole.

[0078] The cloud database anomaly detection model of the application is obtained by the following steps:

[0079] (1) Obtain cloud database key performance indicator (KPI) data, and perform dimension reduction processing on the collected cloud database KPI data to obtain cloud database KPI data after dimension reduction processing.

[0080] It should be noted that the process of collecting KPI data of the cloud database in this step is exactly the same as step (1) described above, and will not be repeated here.

[0081] (2) Preprocess the cloud database KPI data after dimension reduction processing in step (1) to obtain cloud database KPI sample data.

[0082] It should be noted that the process of preprocessing KPI data of the cloud database in this step is exactly the same as step (2) described above, and will not be repeated here.

[0083] (3) Inject abnormal KPI values into the cloud database KPI sample data obtained in step (2) to obtain labeled cloud database KPI sample data.

[0084] Specifically, the abnormal KPI value injection randomly selects samples from the cloud database KPI sample data, and injects abnormal KPI values into the cloud database KPI sample data. Among them, the cloud database KPI sample data injected with abnormal KPI values is called positive samples (label 1), and the cloud database KPI sample data not injected with abnormal KPI values is called negative samples (label 0).

[0085] (4) Divide the labeled cloud database KPI sample data obtained in step (3) into a training set and a test set according to a 7:3 ratio.

[0086] In addition, in cloud database anomaly detection, the cloud database KPI sample data set has a small sample problem. This makes it difficult for the TransAD model to effectively learn the characteristics of the cloud database KPI sample data, and cannot accurately detect cloud database anomalies, seriously affecting the performance of the TransAD model. In order to avoid the performance decline of the TransAD model caused by the lack of abnormal samples, the present application expands the cloud database KPI sample data through data enhancement. Specifically, the present application combines the characteristics of the cloud database anomaly detection scene and the TransAD model, and designs a data enhancement method, which generates new samples by exchanging the positions of the input KPI vectors. The input samples of the model include KPI data of m databases, and each cloud database contains n KPIs. There is no obvious time sequence relationship between different cloud databases. Therefore, the positions of the KPI vectors of different cloud databases in the sample data can be exchanged to generate new samples. The data enhancement process is shown in Figure 3 For a sample containing m cloud databases, a new sample.

[0087] The advantage of this step is that the cloud database KPI sample dataset is expanded, which can effectively improve the training effect of the model and improve the detection performance of the model.

[0088] (5) For the cloud database KPI sample data in the training set obtained in step (4), input the KPI samples of multiple cloud databases in the cluster under the same time window into the Transformer encoder to obtain the KPI encoding vector that fuses the intra-cloud database and inter-cloud database correlation features.

[0089] Specifically, in the encoding process, each input KPI vector fuses different KPI vectors belonging to the same cloud database and the same KPI vectors belonging to different cloud databases for feature fusion to obtain the KPI encoding vector that fuses the intra-cloud database and inter-cloud database correlation features.

[0090] The encoding process of the KPI vector by the Transformer encoder is as shown in Figure 4 The present step includes the following sub-steps:

[0091] (5-1) Input the KPI vector KPI i,j in the KPI sample into the Q network of the Transformer encoder to obtain the Query vector.

[0092] (5-2) Input different KPI vectors (KPI 1,1 , KPI i,2 , KPI i,j-1 , KPI i,j+1 , …, KPI i,n ) belonging to the same cloud database and the same KPI vectors (KPI 2,j , KPI 3,j , …, KPI i-1,j , KPI i+1,j , …, KPI m，j ) belonging to different cloud databases into the K network of the Transformer encoder to obtain the Keys vector.

[0093] It should be noted that the KPI sample data of multiple cloud databases in the cluster are input into the Transformer encoder at the same time, and the Transformer encoder fuses the information of all other KPI vectors for each KPI vector in the encoding process. However, there is no obvious correlation feature between different KPI vectors belonging to different cloud databases, and the invalid information fusion between them will affect the performance of the model. In order to avoid the feature fusion between different KPI vectors of different cloud databases, the application adopts a masking mechanism to avoid the mutual interference between different KPI vectors of different cloud databases.

[0094] Furthermore, the masking mechanism acts on the process of self-attention sublayer calculation. The core idea of the self-attention sublayer is to calculate the attention score between KPI vectors and use it as a weight for feature fusion. Therefore, for those KPI vectors that do not need to be paid attention to, their attention scores can be set to a very small negative number. In this way, these attention scores will become close to 0 after Softmax processing, which is equivalent to masking the corresponding KPI vectors, so as to make the self-attention mechanism focus on the KPI vector information that needs to be paid attention to.

[0095] Furthermore, the masking mechanism is implemented based on a mask matrix, which describes whether there is a correlation between KPI vectors. The mask matrix used in the attention mechanism of the application is as shown in Figure 5 , which is set according to the correlation between KPI vectors, and can avoid the information fusion between different KPI vectors of different cloud databases. Wherein, m represents the number of cloud databases, n represents the number of KPI of cloud databases, KPI i,j represents the jth KPI vector of the ith cloud database. Figure 5 The row vector in the above formula describes the set of KPI vectors that a certain KPI vector needs to pay attention to. If an element in the row vector is 1, it means paying attention to the KPI vector corresponding to the column number of the element; if it is not 1, it needs to be masked.

[0096] The advantage of this step is that the masking mechanism can avoid the problem of model performance degradation caused by invalid information fusion between different KPI vectors of different cloud databases.

[0097] (5-3) The query vector calculated according to step (1) and the key vector obtained in step (2) are calculated to obtain an attention weight (Weights) vector through dot product. Each weight in the attention weight vector represents the degree of attention of the KPI i,j vector corresponding to the weight to the KPI vector.

[0098] (5-4) According to the attention weight vector calculated in step (5-3), the KPI vector that needs to be focused on (i.e. different KPI vectors in the KPI sample data of the same cloud database, and the same KPI vector in the KPI sample data of different cloud databases) is weighted and fused to obtain a KPI encoding vector with fused correlation features.

[0099] (6) The KPI encoding vector obtained in step (5) is input into the cloud database embedding layer, and the cloud database embedding layer performs feature fusion on the n KPI encoding vectors belonging to the same cloud database to obtain a cloud database embedding vector that can represent the running state of the cloud database.

[0100] (7) The cloud database embedding vector obtained in step (6) is input into the anomaly decision layer, and the anomaly decision layer performs feature fusion on the embedding vectors of the m cloud databases in the cluster (obtained by processing the KPI sample data of multiple cloud databases belonging to the same cluster in the same time window) to obtain an anomaly probability vector with m+1 elements.

[0101] (8) The anomaly probability vector obtained in step (7) is further processed using the Softmax and Argmax functions to obtain the abnormal cloud database number, i.e. the anomaly detection result.

[0102] Specifically, the abnormal cloud database number 0 indicates no anomaly, and other numbers indicate that the database with the corresponding number is abnormal.

[0103] (9) The total loss function is calculated based on the anomaly probability vector obtained in step (7) and the true label of the cloud database KPI sample data, and the TransAD model is iteratively trained using the backpropagation method until the TransAD model converges, thereby obtaining a preliminarily trained TransAD model.

[0104] Specifically, the total loss function includes an L2 regularization loss function (which can prevent the model from overfitting during training) and an anomaly classification loss function calculated based on a cross-entropy loss function, and also includes a contrastive loss function. As shown in formula (1), the first term is the L2 regularization loss, the second term is the anomaly classification loss calculated based on the cross-entropy loss function, and the third term is the contrastive loss. Among them, λ in the first term is the coefficient of the L2 regularization term (more artificial experience setting), ω is the TransAD network weight vector; β in the second term is the coefficient of the anomaly classification loss calculated based on the cross-entropy loss function (according to artificial experience setting), L CE represents the cross-entropy loss function, Y i and Y iY and Y respectively represent the true label vector and the predicted probability vector of the KPI sample of the i-th cloud database; M in the third term represents the number of cloud databases in the cluster, and L d The distance loss function for calculating the distance loss of the sample pair is I i Y and Y respectively represent the true label vector and the predicted probability vector of the KPI sample of the i-th cloud database; M in the third term represents the number of cloud databases in the cluster, and L j Y and Y respectively represent the cloud database embedding vectors of the i-th cloud database KPI sample data and the j-th cloud database KPI sample data after being processed by the Transformer encoder and the cloud database embedding layer, Y i,j Y represents whether the i-th cloud database and the j-th cloud database are similar (both abnormal or both normal are similar, and the value is 1; otherwise, the value is 0)

[0105]

[0106] Further, L2 regularization is a method of penalizing model complexity, which adds a regular term of square weight in the loss function. L2 regularization loss is a common method to prevent model overfitting in training model, which will not be described here.

[0107] Further, the cross-entropy loss function is used to measure the difference between the true label of the KPI sample data of the cloud database and the detection label of the TransAD model. Specifically, a true label vector with a length of m+1 is used to describe the abnormal information of multiple cloud databases under the same cluster, and only one element in the vector is 1 and the other elements are 0. If the first element is 1, it means that no cloud database is abnormal, and if the other elements are 1, it means that the corresponding cloud database (the element subscript of the vector corresponds to the cloud database number) is abnormal. Then, the cross-entropy loss is calculated based on the true label vector and the m+1-dimensional abnormal probability vector obtained in step (6). The cross-entropy loss function is a common loss function for training classification models, and its detailed calculation process will not be described here.

[0108] Further, in the model training process, a contrast loss function is introduced, which enables the model to distinguish the embedding vectors of abnormal cloud databases in the cluster from the embedding vectors of normal cloud databases, so that the abnormal decision layer can more accurately identify the abnormal cloud databases.

[0109] The advantage of this step is that through the contrast loss function, the model can learn the similarity between cloud databases and detect cloud database anomalies according to the learned similarity. This similarity is universal for different cloud database KPI loads, allowing the model to have better adaptability to mixed cloud database KPI data sets (KPI data of cloud databases with multiple clusters).

[0110] The contrast loss function calculation process is:

[0111] (9-1) Obtain the embedding vectors of m cloud databases in the cluster, calculate the distance between each pair, and thus obtain the cosine distance between each pair of cloud database embedding vectors.

[0112] Specifically, the cloud database KPI sample data of m cloud databases input to the model are processed by the Transformer encoder and the cloud database embedding layer to obtain the cloud database embedding vector, denoted as [I′1, I′2, ..., I′]. m For m cloud databases, each pair can form a cloud database sample pair, denoted as […]. i ,I j The distance between sample pairs is calculated using cosine distance and normalized to between 0 and 1, as shown in the following formula (2):

[0113]

[0114] (9-2) Based on the cosine distance of the cloud database sample pairs calculated in step (9-1), further calculation and processing are performed to obtain the cosine loss between cloud database KPI sample data pairs.

[0115] Specifically, for similar cloud database sample pairs, we want to minimize their distance in the latent space; for dissimilar sample pairs, we want to maximize their distance in the latent space. Therefore, the goal of the loss function is to minimize the distance between similar sample pairs while maximizing the distance between dissimilar sample pairs. Distance loss L d The calculation is as shown in formula (3). Where, I i 'and I j 'Y' represents the cloud database embedding vectors of the i-th and j-th cloud database KPI sample data after processing by the Transformer encoder and the cloud database embedding layer, respectively; i,j This indicates whether the i-th cloud database and the j-th cloud database are similar (similar if both are anomalous or both are normal, value is 1; otherwise, they are dissimilar, value is 0); h is the boundary distance; considering the imbalance between anomalous and normal samples, a weight γ (value greater than 1) is introduced to reduce the negative impact of this imbalance.

[0116] L d (I′ i ,I′ j ,Y i,j ) = Y i,j d(I′ i ,I′ j )+γ(1-Y i,j )max(hd(I′ i ,I′ j ),0) (3) ​

[0117] The advantage of the sub-step is that the embedding vectors of the abnormal cloud database KPI sample data in the embedding space can be distinguished from the encoding vectors of the normal cloud database KPI sample data in the embedding space, so that the abnormal decision layer can effectively detect the abnormal cloud database.

[0118] (9-3) Accumulate the cosine loss between each pair of cloud database samples calculated in step (9-2) to obtain the distance loss of all pairs of cloud database samples.

[0119] The calculation of the present step is shown in formula 4.

[0120]

[0121] (10) Verify the TransAD model preliminarily trained in step (9) using the test set obtained in step (4) until the detection accuracy obtained reaches the optimal value, thereby obtaining the trained TransAD model.

[0122] Experimental results

[0123] The experimental environment of the present application: CPU is 8 AMD Ryzen 7 3700X@3.60GHz, GPU is 1 NVIDIA GeForce RTX 3060, memory is 16GB DDR4, hard disk capacity is 2TB, under Ubuntu operating system, using Python3.7.0 and machine learning library Pytorch (version 1.13.0), Scikit-learn (version 1.0.1) and other software packages to program the algorithm in this paper.

[0124] In order to illustrate the effectiveness of the detection model in the present application and the superiority of the present application for database performance anomaly detection, the present application has made relevant tests on real data sets (referred to as Dataset-A) and two open source tool generated data sets (referred to as Dataset-B and Dataset-C respectively). The model is compared with the related method JumpStarter and DBCatcher, and the comparison results are shown in Table 1. Among them, P, R and F represent the precision, recall and comprehensive performance of the model respectively.

[0125] Table 1: Anomaly detection effect of different methods

[0126]

[0127] From the table, compared with the JumpStarter method considering only the intra-instance KPI correlation, the present application improves the precision by 14.5%, 15.7% and 16% respectively and improves the comprehensive performance by 11.8%, 12.9% and 13.5% respectively on the three datasets; compared with the DBCatcher method considering only the inter-instance KPI correlation, the present application improves the precision by 10.8%, 9.1% and 8.4% respectively and improves the comprehensive performance by 5.5%, 4.2% and 4.3% respectively on the three datasets, but the recall rate decreases slightly. Although the DBCatcher method achieves a higher recall rate, when the KPI change trends of different instances deviate due to other non-anomalous factors (such as workload changes), the DBCatcher method will produce false positives, thereby causing the precision to decrease. Compared with the DBCatcher method, the present application considers both the overall level of the cloud database cluster system and the instance state level, and detects anomalies by using the correlation characteristics of the cloud database KPI in the inter-instance and intra-instance two dimensions. This greatly improves the anti-interference ability of the model to the deviation of the inter-instance KPI change trend caused by non-anomalous factors, significantly improves the detection precision, and achieves higher comprehensive performance.

[0128] Those skilled in the art will easily understand that the above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A cloud database anomaly detection method based on Transformer, characterized in that, Includes the following steps: A. Obtain key performance indicator (KPI) data for the cloud database, and perform dimensionality reduction processing on the collected cloud database KPI data to obtain the dimensionality-reduced cloud database KPI data. B. Preprocess the KPI data of the cloud database after dimensionality reduction in step A to obtain sample KPI data of the cloud database. C. Input the KPI data of the cloud database obtained in step B into the pre-trained cloud database anomaly detection model TransAD to obtain anomaly detection results; the cloud database anomaly detection model is trained through the following steps: (1) Obtain cloud database KPI data, perform dimensionality reduction processing on the collected cloud database KPI data, and obtain dimensionality-reduced cloud database KPI data. (2) Preprocess the KPI data of the cloud database after dimensionality reduction in step (1) to obtain cloud database KPI sample data; (3) Inject abnormal KPI values ​​into the cloud database KPI sample data obtained in step (2) to obtain labeled cloud database KPI sample data. (4) Divide the labeled cloud database KPI sample data obtained in step (3) into training set and test set in a ratio of 7:3; (5) For the cloud database KPI sample data in the training set obtained in step (4), input the KPI samples of multiple cloud databases in the same time window into the Transformer encoder to obtain the KPI encoding vector that integrates the correlation features within and between cloud databases. (6) Input the KPI encoding vector obtained in step (5) into the cloud database embedding layer. The cloud database embedding layer fuses the features of n KPI encoding vectors belonging to the same cloud database to obtain a cloud database embedding vector that can represent the running status of the cloud database. (7) Input the cloud database embedding vector obtained in step (6) into the anomaly decision layer. The anomaly decision layer performs feature fusion on the embedding vectors of m cloud databases in the cluster to obtain an anomaly probability vector with m+1 elements. This embedding vector is obtained by processing the KPI sample data of multiple cloud databases belonging to the same cluster within the same time window. (8) Use the Softmax and Argmax functions to further process the anomaly probability vector obtained in step (7) to obtain the anomaly cloud database number, i.e. the anomaly detection result; Specifically, an abnormal cloud database number of 0 indicates that there is no abnormality, while other numbers indicate that the corresponding database is abnormal; (9) Calculate the total loss function based on the abnormal probability vector obtained in step (7) and the true label of the cloud database KPI sample data, and use the backpropagation method to iteratively train the TransAD model until the TransAD model converges, thereby obtaining the initially trained TransAD model. (10) Use the test set obtained in step (4) to verify the TransAD model initially trained in step (9) until the detection accuracy reaches the optimal level, thereby obtaining the trained TransAD model.

2. The cloud database anomaly detection method based on Transformer according to claim 1, characterized in that, Step A specifically involves first collecting KPI data from the cloud database at fixed time intervals through the programming interface provided by the cloud database. Then, based on expert experience, the KPI data of the cloud database most relevant to the cloud database anomalies is selected from the collected KPI data as the dimensionality-reduced KPI data of the cloud database. The dimensionality-reduced KPI data of the cloud database is two-dimensional data with time and indicator dimensions, containing n KPI vectors, where the value of n ranges from 10 to 80.

3. The cloud database anomaly detection method based on Transformer according to claim 1 or 2, characterized in that, Step B specifically involves first filling in the missing values ​​in the cloud database's KPI data caused by network jitter using linear interpolation to obtain the filled multidimensional cloud database KPI data; then, using a sliding window of size K, the filled multidimensional cloud database KPI data is divided into multidimensional KPI subsequence samples with fixed time spans and no overlap. Each multidimensional KPI subsequence sample is two-dimensional data with a time dimension and an indicator dimension. The time dimension is K in size, and the indicator dimension is the number of KPIs n in the dimensionality-reduced cloud database KPI data.

4. The cloud database anomaly detection method based on Transformer according to claim 3, characterized in that, The cloud database anomaly detection model consists of three parts connected in sequence: a Transformer encoder, a cloud database embedding layer, and an anomaly decision layer. The first layer is a Transformer encoder, which has multiple coding blocks. Each coding block contains a multi-head self-attention sublayer and a feedforward neural network sublayer. The specific structure is as follows: The multi-head self-attention sub-layer encodes the KPI vectors in the KPI data of multiple cloud databases under the same cluster to obtain KPI encoded vectors. The number of KPI sample data in the cloud database is m, each cloud database KPI sample data contains n KPI vectors, each KPI vector contains k data points, and the number of KPI vectors is m*n. The feedforward neural network sublayer takes the KPI encoding vector output by the multi-head self-attention sublayer as its input and performs a linear transformation on the KPI encoding vector to obtain the linearly transformed KPI encoding vector. The second layer is the cloud database embedding layer, which is a fully connected neural network with 1024 neurons. Its input is the KPI encoding vector output by the first layer. It performs feature fusion processing on n KPI encoding vectors belonging to the same database to output a cloud database embedding vector that can represent the cloud database's operating status. The third layer is the anomaly decision layer, which is a fully connected neural network with 1024 neurons. Its input is the cloud database embedding vector output by the second layer. At the same time, it performs feature fusion processing on the embedding vectors of multiple cloud databases in the cluster to output an m+1 dimensional anomaly probability vector.

5. The cloud database anomaly detection method based on Transformer according to claim 4, characterized in that, Step (5) includes the following sub-steps: (5-1) Transfer the KPI vector from the KPI sample to KPI i,j Input the Q network of the Transformer encoder to obtain the query vector, where i represents the i-th cloud database number in the cluster, j represents the j-th KPI vector in the cloud database sample data, and i∈[1,m], j∈[1,n]; (5-2) Different KPI vectors (KPIs) belonging to the same cloud database 1,1 KPI i,2 KPI i,j-1 KPI i,j+1 KPI i,n ) and the same KPI vector (KPI) across different cloud databases 2,j KPI 3,j KPI i-1,j KPI i+1,j …KPI m，j Input the K network of the Transformer encoder to obtain the key vector; (5-3) Based on the query vector obtained in step (1) and the key vector obtained in step (2), the attention weight vector is calculated by dot product; each weight in the attention weight vector represents a KPI. i,j The degree of attention a vector pays to the KPI vector corresponding to that weight; (5-4) Based on the attention weight vector calculated in step (5-3), the KPI vectors that need to be focused are weighted and fused to obtain the KPI encoding vector with fused relevance features. This KPI vector is different KPI vectors in the KPI sample data of the same cloud database, and the same KPI vector in the KPI sample data of different cloud databases.

6. The cloud database anomaly detection method based on Transformer according to claim 5, characterized in that, The total loss function includes the L2 regularization loss function, the anomaly classification loss function calculated based on the cross-entropy loss function, and the contrastive loss function: The formula consists of three terms: the first term is the L2 regularization loss, the second term is the anomaly classification loss calculated based on the cross-entropy loss function, and the third term is the contrastive loss. In the first term, λ is the coefficient of the L2 regularization term, and ω is the TransAD network weight vector. In the second term, β is the coefficient of the anomaly classification loss calculated based on the cross-entropy loss function. CE Y represents the cross-entropy loss function. i and Y i ' represents the true label vector and predicted probability vector of the KPI sample of the i-th cloud database, respectively; M in the third term represents the number of cloud databases in the cluster, L d To calculate the distance loss function for sample pair distance loss, I i 'and I j 'Y' represents the cloud database embedding vectors of the i-th and j-th cloud database KPI sample data after processing by the Transformer encoder and the cloud database embedding layer, respectively. i,j This indicates whether the i-th cloud database and the j-th cloud database are similar. If both are abnormal or both are normal, they are considered similar and the value is 1; otherwise, the value is 0.

7. The cloud database anomaly detection method based on Transformer according to claim 6, characterized in that, The comparison loss function is calculated as follows: (9-1) Obtain the embedding vectors of m cloud databases in the cluster, calculate the distance between each pair, and thus obtain the cosine distance between each pair of cloud database embedding vectors. (9-2) Based on the cosine distance of the cloud database sample pairs calculated in step (9-1), further calculation and processing are performed to obtain the cosine loss between cloud database KPI sample data pairs. (9-3) Sum the cosine losses between each cloud database sample pair calculated in step (9-2) to obtain the distance loss of all cloud database sample pairs.

8. A cloud database anomaly detection system based on Transformer, characterized in that, include: The first module is used to obtain the key performance indicators (KPIs) data of the cloud database, and to perform dimensionality reduction processing on the collected cloud database KPI data to obtain the dimensionality-reduced cloud database KPI data. The second module is used to preprocess the KPI data of the cloud database after the dimensionality reduction process of the first module in order to obtain cloud database KPI sample data. The third module is used to input the cloud database KPI data obtained by the second module into the pre-trained cloud database anomaly detection model TransAD to obtain anomaly detection results; the cloud database anomaly detection model is trained through the following steps: (1) Obtain cloud database KPI data, perform dimensionality reduction processing on the collected cloud database KPI data, and obtain dimensionality-reduced cloud database KPI data. (2) Preprocess the KPI data of the cloud database after dimensionality reduction in step (1) to obtain cloud database KPI sample data; (3) Inject abnormal KPI values ​​into the cloud database KPI sample data obtained in step (2) to obtain labeled cloud database KPI sample data. (4) Divide the labeled cloud database KPI sample data obtained in step (3) into training set and test set in a ratio of 7:3; (5) For the cloud database KPI sample data in the training set obtained in step (4), input the KPI samples of multiple cloud databases in the same time window into the Transformer encoder to obtain the KPI encoding vector that integrates the correlation features within and between cloud databases. (6) Input the KPI encoding vector obtained in step (5) into the cloud database embedding layer. The cloud database embedding layer fuses the features of n KPI encoding vectors belonging to the same cloud database to obtain a cloud database embedding vector that can represent the running status of the cloud database. (7) Input the cloud database embedding vector obtained in step (6) into the anomaly decision layer. The anomaly decision layer performs feature fusion on the embedding vectors of m cloud databases in the cluster to obtain an anomaly probability vector with m+1 elements. This embedding vector is obtained by processing the KPI sample data of multiple cloud databases belonging to the same cluster within the same time window. (8) Use the Softmax and Argmax functions to further process the anomaly probability vector obtained in step (7) to obtain the anomaly cloud database number, i.e. the anomaly detection result; Specifically, an abnormal cloud database number of 0 indicates that there is no abnormality, while other numbers indicate that the corresponding database is abnormal; (9) Calculate the total loss function based on the abnormal probability vector obtained in step (7) and the true label of the cloud database KPI sample data, and use the backpropagation method to iteratively train the TransAD model until the TransAD model converges, thereby obtaining the initially trained TransAD model. (10) Use the test set obtained in step (4) to verify the TransAD model initially trained in step (9) until the detection accuracy reaches the optimal level, thereby obtaining the trained TransAD model.

Images