Multi-dimensional KPI data anomaly detection method and system

Abstract

The invention discloses a multi-dimensional KPI (Key Performance Indicator) data anomaly detection method. The method comprises the following steps: acquiring existing multi-dimensional KPI data information and preprocessing to construct a training data set; constructing a multi-dimensional KPI data anomaly detection initial model based on a space-time mask mechanism and a GRU network, and training to obtain a multi-dimensional KPI data anomaly detection model; and completing the anomaly detection of the multi-dimensional KPI data of the target by adopting the multi-dimensional KPI data anomaly detection model. The invention also discloses a system for realizing the multi-dimensional KPI data anomaly detection method. According to the method, through construction and training of the multi-dimensional KPI data anomaly detection model based on the space-time mask mechanism and the GRU network, anomaly detection of the multi-dimensional KPI data is achieved, and the method is higher in reliability and better in accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of computers, and in particular relates to a multi-dimensional KPI data anomaly detection method and system. Background Art

[0002] Anomaly detection aims to identify unusual behavior during system operation. Before implementing real-time monitoring of large systems, information operations monitoring platforms preselect certain metrics and collect monitoring data from each machine based on these metrics. When newly observed values ​​in the data deviate significantly from historical patterns, such as sudden spikes or dips, severe jitter, or changes in dependencies, this typically indicates a potential failure in the corresponding metric, machine, or system. Examples include network outages, high service call latency, and software crashes. Anomaly detection primarily identifies these phenomena by analyzing the time series of specific metrics. Upon receiving anomaly alerts, operations personnel immediately address and implement maintenance. The fundamental goal of anomaly detection research is to capture anomalies in time series metrics as accurately and quickly as possible, reducing redundant or false alerts. Because intelligent operations scenarios are characterized by high diversity, time-series dependencies, and inter-metric dependencies, anomaly detection models often need to capture these characteristics when learning data distributions. Therefore, anomaly detection in intelligent operations is particularly challenging. These monitoring metrics, known as key performance indicators (KPIs), are central to assessing system status. Considering the strong correlation between components of modern systems, anomaly detection usually requires simultaneous analysis of multidimensional time series data consisting of multiple KPIs.

[0003] Currently, commonly used multi-dimensional KPI data anomaly detection solutions are mainly based on reconstruction models. They reconstruct the normal data distribution through training models and use the reconstruction error to detect and distinguish abnormal data. However, the fully connected structure of the neural networks used in these solutions can easily cause the model to directly copy the input rather than learn complex distributions, resulting in a small difference in reconstruction error between normal and abnormal samples, which in turn leads to limited detection accuracy (i.e., the ideal shortcut problem). In addition, existing detection solutions do not consider the spatiotemporal dependencies (dynamic changes in time series, strong correlations between indicators) and high diversity (multimodal data distribution) of intelligent operations and maintenance, making it difficult to reconstruct normal samples (i.e., the problem of insufficient scenario adaptability). Finally, the masking or noise injection process of existing solutions, while able to suppress shortcuts, also increases the difficulty of normal learning, reducing the robustness of existing solutions (i.e., the double-edged sword effect problem). Summary of the Invention

[0004] One of the objectives of the present invention is to provide a multi-dimensional KPI data anomaly detection method with high reliability and good accuracy.

[0005] A second object of the present invention is to provide a system for implementing the multi-dimensional KPI data anomaly detection method.

[0006] The multi-dimensional KPI data anomaly detection method provided by the present invention comprises the following steps:

[0007] S1. Obtain existing multi-dimensional KPI data information;

[0008] S2. Preprocess the data information obtained in step S1 to construct a training data set;

[0009] S3. Build an initial model for multi-dimensional KPI data anomaly detection based on the spatiotemporal masking mechanism and GRU network;

[0010] S4. Using the training data set obtained in step S2, the initial model for multidimensional KPI data anomaly detection constructed in step S3 is trained to obtain a multidimensional KPI data anomaly detection model;

[0011] S5. Use the multi-dimensional KPI data anomaly detection model obtained in step S4 to complete the anomaly detection of the target multi-dimensional KPI data.

[0012] The pre-processing described in step S2 specifically includes the following steps:

[0013] The data information obtained in step S1 is subjected to missing value filling and standardization processing to complete the preprocessing process.

[0014] The initial model for multi-dimensional KPI data anomaly detection based on the spatiotemporal mask mechanism and GRU network described in step S3 is constructed, which specifically includes the following steps:

[0015] The constructed initial model for multi-dimensional KPI data anomaly detection includes a spatiotemporal mask module, a first time series capture module, a second time series capture module, a classifier module, an encoder module, and a decoder module;

[0016] The input data of the model is simultaneously input to the spatiotemporal mask module, the first time series capture module and the classifier module; the output of the spatiotemporal mask module is simultaneously input to the first time series capture module; the output of the first time series capture module and the output of the classifier module are together input to the encoder module; the output of the encoder module is input to the second time series capture module; the output of the second time series capture module is input to the decoder module; the output of the decoder module is the output data of the model;

[0017] A spatiotemporal mask module is constructed based on the spatiotemporal mask mechanism. The spatiotemporal mask module is used to introduce perturbations into the input data to facilitate the model’s learning of the temporal sequence of the data and the dependencies between indicators.

[0018] The first time series capture module and the second time series capture module are constructed based on the GRU network; the time series capture module is used to capture the book sequence features between data;

[0019] A classifier module is constructed based on the fully connected layer; the classifier module is used to provide conditional information for the encoder module and the decoder module;

[0020] Construct an encoder module based on the fully connected layer; the encoder module is used to extract latent variable information in the data;

[0021] A decoder module is constructed based on the fully connected layer; the decoder module is used to reconstruct data based on the obtained latent variable information and obtain the reconstruction result.

[0022] The processing process of the spatiotemporal mask module includes the following steps:

[0023] Time dimension masking: For the input data, randomly select a set proportion of time steps; replace the KPI feature value corresponding to the selected time step with the set mask value;

[0024] Spatial dimension masking: For each time step of the input data, randomly select a set proportion of KPI feature values; replace the current value of the selected KPI feature value with the set mask value;

[0025] After the time dimension mask and the space dimension mask, the time point and feature dimension corresponding to the mask are recorded.

[0026] The processing process of the first timing capture module and the second timing capture module is the same, and specifically includes the following steps:

[0027] The time series capture module includes GRU layer and fully connected layer;

[0028] The input is processed through the GRU layer and the fully connected layer in sequence to obtain the output of the timing capture module.

[0029] The processing of the classifier module includes the following steps:

[0030] The classifier module includes a flattening layer, several fully connected layers and a softmax function layer connected in series;

[0031] After the input data is processed by the flattening layer, the flattened data x is obtained. fla is x fla =flatten(x), x is the input data, flatten() is the flattening process;

[0032] The flattened data is processed through several fully connected layers in turn, and then processed through the softmax function layer to obtain the clustering probability clu p ;

[0033] According to the obtained clustering probability clu p , the calculated condition information c is c=argmax(clu p ).

[0034] The processing of the encoder module includes the following steps:

[0035] The encoder receives the time series feature y output by the time series capture module and the condition information c output by the classifier module, merges the time series feature y with the condition information c, and processes them through the fully connected layer to obtain the first intermediate feature y1, which is y1=w1*(y||c)+b1, where w1 is the weight in the fully connected network, || is the vector concatenation operation, and b1 is the bias;

[0036] The first intermediate feature is processed by the Leaky ReLU activation function to obtain the second intermediate feature y2: α p is the probability coefficient in the Leaky ReLU activation function;

[0037] The second intermediate feature y2 is processed by two fully connected layers to obtain the mean feature μ and standard deviation feature σ, μ = w2*y2+b2, σ = w3*y2+b3, where w2 is the weight in the first fully connected network, b2 is the bias in the first fully connected network, w3 is the weight in the second fully connected network, and b3 is the bias in the second fully connected network;

[0038] Finally, the latent variable information z output by the encoder module is calculated as z = μ + softplus(σ), where softplus() is an activation function that ensures that the standard deviation σ is always positive.

[0039] The processing of the decoder module specifically includes the following steps:

[0040] The decoder module receives the latent variable z output by the encoder module and the condition information c output by the classifier module; after concatenating the latent variable z and the condition information c, it is processed through the fully connected layer to obtain the first intermediate feature y3 as y3=w4*(z||c)+b4, where || is the vector concatenation operation, w4 is the weight of the fully connected layer, and b4 is the bias of the fully connected layer;

[0041] The first intermediate feature y3 is processed by the Leaky ReLU activation function to obtain the second intermediate feature y4: where α p is the probability coefficient in the Leaky ReLU activation function;

[0042] The second intermediate feature y4 is input to the gated recurrent unit GRU layer to obtain the temporal reconstruction feature h dec; Reconstruct the time series feature h dec Through a fully connected layer, the feature dimension is mapped back to the dimension of the original multidimensional KPI data, and the final output of the reconstructed multidimensional KPI data x' is x'=w5*h dec +b5, where w5 is the weight of the fully connected layer and b5 is the bias of the fully connected layer.

[0043] The training described in step S4 specifically includes the following steps:

[0044] The first loss is calculated using the following formula:

[0045] L1=ω*MSE msk +(1-ω)*MSE nmsk

[0046] Where L1 is the first loss value; ω is the set weight value; MSE msk is the mask loss, and MSE nmsk is a non-masked loss, and is the mask feature set; The mask part of the reconstructed data; is a set of non-masked features; is the non-masked part of the reconstructed data; MSE() is the mean square error calculation function;

[0047] The second loss is calculated using the following formula:

[0048] L2=α*(1+log((σ z ) 2 )-(μ z ) 2 -(σ z ) 2 )

[0049] Where L2 is the second loss value; α is the set weight value; μ z is the distribution mean of the latent variable information z; σ z is the standard deviation of the distribution of latent variable information z;

[0050] During training, the following formula is used as the total loss function L:

[0051] L=L1+φ*L2

[0052] Where φ is the set weight value.

[0053] The present invention also provides a system for implementing the multidimensional KPI data anomaly detection method, comprising a data acquisition module, a data processing module, a model construction module, a model training module and an anomaly detection module; the data acquisition module, the data processing module, the model construction module, the model training module and the anomaly detection module are connected in series in sequence; the data acquisition module is used to acquire existing multidimensional KPI data information and upload the data information to the data processing module; the data processing module is used to preprocess the acquired data information according to the received data information to construct a training data set, and upload the data information to the model construction module; the model construction module is used to construct a multidimensional KPI data anomaly detection initial model based on a spatiotemporal mask mechanism and a GRU network according to the received data information, and upload the data information to the model training module; the model training module is used to train the constructed multidimensional KPI data anomaly detection initial model according to the received data information using the obtained training data set to obtain a multidimensional KPI data anomaly detection model, and upload the data information to the anomaly detection module; the anomaly detection module is used to complete the target multidimensional KPI data anomaly detection using the obtained multidimensional KPI data anomaly detection model according to the received data information.

[0054] The multidimensional KPI data anomaly detection method and system provided by the present invention not only realizes anomaly detection of multidimensional KPI data by constructing and training a multidimensional KPI data anomaly detection model based on a spatiotemporal mask mechanism and a GRU network, but also has higher reliability and better accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Schematic diagram of the process of the present invention.

[0056] Figure 2 Schematic diagram of the functional modules of the system of the present invention. DETAILED DESCRIPTION

[0057] like Figure 1 The figure shows a flow chart of the method of the present invention: The multi-dimensional KPI data anomaly detection method disclosed in the present invention comprises the following steps:

[0058] S1. Obtain existing multi-dimensional KPI data information;

[0059] S2. Preprocess the data information obtained in step S1 to construct a training data set;

[0060] In specific implementation, the preprocessing includes the following steps:

[0061] The data information obtained in step S1 is subjected to missing value filling and standardization processing to complete the preprocessing process.

[0062] S3. Build an initial model for multi-dimensional KPI data anomaly detection based on the spatiotemporal masking mechanism and the GRU network. This model includes the following steps:

[0063] The constructed initial model for multi-dimensional KPI data anomaly detection includes a spatiotemporal mask module, a first time series capture module, a second time series capture module, a classifier module, an encoder module, and a decoder module;

[0064] The input data of the model is simultaneously input to the spatiotemporal mask module, the first time series capture module and the classifier module; the output of the spatiotemporal mask module is simultaneously input to the first time series capture module; the output of the first time series capture module and the output of the classifier module are together input to the encoder module; the output of the encoder module is input to the second time series capture module; the output of the second time series capture module is input to the decoder module; the output of the decoder module is the output data of the model;

[0065] A spatiotemporal mask module is constructed based on the spatiotemporal mask mechanism. The spatiotemporal mask module is used to introduce perturbations into the input data to facilitate the model’s learning of the temporal sequence of the data and the dependencies between indicators.

[0066] The first time series capture module and the second time series capture module are constructed based on the GRU network; the time series capture module is used to capture the book sequence features between data;

[0067] A classifier module is constructed based on the fully connected layer; the classifier module is used to provide conditional information for the encoder module and the decoder module;

[0068] Construct an encoder module based on the fully connected layer; the encoder module is used to extract latent variable information in the data;

[0069] A decoder module is constructed based on the fully connected layer; the decoder module is used to reconstruct data based on the obtained latent variable information and obtain the reconstruction result.

[0070] In specific implementation, the processing process of the spatiotemporal mask module includes the following steps:

[0071] Time dimension masking: For the input data, a set proportion (usually 3% to 10%) of time steps are randomly selected. The KPI feature values ​​corresponding to the selected time steps are replaced with a set mask value. This mask value can be a fixed 0 or a small random noise value. The intention of this approach is that if the model does not learn and utilize the time information before and after the time point (i.e., the context), it will not be able to accurately reconstruct the masked data. Therefore, this masking method can effectively force the model to capture and learn the dynamic dependencies between the previous and next time points in the KPI sequence.

[0072] Spatial dimension masking: For each time step of the input data, a set proportion (usually 3% to 10%) of KPI feature values ​​are randomly selected; the current value of the selected KPI feature value is replaced with the set mask value; this mask value can be a fixed 0 or a small random noise value. The intention of this approach is that if the model does not learn and utilize other unmasked indicator information at the same time point and the interdependence between them, it will be difficult to accurately infer and restore the masked indicator value. Therefore, this masking method can effectively promote the model to learn and understand the complex and close interdependence between different KPI indicators at the same time.

[0073] After the time dimension mask and the space dimension mask, the time point and feature dimension corresponding to the mask are recorded.

[0074] In specific implementation, the processing process of the first timing capture module and the second timing capture module is the same, which specifically includes the following steps:

[0075] The time series capture module includes GRU layer and fully connected layer;

[0076] The input is processed by the GRU layer and the fully connected layer in sequence to obtain the output of the timing capture module;

[0077] In specific implementation, the processing process of the classifier module includes the following steps:

[0078] The classifier module includes a flattening layer, several fully connected layers and a softmax function layer connected in series;

[0079] After the input data is processed by the flattening layer, the flattened data x is obtained. fla is x fla =flatten(x), x is the input data, flatten() is the flattening process;

[0080] The flattened data is processed through several fully connected layers in turn, and then processed through the softmax function layer to obtain the clustering probability clu p ;

[0081] According to the obtained clustering probability clu p , the calculated condition information c is c=argmax(clu p );

[0082] The classifier of the present invention can adaptively learn classification; this is because there is no guarantee that all data sets of intelligent operation and maintenance can be effectively classified using the same fixed clustering algorithm, and ensure that the classification method is optimal; for example, the K-means algorithm tends to find spherical clusters, while DBSCAN can find clusters of any shape, and they each have their own applicable scenarios; if the clustering algorithm is fixed, once the application scenario or data characteristics change, it is necessary to re-evaluate and possibly exhaustively enumerate multiple clustering algorithms to select the best solution, which in practice will become a labor-intensive task (because clustering algorithms are of many types and mature). The present invention uses a loss function to directly supervise how the classifier splits a complex distribution into multiple simple sub-distributions, thereby ensuring that the splitting method is optimal for the current reconstruction task; if the splitting method is inappropriate, resulting in the complexity of the sub-distribution not being significantly reduced, then the loss function will still be large, and this large loss will be fed back to the classifier through gradient backpropagation, prompting it to find a better and easier to learn splitting method.

[0083] In specific implementation, the processing process of the encoder module includes the following steps:

[0084] The encoder receives the time series feature y output by the time series capture module and the condition information c output by the classifier module, merges the time series feature y with the condition information c, and processes them through the fully connected layer to obtain the first intermediate feature y1, which is y1=w1*(y||c)+b1, where w1 is the weight in the fully connected network, || is the vector concatenation operation, and b1 is the bias;

[0085] The first intermediate feature is processed by the Leaky ReLU activation function to obtain the second intermediate feature y2: α p is the probability coefficient in the Leaky ReLU activation function;

[0086] The second intermediate feature y2 is processed by two fully connected layers to obtain the mean feature μ and standard deviation feature σ, μ = w2*y2+b2, σ = w3*y2+b3, where w2 is the weight in the first fully connected network, b2 is the bias in the first fully connected network, w3 is the weight in the second fully connected network, and b3 is the bias in the second fully connected network;

[0087] Finally, the latent variable information z output by the encoder module is calculated as z = μ + softplus(σ), where softplus() is an activation function that ensures that the standard deviation σ is always positive.

[0088] In specific implementation, the processing process of the decoder module includes the following steps:

[0089] The decoder module receives the latent variable z output by the encoder module and the condition information c output by the classifier module; after concatenating the latent variable z and the condition information c, it is processed through the fully connected layer to obtain the first intermediate feature y3 as y3=w4*(z||c)+b4, where || is the vector concatenation operation, w4 is the weight of the fully connected layer, and b4 is the bias of the fully connected layer;

[0090] The first intermediate feature y3 is processed by the Leaky ReLU activation function to obtain the second intermediate feature y4: where α p is the probability coefficient in the Leaky ReLU activation function;

[0091] The second intermediate feature y4 is input to the gated recurrent unit GRU layer to obtain the temporal reconstruction feature h dec ; Reconstruct the time series feature h dec Through a fully connected layer, the feature dimension is mapped back to the dimension of the original multidimensional KPI data, and the final output of the reconstructed multidimensional KPI data x' is x'=w5*h dec +b5, where w5 is the weight of the fully connected layer and b5 is the bias of the fully connected layer.

[0092] The encoder module, classifier module and decoder module of the present invention adopt conditional generation logic. The conditional generation mechanism itself does not increase the risk of "ideal shortcuts". Assuming that the complex distribution of the original data is G(x), it is very difficult for the model to learn G(x). After conditional generation, G(x) under different conditions c k is split into several simpler distributions G k (x), the model learns these G k (x) will be easier; if the model takes an "ideal shortcut" and learns f(x) = x, then under conditional generation, it only learns f under each condition. k (x) = x; this is essentially no different from the unconditional case f(x) = x, so the main function of conditional generation is to reduce the complexity of normal learning, rather than making the "ideal shortcut" more likely to occur;

[0093] S4. Using the training data set obtained in step S2, the initial model for multidimensional KPI data anomaly detection constructed in step S3 is trained to obtain a multidimensional KPI data anomaly detection model;

[0094] In specific implementation, the training process includes the following steps:

[0095] The first loss is calculated using the following formula:

[0096] L1=ω*MSE msk +(1-ω)*MSEnmsk

[0097] Where L1 is the first loss value; ω is the set weight value (usually set to a larger value, such as 0.8); MSE msk is the mask loss, and MSE nmsk is a non-masked loss, and is the mask feature set; The mask part of the reconstructed data; is a set of non-masked features; is the non-masked part of the reconstructed data; MSE() is the mean square error calculation function;

[0098] In the first loss, the weighted loss function multiplies the reconstruction loss of the masked part by a larger weight, while multiplying the remaining unmasked part by only a smaller weight. This is intended to weaken the reward for restoring the input data through shortcuts, while increasing the reward for restoring the data through the normal learning path. This weakens the impact of the "double-edged sword" problem on normal learning to a lower degree and strengthens the impact on the shortcut problem.

[0099] The second loss is calculated using the following formula:

[0100] L2=α*(1+log((σ z ) 2 )-(μ z ) 2 -(σ z ) 2 )

[0101] Where L2 is the second loss value; α is the set weight value; μ z is the distribution mean of the latent variable information z; σ z is the standard deviation of the distribution of latent variable information z;

[0102] During training, the following formula is used as the total loss function L:

[0103] L=L1+φ*L2

[0104] Where φ is the set weight value.

[0105] The above training process is carried out offline.

[0106] S5. Using the multidimensional KPI data anomaly detection model obtained in step S4, complete the target multidimensional KPI data anomaly detection;

[0107] When performing anomaly detection, the target data is first preprocessed (including missing value filling and normalization). The processed data is then directly processed by the trained encoder and decoder modules to obtain the final reconstruction error.

[0108] Then, a judgment is made based on the obtained reconstruction error: if the reconstruction error is greater than the set value, the data is directly judged to be abnormal; otherwise, the data is judged to be normal.

[0109] The solution of the present invention can effectively solve the "ideal shortcut" problem that is prevalent in unsupervised reconstruction-based anomaly detection methods in intelligent operation and maintenance scenarios; in response to the challenge that traditional reconstruction methods are limited in effectiveness in intelligent operation and maintenance due to data characteristics (such as temporal relationships, dependencies and diversity between indicators), and the problem that "ideal shortcut" solutions in other fields are insufficiently applicable in this scenario, the solution of the present invention has made targeted improvements by introducing a spatiotemporal masking mechanism and adaptive conditional generation logic; the solution of the present invention not only considers the inherent temporal relationships and complex dependencies between temporal indicators of intelligent operation and maintenance data through spatiotemporal masking, but also takes into account the diversity characteristics of data in different operation and maintenance scenarios; more importantly, combined with the designed first loss function, it can effectively weaken the "double-edged sword" effect that the introduction of the masking mechanism may cause on normal mode learning; this design significantly reduces the possibility of model learning "ideal shortcuts", and also reduces the difficulty of the model learning real complex data distribution, thereby improving the reliability and accuracy of the solution of the present invention.

[0110] like Figure 2 The figure shows a schematic diagram of the functional modules of the system of the present invention: the system disclosed in the present invention for implementing the multi-dimensional KPI data anomaly detection method includes a data acquisition module, a data processing module, a model construction module, a model training module and an anomaly detection module; the data acquisition module, the data processing module, the model construction module, the model training module and the anomaly detection module are connected in series in sequence; the data acquisition module is used to acquire existing multi-dimensional KPI data information and upload the data information to the data processing module; the data processing module is used to pre-process the acquired data information based on the received data information to construct a training data set, and upload the data information to the model construction module; the model construction module is used to construct an initial model for multi-dimensional KPI data anomaly detection based on the received data information and a spatiotemporal mask mechanism and a GRU network, and upload the data information to the model training module; the model training module is used to train the constructed initial model for multi-dimensional KPI data anomaly detection based on the received data information using the obtained training data set to obtain a multi-dimensional KPI data anomaly detection model, and upload the data information to the anomaly detection module; the anomaly detection module is used to complete the anomaly detection of the target multi-dimensional KPI data using the obtained multi-dimensional KPI data anomaly detection model based on the received data information.

Claims

1. A multi-dimensional KPI data anomaly detection method, comprising the following steps: S1. Obtain existing multi-dimensional KPI data information; S2. Preprocess the data information obtained in step S1 to construct a training data set; S3. Build an initial model for multi-dimensional KPI data anomaly detection based on the spatiotemporal masking mechanism and GRU network; S4. Using the training data set obtained in step S2, the initial model for multidimensional KPI data anomaly detection constructed in step S3 is trained to obtain a multidimensional KPI data anomaly detection model; S5. Use the multi-dimensional KPI data anomaly detection model obtained in step S4 to complete the anomaly detection of the target multi-dimensional KPI data.

2. The multi-dimensional KPI data anomaly detection method according to claim 1 is characterized in that The pre-processing described in step S2 specifically includes the following steps: The data information obtained in step S1 is subjected to missing value filling and standardization processing to complete the preprocessing process.

3. The multi-dimensional KPI data anomaly detection method according to claim 2 is characterized in that The initial model for multi-dimensional KPI data anomaly detection based on the spatiotemporal mask mechanism and GRU network described in step S3 is constructed, which specifically includes the following steps: The constructed initial model for multi-dimensional KPI data anomaly detection includes a spatiotemporal mask module, a first time series capture module, a second time series capture module, a classifier module, an encoder module, and a decoder module; The input data of the model is simultaneously input to the spatiotemporal mask module, the first temporal capture module and the classifier module; The output of the spatiotemporal mask module is simultaneously input to the first timing capture module; The output of the first timing capture module and the output of the classifier module are used together as the input of the encoder module. The output of the encoder module is used as the input of the second timing capture module; The output of the second timing capture module serves as the input of the decoder module; The output of the decoder module is the output data of the model; Construct a spatiotemporal mask module based on the spatiotemporal mask mechanism; The spatiotemporal mask module is used to introduce perturbations into the input data to facilitate the model’s learning of the temporal sequence of data and the dependencies between indicators. Constructing a first time series capture module and a second time series capture module based on the GRU network; The time sequence capture module is used to capture the book sequence characteristics between data; Build a classifier module based on the fully connected layer; The classifier module is used to provide conditional information for the encoder module and the decoder module; Construct an encoder module based on the fully connected layer; the encoder module is used to extract latent variable information in the data; Build a decoder module based on the fully connected layer; The decoder module is used to reconstruct the data based on the obtained latent variable information and obtain the reconstruction result.

4. The multi-dimensional KPI data anomaly detection method according to claim 3 is characterized in that The processing process of the spatiotemporal mask module includes the following steps: Time dimension masking: For the input data, randomly select a set proportion of time steps; replace the KPI feature value corresponding to the selected time step with the set mask value; Spatial dimension masking: For each time step of the input data, randomly select a set proportion of KPI feature values; replace the current value of the selected KPI feature value with the set mask value; After the time dimension mask and the space dimension mask, the time point and feature dimension corresponding to the mask are recorded.

5. The multi-dimensional KPI data anomaly detection method according to claim 4 is characterized in that The processing process of the first timing capture module and the second timing capture module is the same, and specifically includes the following steps: The time series capture module includes GRU layer and fully connected layer; The input is processed through the GRU layer and the fully connected layer in sequence to obtain the output of the timing capture module.

6. The multi-dimensional KPI data anomaly detection method according to claim 5 is characterized in that The processing of the classifier module includes the following steps: The classifier module includes a flattening layer, several fully connected layers and a softmax function layer connected in series; After the input data is processed by the flattening layer, the flattened data x is obtained. fla is x fla =flatten(x), x is the input data, flatten() is the flattening process; The flattened data is processed through several fully connected layers in turn, and then processed through the softmax function layer to obtain the clustering probability clu p ; According to the obtained clustering probability clu p , the calculated condition information c is c=argmax(clu p ).

7. The multi-dimensional KPI data anomaly detection method according to claim 6 is characterized in that The processing of the encoder module includes the following steps: The encoder receives the time series feature y output by the time series capture module and the condition information c output by the classifier module, merges the time series feature y with the condition information c, and processes them through the fully connected layer to obtain the first intermediate feature y1, which is y1=w1*(y||c)+b1, where w1 is the weight in the fully connected network, || is the vector concatenation operation, and b1 is the bias; The first intermediate feature is processed by the Leaky ReLU activation function to obtain the second intermediate feature y2: α p is the probability coefficient in the Leaky ReLU activation function; The second intermediate feature y2 is processed by two fully connected layers to obtain the mean feature μ and standard deviation feature σ, μ = w2*y2+b2, σ = w3*y2+b3, where w2 is the weight in the first fully connected network, b2 is the bias in the first fully connected network, w3 is the weight in the second fully connected network, and b3 is the bias in the second fully connected network; Finally, the latent variable information z output by the encoder module is calculated as z = μ + softplus(σ), where softplus() is an activation function that ensures that the standard deviation σ is always positive.

8. The multi-dimensional KPI data anomaly detection method according to claim 7 is characterized in that The processing process of the decoder module specifically includes the following steps: the decoder module receives the latent variable z output by the encoder module and the condition information c output by the classifier module; after splicing the latent variable z and the condition information c, the latent variable z and the condition information c are processed through the fully connected layer to obtain the first intermediate feature y3 as y3=w4*(z||c)+b4, where || is the vector splicing operation, w4 is the weight of the fully connected layer, and b4 is the bias of the fully connected layer; The first intermediate feature y3 is processed by the Leaky ReLU activation function to obtain the second intermediate feature y4: where α p is the probability coefficient in the Leaky ReLU activation function; The second intermediate feature y4 is input to the gated recurrent unit GRU layer to obtain the temporal reconstruction feature h dec ; Reconstruct the time series feature h dec Through a fully connected layer, the feature dimension is mapped back to the dimension of the original multidimensional KPI data, and the final output of the reconstructed multidimensional KPI data x' is x'=w5*h dec +b5, where w5 is the weight of the fully connected layer and b5 is the bias of the fully connected layer.

9. The multi-dimensional KPI data anomaly detection method according to claim 8, characterized in that The training described in step S4 specifically includes the following steps: The first loss is calculated using the following formula: L1=ω*MSE msk +(1-ω)*MSE nmsk Where L1 is the first loss value; ω is the set weight value; MSE msk is the mask loss, and MSE nmsk is a non-masked loss, and is the mask feature set; The mask part of the reconstructed data; is a set of non-masked features; is the non-masked part of the reconstructed data; MSE() is the mean square error calculation function; The second loss is calculated using the following formula: L2=α*(1+log((σ z ) 2 )-(m z ) 2 -(s z ) 2 ) Where L2 is the second loss value; α is the set weight value; μ z is the distribution mean of the latent variable information z; σ z is the standard deviation of the distribution of latent variable information z; During training, the following formula is used as the total loss function L: L=L1+φ*L2 Where φ is the set weight value.

10. A system for implementing the multi-dimensional KPI data anomaly detection method according to any one of claims 1 to 9, characterized in that It includes a data acquisition module, a data processing module, a model building module, a model training module, and an anomaly detection module; the data acquisition module, the data processing module, the model building module, the model training module, and the anomaly detection module are connected in series in sequence; the data acquisition module is used to obtain existing multi-dimensional KPI data information and upload the data information to the data processing module; The data processing module is used to pre-process the acquired data information based on the received data information to construct a training data set and upload the data information to the model construction module; The model building module is used to build an initial model for multi-dimensional KPI data anomaly detection based on the received data information, the spatiotemporal mask mechanism and the GRU network, and upload the data information to the model training module; The model training module is used to train the constructed multi-dimensional KPI data anomaly detection initial model based on the received data information and the obtained training data set, obtain the multi-dimensional KPI data anomaly detection model, and upload the data information to the anomaly detection module; The anomaly detection module is used to complete the anomaly detection of the target multidimensional KPI data based on the received data information and the obtained multidimensional KPI data anomaly detection model.

Images