Server anomaly detection method and device, electronic equipment, storage medium and program product
By preprocessing and dimensionality reduction of server KPI data, and using the GRU neural network model with attention mechanism for training, the problem of inaccurate detection of server abnormalities in the existing technology is solved, and efficient and accurate abnormal detection effect is achieved .
Patent Information
- Application Number
- CN202411687400.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-22
- Publication Date
- 2025-02-18
AI Technical Summary
The existing server anomaly detection model cannot accurately identify all problems when facing diverse anomalies, especially when detecting exceptions closely related to time information.
By preprocessing the server KPI data, KPI time series data is obtained; then, the KPI time series data is dimensionally reduced to obtain KPI training data; then, the KPI training data is input to the initial GRU neural network with attention mechanism Training is performed in the model to obtain the server exception detection model.
Accurate detection of server exceptions is achieved, the accuracy and efficiency of abnormality detection is improved, and the false alarm rate and missed rate are reduced.
Smart Images
Figure CN119473689A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computers, and in particular to a server anomaly detection model method, device, electronic device, storage medium and program product. Background Art
[0002] With the rapid development of Internet technology, data centers, as the core infrastructure supporting network application services, are growing in size and complexity. Servers, as the infrastructure supporting the operation of data centers, have a direct impact on the normal operation of services and user experience.
[0003] Existing server anomaly detection models have the defect of being unable to accurately identify all problems when faced with diverse anomalies, especially when detecting anomalies that are closely related to time information.
[0004] Therefore, there is an urgent need for a solution that can accurately detect server anomalies. Summary of the invention
[0005] The embodiments of the present application provide a server anomaly detection method, device, electronic device, storage medium and program product to achieve the effect of accurately detecting server anomalies.
[0006] In a first aspect, an embodiment of the present application provides a method for training a server anomaly detection model, comprising:
[0007] The acquired server KPI data is preprocessed to obtain KPI time series data; wherein the KPI time series data represents the change of performance indicators of the server at different time points.
[0008] The KPI time series data is subjected to dimensionality reduction processing to obtain KPI training data.
[0009] The KPI training data is input into an initial GRU neural network model to process the KPI training data based on the attention mechanism in the initial GRU neural network model to obtain a server anomaly detection model; wherein the server anomaly detection model is used to process the server KPI data to obtain a server anomaly detection result.
[0010] In a second aspect, an embodiment of the present application provides a server anomaly detection method, comprising:
[0011] Get server KPI data.
[0012] The server KPI data is input into a server anomaly detection model to obtain a server anomaly detection result output by the server anomaly detection model, wherein the server anomaly detection model is trained by various possible training methods of the first aspect.
[0013] In a third aspect, an embodiment of the present application provides a training device for a server anomaly detection model, comprising:
[0014] The data processing module is used to pre-process the acquired server KPI data to obtain KPI time series data; wherein the KPI time series data represents the changes in performance indicators of the server at different time points.
[0015] The feature dimension reduction module is used to perform dimension reduction processing on the KPI time series data to obtain KPI training data.
[0016] A model training module is used to input the KPI training data into an initial GRU neural network model to process the KPI training data based on the attention mechanism in the initial GRU neural network model to obtain a server anomaly detection model; wherein the server anomaly detection model is used to process the server KPI data to obtain a server anomaly detection result.
[0017] In a fourth aspect, an embodiment of the present application provides a server anomaly detection device, including:
[0018] The acquisition module is used to obtain server KPI data.
[0019] The detection module is used to input the server KPI data into a server anomaly detection model to obtain a server anomaly detection result output by the server anomaly detection model, wherein the server anomaly detection model is trained by various possible training methods of the first aspect.
[0020] In a fifth aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;
[0021] The memory stores computer executable instructions.
[0022] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.
[0023] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-readable storage medium is stored computer execution instructions, and when the computer execution instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementation methods of the first aspect.
[0024] In a seventh aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.
[0025] The embodiments of the present application provide a server anomaly detection method, device, electronic device, storage medium and program product, which obtains KPI time series data by preprocessing the acquired server KPI data. These data characterize the changes in the performance indicators of the server at different time points. Then, the KPI time series data is subjected to dimensionality reduction processing to obtain KPI training data. Then, the KPI training data is input into the initial GRU neural network model with an attention mechanism for training to obtain a server anomaly detection model. In this way, key information can be efficiently extracted from complex and changeable KPI data, and the attention mechanism can be used to enhance the sensitivity of the model to abnormal features, and finally a model that can accurately detect server anomalies is obtained, thereby improving the accuracy and efficiency of server anomaly detection and reducing the false alarm rate and missed alarm rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0027] Figure 1 A flow chart of a method for training a server anomaly detection model provided in an embodiment of the present application Figure 1 ;
[0028] Figure 2 A flow chart of a method for training a server anomaly detection model provided in an embodiment of the present application Figure 2 ;
[0029] Figure 3 A flow chart of a server anomaly detection method provided in an embodiment of the present application;
[0030] Figure 4 A structural diagram of a server anomaly detection model training device provided in an embodiment of the present application Figure 1 ;
[0031] Figure 5 A structural diagram of a server anomaly detection model training device provided in an embodiment of the present application Figure 2 ;
[0032] Figure 6 A schematic diagram of the structure of a server anomaly detection device provided in an embodiment of the present application;
[0033] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0034] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0035] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0036] With the development of Internet technology, data centers, as core facilities supporting network applications, data storage and processing, are growing in importance and size. Faced with the application of cloud computing, big data, the Internet of Things and other technologies, the complexity of data centers has increased dramatically, and server clusters continue to process massive amounts of data to support online services.
[0037] The expansion of data center scale has brought operational challenges, especially server performance monitoring and exception handling. Server performance directly affects the overall performance and stability of the data center. Therefore, effectively monitoring server performance indicators (Key Performance Indicators, KPIs) and promptly discovering and resolving exceptions are key tasks of operation and maintenance. KPI data covers aspects such as CPU usage, memory occupancy, disk I / O, and network bandwidth, and fully reflects the status of the server. As the number of servers increases and the monitoring frequency increases, the amount of KPI data surges, and how to quickly and accurately detect anomalies becomes a problem.
[0038] Traditional anomaly detection relies on manual experience, is inefficient and difficult to deal with complex situations. Data-driven methods based on machine learning and deep learning can automatically learn KPI data features, build models that accurately identify anomalies, and improve detection efficiency and accuracy. However, in practice, KPI data usually has time series characteristics, and traditional machine learning algorithms often have difficulty effectively processing such time-dependent data; in addition, there are many types of servers in data centers with different performance indicators. How to build a highly general and adaptable anomaly detection model is an urgent problem to be solved.
[0039] The present application provides a server anomaly detection method, device, equipment, storage medium and program product, which can solve the above problems.
[0040] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0041] Figure 1 A flow chart of a method for training a server anomaly detection model provided in an embodiment of the present application Figure 1 ,like Figure 1 As shown, the method includes:
[0042] S101. Preprocess the acquired server KPI data to obtain KPI time series data; wherein the KPI time series data represents the change of performance indicators of the server at different time points.
[0043] For example, server KPI data refers to a series of quantitative indicators used to measure server performance and operating status. Server KPI data includes: CPU usage, memory usage, disk I / O, network bandwidth, response time, process status, number of sessions, temperature and power consumption, etc. KPI time series data refers to server KPI data arranged in chronological order, and each data point represents the server KPI data at a certain point in time.
[0044] Exemplarily, the server KPI data within a period of time is obtained through various means (such as log collection, API interface call, etc.). These data may come from different data sources, such as operating system monitoring, application monitoring, network monitoring, etc. The server KPI data is preprocessed, and the preprocessing methods include standardization processing, outlier processing, and missing value processing. Data standardization refers to the processing method of converting server KPI data into data of the same size. The specific standardization processing methods include minimum-maximum standardization (Min-Max Scaling) and Z-Score standardization. Outliers refer to data that are significantly deviated from other server KPI data, which may be caused by measurement errors, data entry errors or special circumstances. The specific outlier processing method is to set a threshold and use statistical methods (such as Z scores, T scores, etc.) to identify and correct the data. Missing values refer to the situation where some data points in the server KPI data set have no values. The specific missing value processing method is to use filling (such as using mean, median, mode, etc.), deletion or interpolation (such as linear interpolation, spline interpolation, etc.) and other methods for processing. After completing the data preprocessing, a series of KPI data points arranged in chronological order are obtained, and these data points constitute the KPI time series data. KPI time series data represents the changes in performance indicators of the server at different time points.
[0045] S102: Perform dimensionality reduction processing on the KPI time series data to obtain KPI training data.
[0046] Exemplarily, a suitable dimensionality reduction method is selected according to the time characteristics of the KPI time series data. Common dimensionality reduction methods include principal component analysis, linear discriminant analysis, independent component analysis, etc. For time series data, you can also consider using time series decomposition, autoregressive model, sliding average model, uniform manifold approximation and projection (Uniform Manifold Approximation and Projection, UMAP) and other methods for dimensionality reduction. Perform dimensionality reduction on the KPI time series data to obtain KPI training data. The KPI training data should retain the main information in the KPI time series data while removing redundancy and noise.
[0047] S103. Input the KPI training data into the initial GRU neural network model, process the KPI training data based on the attention mechanism in the initial GRU neural network model, and obtain a server anomaly detection model; wherein the server anomaly detection model is used to process the server KPI data to obtain a server anomaly detection result.
[0048] For example, the Gated Recurrent Unit (GRU) is a variant of a recurrent neural network specifically designed to process sequence data. The GRU neural network model has two gating units, the Reset Gate and the Update Gate, which can capture long-term dependencies in sequence data. The attention mechanism is a technology that simulates human visual attention and can help the model pay more attention to important information when processing sequence data. Introducing the attention mechanism in the GRU neural network model allows the model to pay more attention to key indicators that have an important impact on anomaly detection when processing KPI data.
[0049] Exemplarily, the KPI training data is input into a GRU neural network model combined with an attention mechanism to perform model training to obtain a server anomaly detection model. The server anomaly detection model processes the server KPI data to obtain a server anomaly detection result.
[0050] The embodiment of the present application provides a training method for a server anomaly detection model, which obtains KPI time series data by preprocessing the acquired server KPI data. These data characterize the changes in performance indicators of the server at different time points. Then, the KPI time series data is subjected to dimensionality reduction processing to obtain KPI training data. Then, the KPI training data is input into the initial GRU neural network model with an attention mechanism for training to obtain a server anomaly detection model. In this way, key information can be efficiently extracted from complex and changeable KPI data, and the attention mechanism can be used to enhance the sensitivity of the model to abnormal features, and finally a model that can accurately detect server anomalies is obtained, thereby improving the accuracy and efficiency of server anomaly detection and reducing the false alarm rate and missed alarm rate.
[0051] Figure 2 A flow chart of a method for training a server anomaly detection model provided in an embodiment of the present application Figure 2 ,like Figure 2 As shown, in this embodiment Figure 1 Based on the embodiment, a training method for a server anomaly detection model is described in detail, and the method includes:
[0052] S201, performing standardization processing, outlier processing and missing value processing on the acquired server KPI data to obtain pre-processed KPI data; arranging the pre-processed KPI data in chronological order to obtain KPI time series data.
[0053] Exemplarily, server KPI data within a period of time is obtained through various means (such as log collection, API interface call, etc.). These data may come from different data sources, such as operating system monitoring, application monitoring, network monitoring, etc. Since the server KPI data comes from different monitoring indicators, the dimensions and value ranges of each indicator are different. Therefore, it is necessary to standardize the server KPI data and convert it into a unified dimension or value range. The standardization method can choose min-max standardization or z-score standardization. In the KPI data, there will be outliers caused by equipment differences, data recording errors, etc. These outliers will have a negative impact on subsequent analysis, so they need to be identified and processed. The method of outlier processing includes using box plots or statistical methods (such as Z scores, T scores, etc.) to identify outliers, and then processing them by deleting outliers or replacing them with means, medians, etc. In the KPI data, there will be missing values caused by equipment differences, data transmission interruptions, etc. These missing values will affect the integrity and accuracy of the data, so they need to be filled or interpolated. The methods for handling missing values include mean interpolation, median interpolation, mode interpolation, and interpolation.
[0054] After standardization, outlier processing, and missing value processing, preprocessed KPI data is obtained. These data have removed redundancy and noise, and are in the same order of magnitude or range, which is suitable for subsequent model training and analysis. The preprocessed KPI data is arranged in chronological order to obtain KPI time series data. The construction of KPI time series data helps to capture the dynamic changes and trends of data.
[0055] S202. Determine dimensionality reduction parameters by grid search method; wherein the dimensionality reduction parameters include neighborhood size, minimum distance and number of dimensions; and use UMAP dimensionality reduction method to reduce the dimensionality of KPI time series data according to the dimensionality reduction parameters to obtain KPI training data.
[0056] In an example, according to the time characteristics of the KPI time series data, the time window parameters in the UMAP dimension reduction method are adjusted to obtain the optimized time window parameters; according to the optimized time window parameters, the time weight factor is determined; according to the dimension reduction parameters and the time weight factor, the UMAP dimension reduction method is used to reduce the dimension of the KPI time series data to obtain KPI training data.
[0057] Illustratively, the grid search method is a dimensionality reduction parameter tuning method that finds the optimal parameter configuration by exhaustively enumerating all parameter combinations. UMAP is a dimensionality reduction method that preserves the topological structure of the data by optimizing the similarity in high-dimensional space and low-dimensional space. The neighborhood size representation controls the number of nearest neighbors of each data point in the high-dimensional space. Smaller values focus on local structure, and larger values focus on global structure. The minimum distance representation controls the minimum distance between points in the low-dimensional embedding space, affecting the compactness of the cluster. The target dimension representation specifies the target dimension after dimensionality reduction, and 2 or 3 dimensions are usually selected for easy visualization.
[0058] Exemplarily, create a grid containing all possible dimensionality reduction parameter combinations. For example, define the range of neighborhood size as [5, 10, 15, 20, 30, 50, 70, 100]; define the range of minimum distance as [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]; define the range of target dimension as [2, 3, 5, 10]. For each set of parameter combinations in the grid, use the UMAP dimensionality reduction method to reduce the dimensionality of the KPI time series data. Evaluate the dimensionality reduction results under each parameter combination through visualization and score them. Finally, select the parameter combination with the highest evaluation score as the optimal dimensionality reduction parameter.
[0059] Exemplarily, the time characteristics of KPI time series data are analyzed, such as periodicity, trend, seasonality, etc. According to these characteristics, the time window parameters in the UMAP dimensionality reduction method are adjusted to obtain optimized time window parameters. The time window parameters are used to control the consideration range of the time distance between data points, that is, to determine how long the data points should be considered in the dimensionality reduction process. According to the optimized time window parameters, the time weight factor is defined. The time weight factor is used to assign different weights to data at different time points in the dimensionality reduction process. For example, closer time points are assigned higher weights, and the weight decays exponentially as the time points are farther away.
[0060] After determining the dimension reduction parameters (neighborhood size, minimum distance, and target dimension) and the time weight factor, the UMAP dimension reduction method is configured using the dimension reduction parameters and the time weight factor to perform dimension reduction on the KPI time series data to obtain the KPI training data.
[0061] S203. Input the KPI training data into the initial GRU neural network model, process the KPI training data based on the attention mechanism in the initial GRU neural network model, and obtain a context vector; wherein the context vector represents the key time series features in the KPI training data.
[0062] In one example, KPI training data is input into an initial GRU neural network model, and the KPI training data is processed based on the initial GRU neural network model to obtain a hidden state sequence; wherein the hidden state sequence represents sequence data obtained after processing the KPI training data based on the initial GRU neural network; the hidden state sequence is weighted and summed based on the attention mechanism in the initial GRU neural network model to obtain the attention weight; the attention weight is multiplied and summed with the hidden state sequence to obtain a context vector.
[0063] Exemplarily, the KPI training data is input into the initial GRU neural network model. These training data are usually a series of KPI values arranged in chronological order, which constitute the input sequence of the model. The GRU neural network processes the input KPI training data. During the processing, the GRU generates a hidden state sequence. Each element in this sequence represents the internal state of the model when it processes the corresponding time step. The hidden state sequence is actually an encoding of the input sequence, which contains the information in the input sequence and can be used for subsequent detection tasks. The hidden state sequence is weighted and summed using the attention weights to obtain a context vector. The context vector is actually a weighted average of the hidden state sequence, which pays more attention to those elements with higher scores (i.e., important information). Therefore, the context vector can characterize the key time series features in the KPI training data and provide strong support for subsequent tasks. The specific calculation steps are as follows:
[0064] First, calculate the correlation score between the GRU neural network model output at each time step and the hidden state sequence at the current time step. The calculation method is as follows:
[0065] h t1 =W h h t +b h
[0066] s t1 =W s s t +b s
[0067] e t =tanh(h t1 ×s t1 )
[0068] Among them, h t Represents the output of the GRU neural network model at each time step; s t represents the hidden state sequence of the current time step; W h and W s represents the weight matrix; b h and b s represents the bias term; h t1 h t Linear transformation of s t1 For t Linear transformation of t Represents the relevance score.
[0069] Then, the relevance score is normalized through the softmax function to obtain the attention weight. The calculation method is as follows:
[0070] α t =softmax(e t )
[0071]
[0072] Among them, e t represents the correlation score; α t represents the attention weight; T is the total number of time steps.
[0073] Finally, the context vector is obtained by multiplying the attention weight and the hidden state sequence of the current time step and summing them up. The calculation method is as follows:
[0074]
[0075] Among them, α i represents the attention weight; h irepresents the hidden state sequence of the current time step; c t represents the context vector.
[0076] S204, the KPI training data has label information indicating the server anomaly detection result. According to the output layer of the initial GRU neural network, the context vector is processed to obtain a preliminary anomaly detection result; according to the preliminary anomaly detection result and the label information, the parameters of the initial GRU neural network are adjusted to obtain an optimized GRU neural network; the KPI training data is input into the optimized GRU neural network for training to obtain a server anomaly detection model.
[0077] Exemplarily, first, the KPI training data is input into the initial GRU neural network model; the GRU neural network processes the input KPI training data to generate a hidden state sequence; the attention mechanism is used to perform weighted summation on the hidden state sequence to generate a context vector; the context vector is input into the output layer of the GRU neural network to obtain a preliminary anomaly detection result. The KPI training data has label information indicating the server anomaly detection result, and the label information indicates whether the server is abnormal. By comparing the preliminary anomaly detection results with these label information, the performance of the model (such as accuracy, recall, etc.) can be calculated. Based on the performance evaluation of the preliminary anomaly detection results, the back propagation algorithm is used to adjust the parameters of the GRU neural network. This process usually involves calculating the gradient of the loss function and using these gradients to update the weights of the network. After multiple iterations of training, an optimized GRU neural network can be obtained, whose parameters have been adjusted according to the training data to better identify anomalies. The KPI training data is input into the optimized GRU neural network for training to obtain a server anomaly detection model.
[0078] A training method for a server anomaly detection model provided in an embodiment of the present application ensures data quality by preprocessing server KPI data, and then arranges the data in chronological order to obtain KPI time series data; then, accurately determines the dimension reduction parameters through a grid search method, and optimizes the time window parameters and time weight factors in the UMAP dimension reduction method according to the time characteristics of the KPI time series data, thereby achieving effective dimension reduction of the KPI time series data and obtaining high-quality KPI training data; then, the KPI training data is input into the initial GRU neural network model, and the hidden state sequence is weighted by the attention mechanism to obtain a context vector, and anomaly detection is performed based on the context vector, and finally, the GRU neural network is optimized by comparing the preliminary anomaly detection results with the label information, so as to improve the accuracy and robustness of server anomaly detection and reduce false positives and negatives caused by data quality problems. At the same time, by optimizing the dimension reduction method and the attention mechanism, the sensitivity and recognition ability of the model to server abnormal characteristics are further improved.
[0079] Figure 3 A flow chart of a server anomaly detection method provided in an embodiment of the present application is as follows: Figure 3 As shown, the method includes:
[0080] S301. Obtain server KPI data.
[0081] Exemplarily, server KPI data within a certain time sliding window is obtained.
[0082] S302: Input the server KPI data into the server anomaly detection model to obtain the server anomaly detection result output by the server anomaly detection model.
[0083] Exemplarily, the server KPI data is input into the server anomaly detection model to obtain the server anomaly detection result output by the server anomaly detection model; wherein the server anomaly detection model is the server anomaly detection model trained in the above embodiment.
[0084] The server anomaly detection method provided in the embodiment of the present application obtains server KPI data, inputs the server KPI data into a server anomaly detection model, and obtains a server anomaly detection result. The method uses the server anomaly detection model trained in the embodiment of the present invention, which can accurately and quickly detect the abnormal situation of the server.
[0085] Figure 4 A structural diagram of a server anomaly detection model training device provided in an embodiment of the present application Figure 1 ,like Figure 4 As shown, a server anomaly detection model training device 40 provided in this embodiment includes:
[0086] The data processing module 401 is used to pre-process the acquired server KPI data to obtain KPI time series data; wherein the KPI time series data represents the change of the performance indicators of the server at different time points.
[0087] The feature dimension reduction module 402 is used to perform dimension reduction processing on the KPI time series data to obtain KPI training data.
[0088] The model training module 403 is used to input the KPI training data into the initial GRU neural network model to process the KPI training data based on the attention mechanism in the initial GRU neural network model to obtain a server anomaly detection model; wherein the server anomaly detection model is used to process the server KPI data to obtain a server anomaly detection result.
[0089] The present embodiment provides a training device for a server anomaly detection model, which can execute the method provided in the above method embodiment. Its implementation principle and technical effects are similar, and are not described in detail in this embodiment.
[0090] Figure 5 A structural diagram of a server anomaly detection model training device provided in an embodiment of the present application Figure 2 ,like Figure 5 As shown, a server anomaly detection model training device 50 provided in this embodiment includes:
[0091] The data processing module 501 is used to pre-process the acquired server KPI data to obtain KPI time series data; wherein the KPI time series data represents the change of the performance indicators of the server at different time points.
[0092] The feature dimension reduction module 502 is used to perform dimension reduction processing on the KPI time series data to obtain KPI training data.
[0093] The model training module 503 is used to input the KPI training data into the initial GRU neural network model to process the KPI training data based on the attention mechanism in the initial GRU neural network model to obtain a server anomaly detection model; wherein the server anomaly detection model is used to process the server KPI data to obtain a server anomaly detection result.
[0094] In a possible implementation, the feature dimension reduction module 502 includes:
[0095] The determination module 5021 is used to determine the dimension reduction parameters by using a grid search method; wherein the dimension reduction parameters include the neighborhood size, the minimum distance and the number of dimensions.
[0096] The dimension reduction module 5022 is used to reduce the dimension of the KPI time series data using the UMAP dimension reduction method according to the dimension reduction parameters to obtain KPI training data.
[0097] In a possible implementation, the dimension reduction module 5022 includes:
[0098] According to the time characteristics of the KPI time series data, the time window parameters in the UMAP dimensionality reduction method are adjusted to obtain the optimized time window parameters.
[0099] Determine the time weight factor based on the optimized time window parameters.
[0100] According to the dimension reduction parameters and time weight factors, the UMAP dimension reduction method is used to reduce the dimension of the KPI time series data to obtain the KPI training data.
[0101] In a possible implementation, the model training module 503 includes:
[0102] The first processing module 5031 is used to input the KPI training data into the initial GRU neural network model to process the KPI training data based on the attention mechanism in the initial GRU neural network model to obtain a context vector; wherein the context vector represents the key time series features in the KPI training data.
[0103] The second processing module 5032 is used to process the context vector based on the initial GRU neural network to obtain a server anomaly detection model.
[0104] In a possible implementation, the first processing module 5031 includes:
[0105] The KPI training data is input into the initial GRU neural network model, so as to process the KPI training data based on the initial GRU neural network model to obtain a hidden state sequence; wherein the hidden state sequence represents the sequence data obtained after processing the KPI training data based on the initial GRU neural network.
[0106] Based on the attention mechanism in the initial GRU neural network model, the hidden state sequence is weighted and summed to obtain the attention weight.
[0107] The attention weights are multiplied by the hidden state sequence and summed to obtain the context vector.
[0108] In a possible implementation, the second processing module 5032 includes:
[0109] According to the output layer of the initial GRU neural network, the context vector is processed to obtain preliminary anomaly detection results.
[0110] According to the preliminary anomaly detection results and label information, the parameters of the initial GRU neural network are adjusted to obtain the optimized GRU neural network.
[0111] The KPI training data is input into the optimized GRU neural network for training to obtain the server anomaly detection model.
[0112] The present embodiment provides a training device for a server anomaly detection model, which can execute the method provided in the above method embodiment. Its implementation principle and technical effects are similar, and are not described in detail in this embodiment.
[0113] Figure 6 A schematic diagram of the structure of a server anomaly detection device provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, a server anomaly detection model training device 60 provided in this embodiment includes:
[0114] The acquisition module 601 is used to acquire server KPI data.
[0115] The detection module 602 is used to input the server KPI data into the server anomaly detection model to obtain the server anomaly detection result output by the server anomaly detection model.
[0116] The server anomaly detection device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be described in detail here.
[0117] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 70 provided in this embodiment includes: at least one processor 701 and a memory 702. Optionally, the device 70 further includes a communication component 703. The processor 701, the memory 702 and the communication component 703 are connected via a bus 704.
[0118] In a specific implementation process, at least one processor 701 executes the computer-executable instructions stored in the memory 702, so that at least one processor 701 executes the above method.
[0119] The specific implementation process of the processor 701 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.
[0120] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.
[0121] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.
[0122] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.
[0123] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.
[0124] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.
[0125] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.
[0126] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.
[0127] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.
[0128] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0129] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0130] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0131] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0132] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.
Claims
1. A training method for a server anomaly detection model, characterized in that: include: Preprocess the acquired server KPI data to obtain KPI time series data; wherein the KPI time series data represents the change of performance indicators of the server at different time points; Performing dimensionality reduction processing on the KPI time series data to obtain KPI training data; The KPI training data is input into an initial GRU neural network model to process the KPI training data based on the attention mechanism in the initial GRU neural network model to obtain a server anomaly detection model; wherein the server anomaly detection model is used to process the server KPI data to obtain a server anomaly detection result.
2. The method according to claim 1, characterized in that According to the KPI time series data, dimensionality reduction processing is performed to obtain KPI training data, including: Determine dimensionality reduction parameters by grid search method; wherein the dimensionality reduction parameters include neighborhood size, minimum distance and number of dimensions; According to the dimension reduction parameters, the UMAP dimension reduction method is used to reduce the dimension of the KPI time series data to obtain the KPI training data.
3. The method according to claim 2, characterized in that According to the dimension reduction parameters, the UMAP dimension reduction method is used to reduce the dimension of the KPI time series data to obtain the KPI training data, including: According to the time characteristics of the KPI time series data, the time window parameters in the UMAP dimensionality reduction method are adjusted to obtain optimized time window parameters; Determining a time weight factor according to the optimized time window parameter; According to the dimension reduction parameter and the time weight factor, the UMAP dimension reduction method is used to reduce the dimension of the KPI time series data to obtain the KPI training data.
4. The method according to claim 1, characterized in that The KPI training data is input into the initial GRU neural network model to process the KPI training data based on the attention mechanism in the initial GRU neural network model to obtain a server anomaly detection model, including: Inputting the KPI training data into an initial GRU neural network model to process the KPI training data based on an attention mechanism in the initial GRU neural network model to obtain a context vector; wherein the context vector represents key time series features in the KPI training data; The context vector is processed based on the initial GRU neural network to obtain a server anomaly detection model.
5. The method according to claim 4, characterized in that The KPI training data is input into the initial GRU neural network model to process the KPI training data based on the attention mechanism in the initial GRU neural network model to obtain a context vector, including: Inputting the KPI training data into an initial GRU neural network model to process the KPI training data based on the initial GRU neural network model to obtain a hidden state sequence; wherein the hidden state sequence represents sequence data obtained after processing the KPI training data based on the initial GRU neural network; Performing weighted summation on the hidden state sequence based on the attention mechanism in the initial GRU neural network model to obtain an attention weight; The attention weight is multiplied by the hidden state sequence and the sum is taken to obtain the context vector.
6. The method according to claim 4, characterized in that The KPI training data has label information indicating a server anomaly detection result; the context vector is processed based on the initial GRU neural network to obtain a server anomaly detection model, including: Processing the context vector according to the output layer of the initial GRU neural network to obtain a preliminary anomaly detection result; According to the preliminary anomaly detection result and the label information, adjusting the parameters of the initial GRU neural network to obtain an optimized GRU neural network; The KPI training data is input into the optimized GRU neural network for training to obtain a server anomaly detection model.
7. The method according to any one of claims 1 to 6, characterized in that Preprocess the acquired server KPI data to obtain KPI time series data, including: Performing standardization, outlier processing and missing value processing on the acquired server KPI data to obtain pre-processed KPI data; The pre-processed KPI data are arranged in chronological order to obtain the KPI time series data.
8. A server anomaly detection method, characterized in that: include: Get server KPI data; The server KPI data is input into a server anomaly detection model to obtain a server anomaly detection result output by the server anomaly detection model, wherein the server anomaly detection model is trained by the training method described in any one of claims 1 to 7.
9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the method according to any one of claims 1 to 7, and / or the processor executes the method according to claim 8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method according to any one of claims 1 to 7, and / or enable the processor to execute the method according to claim 8.