Database fault prediction and autonomous recovery system and method and related equipment

Abstract

The invention belongs to the technical field of computer network monitoring and database management, and discloses a database fault prediction and autonomous recovery system and method and related equipment, and the method comprises the steps: collecting multi-dimensional performance indexes on a database server in real time, extracting time features of different scales through discrete wavelet transform, and constructing feature vectors; and dynamically calculating a fault prediction threshold by using a sliding window technology, and predicting a fault risk probability by using a multi-scale attention-enhanced LSTM model. And according to a prediction result, the system automatically executes an autonomous recovery strategy, high-precision prediction and autonomous recovery of database faults are realized, and the stability and service continuity of the system are improved.

Description

Technical Field

[0001] The present invention belongs to the technical fields of computer network monitoring and database management, and specifically relates to a database fault prediction and autonomous recovery system, method, and related devices. Background Art

[0002] In today's highly integrated information technology application scenarios, the database system, as the core infrastructure supporting massive data processing and business logic operation, the stability of its operating state is directly related to enterprise service continuity, user experience, and business benefits. With the popularization of cloud computing and distributed architectures, the complexity of database systems has increased exponentially, and the performance metric data generated during operation has characteristics such as high dimensionality, strong temporal correlation, and multi-scale changes, which pose higher requirements for real-time monitoring and fault prediction technologies.

[0003] However, most of the database monitoring methods in the existing technology system adopt a threshold warning mechanism based on static rules. For example, warnings are triggered by presetting fixed thresholds for CPU utilization or memory occupancy. Although such methods can identify obvious abnormal fluctuations, they have significant limitations: First, traditional methods rely on manual experience to set thresholds and are difficult to adapt to dynamically changing load scenarios, and are prone to false alarms during sudden traffic or periodic business peaks; Second, the degradation of database performance often manifests as a progressive anomaly of multi-index coupling. The existing technology's feature extraction of time series data stays at a single time granularity and lacks the ability to jointly analyze long-term trends and short-term mutations; In addition, the fixed threshold mechanism cannot capture the correlation characteristics between indicators, such as the non-linear relationship between disk I / O rate anomalies and query response time delays, resulting in a high miss detection rate for compound faults.

[0004] More prominently, under the existing technology framework, fault recovery heavily relies on manual decision-making, and there is a minute-level delay from anomaly detection to operations such as restarting and switching backups, which cannot meet the stringent requirements for high availability in scenarios such as financial transactions and industrial control. This series of technical defects together lead to three core problems in database operation and maintenance: insufficient fault prediction accuracy, lack of adaptability, and low recovery efficiency, becoming the key bottleneck restricting the reliability of modern information systems. Summary of the Invention

[0005] The present invention provides a database fault prediction and autonomous recovery system, method, and related devices, which solve the problem that traditional fault prediction methods are difficult to accurately capture the complex characteristics of the database operating state, resulting in high false alarm rates and miss detection rates, and affecting the reliability of the system.

[0006] To achieve the above object, the present invention provides the following technical solutions: A database fault prediction and autonomous recovery system, comprising: A data acquisition module, which is used to collect multi-dimensional performance metrics of a database server in real time and generate a multi-dimensional time series data set; A dynamic feature extraction module, which performs multi-scale decomposition on the multi-dimensional time series data through discrete wavelet transform, calculates the statistical features of each scale coefficient, and constructs a feature vector; An adaptive threshold adjustment module, which dynamically calculates the mean and standard deviation of multi-dimensional performance metrics using a sliding window, and generates a dynamic prediction threshold based on the mean and standard deviation; A fault prediction module, which uses a multi-scale attention enhanced LSTM model to train and predict the feature vector, and outputs a fault risk probability; An autonomous recovery module, which executes a hierarchical autonomous recovery strategy according to the comparison result between the fault risk probability and the dynamic prediction threshold.

[0007] Preferably, the dynamic feature extraction module uses Daubechies wavelet basis function to decompose the multi-dimensional time series into three layers, and calculates energy, entropy, skewness and kurtosis statistics in each decomposed scale coefficient.

[0008] Preferably, the multi-scale attention enhanced LSTM model includes: An input layer, which receives the feature vector; An LSTM layer, which is used to extract the temporal features of the time series and output the hidden state and cell state; An attention calculation unit, which calculates the attention weights of each scale feature; A feature fusion layer, which weighted-sums the feature vectors of each scale according to the attention weights to generate a context vector.

[0009] Preferably, the formula for calculating the attention weights of each scale feature is:

[0010] is the hidden state at time step , is the cell state at time step , is the scale of the feature vector, , , are learnable parameters.

[0011] Preferably, the autonomous recovery module includes a three-level response strategy: Primary response: Generate system logs and alarm notifications; Intermediate response: Execute memory release and cache cleaning operations; Advanced response: Trigger service restart or primary / standby switch operation.

[0012] Preferably, generating a dynamic prediction threshold based on the mean and standard deviation is specifically as follows:

[0013] Wherein, is the dynamic prediction threshold, is the mean, is the standard value, is the risk coefficient set according to the system risk preference.

[0014] A database fault prediction and autonomous recovery method, comprising: Collecting multi-dimensional performance metrics of a database server to generate time series data; Performing multi-scale wavelet decomposition on the time series data, calculating statistical features of each scale coefficient, and constructing a feature vector; Using a sliding window to dynamically calculate the mean and standard deviation of multi-dimensional performance metrics, and generating a dynamic prediction threshold based on the mean and standard deviation; Using a multi-scale attention-enhanced LSTM model to train and predict the feature vector, and outputting a fault risk probability; Executing a hierarchical autonomous recovery strategy according to the comparison result between the fault risk probability and the dynamic prediction threshold.

[0015] Preferably, the training of the multi-scale attention-enhanced LSTM model is as follows: Constructing a historical data set including normal and fault states; Using a binary cross-entropy loss function and an Adam optimizer for model training; Applying an early stopping mechanism and a regularization method to prevent model overfitting.

[0016] A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of a database fault prediction and autonomous recovery method are implemented.

[0017] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a database fault prediction and autonomous recovery method are implemented.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a database fault prediction and autonomous recovery system. By collecting multi-dimensional performance metrics in real time on the database server, discrete wavelet transform is used to extract time features at different scales, and feature vectors are constructed. The sliding window technique is adopted to dynamically calculate the fault prediction threshold, and the LSTM model enhanced by multi-scale attention is used to predict the probability of fault risk. According to the prediction results, the system automatically executes the autonomous recovery strategy, realizing high-precision prediction and autonomous recovery of database faults, and improving the stability and business continuity of the system.

[0019] Furthermore, through the multi-scale attention mechanism, the model can effectively capture the complex time series features of database performance metrics, significantly improve the accuracy of fault prediction, and reduce the false alarm and missed alarm rates.

[0020] Furthermore, the adaptive threshold adjustment algorithm enables the system to dynamically respond to load changes, automatically adjust the fault prediction threshold, and adapt to different operating environments and load conditions.

[0021] Furthermore, the system can automatically execute the predefined recovery strategy according to the fault prediction results, reduce manual intervention, improve the operation and maintenance efficiency, and ensure the business continuity. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a block diagram of a database fault prediction and autonomous recovery system of the present invention; Figure 2 It is a schematic diagram of the structure of the LSTM model enhanced by multi-scale attention of the present invention; Figure 3 It is a schematic diagram of the adaptive threshold adjustment process of the present invention; Figure 4 It is a flowchart of a database fault prediction and autonomous recovery of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and illustrated herein can be arranged and designed in various different configurations.

[0024] Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0025] It should be noted that similar reference numerals and letters indicate similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0026] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0027] As Figure 1 shown, the present invention provides a database fault prediction and autonomous recovery system, including: A data acquisition module, configured to collect multi-dimensional performance metrics of a database server in real time and generate a multi-dimensional time series data set; Deployed on the database server, it is used to collect multi-dimensional performance metrics of the database operation in real time to form a multi-dimensional time series data set. The collected metrics include CPU usage rate, memory occupancy rate, disk I / O rate, network bandwidth utilization rate, number of database connections, query response time, etc.

[0028] A dynamic feature extraction module, which performs multi-scale decomposition on the multi-dimensional time series data through discrete wavelet transform, calculates the statistical features of each scale coefficient, and constructs a feature vector; Perform discrete wavelet transform on the collected multi-dimensional time series data to extract time features of different scales. Calculate statistical features such as energy, entropy, skewness, and kurtosis at each scale, and construct a feature vector for fault prediction.

[0029] An adaptive threshold adjustment module, which dynamically calculates the mean and standard deviation of multi-dimensional performance metrics using a sliding window, and generates a dynamic prediction threshold based on the mean and standard deviation; Calculate the fault prediction threshold according to the formula T = μ + βσ, where μ is the mean, σ is the standard deviation, and β is the risk coefficient set according to the system risk preference.

[0030] A fault prediction module, which uses an LSTM model enhanced with multi-scale attention to train and predict the feature vector, and outputs the fault risk probability; The model includes an input layer that receives the feature vector; An LSTM layer, which is used to extract the temporal features of the time series and output the hidden state and cell state; An attention calculation unit, which calculates the attention weights of each scale feature; A feature fusion layer, which weights and sums the feature vectors of each scale according to the attention weights to generate a context vector.

[0031] The autonomous recovery module executes a hierarchical autonomous recovery strategy according to the comparison result between the failure risk probability and the dynamic prediction threshold.

[0032] The autonomous recovery module includes a three-level response strategy: Primary response: Generate system logs and alarm notifications; Intermediate response: Execute memory release and cache cleaning operations; Advanced response: Trigger service restart or primary / standby switch operation.

[0033] Another embodiment of the present invention provides a database failure prediction and autonomous recovery system, including: Data acquisition module The deployment method is: Deploy a lightweight monitoring agent program on the database server to collect the performance metrics of the database and the system in real time.

[0034] The collected metrics are: System level: CPU usage rate, memory occupancy rate, disk I / O rate, network bandwidth utilization rate, etc.

[0035] Database level: Number of database connections, query response time, transaction processing volume, lock waiting time, etc.

[0036] Data transmission: Through secure network communication, transmit the collected data to the data processing center in real time to form a multi-dimensional time series dataset.

[0037] Dynamic feature extraction module Discrete wavelet transform (DWT): Mother wavelet selection: Select the Daubechies wavelet (such as db4) for multi-scale decomposition.

[0038] Decomposition level: According to the data characteristics, select an appropriate decomposition level (such as 3 levels) to obtain the approximation coefficients and detail coefficients at different scales.

[0039] Feature calculation includes: Statistical features: Calculate statistical features such as energy, entropy, skewness, and kurtosis in the coefficients at each scale.

[0040] Feature vector construction: Combine the statistical features at each scale to form a feature vector for failure prediction .

[0041] Adaptive threshold adjustment module The sliding window technique includes: Window size: Set the sliding window size to (such as the data of the last 60 time points).

[0042] Parameter calculation: Calculate the mean value of the performance metrics within the window and the standard deviation .

[0043] Threshold calculation includes: Risk coefficient : Set according to the risk tolerance of the system (such as ).

[0044] Threshold formula: , dynamically adjust the fault prediction threshold.

[0045] The adaptive threshold adjustment process is as Figure 3 shown.

[0046] Fault prediction module As Figure 2 shown, the structure of the LSTM model with multi-scale attention enhancement is:[[]] Input layer: Receive the feature vector .

[0047] LSTM layer: Adopt a single-layer or multi-layer LSTM network, and the number of hidden units is (such as 128). The LSTM layer outputs the hidden state and the cell state .

[0048] The multi-scale attention mechanism is:[[]] Attention weight calculation:[[]]

[0049] Among them, is the hidden state at time step , is the cell state at time step , ] / >is the feature vector at scale , , , are learnable parameters.

[0050] Context vector calculation:[[]] [[ID=.]].

[0051] Output layer: Map the context vector to the fault risk probability .

[0052] Model training includes:[[]] Dataset: Use the labeled historical data, including samples in normal and fault states.

[0053] Loss function: Adopt the binary cross-entropy loss function.

[0054] Optimization algorithm: Use the Adam optimizer with an initial learning rate of 0.001.

[0055] Training strategy: Adopt early stopping strategy and regularization techniques to prevent overfitting.

[0056] Autonomous recovery module Risk judgment includes: If , it is considered that there is a risk of failure.

[0057] According to the risk level, it is divided into levels such as early warning, warning, and emergency failure.

[0058] The execution of the recovery strategy includes: Early warning level: Record logs and send notifications to operation and maintenance personnel.

[0059] Warning level: Execute resource cleaning operations, release memory, and clean caches.

[0060] Emergency failure level: Automatically restart relevant services or switch to a backup server.

[0061] The specific implementation of the multi-scale attention enhanced LSTM model is as follows: Model design includes: Input layer: Input feature vector , including statistical features of different scales.

[0062] The LSTM layer is designed as: Structural parameters: The number of hidden units .

[0063] Activation function: Adopt Sigmoid and Tanh functions.

[0064] Output: Hidden state and cell state .

[0065] The multi-scale attention mechanism includes: Learnable parameters: Initialize the attention vector , weight matrix and bias vector .

[0066] Output layer: Activation function: Adopt the Sigmoid function to map the output to interval, representing the probability of failure risk .

[0067] Model training is as follows: Data preparation: Sample quantity: Collect one-year historical data, including approximately 50,000 normal and faulty samples each.

[0068] Data division: Training set (70%), validation set (15%), test set (15%).

[0069] Training parameters: Batch size: 64.

[0070] Number of training epochs: Train until the validation loss no longer decreases according to the performance of the validation set.

[0071] Evaluation metrics: Accuracy, precision, recall, F1 score.

[0072] AUC value: Used to evaluate the classification performance of the model.

[0073] Parameter selection and hyperparameter tuning Number of hidden units : Select through cross-validation to obtain the best performance.

[0074] Risk coefficient : According to the experimental results, when it is [specific value], both the false alarm rate and the miss rate are relatively low.

[0075] Learning rate adjustment: Adopt the learning rate decay strategy to dynamically reduce the learning rate during training.

[0076] As Figure 4 shown, the present invention also provides a database fault prediction and autonomous recovery method, including: Collect multi-dimensional performance metrics of the database server to generate time series data; Perform multi-scale wavelet decomposition on the time series data, calculate the statistical features of each scale coefficient and construct a feature vector; Adopt a sliding window to dynamically calculate the mean and standard deviation of multi-dimensional performance metrics, and generate a dynamic prediction threshold based on the mean and standard deviation; Adopt an LSTM model enhanced with multi-scale attention to train and predict the feature vector, and output the fault risk probability; Execute a hierarchical autonomous recovery strategy according to the comparison result between the fault risk probability and the dynamic prediction threshold.

[0077] The training of the LSTM model enhanced with multi-scale attention is as follows: Construct a historical data set including normal and faulty states; Adopt a binary cross-entropy loss function and an Adam optimizer to train the model; Early stopping mechanism and regularization methods are applied to prevent model overfitting.

[0078] The dynamic feature extraction module uses Daubechies wavelet basis function to perform three-level decomposition on the multidimensional time series and calculates the energy, entropy, skewness and kurtosis statistics in each scale coefficient after decomposition.

[0079] The multi-scale attention-enhanced LSTM model includes: Input layer, receiving feature vectors; LSTM layer, used to extract the temporal features of the time series and output the hidden state and cell state; Attention calculation unit, calculates the attention weight of each scale feature; The feature fusion layer generates a context vector by weighting the feature vectors of each scale according to the attention weight.

[0080] The formula for calculating the attention weight of each scale feature is:

[0081] is the time step The hidden state of is the time step The cell state, For scale The eigenvector of 、 、 are learnable parameters.

[0082] The autonomous recovery module includes three levels of response strategies: Primary response: Generate system logs and alarm notifications; Intermediate response: perform memory release and cache cleanup operations; Advanced response: Trigger service restart or active / standby switchover.

[0083] The dynamic prediction thresholds generated based on the mean and standard deviation are as follows:

[0084] in, is the dynamic prediction threshold, is the mean, is the standard value, It is the risk factor set according to the system risk preference.

[0085] The terminal device provided by an embodiment of the present invention. The terminal device of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0086] The computer program can be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention.

[0087] The terminal device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory.

[0088] The processor can be a Central Processing Unit (CPU), or can also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0089] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the terminal device by running or executing the computer program and / or modules stored in the memory, and by calling the data stored in the memory.

[0090] If the modules / units integrated in the terminal device are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of the present invention, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0091] Although the embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the above specific embodiments and application fields. The above specific embodiments are merely illustrative and guiding, rather than restrictive. Under the inspiration of the specification, those of ordinary skill in the art can also make many forms without departing from the scope protected by the claims of the present invention, and these all fall within the scope of protection of the present invention.

Claims

1. A database fault prediction and autonomous recovery system, characterized in that, It includes: A data acquisition module, which is used to collect multi-dimensional performance metrics of the database server in real time and generate a multi-dimensional time series dataset; A dynamic feature extraction module, which performs multi-scale decomposition on the multi-dimensional time series data through discrete wavelet transform, calculates the statistical features of each scale coefficient, and constructs a feature vector; An adaptive threshold adjustment module, which dynamically calculates the mean and standard deviation of multi-dimensional performance metrics using a sliding window, and generates a dynamic prediction threshold based on the mean and standard deviation; A fault prediction module, which uses an LSTM model enhanced with multi-scale attention to train and predict the feature vector, and outputs the fault risk probability; An autonomous recovery module, which executes a hierarchical autonomous recovery strategy according to the comparison result between the fault risk probability and the dynamic prediction threshold.

2. The database fault prediction and autonomous recovery system according to claim 1, characterized in that, The dynamic feature extraction module uses Daubechies wavelet basis function to decompose the multi-dimensional time series into three layers, and calculates energy, entropy, skewness, and kurtosis statistics in the decomposed scale coefficients of each layer.

3. The database failure prediction and autonomous recovery system according to claim 1, characterized in that The LSTM model enhanced with multi-scale attention includes: An input layer, which receives the feature vector; An LSTM layer, which is used to extract the temporal features of the time series and output the hidden state and cell state; An attention calculation unit, which calculates the attention weights of each scale feature; A feature fusion layer, which generates a context vector by weighted summation of each scale feature vector according to the attention weights.

4. A database fault prediction and autonomous recovery system according to claim 3, wherein The formula for calculating the attention weights of each scale feature is: is the hidden state at time step , is the cell state at time step , is the feature vector at scale , , , are learnable parameters.

5. A database fault prediction and autonomous recovery system according to claim 1, characterized in that The autonomous recovery module includes a three-level response strategy: Primary response: Generate system logs and alarm notifications; Intermediate response: Execute memory release and cache cleaning operations; Advanced response: Trigger service restart or primary / standby switch operation.

6. The database fault prediction and autonomous recovery system according to claim 1, characterized in that, Generating a dynamic prediction threshold based on the mean and standard deviation specifically includes: Among them, is the dynamic prediction threshold, is the mean value, is the standard value, is the risk coefficient set according to the system risk preference.

7. A method for database fault prediction and autonomous recovery, characterized in that It includes: Collect multi-dimensional performance metrics of the database server to generate time series data; Perform multi-scale wavelet decomposition on the time series data, calculate the statistical features of each scale coefficient, and construct a feature vector; Dynamically calculate the mean and standard deviation of multi-dimensional performance metrics using a sliding window, and generate a dynamic prediction threshold based on the mean and standard deviation; Use an LSTM model enhanced with multi-scale attention to train and predict the feature vector, and output the fault risk probability; Execute a hierarchical autonomous recovery strategy according to the comparison result between the fault risk probability and the dynamic prediction threshold.

8. A database fault prediction and autonomous recovery method according to claim 7, characterized in that, The training of the LSTM model enhanced with multi-scale attention is as follows: Construct a historical dataset including normal and fault states; Use the binary cross-entropy loss function and Adam optimizer for model training; Apply an early stopping mechanism and regularization method to prevent model overfitting.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a database fault prediction and autonomous recovery method as described in any one of claims 7 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of a database fault prediction and autonomous recovery method as described in any one of claims 7 to 8.

Images