Fault detection method for persistent memory server

Abstract

The invention discloses a fault detection method for a persistent memory server, and relates to the technical field of fault detection and persistent memories, and the method comprises the following steps: collecting various historical index data from a persistent memory during the operation of a system, carrying out the preprocessing, and constructing an initial sample set; expanding the initial sample set by using a generative adversarial network to obtain an expanded sample set; a CNN-LSTM model is constructed, the expanded sample set is used for model training until network convergence, and a trained persistent memory fault prediction model is obtained; and monitoring the running state of the system in real time by using the trained persistent memory fault prediction model, predicting potential faults of the persistent memory, and determining whether to send out fault early warning or not by setting a threshold value. According to the method, the fault detection accuracy can be improved, memory faults can be found in time, measures are taken for repairing, and stable operation of the system is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of fault detection and persistent memory, and more specifically to a fault detection method for a persistent memory server. Background Art

[0002] With the continuous development of smart hardware technology, persistent memory has emerged. It is a new hardware technology that combines memory and storage functions. It has the following advantages: it has high-speed data reading and writing capabilities, which can meet the real-time and low-latency requirements of smart hardware; the data persistence of persistent memory can ensure that the status and data of the device can be restored even after power failure or restart, improving the reliability and stability of the system; the capacity of persistent memory is large, which can meet the needs of smart hardware for large-scale data storage.

[0003] The existing computing environment faces many challenges. Memory failure will affect the computer's operating performance. Once an uncorrectable failure occurs, the computer equipment will not work. If the operation and maintenance personnel only replace or repair the memory when an uncorrectable failure occurs, the operation and maintenance work will have problems such as passive fault handling, untimely handling and low processing efficiency.

[0004] The current detection methods for persistent memory failures do not consider the long-term dependencies in time series data, and are still insufficient in terms of the accuracy and timeliness of failure detection. Therefore, how to timely and effectively detect persistent memory failures and take measures to repair them to ensure the stable operation of the system is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a fault detection method for a persistent memory server, which solves the problems existing in the background technology.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A fault detection method for a persistent memory server includes the following steps:

[0008] Collect various historical indicator data during system operation from persistent memory and perform preprocessing to build an initial sample set;

[0009] The initial sample set is expanded using a generative adversarial network to obtain an expanded sample set;

[0010] Build a CNN-LSTM model and use the expanded sample set to train the model until the network converges to obtain a trained persistent memory fault prediction model;

[0011] Use the trained persistent memory fault prediction model to monitor the system's operating status in real time, predict potential persistent memory failures, and decide whether to issue a fault warning by setting thresholds.

[0012] Optional historical indicator data includes: CPU usage, memory usage, disk I / O, network traffic, and system logs.

[0013] Optionally, the preprocessing operation specifically includes the following steps:

[0014] Clean different types of historical indicator data according to predefined data cleaning rules;

[0015] The Z-score standardization method was used to normalize the cleaned data, and the initial sample set was constructed based on the standardized data.

[0016] Optionally, the initial sample set is expanded using a generative adversarial network, specifically including the following steps:

[0017] Use random noise to train the generative adversarial network, update the parameters of the discriminator and generator through the discriminator loss function and the generator loss function until the loss function converges, and use the parameters of the discriminator and generator at the end of the training as the parameters of the generative adversarial network to generate a sample expansion model;

[0018] The initial sample set is input into the sample expansion model to generate an expanded sample set that is close to the actual historical indicator data.

[0019] Optionally, the CNN-LSTM model includes: an input layer, a CNN network, an LSTM network, a fusion network, a fully connected layer, and an output layer;

[0020] Input layer, used to input the expanded sample set;

[0021] The CNN network includes a convolutional layer, a maximum pooling layer, and a dense layer connected in sequence, which are used to extract spatial features from the expanded sample set;

[0022] The LSTM network includes a forget gate, an input gate, and an output gate, which are used to extract temporal features from the expanded sample set;

[0023] The fusion network is used to connect the extracted spatial features and temporal features in series to obtain fusion features;

[0024] Fully connected layer, used to identify persistent memory failures based on fused features;

[0025] The output layer is used to output the fault prediction results.

[0026] Optionally, model training specifically includes the following steps:

[0027] Based on the preset ratio, the expanded sample set is divided into a training set and a test set;

[0028] Randomly initialize the weights of the CNN-LSTM model and train the CNN-LSTM model based on the training set;

[0029] The back propagation algorithm is used to obtain the changing trend of the prediction accuracy of the test set, the loss is calculated by combining the cross entropy loss function, and the Adam optimizer is selected to optimize the CNN-LSTM model;

[0030] The training is terminated until the loss function converges, and a trained persistent memory fault prediction model is obtained.

[0031] Optionally, after predicting potential failure of persistent memory, the following is also included:

[0032] When the potential fault of the persistent memory is an uncorrectable error, determining to repair the fault based on the business load and / or the business load growth;

[0033] When the potential fault of the persistent memory is a correctable error, the memory page corresponding to the fault is isolated through a preset interface and divided into a fault isolation area;

[0034] When there is no potential failure in the persistent memory, continue to monitor the operating status of the system.

[0035] Through the above technical solutions, it can be seen that compared with the prior art, the present invention discloses a fault detection method for persistent memory servers, which can capture long-term dependencies in time series data and improve the accuracy of fault detection; it can monitor the operating status of the system in real time and discover potential faults in time; by continuously learning new data, the model can adapt to changes in the system environment. The present invention can be applied to scenarios such as data centers, cloud computing platforms, and large servers to help operation and maintenance personnel discover and handle system faults in time and ensure stable system operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0037] Figure 1 A flowchart of a fault detection method for a persistent memory server provided by the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The embodiment of the present invention discloses a fault detection method for a persistent memory server. Figure 1 As shown, the following steps are included:

[0040] Collect various historical indicator data during system operation from persistent memory and perform preprocessing to build an initial sample set;

[0041] The initial sample set is expanded using a generative adversarial network to obtain an expanded sample set;

[0042] Build a CNN-LSTM model and use the expanded sample set to train the model until the network converges to obtain a trained persistent memory fault prediction model;

[0043] Use the trained persistent memory fault prediction model to monitor the system's operating status in real time, predict potential persistent memory failures, and decide whether to issue a fault warning by setting thresholds.

[0044] Furthermore, historical indicator data includes: CPU usage, memory usage, disk I / O, network traffic, system logs, etc.

[0045] Furthermore, in order to remove noise and improve data quality, this embodiment proposes to preprocess the collected initial data, which specifically includes the following steps:

[0046] According to the predefined data cleaning rules, different types of historical indicator data are cleaned, including missing value processing, duplicate value processing, outlier processing, etc.;

[0047] The Z-score standardization method was used to normalize the cleaned data, and the initial sample set was constructed based on the standardized data.

[0048] Specifically, missing value processing can be achieved by missing value deletion or missing value interpolation, duplicate value processing can be achieved by a multi-pass nearest neighbor sorting algorithm, and outlier processing can be achieved by an improved isolation forest algorithm based on PCA.

[0049] Furthermore, the initial sample set is expanded using a generative adversarial network, which specifically includes the following steps:

[0050] Use random noise to train the generative adversarial network, update the parameters of the discriminator and generator through the discriminator loss function and the generator loss function until the loss function converges, and use the parameters of the discriminator and generator at the end of the training as the parameters of the generative adversarial network to generate a sample expansion model;

[0051] The initial sample set is input into the sample expansion model to generate an expanded sample set that is close to the actual historical indicator data.

[0052] In this embodiment, data expansion using a generative adversarial network can obtain and learn the advantages of the underlying data distribution of the input data, generate data that is relatively similar to the real indicator data collected, solve the problem of lack of data in the field of persistent memory research, and achieve the expansion of various indicator data in persistent memory, providing more data for model training, and improving the prediction rate and accuracy of the prediction model.

[0053] LSTM (Long Short-Term Memory) is a powerful deep learning model that can effectively process sequence data and learn long-term dependencies. It has been widely used in fields such as time series prediction and pattern recognition. Therefore, LSTM is also used in the fault prediction model constructed in this embodiment to capture the long-term dependencies in time series data and improve the accuracy of fault detection.

[0054] Furthermore, the CNN-LSTM model includes: input layer, CNN network, LSTM network, fusion network, fully connected layer and output layer;

[0055] Input layer, used to input the expanded sample set;

[0056] The CNN network includes a convolutional layer, a maximum pooling layer, and a dense layer connected in sequence, which are used to extract spatial features from the expanded sample set;

[0057] The LSTM network includes a forget gate, an input gate, and an output gate, which are used to extract temporal features from the expanded sample set;

[0058] The fusion network is used to connect the extracted spatial features and temporal features in series to obtain fusion features;

[0059] Fully connected layer, used to identify persistent memory failures based on fused features;

[0060] The output layer is used to output the fault prediction results.

[0061] In this embodiment, a fault prediction model is constructed based on CNN and LSTM. Low-dimensional deep features are extracted based on CNN, and the horizontal features in the low-dimensional deep features are further extracted and upgraded. At the same time, the features in the longitudinal time dimension of the low-dimensional deep features are extracted and reduced. When LSTM is used for training, the time overhead of model training can be reduced and the precision and accuracy of the model can be improved.

[0062] Furthermore, model training specifically includes the following steps:

[0063] Based on the preset ratio, the expanded sample set is divided into a training set and a test set;

[0064] Randomly initialize the weights of the CNN-LSTM model and train the CNN-LSTM model based on the training set;

[0065] The back propagation algorithm is used to obtain the changing trend of the prediction accuracy of the test set, the loss is calculated by combining the cross entropy loss function, and the Adam optimizer is selected to optimize the CNN-LSTM model;

[0066] The training is terminated until the loss function converges, and a trained persistent memory fault prediction model is obtained.

[0067] In this embodiment, the CNN-LSTM model is trained through labeled historical data so that it can learn the behavior patterns and fault characteristics of the memory system. Continuously learning new data will enable the model to better adapt to changes in the system environment and establish a more accurate memory fault prediction model, so as to timely detect memory faults and take measures to repair them, thereby ensuring the stable operation of the system.

[0068] Furthermore, in another embodiment, after predicting a potential failure of the persistent memory, the method further includes:

[0069] When the potential fault of the persistent memory is an uncorrectable error, determining to repair the fault based on the business load and / or the business load growth;

[0070] When the potential fault of the persistent memory is a correctable error, the memory page corresponding to the fault is isolated through a preset interface and divided into a fault isolation area;

[0071] When there is no potential failure in the persistent memory, continue to monitor the operating status of the system.

[0072] Since ignoring correctable memory faults will cause a small number of correctable memory faults to gradually spread and worsen, and eventually develop into uncorrectable faults, if the uncorrectable faults exceed the fault tolerance range of the hardware platform, it will cause the system to crash or the application to stop running. In this embodiment, when a correctable error is detected, the correctable memory fault that occurs is handled at the software level, and the memory page is isolated through a preset interface for easy repair. The memory page is divided into a fault isolation area and all data in the isolation area is migrated to a fault-free memory area, so that the correctable errors will not accumulate to form uncorrectable errors.

[0073] In addition, when an uncorrectable error is detected, the system business load is taken into consideration. When the business load or business load growth is greater than or equal to the corresponding threshold, the system determines to repair the fault. The detected memory fault can be repaired at a more appropriate time, and self-healing resources can be used reasonably to reduce the probability of subsequent system downtime.

[0074] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0075] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fault detection method for a persistent memory server, characterized in that: The following steps are involved: Collect various historical indicator data during system operation from persistent memory and perform preprocessing to build an initial sample set; The initial sample set is expanded using a generative adversarial network to obtain an expanded sample set; Build a CNN-LSTM model and use the expanded sample set to train the model until the network converges to obtain a trained persistent memory fault prediction model; Use the trained persistent memory fault prediction model to monitor the system's operating status in real time, predict potential persistent memory failures, and decide whether to issue a fault warning by setting thresholds.

2. A method for fault detection of a persistent memory server according to claim 1, characterized in that: Historical indicator data includes: CPU usage, memory usage, disk I / O, network traffic, and system logs.

3. A fault detection method for a persistent memory server according to claim 1, characterized in that: The preprocessing operation specifically includes the following steps: Clean different types of historical indicator data according to predefined data cleaning rules; The Z-score standardization method was used to normalize the cleaned data, and the initial sample set was constructed based on the standardized data.

4. A method for fault detection of a persistent memory server according to claim 1, characterized in that: The initial sample set is expanded using a generative adversarial network, which includes the following steps: Use random noise to train the generative adversarial network, update the parameters of the discriminator and generator through the discriminator loss function and the generator loss function until the loss function converges, and use the parameters of the discriminator and generator at the end of the training as the parameters of the generative adversarial network to generate a sample expansion model; The initial sample set is input into the sample expansion model to generate an expanded sample set that is close to the actual historical indicator data.

5. A fault detection method for a persistent memory server according to claim 1, characterized in that: The CNN-LSTM model includes: input layer, CNN network, LSTM network, fusion network, fully connected layer and output layer; Input layer, used to input the expanded sample set; The CNN network includes a convolutional layer, a maximum pooling layer, and a dense layer connected in sequence, which are used to extract spatial features from the expanded sample set; The LSTM network includes a forget gate, an input gate, and an output gate, which are used to extract temporal features from the expanded sample set; The fusion network is used to connect the extracted spatial features and temporal features in series to obtain fusion features; Fully connected layer, used to identify persistent memory failures based on fused features; The output layer is used to output the fault prediction results.

6. A method for fault detection of a persistent memory server according to claim 1, characterized in that: Model training specifically includes the following steps: Based on the preset ratio, the expanded sample set is divided into a training set and a test set; Randomly initialize the weights of the CNN-LSTM model and train the CNN-LSTM model based on the training set; The back propagation algorithm is used to obtain the changing trend of the prediction accuracy of the test set, the loss is calculated by combining the cross entropy loss function, and the Adam optimizer is selected to optimize the CNN-LSTM model; The training is terminated until the loss function converges, and a trained persistent memory fault prediction model is obtained.

7. A fault detection method for a persistent memory server according to claim 1, characterized in that: After predicting potential failures of persistent memory, it also includes: When the potential fault of the persistent memory is an uncorrectable error, determining to repair the fault based on the business load and / or the business load growth; When the potential fault of the persistent memory is a correctable error, the memory page corresponding to the fault is isolated through a preset interface and divided into a fault isolation area; When there is no potential failure in the persistent memory, continue to monitor the operating status of the system.

Images