A method, device, system and readable storage medium for detecting disk failures

Abstract

The present invention discloses a method, device, system and computer-readable storage medium for detecting disk failures. The method includes: pre-establishing a state information sample pool based on faulty disks and healthy disks of multiple disk models, where the state information sample pool includes state information samples of multiple faulty disks and state information samples of multiple healthy disks; training a pre-established twin network based on the state information samples in the state information sample pool to obtain a trained disk failure prediction model; using the disk failure prediction model to process and analyze the current state information of the disk to be detected to obtain the current state of the disk to be detected; since the disk failure prediction model in the present invention is established based on the state information of multiple faulty disks and multiple healthy disks of multiple disk models, the disk failure prediction model can more accurately predict the disk state, and has high prediction efficiency and wide application scope.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular, to a method, apparatus, system, and computer-readable storage medium for detecting disk failures. Background Art

[0002] In a storage system, there are numerous disk manufacturers and models, and there are hundreds or thousands of disks for a single model. Additionally, in an actual production environment, as time goes by, disks are updated, and the emergence of new models poses new challenges to fault prediction.

[0003] The current disk fault prediction methods include the following: threshold-based, machine learning-based, deep learning-based, transfer learning, etc. Among them, the threshold-based method is relatively conservative and has low accuracy. Machine learning and deep learning-based models require a large amount of data for training and perform well only on single-model disks, with weak generalization ability for other models that have not been seen before. Transfer learning can solve the problem of insufficient data for the target model disk, but it is difficult to find a source model disk with a sufficient amount of data for the target model disk in practical applications.

[0004] In view of this, how to provide a disk fault detection method, apparatus, system, and computer-readable storage medium with high detection accuracy and applicable to multiple models has become a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a method, apparatus, system, and computer-readable storage medium for detecting disk failures, which can more accurately predict the disk state during use, with high prediction efficiency and a wide range of applications.

[0006] To solve the above technical problems, the embodiments of the present invention provide a method for detecting disk failures, including:

[0007] Pre-establish a state information sample pool based on faulty disks and healthy disks of multiple disk models, where the state information sample pool includes state information samples of multiple faulty disks and state information samples of multiple healthy disks;

[0008] Train a pre-established siamese network based on the state information samples in the state information sample pool to obtain a trained disk fault prediction model;

[0009] Use the disk fault prediction model to process and analyze the current state information of the disk to be detected to obtain the current state of the disk to be detected.

[0010] Optionally, the process of pre - establishing a state information sample pool based on faulty disks and healthy disks of multiple disk models, where the state information sample pool includes state information samples of multiple faulty disks and state information samples of multiple healthy disks is as follows:

[0011] For each disk model, determine multiple faulty disks and multiple healthy disks;

[0012] For each of the faulty disks or each of the healthy disks, obtain each historical SMART feature information of each disk, where the disk is a faulty disk or a healthy disk;

[0013] Process each historical SMART feature information of each disk to obtain a SMART feature matrix sample corresponding to each faulty disk and a SMART feature matrix sample corresponding to each healthy disk respectively, and form a state information sample pool.

[0014] Optionally, the process of training a pre - established siamese network based on the state information samples in the state information sample pool to obtain a trained disk fault prediction model is as follows:

[0015] According to the SMART feature matrix samples corresponding to each faulty disk and the SMART feature matrix samples corresponding to each healthy disk in the state information sample pool, construct multiple sample pairs and labels corresponding to each sample pair respectively;

[0016] Based on the stochastic gradient descent method, use each sample pair to train the pre - established siamese network to obtain the optimal network parameters after training and the corresponding disk fault prediction model.

[0017] Optionally, the process of training a pre - established siamese network based on the stochastic gradient descent method using each sample pair to obtain the optimal network parameters after training and the corresponding disk fault prediction model is as follows:

[0018] Input the first sample and the second sample in the current sample pair into the first encoding network and the second encoding network of the siamese network respectively to obtain a first feature vector corresponding to the first sample and a second feature vector corresponding to the second sample; where the network parameters of the first encoding network and the second encoding network are the same and are both the current network parameters;

[0019] According to the first feature vector, the second feature vector, and the label corresponding to the current sample pair, calculate the Euclidean distance between the first sample and the second sample;

[0020] Establish a loss function based on the Euclidean distance, and update the current network parameters using the gradient of the loss function;

[0021] Determine whether the termination condition is satisfied. If it is satisfied, use the updated network parameters as the optimal network parameters, and use the Siamese network based on the optimal network parameters as the trained disk fault prediction model;

[0022] If not, use the next sample pair as the current sample pair, use the updated network parameters as the current network parameters, and return to execute the process of inputting the first sample and the second sample in the current sample pair into the first encoding network and the second encoding network of the Siamese network respectively to obtain the first feature vector corresponding to the first sample and the second feature vector corresponding to the second sample.

[0023] Optionally, the process of calculating the Euclidean distance between the first sample and the second sample according to the first feature vector, the second feature vector, and the label corresponding to the current sample pair is:

[0024] Calculate the Euclidean distance between the first sample and the second sample according to the first feature vector, the second feature vector, the label corresponding to the current sample pair, and the first calculation formula, where:

[0025] The first calculation formula is where, ED w (S1, S2, Y) is the Euclidean distance between the first sample S1 and the second sample S2, Y is the label of the sample pair (S1, S2, Y), F sig is the sigmoid activation function, F w (S1) is the first feature vector, F w (S2) is the second feature vector, and N' is the dimension of the feature vector.

[0026] Optionally, the loss function is:

[0027] L(w, S1, S2, Y) = Y × ED w 2 + (1 - Y) × (1 - ED w ) 2 , when the states of S1 and S2 are the same, Y is 1, and the loss function is ED w 2 ; when the states of S1 and S2 are different, Y is 0, and the loss function is (1 - ED w ) 2 , where, w is the current network parameter.

[0028] Optionally, the process of processing the current status information of the disk to be detected by using the disk failure prediction model to determine the current status of the disk to be detected is as follows:

[0029] Obtain each SMART feature information corresponding to the disk to be detected and between the current moment and a preset historical moment, and form a SMART feature matrix;

[0030] Identify the disk model of the disk to be tested, and determine whether there is a SMART feature matrix sample of a target failed disk with the same disk model in the status information sample pool;

[0031] If there is a SMART feature matrix sample of a target failed disk with the same disk model, obtain multiple first target SMART feature matrix samples;

[0032] Combine the SMART feature matrix with each of the first target SMART feature matrix samples respectively to form multiple first samples to be tested;

[0033] Use the disk failure prediction model to process each of the first samples to be tested respectively, and obtain a first Euclidean distance corresponding to each of the first samples to be tested;

[0034] When there is a first Euclidean distance less than a first preset value among the first Euclidean distances, determine that the current status of the disk to be detected is a failure status;

[0035] If there is no SMART feature matrix sample of a target failed disk with the same disk model, determine whether there is a second target SMART feature matrix sample of a target healthy disk with the same disk model in the status information sample pool;

[0036] If there is a second target SMART feature matrix sample of a target healthy disk with the same disk model, obtain multiple second target SMART feature matrix samples;

[0037] Combine the SMART feature matrix with each of the second target SMART feature matrix samples respectively to form multiple second samples to be tested;

[0038] Use the disk failure prediction model to process each of the second samples to be tested respectively, and obtain a second Euclidean distance corresponding to each of the second samples to be tested;

[0039] When all the second Euclidean distances are less than a second preset value, determine that the current status of the disk to be detected is a healthy status;

[0040] If there is no second target SMART feature matrix sample of a target healthy disk with the same disk model as the disk to be detected, determine whether there is a third target SMART feature matrix sample corresponding to a target faulty disk of other disk models in the status information sample pool;

[0041] If there is a third target SMART feature matrix sample corresponding to a target faulty disk of other disk models, obtain multiple third target SMART feature matrix samples;

[0042] Combine the SMART feature matrix with each of the third target SMART feature matrix samples to form multiple third test sample pairs;

[0043] Use the disk fault prediction model to process each of the third test sample pairs to obtain a third Euclidean distance corresponding to each of the third test sample pairs;

[0044] When a preset number of the third Euclidean distances among all the third Euclidean distances are less than a third preset value, determine that the current state of the disk to be detected is a faulty state;

[0045] If there is no third target SMART feature matrix sample corresponding to a target faulty disk of other disk models, determine whether there is a fourth target SMART feature matrix sample corresponding to a target healthy disk of other disk models in the status information sample pool;

[0046] If there is a fourth target SMART feature matrix sample corresponding to a target healthy disk of other disk models, obtain multiple fourth target SMART feature matrix samples;

[0047] Combine the SMART feature matrix with each of the fourth target SMART feature matrix samples to form multiple fourth test sample pairs;

[0048] Use the disk fault prediction model to process each of the fourth test sample pairs to obtain a fourth Euclidean distance corresponding to each of the fourth test sample pairs;

[0049] When all the fourth Euclidean distances are less than a fourth preset value, determine that the current state of the disk to be detected is a healthy state.

[0050] An embodiment of the present invention further provides a disk fault detection device, including:

[0051] A building module, configured to pre - establish a status information sample pool based on faulty disks and healthy disks of multiple disk models, where the status information sample pool includes status information samples of multiple faulty disks and status information samples of multiple healthy disks;

[0052] A training module, configured to train a pre - established twin network based on the status information samples in the status information sample pool to obtain a trained disk fault prediction model;

[0053] A prediction module, configured to process and analyze the current status information of a disk to be detected by using the disk fault prediction model to obtain the current status of the disk to be detected.

[0054] An embodiment of the present invention further provides a disk fault detection system, including:

[0055] A memory, configured to store a computer program;

[0056] A processor, configured to implement the steps of the disk fault detection method as described above when executing the computer program.

[0057] An embodiment of the present invention further provides a computer - readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the disk fault detection method as described above are implemented.

[0058] An embodiment of the present invention provides a disk fault detection method, device, system and computer - readable storage medium. The method includes: pre - establishing a status information sample pool based on faulty disks and healthy disks of multiple disk models, where the status information sample pool includes status information samples of multiple faulty disks and status information samples of multiple healthy disks; training a pre - established twin network based on the status information samples in the status information sample pool to obtain a trained disk fault prediction model; processing and analyzing the current status information of a disk to be detected by using the disk fault prediction model to obtain the current status of the disk to be detected.

[0059] It can be seen that when establishing the disk fault prediction model in the embodiments of the present invention, the state information of multiple faulty disks and the state information of multiple healthy disks are first determined based on the faulty disks and healthy disks of multiple disk models, and a state information sample pool is established. Then, the twin network is trained according to each state information in the state information sample pool to obtain the trained disk fault prediction model. When predicting the fault of the disk to be detected, the current state information of the disk to be detected is obtained, and the disk fault prediction model is used to process and analyze the current state information to obtain the current state of the disk to be detected. Since the disk fault prediction model in the present invention is established based on the state information of multiple faulty disks and multiple healthy disks of multiple disk models, the disk fault prediction model can more accurately predict the disk state, and has high prediction efficiency and wide application range. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the prior art and the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0061] Figure 1 It is a schematic flowchart of a disk fault detection method provided by an embodiment of the present invention;

[0062] Figure 2 It is a schematic structural diagram of a twin network provided by an embodiment of the present invention;

[0063] Figure 3 It is a schematic structural diagram of a disk fault detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0064] The embodiments of the present invention provide a disk fault detection method, device, system and computer-readable storage medium, which can more accurately predict the disk state during use, and have high prediction efficiency and wide application range.

[0065] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0066] Please refer to Figure 1 , Figure 1The flowchart of a disk failure detection method provided by an embodiment of the present invention. The method includes:

[0067] S110: Based on the failed disks and healthy disks of multiple disk models, establish a state information sample pool in advance. The state information sample pool includes state information samples of multiple failed disks and state information samples of multiple healthy disks;

[0068] It should be noted that in the embodiment of the present invention when establishing a disk failure prediction model, multiple disk models are determined in advance, and for each disk model, multiple failed disks and multiple healthy disks are further determined. For example, for disk model M1, m1 failed disks and m2 healthy disks are determined, and for disk model M2, n1 failed disks and n2 healthy disks are determined. Among them, the ratios of m1:m2 and n1:n2 can both be 1:3, and m1 can be equal to n1, and m2 can be equal to n2. The specific values can be determined according to actual needs, and the embodiment of the present invention does not make special limitations. Then, for each disk (failed disk or healthy disk), multiple state information samples are obtained according to the historical state data of the disk, and then multiple state information samples are obtained according to the historical state data corresponding to each disk. All the state information samples form the state information sample pool.

[0069] Further, the process of establishing the state information sample pool based on the failed disks and healthy disks of multiple disk models in advance, where the state information sample pool includes state information samples of multiple failed disks and state information samples of multiple healthy disks, can specifically be:

[0070] For each disk model, determine multiple failed disks and multiple healthy disks;

[0071] For each failed disk or each healthy disk, obtain each historical SMART feature information of the disk, where the disk is a failed disk or a healthy disk;

[0072] Process each historical SMART feature information of each disk to obtain a SMART feature matrix sample corresponding to each failed disk and a SMART feature matrix sample corresponding to each healthy disk, and form the state information sample pool.

[0073] It should be noted that in the embodiments of the present invention, for each hard disk (failed disk or healthy disk), historical SMART feature information of the disk is obtained. Specifically, historical SMART feature information of the last D days of the disk can be obtained. Among them, the number of SMART features is N, and N is a positive integer. These historical SMART feature information constitute a two-dimensional SMART feature matrix of D×N. The SMART values in the two-dimensional SMART feature matrix are subjected to maximum-minimum normalization and normalized to between 0 and 1. The normalized two-dimensional SMART feature matrix is used as a two-dimensional SMART feature matrix sample, so that a two-dimensional SMART feature matrix corresponding to each disk can be obtained. These two-dimensional SMART feature matrices constitute a state information sample pool.

[0074] S120: Train a pre-established twin network based on the state information samples in the state information sample pool to obtain a trained disk failure prediction model;

[0075] Specifically, in the embodiments of the present invention, a twin network can be pre-established. For details, please refer to Figure 2 , and the twin network includes two encoding networks. Among them, the left encoding network can be used as the first encoding network, and the right encoding network can be used as the second encoding network. Then, the twin network can be trained according to the state information samples in the state information sample pool to obtain a trained disk failure prediction model.

[0076] Further, the process of training the pre-established twin network based on the state information samples in the state information sample pool to obtain a trained disk failure prediction model in S120 above can specifically be:

[0077] According to the SMART feature matrix samples corresponding to each failed disk and the SMART feature matrix samples corresponding to each healthy disk in the state information sample pool, a plurality of sample pairs and labels corresponding to each sample pair are constructed;

[0078] Based on the stochastic gradient descent method, each sample pair is used to train the pre-established twin network to obtain the optimal network parameters after training and the corresponding disk failure prediction model.

[0079] It should be noted that two samples can be randomly selected with replacement from the SMART feature matrix samples in the status information sample pool to form a sample pair, and the label of the formed sample pair is determined according to the respective statuses of the two samples. Among them, when the statuses of the two samples are the same, the corresponding label is 1, and when the statuses of the two samples are different, the corresponding label is 0. Among them, the status of the SMART feature matrix sample of the faulty disk is faulty, and the status of the SMART feature matrix sample of the healthy disk is healthy. For example, sample S1 and sample S2 form a sample pair, and its label is Y, then the sample pair format can be (S1, S2, Y). All sample pairs can form a sample pair data pool, and the sample pairs in the sample pair data pool can also be randomly divided into a training set and a test set according to a preset ratio (such as 8:2).

[0080] Specifically, the hidden layer of the twin network in the embodiment of the present invention (please refer to Figure 2 ) is composed of two encoding networks with the same network parameters. Among them, the encoding network consists of two layers of convolutional networks followed by two layers of fully connected networks for automatically extracting features. The specific structure of the encoding network is as follows:

[0081] Create a Sequential model using Keras

[0082] Add a two-dimensional convolutional layer with 5 convolutional kernels, kernel size 3x3, and activation function relu

[0083] Add a pooling layer with size 2x2

[0084] Add a two-dimensional convolutional layer with 10 convolutional kernels, kernel size 3x3, and activation function relu

[0085] Add a pooling layer with size 2x2

[0086] Add a Flatten layer to "flatten" the output of the convolutional layer, that is, to make the multi-dimensional input one-dimensional

[0087] Add a fully connected layer with 100 neurons and activation function relu

[0088] Add a fully connected layer with 50 neurons and activation function relu. Specifically, when performing the disk fault prediction model, first initialize the network parameters of the twin network, and sequentially input each sample pair into the twin network. Each time, two feature vectors corresponding to the two samples in the input sample pair are obtained, then calculate the Euclidean distance between the two feature vectors, and establish a loss function based on the Euclidean distance. Then, use the stochastic gradient descent method to update the network parameters until the training ends to obtain the trained disk fault prediction model.

[0089] Further, the process of training the pre-established Siamese network with each sample pair based on the stochastic gradient descent method to obtain the optimal network parameters after training and the corresponding disk fault prediction model is as follows:

[0090] Input the first sample and the second sample in the current sample pair into the first encoding network and the second encoding network of the Siamese network respectively to obtain a first feature vector corresponding to the first sample and a second feature vector corresponding to the second sample; wherein, the network parameters of the first encoding network and the second encoding network are the same and are both the current network parameters;

[0091] Calculate the Euclidean distance between the first sample and the second sample according to the first feature vector, the second feature vector and the label corresponding to the current sample pair;

[0092] Establish a loss function according to the Euclidean distance, and update the current network parameters with the gradient of the loss function;

[0093] Judge whether the termination condition is satisfied. If it is satisfied, use the updated network parameters as the optimal network parameters, and use the Siamese network based on the optimal network parameters as the trained disk fault prediction model;

[0094] If not, use the next sample pair as the current sample pair, use the updated network parameters as the current network parameters, and return to execute the process of inputting the first sample and the second sample in the current sample pair into the first encoding network and the second encoding network of the Siamese network respectively to obtain a first feature vector corresponding to the first sample and a second feature vector corresponding to the second sample.

[0095] Specifically, the hidden layer of the Siamese network in the embodiment of the present invention (please refer to Figure 2 ) is composed of two encoding networks with the same network parameters, namely the first encoding network and the second encoding network. When performing

[0096] The first calculation formula is where ED w (S1, S2, Y) is the Euclidean distance between the first sample S1 and the second sample S2, Y is the label of the sample pair (S1, S2, Y), F sig is the sigmoid activation function (used to normalize the Euclidean distance to between 0 and 1), F w (S1) is the first feature vector, F w (S2) is the second feature vector, and N' is the dimension of the feature vector.

[0097] Construct the loss function L(w, S1, S2, Y) = Y × ED w 2 +(1 - Y) × (1 - EDw ) 2 When the states of S1 and S2 are the same, Y is 1 and the loss function is ED w 2 The greater the distance, the greater the loss; when the states of S1 and S2 are different, Y is 0 and the loss function is (1 - ED w ) 2 The smaller the distance, the greater the loss, where w are the current network parameters.

[0098] Specifically, update the current network parameter w according to the derivative of the loss function with respect to w, and obtain the updated network parameter. After each update is completed, determine whether the termination condition is satisfied. The termination condition can be that the current cumulative number of updates reaches a preset number, or the loss of the loss function is less than a preset loss value. If the termination condition is satisfied, use the updated network parameter as the optimal network parameter, and use the siamese network based on the optimal network parameter as the trained disk failure prediction model; if the termination condition is not satisfied, use the next sample pair as the current sample pair, use the updated network parameter as the current network parameter, and return to execute the process of inputting the first sample and the second sample in the current sample pair into the first encoding network and the second encoding network of the siamese network respectively, to obtain the first feature vector corresponding to the first sample and the second feature vector corresponding to the second sample, until the termination condition is finally satisfied and the trained disk failure prediction model is obtained.

[0099] It should be noted that the training process of the disk failure prediction model in S110 and S120 in the embodiments of the present invention only needs to be executed once in advance. When predicting the state of the disk to be detected subsequently, it is not necessary to perform model training every time.

[0100] S130: Use the disk failure prediction model to process and analyze the current state information of the disk to be detected, and obtain the current state of the disk to be detected.

[0101] It can be understood that when predicting the disk to be detected in the embodiments of the present invention, the current state information can be obtained. Specifically, the SMART feature information corresponding to the disk to be detected and between the current moment and the preset historical moment can be obtained to form a SMART feature matrix; for example, obtain the SMART feature information of the disk to be detected in the last D days, the number of features of the SMART feature information is N, and then form a two-dimensional SMART feature of D×N, and normalize the maximum and minimum values of the SMART values in the two-dimensional SMART feature to between 0 and 1.

[0102] Since the SMART feature matrix samples of multiple disk models are stored in the status information sample pool, the disk model of the disk to be detected can be identified, and it can be determined whether there is a SMART feature matrix sample of the target faulty disk with the same disk model in the status information sample pool; that is, it is determined whether there is a faulty disk in the corresponding faulty disks in the status information sample pool with the same disk model as the disk to be detected. If there is, the faulty disk is used as the target faulty disk.

[0103] If there is a SMART feature matrix sample of the target faulty disk with the same disk model in the status information sample pool, multiple first target SMART feature matrix samples are obtained, and the SMART feature matrix of the disk to be detected is combined with each of the first target SMART feature matrix samples respectively to form multiple first pairs of samples to be tested; specifically, 20% of the samples can be randomly selected from each of the SMART feature matrix samples corresponding to the target faulty disk as each of the first target SMART feature matrix samples, and the first pairs of samples to be tested are constructed. For example, if the SMART feature matrix of the disk to be detected is Q, and each of the first target SMART feature matrix samples is A1, A2, A3, A4 respectively, then the composed pairs of samples are (Q, A1, Y1), (Q, A2, Y2), (Q, A3, Y4), (Q, A4, Y4).

[0104] The disk fault prediction model is used to process each of the first pairs of samples to be tested respectively to obtain the first Euclidean distance corresponding to each of the first pairs of samples to be tested. When there is a first Euclidean distance less than the first preset value among the first Euclidean distances, it is determined that the current state of the disk to be detected is a faulty state; specifically, the disk fault prediction model can be used to process each of the first pairs of samples to be tested in turn. If the currently obtained first Euclidean distance is less than the first preset value, the processing of the subsequent other first pairs of samples to be tested can be stopped, and the conclusion that the current state of the disk to be detected is a faulty state can be drawn.

[0105] If there is no SMART feature matrix sample of the target faulty disk with the same disk model in the status information sample pool, it can be further determined whether there is a second target SMART feature matrix sample of the target healthy disk with the same disk model in the status information sample pool; that is, it is determined whether there is a healthy disk in the corresponding healthy disks in the status information sample pool with the same disk model as the disk to be detected. If there is, the healthy disk is used as the target faulty disk.

[0106] If there is a second target SMART feature matrix sample of a target healthy disk with the same disk model, obtain multiple second target SMART feature matrix samples. Specifically, 10% of the samples can be randomly selected from each SMART feature matrix sample corresponding to the target healthy disk as each second target SMART feature matrix sample, and a second sample pair to be tested is constructed. Then, the SMART feature matrix is combined with each second target SMART feature matrix sample respectively to form multiple second sample pairs to be tested. Next, the disk failure prediction model is used to process each second sample pair to be tested respectively, and a second Euclidean distance corresponding to each second sample pair to be tested is obtained. And when each second Euclidean distance is less than a second preset value, it is determined that the current state of the disk to be detected is a healthy state.

[0107] If there is no second target SMART feature matrix sample of a target healthy disk with the same disk model in the state information sample pool, further determine whether there is a third target SMART feature matrix sample corresponding to a target failed disk of another disk model (i.e., a disk model different from the disk model of the disk to be detected) in the state information sample pool. If there is a third target SMART feature matrix sample corresponding to a target failed disk of another disk model, obtain multiple third target SMART feature matrix samples. Specifically, 20% of the samples can be obtained from each third target SMART feature matrix sample corresponding to the target failed disk of another disk model as each third target SMART feature matrix sample. The SMART feature matrix is combined with each third target SMART feature matrix sample respectively to form multiple third sample pairs to be tested. Then, the disk failure prediction model is used to process each third sample pair to be tested respectively, and a third Euclidean distance corresponding to each third sample pair to be tested is obtained. And when a preset number of third Euclidean distances among each third Euclidean distance are less than a third preset value, it is determined that the current state of the disk to be detected is a failed state.

[0108] If there is no third target SMART feature matrix sample corresponding to the target faulty disk of other disk models in the status information sample pool, continue to determine whether there is a fourth target SMART feature matrix sample corresponding to the target healthy disk of other disk models in the status information sample pool. If there is a fourth target SMART feature matrix sample corresponding to the target healthy disk of other disk models, obtain multiple fourth target SMART feature matrix samples. Specifically, it can be obtained that the SMART feature matrix is combined with each fourth target SMART feature matrix sample respectively to form multiple fourth samples to be tested. And the disk fault prediction model is used to process each fourth sample to be tested respectively, and the fourth Euclidean distance corresponding to each fourth sample to be tested is obtained. When each fourth Euclidean distance is less than the fourth preset value, it is determined that the current state of the disk to be detected is a healthy state.

[0109] It can be seen that when establishing the disk fault prediction model in the embodiment of the present invention, first, based on the faulty disks and healthy disks of multiple disk models, the status information of multiple faulty disks and the status information of multiple healthy disks are determined, and a status information sample pool is established. Then, according to each status information in the status information sample pool, the twin network is trained to obtain the trained disk fault prediction model. When predicting the fault of the disk to be detected, the current status information of the disk to be detected is obtained and the disk fault prediction model is used to process and analyze the current status information to obtain the current state of the disk to be detected. Since the disk fault prediction model in the present invention is established based on the status information of multiple faulty disks and multiple healthy disks of multiple disk models, the disk fault prediction model can more accurately predict the disk status, and has high prediction efficiency and wide application range.

[0110] Based on the above embodiments, the embodiment of the present invention also provides a disk fault detection device, specifically referring to Figure 3 , the device includes:

[0111] A building module 21, configured to pre-establish a status information sample pool based on faulty disks and healthy disks of multiple disk models. The status information sample pool includes status information samples of multiple faulty disks and status information samples of multiple healthy disks;

[0112] A training module 22, configured to train a pre-established twin network based on the status information samples in the status information sample pool to obtain a trained disk fault prediction model;

[0113] A prediction module 23, configured to process and analyze the current status information of the disk to be detected by using the disk fault prediction model to obtain the current state of the disk to be detected.

[0114] It should be noted that the disk failure detection device in the embodiments of the present invention has the same beneficial effects as the disk failure detection method in the above embodiments. For the specific introduction of the disk failure detection method involved in the embodiments of the present invention, please refer to the above embodiments, and the present invention will not elaborate here.

[0115] Based on the above embodiments, the embodiments of the present invention further provide a disk failure detection system, including:

[0116] A memory for storing a computer program;

[0117] A processor for implementing the steps of the disk failure detection method as described above when executing the computer program.

[0118] For example, the processor in the embodiments of the present invention can specifically be used to pre-establish a state information sample pool based on the faulty disks and healthy disks of multiple disk models. The state information sample pool includes state information samples of multiple faulty disks and state information samples of multiple healthy disks; train a pre-established twin network based on the state information samples in the state information sample pool to obtain a trained disk failure prediction model; use the disk failure prediction model to process and analyze the current state information of the disk to be detected to obtain the current state of the disk to be detected.

[0119] Based on the above embodiments, the embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the disk failure detection method as described above are implemented.

[0120] The computer-readable storage medium may include various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc.

[0121] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0122] It should also be noted that in this specification, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0123] The foregoing description of the disclosed embodiments enables those skilled in the art to practice or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A disk failure detection method, characterized in that, Including: Pre - establish a state information sample pool based on faulty disks and healthy disks of multiple disk models. The state information sample pool includes state information samples of multiple faulty disks and state information samples of multiple healthy disks; the number of faulty disks of each disk model is the same, and the number of healthy disks of each disk model is the same; Train a pre - established twin network based on the state information samples in the state information sample pool to obtain a trained disk fault prediction model; Use the disk fault prediction model to process and analyze the current state information of the disk to be detected to obtain the current state of the disk to be detected; where: The process of pre - establishing a state information sample pool based on faulty disks and healthy disks of multiple disk models, where the state information sample pool includes state information samples of multiple faulty disks and state information samples of multiple healthy disks is as follows: For each disk model, determine multiple faulty disks and multiple healthy disks; For each of the faulty disks or each of the healthy disks, obtain each historical SMART feature information of each disk, where the disk is a faulty disk or a healthy disk; Process each historical SMART feature information of each disk to obtain a SMART feature matrix sample corresponding to each faulty disk and a SMART feature matrix sample corresponding to each healthy disk, and form a state information sample pool; The process of training a pre - established twin network based on the state information samples in the state information sample pool to obtain a trained disk fault prediction model is as follows: According to the SMART feature matrix samples corresponding to each faulty disk and the SMART feature matrix samples corresponding to each healthy disk in the state information sample pool, construct multiple sample pairs and labels corresponding to each sample pair; Input the first sample and the second sample in the current sample pair into the first encoding network and the second encoding network of the twin network respectively to obtain a first feature vector corresponding to the first sample and a second feature vector corresponding to the second sample; where the network parameters of the first encoding network and the second encoding network are the same and are both the current network parameters; Calculate the Euclidean distance between the first sample and the second sample according to the first feature vector, the second feature vector, and the label corresponding to the current sample pair; Establish a loss function according to the Euclidean distance, and update the current network parameters using the gradient of the loss function; Judge whether the termination condition is met. If it is met, take the updated network parameters as the optimal network parameters, and take the twin network based on the optimal network parameters as the trained disk fault prediction model; If not satisfied, take the next sample pair as the current sample pair, take the updated network parameters as the current network parameters, and return to execute the process of respectively inputting the first sample and the second sample in the current sample pair into the first encoding network and the second encoding network of the siamese network to obtain a first feature vector corresponding to the first sample and a second feature vector corresponding to the second sample; The process of using the disk fault prediction model to process the current state information of the disk to be detected and determine the current state of the disk to be detected is as follows: Obtain each SMART feature information corresponding to the disk to be detected between the current moment and a preset historical moment to form a SMART feature matrix; Identify the disk model of the disk to be detected, and determine whether there is a SMART feature matrix sample of a target faulty disk with the same disk model in the state information sample pool; If there is a SMART feature matrix sample of a target faulty disk with the same disk model, obtain multiple first target SMART feature matrix samples; Combine the SMART feature matrix with each of the first target SMART feature matrix samples to form multiple first samples to be tested; Use the disk fault prediction model to process each of the first samples to be tested to obtain a first Euclidean distance corresponding to each of the first samples to be tested; When there is a first Euclidean distance less than a first preset value among the first Euclidean distances, determine that the current state of the disk to be detected is a faulty state; If there is no SMART feature matrix sample of a target faulty disk with the same disk model, determine whether there is a second target SMART feature matrix sample of a target healthy disk with the same disk model in the state information sample pool; If there is a second target SMART feature matrix sample of a target healthy disk with the same disk model, obtain multiple second target SMART feature matrix samples; Combine the SMART feature matrix with each of the second target SMART feature matrix samples to form multiple second samples to be tested; Use the disk fault prediction model to process each of the second samples to be tested to obtain a second Euclidean distance corresponding to each of the second samples to be tested; When each of the second Euclidean distances is less than a second preset value, determine that the current state of the disk to be detected is a healthy state; If there is no second target SMART feature matrix sample of a target healthy disk with the same disk model, determine whether there is a third target SMART feature matrix sample corresponding to a target faulty disk of another disk model in the state information sample pool; If there is a third target SMART feature matrix sample corresponding to a target faulty disk of another disk model, obtain multiple third target SMART feature matrix samples; Combine the SMART feature matrix with each of the third target SMART feature matrix samples to form multiple third samples to be tested; Process each of the third pairs of samples to be measured using the disk failure prediction model to obtain a third Euclidean distance corresponding to each of the third pairs of samples to be measured. When a preset number of the third Euclidean distances among all the third Euclidean distances are less than a third preset value, determine that the current state of the disk to be detected is a failure state. If there is no third target SMART feature matrix sample corresponding to the target failed disk of other disk models, then determine whether there is a fourth target SMART feature matrix sample corresponding to the target healthy disk of other disk models in the state information sample pool. If there is a fourth target SMART feature matrix sample corresponding to the target healthy disk of other disk models, then obtain a plurality of the fourth target SMART feature matrix samples. Combine the SMART feature matrix with each of the fourth target SMART feature matrix samples to form a plurality of fourth pairs of samples to be measured. Process each of the fourth pairs of samples to be measured using the disk failure prediction model to obtain a fourth Euclidean distance corresponding to each of the fourth pairs of samples to be measured. When all the fourth Euclidean distances are less than a fourth preset value, determine that the current state of the disk to be detected is a healthy state.

2. The disk failure detection method according to claim 1, characterized in that, The process of calculating the Euclidean distance between the first sample and the second sample according to the first eigenvector, the second eigenvector, and the label corresponding to the current pair of samples is as follows: Calculate the Euclidean distance between the first sample and the second sample according to the first eigenvector, the second eigenvector, the label corresponding to the current pair of samples, and a first calculation formula, where: The first calculation relation is , where is the Euclidean distance between the first sample and the second sample . is the label of the sample pair . is the sigmoid activation function is the first feature vector is the second feature vector is the dimension of the feature vector.

3. The disk failure detection method according to claim 2, wherein The loss function is: When and are in the same state, Y is 1 and the loss function is ; when and are in different states, Y is 0 and the loss function is , where w is the current network parameter.

4. A disk failure detection device, characterized in that, It includes: A building module, configured to pre-establish a state information sample pool based on the failed disks and healthy disks of multiple disk models, where the state information sample pool includes state information samples of multiple failed disks and state information samples of multiple healthy disks; the number of failed disks of each disk model is the same, and the number of healthy disks of each disk model is the same. A training module, configured to train a pre-established siamese network based on the state information samples in the state information sample pool to obtain a trained disk failure prediction model. A prediction module, configured to process and analyze the current state information of the disk to be detected using the disk failure prediction model to obtain the current state of the disk to be detected. The building module is configured to, for each disk model, determine a plurality of failed disks and a plurality of healthy disks; for each of the failed disks or each of the healthy disks, obtain each historical SMART feature information of each disk, where the disk is a failed disk or a healthy disk; process each of the historical SMART feature information of each disk to obtain a SMART feature matrix sample corresponding to each of the failed disks and a SMART feature matrix sample corresponding to each of the healthy disks, and form a state information sample pool; where: The building module is configured to: For each disk model, a plurality of faulty disks and a plurality of healthy disks are determined; for each of the faulty disks or each of the healthy disks, respective historical SMART characteristic information of each disk is obtained, where the disk is a faulty disk or a healthy disk; the respective historical SMART characteristic information of each disk is processed to obtain a SMART characteristic matrix sample corresponding to each of the faulty disks and a SMART characteristic matrix sample corresponding to each of the healthy disks, and a state information sample pool is formed; The training module is used for: According to the SMART characteristic matrix samples corresponding to each of the faulty disks and the SMART characteristic matrix samples corresponding to each of the healthy disks in the state information sample pool, a plurality of sample pairs and labels corresponding to each of the sample pairs are constructed; The first sample and the second sample in the current sample pair are respectively input into the first encoding network and the second encoding network of the siamese network to obtain a first feature vector corresponding to the first sample and a second feature vector corresponding to the second sample; wherein, the network parameters of the first encoding network and the second encoding network are the same and are both the current network parameters; According to the first feature vector, the second feature vector and the label corresponding to the current sample pair, the Euclidean distance between the first sample and the second sample is calculated; A loss function is established according to the Euclidean distance, and the current network parameters are updated by using the gradient of the loss function; It is judged whether a termination condition is satisfied. If so, the updated network parameters are used as the optimal network parameters, and the siamese network based on the optimal network parameters is used as the trained disk fault prediction model; If not, the next sample pair is used as the current sample pair, the updated network parameters are used as the current network parameters, and the process of inputting the first sample and the second sample in the current sample pair into the first encoding network and the second encoding network of the siamese network respectively to obtain a first feature vector corresponding to the first sample and a second feature vector corresponding to the second sample is returned; The prediction module is used for: Obtain respective SMART characteristic information corresponding to the disk to be detected between the current moment and the preset historical moment to form a SMART characteristic matrix; Identify the disk model of the disk to be detected, and judge whether there is a SMART characteristic matrix sample of a target faulty disk with the same disk model in the state information sample pool; If there is a SMART characteristic matrix sample of a target faulty disk with the same disk model, obtain a plurality of first target SMART characteristic matrix samples; The SMART characteristic matrix is respectively combined with each of the first target SMART characteristic matrix samples to form a plurality of first samples to be detected; The disk fault prediction model is used to process each of the first samples to be detected to obtain a first Euclidean distance corresponding to each of the first samples to be detected; When there is a first Euclidean distance less than a first preset value among the first Euclidean distances, determine that the current state of the disk to be detected is a faulty state; If there is no sample of the SMART feature matrix of the target faulty disk with the same disk model, determine whether there is a second target SMART feature matrix sample of the target healthy disk with the same disk model in the state information sample pool; If there is a second target SMART feature matrix sample of the target healthy disk with the same disk model, obtain multiple second target SMART feature matrix samples; Combine the SMART feature matrix with each of the second target SMART feature matrix samples to form multiple second samples to be tested; Use the disk fault prediction model to process each of the second samples to be tested, and obtain a second Euclidean distance corresponding to each of the second samples to be tested; When each of the second Euclidean distances is less than a second preset value, determine that the current state of the disk to be detected is a healthy state; If there is no second target SMART feature matrix sample of the target healthy disk with the same disk model, determine whether there is a third target SMART feature matrix sample corresponding to the target faulty disk of other disk models in the state information sample pool; If there is a third target SMART feature matrix sample corresponding to the target faulty disk of other disk models, obtain multiple third target SMART feature matrix samples; Combine the SMART feature matrix with each of the third target SMART feature matrix samples to form multiple third samples to be tested; Use the disk fault prediction model to process each of the third samples to be tested, and obtain a third Euclidean distance corresponding to each of the third samples to be tested; When a preset number of the third Euclidean distances among the third Euclidean distances are less than a third preset value, determine that the current state of the disk to be detected is a faulty state; If there is no third target SMART feature matrix sample corresponding to the target faulty disk of other disk models, determine whether there is a fourth target SMART feature matrix sample corresponding to the target healthy disk of other disk models in the state information sample pool; If there is a fourth target SMART feature matrix sample corresponding to the target healthy disk of other disk models, obtain multiple fourth target SMART feature matrix samples; Combine the SMART feature matrix with each of the fourth target SMART feature matrix samples to form multiple fourth samples to be tested; Use the disk fault prediction model to process each of the fourth samples to be tested, and obtain a fourth Euclidean distance corresponding to each of the fourth samples to be tested; When each of the fourth Euclidean distances is less than a fourth preset value, determine that the current state of the disk to be detected is a healthy state.

5. A disk failure detection system, characterized in that, Including: A memory for storing a computer program; A processor for implementing the steps of the disk failure detection method according to any one of claims 1 to 3 when executing the computer program.

6. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the disk failure detection method according to any one of claims 1 to 3 are implemented.

Images