Fault hard disk prediction method based on imitation learning

Abstract

The invention relates to the technical field of hard disk fault prediction, in particular to a fault hard disk prediction method based on imitation learning, which comprises the following steps: S1, constructing a data set, and sampling the data set to obtain expert data; s2, establishing a teacher model based on a generative adversarial imitation learning architecture, and training the teacher model by using the data set to obtain fault discrimination knowledge; s3, distilling fault discrimination knowledge from the teacher model to the student model, respectively calculating soft label loss and prediction loss to construct a total loss function, and strengthening the student model through the total loss function; s4, inputting to-be-tested data into the reinforced student model, outputting a prediction result, and judging whether the hard disk fails or not according to the prediction result; on the basis of a teacher model and a student model, the problem of unbalanced distribution of normal data and fault data is avoided through generative adversarial imitation learning in combination with a knowledge distillation technology, and whether the hard disk is about to fail or not can be judged more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of hard disk failure prediction, and in particular to a method for predicting failed hard disks based on imitation learning. Background Art

[0002] As the most commonly used storage device at present, with the rapid development of cloud computing, big data, and cloud storage services, the hard disk drive has become a core component in the storage system of data centers; however, the hard disk failure rate is relatively high, which has become one of the main problems affecting system stability and data security. Research shows that about 78% of cloud server failures are due to hard disk failures, which not only cause service interruptions but may also lead to data loss, resulting in economic losses to service providers and users; therefore, in order to improve the reliability and availability of data storage systems, data redundancy schemes such as redundant backup and erasure codes are widely deployed to cope with the risks brought by hard disk failures; however, these schemes are usually passive fault-tolerant measures that can only reconstruct data after a failure, and there are problems such as low space utilization efficiency and high bandwidth requirements; therefore, it is particularly important to predict the failure of hard disk drives in advance and take corresponding preventive measures.

[0003] With the popularization of the Self-Monitoring, Analysis and Reporting Technology (SMART) of hard disks, most hard disks can monitor their health status in real time and provide fault warnings, that is, the SMART system detects faults based on set thresholds by collecting various performance indicators of hard disks; however, the threshold setting is usually too conservative, often resulting in healthy hard disks being misjudged as failed hard disks, thus reducing the accuracy of fault prediction. In order to improve the fault prediction performance, in recent years, prediction methods based on machine learning and deep learning are usually used, and the SMART data is used as input and combined with supervised learning algorithms for hard disk fault prediction. However, due to the large number of normal hard disks and the small proportion of failed hard disks, the data distribution is unbalanced, making the model prone to bias towards the characteristics of normal hard disk drives during training, affecting the accuracy of fault prediction. Summary of the Invention

[0004] In order to solve the technical problem that the existing prediction method has a low prediction accuracy for failed hard disks due to the unbalanced distribution of normal data and failed data, the purpose of the present invention is to provide a method for predicting failed hard disks based on imitation learning, and the specific technical solution adopted is as follows:

[0005] Step S1: Collect hard disk data, perform preprocessing, construct a data set, and sample the data set to obtain expert data;

[0006] Step S2: Establish a teacher model based on the generative adversarial imitation learning architecture, and train the teacher model using a dataset to obtain fault discrimination knowledge;

[0007] Step S3: Distill the fault discrimination knowledge from the teacher model to the student model, calculate the soft label loss and the prediction loss respectively to construct the total loss function, and strengthen the student model through the total loss function;

[0008] Step S4: Obtain the data to be measured, input the data to be measured into the strengthened student model, output the prediction result, and judge whether the hard disk is faulty according to the prediction result.

[0009] Preferably, in step S1, it includes:

[0010] Step S11: Collect hard disk data and perform standardization processing on the hard disk data. The corresponding calculation formula is:

[0011]

[0012] where x norm represents the standardized hard disk data; x represents the currently input hard disk data; x min and x max represent the minimum value and the maximum value in the hard disk data respectively;

[0013] Step S12: Perform zero-padding processing on the standardized hard disk data, and construct a dataset in units of days;

[0014] Step S13: Randomly sample the dataset to obtain expert data.

[0015] Preferably, in step S2, it includes:

[0016] Step S21: Obtain the generator and the discriminator. The generator and the environment interact through Markov decision-making, generate several intermediate states based on the dataset to form a complete trajectory, and input it into the discriminator;

[0017] Step S22: The discriminator outputs a score for the complete trajectory, and combines the expert data and the complete trajectory for binary classification training;

[0018] Step S23: Store the generated result state based on the trained discriminator, and update the generator parameters;

[0019] Step S24: Repeat steps S21 to S23 until the discriminator cannot distinguish the generator from the expert data to obtain the fault discrimination knowledge.

[0020] Preferably, in step S21, it includes:

[0021] Define the initial information in the generator as {S, A, P, R, γ}, where S represents the state space; A represents the action space; P represents the state transition function; R represents the reward function; γ represents the discount factor, i.e., a constant;

[0022] Based on the initial information, it is segmented into several {s, a} state - actions, and the data in the first window after segmentation is the initial state s0, where s represents the s - state in the state space; a represents the a - action in the action space; and it interacts with the environment to obtain the next state - action of any current state - action, and generates several [(s, a, s′)] intermediate states, where s′ represents the s′ - state obtained by the s - state taking the a - action and interacting with the environment;

[0023] Store the intermediate states until a complete trajectory is formed, and input the complete trajectory into the discriminator.

[0024] Preferably, the generator is optimized by the PPO strategy; LSTM time - series modeling is used in the discriminator.

[0025] Preferably, in step S23, it includes:

[0026] Combine the score with the [(s, a, s′)] intermediate states to form [(s, a, s′, r)] result states, where r represents the reward function corresponding to the current state, store the result states, and update the generator parameters using the PPO truncation method.

[0027] Preferably, in step S3, it includes:

[0028] Screen the target data based on the data set, distill the fault discrimination knowledge from the teacher model to the student model, and train the student model with the target data;

[0029] Calculate the soft - label loss and the prediction loss respectively, and determine the difference between the student model and the teacher model and the difference between the predicted value and the true value obtained by the student model in turn;

[0030] Perform a weighted sum of the soft - label loss and the prediction loss to obtain the total loss function and strengthen the student model.

[0031] Preferably, it is characterized in that screening the target data based on the data set, distilling the fault discrimination knowledge from the teacher model to the student model, and training the student model with the target data includes:

[0032] Input any state in the state space into the teacher model to obtain the corresponding soft - label output y t and the action a t , and obtain the output y s of the current student model through the target data and the action a s, generate expert action pairs [(s, a t ), and based on the student model, generate several [(s, a s , s′)] intermediate states, store the intermediate states until a complete trajectory is formed;

[0033] The discriminator identifies the output of the teacher model as expert data, outputs a score for the complete trajectory, and conducts binary classification training;

[0034] Combine the score with [(s, a s , s′)] intermediate states to form [(s, a s , s′, r)] result states for storage;

[0035] Calculate the update loss of generative adversarial imitation learning through the PPO truncation method, calculate the loss between the soft labels of the student model and the soft labels of the teacher model using KL divergence, and combine them to update the generator parameters;

[0036] Train the student model in a loop until the discriminator can no longer distinguish between the generator and expert data.

[0037] The present invention has the following beneficial effects:

[0038] By collecting hard disk data to construct a dataset, and selecting a certain number of healthy hard disk data as expert data, the degradation law of the hard disk is learned through generative adversarial imitation learning, and a hard disk degradation law model is established based on the teacher model, enabling it to have the ability to identify diverse fault modes; through knowledge distillation technology, the fault discrimination knowledge obtained by the teacher model is distilled into the student model, improving the generalization ability of the student model, enabling the student model to adapt to multiple hard disk models, reducing the complexity and deployment cost of the student model while maintaining performance, and effectively reducing the hard disk operation and maintenance cost of the data center in practical applications. The student model can learn the fault discrimination knowledge of the teacher model for multiple different types of hard disk data, enabling the student model to obtain strong robustness and prediction ability for specific types of hard disks; that is, through generative adversarial imitation learning, the problem of unbalanced distribution of normal data and fault data is avoided, and it is possible to more accurately judge whether the hard disk is about to fail. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0040] Figure 1The flowchart of the steps of a method for predicting faulty hard disks based on imitation learning provided by an embodiment of the present invention;

[0041] Figure 2 The implementation flowchart of a method for predicting faulty hard disks based on imitation learning provided by an embodiment of the present invention. Detailed implementation manners

[0042] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following combines the accompanying drawings and preferred embodiments to detail the specific implementation manners, structures, features and effects of a method for predicting faulty hard disks based on imitation learning proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0044] The following specifically describes the specific solution of a method for predicting faulty hard disks based on imitation learning provided by the present invention with reference to the accompanying drawings.

[0045] Please refer to Figure 1 and Figure 2 , which respectively show the flowchart of the steps and the implementation flowchart of a method for predicting faulty hard disks based on imitation learning provided by an embodiment of the present invention. The method includes:

[0046] Step S1: Collect hard disk data, perform preprocessing, construct a data set, and sample the data set to obtain expert data;

[0047] Step S2: Establish a teacher model based on the generative adversarial imitation learning architecture, and use the data set to train the teacher model to obtain fault discrimination knowledge;

[0048] Step S3: Distill the fault discrimination knowledge from the teacher model to the student model, calculate the soft label loss and the prediction loss respectively to construct a total loss function, and strengthen the student model through the total loss function;

[0049] Step S4: Obtain the data to be measured, input the data to be measured into the strengthened student model, output the prediction result, and judge whether the hard disk is faulty according to the prediction result.

[0050] For better illustration, Generative Adversarial Imitation Learning (GAIL), as an emerging technology, provides a more precise solution. It combines the characteristics of imitation learning and generative adversarial networks. By simulating the decision-making trajectories of experts, it learns the degradation process of hard drives during interactions with the environment. That is, in hard drive fault prediction, the degradation process can be regarded as an expert trajectory. Through learning this trajectory, generative adversarial imitation learning can accurately identify the degradation patterns of hard drives and avoid problems caused by data imbalance. By comparing the degradation pattern differences between the target hard drive, i.e., the hard drive to be detected, and a healthy hard drive, it can predict in advance the occurrence of faults in the hard drive to be detected, providing a more precise warning for maintenance.

[0051] Through the Knowledge Distillation technology, the capabilities of the teacher model can be transferred to the student model, that is, the model can be extended from a specific model of hard drive to other models, improving the generalization ability and applicability of the model, and further enhancing the accuracy and reliability of fault prediction. Among them, knowledge distillation is a model transfer and compression technology. It transfers the "high-level knowledge" contained in the teacher model that has been trained on multi-model hard drive data and has certain prediction capabilities to the student model with a lighter structure and lower computational cost through the method of soft labels. It can be shown that through the knowledge distillation technology, only by obtaining a teacher model trained on all hard drive data can it be quickly transferred to a specific model of hard drive, avoiding the consumption caused by retraining and improving the detection efficiency.

[0052] Furthermore, in step S1, it includes:

[0053] Step S11: Collect hard drive data and perform standardization processing on the hard drive data. The corresponding calculation formula is:

[0054]

[0055] where, x norm represents the standardized hard drive data; x represents the currently input hard drive data; x min and x max represent the minimum and maximum values in the hard drive data respectively;

[0056] Step S12: Perform zero-padding processing on the standardized hard drive data and construct a data set in units of days;

[0057] Step S13: Randomly sample the data set to obtain expert data.

[0058] Optionally, in this embodiment, the hard disk data refers to the SMART attribute value; it should be noted that the preprocessing operation for the hard disk data is to clear the attributes that are not related to the hard disk failure; among them, the SMART attribute value includes the original value and the normalized value. The original value is the value of the actual number of occurrences of this type of failure counted by the counter during the use of the hard disk, and the normalized value is the value obtained by calculation through a formula based on the current value during the operation of the hard disk, with a range between 1 and 253. 1 represents the worst case, and 253 represents the best case; when the hard disk leaves the factory, each SMART attribute has a preset maximum normal value, that is, the factory value. The preset basis and calculation method of this factory value are not disclosed by the hard disk manufacturer, so the hard disks of different models are different. Generally speaking, the maximum normal value is usually 100 or 200 or 253.

[0059] Specifically, since the ranges of different SMART attribute values are different, in order to avoid errors caused by scale differences, the hard disk data is standardized, and zero-padding processing is performed on the standardized hard disk data to complement the non-existent data and unify the data scale; moreover, the manufacturer records the SMART attribute values in units of days, and records the SMART attribute values of all hard disks every day; that is, the dataset used in this application for training the teacher model construction is the change sequence of the SMART attribute values of the hard disk over multiple days; then expert data, that is, healthy hard disk data, is randomly sampled from the dataset for the teacher model to learn the hard disk degradation law.

[0060] Further, in step S2, it includes:

[0061] Step S21: Obtain the generator and the discriminator. The generator and the environment interact through Markov decision-making, generate several intermediate states based on the dataset, form a complete trajectory, and input it into the discriminator.

[0062] Further, in step S21, it includes:

[0063] Step S211: Define the initial information in the generator as {S, A, P, R, γ}, where S represents the state space; A represents the action space; P represents the state transition function; R represents the reward function; γ represents the discount factor, that is, a constant;

[0064] Step S212: Based on the initial information, it is segmented into several {s, a} state actions, and the data in the first window after segmentation is the initial state s0, where s represents the s state in the state space; a represents the a action in the action space; and it interacts with the environment to obtain the next state action of any current state action, and generates several [(s, a, s′)] intermediate states, where s′ represents the s′ state obtained after the s state takes the a action and interacts with the environment.

[0065] Step S213: Store the intermediate state until a complete trajectory is formed, and input the complete trajectory into the discriminator.

[0066] Understandably, the initial information is represented by the five-tuple {S, A, P, R, γ}. Among them, S represents the state space, that is, the information useful for decision-making that the generator can obtain. Since the hard disk damage is gradual, using only the SMART attribute values of one day cannot enable the generator to grasp the concept of time. Therefore, in this embodiment, the SMART attribute values for a period of time are collected and regarded as a state. Specifically, assuming that the instantaneous state x has n variables, and m vectors accumulated by continuous instantaneous states within a time window are used as the time window to form an m×n-dimensional matrix M mn to represent the current state of the hard disk, which is used to reflect the inherent state of the hard disk during this period of time. For a hard disk sequence data, that is, after the dataset is divided by a sliding window, k m×n-dimensional matrices will be obtained. k represents the length of the hard disk sequence data, reflecting the change process of the current hard disk running state. Based on the segmentation of the time window, a set is formed and denoted as the state space S. Among them, since each SMART attribute value is normalized to a decimal between 0 and 1, the entire state space is a finite continuous state space.

[0067] A represents the action space, that is, the set of actions that the generator can choose in the current reinforcement learning task. The generator is used to predict the state change based on the currently observed state and the previous state of the hard disk. Specifically, any action a is defined as a 1×g matrix, q represents the number of SMART attribute values of the currently analyzed hard disk, reflecting the next state of the hard disk predicted by the generator, and the action space A is the set of all actions a. Since each value in the action a is a decimal between 0 and 1, the entire action space is a finite continuous state space.

[0068] P represents the state transition function, and p(s′|s,a) represents the probability of transitioning to s′ after taking action a in state s. That is, P represents the policy, corresponding to the probability of the generator executing each action, which is divided into a deterministic policy and a stochastic policy. Specifically, when training the teacher model, the corresponding policy is selected according to actual needs. For example, in order to explore the state space and action space more, a stochastic policy is adopted, and proximal policy optimization (PPO) is used to limit the amplitude of each policy update through the proximal policy optimization method to ensure the balance between exploration and training. Furthermore, a generator that can make a similar expert policy is trained.

[0069] R represents the reward function, which is usually set manually to judge the learning effect of the generator; the reward function is used to guide the generator to complete a task. If the reward for completing the task is set as a large positive number and the rewards for all other states are set as negative numbers, the generator will complete the task as quickly as possible in order to obtain the maximum total reward. Preferably, in this embodiment, the reward is not given manually, but is given by the output result of the discriminator. Specifically, the discriminator will judge whether the samples generated by the generator come from the expert data and output a real number between 0 and 1, indicating the probability that the discriminator believes that the state-action pair comes from the generator's policy rather than the expert. Among them, the closer the samples generated by the generator are to 0, the more the discriminator believes that they come from the expert data, indicating that the corresponding generator has a better effect; on the contrary, if it approaches 1, the discriminator believes that it comes from the generator's policy. Generally, taking 0.5 as the dividing line, the data less than 0.5 is defined as the expert policy, and the data greater than 0.5 is defined as the generator's policy. Therefore, in this embodiment, the negative logarithm form is used to use the discriminator result as the reward of the generator; γ represents the discount factor, γ ∈ [0, 1], which is used to calculate the cumulative return.

[0070] Furthermore, the generator is optimized through the PPO strategy; LSTM time series modeling is used in the discriminator.

[0071] It can be explained that PPO (Proximal Policy Optimization) is introduced as a generation network within the generator, and the generation policy is learned through the reinforcement learning generator. Specifically, in this embodiment, the state s and action a are both defined as continuous values, and PPO has significant advantages in processing continuous state space and continuous action space. That is, PPO can directly optimize in the policy space, and limit the step size of each policy update through the PPO truncated update mechanism, effectively preventing drastic fluctuations in policy parameters and ensuring a more stable training process. In addition, PPO has a strong fitting ability for high-dimensional continuous space, can efficiently explore and approximate the optimal policy, and is suitable for scenarios with continuous states and actions. The training process of PPO supports batch offline training, that is, it can be repeatedly trained on historical hard disk data until the policy converges, and then the optimized policy is deployed in the actual environment for online inference. For the task of outputting recognition samples based on the generator for the discriminator, the high response speed requirement is not suitable for updating the policy while inferring, so the offline-online separation feature of PPO just meets the needs of this scenario. The training process can be completed in the background using a large amount of data and computing power, and after deployment, only a lightweight inference model is needed to quickly output the prediction results, which improves the real-time and reliability of the overall method; since the generator of generative adversarial imitation learning needs to be updated stably and efficiently in a continuous strategy space, the training process involves an adversarial game between the generator and the discriminator. In order to avoid violent fluctuations in the generator strategy and affect the convergence of the entire adversarial process, PPO, with its stable small-step strategy update mechanism and good sample efficiency, is more suitable for adversarial game scenarios such as generative adversarial imitation learning.

[0072] Expert data is defined as normal samples, and the discriminator is used to determine whether the generator output samples, that is, any state-action pair, originates from the expert data. Since the SMART attribute values of hard drives exhibit obvious sequential and stage-specific characteristics over time, the use of traditional fully connected neural networks may not be able to fully model the temporal dependencies in the sequence. However, the Long Short-Term Memory Network (LSTM) is a recurrent neural network structure specifically designed to process sequence data. It can retain information over long time spans and capture temporal patterns in state changes. Therefore, the use of LSTM in the discriminator can significantly improve its ability to perceive the hard drive degradation process patterns, enabling it to more accurately distinguish the subtle differences between the generator and expert data, thereby providing a more effective training signal to promote the optimization of the generator.

[0073] Step S22: The discriminator scores the complete trajectory output and combines the expert data and the complete trajectory for binary classification training.

[0074] Explanation is made that the expert data is normal samples, which are used as the prior knowledge for the teacher model training, that is, the degradation sequence data of healthy hard disks; specifically, based on all the data of the hard disk, it is segmented, that is, the initial information is segmented into multiple {s, a} state actions to participate in subsequent training, and the data within the first window after segmentation is used as the initial state s0, which is input into the generator. The generator predicts the current state of the hard disk, outputs the predicted action, and inputs the state-action pair {s0, a0} into the discriminator. The output of the discriminator is used as the reward, and the next state s1 of the initial state s0 is obtained by interacting with the environment. This process is continuously repeated until the end of this cycle and the next cycle starts. The intermediate states of the triples [(s, a, s′)] generated during this period are stored in the buffer to form a complete trajectory; then the discriminator outputs a score for the complete trajectory to evaluate whether the predicted trajectory output conforms to the actual scenario, and the expert data and the complete trajectory output by the generator are used for binary classification training using the minimized cross-entropy loss, effectively improving the discriminator's ability to distinguish between the real trajectory and the complete trajectory output by the generator, ensuring the authenticity and integrity of the complete trajectory, and enhancing the effect of the discriminator.

[0075] Step S23: Store the generated result state based on the trained discriminator and update the generator parameters.

[0076] Further, in step S23, it includes:

[0077] Combine the score with the intermediate state of [(s, a, s′)] to form a result state of [(s, a, s′, r)], where r represents the reward function corresponding to the current state, store the result state, and update the generator parameters using the PPO truncation method.

[0078] Specifically, combine the score output by the discriminator with the intermediate state of the triple [(s, a, s′)] to form a quadruple [(s, a, s′, r)] result state, that is, form a quadruple [(s, a, s′, r)] result state containing the original input, state, action, and score, and store it in the experience buffer to retain more training information, improve the efficiency and effect of training, that is, the experience buffer is used to store training data to enable better utilization of historical data during training; update the generator parameters using the PPO truncation method, which avoids excessive update steps through truncated policy gradients to improve the stability and convergence speed of training, so that the output samples of the generator can be closer to the expert data.

[0079] Step S24: Repeat steps S21 to S23 until the discriminator cannot distinguish between the generator and the expert data to obtain fault discrimination knowledge; through cyclic training, both the generator and the discriminator can achieve good effects, enabling the teacher model to obtain fault discrimination knowledge.

[0080] Further, in step S3, it includes:

[0081] Step S31: Filter the target data based on the dataset, distill the fault discrimination knowledge from the teacher model to the student model, and train the student model with the target data.

[0082] Preferably, in the scenario of actual deployment for hard disk detection, there are significant differences in the characteristic distribution, fault modes, and time evolution trends of SMART attribute values among hard disks of different manufacturers or models. This makes it difficult for a unified training model to achieve the optimal effect for the hard disk data of a specific model. Therefore, through the knowledge distillation technology, the fault discrimination knowledge obtained by the teacher model is distilled into the student model to achieve cross-model fault prediction knowledge transfer.

[0083] Further, in step S31, it includes:

[0084] Step S311: Input any state in the state space into the teacher model to obtain the corresponding soft label output y t and action a t , and obtain the output y s and action a s obtained by the current student model through the target data, generate the expert action pair [(s, a t )], generate several [(s, a s , s′)] intermediate states based on the student model, store the intermediate states until a complete trajectory is formed;

[0085] Step S312: The discriminator identifies the output of the teacher model as expert data, outputs a score for the complete trajectory, and performs binary classification training;

[0086] Step S313: Combine the score with the [(s, a s , s′)] intermediate states to form the [(s, a s , s′, r)] result state for storage;

[0087] Step S314: Calculate the generative adversarial imitation learning update loss through the PPO truncation method, calculate the loss between the soft label of the student model and the soft label of the teacher model using the KL divergence, and combine them to update the generator parameters;

[0088] Step S315: Continuously train the student model until the discriminator can no longer distinguish between the generator and the expert data.

[0089] Understandably, through the knowledge distillation technique, the training of the student model is similar to that of the teacher model, that is, the high-level fault discrimination knowledge possessed by the teacher model is transferred to the student model, so that the student model can also obtain strong robustness and predictive ability with a lead time on a specific model of hard disk; it can be explained that the teacher model has the ability to identify diverse fault modes of hard disks after training. When its knowledge is transferred to the student model for training, the training data between the teacher model and the student model is different. The teacher model is trained on the dataset, while the student model is the target data selected from the dataset. In this embodiment, the target data refers to the hard disk data of any specific model; that is, a small-scale lightweight student model is trained with the target data to reduce the complexity and deployment cost of the student model while maintaining performance, and the preprocessing of the target data is consistent with the preprocessing of the dataset to ensure the consistency of knowledge transfer.

[0090] Specifically, the discriminator identifies the output of the teacher model as expert data, and then the discriminator scores the complete trajectory output by the generator in the student model. Similarly, binary classification training is performed on the expert data and the complete trajectory using the minimized cross-entropy loss to improve the effect of the discriminator; for the update of the generator parameters, a combination of the losses of the soft labels of the student model and the soft labels of the teacher model is calculated using the KL (Kullback-Leibler) divergence. Among them, the KL divergence, that is, relative entropy, is used to measure the uncertainty of one probability distribution relative to another probability distribution, so that the soft labels output by the student model can be as close as possible to the soft labels output by the teacher model, ensuring that the update direction of the generator parameters is consistent with the guidance of the teacher model.

[0091] Step S32: Calculate the soft label loss and the prediction loss respectively, and successively determine the difference between the student model and the teacher model and the difference between the predicted value obtained by the student model and the true value.

[0092] It is explained that the teacher model provides two types of soft targets. One is the final class probability output, that is, the output soft label. The other is the feature representation of the intermediate layer, that is, the LSTM hidden state or the policy vector. While the student model is supervised and trained with the true label, it learns the fault discrimination knowledge of the teacher model by minimizing the difference from the output of the teacher model, such as KL divergence or MSE loss. Specifically, the soft label loss and the prediction loss are calculated respectively. Among them, the soft label usually contains the class probability distribution information output by the teacher model, which provides richer inter-class similarity information than the true label and helps the student model better perform knowledge transfer. The soft label loss is the core of the knowledge distillation technology and is used to measure the difference between the soft label output by the student model and the teacher model. The prediction loss is the loss in generative adversarial imitation learning, and the difference between the prediction result of the student model and the true result is measured by this prediction loss.

[0093] Step S33: Perform a weighted sum of the soft label loss and the prediction loss to obtain the total loss function and strengthen the student model.

[0094] It can be explained that the total loss function is the weighted sum of the soft label loss and the prediction loss. The soft label loss guides the student model to learn the fault discrimination knowledge from the teacher model, and the prediction loss ensures that the student model can accurately predict the faults of specific model hard disks. By minimizing the total loss function, the student model gradually approaches the performance of the teacher model on the data of this specific model hard disk. For some scenarios, a reinforcement learning transfer strategy can also be adopted to let the student model refer to the action suggestions or policy outputs provided by the teacher model during the training process for policy distillation.

[0095] It is explained that in step S4, the data to be measured is obtained, the data to be measured is input into the strengthened student model, and the prediction result is output. Whether the hard disk is faulty is judged according to the prediction result. It can be understood that if the data to be measured is a commonly seen hard disk model, both the teacher model and the student model can achieve fault prediction. If the data to be measured is the data of a specific model hard disk, using the strengthened student model for prediction can obtain the prediction result faster and more accurately. Among them, when the discriminator in the model architecture can distinguish the samples output by the generator from the expert data, it means that the hard disk corresponding to the data to be measured has a fault, and the operation and maintenance personnel are notified to take corresponding maintenance measures.

[0096] Understandably, a data set is constructed by collecting hard disk data, and a certain amount of healthy hard disk data is selected as expert data. The degradation law of the hard disk is learned through generative adversarial imitation learning. A hard disk degradation law model is established based on the teacher model, enabling it to have the ability to identify diverse fault modes. Through knowledge distillation technology, the fault discrimination knowledge obtained by the teacher model is distilled into the student model, improving the generalization ability of the student model, enabling the student model to adapt to multiple hard disk models, reducing the complexity and deployment cost of the student model while maintaining performance, and effectively reducing the hard disk operation and maintenance cost of the data center in practical applications. The student model can learn the fault discrimination knowledge of the teacher model for the data of multiple different hard disk models, enabling the student model to obtain strong robustness and prediction ability for specific hard disk models. That is, through generative adversarial imitation learning, the problem of unbalanced distribution of normal data and fault data is avoided, and it is possible to more accurately determine whether the hard disk is about to fail.

[0097] It should be noted that the above sequence of the embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0098] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the key points of each embodiment are the differences from other embodiments.

Claims

1. A fault hard disk prediction method based on imitation learning, characterized in that, The method includes: Step S1: Collect hard disk data, perform preprocessing, construct a data set, and sample the data set to obtain expert data; Step S2: Establish a teacher model based on the generative adversarial imitation learning architecture, and use the data set to train the teacher model to obtain fault discrimination knowledge; Step S3: Distill the fault discrimination knowledge from the teacher model to the student model, calculate the soft label loss and the prediction loss respectively to construct a total loss function, and strengthen the student model through the total loss function; Step S4: Obtain the data to be measured, input the data to be measured into the strengthened student model, output the prediction result, and judge whether the hard disk is faulty according to the prediction result.

2. The method for predicting a faulty hard disk based on imitation learning according to claim 1, characterized in that, In step S1, it includes: Step S11: Collect hard disk data and perform standardization processing on the hard disk data. The corresponding calculation formula is: Among them, x norm represents the standardized hard disk data; x represents the currently input hard disk data; x min , x max respectively represent the minimum and maximum values in the hard disk data; Step S12: Perform zero-padding processing on the standardized hard disk data, and construct a data set in units of days; Step S13: Randomly sample the data set to obtain expert data.

3. The method for predicting a faulty hard disk based on imitation learning according to claim 1, characterized in that In step S2, it includes: Step S21: Obtain the generator and the discriminator. The generator and the environment interact through Markov decision-making, generate several intermediate states based on the data set, form a complete trajectory, and input it into the discriminator; Step S22: The discriminator outputs a score for the complete trajectory, and combines the expert data and the complete trajectory for binary classification training; Step S23: Store the result state generated based on the trained discriminator, and update the generator parameters; Step S24: Repeat steps S21 to S23 until the discriminator cannot distinguish between the generator and the expert data to obtain fault discrimination knowledge.

4. The method for predicting a faulty hard disk based on imitation learning according to claim 3, wherein In step S21, it includes: Define the initial information in the generator as {S, A, P, R, γ}, where S represents the state space; A represents the action space; P represents the state transition function; R represents the reward function; γ represents the discount factor, that is, a constant; Based on the initial information, divide it into several {s, a} state actions, and the data in the first window after division is the initial state s0, where s represents the s state in the state space; a represents the a action in the action space; and interact with the environment to obtain the next state action of any current state action, and generate several [(s, a, s′)] intermediate states, where s′ represents the s′ state obtained by transferring the s state after taking the a action and interacting with the environment; Store the intermediate states until a complete trajectory is formed, and input the complete trajectory into the discriminator.

5. The method for predicting a faulty hard disk based on imitation learning according to claim 4, characterized in that, The generator is optimized through the PPO strategy; LSTM time series modeling is used in the discriminator.

6. The method for predicting a faulty hard disk based on imitation learning according to claim 4, wherein In step S23, it includes: Combine the score with the [(s, a, s′)] intermediate states to form [(s, a, s′, r)] result states, where r represents the reward function corresponding to the current state, store the result states, and update the generator parameters using the PPO truncation method.

7. A fault hard disk prediction method based on imitation learning according to claim 1, characterized in that, In step S3, it includes: Screen the target data based on the data set, distill the fault discrimination knowledge from the teacher model to the student model, and train the student model through the target data; Calculate the soft label loss and the prediction loss respectively, and determine the difference between the student model and the teacher model and the difference between the prediction value obtained by the student model and the true value in turn; A weighted sum of the soft label loss and the prediction loss is obtained to get the total loss function, strengthening the student model.

8. A method for predicting a faulty hard disk based on imitation learning according to any one of claims 5 or 7, characterized in that Target data is obtained through screening based on the dataset, and the fault discrimination knowledge is distilled from the teacher model to the student model, and the student model is trained through the target data, including: Input any state in the state space into the teacher model to obtain the corresponding soft label output y t and action a t , and obtain the output y s and action a s of the current student model through the target data, generate the expert action pair [(s, a t ), and generate several [(s, a s , s′)] intermediate states based on the student model, store the intermediate states until a complete trajectory is formed; The discriminator identifies the output of the teacher model as expert data, scores the complete trajectory, and conducts binary classification training; Combine the score with the intermediate state [(s, a s , s′)] to form the result state [(s, a s , s′, r)] and store it; The update loss of generative adversarial imitation learning is calculated through the PPO truncation method, and the loss between the soft label of the student model and the soft label of the teacher model is calculated using the KL divergence, and the generator parameters are updated by combining them; The student model is trained in a loop until the discriminator can no longer distinguish the generator from the expert data.

Images